Vehicle track recovery method and device in shielding state and computer equipment

Abstract

The invention relates to a vehicle track recovery method and device in a shielding state and computer equipment, and the method comprises the steps: obtaining a target video frame of vehicle driving shot by an unmanned plane, clustering the lane pixel coordinates in the target video frame to obtain an intersection, taking the intersection as a node, and obtaining a target video frame of vehicle driving; the connection relation between the intersections is the edge construction lane topological graph; in the lane topological graph, at least two to-be-bound lanes are determined according to the driving state of the vehicle, the binding score of the vehicle track and each to-be-bound lane is calculated, and a bound lane corresponding to the vehicle track is determined according to the binding score; when the unmanned aerial vehicle detects that the vehicle is shielded, generating at least two candidate trajectories according to the bound lane and lane information in the lane topological graph, and determining a target trajectory in the candidate trajectories through a decision model; and if at least two target trajectories exist and the target trajectories meet a preset separation condition, fusing the target trajectories to obtain a complete trajectory. According to the invention, the accuracy of track recovery is improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle trajectories, and in particular to a method, apparatus, and computer device for recovering vehicle trajectories in an occluded state. Background Art

[0002] When a drone tracks a vehicle, it can be obscured by obstacles such as buildings and trees, causing the drone to lose the target. In the event of a target loss, the drone predicts the vehicle's trajectory to re-identify the obscured target. However, if the target changes direction or lanes during an occlusion, the actual vehicle trajectory can differ significantly from the predicted one, reducing the drone's accuracy in re-identifying the target after the occlusion. Therefore, improving the accuracy of drone trajectory prediction after vehicle occlusion has become a pressing issue. Summary of the Invention

[0003] The embodiments of the present application provide a method, device, and electronic device for recovering a vehicle trajectory in an occluded state, to at least solve the problem in the related art of low accuracy of vehicle trajectory prediction by a drone after the vehicle is occluded.

[0004] In a first aspect, an embodiment of the present application provides a method for recovering a vehicle trajectory in an occluded state, the method being applied to a drone carrying a camera assembly, the method comprising: Obtain a target video frame of a vehicle traveling shot by the drone, cluster lane pixel coordinates in the target video frame to obtain intersections, and construct a lane topology map using the intersections as nodes and the connections between the intersections as edges; In the lane topology map, determining at least two lanes to be bound according to the driving state of the vehicle, calculating a binding score between the vehicle trajectory and each of the lanes to be bound, and determining a bound lane corresponding to the vehicle trajectory according to the binding score; When the UAV detects that the vehicle is obscured, generating at least two candidate trajectories according to the bound lane and lane information in the lane topology map, and determining a target trajectory from the candidate trajectories through a decision model; If there are at least two target trajectories, and the target trajectories meet the preset separation condition, the target trajectories are fused to obtain a complete trajectory.

[0005] In one embodiment, the driving state includes the vehicle's position information, heading angle, and driving speed, and the calculating of the binding score between the vehicle trajectory and each lane to be bound includes: For any lane to be bound, determining a distance feature based on the distance between the position information and the center of the lane to be bound; determining an angle feature according to the lane direction of the lane to be bound and the heading angle; determining a speed characteristic according to the lane speed limit of the lane to be bound and the driving speed; Based on the current moment, obtaining the binding score between the vehicle and the corresponding bound lane at the previous moment to determine the historical characteristics; A binding score between the vehicle trajectory and the lane to be bound is obtained based on the distance feature, the angle feature, the speed feature, and the history feature.

[0006] In one embodiment, determining the bound lane corresponding to the vehicle trajectory according to the binding score further includes: If, in a first preset number of consecutive first target video frames, the binding score of each of the target video frames is greater than a first preset threshold, an intermediate pairing set is obtained, where the intermediate pairing set is a pairing of the vehicle trajectory and the lane to be bound; In the intermediate pairing set, if, in a second preset number of consecutive second target video frames, the binding score of each of the consecutive second target video frames is greater than a second preset threshold, determining the current to-be-bound lane as the bound lane of the vehicle, wherein the first preset number is less than the second preset number, and the first preset threshold is less than the second preset threshold; If the vehicle trajectory satisfies the lane change condition, the binding relationship between the vehicle trajectory and the bound lane is released, and the vehicle is bound to the lane again.

[0007] In one embodiment, clustering lane pixel coordinates in the target video frame to obtain intersections, and constructing a lane topology graph with the intersections as nodes and connections between the intersections as edges, includes: Clustering the lane pixel coordinates according to preset lane requirements to obtain a structured road network, and determining intersections in the structured road network according to preset curvature requirements; According to preset lane design requirements, a lane topology graph is constructed with the intersections as nodes and the connection relationships between the intersections as edges.

[0008] In one embodiment, when the drone detects that the vehicle is blocked, the process includes: Obtain an average confidence level according to the confidence level of the vehicle identified in each frame in a third preset number of consecutive video frames, and obtain a confidence level decrease rate according to the average confidence level and the confidence level of the last consecutive video frame; If the drone does not detect the vehicle in the continuous video frames and the confidence decrease rate is greater than a preset confidence threshold, the drone detects that the vehicle is blocked.

[0009] In one embodiment, generating at least two candidate trajectories based on the bound lane and lane information in the lane topology map includes: Determining, based on lane speed limit information and lane turning information of each lane in the lane topology map, a lane to be matched that the vehicle can drive to from the bound lane; Obtain a travel distance of the vehicle during the occlusion time, and generate at least two candidate trajectories based on the travel distance and the lane to be matched.

[0010] In one embodiment, determining the target trajectory from the candidate trajectories using a decision model includes: For each candidate trajectory, obtaining a predicted speed of the vehicle in the candidate trajectory and a distance between the candidate trajectory and an adjacent lane; Obtaining a kinematic score based on the actual speed of the vehicle and the predicted speed; Obtaining a topological consistency score based on the distance between the candidate trajectory and the center of the adjacent lane; When the vehicle reappears, obtaining the characteristics and location information of the vehicle, and determining a re-identification score based on the similarity between the characteristics of the vehicle and the characteristics of the candidate trajectory, and the norm of the location information and the location information of the candidate trajectory; Obtaining an optimization score according to the kinematic score, the topological consistency score, and the re-identification score; The candidate trajectory with the highest optimization score is used as the target trajectory.

[0011] In one embodiment, after determining the optimization score according to the kinematic score, the topological consistency score, and the re-identification score, the method further includes: If the difference between the optimization scores of at least two candidate trajectories is less than a preset difference threshold, taking the straight candidate trajectory as the target trajectory; If any two candidate trajectories intersect, the candidate trajectory whose predicted speed of the vehicle is greater than a preset speed threshold is used as the target trajectory.

[0012] In one embodiment, the preset separation condition satisfies the following configuration: The difference between the end time and the start time of each target track is within a preset time range; The types of vehicles traveling on different target trajectories are consistent.

[0013] In one embodiment, fusing the target trajectories to obtain a complete trajectory includes: Obtaining the running speed of the vehicle before being blocked and the running time of the vehicle when being blocked; Obtaining a motion continuity score based on the predicted acceleration at the end of the target trajectory and the actual acceleration of the vehicle; If the driving duration is within the driving time range of the running speed and the motion continuity score meets the continuity requirement, a fusion order of the target trajectories is determined, and the target trajectories are fused according to the fusion order through the decision model to obtain a complete trajectory.

[0014] In a second aspect, an embodiment of the present application provides a vehicle trajectory recovery device in an obstructed state, the device being applied to a drone carrying a camera assembly, the device comprising: A lane topology map construction module is used to obtain a target video frame of a vehicle being driven by the drone, cluster lane pixel coordinates in the target video frame to obtain intersections, and construct a lane topology map using the intersections as nodes and the connections between the intersections as edges; a lane and vehicle trajectory binding module, configured to determine, in the lane topology map, at least two lanes to be bound based on the vehicle's driving state, calculate a binding score between the vehicle trajectory and each of the lanes to be bound, and determine a bound lane corresponding to the vehicle based on the binding score; a target trajectory determination module, configured to generate at least two candidate trajectories based on the bound lanes and lane information in the lane topology map when the drone detects that the vehicle is obscured, and determine a target trajectory from the candidate trajectories using a decision model; The target trajectory fusion module is used to fuse the target trajectories to obtain a complete trajectory if there are at least two target trajectories and the target trajectories meet a preset separation condition.

[0015] In a third aspect, an embodiment of the present application provides a computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the method for recovering the vehicle trajectory under the occlusion state as described in the first aspect above is implemented.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the vehicle trajectory recovery method under an occlusion state as described in the first aspect above.

[0017] The embodiments of the present application provide a method, apparatus, and computer device for recovering vehicle trajectories in an occluded state, which have at least the following technical effects.

[0018] A lane topology map is constructed by capturing target video frames of a vehicle in motion using a drone. By calculating the binding score between the lane to be bound and the vehicle trajectory, the vehicle trajectory is bound to the corresponding bound lane. This provides physical constraints between the vehicle trajectory and the bound lane, ensuring that the vehicle's trajectory remains consistent in occluded scenarios, using the lane information in the bound lane as a constraint. When an occlusion is detected, at least two candidate trajectories are generated based on the bound lane and lane information in the lane topology map. The bound lane and lane information provide lane geometry constraints and traffic regulations for generating candidate trajectories, ensuring that the candidate trajectories conform to the actual situation. Determining the target trajectory from the candidate trajectories ensures that the target trajectory is close to the vehicle's actual trajectory, ensuring the accuracy of the target trajectory. Target trajectories that meet the preset separation conditions are fused to obtain a globally consistent complete trajectory, thus overcoming the trajectory fragmentation problem caused by frame-by-frame processing.

[0019] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 is a flow chart of a method for restoring a vehicle trajectory in an occluded state according to an exemplary embodiment; Figure 2 is a schematic diagram showing binding of a vehicle to a lane according to an exemplary embodiment; Figure 3 is a flowchart of determining a bound lane according to another exemplary embodiment; Figure 4 is a flowchart illustrating lane binding according to another exemplary embodiment; Figure 5 is a flow chart of a method for recovering a vehicle trajectory in an occluded state according to another exemplary embodiment; Figure 6 is a flow chart of a method for recovering a vehicle trajectory in an occluded state according to another exemplary embodiment; Figure 7 is a block diagram of a vehicle trajectory recovery device in an occlusion state according to an exemplary embodiment; Figure 8 is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is described and illustrated below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely used to explain this application and are not intended to limit this application. Based on the embodiments provided in this application, all other embodiments obtained by those of ordinary skill in the art without making any creative effort are within the scope of protection of this application.

[0022] Obviously, the drawings described below are merely examples or embodiments of the present application. Those skilled in the art can, without inventive effort, apply the present application to other similar scenarios based on these drawings. Furthermore, it is also understood that, although the effort involved in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, changes in design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as an insufficiency of the content disclosed in this application.

[0023] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments unless there is a conflict.

[0024] Unless otherwise defined, technical or scientific terms used herein shall have the ordinary meaning as understood by persons of ordinary skill in the art to which this application belongs. The terms "a," "an," "an," "the," and similar expressions used herein do not denote quantitative limitations and may refer to either the singular or the plural. The terms "comprise," "include," "have," and any variations thereof, used herein, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or modules (units) is not limited to the listed steps or units but may also include steps or units not listed, or may include other steps or units inherent to the process, method, product, or apparatus. The terms "connected," "connected," "coupled," and similar expressions used herein are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. As used herein, "plurality" means two or more. "And / or" describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" may mean: A exists alone; A and B exist simultaneously; or B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0025] In a first aspect, the embodiments of the present application provide a method for recovering a vehicle trajectory in an occluded state. Figure 1 FIG. 1 is a flow chart of a method for restoring a vehicle trajectory in an occluded state according to an exemplary embodiment. Figure 1 As shown, the vehicle trajectory recovery method under occlusion state includes: Step S101: Obtain a target video frame of a vehicle traveling by a drone, cluster lane pixel coordinates in the target video frame to obtain intersections, and construct a lane topology map using intersections as nodes and connections between intersections as edges.

[0026] A drone captures a moving vehicle and obtains video frames. The scale-invariant feature transform (SIFT) of each frame is obtained from the drone video frames, and the difference between each frame is obtained. The difference satisfies the following formula:

[0027] in, f t is the video frame at time t, f t-1 is the video frame at time t-1, Δ( f t ，ft-1 ) is the difference, SIFT ( f t ) is the SIFT feature of the video frame at time t, SIFT ( f t-1 ) is the SIFT feature of the video frame at time t-1.

[0028] If the difference between consecutive frames exceeds a preset difference threshold, the drone video frame is captured based on the specific situation of the drone video frame to obtain the target video frame. For example, if the drone video frame is 30 frames, the drone video frame is captured to obtain a target video frame of 15 frames.

[0029] The target video frame is obtained by judging whether the difference between consecutive frames exceeds a preset difference threshold, so as to ensure that the analysis is performed only when the scene changes significantly.

[0030] Continuing with step S101, the target video frame is processed using a dual-branch hybrid neural network to obtain the pixel coordinates of the lane lines. A confidence score is then assigned to the pixel coordinates, and pixel coordinates with confidence scores greater than a preset score threshold are retained. Using a clustering algorithm, lane pixel coordinates are clustered according to preset lane requirements to obtain a structural road network. The preset lane requirements include lane line length requirements and lane obstacle settings. The lane length requirement specifically includes that the aggregated lane length is greater than or equal to a preset length threshold to ensure that the lane length formed by the cluster meets actual requirements. Lane obstacle settings specifically include road obstacle settings due to construction and temporary lane changes.

[0031] In the structured road network, pixel coordinates are sampled in the structured road network according to the preset interval and the same direction, and the three-point curvature calculation is performed on the pixel coordinates, and the curvature is quickly obtained through the discrete pixel coordinates. The smoothing coefficient of the curvature is adaptively adjusted so that the curvature maintains the real characteristics and suppresses noise. Based on the entire curvature, the extreme value of the curvature is determined. Among them, the extreme value of the curvature includes the maximum value and the minimum value. The maximum value indicates that the curvature of the curve near the pixel coordinate is greater than the surrounding area. Therefore, the maximum value can accurately identify the vertex of the turn and the entrance point of the intersection. The minimum value indicates that the curvature of the pixel coordinate is smaller than that of the adjacent area. Therefore, the pixel coordinate is part of a straight line segment. And the curvature lower limit is set so that the curvature is greater than the curvature lower limit to ensure that the lane meets the minimum turning radius. Among them, the three-point curvature calculation satisfies:

[0032] Where k is the curvature, ( x １ , y 1), ( x ２ , y２ ),( x ３ , y ３ ) are three different pixel coordinates.

[0033] After determining the maximum and minimum values ​​of the curvature, the Gaussian smoothing coefficient is dynamically adjusted to address the pixel coordinate jitter caused by the vibration of the drone.

[0034] Curvature extrema can be used to accurately locate curve vertices and intersection entrances, providing curvature continuity constraints for subsequent vehicle trajectories and lane binding. Dynamically adjusting the Gaussian smoothing coefficient can also reduce the error rate in constructing lane topology maps, providing curvature constraints for subsequent accurate trajectory generation and improving the rationality of generated trajectories.

[0035] The pixel coordinates corresponding to the maximum curvature value are determined as the first node, and the pixel coordinates corresponding to the minimum curvature value are determined as the second node. If the intersection node has at least three edges connected to adjacent pixels, and the angle between any two edges is greater than a first preset angle and less than a second preset angle, the first node that meets the conditions is determined to be an intersection node. If the curvature change rate between the second node and the adjacent pixels is less than a change threshold, the second node is considered a normal node.

[0036] Fit the lane pixel coordinates to obtain a fitted curve. Equidistant sampling is performed within the fitted curve to obtain a number of sampling points. The Hausdorff distance between each adjacent sampling point is calculated. If the number of Hausdorff distances greater than a preset distance threshold reaches a preset proportion of the total Hausdorff distances, the lane pixels meet the first connection condition. Within the fitted curve, the vector formed by adjacent lane pixels is determined. Based on the vector, the angle formed by the lane pixels is determined. If the angle is less than a third preset angle, the lane pixels meet the second connection condition, meaning that the directions formed by the lane pixels are all in the same direction.

[0037] When lane pixels meet both the first and second connection conditions, they are connected. Therefore, intersection nodes and regular nodes can be connected. Furthermore, based on regular nodes and intersection nodes, the connections between regular nodes and intersection nodes are used as edges, and a lane topology map is constructed according to the preset lane design requirements.

[0038] It should be noted that the preset lane design requirements include highway alignment design specifications and urban road intersection design standards. The preset lane design requirements are determined based on the video frames captured by the drone. If the video frame is a highway, the preset lane design requirements are based on highway alignment design specifications; if the video frame is a city road, the preset lane design requirements are based on urban road intersection design standards.

[0039] In addition, based on the changes in lanes caused by road construction, update rules are established. The lane topology graph changes its edges according to the update rules to establish a real-time updated lane topology graph. The updated lane topology graph satisfies:

[0040] in: G t is the lane topology at time t, G t+1 is the lane topology at time t+1, representing the updated lane topology, ΔG is the incremental observation data, representing the detection of new lane elements, When there is an overlapping area in the topological data at time t and time t+1, the conflicting lane elements are eliminated by updating the rules to ensure the global consistency of the updated topological graph. is the overlap ratio between the lane topology map at time t and the incremental observation data, It is the overlapping area between the lane topology map and the incremental observation data at time t; is the area of ​​the lane topology at time t, is the overlap threshold.

[0041] Continuing with step S101, the target video frame is acquired through drone recognition, and the pixel coordinates of the lane lines in the target video frame are obtained. These pixel coordinates are clustered to obtain a preliminary structural road network. Within the structural road network, the degree of curvature is further determined using curvature. Intersections and lane straight-through intersections are determined based on the curvature, ensuring that the lane topology map accurately reflects the actual lane conditions.

[0042] Step S102: In the lane topology map, determine at least two lanes to be bound based on the vehicle's driving state, calculate a binding score between the vehicle and each lane to be bound, and determine a bound lane corresponding to the vehicle trajectory based on the binding score.

[0043] Figure 2 is a schematic diagram showing the binding of a vehicle to a lane according to an exemplary embodiment. Figure 2 As shown, the vehicle detection module on the drone acquires the vehicle's driving status. This status includes the vehicle's location, heading angle, and speed. The vehicle's location information is sent to the binding engine. Based on this location information, the binding engine searches the lane topology map for lanes near the vehicle's location and identifies at least two lanes to be bound. The binding engine calculates the binding score between the vehicle's trajectory and each lane to be bound. Based on the binding score, the binding lane corresponding to the vehicle's trajectory is determined, the binding relationship is updated, and the vehicle's trajectory in the bound lanes is recorded.

[0044] The binding scores include: For any lane to be bound, the distance feature is determined based on the distance between the position information and the center of the lane to be bound, so as to determine the proximity of the vehicle to the center of the lane to be bound. The distance feature satisfies the following formula:

[0045] in, f dist is the distance feature, V t .pos is the world coordinate of the vehicle center point, e.proj is the vertical projection coordinate of the vehicle position on the lane center line, δ d is the distance tolerance, and δ d The value is half of the width of the lane to be bound.

[0046] According to the lane direction and heading angle of the lane to be bound, the angle feature is determined to ensure that the vehicle's heading is consistent with the specified direction of the lane based on the angle feature. The angle feature satisfies the following formula:

[0047] in, f angle is the score feature, V t .θ is the heading angle of the vehicle, e.θ proj It is the tangent direction angle of the lane centerline at the vehicle projection point.

[0048] Based on the lane speed limit and driving speed of the lane to be bound, the speed feature is determined to ensure that the vehicle's driving speed matches the lane speed limit. The speed feature satisfies the following formula:

[0049] in, f speed is the speed characteristic, V t .v is the vehicle's speed, e.speed is the lane speed limit for the lane.

[0050] Based on the current moment, the binding score between the vehicle trajectory and the corresponding bound lane at the previous moment is obtained to determine the historical features to provide time continuity constraints. The historical features satisfy the following formula:

[0051] in, f history It is a historical feature, S(V t-1 ,e) is the binding score of the vehicle trajectory to the same lane at time t-1, is the historical confidence decay coefficient.

[0052] Based on the distance feature, angle feature, speed feature, and history feature, the binding score between the vehicle trajectory and the lane to be bound is obtained. Each feature is multiplied by its own weight, and the results of the multiplication are added together to meet the following formula:

[0053] Among them, S (V t , e) is the binding score, W1 is the first weight, W2 is the second weight, W3 is the third weight, W4 is the fourth weight, f dist is the distance feature, f angele is the angle feature, f speed is the speed characteristic, f history It is a historical feature.

[0054] It should be noted that when a turn signal is detected, the second weight is increased by a preset value to achieve dynamic adjustment of the weight.

[0055] In one embodiment, the first weight is 0.4, the second weight is 0.3, the third weight is 0.2, and the fourth weight is 0.1. Then the binding score is:

[0056] The vehicle determines the binding score between its trajectory and each lane to be bound while complying with traffic rules, such as the prohibition of crossing solid lines. Figure 3 is a flowchart of determining a bound lane according to another exemplary embodiment. Figure 3 As shown, the bound lane of the vehicle trajectory is determined according to the binding score, specifically including: If the binding score of each target video frame in a first preset number of consecutive first target video frames is greater than a first preset threshold, an intermediate pairing set is obtained, where the intermediate pairing set is a pairing of the vehicle trajectory and the lane to be bound.

[0057] For a first predetermined number of consecutive first target video frames, a binding score is obtained for each vehicle trajectory and lane to be bound in each frame. If the binding scores for a vehicle trajectory and lane to be bound are all greater than a first predetermined threshold in consecutive video frames, the vehicle trajectory and lane to be bound combinations with the binding scores that meet the condition are retained to form an intermediate pairing set. The intermediate pairing set contains multiple combinations of vehicle trajectories and vehicles to be bound.

[0058] In one embodiment, the first preset number is 3, the first preset threshold is 0.6, the lanes to be bound include the first lane, the second lane, and the third lane, and the binding scores of the vehicle trajectory to the first lane in three consecutive frames are 0.6, 0.7, and 0.8, respectively; the binding scores of the vehicle trajectory to the second lane in three consecutive frames are 0.8, 0.8, and 0.9, respectively; and the binding scores of the vehicle trajectory to the third lane in three consecutive frames are 0.3, 0.7, and 0.8, respectively. The vehicle trajectory is retained in the first lane, and the vehicle is retained in the second lane.

[0059] In the intermediate pairing set, if in a second preset number of consecutive second target video frames, the binding score of each consecutive second target video frame is greater than a second preset threshold, then the current to-be-bound lane is determined to be a bound lane of the vehicle trajectory, wherein the first preset number is less than the second preset number, and the first preset threshold is less than the second preset threshold.

[0060] Based on the intermediate pairing set, a second preset number of consecutive second target video frames are tested. If a binding score is obtained for each of the second preset number of consecutive second target video frames, and if the binding score for the vehicle trajectory and the lane to be bound is greater than a second preset threshold in consecutive video frames, the vehicle trajectory and lane to be bound combinations with the binding score that meet the condition are retained.

[0061] In another embodiment, the second preset number is 5, the second preset threshold is 0.8, and in the intermediate pairing set, the binding scores of the vehicle trajectory in 5 consecutive frames of the second target video frame and the first lane are 0.6, 0.7, 0.8, 0.6, and 0.8 respectively; the binding scores of the vehicle trajectory in 5 consecutive frames of the second target video frame and the second lane are 0.8, 08, 0.9, 0.9, and 0.8 respectively, then the second lane can be determined as the bound vehicle of the vehicle.

[0062] If the vehicle trajectory meets the lane change conditions, the binding relationship between the vehicle trajectory and the bound lane is released, and the vehicle is bound to the lane again.

[0063] The vehicle's lane change conditions must simultaneously meet the following: the vehicle's speed is greater than a preset speed threshold, the vehicle's yaw angle is greater than a preset yaw angle threshold, and the lane marking is a dashed line. For example, the vehicle's lane change conditions must meet the following conditions: the vehicle's speed is greater than 0.5 m / s, the vehicle's yaw angle is greater than 15°, and the lane marking is a dashed line.

[0064] When the vehicle trajectory meets the lane change conditions, it means that the vehicle has left the bound lane. Therefore, when it is detected that the vehicle meets the lane change conditions, lane binding is performed again based on the vehicle's position to determine a new bound lane.

[0065] In another embodiment, Figure 4FIG. 1 is a flowchart illustrating lane binding according to another exemplary embodiment, wherein the first preset number is 3, the second preset number is 5, the first preset threshold is 0.6, and the second preset threshold is 0.8. Figure 4 As shown in the figure, when a drone detects a new vehicle during vehicle detection, if the vehicle is not bound to a lane, the vehicle's trajectory is calculated to have a score greater than 0.6 over three consecutive frames relative to the lane to be bound. This preliminarily determines the possibility of the vehicle being associated with a lane. Furthermore, if the vehicle's trajectory has a score greater than 0.8 over five consecutive frames relative to the lane to be bound, the vehicle's trajectory is bound to the vehicle. If the vehicle changes lanes, the lanes are re-bound to determine a new lane. If the vehicle is suddenly obscured during detection, the binding score continues to decrease. If the binding score in a single frame is less than 0.4 during the preliminarily determined possibility of the vehicle being associated with a lane, rebinding is performed. If the binding score for two consecutive frames is less than 0.2, rebinding is performed.

[0066] Continuing with step S102, the vehicle trajectory is bound to the lane it is traveling in, providing physical constraints for the vehicle. Constraining subsequent trajectory generation based on the binding relationship between the vehicle trajectory and the bound lane provides a basis for trajectory changes and greater accuracy.

[0067] Step S103: When the UAV detects that the vehicle is blocked, it generates at least two candidate trajectories based on the bound lanes and the lane information in the lane topology map, and determines the target trajectory from the candidate trajectories through the decision model.

[0068] When a drone is tracking a vehicle, it needs to detect whether the vehicle is obscured. Detecting obscuration specifically includes: For a third preset number of consecutive video frames, an average confidence level is calculated based on the confidence level of the vehicle identified in each frame. A confidence level drop rate is then calculated based on the average confidence level and the confidence level of the last consecutive video frame. If the drone fails to detect the vehicle in the consecutive video frames and the confidence level drop rate exceeds a preset confidence threshold, the drone detects that the vehicle is obscured. The confidence level drop rate satisfies the following criteria: the ratio of the difference between the average confidence level and the confidence level of the last frame over N consecutive frames to the average confidence level. The confidence level represents the probability that the drone has recognized the vehicle.

[0069] In one embodiment, the third preset number is 3. If the drone does not detect a vehicle in three consecutive video frames and the confidence drop rate of the three consecutive frames exceeds 30%, the drone detects an occlusion event, that is, detects that the vehicle is occluded.

[0070] By detecting multiple consecutive frames and adding the confidence change rate as a detection indicator, it is possible to effectively distinguish between real vehicle occlusion and unrecognizable content caused by brief jitter, thereby improving the accuracy of detecting vehicle occlusion.

[0071] Continuing with step S103, when the drone detects that the vehicle is obscured, it generates at least two candidate trajectories based on the bound lanes and lane information in the lane topology map, specifically including: Step S131: Determine a lane to be matched that the vehicle can drive to from the bound lane based on the lane speed limit information and lane turning information of each lane in the lane topology map.

[0072] The vehicle's bound lane is used as the starting lane. Adjacent lanes are determined based on the lane topology. Based on the speed limit and lane turning information in each lane, the matching lanes to which the vehicle can change from the bound lane are determined. The matching lanes represent lanes to which the vehicle can change from the bound lane, subject to traffic regulations and lane turning and speed limits, to ensure passability between lanes.

[0073] Step S132: Obtain the distance traveled by the vehicle during the occlusion period, and generate at least two candidate trajectories based on the travel distance and the lane to be matched.

[0074] Obtain the vehicle's operating parameters and determine the distance traveled during the time the vehicle was obscured. Based on the distance traveled and the lane to be matched, generate at least two candidate trajectories between the bound lane and the lane to be matched.

[0075] After determining the candidate trajectories, for each candidate trajectory, the predicted speed of the vehicle in the candidate trajectory is obtained. Based on the actual speed and the predicted speed of the vehicle, a kinematic score is obtained to determine whether the candidate trajectory meets the vehicle kinematic constraints. The kinematic score specifically satisfies:

[0076] Among them, KinematicScore is the kinematic score, v pred is the predicted velocity, v hist is the actual speed, σ v is the historical velocity standard deviation.

[0077] Based on the distance between the candidate trajectory and the center of the adjacent lane, a topological consistency score is obtained to determine the degree of matching between the candidate trajectory and the road topology structure. The topological consistency score satisfies:

[0078] Among them, TopSore is the topological consistency score, di is the minimum distance from the candidate trajectory to the adjacent lane, σ d is the distance tolerance factor.

[0079] In the topological consistency score, a larger distance tolerance coefficient indicates that the candidate trajectory can tolerate greater deviations from the center of the adjacent lane. A smaller distance tolerance coefficient indicates that the candidate trajectory can tolerate less deviations from the center of the adjacent lane. A smaller distance tolerance coefficient indicates that the candidate trajectory is closer to the lane.

[0080] When the vehicle reappears in the drone's field of view, the vehicle's features and location information are obtained. Based on the similarity between the vehicle's visual features and the corresponding visual features of the candidate track, as well as the norm of the location information and the location information of the candidate track, a re-identification score is determined to determine the degree of match between the vehicle's visual features after the occlusion is removed and the visual features before the occlusion. The re-identification score satisfies: , Among them, ReIDScore is the re-identification score, feat is the similarity, and pos is the norm of the position information.

[0081] The optimization score is obtained based on the kinematic score, topological consistency score and re-identification score. The optimization score satisfies: , where OptimalScore is the optimal score, KinematicScore is the kinematic score, TopoScore is the topological consistency score, ReIDScore is the re-identification score, w1 is the first optimization weight, w2 is the second optimization weight, and w3 is the third optimization weight.

[0082] The duration of vehicle occlusion is divided into three stages: the initial occlusion period, which lasts for the first preset duration; the mid-occlusion period, which starts at the end of the first preset duration; and the late occlusion period, which lasts for the third preset duration. The first, second, and third optimization weights are dynamically configured based on the vehicle occlusion stage.

[0083] Optionally, the first preset duration is 2 seconds, the second preset duration is 5 seconds, and the third preset duration is greater than 10 seconds. The weight distribution during the initial occlusion period (0-2 seconds) is: w1=0.7, w2=0.3, w3=0; the weight distribution during the middle occlusion period (3-8 seconds) is: w1=0.4, w2=0.5, w3=0.1; and the weight distribution during the long occlusion period (greater than 8 seconds) is: w1=0.2, w2=0.3, w3=0.5.

[0084] The candidate trajectory with the highest optimization score is used as the target trajectory. The target trajectory is then smoothed to make it smoother. Optionally, the smoothing method uses topologically constrained B-spline smoothing, specifically satisfying the following formula:

[0085] Among them, Q ( t i ) is the original trajectory point of the target trajectory, P( t i ) is the smoothed trajectory point, L j The center line of the topological lane, λ is the constraint weight, P(uj) is the target trajectory after smoothing under the parameter u j The three-dimensional coordinates, u j are the parameters of the B-spline curve, which are used to locate the points to be constrained on the target trajectory.

[0086] After obtaining the optimization scores, if the difference between the optimization scores of at least two candidate trajectories is less than a preset difference threshold, the straight candidate trajectory is selected as the target trajectory. Optionally, the preset difference threshold is 5%. If the difference in the optimization scores of at least two candidate trajectories is less than 5%, indicating that the trajectory paths of the two candidate trajectories are very similar, the straight candidate trajectory is preferentially selected as the target trajectory.

[0087] If any two candidate trajectories intersect, the candidate trajectory with a predicted speed greater than a preset speed threshold is selected as the target trajectory. When candidate trajectories intersect, a speed-priority strategy is adopted, with the candidate trajectory with a predicted speed greater than a preset speed threshold selected as the target trajectory.

[0088] If the target trajectory cannot be determined from the candidate trajectories within the target preset time, the manual annotation interface will be started for manual selection.

[0089] Continuing to refer to step S103, the accuracy of detecting vehicle occlusion is improved through dual detection of vehicle occlusion using multi-frame detection and confidence reduction rate. An accurate mapping relationship is established between the vehicle and the lane in the lane topology map. After accurately detecting that the vehicle is occluded, the bound lane is used as the starting lane, and candidate trajectories are formed in combination with the lane topology map. The candidate trajectories meet the traffic rules and the geometric requirements of the lanes, and each candidate trajectory is achievable, avoiding the generation of unreasonable trajectories. The target trajectory is determined in the candidate trajectories through the kinematic score, topological consistency score and re-identification score, so that the target trajectory can be highly consistent with the actual situation, thereby improving the accuracy of the target trajectory.

[0090] Step S104: If there are at least two target trajectories and the target trajectories meet the preset separation condition, the target trajectories are merged to obtain a complete trajectory.

[0091] When a vehicle is blocked, at least two target trajectories are generated. Alternatively, if there are multiple vehicles on the lane, different target trajectories are generated for different vehicles.

[0092] When it is determined that the vehicle is obscured, the target trajectory is checked to see if it meets the preset separation criteria. Specifically, if there are at least two target trajectories, a set of isolated trajectory segments is created for the determined target trajectories. For each target trajectory in the set of isolated trajectory segments, if the difference between the end time and the start time of each target trajectory is within a preset time range, and if a connected path exists between the start and end points of the target trajectory, as verified by traffic regulations and lane steering, the type of vehicle traveling on the target trajectory is obtained, and the type of vehicles traveling on different target trajectories is determined to be consistent.

[0093] The operating speed determines the range of the vehicle's obstructed driving duration. For example, if the operating speed is less than 5 m / s, the maximum obstructed driving duration is 60 seconds. If the operating speed is greater than or equal to 5 m / s and less than 10 m / s, the maximum obstructed driving duration is 30 seconds. If the operating speed is greater than or equal to 10 m / s, the maximum obstructed driving duration is 15 seconds.

[0094] When the target trajectory is determined to meet the preset separation conditions, the vehicle's running speed before being blocked and the driving time when blocked are obtained. Based on the predicted acceleration at the end of the target trajectory and the actual acceleration of the vehicle, a motion continuity score is obtained to verify whether the target trajectory meets the running continuity. The motion continuity score specifically meets the following requirements:

[0095] Among them, ConsistencySore is the motion continuity score, n is the number of historical frames to verify the continuity of the trajectory, and a k pered is the predicted acceleration at the end of the k-th target trajectory, a k actual is the acceleration of the vehicle.

[0096] If the driving duration is within the driving time range of the running speed and the motion continuity score meets the continuity requirement, the fusion order of the target trajectory is determined by the driving duration and driving time to provide a guarantee for forming a complete trajectory and avoid a complete trajectory with time error.

[0097] Based on the fusion order, a complete trajectory is formed through the Markov decision model. The complete trajectory obtained through the Markov decision model specifically includes: The state space is constructed based on the target trajectory. The state space is the set of all vehicle states. The state space satisfies: Position status:s p ∈{lane keeping, lane changing, intersection turning} Speed ​​status:s v ∈{acceleration, uniform speed, deceleration} Topology status: s t ∈{main road, ramp, intersection} The state space is used as the input of the Markov decision model to ensure that the trajectory prediction conforms to the physical and regulatory constraints of the actual driving scenario.

[0098] The transition probability matrix is ​​determined based on the actual traffic flow data of the current lane. For example, the current position state is used as the row of the transition probability matrix, and the next position state is used as the column of the transition probability matrix. Each element in the transition probability matrix represents the probability of transitioning to the next position state after performing an action in the current position state, that is, P(s'|s,a), where s' is the next position state, s is the current position state, and a is the switching action. The dimension of the transition probability matrix is ​​determined by the state space. Among them, the switching action refers to the action of the vehicle switching lanes, including turning left, turning right, and going straight. The transition probability matrix satisfies:

[0099] According to the above transition concept matrix and state space, the complete trajectory is obtained through the decision function. Specifically, in the Markov decision model, the state space provides the values ​​of the current state s and the next state s', the transition probability matrix provides the value of P(s'|s,a), the reward function R(s,a) evaluates the immediate benefit of the action, and the value function Evaluate the long-term benefits of the action to determine the decision function. The decision function satisfies:

[0100] Among them, π(s) is the decision function, R(s,a) is the immediate reward, γ is the discount factor, is the state value function, P(s'|s,a) is the state transition function, which represents the probability that the system transfers to s' after executing action a in state s, a is the switching action, and A is the set of vehicle switching actions.

[0101] The vehicle's switching action is determined by the decision function, but the vehicle trajectory is a continuous sequence of path points. A kinematic model is used to convert the switching action into specific vehicle control commands (such as steering wheel angle and acceleration). Then, based on the geometric centerline of the bound lane and the kinematic constraints, a smooth trajectory point is generated. Therefore, the complete trajectory is smoothed using a B-spline smoothing constraint to ensure that the rate of change of acceleration in the complete trajectory conforms to vehicle dynamics. The B-spline smoothing constraint specifically satisfies:

[0102] Among them, T fused (t) is the complete trajectory function, which represents the spatial position of the vehicle at time t, is the rate of change of acceleration, a max is the acceleration rate of change threshold.

[0103] The target trajectory timestamps during occlusion are predicted based on the vehicle's speed, for example, one trajectory point every 0.1 seconds. Multiple target trajectories are then temporally aligned based on the timestamps. If any two target trajectories overlap in time, they are weighted and fused, and the fused trajectory points are smoothed using topological constraints.

[0104] Continuing with step S104, the timing information is used and the fusion order is determined by the driving duration and motion continuity score to ensure that the connection between the target trajectories conforms to the temporal and spatial consistency, avoiding the chaotic connection of target trajectories in different time periods. In addition, through B-spline smoothing and target trajectory fusion, a high-precision complete trajectory can be obtained, overcoming the problem of fragmentation of different target trajectories.

[0105] In step S101, the target video frame is processed by a dual-branch hybrid neural network to obtain the pixel coordinates of the lane line, which specifically includes: The local geometric features and global context information of the lane in the target video frame are captured through a two-branch hybrid neural network. Among them, the local geometric feature information includes the lane line and the indicator arrow in the lane. The global context information includes the background information in the video frame and the association relationship between different objects. If part of the content in the lane is blocked, making the local geometric features incomplete, the missing content of the local geometric features is supplemented by the global context information to obtain the complete lane features. Therefore, based on the local geometric features and the global context features, the pixel coordinates of the lane line in the target video frame are obtained. The confidence score of each pixel coordinate is calculated, and the pixel coordinates with a confidence score greater than the preset score threshold are retained. Among them, the confidence score satisfies the following formula:

[0106] in, S i It isi The confidence score of the pixel coordinates, f frature ( u i , v i ) is the pixel coordinate of the lane line, MLP ( f frature (u i , v i )) is the multi-layer perceptron processing of the pixel coordinates of the lane line, and δ is the activation function used to map the MLP input to a probability value.

[0107] Convert the retained pixel coordinates from the image coordinate system to the world coordinate system so that the vehicle's position in the lane topology can be directly determined by the pixel coordinates, satisfying the following formula:

[0108] Among them, X i is the horizontal coordinate in the world coordinate system, Y i is the ordinate in the world coordinate system, f is the focal length of the camera assembly, R(Φ,θ) is the selection matrix, z is the altitude of the drone, u i is the horizontal coordinate of the pixel coordinate in the picture, v i is the vertical coordinate of the pixel coordinate in the picture, x t is the horizontal position coordinate of the UAV at time t, y t is the vertical position coordinate of the UAV at time t, Φ is the yaw angle of the UAV, and θ is the pitch angle of the UAV.

[0109] After determining the pixel coordinates, they need to be clustered. The clustering process also includes calculating the comprehensive distance of the geometric and semantic features of the fused lanes to distinguish lanes with different attributes, such as the main road and the emergency lane. The comprehensive distance satisfies:

[0110] Among them, d combined is the comprehensive distance, α is the empirical optimization value, d euclidean It is a geometric distance item that represents the three-dimensional Euclidean distance between lane line pixel coordinates. is the semantic feature vector of the current lane line pixel coordinate, is the semantic feature vector of the lane line, It is the feature difference term, which represents the L2 norm distance between the two.

[0111] In step S101, after obtaining the target video frame, it is necessary to correct the drone's perspective distortion. Correcting the target video frame specifically involves determining a homography matrix based on the drone's altitude and pitch angle. For each target video frame, a perspective transformation is performed based on the homography matrix. This eliminates perspective changes caused by the drone's posture changes, ensuring geometric consistency for subsequent lane detection.

[0112] In one embodiment, Figure 5 is a flow chart of a method for restoring a vehicle trajectory in an occluded state according to another exemplary embodiment. Figure 5 As shown, a video of a vehicle is captured from a drone's camera. Based on SIFT feature changes, target video frames are identified within the video. Vehicle detection and tracking are performed within the video frames, and a lane topology map is constructed. The vehicle trajectory is calculated and bound to the lane. Based on this, a dual detection method based on the absence of vehicle detection in consecutive frames and the rate of confidence drop in vehicle detection is used to detect whether the vehicle is occluded. If the vehicle is not occluded, the vehicle is tracked to obtain its trajectory. If the vehicle is occluded, topology-constrained trajectory detection is performed to obtain candidate trajectories, and a target trajectory that matches the actual trajectory is determined within the candidate trajectories. The target trajectory is then smoothed and fused to optimize it, and a complete trajectory is output.

[0113] In another embodiment, Figure 6 is a flow chart of a method for restoring a vehicle trajectory in an occluded state according to another exemplary embodiment. Figure 6 As shown, a video of a vehicle is captured from the drone's camera assembly. Based on SIFT feature changes in the video, the target video frame is identified. Vehicle detection and tracking are performed within the video frames, and a lane topology map is constructed. The lane topology map and vehicle detection evidence are analyzed by the trajectory-lane analysis module. In the trajectory-lane analysis module, the vehicle trajectory is bound to the lane. When an occlusion is detected, a topology-constrained trajectory prediction is performed, ensuring that the predicted trajectory is complete and consistent with the actual trajectory. If the vehicle is not detected as occluded, the vehicle's trajectory is recorded. Ultimately, a complete vehicle trajectory is formed based on the unobstructed and occluded trajectories.

[0114] In summary, the embodiments of the present application provide a method, device and computer equipment for recovering vehicle trajectories under occlusion. The method constructs a lane topology map through the target video frame of the drone, and implements the binding of the vehicle and the lane through lane geometry constraints and traffic rule constraints in the lane map topology map, providing physical constraints to ensure that the generated path conforms to the actual road conditions. During the period of vehicle occlusion, candidate trajectories are generated, and the candidate trajectories conform to topological consistency, kinematic continuity, and temporal continuity, and the accuracy of the candidate trajectories can be generated. Through timing information and B-spline smoothing, a globally consistent complete trajectory is obtained, avoiding the generation of a fragmented trajectory, so that the complete trajectory can be close to the actual trajectory, thereby improving the accuracy of the recovered trajectory.

[0115] In a second aspect, an embodiment of the present application provides a vehicle trajectory recovery device in an obstructed state. Figure 7 FIG. 1 is a block diagram of a vehicle trajectory recovery device under an occlusion state according to an exemplary embodiment. Figure 7 As shown, the vehicle trajectory recovery device under occlusion is applied to a drone equipped with a camera assembly. The vehicle trajectory recovery device under occlusion includes: a lane topology map construction module for obtaining a target video frame shot by the drone of a vehicle driving, clustering lane pixel coordinates in the target video frame to obtain intersections, and constructing a lane topology map with intersections as nodes and connections between intersections as edges; A lane and vehicle trajectory binding module is used to determine at least two lanes to be bound based on the vehicle's driving status in the lane topology map, calculate the binding score between the vehicle and each lane to be bound, and determine the corresponding bound lane for the vehicle based on the binding score; The target trajectory determination module is used to generate at least two candidate trajectories based on the lane information in the bound lane and lane topology map when the UAV detects that the vehicle is obscured, and determine the target trajectory from the candidate trajectories through the decision model; The target trajectory fusion module is used to fuse the target trajectories to obtain a complete trajectory if there are at least two target trajectories and the target trajectories meet the preset separation conditions.

[0116] In summary, the vehicle trajectory recovery device under occlusion state provided by the present application constructs a lane topology map by shooting the target video frame of the vehicle driving through a drone. By calculating the binding score between the lane to be bound and the vehicle, binding the vehicle and the corresponding bound lane, and binding the vehicle to the bound lane, a physical constraint is provided between the vehicle and the bound lane, so that the vehicle uses the lane information in the bound lane as a constraint condition in the occlusion scene, ensuring the continuity of the vehicle's running trajectory in the occlusion scene; when it is detected that the vehicle is occluded, at least two candidate trajectories are generated according to the lane information in the bound lane and the lane topology map, wherein the bound lane and lane information provide lane geometry constraints and traffic rules constraints for generating candidate trajectories, ensuring that the candidate trajectories conform to the actual situation, thereby determining the target trajectory in the candidate trajectories to ensure that the target trajectory is close to the actual trajectory of the vehicle, ensuring the accuracy of the target trajectory. The target trajectories that meet the preset separation conditions are merged to obtain a globally consistent complete trajectory, thereby overcoming the trajectory fragmentation problem caused by frame-by-frame processing.

[0117] It should be noted that the vehicle trajectory recovery device under occlusion provided in this embodiment is used to implement the above-mentioned embodiments, and the previously described aspects will not be repeated. As used above, the terms "module," "unit," "subunit," etc. may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the above embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0118] In a third aspect, an embodiment of the present application provides an electronic device, Figure 8 FIG is a block diagram of an electronic device according to an exemplary embodiment. Figure 8 As shown, the electronic device may include a processor 81 and a memory 82 storing computer program instructions.

[0119] Specifically, the processor 81 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0120] Memory 82 may include a large-capacity memory for data or instructions. By way of example, and not limitation, memory 82 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 82 may include removable or non-removable (or fixed) media. Where appropriate, memory 82 may be internal or external to the data processing device. In certain embodiments, memory 82 is non-volatile memory. In certain embodiments, memory 82 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), where the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data out dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0121] The memory 82 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 81 .

[0122] The processor 81 reads and executes computer program instructions stored in the memory 82 to implement vehicle trajectory recovery in any of the above-mentioned embodiments under any of the occlusion states.

[0123] In one embodiment, the vehicle trajectory recovery device in the obstructed state may further include a communication interface 83 and a bus 80. Figure 8 As shown, the processor 81, the memory 82, and the communication interface 83 are connected via a bus 80 and communicate with each other.

[0124] The communication interface 83 is used to implement communication between the various modules, devices, units, and / or devices in the embodiments of the present application. The communication interface 83 can also implement data communication with other components such as: external devices, image / data acquisition equipment, databases, external storage, and image / data processing workstations.

[0125] Bus 80 , which includes hardware, software, or both, couples the components of the vehicle trajectory recovery device under obstruction. Bus 80 includes, but is not limited to, at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 80 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Bus 80 may include one or more buses, where appropriate. Although embodiments herein describe and illustrate a particular bus, this application contemplates any suitable bus or interconnect.

[0126] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the vehicle trajectory recovery method under an occlusion state provided in the first aspect is implemented.

[0127] The readable storage medium may include, but is not limited to, a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0128] In a possible embodiment, the present invention can also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of the vehicle trajectory recovery method under the occlusion state provided in the first aspect.

[0129] The program code for executing the present invention may be written in any combination of one or more programming languages, and may be executed entirely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or entirely on the remote device.

[0130] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0131] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for recovering vehicle trajectory under occlusion, characterized in that: The method is applied to a drone carrying a camera assembly, and the method includes: Obtain a target video frame of a vehicle traveling shot by the drone, cluster lane pixel coordinates in the target video frame to obtain intersections, and construct a lane topology map using the intersections as nodes and the connections between the intersections as edges; In the lane topology map, determining at least two lanes to be bound according to the driving state of the vehicle, calculating a binding score between the vehicle trajectory and each of the lanes to be bound, and determining a bound lane corresponding to the vehicle trajectory according to the binding score; When the UAV detects that the vehicle is obscured, generating at least two candidate trajectories according to the bound lane and lane information in the lane topology map, and determining a target trajectory from the candidate trajectories through a decision model; If there are at least two target trajectories, and the target trajectories meet the preset separation condition, the target trajectories are fused to obtain a complete trajectory.

2. The vehicle trajectory recovery method under occlusion state according to claim 1, characterized in that: The driving state includes the vehicle's position information, heading angle, and driving speed. The calculating of the binding score between the vehicle trajectory and each lane to be bound includes: For any lane to be bound, determining a distance feature based on the distance between the position information and the center of the lane to be bound; determining an angle feature according to the lane direction of the lane to be bound and the heading angle; determining a speed characteristic according to the lane speed limit of the lane to be bound and the driving speed; Based on the current moment, obtaining the binding score between the vehicle and the corresponding bound lane at the previous moment to determine the historical characteristics; A binding score between the vehicle trajectory and the lane to be bound is obtained based on the distance feature, the angle feature, the speed feature, and the history feature.

3. The vehicle trajectory recovery method under occlusion state according to claim 2, characterized in that: Determining a bound lane corresponding to the vehicle trajectory according to the binding score further includes: If, in a first preset number of consecutive first target video frames, the binding score of each of the target video frames is greater than a first preset threshold, an intermediate pairing set is obtained, where the intermediate pairing set is a pairing of the vehicle trajectory and the lane to be bound; In the intermediate pairing set, if, in a second preset number of consecutive second target video frames, the binding score of each of the consecutive second target video frames is greater than a second preset threshold, determining the current to-be-bound lane as the bound lane of the vehicle, wherein the first preset number is less than the second preset number, and the first preset threshold is less than the second preset threshold; If the vehicle trajectory satisfies the lane change condition, the binding relationship between the vehicle trajectory and the bound lane is released, and the vehicle is bound to the lane again.

4. The vehicle trajectory recovery method under occlusion state according to claim 1, characterized in that: Clustering lane pixel coordinates in the target video frame to obtain intersections, and constructing a lane topology graph with the intersections as nodes and connections between the intersections as edges, including: Clustering the lane pixel coordinates according to preset lane requirements to obtain a structured road network, and determining intersections in the structured road network according to preset curvature requirements; According to preset lane design requirements, a lane topology graph is constructed with the intersections as nodes and the connection relationships between the intersections as edges.

5. The vehicle trajectory recovery method under occlusion state according to claim 1, characterized in that: When the drone detects that the vehicle is blocked, the method includes: Obtain an average confidence level according to the confidence level of the vehicle identified in each frame in a third preset number of consecutive video frames, and obtain a confidence level decrease rate according to the average confidence level and the confidence level of the last consecutive video frame; If the drone does not detect the vehicle in the continuous video frames and the confidence decrease rate is greater than a preset confidence threshold, the drone detects that the vehicle is blocked.

6. The vehicle trajectory recovery method under occlusion state according to claim 5, characterized in that: The generating at least two candidate trajectories according to the bound lane and the lane information in the lane topology map includes: Determining, based on lane speed limit information and lane turning information of each lane in the lane topology map, a lane to be matched that the vehicle can drive to from the bound lane; Obtain a travel distance of the vehicle during the occlusion time, and generate at least two candidate trajectories based on the travel distance and the lane to be matched.

7. The vehicle trajectory recovery method under occlusion state according to claim 5, characterized in that: Determining the target trajectory from the candidate trajectories by using the decision model includes: For each candidate trajectory, obtaining a predicted speed of the vehicle in the candidate trajectory and a distance between the candidate trajectory and an adjacent lane; Obtaining a kinematic score based on the actual speed of the vehicle and the predicted speed; Obtaining a topological consistency score based on the distance between the candidate trajectory and the center of the adjacent lane; When the vehicle reappears, obtaining the characteristics and location information of the vehicle, and determining a re-identification score based on the similarity between the characteristics of the vehicle and the characteristics of the candidate trajectory, and the norm of the location information and the location information of the candidate trajectory; Obtaining an optimization score according to the kinematic score, the topological consistency score, and the re-identification score; The candidate trajectory with the highest optimization score is used as the target trajectory.

8. The vehicle trajectory recovery method under occlusion state according to claim 7, characterized in that: After determining the optimization score according to the kinematic score, the topological consistency score and the re-identification score, the method further includes: If the difference between the optimization scores of at least two candidate trajectories is less than a preset difference threshold, taking the straight candidate trajectory as the target trajectory; If any two candidate trajectories intersect, the candidate trajectory whose predicted speed of the vehicle is greater than a preset speed threshold is used as the target trajectory.

9. The vehicle trajectory recovery method under occlusion state according to claim 1, characterized in that: The preset separation conditions meet the following configurations: The difference between the end time and the start time of each target track is within a preset time range; The types of vehicles traveling on different target trajectories are consistent.

10. The vehicle trajectory recovery method under occlusion state according to claim 1, characterized in that: The step of fusing the target trajectories to obtain a complete trajectory includes: Obtaining the running speed of the vehicle before being blocked and the running time of the vehicle when being blocked; Obtaining a motion continuity score based on the predicted acceleration at the end of the target trajectory and the actual acceleration of the vehicle; If the driving duration is within the driving time range of the running speed and the motion continuity score meets the continuity requirement, a fusion order of the target trajectories is determined, and the target trajectories are fused according to the fusion order through the decision model to obtain a complete trajectory.

Images