A key parking scene extraction method for complex interactive parking data set

Abstract

The application belongs to the technical field of automobiles, and particularly relates to a key parking scene extraction method for complex interactive parking data sets. The method comprises the following steps: step one, multi-target trajectory data extraction; step two, state estimation of each target according to a detection result to generate time sequence trajectory data containing position, speed and acceleration; step three, a parking behavior recognition model based on multi-feature fusion is used to screen the trajectory of the parking behavior from the extracted trajectory sequence to realize the judgment of the parking behavior; step four, a key traffic participant recognition model is constructed to intercept the trajectory sequence of other traffic participants in the range; step five, an interactive intensity quantitative index is designed to comprehensively consider the spatial proximity and the maneuvering degree to quantitatively determine whether the interaction between the host vehicle and the traffic participants occurs, and a neighborhood traffic participant set is screened; and step six, a parking scene key index is designed to extract the key parking scene from the entire parking data set. The application can be used for key parking scene extraction of parking data sets with complex interaction, and is more concerned about the influence of other traffic participants on the decision planning of the host vehicle in the parking process. The extracted key parking scene is real, and can be used for scene testing of an intelligent parking system.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of automobiles, in particular to a key parking scene extraction method for complex interactive parking data sets. BACKGROUND

[0002] As a core component of future transportation systems, intelligent vehicle technology is working to solve a series of serious traffic problems faced by traditional vehicles. In order to ensure the reliability and safety of intelligent vehicle technology, a scientific and rigorous testing and evaluation method is indispensable. Traditional testing methods cannot fully meet the performance verification needs of intelligent vehicles in complex and variable real traffic environments. Therefore, scene-based testing methods have gradually become the mainstream, which focuses on simulating real driving scenes and can more comprehensively evaluate the ability of intelligent vehicles to respond to sudden conditions, complex interactions and environmental changes.

[0003] Although there are many scene testing technologies for high-difficulty scenes such as lane changing and following, emergency braking in the field of intelligent vehicle research, the research on parking scenes in low-speed complex interactive environments is relatively insufficient. The urban parking environment brings a series of challenging tasks such as narrow lane reversing, parallel parking, obstacle avoidance driving to intelligent parking systems due to its unique space constraints, frequent human-vehicle interactions and dynamic obstacles, which puts higher requirements on the real-time perception, accurate decision-making and dynamic planning capabilities of intelligent parking systems. Therefore, in order to comprehensively and rigorously test and evaluate intelligent parking systems, a key parking scene extraction method for complex interactive parking data sets is urgently needed. SUMMARY

[0004] To solve the above technical problems, the application provides a key parking scene extraction method for complex interactive parking data sets, which can be used for key parking scene extraction of parking data sets with complex interactions, and focuses more on the influence of other traffic participants on the decision planning of the vehicle during parking. The extracted key parking scenes are real and can be used in the scene testing process of intelligent parking systems.

[0005] The technical scheme of the application is explained as follows in combination with the drawings:

[0006] A key parking scene extraction method for complex interactive parking data sets, comprising the following steps:

[0007] Step one, multi-target trajectory data extraction, using the target detection model YOLO and the multi-target tracking algorithm DeepSORT to detect and track the moving targets in the parking data set;

[0008] Step two, according to the detection result, the state of each target is estimated, and the time sequence trajectory data containing position, speed, heading angle and acceleration are generated;

[0009] Step three, a parking behavior recognition model based on multi-feature fusion is used to screen the trajectory of parking behavior from the extracted trajectory sequence; the parking behavior recognition model integrates the key dynamic features of the vehicle at the terminal of the trajectory and the position information of the idle parking space to realize the judgment of the parking behavior;

[0010] Step four, a key traffic participant recognition model is constructed, and the interaction range is determined according to the speed of the host vehicle and the traffic flow density in the parking process, and the trajectory sequence of other traffic participants in the range is intercepted;

[0011] Step five, an interaction intensity quantification index is designed, and the spatial proximity and maneuvering degree are considered to quantitatively judge whether the interaction between the host vehicle and the traffic participants occurs, and the neighborhood traffic participant set is screened;

[0012] Step six, a key parking scene index is designed, and the overall key of the parking scene is evaluated in combination with the interaction intensity, parking space and other factors, and finally the key parking scene is extracted from the whole parking data set.

[0013] Further, the specific method of step one is as follows:

[0014] 11) Target detection, YOLO is used for real-time target detection of each frame image of the parking lot monitoring video; YOLO predicts the bounding box and corresponding class probability of all targets in the image in a single forward propagation process through a convolutional neural network architecture; the output result contains the center coordinates, width, height and confidence score of each target;

[0015] 12) Target tracking, all targets detected by YOLO in the first frame of the video are used as initial trajectories, and a unique tracking ID is assigned to each target, and the detection result of the first frame is recorded as the initial state; for each known tracking target, the Kalman filter is used to predict the target state of the next frame through the current frame state, and the estimated value is corrected according to the YOLO target detection result of the next frame; DeepSORT uses the target appearance features extracted by CNN and the target motion features calculated by Kalman filter to measure the similarity between the current frame detection result and the historical appearance features or predicted motion features, and obtains the cost matrix; the Hungarian algorithm is used to process the cost matrix to find the optimal matching between the detection result and the known target trajectory; for the successfully matched detection result, the state of the corresponding target trajectory is updated; for the detection result that is not matched to any known trajectory, a new target trajectory is created and added to the tracking list; for the existing trajectory that fails to match the detection result for 100 consecutive frames, the trajectory is terminated.

[0016] Further, the specific method of step two is as follows:

[0017] 21) The bounding box coordinates of the target detected by YOLO are regarded as spatial position features;

[0018] 22) The motion state is calculated according to the spatial position features, specifically as follows:

[0019] (1) Velocity estimation, the position difference of the target detection frame between two frames is used to calculate the instantaneous velocity of the target combined with the frame rate, which is expressed as:

[0020]

[0021] Wherein, the frame rate is 25fps; Δt is 0.04s; (x1, y1) is the center coordinate of the target detection frame in the previous frame; (x2, y2) is the center coordinate of the same target detection frame in the next frame;

[0022] (2) Heading angle estimation, the heading angle is defined as the angle between the direction of target movement and the positive direction of the horizontal axis, and the spatial position relationship of the center position of the target detection frame between the continuous frames is compared to estimate the heading angle, which is expressed as:

[0023]

[0024] (3) Acceleration calculation, the acceleration is estimated by the second derivative of the position change of the target, that is, the time series of the velocity is differentiated, which is expressed as:

[0025]

[0026] Further, the specific method of step three is as follows:

[0027] 31) A parking behavior recognition model based on multi-feature fusion is established to recognize the parking behavior of the vehicle in the predetermined scene;

[0028] 32) The parking behavior recognition model realizes the judgment of the parking behavior by integrating the key dynamic features of the vehicle at the trajectory terminal, including the state features at the vehicle trajectory terminal, the spatial features related to the position of the idle parking space and the time features; wherein the vehicle trajectory terminal of the completed parking behavior has obvious identifiable state features, which are expressed as indicators considering the velocity, acceleration and heading angle features, specifically as follows:

[0029] (1) The indicator considering the velocity feature is expressed as:

[0030] VScore = 1 - Velocity_End / V_threshold

[0031] where V_threshold = 3 m / s is the velocity threshold, Velocity_End is the velocity at the end of the vehicle trajectory, the closer to zero the higher the score;

[0032] (2) The index considering the acceleration feature is represented as:

[0033] AScore = 1 - Acceleration_End / A_threshold

[0034] where A_threshold = 0.1 m / s is the acceleration threshold, Acceleration_End is the acceleration at the end of the vehicle trajectory, the closer to zero the higher the score; 2

[0035] (3) The index considering the heading angle feature is represented as:

[0036]

[0037] where I is the set of idle parking spaces, VehicleHeading_End represents the heading angle at the end of the vehicle trajectory, IdealSpotDirection i represents the ideal pointing angle of the i-th parking space;

[0038] (4) When the vehicle performs a parking behavior, its final position will be very close to the idle parking space, so by calculating the distance between the end of the vehicle trajectory and the center point of the idle parking space, as an important basis for judging the parking behavior; The index considering the trajectory end position feature is represented as:

[0039]

[0040] where DistanceToSpot i is the actual distance from the midpoint of the vehicle bounding box to the center point of the i-th parking space; SpotRadius i is half the width of the target parking space;

[0041] (5) The time a vehicle stays at a certain point or near a parking space is taken as a time feature for identifying parking behavior; The index considering the stopping time feature is represented as:

[0042] TScore = 1 - exp(-(TimeStopped / T_threshold))

[0043] where TimeStopped is the time the vehicle stays stationary in the target parking space area; T_threshold is the stopping time threshold, the longer the stopping time, the higher the probability of parking behavior;

[0044] ​The score distribution of the parking behavior recognition model considering the above indicators is in [0, 5], and considering the existence of non-standard parking drivers, the trajectory with a total index score exceeding 4 is considered as a parking trajectory:

[0045] Score=VScore+AScore+HScore+PScore+TScore.

[0046] Further, the specific method of step four is as follows:

[0047] 41) Construct a key traffic participant recognition model; the key traffic participant recognition model includes interaction range determination and interaction feature extraction to determine a set of neighborhood traffic participants;

[0048] 42) Focus on participants within the interaction range and possibly interacting with the host vehicle by the key traffic participant recognition model, extract and analyze the features of each participant within the defined interaction range; the specific method is as follows:

[0049] (1) Interaction range determination, for the host vehicle with parking behavior, the range radius determined by considering the vehicle speed and the basic safety distance is represented as:

[0050] InteractiveRadius=GeometricDistance+SafetyBaseDistance+Speed*k1

[0051] Wherein, GeometricDistance is half of the diagonal line in the simplified model of the vehicle body rectangle, which is related to the size of the vehicle body; SafetyBaseDistance is the basic safety distance constant; Speed is the speed of the host vehicle; k1 is the speed influence coefficient, which is related to the flow density in the parking lot and is set as a constant;

[0052] (2) Interaction feature extraction, intercept the trajectory data of other traffic participants within the interaction range and within the parking time range of the host vehicle, and sort them according to the time sequence, including position, speed, heading angle and acceleration sequence;

[0053] Further, the specific method of step five is as follows:

[0054] 51) Design an interaction intensity quantification index, analyze the trajectory features of the host vehicle parking trajectory and other traffic participants, design a quantification index considering the spatial proximity and maneuvering degree of the host vehicle and the participants, convert the extracted participant trajectory features into numerical values, calculate the interaction intensity score between each participant and the host vehicle, and screen out the participants who have a key impact on the parking behavior, represented as:

[0055] InteractionScore = SpatialProximityScore + ManeuverDegree

[0056] where InteractionScore is the interaction intensity score of the host vehicle with any participant; SpatialProximityScore is the spatial proximity; ManeuverDegree is the maneuver degree;

[0057] 52) The closer the distance between the host vehicle and the participant within the interaction range, the greater the possibility of interaction, thus defining the spatial proximity, which is expressed as:

[0058] SpatialProximityScore = 1 - (Distance / InteractiveRadius)

[0059] where Distance is the distance between the host vehicle and any participant during the interaction;

[0060] Considering the acceleration and the rate of change of the heading angle of the participant within the interaction range, the maneuver degree is defined as the absolute value of the acceleration and the rate of change of the heading angle of the participant within the interaction range, which is expressed as:

[0061]

[0062] where Heading_change_threshold is the threshold of the rate of change of the heading angle, which is a constant; a(t) is the acceleration of any participant within the interaction range; A_threshold is the acceleration threshold; Heading(t) is the heading angle of any participant within the interaction range at time t; Heading(t-Δt) is the heading angle of the same participant at time t-Δt; and Δt is 0.04s.

[0063] Further, the specific method of step six is as follows:

[0064] 61) A parking operation difficulty index is designed to comprehensively reflect the influence of the parking space and the size of the parking space on the difficulty of the parking operation, which is expressed as:

[0065] (1) For a vertical parking space, the occupancy states of the left, right, and rear parking spaces are considered, and for a parallel parking space, the occupancy states of the front, rear, and side parking spaces are considered, and the operation difficulty score is designed to represent the size of the adjustable space of the vehicle during the parking process, which is expressed as:

[0066]

[0067] where N avail ​N is the number of available parking spaces; N max N is the total number of parking spaces; LineWidth is the width of the main road of the parking lot; when there is no occupation around the target parking space, N avail N is 3; when there is no available parking space, N avail N is 1, the path planning difficulty of the intelligent parking system increases;

[0068] (2) According to the size of the vehicle and the size of the parking space, the feasibility of successful parking is evaluated, and the parking space size matching score is designed, which is represented as:

[0069]

[0070] Where S match is the overlapping area of the size of the parking space and the size of the main vehicle; S main is the size of the main vehicle; when the parking space completely adapts to the size of the main vehicle, the PSSM value is 0, indicating the best match, and if it does not match, a value between 0 and 1 will be obtained.

[0071] (3) Finally, the parking operation difficulty index is calculated, which is represented as:

[0072] PSCI = A3S + PSSM

[0073] 62) According to the interaction intensity quantification index described in step five, the sum of the interaction intensity of each key traffic participant extracted and the parking space difficulty index are calculated as the quantification of the scene key, and the score exceeding a certain threshold is the key parking scene where interaction occurs, which is represented as:

[0074]

[0075] Where J is the set of neighborhood traffic participants, InteractionScore j is the interaction intensity score of each participant; w1 is the weight coefficient considering static obstacles; w2 is the weight coefficient considering dynamic interaction participants, and satisfies w1 + w2 = 1;

[0076] 63) Key parking scene extraction:

[0077] The extraction is as follows: obtaining a target trajectory sequence from the parking data set according to the method in step one and step two; selecting a plurality of dynamic features through analysis and comparison of the target trajectory sequence, and screening the trajectory sequence into a parking task class and a non-parking task class; for the parking task class, a spatial interaction range is determined according to the position and speed sequence of the host vehicle, and a neighborhood traffic participant set is screened from other participant sequences; when the distance between the traffic participant and the host vehicle is less than the interaction range, the trajectory start point is regarded, and when the distance is greater than the interaction range, the trajectory end point is regarded; according to the interaction intensity of the host vehicle and each key traffic participant calculated in step five, the spatial proximity and maneuvering degree indexes are calculated according to the position feature, acceleration feature and heading angle feature of each participant; according to the calculation of the parking operation difficulty index in step six, the parking space size and the main road width of the parking lot are known, and finally the key scene index is calculated.

[0078] The beneficial effects of the present application are:

[0079] The present application deeply analyzes the real parking scene with complex interaction, proposes a key parking scene extraction method, comprehensively considers the interaction intensity determined by dynamic traffic participants and the parking space determined by static traffic participants, and the extracted key scene has reality and representativeness, which is helpful for algorithm optimization of an intelligent parking system and improves the safety and efficiency of an autonomous vehicle in a complex parking environment. BRIEF DESCRIPTION OF DRAWINGS

[0080] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0081] Figure 1 Part of the flowchart of the present application;

[0082] Figure 2 Another part of the flowchart of the present application;

[0083] Figure 3 The shooting angle diagram of the parking lot video suitable for the present application;

[0084] Figure 4 The structure diagram of the position feature sequence;

[0085] Figure 5 The structure diagram of the speed feature sequence;

[0086] Figure 6 is a structural diagram of an acceleration feature sequence;

[0087] Figure 7 is a schematic diagram of the scenario described in step four;

[0088] Figure 8 is a schematic diagram of the extracted trajectory of the traffic participant;

[0089] Figure 9 is a schematic diagram of the key quantitative indicators of a certain scenario compared with TTC. DETAILED DESCRIPTION

[0090] The application will be described in further detail below with reference to the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the application, and not to limit the application. In addition, it should be noted that, for the sake of description, only the parts related to the application are shown in the drawings, not all the structures.

[0091] Embodiment one

[0092] Referring to Figure 1 and Figure 2 , the embodiment provides a key parking scene extraction method for complex interactive parking data set, including the following steps:

[0093] Step one, multi-target trajectory data extraction, using target detection model YOLOv5 and multi-target tracking algorithm DEEP-SORT to detect and track the moving targets in the parking data set, as follows:

[0094] 11) target detection, using YOLO to perform real-time target detection on each frame of image of the parking lot monitoring video; YOLO predicts the bounding box of all targets in the image and the corresponding class probability through the convolutional neural network architecture in a single forward propagation process; the output result contains the center coordinates, width, height and confidence score of each target;

[0095] 12) Target tracking, for all vehicle targets detected by YOLOv5, use DeepSORT algorithm to take all vehicle targets as initial tracking targets, assign a unique tracking ID to each target, and record the detection results of the first frame as the initial state; for each known tracking target (such as position, heading angle, speed and acceleration), DeepSORT uses Kalman filter to predict the target position of the next frame; update the predicted state according to the target state of the last frame, get the predicted bounding box; in the new frame image, YOLOv5 performs target detection again to produce new detection results; use the Hungarian matching algorithm to associate and match the predicted bounding box with the new detection results; by calculating the cost matrix, that is, the IoU feature distance in the non-deterministic state and the Mahalanobis distance and cosine distance in the deterministic state, associate the detection results with the existing tracking targets, and also identify possible new targets; according to the matching results, update the state of the existing tracking targets; if the predicted bounding box matches the detected bounding box successfully, update the position and other state parameters of the tracking target according to the detection results; for the old tracking targets that are not matched successfully, decide whether to continue tracking or end tracking according to the survival threshold; for the new detected bounding box that is not matched successfully with the existing targets, create a new tracking target and add it to the tracking queue.

[0096] Step two, according to the detection results, estimate the state of each target to generate time series trajectory data containing position, speed, heading angle and acceleration, as follows:

[0097] 21) Take the bounding box coordinates of the vehicle detected by YOLOv5 as spatial position features;

[0098] 22) Calculate the motion state according to the spatial position features, as follows:

[0099] (1) Speed estimation, use the position difference of the vehicle detection box between two frames to calculate the instantaneous speed of the vehicle combined with the frame rate, represented as:

[0100]

[0101] Where the frame rate is 25fps; Δt is 0.04s; (x1, y1) is the center coordinate of the vehicle detection box in the previous frame; (x2, y2) is the center coordinate of the same vehicle detection box in the next frame;

[0102] (2) Heading angle estimation, define the heading angle as the angle between the direction of vehicle movement and the positive direction of the horizontal axis, compare the spatial position relationship of the center position of the vehicle detection box between consecutive frames to estimate the heading angle, represented as:

[0103]

[0104] (3) Acceleration calculation, the acceleration is estimated by the second derivative of the vehicle position change, i.e. the differential calculation of the time series of the speed, which is expressed as:

[0105]

[0106] Step three, a parking behavior recognition model based on multi-feature fusion is used to screen the trajectory of the parking behavior from the extracted trajectory sequence; the parking behavior recognition model integrates the key dynamic features of the vehicle at the trajectory terminal and the idle parking space position information to realize the judgment of the parking behavior, which is specifically as follows:

[0107] 31) A parking behavior recognition model based on multi-feature fusion is established to identify the parking behavior of the vehicle in the predetermined scene;

[0108] 32) The parking behavior recognition model integrates the key dynamic features of the vehicle at the trajectory terminal, including the state features at the end of the vehicle trajectory, the space features related to the idle parking space position and the time features, to realize the judgment of the parking behavior; wherein the end of the vehicle trajectory with the completed parking behavior has obvious identifiable state features, which are expressed as the indexes considering the speed, acceleration and heading angle features, which are specifically as follows:

[0109] (1) The index considering the speed feature is expressed as:

[0110] VScore = 1 - Velocity_End / V_threshold

[0111] wherein V_threshold = 3 m / s is the speed threshold, Velocity_End is the speed at the end of the vehicle trajectory, and the closer to zero, the higher the score;

[0112] (2) The index considering the acceleration feature is expressed as:

[0113] AScore = 1 - Acceleration_End / A_threshold

[0114] wherein A_threshold = 0.1 m / s 2 is the acceleration threshold, Acceleration_End is the acceleration at the end of the vehicle trajectory, and the closer to zero, the higher the score;

[0115] (3) The index considering the heading angle feature is expressed as:

[0116]

[0117] where I is the set of free parking spots, VehicleHeading_End represents the heading angle at the end of the vehicle trajectory, IdealSpotDirection i represents the ideal pointing angle of the i-th parking spot.

[0118] (4) When the vehicle performs parking behavior, its final position will be very close to the free parking spot, so by calculating the distance between the end of the vehicle trajectory and the center of the free parking spot, it is an important basis for judging parking behavior. The index considering the position characteristics of the trajectory end is represented as:

[0119]

[0120] where DistanceToSpot i is the actual distance from the midpoint of the vehicle bounding box to the center of the i-th parking spot; SpotRadius i is half the width of the target parking spot.

[0121] (5) A normally driving vehicle will not stay at a fixed location for a long time, but a vehicle will remain stationary after completing parking. Therefore, the time a vehicle stays at a certain point or near a parking spot can be used as a time feature to identify parking behavior. The index considering the stay time feature can be represented as:

[0122] TScore = 1 - exp(-(TimeStopped / T_threshold))

[0123] where TimeStopped is the time the vehicle stays stationary in the target parking spot area; T_threshold is the stay time threshold, the longer the stay time, the higher the probability of parking behavior;

[0124] The score distribution of the parking behavior recognition model considering the above indexes is in [0, 5]. Considering the existence of non-standard parking drivers, trajectories with a total index score exceeding 4 are considered as parking trajectories:

[0125] Score = VScore + AScore + HScore + PScore + TScore.

[0126] Step four, build a key traffic participant recognition model, determine the interaction range according to the speed of the host vehicle and the density of the traffic flow, and intercept the trajectory sequence of other traffic participants in the range, as follows:

[0127] 41) Build a key traffic participant recognition model; the key traffic participant recognition model includes interaction range determination and interaction feature extraction to determine the set of neighborhood traffic participants.

[0128] 42) Focus on the participants in the interaction range with the main car distance and may produce interaction through the key traffic participant identification model, extract and analyze the characteristics of each participant in the limited interaction range; Specifically as follows:

[0129] (1) Interaction range determination, for the main car that occurs parking behavior, the range radius determined by considering the speed and the basic safety distance is represented as:

[0130] InteractiveRadius = GeometricDistance + SafetyBaseDistance + Speed * k1

[0131] Wherein, GeometricDistance is half of the diagonal line in the rectangular simplified model of the vehicle body, which is related to the size of the vehicle body; SafetyBaseDistance is the basic safety distance constant; Speed is the speed of the main car; k1 is the speed influence coefficient, which is related to the flow density in the parking lot, and is set as a constant;

[0132] (2) Interaction feature extraction, intercept the trajectory data of other traffic participants in the interaction range and the parking time range of the main car, and sort them according to the time sequence, including position, speed, heading angle and acceleration sequence;

[0133] Step five, design interaction intensity quantitative index, comprehensively consider the spatial proximity and maneuver degree to quantitatively distinguish whether the interaction between the main car and the traffic participant occurs, and screen the neighborhood traffic participant set, specifically as follows:

[0134] 51) Design interaction intensity quantitative index InteractionScore (the calculation method of interaction intensity quantitative index is to sum the spatial proximity and maneuver degree of the main car and each participant), analyze the trajectory characteristics of the main car parking trajectory and other traffic participants, and the interaction intensity quantitative index comprehensively considers the spatial proximity and maneuver degree of the main car and the participant. Convert the extracted participant trajectory characteristics into a numerical value, calculate the interaction intensity score between each participant and the main car, and screen out the participants who have a key influence on the parking behavior, represented as:

[0135] InteractionScore = SpatialProximityScore + ManeuverDegree

[0136] 52) In the interaction range of traffic participants, the closer the distance between the main car and the participant, the greater the possibility of interaction, so the spatial proximity is defined, represented as:

[0137] SpatialProximityScore = 1 - (Distance / InteractiveRadius)

[0138] wherein, is the distance between the host vehicle and other participants during the interaction;

[0139] Considering the behavior actions of the traffic participants within the interaction range and the host vehicle, the maximum acceleration and the maximum heading angle change rate are comprehensively considered to define the degree of maneuvering, which is calculated within the interaction range, and the maximum absolute value of the acceleration and the heading angle change rate of the host vehicle and the participants are represented as:

[0140]

[0141] wherein, in order to unify the dimension, the heading angle change rate threshold Heading_change_threshold is introduced, which is a constant;

[0142] Step six, design the key indicators of the parking scene, evaluate the overall key of the parking scene by combining the interaction intensity, parking space and other factors, and finally extract the key parking scene from the entire parking data set based on this, which is as follows:

[0143] 61) Design the parking operation difficulty index to comprehensively reflect the influence of the parking space and the parking space size on the parking operation difficulty, which is represented as:

[0144] (1) For the vertical parking space, the occupancy state of the left, right and rear parking spaces is considered, and for the parallel parking space, the occupancy state of the front, rear and side parking spaces is considered, and the operation difficulty score is designed to represent the adjustable space size of the vehicle during the parking process, which is represented as:

[0145]

[0146] wherein, N avail is the number of currently available parking spaces; N max is the maximum total number of parking spaces; LineWidth is the main road width of the parking lot; when there is no occupancy around the target parking space, N avail value is 3; when there is no available parking space, N avail value is 1, and the path planning difficulty of the intelligent parking system increases;

[0147] (2) According to the vehicle size and the parking space size, the feasibility of parking success is evaluated, and the parking space size matching score is designed, which is represented as:

[0148]

[0149] wherein, S matchS is the area of the overlap between the size of the parking space and the size of the host vehicle; S main PSSM is the size of the host vehicle; when the parking space fully matches the size of the host vehicle, the PSSM value is 0, indicating the best match, and if there is no match, a value between 0 and 1 will be obtained.

[0150] (3) Finally, the parking operation difficulty index is calculated and expressed as:

[0151] PSCI=A3S+PSSM

[0152] 62) According to the interaction intensity quantification index of step five, the sum of the interaction intensity of each key traffic participant and the parking space difficulty index are calculated as the quantification of the scene criticality, and the score exceeding a certain threshold is the key parking scene where interaction occurs, which is expressed as:

[0153]

[0154] where InteractionScore j is the interaction intensity score of each participant; w1 is the weight coefficient considering static obstacles; w2 is the weight coefficient considering dynamic interaction participants, and satisfies w1+w2=1;

[0155] 63) Key parking scene extraction:

[0156] The Dragon Lake Parking dataset and other parking lot scenes using unmanned aerial orthographic aerial view videos are used for extraction, and the extraction process is as follows: according to the method of step one and step two, the target trajectory sequence is obtained from the parking dataset; through analysis and comparison of the target trajectory sequence, multiple dynamic features are selected to screen the trajectory sequence, and the trajectory sequence is divided into parking task class and non-parking task class; for the parking task class, the spatial interaction range is determined according to the position and speed sequence of the host vehicle, and the neighborhood traffic participant set is selected from the other participant sequence; the distance between the traffic participant and the host vehicle is less than the interaction range, which is regarded as the starting point of the trajectory, and the distance is greater than the interaction range, which is regarded as the end point of the trajectory; according to the interaction intensity of the host vehicle and each key traffic participant calculated in step five, the spatial proximity and maneuverability index are calculated according to the position characteristics, acceleration characteristics and heading angle characteristics of each participant; the parking operation difficulty index is calculated according to step six, wherein the size of the parking space and the width of the main road of the parking lot are known, and finally the scene criticality index is calculated.

[0157] Example two

[0158] This example provides a method for extracting key parking scenes from a complex interaction parking dataset using the method of example one;

[0159] According to the interaction range determined in step four, the extracted trajectory of the traffic participant is as shown in Figure 8

[0160] According to the keyness quantization index of a certain scene calculated in step five and step six, the comparison with TTC is as shown in Figure 9

[0161] According to Figure 9 It can be known that, compared with TTC, the index proposed in the application is relatively smooth in the interaction process, and the index starts to rise after the traffic participant enters the interaction range, and then the index decreases after reaching a certain degree, which is consistent with the characteristics of the interaction between the host vehicle and the participant;

[0162] Compared with TTC, the interaction intensity quantization index of the application comprehensively considers the changes of the host vehicle and the participant in acceleration and heading angle;

[0163] In addition, unlike the high-speed scene, the keyness index of the parking scene needs to pay attention to the interaction intensity quantization index proposed in the application, and also needs to pay attention to the size of the parking space. For this problem, the application designs a corresponding parking operation difficulty index, which together with the interaction intensity quantization index constitutes the scene keyness quantization index.

[0164] Although the embodiments of the application have been shown and described, it can be understood by those of ordinary skill in the art that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirits of the application, and the scope of the application is defined by the appended claims and their equivalents.​​

Claims

1. A method for key parking scene extraction oriented to complex interactive parking dataset, characterized in that, Comprising the following steps: Step one, multi-target trajectory data extraction, using target detection model YOLO and multi-target tracking algorithm DeepSORT to detect and track the moving targets in the parking data set; Step two, according to the detection result, the state of each target is estimated, and the time series trajectory data containing position, speed, heading angle and acceleration are generated; Step three, based on the parking behavior recognition model of multi-feature fusion, the trajectory of the parking behavior is screened from the extracted trajectory sequence; the parking behavior recognition model integrates the key dynamic features of the vehicle at the terminal of the trajectory and the idle parking space position information, and realizes the judgment of the parking behavior; Step four, construct a key traffic participant identification model, according to the interaction range of the host vehicle speed and traffic flow density in the parking process, intercept the trajectory sequence of other traffic participants in the range; Step five, design the interaction intensity quantification index, comprehensively consider the spatial proximity and the maneuvering degree to quantitatively judge whether the interaction between the host vehicle and the traffic participants occurs, and screen the neighborhood traffic participant set; Step six, design the keyness index of parking scene, evaluate the overall keyness of parking scene combining with the interaction intensity and parking space, and finally extract the key parking scene from the whole parking data set according to this; The specific method of step six is as follows: 61) design the parking operation difficulty index, which comprehensively reflects the influence of parking space and parking space size on the difficulty of parking operation, and is represented as: (1) for vertical parking space, considering the occupancy state of left, right and rear parking space, for parallel parking space, considering the occupancy state of front, rear and side parking space, design operation difficulty score to represent the adjustable space size of vehicle in parking process, represented as: Wherein, N avail is the current available parking space number; N max is the maximum total number of parking spaces; LineWidth is the main road width of the parking lot; when the target parking space is surrounded by unoccupied parking spaces, N avail is 3; when there is no available parking space, N avail is 1, and the path planning difficulty of the intelligent parking system increases; (2) according to the size of vehicle and parking space, evaluate the feasibility of parking success, design parking space size matching degree score, represented as: where S match is the area of the overlap of the parking space size and the host vehicle size; S main is the size of the host vehicle; when the parking space fully accommodates the host vehicle size, the PSSM value is 0, indicating the best match, and if not, a value between 0-1 is obtained; (3) finally, calculate the parking operation difficulty index, represented as: PSCI=A3S+PSSM 62) according to the interaction intensity quantification index in step five, calculate the sum of the interaction intensity of each key traffic participant and the parking space difficulty index as the quantification of scene keyness, and the score exceeding a certain threshold is the key parking scene with interaction, represented as: wherein J is a set of neighborhood traffic participants, InteractionScore j is the interaction intensity score for each participant; w1 is a weight coefficient considering static obstacles; w2 is a weight coefficient considering dynamic interaction participants, and satisfies w1+w2=1; 63) key parking scene extraction: The extraction process is as follows: obtaining the target trajectory sequence from the parking data set according to the method of step one and step two; selecting the multi-element dynamic characteristics through the analysis and comparison of the target trajectory sequence, and screening the trajectory sequence into the parking task class and the non-parking task class; for the parking task class, the spatial interaction range is determined according to the position and speed sequence of the host vehicle, and the neighborhood traffic participant set is screened from the other participant sequence; the distance between the traffic participant and the host vehicle is less than the interaction range, and the trajectory starting point is regarded as the trajectory ending point; according to the interaction intensity of the host vehicle and each key traffic participant calculated in step five, the spatial proximity and maneuvering degree indexes are calculated according to the position characteristics, acceleration characteristics and heading angle characteristics of each participant; according to the calculation of the parking operation difficulty index in step six, the parking space size and the main road width of the parking lot are known, and finally the scene key index is calculated.

2. The method of claim 1, wherein, The specific method of step one is as follows: 11) target detection, using YOLO to perform real-time target detection on each frame of image of the parking lot monitoring video; YOLO predicts the bounding box of all targets in the image and the corresponding class probability in a single forward propagation process through the convolutional neural network architecture; the output result contains the center coordinates, width, height and confidence score of each target; 12) target tracking, all targets detected by YOLO in the first frame of the video are taken as initial trajectories, and a unique tracking ID is assigned to each target, and the detection result of the target in the first frame is recorded as the initial state; for each known tracking target, the Kalman filter is used to predict the target state of the next frame through the state of the current frame, and the estimated value is corrected according to the YOLO target detection result of the next frame; DeepSORT uses the target appearance features extracted by CNN and the target motion features calculated by Kalman filter to measure the similarity between the current frame detection result and the historical appearance features or the predicted motion features, and obtains the cost matrix; the Hungarian algorithm is used to process the cost matrix to find the optimal matching between the detection result and the known target trajectory; for the successfully matched detection result, the state of the corresponding target trajectory is updated; for the detection result that is not matched to any known trajectory, a new target trajectory is created and added to the tracking list; for the existing trajectory that fails to match the detection result for 100 consecutive frames, the trajectory is terminated.

3. The method of claim 1, wherein, The specific method of step two is as follows: 21) the bounding box coordinates of the target detected by YOLO are regarded as the spatial position characteristics; 22) the motion state is calculated according to the spatial position characteristics, specifically as follows: (1) velocity estimation, the position difference of the target detection box between two frames is used to calculate the instantaneous velocity of the target combined with the frame rate, which is represented as: wherein the frame rate is 25fps;△t is 0.04s;(x1,y1) is the center coordinate of the target detection box in the previous frame; and(x2,y2) is the center coordinate of the same target detection box in the next frame. (2) Heading angle estimation, the heading angle is defined as the angle between the direction of the target movement and the positive direction of the horizontal axis, the heading angle is estimated by comparing the spatial position relationship of the center position of the target detection frame between consecutive frames, which is represented as: (3) Acceleration calculation, the acceleration is estimated by the second derivative of the position change of the target, that is, the time series of the velocity is differentiated, which is represented as:

4. The method of claim 1, wherein, The specific method of step three is as follows: 31) Establish a parking behavior recognition model based on multi-feature fusion to recognize the parking behavior of the vehicle in the predetermined scene; 32) The parking behavior recognition model realizes the judgment of the parking behavior by integrating the key dynamic features of the vehicle at the trajectory terminal, including the state features at the trajectory terminal of the vehicle, the spatial features related to the position of the idle parking space, and the time features; wherein the trajectory terminal of the vehicle completing the parking behavior has obvious identifiable state features, which are represented as indicators considering the speed, acceleration, and heading angle features, and are specifically as follows: (1) The indicator considering the speed feature is represented as: VScore=1-|Velocity_End| / V_threshold Wherein, V_threshold=3m / s is the speed threshold, Velocity_End is the speed at the trajectory terminal of the vehicle, and the closer to zero, the higher the score; (2) The indicator considering the acceleration feature is represented as: AScore=1-|Acceleration_End| / A_threshold where A_threshold = 0.1 m / s 2 Acceleration_End is the acceleration at the end of the vehicle trajectory, with a higher score for being closer to zero. (3) The indicator considering the heading angle feature is represented as: where I is the set of free parking spots, VehicleHeading_End represents the heading angle at the end of the vehicle trajectory, IdealSpotDirection i represents the ideal pointing angle of the i-th parking spot. (4) When the vehicle has a parking behavior, its final position will be very close to the idle parking space, so by calculating the distance between the trajectory terminal of the vehicle and the center point of the idle parking space, an important basis for judging the parking behavior is obtained; the indicator considering the trajectory terminal position feature is represented as: wherein DistanceToSpot i is the actual distance from the midpoint of the vehicle bounding box to the center point of the i-th parking spot; SpotRadius i is half the width of the target parking spot; (5) The time that the vehicle stays at a certain point or near a parking space is taken as the time feature for recognizing the parking behavior; the indicator considering the stay time feature is represented as: TScore=1-exp(-(TimeStopped / T_threshold)) Wherein, TimeStopped is the time that the vehicle continuously stops in the target parking area; T_threshold is the stay time threshold, and the longer the stay time, the higher the probability of parking behavior; The score distribution of the parking behavior recognition model considering the above indicators is in [0, 5], and considering the existence of non-standard parking drivers, the trajectory with a total indicator score exceeding 4 is considered as a parking trajectory: Score=VScore+AScore+HScore+PScore+TScore.

5. The method of claim 1, wherein, The specific method of step four is as follows: 41) Construct a key traffic participant recognition model; the key traffic participant recognition model includes interaction range determination and interaction feature extraction to determine the neighbor traffic participant set; 42) Through the key traffic participant recognition model, pay attention to the participants with a distance from the host vehicle within the interaction range and possible interaction, and extract and analyze the features of each participant within the limited interaction range; specifically as follows: (1) Interactive range determination, for the host vehicle that occurs parking behavior, with the parking trajectory as the center, the range radius determined by considering the vehicle speed and the basic safety distance is expressed as: InteractiveRadius = GeometricDistance + SafetyBaseDistance + Speed * k1 Wherein, GeometricDistance is the half of the diagonal line in the simplified model of the vehicle body rectangle, which is related to the size of the vehicle body; SafetyBaseDistance is the basic safety distance constant; Speed is the speed of the host vehicle; k1 is the speed influence coefficient, which is related to the flow density in the parking lot, and is set as a constant; (2) Interactive feature extraction, intercept the trajectory data of other traffic participants within the interactive range and the parking time range of the host vehicle, and sort them according to the time sequence, including the position, speed, heading angle and acceleration sequence.

6. The method of claim 5, wherein, The specific method of step five is as follows: 51) Design the interaction intensity quantification index, analyze the trajectory features of the host vehicle parking trajectory and other traffic participants, design the quantification index considering the spatial proximity and maneuver degree of the host vehicle and the participants, convert the extracted participant trajectory features into numerical values, calculate the interaction intensity score between each participant and the host vehicle, and screen out the participants who have a key influence on the parking behavior, which is expressed as: InteractionScore = SpatialProximityScore + ManeuverDegree Wherein, InteractionScore is the interaction intensity score between the host vehicle and any participant; SpatialProximityScore is the spatial proximity; ManeuverDegree is the maneuver degree; 52) The closer the distance between the traffic participant in the interactive range and the host vehicle, the greater the possibility of interaction, so the spatial proximity is defined, which is expressed as: SpatialProximityScore = 1-(Distance / InteractiveRadius) wherein, is the distance between the host vehicle and any participant during the interaction; Considering the acceleration and heading angle change rate of the traffic participant in the interactive range, the maneuver degree is defined, the absolute value of the acceleration and the heading angle change rate of the participant in the interactive range is calculated respectively, which is expressed as: Wherein, in order to unify the dimension, the heading angle change rate threshold Heading_change_threshold is introduced, which is a constant; a(t) is the acceleration of any participant in the interactive range; A_threshold is the acceleration threshold; Heading(t) is the heading angle of any participant in the interactive range at t; Heading(t-△t) is the heading angle of the same participant at t-△t; △t is 0.04s.

Images