Blind zone parking track generation method and device and storage medium

Abstract

The embodiment of the invention discloses a blind area parking track generation method, a blind area parking track generation device, electronic equipment, a chip and a computer readable storage medium, and the method comprises the steps: obtaining parking track data of a target vehicle; the parking track data is structured data of T frames of images before the target vehicle enters a monitoring blind area, and T is a positive integer; based on the parking track data, determining a parking intention set of the target vehicle and coordinates of a parking position of the target vehicle; determining a target parking space number of the target vehicle based on the parking intention set of the target vehicle and the coordinates of the parking position of the target vehicle; generating a parking track based on the T-frame image structured data and the parking space coordinates corresponding to the target parking space number; according to the blind area parking track generation method provided by the embodiment of the invention, parking positioning and parking track generation of the vehicle in the monitoring blind area can be realized.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, and in particular to a blind spot parking trajectory generation method, a blind spot parking trajectory generation device, an electronic device, a chip and a computer-readable storage medium. Background Art

[0002] At present, vehicle identification and tracking relies on surveillance videos. When a vehicle enters a surveillance blind spot, it becomes impossible to track the vehicle’s parking position and parking trajectory within the blind spot. This is a technical problem that needs to be solved urgently. Summary of the invention

[0003] Embodiments of the present application provide a blind spot parking trajectory generation method, a blind spot parking trajectory generation device, an electronic device, a chip, and a computer-readable storage medium.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides a method for generating a blind spot parking trajectory, comprising:

[0006] Acquire parking trajectory data of the target vehicle; the parking trajectory data is T-frame image structured data before the target vehicle enters the monitoring blind spot, where T is a positive integer;

[0007] Based on the parking trajectory data, determining a parking intention set of the target vehicle through a parking intention judgment model and determining the coordinates of the parking position of the target vehicle through a prediction model; the parking intention set includes at least one parking space number;

[0008] Determining a target parking space number of the target vehicle based on the parking intention set of the target vehicle and the coordinates of the parking position of the target vehicle;

[0009] A parking trajectory is generated based on the T-frame image structured data and the parking space coordinates corresponding to the target parking space number.

[0010] In a second aspect, an embodiment of the present application provides a blind spot parking trajectory generation device, comprising:

[0011] Video acquisition module: used to obtain parking trajectory data of the target vehicle; the parking trajectory data is T-frame image structured data before the target vehicle enters the monitoring blind spot, where T is a positive integer;

[0012] Trajectory deduction module: used to determine the parking intention set of the target vehicle through a parking intention judgment model based on the parking trajectory data and to determine the coordinates of the parking position of the target vehicle through a prediction model; the parking intention set includes at least one parking space number; based on the parking intention set of the target vehicle and the coordinates of the parking position of the target vehicle, determine the target parking space number of the target vehicle; based on the T-frame image structured data and the parking space coordinates corresponding to the target parking space number, generate a parking trajectory.

[0013] In a third aspect, the present application provides an electronic device, comprising: a processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory, and executing any one of the blind spot parking trajectory generation methods provided in the embodiments of the present application.

[0014] In a fourth aspect, the present application provides a chip, including: a processor, used to call and run a computer program from a memory, so that a device equipped with the chip executes any blind spot parking trajectory generation method provided in the embodiments of the present application.

[0015] In a fifth aspect, the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program enables a computer to execute any blind spot parking trajectory generation method provided in the embodiments of the present application.

[0016] The blind spot parking trajectory generation method provided in the embodiment of the present application determines the parking intention set of the vehicle and predicts the parking position coordinates of the vehicle through the T-frame data before the vehicle enters the monitoring blind spot, then determines the target parking space coordinates of the vehicle through the parking intention set of the vehicle and the parking position coordinates of the vehicle, and finally obtains the parking trajectory of the vehicle in the blind spot through the T-frame data before the vehicle enters the monitoring blind spot and the target parking space coordinates of the vehicle, thereby realizing the parking positioning and parking trajectory generation of the vehicle in the monitoring blind spot. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of the implementation process of the method for generating a blind spot parking trajectory provided in an embodiment of the present application;

[0018] Figure 2 A schematic diagram of the overlapping relationship of cameras provided in an embodiment of the present application;

[0019] Figure 3 A schematic diagram of candidate trajectories provided in an embodiment of the present application;

[0020] Figure 4 A schematic diagram of the structure of a blind spot parking trajectory generating device provided in an embodiment of the present application;

[0021] Figure 5A schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0022] Figure 6 A schematic structural diagram of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] It should be noted that in the embodiments of the present application, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the embodiments of the present application, the character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0025] In the description of the embodiments of the present application, the term "corresponding" may indicate a direct or indirect correspondence between two items, or an association relationship between the two items, or a relationship between indication and being indicated, configuration and being configured, and the like.

[0026] To facilitate understanding of the technical solutions of the embodiments of the present application, the relevant technologies of the embodiments of the present application are described below. The following related technologies can be arbitrarily combined with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.

[0027] In the related art, the implementation of automatic parking adopts the following method, including: step 1, the parking space image acquisition module acquires the parking space image in the parking lot, and transmits the acquired parking space video image to the parking space image recognition processing module; step 2, the parking space image recognition processing module recognizes and processes the received parking space video image frame by frame, until a completely photographed parking space is recognized in a certain frame of video image, the complete parking space recognized in the frame of video image is determined to be the target parking space, and the image coordinates of the four corner points of the target parking space are transmitted to the parking space tracking module; step 3, the target parking space recognized by the parking space image recognition processing module is tracked and recognized, the target parking space and the coordinate information of the target parking space in the world coordinate system are drawn and transmitted to the parking space display module until the automatic parking process is completed; step 4, the parking space display module displays the coordinate information of the target parking space and the target parking space in the world coordinate system detected and drawn by the parking space tracking module in real time for the driver's reference, thereby realizing the human-computer interaction function.

[0028] This method has the following technical problems: 1. The target vehicle must park in the parking space according to the parking path generated by the planning path module. The premise for the realization of this technology is that the target vehicle must have an automatic driving system. 2. This method requires that there should be no blind spots during the camera shooting process and the target vehicle cannot be out of the video image, otherwise it will be impossible to locate the target parking space.

[0029] Figure 1 A schematic diagram of the implementation flow of the method for generating a blind spot parking trajectory provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the embodiment of the present application provides a method for generating a blind spot parking trajectory, the method comprising the following steps:

[0030] Step 101: Acquire parking trajectory data of a target vehicle; the parking trajectory data is T-frame image structured data before the target vehicle enters a monitoring blind spot, where T is a positive integer.

[0031] In an optional implementation of the present application, the parking trajectory data of the target vehicle is obtained, including: collecting real-time video stream data and camera internal and external parameter information of the monitoring camera, and using target recognition and positioning algorithms, such as CenterNet algorithm, YOLO (You Only Look Once) algorithm, etc. to extract the target object category, position, heading angle, size, timestamp and other information within the camera's perception range as frame image structured data.

[0032] For example, take the CenterNet algorithm as an example. The video stream data is read frame by frame, passed into the network in the form of an image, and the timestamp of the current frame is recorded. The 50-layer residual network (Residual Network-50, ResNet-50) is used to extract image features, the deconvolution module upsamples the feature map, and the convolution network predicts the heat map, size, center point coordinates and offset of the target; the most likely category of the target is obtained based on the heat map information; the center point coordinates are regarded as the position of the target, and the pixel coordinates of the image are converted into world coordinates using external parameters to obtain the position information of the target; the heading angle information is obtained using the offset; the target category, position, heading angle, size, timestamp and other information contained in the current frame are obtained as the frame image structured data of the current frame.

[0033] In an optional implementation of the present application, when the target vehicle appears simultaneously in the data captured by at least two cameras at a specific moment, the data captured by the at least two cameras at the specific moment are fused, and the fused data is used as the frame image structured data of the target vehicle at the specific moment.

[0034] refer to Figure 2 , Figure 2A schematic diagram of the overlapping relationship of cameras provided in the embodiment of the present application is shown in FIG. Figure 2 As shown, there is an overlapping area between cameras c1 and c2, and the differences in category, position, heading angle, and size in the frame image structured data taken by the two cameras at the same time are compared; the Hungarian algorithm is used to regard two targets with the same category and the smallest comprehensive difference in position, heading angle and size that is less than the matching threshold as the same target; when the target is a target vehicle, it represents that the target vehicle appears in the data taken by at least two cameras at the specific moment at the same time, and the position, heading angle, size and other information of the successfully matched target are fused, and the fused result is used to update the position, heading angle, size and other information of the target as the frame image structured data at the current moment.

[0035] In an optional implementation of the present application, an association relationship is constructed based on the overlapping relationship of cameras.

[0036] For example, continue to refer to Figure 2 , camera c1 has overlapping coverage area with camera c2 and camera c6, camera c2 has overlapping coverage area with camera c3, camera c4 has overlapping coverage area with camera c5, according to Figure 2 The overlapping coverage of each camera in the image is used to construct an association relationship T = {c1:c2, c1:c6, c2:c3, c4:c5}, that is, the data of c1 and c2 can be matched and fused, the data of c1 and c6 can be matched and fused, the data of c2 and c3 can be matched and fused, and the data of c4 and c5 can be matched and fused.

[0037] In an optional implementation of the present application, internal and external parameter calibration is performed on cameras within the monitoring area.

[0038] For example, the internal parameters of the surveillance camera in the parking lot are calibrated for vehicle positioning, and the camera's Real Time Streaming Protocol (RTSP) stream is connected; the black and white checkerboard calibration plate is moved so that the calibration plate moves up and down, left and right, and at a certain tilt angle in the camera screen to ensure that the calibration plate is evenly spread over the entire camera screen; the camera internal parameter calibration program of the Robot Operating System (ROS) is run to obtain the camera's internal parameters. When the position accuracy requirement is not high or it is difficult to calibrate the internal parameters with the help of a calibration plate, the camera's internal parameters can be calculated using a formula. The specific steps are: Get the camera's horizontal field of view Fov through the camera's device information w , vertical viewing angle Fov h , resolution c w *c h , c w is the number of pixels in the horizontal direction, c his the number of pixels in the vertical direction, focal length f; according to the formula The horizontal focal length is The vertical focal length is The intrinsic parameter matrix of the camera is

[0039] For example, the external parameters of the surveillance camera in the parking lot are calibrated to obtain the conversion relationship between the camera coordinates and the world coordinates. At least 6 feature points (angular and stable points whose pixel values ​​are easy to obtain) are selected in the camera image, and cones can also be placed manually; the position information of the feature points is collected; the camera image is intercepted to obtain the pixel value corresponding to each feature point; according to the pixel value and position information of each feature point, the camera external parameters are obtained using the perspective-n-point (PnP) algorithm.

[0040] Step 102: Based on the parking trajectory data, determine the parking intention set of the target vehicle through a parking intention judgment model and determine the coordinates of the parking position of the target vehicle through a prediction model; the parking intention set includes at least one parking space number.

[0041] Before determining the parking intention set of the target vehicle by using the parking intention judgment model, the method further includes: training the parking intention judgment model in the following manner:

[0042] Performing cluster analysis on the first historical trajectory data set to obtain a plurality of parking intention sets; wherein any parking intention set of the plurality of parking intention sets includes at least one parking space number, and the parking space numbers in any parking intention set are not repeated; the first historical trajectory data set includes a plurality of historical trajectory data, the historical trajectory data is T-frame image structured data before the vehicle enters the monitoring blind spot, and one historical trajectory data corresponds to one parking intention set;

[0043] splicing the parking intention set corresponding to each historical trajectory data in the plurality of historical trajectory data to obtain a second historical trajectory data set;

[0044] Performing oversampling processing on the second historical trajectory data set to obtain a third historical trajectory data set;

[0045] The parking intention judgment model is trained using the third historical trajectory data set.

[0046] In an optional implementation of the present application, a synthetic minority oversampling technique (SMOTE) method is used to perform oversampling processing on the second historical trajectory dataset.

[0047] A conventional deep learning model is a data fitting model that requires a large amount of training data to achieve the expected model evaluation indicators. However, in real-world scenarios or project implementation, it takes a long time to accumulate training data to achieve the required training data volume required for model training. In response to this actual scenario problem, this application preprocesses the data to solve the problem that the actual scenario training data set is difficult to collect, takes a long time, and has a large volume, thereby reducing resource consumption and reducing model training costs.

[0048] In an optional implementation manner of the present application, before determining the parking intention set of the target vehicle through a parking intention judgment model based on the parking trajectory data and determining the coordinates of the parking position of the target vehicle through a prediction model or performing cluster analysis on the first historical trajectory data set, the method further includes:

[0049] A preprocessing operation is performed on the parking trajectory data or the first historical trajectory data set; the preprocessing operation includes at least one of the following operations:

[0050] Delete the frame image structured data with the number of missing features greater than or equal to M;

[0051] Perform feature completion on the frame image structured data with the number of missing features less than M;

[0052] Deleting frame image structured data in which the difference in vehicle speed between two adjacent frames is greater than a first preset threshold; and,

[0053] Delete the frame image structured data whose vehicle longitude and latitude offset between two adjacent frames is greater than a second preset threshold; wherein M is a positive integer.

[0054] Due to problems such as equipment recognition errors, the original structured data may have problems such as missing data, data duplication, and changes in the longitude and latitude of the target vehicle. The trajectory data is cleaned and feature-screened, and then the vehicle's parking intention is judged to determine the set of parking spaces that the target vehicle may enter in the blind spot.

[0055] Exemplarily, the trajectory data is divided according to the vehicle ID. For a single target vehicle, each frame of data is first arranged in chronological order, and then the frame image structured data that is missing M types of feature information is eliminated. For data with less than M missing feature categories, it is supplemented. The supplementation method can use a variety of supplementation methods such as interpolation supplementation, mean supplementation, and nearest neighbor supplementation. Since the vehicle entering the parking space is a driving process under a low speed state and can be regarded as a uniform motion, the data with large fluctuations in the speed of the previous and next frames are eliminated. Eliminate the data with large jumps in the longitude and latitude of the target vehicle to avoid the impact of camera acquisition errors on subsequent processing. In an optional embodiment of the present application, the value of M is 3.

[0056] Since the target vehicle's driving trajectory and heading angle will be different when it enters different parking spaces, the trajectory of the target vehicle before entering the camera's blind spot will also be different. A similarity trajectory clustering analysis is performed on the preprocessed first historical trajectory dataset to fuzzy determine the parking space that the target vehicle may park in the camera's blind spot.

[0057] For example, assume that there are m cameras c in the parking lot 1 ,c 2 ,...,c m There is a blind spot, and there are n 1 ,n 2 ,...,n m parking spaces, the first historical trajectory dataset after preprocessing is D 1 ,D 2 ,...,D m The similarity trajectory clustering analysis process is explained by taking the blind spot of camera c1 as an example. m The parking space number corresponding to the parking space is n 11 ,n 12 ,...,n 1m . Assume that the historical trajectory dataset D 1 include Each piece of historical trajectory data is serialized data, which is the historical T-frame trajectory of the vehicle before it enters the parking space. Clustering methods such as hierarchical clustering algorithm and K-Means clustering algorithm are used. After the serialized data is clustered, we can get Then, each cluster corresponds to a set of vehicles entering parking spaces Ω τ , The elements in the parking space set are different parking space numbers, that is, right satisfy Among them, the set Ω τ The elements in are different from each other, that is, any And any element satisfy Naturally, according to the clustering results, it can be concluded that in the blind area of ​​camera c1, Parking intention τ ,Any parking intention includes a set of non-repeated parking space numbers, and one historical trajectory corresponds to one parking intention.

[0058] Assume that at time T, camera c1 can capture the target vehicle, and at time T+1, the target vehicle enters the camera's blind spot. 1 The T frame data of a historical trajectory is d i =[X1 ,X 2 ,...,X T ], where X T ={x 1 ,x 2 ,...,x S} is the last frame of data that the camera can capture of the vehicle. One historical trajectory corresponds to one parking intention. Since the last frame of data of the target vehicle before the blind spot can best reflect the driving intention of the vehicle, when training the parking intention judgment model, the present invention selects the historical trajectory T frame data including the last frame of data before the blind spot as the model training data. The parking intention Ω corresponding to the historical trajectory is τ As a soft label, it is spliced ​​after the corresponding last frame of data to form a new training data set That is, the second historical trajectory dataset. Because the target vehicle is driven by humans, the driver will be more inclined to choose a certain parking space to park due to factors such as the location of the parking space and the difficulty of entering the parking space, so the data set with parking intention soft labels This is an unbalanced dataset. During the training process, the model will be generated in the direction of the least expensive classification of the majority class labels, which will eventually lead to poor overall classification performance of the model. This application uses the SMOTE oversampling method to classify the data set to solve this problem. Upsample the minority class data in .

[0059] For example, assume that the dataset There are three types of parking intention soft tags in Ω 1 ,Ω 2 ,Ω 3 , corresponding to σ 1 ,σ 2 ,σ 3 Data σ 3 <σ 2 <σ 1 ,The specific steps of building the parking intention judgment model are as follows:

[0060] (1) Statistical data set The proportion of the number of soft tags in the three categories: Ω 1 :Ω 2 :Ω 3 =σ 1 :σ 2 :σ 3 ;

[0061] (2) Determine the sampling rate: For the label Ω 2 , whose sampling rate is σ 1 / σ 2 , for the label Ω 3 , whose sampling rate is σ 1 / σ 3 ;

[0062] (3) For the pair labeled Ω 2 For each data sample, the distance from it to all samples in the minority class sample set is calculated using the Euclidean distance as the standard, and these samples are recorded as "neighbors"; for the label Ω 1 The same operation is performed on each data sample.

[0063] (4) For the pair labeled Ω 2 For each data sample, the sampling rate σ is determined according to the sampling rate in step (2). 1 / σ 2 For each minority class sample A, randomly select several neighbors from its "neighbors". Assume that the selected neighbor is B. For neighbor B, according to: C = A + rand (0, 1) * |AB|, a new sample C is obtained. Among them, rand (0, 1) means randomly taking a value between 0 and 1. Finally, perform the same operation for other samples similar to B in the selection to complete the sampling. Similarly, for the sample labeled Ω 3 Each data sample is represented by σ 1 / σ 3 The same operation is performed as above for the sampling rate;

[0064] (5) After the above sampling operation, the proportion of intended labels can be obtained as Ω 1 :Ω 2 :Ω 3 = New dataset with a 1:1:1 ratio That is the third historical trajectory data set, and there are 3σ in the data set 1 samples;

[0065] (6) Using a new dataset A classification model is trained, such as a Light Gradient Boosting Machine (LightGBM) model. The classification model obtained by training is the parking intention judgment model corresponding to the camera c1.

[0066] Similarly, for camera c 2 ,...,c m The corresponding trajectory dataset D 2 ,...,D m After the above processing, we get a new data set Correspondingly, m-1 different parking intention judgment models are generated.

[0067] In an optional implementation manner of the present application, before determining the coordinates of the parking position of the target vehicle by using the prediction model, the method further includes:

[0068] The prediction model is trained by the first historical trajectory data set; the loss function of the prediction model is expressed as: Loss = L gp (P g ,P p )+γ, where represents the actual end point position P g and the predicted end position P p The distance between g is the actual end point position, P p is the predicted endpoint position, and γ is the data distribution loss error obtained by processing multiple historical trajectory data collected by multiple cameras through KL divergence.

[0069] In the related art, the prediction model requires that the training data set and the test data set satisfy the independent and identical distribution. However, due to reasons such as lighting, background, shooting angle, image quality, etc., the data obtained in the project deployment environment (test set) and the data used by the training algorithm (training set) often do not satisfy the independent and identical distribution conditions, which leads to a sharp decline in the performance of the algorithm in the project environment application. Therefore, the applicant added the data distribution loss error obtained by processing multiple historical trajectory data collected by multiple cameras through KL divergence to the loss function of the prediction model.

[0070] For example, assuming that different shooting angles of cameras c 1 ,c 2 ,...,c m The collected target vehicle trajectory dataset is D 1 ,D 2 ,...,D m For any two data sets D K , D L ,set up and Denote the feature set D K and D L The jth data of D K , D L The covariance matrix of K , C L , and n K ,n L Respectively represent the number of samples in the two data sets. The calculation formula of the covariance matrix is ​​as follows: Then, the distribution difference loss calculation formula for the two scenarios is: represents the L2 regular term, d is the data distribution distance between the two scene data sets, and the data distribution loss for i scenes is calculated as: Among them, K and L are any two different data sets in the i scene sets.

[0071] When training the prediction model, the distribution loss term is included in the model loss function. In this way, the model can pay attention to the data distribution differences during the training process and iterate in the direction of decreasing the distribution differences.

[0072] It should be noted that in the underground parking lot scenario, the data distribution difference is mainly caused by the different camera installation positions. Therefore, the present invention uses the data distribution difference caused by the camera viewing angle difference as an example to illustrate how to solve the existing problem. It should be pointed out that the method proposed in the present invention is still applicable to data distribution problems caused by illumination differences, background differences, image quality differences, etc. in other scenarios.

[0073] In an optional implementation of the present application, the prediction model is composed of a history trajectory encoding unit and an endpoint position decoding unit. In the encoding unit, several gated recurrent units (GRU) are used as encoding units. The hidden state vector output by a single encoding unit at any time t can be expressed by the following formula: t =GRU(h t-1 ,D t ), where ht- 1 is the latent vector generated by the previous GRU encoding unit, D t is the feature vector at time t.

[0074] In the decoding unit, the MLP network is used as the decoding unit to output the end position, that is, the last trajectory point when the vehicle stops, the parking position of the vehicle. The loss function can be expressed by the following formula: loss = L gp (P g ,P p )+γ, where P g is the actual end point position, P p is the predicted endpoint position, and γ is the data distribution loss error.

[0075] Step 103: Determine the target parking space number of the target vehicle based on the parking intention set of the target vehicle and the coordinates of the parking position of the target vehicle.

[0076] The distances between the coordinates of each parking space in the parking intention set of the target vehicle and the coordinates of the parking position of the target vehicle are calculated respectively, and the parking space closest to the coordinates of the parking position of the target vehicle is the target parking space of the target vehicle.

[0077] For example, the process of determining the target parking space number is described by taking camera c1 as an example. Query the longitude and latitude position of the parking space in the parking intention set of the target vehicle. Where n represents the parking space number, and the parking position P of the target vehicle is calculated respectively. p and parking space location The Euclidean distance between them is obtained by Find the Euclidean distance set E dis The minimum value in and the parking space number corresponding to this value are recorded as n final , then n final The parking space is the target parking space for the target vehicle.

[0078] Step 104: Generate a parking trajectory based on the T-frame image structured data and the parking space coordinates corresponding to the target parking space number.

[0079] In an optional implementation manner of the present application, generating a parking trajectory based on the T-frame image structured data and the parking space coordinates corresponding to the target parking space number includes:

[0080] A grid point set is constructed with the parking space coordinates corresponding to the target parking space number as the center; the grid point set includes N*N grid points, where N is an integer greater than or equal to 2;

[0081] Based on the coordinates of the target vehicle in the last frame of structured image data of the target vehicle entering the monitoring blind spot and the N*N grid points, a trajectory curve is generated by a polynomial to generate N*N candidate trajectories;

[0082] Based on the T-frame image structured data before the target vehicle enters the monitoring blind spot, trajectory deduction is performed through a motion model to generate a kinematic trajectory;

[0083] The similarities between the N*N candidate trajectories and the kinematic trajectory are compared, and the candidate trajectory with the highest similarity is determined as the parking trajectory.

[0084] Exemplary, reference Figure 3 , Figure 3 The candidate trajectory diagram provided in the embodiment of the present application is based on parking space n final Location point Construct a set of N×N grid points P with uniform spacing for the center grid , the elements in the set are longitude and latitude points, and the N×N grid points are selected as the end points of the entry trajectory and meet the following conditions: in, for The longitude coordinates of for The latitude coordinates of , interval indicates the spacing of grid points: n width For parking space n finalThe width of β is an integer in the interval [-N, N]. The latitude and longitude position of the target vehicle in the Tth frame, i.e. the last frame of structured image data before entering the blind spot, is taken as the starting point, and the N×N grid points are taken as the end points. The trajectory curve is generated based on the polynomial, which is: y=a 0 +a 1 x+a 2 x 2 +…+a n x n , where x represents the longitude coordinate, y represents the latitude coordinate, and a0…an are polynomial coefficients; finally, N×N candidate trajectories are generated.

[0085] For example, assuming that the vehicle's driving in the parking lot conforms to the constant acceleration and speed motion model, the position information of the last frame is x n =[lon n ,lat n ,θ n ,v n ,a,ω], where lon n is the longitude coordinate of the vehicle in the last frame, lat n is the latitude coordinate of the vehicle in the last frame. The speed ω is set according to historical experience and can be adjusted according to the actual test situation. In this example, ω is set to 1. The speed v at the last frame time point is obtained based on the position change information and time difference Δt of the last three consecutive frames of data. n , speed heading angle θ n 、Acceleration a: Assuming the data transmission frame rate is 10fps / s, the position at the next time point, that is, after t=0.1s, is x n+1 =[lon n ,lat n ,θ n ,v n ,a,ω]+[g lon ,g lat ,ωt,at,0,0],

[0086] in,

[0087] g lon Indicates the change in position in the longitude direction, g latIndicates the position change in the latitude direction. The next frame of data is updated according to the speed and speed heading angle information of the latest frame, and a total of 50 frames are updated (the deduction time is 5s) to obtain the kinematic trajectory. For example, if the positions of the last three consecutive frames of data are (121.6132126, 31.2505352), (121.6132105, 31.2505345), (121.6132088, 31.2505341), then the data in Table 1 can be obtained:

[0088] longitude latitude θ ν α ω <![CDATA[x n ]]> 121.6132088 31.2505341 254.61 1.91 -4.67 1 <![CDATA[x n+1 ]]> 121.6132119 31.2505330 256.44 1.44 -4.67 1 <![CDATA[x n+2 ]]> 121.613215 31.2505318 258.26 0.97 -4.67 1 … … … … … … …

[0089] Table 1

[0090] For example, assume that the candidate trajectory set Calculate each trajectory q and the kinematic trajectory q * The Fréchet distance of 1 With q * For example, from q 1 Take out 50 trajectory points and calculate q 1 With q * The corresponding point q 1-k ,q *-k The largest of these is the Fréchet distance:

[0091]

[0092] …

[0093]

[0094] D F (q 1 ,q * )=max(d 1 ,...,d 50 ), then the trajectory q 1 With q * The curve similarity is Repeat the calculation to get q * The curve similarity with all trajectories in the candidate trajectory set Q The trajectory corresponding to the maximum value of the similarity set I is the parking trajectory of the vehicle in the monitoring blind spot.

[0095] In an optional implementation manner of the present application, the present invention further includes: obtaining complete trajectory data of the target vehicle; the complete trajectory data is trajectory data from the time when the target vehicle enters the monitoring area to before entering the monitoring blind area;

[0096] A global trajectory of the vehicle is generated based on the complete trajectory data and the parking trajectory.

[0097] In actual application, the complete trajectory data of the vehicle from entering the monitoring area to entering the monitoring blind spot is obtained, and then the generated parking trajectory is spliced ​​with the complete trajectory data of the vehicle to obtain the entire trajectory of the vehicle in the monitoring area and in the blind spot, that is, the global trajectory of the vehicle.

[0098] The blind spot parking trajectory generation method provided in the embodiment of the present application constructs a parking intention judgment model, and innovatively uses data feature engineering, clustering methods, classification models and other means to analyze and extract the target vehicle's intention set to enter the parking space; considering the data distribution error problem caused by the camera shooting angle, shooting environment differences and other issues, the distribution error measurement function is innovatively added to the trajectory prediction model loss function, so that the prediction model has better robustness and accuracy, and realizes the accurate positioning of the parking space; the kinematic model and the prediction model are combined to deduce the vehicle trajectory, which increases the interpretability of the trajectory; the existing camera is used without the need to adjust the posture and angle of the shooting camera in real time, so that the vehicle can be tracked in the entire domain, and the parking trajectory can be traced back, reducing the deployment cost of the equipment.

[0099] The present application embodiment also provides a blind spot parking trajectory generating device 400, referring to Figure 4 The blind spot parking trajectory generating device 400 in this embodiment includes:

[0100] The video acquisition module 410 is used to obtain the parking trajectory data of the target vehicle; the parking trajectory data is T-frame image structured data before the target vehicle enters the monitoring blind spot, where T is a positive integer;

[0101] Trajectory deduction module 420: used to determine the parking intention set of the target vehicle based on the parking trajectory data through a parking intention judgment model and to determine the coordinates of the parking position of the target vehicle through a prediction model; the parking intention set includes at least one parking space number; based on the parking intention set of the target vehicle and the coordinates of the parking position of the target vehicle, determine the target parking space number of the target vehicle; based on the T-frame image structured data and the parking space coordinates corresponding to the target parking space number, generate a parking trajectory.

[0102] In an embodiment of the present application, before determining the parking intention set of the target vehicle through the parking intention judgment model, the trajectory deduction module 420 is also used to train the parking intention judgment model in the following manner: cluster analysis is performed on the first historical trajectory data set to obtain multiple parking intention sets; wherein, any parking intention set of the multiple parking intention sets includes at least one parking space number, and the parking space numbers in any parking intention set are not repeated; the first historical trajectory data set includes multiple historical trajectory data, and the historical trajectory data is T-frame image structured data before the vehicle enters the monitoring blind spot, and one historical trajectory data corresponds to one parking intention set; each historical trajectory data of the multiple historical trajectory data is spliced ​​with its corresponding parking intention set to obtain a second historical trajectory data set; the second historical trajectory data set is oversampled to obtain a third historical trajectory data set; and the parking intention judgment model is trained through the third historical trajectory data set.

[0103] In the embodiment of the present application, before determining the coordinates of the parking position of the target vehicle through the prediction model, the trajectory deduction module 420 is also used to train the prediction model through the first historical trajectory data set; the loss function of the prediction model is expressed as: Loss = L gp (P g ,P p )+γ, where represents the actual end point position P g and the predicted end position P p The distance between g is the actual end point position, P p is the predicted endpoint position, and γ is the data distribution loss error obtained by processing multiple historical trajectory data collected by multiple cameras through KL divergence.

[0104] In an embodiment of the present application, the trajectory deduction module 420 is specifically used to construct a grid point set with the parking space coordinates corresponding to the target parking space number as the center; the grid point set includes N*N grid points, where N is an integer greater than or equal to 2; based on the coordinates of the target vehicle in the last frame of image structured data when the target vehicle enters the monitoring blind spot and the N*N grid points, a trajectory curve is generated through a polynomial to generate N*N candidate trajectories; based on the T-frame image structured data before the target vehicle enters the monitoring blind spot, a trajectory is deduced through a motion model to generate a kinematic trajectory; the similarity between the N*N candidate trajectories and the kinematic trajectory is compared, and the candidate trajectory with the highest similarity is determined as the parking trajectory.

[0105] In an embodiment of the present application, before determining the parking intention set of the target vehicle through a parking intention judgment model based on the parking trajectory data and determining the coordinates of the parking position of the target vehicle through a prediction model or performing cluster analysis on the first historical trajectory data set, the trajectory deduction module 420: is also used to perform a preprocessing operation on the parking trajectory data or the first historical trajectory data set; the preprocessing operation includes at least one of the following operations: deleting frame image structured data with a number of missing features greater than or equal to M; performing feature completion on frame image structured data with a number of missing features less than M; deleting frame image structured data with a speed difference between two adjacent frames greater than a first preset threshold; and deleting frame image structured data with a vehicle latitude and longitude offset greater than a second preset threshold; wherein M is a positive integer.

[0106] In an embodiment of the present application, the video acquisition module 410 is also used to obtain complete trajectory data of the target vehicle; the complete trajectory data is the trajectory data from the time when the target vehicle enters the monitoring area to before entering the monitoring blind spot; the trajectory deduction module 420 generates the global trajectory of the vehicle based on the complete trajectory data and the parking trajectory.

[0107] Those skilled in the art should understand that Figure 4 The functions of each unit in the blind spot parking trajectory generating device 400 can be understood by referring to the related description of the aforementioned method. Figure 4 The functions of the various units in the blind spot parking trajectory generating device 400 shown may be implemented by a program running on a processor, or may be implemented by a specific logic circuit.

[0108] Figure 5 It is a schematic structural diagram of an electronic device 500 provided in an embodiment of the present application. Figure 5 The electronic device 500 shown includes a processor 510, which can call and run a computer program from a memory to implement the method in the embodiment of the present application.

[0109] Alternatively, if Figure 5 As shown, the electronic device 500 may further include a memory 520. The processor 510 may call and run a computer program from the memory 520 to implement the method in the embodiment of the present application.

[0110] The memory 520 may be a separate device independent of the processor 510 , or may be integrated into the processor 510 .

[0111] Alternatively, if Figure 5As shown, the electronic device 500 may further include a transceiver 530, and the processor 510 may control the transceiver 530 to communicate with other devices, specifically, may send information or data to other devices, or receive information or data sent by other devices.

[0112] The transceiver 530 may include a transmitter and a receiver. The transceiver 530 may further include an antenna, and the number of the antennas may be one or more.

[0113] The electronic device 500 may specifically be a blind spot parking trajectory generating device according to an embodiment of the present application, and the electronic device 500 may implement the corresponding processes implemented by the blind spot parking trajectory generating device in each method according to an embodiment of the present application, which will not be described in detail for the sake of brevity.

[0114] Figure 6 It is a schematic structural diagram of the chip of an embodiment of the present application. Figure 6 The chip 600 shown includes a processor 610, which can call and run a computer program from a memory to implement the method in the embodiment of the present application.

[0115] Alternatively, if Figure 6 As shown, the chip 600 may further include a memory 620. The processor 610 may call and run a computer program from the memory 620 to implement the method in the embodiment of the present application.

[0116] The memory 620 may be a separate device independent of the processor 610 , or may be integrated into the processor 610 .

[0117] Optionally, the chip 600 may further include an input interface 630. The processor 610 may control the input interface 630 to communicate with other devices or chips, and specifically, may obtain information or data sent by other devices or chips.

[0118] Optionally, the chip 600 may further include an output interface 640. The processor 610 may control the output interface 640 to communicate with other devices or chips, and specifically, may output information or data to other devices or chips.

[0119] The chip can be applied to the blind spot parking trajectory generating device in the embodiment of the present application, and the chip can implement the corresponding processes implemented by the blind spot parking trajectory generating device in each method of the embodiment of the present application, which will not be described in detail here for the sake of brevity.

[0120] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0121] It should be understood that the processor of the embodiment of the present application may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor are combined and performed. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, and other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0122] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0123] It should be understood that the above-mentioned memory is exemplary but not restrictive. For example, the memory in the embodiments of the present application may also be static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synch link DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM), etc. That is to say, the memory in the embodiments of the present application is intended to include but not limited to these and any other suitable types of memory.

[0124] The embodiment of the present application also provides a computer-readable storage medium for storing a computer program. The computer-readable storage medium can be applied to the blind spot parking trajectory generation device in the embodiment of the present application, and the computer program enables the computer to execute the corresponding processes implemented by the blind spot parking trajectory generation device in each method of the embodiment of the present application, which will not be described in detail here for the sake of brevity.

[0125] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0126] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0127] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0128] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0129] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0130] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a blind spot parking trajectory generation device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0131] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for generating a blind - spot parking trajectory, characterized in that, it includes: Obtain the parking trajectory data of the target vehicle; The parking trajectory data is the structured data of T - frame images before the target vehicle enters the monitoring blind spot, where T is a positive integer; Based on the parking trajectory data, determine the parking intention set of the target vehicle through a parking intention judgment model and determine the coordinates of the parking position of the target vehicle through a prediction model; the parking intention set includes at least one parking space number; Based on the parking intention set of the target vehicle and the coordinates of the parking position of the target vehicle, determine the target parking space number of the target vehicle; Generate a parking trajectory based on the structured data of the T - frame images and the parking space coordinates corresponding to the target parking space number.

2. The blind - spot parking trajectory generation method according to claim 1, characterized in that, Before determining the parking intention set of the target vehicle through the parking intention judgment model, it further includes: training the parking intention judgment model in the following way: Perform clustering analysis on the first historical trajectory data set to obtain multiple parking intention sets; wherein, any one of the multiple parking intention sets includes at least one parking space number, and the parking space numbers in any one of the parking intention sets are not repeated; the first historical trajectory data set includes multiple historical trajectory data, the historical trajectory data is the structured data of T - frame images before the vehicle enters the monitoring blind spot, and one historical trajectory data corresponds to one parking intention set; Concatenate each historical trajectory data in the multiple historical trajectory data with its corresponding parking intention set to obtain a second historical trajectory data set; Perform oversampling processing on the second historical trajectory data set to obtain a third historical trajectory data set; Train the parking intention judgment model through the third historical trajectory data set.

3. The blind - spot parking trajectory generation method according to claim 1, characterized in that, Before determining the coordinates of the parking position of the target vehicle through the prediction model, it further includes: Train the prediction model through the first historical trajectory dataset; the loss function of the prediction model is expressed as: Loss = L gp (P g , P p ) + γ, where P g is the actual end position, P p is the predicted end position, L gp represents the distance between the actual end position P g and the predicted end position P p , and γ is the data distribution loss error obtained by processing multiple historical trajectory data collected by multiple cameras through KL divergence.

4. The blind - spot parking trajectory generation method according to claim 1, characterized in that, The generating a parking trajectory based on the structured data of the T - frame images and the parking space coordinates corresponding to the target parking space number includes: Construct a grid point set with the parking space coordinates corresponding to the target parking space number as the center; the grid point set includes N * N grid points, where N is an integer greater than or equal to 2; Based on the coordinates of the target vehicle in the structured data of the last frame of the target vehicle entering the monitoring blind spot and the N * N grid points, generate trajectory curves through polynomials to generate N * N candidate trajectories; Based on the structured data of the T - frame images before the target vehicle enters the monitoring blind spot, perform trajectory deduction through a motion model to generate a kinematic trajectory; Compare the similarity between the N * N candidate trajectories and the kinematic trajectory, and determine the candidate trajectory with the highest similarity as the parking trajectory.

5. The blind - spot parking trajectory generation method according to claim 1 or 2, characterized in that, Before determining the parking intention set of the target vehicle through the parking intention judgment model and determining the coordinates of the parking position of the target vehicle through the prediction model based on the parking trajectory data or before performing clustering analysis on the first historical trajectory data set, it further includes: Performing a preprocessing operation on the parking trajectory data or the first historical trajectory data set; the preprocessing operation includes at least one of the following operations: Deleting frame image structured data with the number of missing features greater than or equal to M; Completing the features of frame image structured data with the number of missing features less than M; Deleting frame image structured data with the speed difference between two adjacent frames greater than a first preset threshold; and, Deleting frame image structured data with the vehicle longitude and latitude offset greater than a second preset threshold; where M is a positive integer.

6. The method for generating a blind area parking trajectory according to any one of claims 1-4, characterized in that, it further includes: Obtaining the complete trajectory data of the target vehicle; The complete trajectory data is the trajectory data from when the target vehicle enters the monitoring area to before entering the monitoring blind area; Generating the global trajectory of the vehicle based on the complete trajectory data and the parking trajectory.

7. The method for generating a blind area parking trajectory according to any one of claims 1-4, characterized in that, When the target vehicle appears in the data captured by at least two cameras at a specific moment, fusing the data captured by the at least two cameras at the specific moment, and using the fused data as the frame image structured data of the target vehicle at the specific moment.

8. A blind area parking trajectory generation device, characterized in that, it includes: A video acquisition module: used to obtain the parking trajectory data of the target vehicle; The parking trajectory data is the T-frame image structured data before the target vehicle enters the monitoring blind area, and T is a positive integer; A trajectory deduction module: used to determine the parking intention set of the target vehicle through a parking intention judgment model and determine the coordinates of the parking position of the target vehicle through a prediction model based on the parking trajectory data; the parking intention set includes at least one parking space number; determining the target parking space number of the target vehicle based on the parking intention set of the target vehicle and the coordinates of the parking position of the target vehicle; Generating a parking trajectory based on the T-frame image structured data and the parking space coordinates corresponding to the target parking space number.

9. An electronic device, characterized in that, it includes: A processor and a memory, the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method for generating a blind area parking trajectory according to any one of claims 1-7.

10. A chip, characterized in that, it includes: A processor, used to call and run a computer program from a memory, so that a device installed with the chip executes the method for generating a blind area parking trajectory according to any one of claims 1-7.

11. A computer-readable storage medium, characterized in that, For storing a computer program which causes a computer to execute the blind spot parking trajectory generation method according to any one of claims 1-7.

Images