An IOU matching method, device and equipment, a storage medium and a parking space tracking method based on the IOU matching method

Abstract

The present application relates to a kind of IOU matching method, device, equipment, storage medium and parking stall tracking method based on IOU matching method.The IOU matching method includes S1, the parking stall set O obtained based on current frame detection result and the IOU matrix of the prediction trajectory set P of current frame is established with prediction trajectory set P of parking stall set O;S2, matching judgment, obtain the set of IOU value that meets optimal matching in IOU matrix, the parking stall and prediction trajectory corresponding to this set of IOU value are regarded as matching result output.The present application proposes a kind of IOU matching method, device, equipment, storage medium and parking stall tracking method based on IOU matching method, can improve the accuracy and stability of parking stall tracking result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle automatic driving, and in particular to an IOU matching method, device and equipment, a storage medium and a parking space tracking method based on the IOU matching method. BACKGROUND

[0002] With the rapid development of society, private cars are becoming more and more in big cities, which makes it difficult to find free parking spaces. A study shows that more than 50% of drivers cannot find free parking spaces in traffic-intensive areas. In addition, 23% of all car accidents occur in parking lots.

[0003] In this case, the park assist system (PAS) emerges as the times require, and the parking space needs to be tracked in the process of parking, so as to ensure that the car is parked in the reserved parking space.

[0004] Target tracking is usually divided into five main research contents: (1) motion model: how to model and estimate the motion trajectory of the target in the video. (2) feature extraction: what kind of feature represents the target. (3) observation model: how to score a large number of candidate samples. (4) model updating: how to update the observation model to adapt to the changes of the target. (5) integration method: how to integrate multiple decisions to obtain a better decision result.

[0005] Among them, the motion model: models and estimates the motion trajectory of the target in the video. There are two common methods: particle filtering, Kalman filtering and sliding window. The idea of particle filtering is based on the Monte Carlo method, which uses a particle set to represent the probability, which can be used on any form of state space model. Its core idea is to express its distribution by randomly sampling particles from the posterior probability, which is a sequential importance sampling method. It infers the hidden state of the target through recursion. And the sliding window is an exhaustive search method, which lists all possible samples near the target as candidate samples.

[0006] Feature extraction: discriminative feature representation is one of the keys to target tracking. Commonly used features are divided into two types: hand-designed features and deep features. Commonly used hand-designed features include gray features, direction gradient histogram, Haar features, scale invariant features, etc. Unlike human-designed features, deep features are learned features through a large number of training samples, which are more discriminative than hand-designed features. Therefore, tracking methods using deep features usually easily obtain a good effect.

[0007] Observation model: According to different ideas, the observation model can be divided into two categories: generative model and discriminative model. The generative model usually finds the most similar candidate to the target template as the tracking result, which can be regarded as template matching. Common theoretical methods include: subspace, sparse representation, dictionary learning, etc. The discriminative model distinguishes the target from the background by training a classifier, and selects the candidate sample with the highest confidence as the prediction result. Discriminative methods have become the mainstream method in target tracking, because a large number of machine learning methods can be used. Common theoretical methods include: logistic regression, ridge regression, support vector machine, multiple instance learning, correlation filtering, etc.

[0008] Model update: Model update is mainly to update the observation model to adapt to the changes of the target appearance and prevent the tracking process from drifting. There is no uniform standard for model update. It is generally believed that the appearance of the target changes continuously, so the model is often updated once every frame.

[0009] Integrated method: Integrated method is beneficial to improve the prediction accuracy of the model, and is often regarded as an effective means to improve the tracking accuracy. Integrated method can be roughly divided into two categories: selecting the best one from multiple prediction results, or using all predictions for weighted average.

[0010] Multi-target tracking algorithm can be divided into DBT (Detection-Based tracking) and DFT (Detection-Free tracking) according to whether it is based on detector to initialize the target. DFT is a common initialization method in single target tracking field, that is, whenever a new target appears, the algorithm is told the position of the new target. The advantage of this method is target free, but the disadvantage is that the calculation process is more complicated and there is too much interaction. The DBT method first detects the target, and then associates the target into the tracking trajectory.

[0011] Sort and Deepsort are the most representative DBT target tracking frameworks at present, both of which use uniform speed Kalman filter (KF) as the motion model. The biggest difference between the two is the data association part. Deepsort uses a re-id network for feature extraction and Mahalanobis distance to judge the similarity between features, but it is very difficult to train a re-id network. Sort directly calculates the IOU to associate data, which is fast and effective.

[0012] If the parking space is tracked, the DBT tracking method is the first choice, the re-identification network extracts the feature, which increases the cost of the parking space, and the network is not easy to achieve better results, so the method of geometric data association, that is, the IOU method, should also be used for data association, which can greatly reduce the complexity and tracking cost of the tracking model, but the current DBT tracking framework mostly uses the parameters of the target rectangular frame or the picture features corresponding to the rectangular frame as the input, and the output of the tracking framework is also related to the parameters of the rectangular frame, because of the perspective problem of the surround view, the parking space will change more or less into a general parallelogram, and the traditional rectangular frame tracking framework is no longer applicable, and cannot complete the tracking, and cannot be calculated in data association, and the existing technology needs to be improved. SUMMARY

[0013] In view of the above problems of the prior art, the present application provides an IOU matching method, device, equipment, storage medium and parking space tracking method based on the IOU matching method, which can improve the accuracy and stability of the parking space tracking result.

[0014] Specifically, the present application provides an IOU matching method, comprising the steps of:

[0015] S1, establishing an IOU matrix of a parking space set O and a predicted trajectory set P based on the parking space set O obtained from the current frame detection result and the predicted trajectory set P of the current frame, comprising the following steps:

[0016] S11, finding the intersection of the position overlapping area of each parking space OI in the parking space set O and each predicted trajectory Pn in the predicted trajectory set P, wherein the parking space OI is in the parking space set O, and the predicted trajectory Pn is in the predicted trajectory set P;

[0017] The position overlapping area is a polygon, and the area of the position overlapping area, the area of the parking space OI corresponding to the position overlapping area and the area of the predicted trajectory Pn are calculated according to the intersection of the polygon, and the IOU value is calculated according to the area of the position overlapping area, the area of the parking space OI and the area of the predicted trajectory Pn;

[0018] S12, establishing the IOU matrix based on the corresponding relationship between the IOU value and the parking space OI and the predicted trajectory Pn;

[0019] S2, matching judgment, obtaining a group of IOU values that meet the optimal matching in the IOU matrix, and the parking space and the predicted trajectory corresponding to the group of IOU values are taken as the matching result and output.

[0020] According to one embodiment of the present application, in step S11, the area of the polygon represented by the position overlapping region is area1, the area of the polygon represented by the parking space Oi corresponding to the position overlapping region is area2, and the area of the polygon represented by the predicted trajectory Pn is area3.

[0021] IOU = area1 / (area2+area3-area1).

[0022] According to one embodiment of the present application, each parking space Oi in the parking space set O includes parking space position information, which includes four vertex information of the parking space, or two vertex information of the parking space and direction information of the parking space; each predicted trajectory Pn in the predicted trajectory set P includes predicted position information, which includes four vertex information of the predicted trajectory, or two vertex information of the predicted trajectory and direction information of the predicted trajectory.

[0023] In step S11, four vertex information of the position overlapping region of the parking space Oi and the predicted trajectory Pn is obtained based on the parking space position information and the predicted position information; the area of the position overlapping region is calculated based on the four vertex information of the position overlapping region, the area of the parking space Oi is calculated based on the parking space position information, and the area of the predicted trajectory Pn is calculated based on the predicted position information.

[0024] According to one embodiment of the present application, the polygon represented by the position overlapping region is a parallelogram.

[0025] The present application also provides a parking space tracking method, including the steps of:

[0026] T1, obtaining a parking space set O based on the detection result of the current frame, and performing parking space prediction on the current frame based on the updated trajectory set T of the previous frame to obtain a predicted trajectory set P of the current frame;

[0027] T2, performing the IOU matching method according to claim 1 to obtain the matching result between the parking space Oi and the predicted trajectory Pn;

[0028] T3, updating the trajectory set T, updating the trajectory Tm corresponding to the predicted trajectory Pn according to the parking space Oi in the matching result respectively, obtaining the updated trajectory set P and returning to step T1, wherein Tm belongs to the trajectory set T, and the updating of the trajectory Tm includes the updating of the actual position information TL representing the trajectory Tm in the current frame, the updating of the occupancy state information TC representing the occupancy state of the trajectory, the updating of the motion model, and the updating of the life cycle representing the validity of the trajectory.

[0029] According to one embodiment of the present application, the parking space Oi comprises parking space position information OL obtained based on actual detection of a current frame and occupancy state information OC used to indicate occupancy state of the parking space, and the predicted trajectory Pn comprises predicted position information PL and occupancy state information PC used to indicate occupancy state of the predicted trajectory Pn;

[0030] The actual position information TL is determined based on the parking space position information OL and the predicted position information PL.

[0031] The occupancy state information TC of the trajectory Tm corresponding to the predicted trajectory Pn is updated based on the occupancy state information OC of the parking space Oi.

[0032] The motion model of the trajectory Tm corresponding to the predicted trajectory Pi is updated based on the parking space position information OL.

[0033] For the trajectory Ti with a matching result, the life cycle of the trajectory Tm is initialized.

[0034] According to one embodiment of the present application, if a certain parking space Oi does not match all predicted trajectories Pn, the parking space Oi is initialized to generate a new trajectory Tm.

[0035] If a certain predicted trajectory Pn does not match all parking spaces Oi, the trajectory Tm corresponding to the predicted trajectory Pn is retained or deleted according to the life cycle of the trajectory Tm.

[0036] According to one embodiment of the present application, in step T1, the parking space is predicted based on the trajectory set P updated in the previous frame for the current frame, and the predicted trajectory set P of the current frame is obtained, comprising the following steps:

[0037] For each trajectory Tm in the trajectory set T, the possible position of the trajectory Ti in the current frame is predicted based on the motion model of the trajectory Tm, and the obtained prediction result is used as the predicted position information PL of the predicted trajectory Pi. The occupancy state information TC of the trajectory Ti is used as the occupancy state information PC of the corresponding predicted trajectory.

[0038] The present application also provides an IOU matching device for implementing the above-mentioned IOU matching method, comprising:

[0039] An acquisition unit is configured to acquire the parking space set O and the predicted trajectory set P.

[0040] The computing unit calculates the intersection of the polygons represented by the position overlapping area, calculates the area of the position overlapping area based on the intersection of the polygons, calculates the IOU value according to the area of the position overlapping area, the area of the parking space Oi and the area of the predicted trajectory Pn.

[0041] The matrix establishing unit establishes the IOU matrix based on the IOU value and the corresponding relationship between the parking space Oi and the predicted trajectory Pn.

[0042] The judging unit obtains a group of IOU values that meet the optimal matching in the IOU matrix, and outputs the parking space and the predicted trajectory corresponding to the group of IOU values as the matching result.

[0043] The present application also provides an IOU matching device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the IOU matching method of any one of the preceding embodiments when executing the computer program.

[0044] The present application also provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the IOU matching method of any one of the preceding embodiments when executed by a processor.

[0045] The present application provides an IOU matching method, device, equipment, storage medium and parking space tracking method based on the IOU matching method, which tracks the polygons represented by the parking spaces, establishes the IOU matrix, and can improve the accuracy and stability of the parking space tracking result.

[0046] It should be understood that the above general description and the following detailed description of the present application are exemplary and illustrative, and are intended to provide further explanation of the present application as claimed. BRIEF DESCRIPTION OF DRAWINGS

[0047] The accompanying drawings are included to provide further explanation of the present application, which are incorporated and constitute a part of the present application, and show embodiments of the present application, and together with the present specification, serve to explain the principles of the present application. In the drawings:

[0048] Figure 1 A flowchart of the IOU matching method of one embodiment of the present application is shown.

[0049] Figure 2 A schematic diagram of the polygons represented by the position overlapping area of the parking space and the predicted trajectory of one embodiment of the present application is shown.

[0050] Figure 3 A flowchart of the parking space tracking method of one embodiment of the present application is shown.

[0051] Figure 4 Fig. 1 shows a structural schematic diagram of an IOU matching device according to an embodiment of the present application. DETAILED DESCRIPTION

[0052] It should be noted that the embodiments and features of the embodiments in the present application can be combined with each other without conflict.

[0053] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. The following description of at least one exemplary embodiment is actually only illustrative, but not as any limitation on the present application and its application or use. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0054] It should be noted that the terms used herein are only for describing specific embodiments, and are not intended to limit the exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form, unless the context clearly indicates otherwise, and it should also be understood that when the terms "comprise" and / or "include" are used in the specification, there is a presence of the features, steps, operations, devices, components and / or combinations thereof.

[0055] Unless specifically stated otherwise, the relative arrangement of the components and steps illustrated in these embodiments and the numerical expressions and values set forth herein are not limiting of the scope of the present application. It should be understood that the various parts of the drawings are not necessarily drawn to scale, and that, for the purpose of convenience and clarity, not all components can be shown in a given figure. Techniques, methods, and devices known to those of ordinary skill in the art can not be discussed in detail, but can be assumed to be part of the present application. In all examples shown and discussed herein, any specific value should be interpreted as merely an example, and not as a limitation. Thus, other examples of the exemplary embodiments can have different values. It should be noted that like reference numerals and letters refer to like items in the following drawings, and thus, once an item is defined in one drawing, it need not be discussed further in subsequent drawings.

[0056] In the description of the present application, it should be understood that the orientation words such as "front, back, up, down, left, right", "transverse, vertical, perpendicular, horizontal" and "top, bottom" and the like indicated orientation or position relationship is generally based on the orientation or position relationship shown in the drawings, only for the convenience of describing the present application and simplifying the description, without making the opposite statement, these orientation words do not indicate and imply that the device or element referred to must have a particular orientation or be constructed and operated in a particular orientation, therefore, it cannot be understood as a limitation on the scope of protection of the present application; the orientation words "inner, outer" refer to the inner and outer of the contour of each component itself.

[0057] In addition, it should be noted that the use of "first", "second" and the like to limit the parts is only for the convenience of distinguishing the corresponding parts, and the above words have no special meaning unless otherwise stated, therefore, it cannot be understood as a limitation on the scope of protection of the present application. In addition, although the terms used in the present application are selected from the commonly known and used terms, some terms mentioned in the specification of the present application may be selected by the applicant according to his or her judgment, and the detailed meaning of each term is explained in the relevant part of the description. In addition, the present application is required to be understood not only by the actual terms used, but also by the meaning implied by each term.

[0058] Figure 1 The flow chart of the IOU matching method of one embodiment of the present application is shown. As shown in the figure, an IOU matching method comprises the following steps:

[0059] S1, establishing the IOU matrix of the parking space set O and the predicted trajectory set P based on the parking space set O obtained based on the current frame detection result and the predicted trajectory set P of the current frame, comprising the following steps:

[0060] S11, finding the intersection of the position overlapping area of each parking space OI in the parking space set O and each predicted trajectory Pn in the predicted trajectory set P, wherein OI∈parking space set O; Pn∈P;

[0061] The position overlapping area is a polygon, the area of the position overlapping area, the area of the parking space OI corresponding to the position overlapping area and the area of the predicted trajectory Pn are calculated according to the intersection of the polygon, and the IOU value is calculated according to the area of the position overlapping area, the area of the parking space OI and the area of the predicted trajectory Pn;

[0062] S12, establishing the IOU matrix based on the corresponding relationship between the IOU value and the parking space OI and the predicted trajectory Pn, each IOU value in the row and column of the IOU matrix corresponds to a parking space and a predicted trajectory;

[0063] S2, matching judgment, obtaining a group of IOU values in the IOU matrix that meet the optimal matching, and the parking space and the predicted trajectory corresponding to the group of IOU values are taken as the matching result and output.

[0064] The IOU matching method provided by the application calculates the IOU through the polygon area represented by the position overlapping area of each parking space Oi in the parking space set O and each trajectory Pn in the predicted trajectory set P, and establishes the corresponding IOU matrix. The traditional IOU calculation is to calculate the intersection over union of the rectangular frame of the parking space and the predicted parking space. Due to the perspective problem of the surround view picture obtained in the parking space detection process, the obtained parking space is not a rectangle but a polygon. If the traditional method is used to calculate the IOU, the parking space is approximated to a rectangle for calculation, which leads to redundancy in area calculation, thus increasing the false matching and reducing the parking space tracking effect.

[0065] Preferably, in step S11, the area of the polygon represented by the position overlapping area is area1, the area of the polygon represented by the parking space corresponding to the position overlapping area is area2, and the area of the polygon represented by the predicted trajectory is area3.

[0066] IOU = area1 / (area2 + area3 - area1).

[0067] Preferably, each parking space Oi in the parking space set O includes parking space position information, and the parking space position information includes four vertex information of the parking space, or two vertex information of the parking space and direction information of the parking space; each predicted trajectory Pn in the predicted trajectory set P includes predicted position information, and the predicted position information includes four vertex information of the predicted trajectory, or two vertex information of the predicted trajectory and direction information of the predicted trajectory. It is easy to understand that the four vertex information is equivalent to the two vertex information and the direction information of the parking space or the predicted trajectory. Both of them can be used to determine the position of the parking space or the predicted trajectory and calculate the area of the parking space. If the parking space position information is two vertex information of the parking space and direction information of the parking space. According to the definition of the parking space standard, the distance between the long side and the short side of the parking space standard frame is fixed. Therefore, the distance between the two vertices can be used to determine whether the two vertices belong to the long side or the short side of the parking space. Once the two vertices are determined as the long side or the short side of the parking space, the specific position of the other two vertices of the parking space can be easily obtained by calculation combined with the direction information of the parking space, so that the complete four vertex information can be obtained.

[0068] In step S11, the four vertex information of the position overlapping area of the parking space Oi and the predicted trajectory Pn is obtained based on the parking space position information and the predicted position information; in step S12, the area of the position overlapping area is calculated based on the four vertex information of the position overlapping area, and the area of the parking space Oi and the area of the predicted trajectory Pn are calculated based on the parking space position information and the predicted position information respectively.

[0069] Preferably, the polygon represented by the position overlapping region is a parallelogram, which is different from the rectangle used in the traditional IOU calculation.

[0070] Preferably, in step S11, the Sather-Hodgman algorithm is used to calculate the four vertex information of the position overlapping region, and the shoelace formula is used to calculate the area of the position overlapping region, the area of the parking space and the area of the predicted trajectory. As an example but not limitation, the skilled in the art can also use other calculation methods to calculate the area of the polygon represented by the position overlapping region, the area of the parking space and the predicted trajectory, for example, cutting the polygon into multiple triangles, calculating the area of each triangle respectively, and then obtaining the area of the polygon by summation.

[0071] Preferably, in step S2, the Hungarian algorithm is used to obtain a set of IOU values that meet the optimal matching in the IOU matrix. It should be noted that the Hungarian algorithm mainly solves the problem of reasonable allocation, which allocates the boundary box (the boundary of the polygon) of a certain parking space Oi in the parking space set O of the current frame to the boundary box of a certain predicted trajectory Pn in the predicted trajectory set P, so that the boundary box of the predicted trajectory Pn finds the boundary box of the parking space Oi that best matches it, to achieve the optimal matching effect.

[0072] The process of establishing the IOU matrix is described in detail below. Figure 2 A schematic diagram of the polygon represented by the position overlapping region of the parking space and the predicted trajectory according to an embodiment of the present application is shown. As shown in the figure, in step S11, a certain parking space in the parking space set O of the current frame is schematically shown as a polygon S1, and the parking space position information of the parking space includes the four vertices t1, t2, m3 and t4 of the polygon S1. A certain predicted trajectory in the predicted trajectory set P of the current frame is schematically shown as a polygon S2, and the predicted position information of the predicted trajectory includes the four vertices m1, n2, n3 and n4 of the polygon S2. The Sather-Hodgman algorithm is used to calculate the four vertices m1, m2, m3 and m4 of the polygon S3 represented by the position overlapping region of the polygons S1 and S2.

[0073] Then, based on the four vertices m1, m2, m3 and m4 of the polygon S1 of the position overlapping region, the shoelace formula is used to calculate the area of the polygon S1 as area1, based on the four vertices t1, t2, m3 and t4 of the polygon S2, the shoelace formula is used to calculate the area of the polygon S2 represented by the parking space as area2, and based on the four vertices m1, n2, n3, n4 of the polygon S3, the shoelace formula is used to calculate the area of the polygon S3 represented by the predicted trajectory as area3. Then, IOU = area1 / (area2+area3-area1) is calculated.

[0074] In step S12, the IOU value of each parking space in the parking space set O and each trajectory in the corresponding predicted trajectory set P is assigned to the IOU matrix. It should be noted that if a certain parking space and a certain predicted trajectory have no overlap, it means that the area area3 of the position overlap region of the parking space and the predicted trajectory is 0. Therefore, the IOU value calculated by the calculation formula of IOU is also 0, and 0 is assigned to the corresponding associated position of the parking space and the predicted trajectory in the IOU matrix.

[0075] Figure 3 A flow chart of a parking space tracking method according to an embodiment of the present application is shown. As shown in the figure, a parking space tracking method includes the following steps:

[0076] T1, obtaining a parking space set O based on the current frame detection result, performing parking space prediction on the current frame based on the trajectory set T updated in the previous frame to obtain a predicted trajectory set P of the current frame;

[0077] T2, performing the aforementioned IOU matching method to obtain the optimal matching result between the parking space Oi and the predicted trajectory Pn;

[0078] T3, updating the trajectory set T, updating the trajectory Tm corresponding to the predicted trajectory Pn according to the parking space Oi in the matching result respectively, obtaining the updated trajectory set P and returning to step T1, wherein Tm∈trajectory set T, the update of the trajectory Tm includes the update of the actual position information TL of the trajectory Tm in the current frame, the update of the occupancy state information TC of the trajectory, the update of the motion model and the update of the life cycle of the trajectory validity.

[0079] The parking space tracking method provided by the present application uses the aforementioned IOU matching method to calculate the IOU value by using a polygon instead of a rectangle, obtains the optimal matching result of the parking space set O and the predicted trajectory set P according to the established IOU matrix, and updates the trajectory Tm corresponding to the predicted trajectory Pn through the matching result. No matter how the shape represented by the parking space changes, the parking space position information will not be transient, so that the result of the parking space tracking prediction is more accurate and more stable.

[0080] Preferably, the parking space position information OL acquired based on the actual detection of the current frame includes the position information of the parking space Oi and the occupancy state information OC indicating the occupancy state of the parking space; the predicted trajectory Pn includes the predicted position information PL and the occupancy state information PC indicating the occupancy state of the predicted trajectory Pn; each trajectory Tm in the trajectory set includes the actual position information TL of the trajectory in the current frame, the occupancy state information TC indicating the occupancy state of the trajectory, the motion model, and the life cycle indicating the validity of the trajectory; the occupancy state information can be understood as a flag bit, usually represented by 0 or 1, 0 representing an idle occupancy state and 1 representing an occupied state. The motion model can be one of a uniform Kalman model, an infinite Kalman model, a particle filter Kalman model, a kernel correlation filter model, and a Monte Carlo model. The life cycle can be represented by a numerical value, and the larger the numerical value, the more times the trajectory is not updated, and the smaller the numerical value, the fewer times the trajectory is not updated. The initial value of the life cycle is set to 0.

[0081] Confirming the actual position information TL based on the parking space position information OL and the predicted position information PL;

[0082] Updating the occupancy state information TC of the trajectory Tm corresponding to the predicted trajectory Pn based on the occupancy state information OC of the parking space Oi;

[0083] Updating the motion model of the trajectory Tm corresponding to the predicted trajectory Pn based on the parking space position information OL;

[0084] Initializing the life cycle of the trajectory Tm for the trajectory Ti with a matching result.

[0085] Preferably, if a parking space Oi does not match all predicted trajectories Pn, it indicates that a new parking space is acquired based on the detection result of the current frame, and a new trajectory Tm is initialized for the parking space Oi; if a predicted trajectory Pn does not match all parking spaces, it indicates that the predicted trajectory Pn is not detected by the current frame, and the trajectory Tm corresponding to the predicted trajectory Pn is retained or deleted according to the life cycle of the trajectory Tm. For example, if the numerical value represented by the life cycle exceeds a set threshold, the trajectory is deleted, otherwise it is retained, the life cycle is increased by a set step value, and the next frame is waited for processing.

[0086] Preferably, in step T1, the trajectory set P updated based on the previous frame is used to predict the parking space of the current frame to acquire the predicted trajectory set P of the current frame, including the following steps:

[0087] For each trajectory Tm in the trajectory set T, the possible position of the trajectory Ti in the current frame is predicted based on the motion model of the trajectory Tm, and the obtained prediction result is taken as the prediction position information PL of the prediction trajectory Pi; the occupancy state information TC of the trajectory Ti is taken as the occupancy state information PC of the corresponding prediction trajectory.

[0088] Preferably, in step T1, if the trajectory set T of the previous frame is empty, all parking spaces Oi in the parking space set O obtained from the current frame detection result are initialized to generate corresponding new trajectories Tm and added to the trajectory set T.

[0089] Table 1 is an IOU matrix of 3 parking spaces (parking space A, parking space B, parking space D) contained in the parking space set O obtained based on the current frame detection result and 3 prediction trajectories (prediction trajectory A, prediction trajectory B, prediction trajectory C) contained in the prediction trajectory set P of the current frame.

[0090]

[0091]

[0092] Table 1

[0093] Referring to Table 1, if the first set threshold is 0.5, it can be considered that the parking space and the prediction trajectory corresponding to the IOU value are matched if the IOU value is greater than 0.5. It can be known from Table 1 that the parking space A matches the prediction trajectory A, the parking space A matches the prediction trajectory B, and the parking space B matches the prediction trajectory B. The Hungarian algorithm can be used to find a set of IOU values that meet the optimal matching from the IOU matrix, that is, the parking space A matches the prediction trajectory A, and the parking space B matches the prediction trajectory B. If it is not the optimal matching, it is assumed that the parking space A matches the prediction trajectory B, and another matching will be lost. After obtaining the optimal matching, the trajectory A corresponding to the prediction trajectory A and the trajectory B corresponding to the prediction trajectory B are updated respectively. Further, the IOU value of the parking space D and all prediction trajectories is less than the first set threshold, which indicates that the parking space D is not matched to any prediction trajectory, and the parking space D is initialized to generate a new trajectory and added to the trajectory set. The IOU value of the prediction trajectory C and all parking spaces is less than the first set threshold, and the prediction trajectory C is not matched to any parking space, so the life cycle of the prediction trajectory C is increased by 1. If the value of the life cycle is greater than the second set threshold, the trajectory C is deleted from the trajectory set, and if not, it is retained.

[0094] The following illustrates a parking space tracking method provided by the present application.

[0095] The surround view camera of the vehicle is turned on to obtain a first frame of surround view picture, a parking space set O of detection result of the surround view picture is obtained, each parking space in the parking space set O contains parking space position information and occupancy state information (i.e. whether occupied), and it is assumed that the parking space set O of detection result of the first frame of surround view picture contains parking space A, parking space B and parking space C. Since the current track set is empty, the predicted track set P is also empty, and a new track is generated by using all parking spaces to be added to the track set. The initialization operation contains generating actual position information of the track by using the parking space position information of the parking space, initializing the motion model, the life cycle being 0, and the occupancy state information being idle. The corresponding track A, track B and track C are generated from the parking space A, parking space B and parking space C respectively.

[0096] A second frame of surround view picture is obtained to obtain a parking space set O of detection result of the surround view picture, and it is assumed that the parking space set O contains parking space B, parking space C, parking space D and parking space E. The track A, track B and track C generate a predicted track set P at the current time, and the predicted track set P contains the predicted tracks predictA, predictB and predictC. The predicted tracks predictA, predictB and predictC are matched with the four parking spaces by using IOU. If the optimal matching result is that the parking space B is matched with the predicted track predictB and the parking space C is matched with the predicted track predictC, the actual position information, occupancy state information, motion model and life cycle of the track B are updated by using the parking space B, and the actual position information, occupancy state information, motion model and life cycle of the track C are updated by using the parking space C. For the track that is not matched, such as the track A, the life cycle of the track A is increased by 1, and it is judged whether the life cycle of the track A is greater than a second set threshold value. If yes, the track A is deleted, and if no, the track A is retained. It is assumed that the second set threshold value is 0.8, and the track A is deleted. For the parking space that is not matched with the track, such as the parking space D and parking space E, new tracks, i.e. track D and track E, are generated by using the parking space D and parking space E for initialization. After the second frame of surround view picture is tracked, the tracks that are updated and retained are the track B, track C, track D and track E.

[0097] The third frame of surround view picture is obtained, a parking space set O of detection results of the surround view picture is obtained, and it is assumed that the parking space set O includes parking space C and parking space F. The predicted trajectories of trajectory B, trajectory C, trajectory D and trajectory E at the current time are predictB, predictC, predictD and predictE. The predicted trajectories of trajectory B, trajectory C, trajectory D and trajectory E are matched with two parking spaces of the third frame in IOU. If the optimal matching result is that parking space C and predicted trajectory predictC are matched, the actual position information, occupancy state information, motion model and life cycle of trajectory C are updated using parking space C. For trajectories that are not matched, such as trajectory B, trajectory D and trajectory E, the life cycle is increased by 1, and the life cycle of the trajectory is greater than the second set threshold 0.8, so the trajectories B, D and E are deleted. For the parking space F that is not matched, a new trajectory F is initialized and generated using the parking space F. After the third frame of surround view picture is tracked, the updated and retained trajectories are trajectory C and trajectory F.

[0098] After each time a new surround view picture is obtained, the same parking space tracking method is used for processing.

[0099] Figure 4 The structure of the IOU matching device of one embodiment of the application is shown. As shown in the figure, the application also provides an IOU matching device 400 for implementing the foregoing IOU matching method. The IOU matching device 400 mainly includes an acquisition unit 401, a calculation unit 402, a matrix establishing unit 403 and a judgment unit 404.

[0100] The acquisition unit 401 is used to acquire the parking space set O and the predicted trajectory set P.

[0101] The calculation unit 402 is used to calculate the intersection of the polygons represented by the position overlapping regions, calculate the area of the position overlapping region based on the intersection of the polygons, calculate the area of the parking space Oi corresponding to the position overlapping region and the area of the predicted trajectory Pn, and calculate the IOU value according to the area of the position overlapping region, the area of the parking space Oi and the area of the predicted trajectory Pn.

[0102] The matrix establishing unit 403 establishes the IOU matrix based on the correspondence between the IOU value and the parking space Oi and the predicted trajectory Pn.

[0103] The judgment unit 404 acquires a group of IOU values that meet the optimal matching in the IOU matrix, and outputs the parking space and the predicted trajectory corresponding to the group of IOU values as the matching result.

[0104] The application further provides an IOU matching device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the foregoing IOU matching methods when executing the computer program.

[0105] The application further provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the steps of any of the foregoing IOU matching methods.

[0106] The specific implementation manners and technical effects of the IOU matching apparatus, the IOU matching device, and the computer readable storage medium can be referred to the foregoing embodiments of the IOU matching method provided by the application, and will not be described herein again.

[0107] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0108] The various illustrative logical blocks, and circuits described in connection with the embodiments disclosed herein can be implemented or performed with a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general purpose processor can be a microprocessor, but in the alternative, the processor can be any conventional processor, controller, microcontroller, or state machine. A processor can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0109] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor. The processor and the storage medium can reside in an ASIC. The ASIC can reside in a user terminal. In the alternative, the processor and the storage medium can reside as discrete components in a user terminal.

[0110] In one or more exemplary embodiments, the functions described can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functions can be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media can be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray® disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0111] As will be apparent to those of ordinary skill in the art, various modifications and variations can be made to the above-described exemplary implementations of the present application without departing from the spirit or scope of the present application. Thus, it is intended that the present application cover the modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.

Claims

1. An IOU matching method, comprising the steps of: S1, establishing an IOU matrix of a parking space set O and a predicted trajectory set P based on the parking space set O obtained from the current frame detection result and the predicted trajectory set P of the current frame, comprising the steps of: S11, find the intersection of the position overlapping area of each of the parking spaces Oi in the parking space set O and each of the predicted trajectory set P, wherein, a parking space Oi in the parking space set O; a predicted trajectory Pn in P; the position overlap area is a polygon, the area of the position overlap area is calculated according to the intersection of the polygon, the area of the parking space Oi corresponding to the position overlap area and the area of the predicted trajectory Pn, and the IOU value is calculated according to the area of the position overlap area, the area of the parking space Oi and the area of the predicted trajectory Pn; S12, establishing the IOU matrix based on the IOU value and the corresponding relationship between the parking space Oi and the predicted trajectory Pn; S2, matching judgment, obtaining a group of IOU values that meet the optimal matching in the IOU matrix, and the parking space and the predicted trajectory corresponding to the group of IOU values are taken as the matching result and output; In step S11, the area of the polygon represented by the position overlap area is area1, the area of the polygon represented by the parking space Oi corresponding to the position overlap area is area2, and the area of the polygon represented by the predicted trajectory Pn is area3; IOU = area1 / (area2+area3-area1).

2. The IOU matching method of claim 1, wherein, Each parking space Oi in the parking space set O includes parking space position information, and the parking space position information includes four vertex information of the parking space, or two vertex information of the parking space and direction information of the parking space; each predicted trajectory Pn in the predicted trajectory set P includes predicted position information, and the predicted position information includes four vertex information of the predicted trajectory, or two vertex information of the predicted trajectory and direction information of the predicted trajectory; In step S11, four vertex information of the position overlap area of the parking space Oi and the predicted trajectory Pn is obtained based on the parking space position information and the predicted position information; The area of the position overlap area is calculated based on the four vertex information of the position overlap area, and the area of the parking space Oi and the area of the predicted trajectory Pn are calculated based on the parking space position information and the predicted position information respectively.

3. The IOU matching method of claim 1, wherein, The polygon represented by the position overlap area is a parallelogram.

4. A parking space tracking method characterized by, comprising the steps of: T1, obtaining a parking space set O based on the current frame detection result, and performing parking space prediction on the current frame based on a trajectory set T updated in the previous frame to obtain a predicted trajectory set P of the current frame; T2, performing the IOU matching method as claimed in claim 1 to obtain the matching result between the parking space Oi and the predicted trajectory Pn; T3, updating the trajectory set T, updating the trajectory Tm corresponding to the predicted trajectory Pn in the matching result respectively to obtain an updated trajectory set P and return to step T1, wherein Tm is in the trajectory set T, and the updating of the trajectory Tm includes updating of actual position information TL used to represent the actual position of the trajectory Tm in the current frame, updating of occupancy state information TC used to represent the occupancy state of the trajectory, updating of the motion model, and updating of the life cycle used to represent the validity of the trajectory.

5. The parking space tracking method according to claim 4, characterized in that, The parking space Oi includes parking space position information OL obtained based on actual detection of a current frame and occupancy state information OC indicating an occupancy state of the parking space, and the predicted trajectory Pn includes predicted position information PL and occupancy state information PC indicating an occupancy state of the predicted trajectory Pn; The actual position information TL is determined based on the parking space position information OL and the predicted position information PL; The occupancy state information TC of the trajectory Tm corresponding to the predicted trajectory Pn is updated based on the occupancy state information OC of the parking space Oi; The motion model of the trajectory Tm corresponding to the predicted trajectory Pi is updated based on the parking space position information OL; For the trajectory Ti with a matching result, the life cycle of the trajectory Tm is initialized.

6. The parking space tracking method according to claim 5, characterized in that, If a parking space Oi does not match any predicted trajectory Pn, a new trajectory Tm is initialized for the parking space Oi; If a predicted trajectory Pn does not match any parking space Oi, the trajectory Tm corresponding to the predicted trajectory Pn is retained or deleted according to the life cycle of the trajectory Tm.

7. The parking space tracking method according to claim 4, characterized in that, In step T1, the parking spaces in the current frame are predicted based on the trajectory set P updated in the previous frame, and the predicted trajectory set P of the current frame is obtained, including the following steps: For each trajectory Tm in the trajectory set T, the possible position of the trajectory Ti in the current frame is predicted based on the motion model of the trajectory Tm, and the obtained prediction result is taken as the predicted position information PL of the predicted trajectory Pi. The occupancy state information TC of the trajectory Ti is used as the occupancy state information PC of the corresponding predicted trajectory.

8. An IOU matching apparatus for implementing the IOU matching method of claim 1, characterized by, The method comprises the following steps: An acquisition unit is configured to acquire the parking space set O and the predicted trajectory set P; A calculation unit is configured to calculate the intersection of the polygons represented by the position overlap region, calculate the area of the position overlap region based on the intersection of the polygons, calculate the area of the parking space Oi corresponding to the position overlap region and the area of the predicted trajectory Pn, and calculate the IOU value based on the area of the position overlap region, the area of the parking space Oi, and the area of the predicted trajectory Pn; A matrix establishment unit is configured to establish the IOU matrix based on the IOU value and the correspondence between the parking space Oi and the predicted trajectory Pn; A judgment unit is configured to acquire a group of IOU values that meet the optimal matching in the IOU matrix, and output the parking space and the predicted trajectory corresponding to the group of IOU values as the matching result.

9. An IOU matching device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the IOU matching method according to any one of claims 1-3.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the IOU matching method according to any one of claims 1-3.

Images