A Method for Detecting Parking Space Marking Points and Tracking and Locating Vehicles Based on Panoramic Images

Abstract

The present invention provides a vehicle positioning method for detecting and tracking parking space landmark points in a panoramic image, which takes into account both motion information and appearance information and correlates them. By using a convolutional neural network to re-identify the features of the parking space landmark points in each frame of the image, feature vectors are obtained as the input for tracking. Through the cascaded matching and IOU matching of the feature vectors of each detection result in the current frame with the feature vectors saved for each predicted tracking target, the probability of successful feature vector matching is increased, which can improve the problem of ID jumps in target tracking under occlusion, improve the accuracy of real-time vehicle positioning, and the positioning method for panoramic images has a lower cost.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and automatic parking, and in particular to a method for detecting parking space marker points and tracking and positioning a vehicle based on a panoramic image. Background Art

[0002] The vehicle positioning of automatic parking is a prerequisite for the automatic parking system to perform path planning and vehicle control. At present, the automatic parking system based on panoramic surround images mainly detects parking spaces on panoramic images, and then controls the vehicle to complete automatic parking through the vehicle positioning method of the Ackerman steering model. However, during the parking process of the vehicle positioning method of the Ackerman steering model, it is necessary to calculate parameters such as turning radius, angular velocity, and heading angle to obtain the change value of the vehicle coordinates. There are problems such as large cumulative errors and the inability to correct them in real time, which leads to large errors in subsequent path planning and vehicle control, affecting the effect of automatic parking.

[0003] In order to better locate the car, a variety of technical means are used, such as an existing image target tracking method based on YOLO, which includes the following steps: S1, input video; S2, use the target detection network YOLO to detect the target and initialize the Kalman filter; S3, detect the current frame image, if the target is detected, go to step S4, otherwise execute step S5; S4, calculate the intersection and union ratio of the detected position and the predicted position of the target in the current frame image, if the intersection and union ratio is greater than the preset threshold, use the detected position of the target as the position of the target in the current frame; S5, match the position of the target in the previous frame with the predicted position of the target in the current frame for key points, if the matching pair is greater than the preset threshold, the position of the target in the current frame is obtained; S6, check whether the video detection is completed, if so, end the tracking, otherwise return to step S3. The key point matching of the target position in the previous frame and the predicted position of the target in the current frame obtained by the Kalman filter can effectively improve the tracking accuracy. However, this method only considers the information of adjacent frames. Although the use of key point matching can effectively avoid the limitation of target tracking loss, it is prone to tracking loss when the number of identity ID changes is large in the face of long-term occlusion, and it is not very robust to omissions and occlusions. Summary of the invention

[0004] In order to solve the problem that the above technical solution is prone to tracking loss and identity ID changes frequently in the face of long-term occlusion, and the robustness to omissions and occlusions is not high, the present invention provides a parking space marker detection and vehicle tracking and positioning method based on panoramic images. The method in this solution considers motion information and appearance information at the same time and associates them to improve the target tracking effect in the case of occlusion and reduce the problem of target tracking ID jump.

[0005] The technical solution adopted by the present invention is: a method for detecting parking space markers and tracking and locating vehicles based on a panoramic image, comprising the following steps:

[0006] Step 1: Before the car performs automatic parking, obtain the mapping relationship between the pixel coordinate distance in the pixel coordinate system on the panoramic surround image and the actual coordinate distance in the world coordinate system;

[0007] Step 2: When the car is automatically parking, the multi-target detector based on the convolutional neural network is used to detect the location information and feature vector of the parking space mark point in the current frame according to the sampling time interval;

[0008] Step 3: Initialize the detector, screen the first detection target, re-identify the features and output the second detection target that meets the preset indicators; wherein the first detection target includes the feature vector of the parking space mark point in step 2;

[0009] Step 4: Initialize the tracker, track the first tracking target and determine its tracking state; wherein the first tracking target includes the second detection target in step 3; when the tracker is initialized, a tracking ID of the first tracking target is automatically generated; the tracking state of the first tracking target includes a determined tracking state and an uncertain tracking state;

[0010] Step 5: Use the Kalman filter algorithm to predict the tracking target and obtain the predicted tracking target in the next frame; wherein the first tracking target in the determined tracking state corresponds to the first determined predicted tracking target in the next frame, and the first tracking target in the uncertain tracking state corresponds to the first uncertain predicted tracking target in the next frame;

[0011] Step 6: Use cascade matching and Hungarian algorithm to match the second detection target and the first determined predicted tracking target and output the first target set generated after matching; wherein the initial frame detection result in step 3 does not participate in the matching, and the matching starts from the detection result of the second frame; the first target set includes the first successfully matched target formed by the successful matching of the second detection target and the first predicted tracking target, the third detection target that is not successfully matched in the second detection target, and the second determined predicted tracking target that is not successfully matched in the first determined predicted tracking target;

[0012] Step 7: Perform IOU matching on the IOU candidate prediction tracking target and the third detection target, and output a second target set generated after the IOU matching; wherein the IOU candidate prediction tracking target includes the first uncertain prediction tracking target and the second certain prediction tracking target; the second target set includes the second successful matching target formed by the successful IOU matching between the IOU candidate prediction tracking target and the third detection target, the second IOU candidate prediction tracking target that failed to perform IOU matching among the IOU candidate prediction tracking targets, and the fourth detection target that failed to perform IOU matching on the third detection target;

[0013] Step 8: Calculate the average value of the coordinate changes of two consecutive frames of the same tracking ID in the pixel coordinate system based on the first successful matching target and the second successful matching target, and calculate the position change of the vehicle in the world coordinate system based on the mapping relationship between the pixel coordinate distance and the actual coordinate distance obtained in step 1.

[0014] The present invention uses a convolutional neural network to re-identify the features of the parking space marking points in each frame of the image, obtains a feature vector as the input for tracking, and performs cascade matching and IOU matching on the feature vector of each detection result of the current frame with the saved feature vector of each predicted tracking target, thereby increasing the probability of successful feature vector matching. This can improve the problem of tracking ID jumping caused by the tracking target being assigned a new tracking ID as a detection target in the next frame due to matching failure in the case of occlusion, thereby improving the accuracy of real-time vehicle positioning. Moreover, based on the panoramic image, a convolutional neural network is used to detect and track the parking space marking points to achieve real-time vehicle positioning. This is a positioning method based on pure vision, and only four fisheye cameras arranged around the vehicle are required to generate a panoramic surround view image, which is lower in cost than the positioning sensors currently commonly used in vehicles.

[0015] Preferably, the first successfully matched target and the second successfully matched target are output to step 4 to become new tracking targets for determining the tracking state, thereby completing the update iteration of the tracking information.

[0016] Preferably, the fourth detection target is output to step 4 to make it a tracking target of a new uncertain tracking state. The second IOU candidate predicted tracking target is screened, and it is selected to be deleted or output to step 4 to make it a new tracking target; wherein the uncertain predicted tracking target in the second IOU candidate predicted tracking target is deleted; and the second determined predicted tracking target in the second IOU candidate predicted tracking target is screened. When screening the second determined predicted tracking target in the second IOU candidate predicted tracking target, the second determined predicted tracking target whose matching times do not exceed the threshold is output to step 4 to make it a new tracking target of a determined tracking state; the second determined predicted tracking target whose matching times exceed the threshold is deleted, and the threshold of the matching times is 100. A container is constructed for each tracking target to store the feature vectors of the most recent 100 frames that are successfully associated with each tracking target, thereby increasing the probability of successful matching of the tracking target and thereby increasing the number of successfully matched targets, so that the vehicle position change reflected by the average value of the coordinate change of two consecutive frames of the tracking ID is more accurate.

[0017] Preferably, in step 2, the "YOLO v5" algorithm is used as a multi-target detector to simultaneously detect parking space markers on the panoramic surround image to obtain the position information of the detection frame. YOLO has the advantages of fast speed, strong generalization ability, and low background prediction error rate. YOLO v3 uses a feature pyramid structure to achieve multi-scale prediction, which has a good effect on the detection of small targets such as markers. YOLO v5 implements adaptive anchor frame calculation and adaptive grayscale filling on this basis, and both accuracy and speed have been greatly improved.

[0018] Preferably, in step 3, the detector automatically generates the location information, confidence and feature vector of the detection target when it is initialized; wherein the detection target is selected and represented by a detection box. The screening of the detection target includes removing the detection boxes whose confidence is less than a threshold, and using the non-maximum suppression algorithm for screening to eliminate the situation where multiple detection boxes are on a target. The purpose of the non-maximum suppression algorithm is to suppress non-maximum targets, thereby searching for local maximum targets.

[0019] Preferably, in step 6, the cascade matching includes motion information association and appearance feature association. Appearance feature association is to calculate the minimum cosine distance matrix between the feature vector of each detection result of the current frame and the feature vector set saved for each predicted tracking target; the motion information association is to calculate the Mahalanobis distance between the predicted tracking target and the detection result in the cosine distance matrix, and set the value of the Mahalanobis distance of the corresponding predicted tracking target greater than the threshold in the cosine distance matrix to infinity. The Mahalanobis distance formula between the predicted tracking target and the detection result is:

[0020]

[0021] Among them, d j Indicates the position of the jth detection box, y i represents the predicted position of the target by the ith tracker, S i Represents the covariance matrix between the detected positions and the average tracked positions.

[0022] The processed cosine distance matrix is used as the input of the Hungarian algorithm to obtain the cascade matching results and remove the matching pairs with large gaps.

[0023] Preferably, in step 7, the IOU calculation method is:

[0024]

[0025] Among them, Area(Bbx i ∩Bbx j ) represents the area where the detected target and the tracked target intersect, Area(Bbx i ∪Bbxj ) represents the area where the detected target and the tracked target overlap.

[0026] Preferably, in step 8, when calculating the average position change of the center points of the tracking boxes with the same tracking ID in two consecutive frames in the pixel coordinate system, a new vehicle coordinate system needs to be established in the pixel coordinate system and the initial coordinates of the vehicle are set within the vehicle coordinate system. Since it is not easy to set the origin of the coordinate system in the pixel coordinate system, the vehicle coordinate system is reset based on the pixel coordinate system. The origin of the vehicle coordinate system is set according to the actual situation, which is convenient for determining the initial coordinates of the vehicle within the vehicle coordinate system.

[0027] Compared with the prior art, the present invention simultaneously considers motion information and appearance information and correlates them. By using a convolutional neural network to re-identify the feature points of the parking space in each frame of the image and obtain feature vectors as the input for tracking, through the cascaded matching and IOU matching of the feature vectors of each detection result in the current frame with the feature vectors saved for each predicted tracking target, the probability of successful feature vector matching is increased, which can improve the problem of tracking ID jumps caused by the tracked target being re-assigned a new tracking ID as a detected target in the next frame due to matching failure under occlusion, improve the accuracy of vehicle real-time positioning, and the positioning method for panoramic images has a lower cost. Description of the Drawings

[0028] Figure 1 is a flowchart of a method for detecting parking space feature points and vehicle tracking and positioning based on panoramic images according to the present invention.

[0029] Figure 2 is a schematic diagram of using a multi-object detector to detect parking space feature points during automatic parking in a method for detecting parking space feature points and vehicle tracking and positioning based on panoramic images according to the present invention.

[0030] Figure 3 is a flowchart of the Hungarian algorithm in a method for detecting parking space feature points and vehicle tracking and positioning based on panoramic images according to the present invention. Detailed Embodiments

[0031] The drawings are only for illustrative purposes and should not be construed as a limitation to this patent; for better illustration of this embodiment, some parts in the drawings may be omitted, enlarged or reduced, which do not represent the dimensions of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted. The positional relationships described in the drawings are only for illustrative purposes and should not be construed as a limitation to this patent.

[0032] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "long", "short" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limitations on this patent. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0033] The technical solution of the present invention is further described in detail below through specific embodiments and in conjunction with the accompanying drawings:

[0034] Example 1

[0035] like Figure 1 - Figure 2 An embodiment of a method for detecting parking space markers and tracking and positioning a vehicle based on a panoramic image is shown, comprising the following steps:

[0036] Step 1: Before the car performs automatic parking, obtain the mapping relationship between the pixel coordinate distance in the pixel coordinate system on the panoramic surround image and the actual coordinate distance in the world coordinate system;

[0037] Step 2: When the car is performing automatic parking, the multi-target detector based on the convolutional neural network is used to detect the location information and feature vector of the parking space mark point in the current frame according to the sampling time interval;

[0038] Step 3: Initialize the detector, screen the first detection target, re-identify the features and output the second detection target that meets the preset indicators; wherein the first detection target includes the feature vector of the parking space mark point in step 2;

[0039] Step 4: Initialize the tracker, track the first tracking target and determine its tracking state; wherein the first tracking target includes the second detection target in step 3; when the tracker is initialized, a tracking ID of the first tracking target is automatically generated; the tracking state of the first tracking target includes a determined tracking state and an uncertain tracking state;

[0040] Step 5: Use the Kalman filter algorithm to predict the tracking target and obtain the predicted tracking target in the next frame; wherein the first tracking target in the determined tracking state corresponds to the first determined predicted tracking target in the next frame, and the first tracking target in the uncertain tracking state corresponds to the first uncertain predicted tracking target in the next frame;

[0041] Step 6: Use cascaded matching and the Hungarian algorithm to match the second detection target with the first determined predicted tracking target and output the first target set generated after matching; among which, the initial frame detection result in Step 3 does not participate in the matching, and the matching starts from the detection result of the second frame; the first target set includes the first successfully matched target formed by the successful matching of the second detection target with the first predicted tracking target, the third detection target that fails to be successfully matched among the second detection targets, and the second determined predicted tracking target that fails to be successfully matched among the first determined predicted tracking targets;

[0042] Step 7: Perform IOU matching on the IOU candidate predicted tracking targets and the third detection target, and output the second target set generated after IOU matching; among which, the IOU candidate predicted tracking targets include the first uncertain predicted tracking target and the second determined predicted tracking target; the second target set includes the second successfully matched target formed by the successful IOU matching of the IOU candidate predicted tracking targets with the third detection target, the second IOU candidate predicted tracking target that fails to be successfully IOU matched among the IOU candidate predicted tracking targets, and the fourth detection target that fails to be successfully IOU matched among the third detection targets;

[0043] Step 8: Calculate the average value of the coordinate changes of the same tracking ID in two consecutive frames in the pixel coordinate system based on the first successfully matched target and the second successfully matched target, and calculate the position change of the vehicle in the world coordinate system according to the mapping relationship between the pixel coordinate distance and the actual coordinate distance obtained in Step 1.

[0044] Specifically, output the first successfully matched target and the second successfully matched target to Step 4 to make them the tracking targets in the new determined tracking state.

[0045] Specifically, output the fourth detection target to Step 4 to make it the tracking target in the new uncertain tracking state. Screen the second IOU candidate predicted tracking targets, and select to delete them or output them to Step 4 to make them the new tracking targets; among which, delete the uncertain predicted tracking targets in the second IOU candidate predicted tracking targets; screen the second determined predicted tracking targets in the second IOU candidate predicted tracking targets. When screening the second determined predicted tracking targets in the second IOU candidate predicted tracking targets, output the second determined predicted tracking targets with the number of matches not exceeding the threshold to Step 4 to make them the tracking targets in the new determined tracking state; delete the second determined predicted tracking targets with the number of matches exceeding the threshold, and the threshold of the number of matches is 100.

[0046] Beneficial effects of this embodiment: The present invention considers motion information and appearance information at the same time and associates them. By using a convolutional neural network to re-identify the features of the parking space marking points in each frame of the image, a feature vector is obtained as the input for tracking, and the feature vector of each detection result in the current frame is cascade matched and IOU matched with the saved feature vector of each predicted tracking target to increase the probability of successful feature vector matching. This can improve the problem of tracking ID jumping caused by the tracking target being re-assigned a new tracking ID as a detection target in the next frame due to matching failure in the case of occlusion, improve the accuracy of real-time vehicle positioning, and the positioning method of panoramic images is more cost-effective.

[0047] The first successfully matched target and the second successfully matched target are output to step 4 to become the new tracking targets for determining the tracking status, completing the update iteration of the tracking information. A container is constructed for each tracking target to store the feature vectors of the last 100 frames successfully associated with each tracking target, increasing the probability of successful matching of the tracking target and thus increasing the number of successful matching targets, so that the average value of the coordinate change of two consecutive frames of the tracking ID reflects the vehicle position change more accurately.

[0048] Example 2

[0049] Embodiment 2 of a method for detecting parking space marker points and tracking and positioning vehicles based on a panoramic image, based on embodiment 1, as Figure 1 - Figure 2 As shown, steps 2 and 3 are further limited.

[0050] Specifically, in step 2, the "YOLO v5" algorithm is used as a multi-target detector to simultaneously detect parking space marking points on the panoramic surround image to obtain position information of the detection frame.

[0051] Specifically, in step 3, the detector automatically generates the location information, confidence and feature vector of the detection target when it is initialized; the detection target is selected and represented by a detection box. The screening of the detection target includes removing the detection boxes with confidence less than the threshold, and using the non-maximum suppression algorithm to screen and eliminate the situation where there are multiple detection boxes on a target.

[0052] Beneficial effects of this embodiment: YOLO has the advantages of fast speed, strong generalization ability, low background prediction error rate, etc. YOLO v3 uses a feature pyramid structure to achieve multi-scale prediction, which has a good effect on small target detection such as landmarks. YOLO v5 implements adaptive anchor box calculation and adaptive grayscale filling on this basis, and the accuracy and speed are greatly improved. The purpose of the non-maximum suppression algorithm is to suppress non-maximum targets, thereby searching for local maximum targets.

[0053] Example 3

[0054] Example 3 of a method for detecting parking space landmark points and vehicle tracking and positioning based on panoramic images. On the basis of Example 1 or Example 2, steps 6 to 8 are further defined.

[0055] Specifically, in step 6, the cascade matching includes motion information association and appearance feature association. Appearance feature association is to calculate the minimum cosine distance matrix between the feature vectors of each detection result in the current frame and the set of feature vectors saved for each determined predicted tracking target; the association of motion information is to calculate the Mahalanobis distance between the determined predicted tracking target and the detection result in the cosine distance matrix, and set the values in the cosine distance matrix where the Mahalanobis distance of the corresponding determined predicted tracking target is greater than the threshold to infinity. The formula for the Mahalanobis distance between the predicted tracking target and the detection result is:

[0056]

[0057] where d j represents the position of the j-th detection box, and y i represents the predicted position of the target by the i-th tracker, and S i represents the covariance matrix between the detection position and the average tracking position.

[0058] Take the processed cosine distance matrix as the input of the Hungarian algorithm to obtain the cascade matching result and remove the matching pairs with large differences.

[0059] Specifically, in step 7, the IOU calculation method is:

[0060]

[0061] where Area(Bbx i ∩Bbx j ) represents the area where the detection target and the tracking target intersect, and Area(Bbx i ∪Bbx j ) represents the area where the detection target and the tracking target are combined.

[0062] Specifically, when the successful matching results in step 6 and the IOU successful matching results in step 7 are output to step 4, the successful matching results in step 6 and the IOU successful matching results in step 7 are used as the tracking targets for determining the tracking state, and the update iteration of the tracking information is completed. The matching times threshold for the determined predicted tracking targets that are not successfully matched is 100.

[0063] Specifically, when calculating the average position change of the center points of the tracking boxes with the same tracking ID in two consecutive frames in the pixel coordinate system in step 8, a new vehicle coordinate system needs to be established in the pixel coordinate system and the initial coordinates of the vehicle need to be set within the vehicle coordinate system.

[0064] Advantages of this embodiment: By calculating the minimum cosine distance matrix between the feature vectors of each detection result in the current frame and the set of saved feature vectors of each predicted tracking target, and calculating the Mahalanobis distance matrix between the predicted tracking target and the detection result, the result after linearly weighting the two is used as the input of the Hungarian matching algorithm to obtain the matching result, making the matching result more accurate. Based on the pixel coordinate system, the vehicle coordinate system is reset. The origin of the vehicle coordinate system is set according to the actual situation, which is convenient for determining the initial coordinates of the vehicle within the vehicle coordinate system and preventing the difficulty in determining the origin of the coordinate system within the pixel coordinate system.

[0065] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, rather than limiting the implementation manners of the present invention. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to enumerate all the implementation manners here. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A method for detecting parking space landmark points and vehicle tracking and positioning based on panoramic images, characterized in that, The steps include: Step 1: Before the car performs automatic parking, obtain the mapping relationship between the pixel coordinate distance in the pixel coordinate system on the panoramic surround image and the actual coordinate distance in the world coordinate system; Step 2: When the car is performing automatic parking, the multi-target detector based on the convolutional neural network is used to detect the location information and feature vector of the parking space mark point in the current frame according to the sampling time interval; Step 3: Initialize the detector, screen the first detection target, re-identify the features and output the second detection target that meets the preset indicators; wherein the first detection target includes the feature vector of the parking space mark point in step 2; Step 4: Initialize the tracker, track the first tracking target and determine its tracking status; The first tracking target includes the second detection target in step 3; the tracking ID of the first tracking target is automatically generated when the tracker is initialized; the tracking state of the first tracking target includes a determined tracking state and an uncertain tracking state; Step 5: Use the Kalman filter algorithm to predict the tracking target and obtain the predicted tracking target in the next frame; wherein the first tracking target in the determined tracking state corresponds to the first determined predicted tracking target in the next frame, and the first tracking target in the uncertain tracking state corresponds to the first uncertain predicted tracking target in the next frame; Step 6: Use cascade matching and Hungarian algorithm to match the second detection target and the first determined predicted tracking target and output the first target set generated after matching; wherein the initial frame detection result in step 3 does not participate in the matching, and the matching starts from the detection result of the second frame; the first target set includes the first successfully matched target formed by the successful matching of the second detection target and the first predicted tracking target, the third detection target that is not successfully matched in the second detection target, and the second determined predicted tracking target that is not successfully matched in the first determined predicted tracking target; Step 7: Perform IOU matching on the IOU candidate prediction tracking target and the third detection target, and output a second target set generated after the IOU matching; wherein the IOU candidate prediction tracking target includes the first uncertain prediction tracking target and the second certain prediction tracking target; the second target set includes the second successful matching target formed by the successful IOU matching between the IOU candidate prediction tracking target and the third detection target, the second IOU candidate prediction tracking target that failed to perform IOU matching among the IOU candidate prediction tracking targets, and the fourth detection target that failed to perform IOU matching on the third detection target; Step 8: Calculate the average value of the coordinate changes of two consecutive frames of the same tracking ID in the pixel coordinate system based on the first successful matching target and the second successful matching target, and calculate the position change of the vehicle in the world coordinate system based on the mapping relationship between the pixel coordinate distance and the actual coordinate distance obtained in step 1.

2. A method for detecting parking space landmark points and vehicle tracking and positioning based on panoramic images according to claim 1, characterized in that, The first successfully matched target and the second successfully matched target are output to step 4 to become new tracking targets for determining the tracking state.

3. A method for detecting parking space landmark points and vehicle tracking and positioning based on panoramic images according to claim 1, characterized in that, Output the fourth detection target to step 4 to make it a tracking target in a new uncertain tracking state; screen the second IOU candidate predicted tracking target and choose to delete it or output it to step 4 to make it a new tracking target; wherein the uncertain predicted tracking target in the second IOU candidate predicted tracking target is deleted; and the second determined predicted tracking target in the second IOU candidate predicted tracking target is screened.

4. A method for detecting parking space marker points and vehicle tracking and positioning based on panoramic images according to claim 3, characterized in that When screening the second determined predicted tracking target in the second IOU candidate predicted tracking target, the second determined predicted tracking target whose matching times do not exceed the threshold is output to step 4 to make it a tracking target with a new determined tracking state; the second determined predicted tracking target whose matching times exceed the threshold is deleted.

5. A method for detecting parking space landmark points and vehicle tracking and positioning based on panoramic images according to claim 1, characterized in that, In step 2, the "YOLO v5" algorithm is used as a multi-target detector to simultaneously detect parking space landmarks on the panoramic surround image and obtain the location information of the detection frame.

6. A method for detecting parking space landmark points and vehicle tracking and positioning based on panoramic images according to claim 1, characterized in that, In step 3, the detector automatically generates the location information and confidence of the detection target when it is initialized; the detection target is selected and represented by a detection box.

7. A method for detecting parking space landmark points and vehicle tracking and positioning based on panoramic images according to claim 1, characterized in that, In step 3, the screening of detection targets includes removing detection frames whose confidence is less than a threshold, and using a non-maximum suppression algorithm for screening to eliminate the situation where there are multiple detection frames on a detection target.

8. A method for detecting parking space landmark points and vehicle tracking and positioning based on panoramic images according to claim 1, characterized in that, In step 6, the cascade matching includes motion information association and appearance feature association.

9. A method for detecting parking space marker points and vehicle tracking and positioning based on panoramic images according to claim 8, characterized in that, Appearance feature association is to calculate the minimum cosine distance matrix between the feature vector of each detection result in the current frame and the saved feature vector set of each determined tracking prediction result.

10. A method for detecting parking space marker points and vehicle tracking and positioning based on panoramic images according to claim 8, characterized in that, The association of motion information is to calculate the Mahalanobis distance between the tracking prediction result and the detection result in the cosine distance matrix, and set the value of the Mahalanobis distance of the corresponding tracking prediction result greater than the threshold in the cosine distance matrix to infinity.

Images