Precise roadside parking detection method of high-position camera based on monocular three-dimensional vision
By applying monocular three-dimensional vision technology on roadside cameras, the precise positioning and identification of vehicles in three-dimensional space is achieved, and the problem of lack of depth information in 2D parking detection technology is solved, the accuracy and adaptability of detection is improved, and the accuracy and adaptability of detection is reduced. Cost and improve parking lot management efficiency.
Patent Information
- Application Number
- CN202411234095.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2025-01-03
AI Technical Summary
The existing 2D parking detection technology lacks in-depth information, cannot accurately measure the distance between a vehicle and an obstacle, and cannot effectively identify complex parking space layouts and vehicle postures, resulting in detection failure or false inspection.
The high-level camera detection method based on monocular three-dimensional vision is adopted, and the accurate positioning and identification of the vehicle in the three-dimensional space is achieved through camera calibration, distortion correction, 3D object detection network and real-time vehicle tracking module, and the vehicle's 3D information is provided.
Improve the accuracy of vehicle positioning and identification, can effectively avoid obstacles and identify complex parking space layouts in complex environments, fill the gap that 2D technology cannot provide in-depth information, reduce parking management costs, and improve parking lot utilization .
Smart Images
Figure CN119251795A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision technology, and in particular to an accurate roadside parking detection method based on a high-position camera with monocular three-dimensional vision. Background Art
[0002] Roadside monocular vision 3D detection is one of the key technologies to ensure that the parking of vehicles is consistent with the parking space lines. It can monitor the spatial position relationship between the car and the parking space lines in real time during the parking process, providing important guarantees for smart transportation and smart parking. In traditional parking detection, parking detection usually relies on manual inspection, which is inefficient, costly and cannot be detected in real time.
[0003] In recent years, with the continuous advancement of science and technology, roadside parking detection systems have adopted 2D detection technology such as the YOLOV5 algorithm, which has subverted the traditional manual inspection method and greatly improved the detection speed. However, 2D detection cannot perceive the spatial relationship between the car and the parking space, and can only provide information in the horizontal and vertical directions. It lacks depth information, resulting in the inability to perceive the spatial position relationship between the vehicle and the parking space; secondly, 2D detection cannot effectively solve the occlusion problem between vehicles, resulting in detection failure or false detection.
[0004] In summary, the main shortcomings of existing two-dimensional parking detection methods are as follows:
[0005] 1) Lack of depth information makes it impossible to accurately measure the distance between the vehicle and the obstacle;
[0006] 2) Limited scene understanding capabilities, which can only recognize the outer contour and approximate position of the vehicle, but cannot accurately detect the position and posture of the vehicle;
[0007] 3) Limited adaptability to complex environments, i.e., it cannot provide three-dimensional spatial information, cannot effectively avoid obstacles or recognize complex parking space layouts. Summary of the invention
[0008] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a precise roadside parking detection method based on a high-position camera with monocular three-dimensional vision, which has high precision and high real-time performance, realizes the three-dimensional spatial positioning of the vehicle, and improves the accuracy of positioning and recognition. After the roadside camera is calibrated, when the vehicle is monitored, the method inputs the acquired monocular image into the feature extraction network to input the two-dimensional detection result; based on the two-dimensional detection, the monocular three-dimensional detection is performed through the camera parameters.
[0009] In order to achieve the above object, the technical solution adopted by the present invention is:
[0010] A precise roadside parking detection method based on a high-position camera with monocular three-dimensional vision comprises the following steps;
[0011] Step 1: Fix the roadside camera posture, perform camera calibration and distortion correction;
[0012] Step 2: Use a high-position camera to collect raw video, pre-process it and create a vehicle dataset;
[0013] Step 3: Input the collected vehicle data set into the 3D target detection network model for training to obtain the target detection model, select the optimal detection model to detect the vehicle, and output the 3D information of the vehicle in space;
[0014] Step 4: Input the vehicle information obtained in step 3 into the 3D vehicle tracking module to achieve real-time tracking.
[0015] The specific implementation steps of step 1 are:
[0016] Step 1.1: Obtain the camera's internal and external parameters through camera calibration;
[0017] The internal parameters are focal length, image center, and distortion coefficient;
[0018] The external parameters are the rotation matrix and the translation matrix;
[0019] Step 1.2: Distortion correction: correct the radial and tangential distortions of the camera that collects the data; use the internal parameters and distortion coefficients obtained in step 1.1 to correct the distortion of the collected image; use the two functions "cv2.getOptimalNewCameraMatrix" and "cv2.initUndistortRectifyMap" in OpenCV to calculate the correction map; apply the correction map and verify the correction effect through visual inspection and accuracy evaluation to determine the quality of the corrected image;
[0020] Step 1.3: Arrange the camera internal and external parameters obtained in step 1.1 into a 3×4 matrix format and save it as a file for model training.
[0021] The specific implementation steps of step 2 are:
[0022] Step 2.1: According to the camera calibration and distortion correction in step 1, ensure the quality of the collected data; use a high-position camera to collect vehicle video, capture each frame of the image and save the valid image;
[0023] Step 2.2: Perform denoising, brightening preprocessing and annotation on the saved images. The annotation format is KITTI format, where the KITTI label contains the vehicle's two-dimensional bounding box coordinate information, three-dimensional bounding box information, and spatial direction information, etc. The annotated images are divided into training set, verification set and test set in a ratio of 6:2:2.
[0024] The specific implementation steps of step 3 are:
[0025] Step 3.1: Use the improved GUPNet 3D target detection algorithm for detection. On the basis of the original algorithm, Mixup3D data enhancement is introduced to generate new training samples by linearly combining two different 3D samples and labels; thereby improving the generalization ability and robustness of the model, smoothing the decision boundary, and effectively preventing overfitting; in terms of depth prediction, a normalized depth module is introduced to eliminate pitch angle and focal length problems;
[0026] The training model starts with 2D target detection based on CenterNet. The feature extraction network outputs the feat feature map. Based on the feat feature map, it outputs the heatmap, the 2D center point offset offset_2d, and the size of the 2D box size_2d. The heatmap is used to predict the target category and the rough 2D box center point coordinates; offset_2d is used to predict the offset of the 2D center point coordinates; size_2d refers to the width and height of the 2D bounding box of the detected vehicle.
[0027] Step 3.2: Construct a coordinate tensor map coord_map to store the position information in the image coordinate system, obtain the center position box2d_center of the 2d box according to coord_map and offset_2d, and obtain the 2D bounding box map box2d_map according to box2d_center and size_2d;
[0028] Step 3.3: Use RoIAlign to extract ROI features from box2d_maps in step 3.2 to obtain the cropped ROI feature map. According to the camera calibration information in step 1.3, the coordinate map is normalized, and the normalized coordinate map is connected to the cropped ROI feature map to obtain the final ROI feature map. According to the camera calibration information, the image coordinates are converted to the camera coordinate system using internal parameters to achieve the derivation of the target 3D information.
[0029] Step 3.4: Based on the final ROI feature map obtained in step 3.3, predict the 3D information including four branches: ① heading information; ② offset_3d of the 3D center on the image plane; ③ size_3d of the 3D bounding box; ④ depth of the vehicle.
[0030] Step 3.5: After training is completed, select the optimal detection model for vehicle detection.
[0031] The branch ① is the vehicle's deflection angle information. The ROI feature outputs the deflection angle information through the heading branch. The MultiBin strategy is used to divide the 360° angle range into 12 Bins, i.e., 24 components. Among them, 12 components are the probabilities of classifying the deflection angle into a certain angle range, and the other 12 components correspond to residuals in different ranges. The vehicle's deflection angle can be obtained by adding the category with the maximum probability and its corresponding residual.
[0032] The purpose of branch ② is to obtain the 3D position of the vehicle. After the offset_3d branch, the output is the offset of the 3D center on the image plane. The vehicle center estimated in the heatmap plus the offset is used to obtain the projection position of the 3Dbox center on the image plane. The camera calibration information can be used to obtain its xy coordinates in the 3D space, and the estimated vehicle depth z is added to obtain the 3D position of the vehicle.
[0033] The purpose of branch ③ is to obtain the final size of the vehicle. size_3d outputs the offset of the vehicle's 3D size. The final size of the vehicle is obtained by adding the average size of the corresponding category of vehicle information plus the offset.
[0034] The purpose of branch ④ is to obtain the depth value depth of the vehicle.
[0035] The specific steps of branch ④ are as follows: The imaging principle of the monocular camera can be obtained based on the triangle similarity relationship:
[0036]
[0037] z represents the depth of the detected vehicle, f represents the focal length of the monocular camera obtained in step 1, and h 2d represents the imaging height of the detected vehicle obtained in step 3.1, H 3d ′ represents the height of the center point of the detected vehicle, which is obtained from h in step 3.4. 3d Calculated.
[0038] After geometric calculation, we can get:
[0039] H 3d ′=h 3d *(cosθ-sinθ*tanδ)*f
[0040] Substituting into the above formula, we get:
[0041]
[0042] By introducing the ground equation αx+βy+γz+d=0, we get the angle θ, that is, θ=arctan(γ / β), and the angle δ is calculated by projection, that is, Where α is the normal vector component in the x direction, representing the tilt of the plane on the x axis, β is the normal vector component in the y direction, representing the tilt of the plane on the y axis, γ is the normal vector component in the z direction, representing the tilt of the plane on the z axis, d represents the offset between the plane and the origin (0, 0, 0), which determines the position of the plane in three-dimensional space, V p is the y-axis pixel coordinate of point P, C y is the principal point coordinate of the y-axis, so the final depth is expressed as:
[0043]
[0044] The specific implementation steps of step 4 are:
[0045] Step 4.1: Input the output of step 3 into the 3D vehicle tracking module and use the Kalman filter to predict the vehicle trajectory;
[0046] Step 4.2: Use the Hungarian algorithm to associate and match the detection result of the latest image frame received and extracted with the tracked vehicle in the previous frame;
[0047] Step 4.3: For associated targets, use the Kalman filter to update their states to reflect their positions in the current frame. For unassociated targets, start new trajectory tracking.
[0048] Beneficial effects of the present invention:
[0049] The present invention utilizes a roadside monocular camera and adds multiple 3D detection heads on the basis of the traditional 2D parking detection method, so that parking detection is no longer limited by two-dimensional space and the three-dimensional position information of the vehicle can be calculated; compared with parking detection with 2D technology, it fills the gap that 2D technology cannot provide depth information, can more accurately perceive the spatial relationship between the car and the parking space, and provide more information to effectively avoid obstacles or identify complex parking space layouts; at the same time, the invention is one of the important applications in the field of smart transportation, and its promotion and application can promote the development and application of smart transportation technology, which can not only reduce parking management costs, but also improve parking lot utilization and reduce parking accident losses; in addition, it can also promote the development of smart transportation and bring more economic benefits.
[0050] This invention improves the accuracy and stability of the roadside parking detection system through precise camera calibration and distortion correction, high-quality data set production, advanced 3D target detection and real-time 3D vehicle tracking technology. The system can provide accurate vehicle location and status information in real time, providing strong support for traffic management and parking management in smart cities.
[0051] In summary, the method proposed in the present invention has high vehicle detection accuracy and low equipment cost, and can achieve accurate three-dimensional detection tasks in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Figure 1 This is a flow chart of the precise roadside parking detection method using a high-position camera with monocular three-dimensional vision of the present invention.
[0053] Figure 2 This is a network framework diagram for 3D target detection in the present invention.
[0054] Figure 3 This is the camera imaging principle diagram.
[0055] Figure 4 This is the parking detection effect diagram. DETAILED DESCRIPTION
[0056] The present invention will be further described in detail below in conjunction with the accompanying drawings.
[0057] Reference Figure 1 and Figure 2 , the specific steps of the accurate roadside parking detection method based on a high-position camera with monocular three-dimensional vision of the present invention are described in detail.
[0058] Step 1: Fix the roadside camera posture, install the camera at a height of 3-5 meters, and set the angle to be slightly tilted downward, about 30-45 degrees, perform single-target positioning and distortion correction to ensure the collection of high-quality data.
[0059] The specific implementation steps are:
[0060] Step 1.1: Obtain the camera's internal parameters (focal length f, image center, distortion coefficient, etc.) and external parameters (rotation matrix and translation matrix) through camera calibration.
[0061] Step 1.2: Distortion correction: correct the radial and tangential distortions of the camera that collects the data; use the internal parameters and distortion coefficients obtained in step 1.1 to correct the distortion of the collected image; use the two functions "cv2.getOptimalNewCameraMatrix" and "cv2.initUndistortRectifyMap" in OpenCV to calculate the correction map; apply the correction map and verify the correction effect through visual inspection and accuracy evaluation to determine the quality of the corrected image.
[0062] Step 1.3: Arrange the camera internal and external parameters obtained in step 1.1 into a 3×4 matrix format and save it as a file for model training.
[0063] Step 2: Use a high-position camera to collect raw video, preprocess it and create a vehicle dataset.
[0064] The specific implementation steps are:
[0065] Step 2.1: Use a high-position camera to capture vehicle video, extract each frame of the image, filter out valid images based on image clarity, brightness, and contrast, and save them.
[0066] Step 2.2: Perform preprocessing and annotation of the saved images such as denoising and brightening. The annotated labels are in KITTI format, which contains information such as vehicle 2D coordinates, 3D coordinates, and vehicle direction angles. They are divided into training set, validation set, and test set in a ratio of 6:2:2.
[0067] Step 3: Training the model, inputting the vehicle data set into the 3D object detection network model for training, selecting the optimal model and applying it to the actual data, detecting the vehicle, and outputting the 3D information of the vehicle in space.
[0068] The specific implementation steps are:
[0069] Step 3.1: Use the improved GUPNet 3D target detection algorithm for detection. On the basis of the original algorithm, Mixup3D data enhancement is introduced. By linearly combining two different 3D samples and labels, new training samples are generated to improve the generalization ability and robustness of the model, smooth the decision boundary, and effectively prevent overfitting. In terms of depth prediction, a normalized depth module is introduced to eliminate pitch angle and focal length problems.
[0070] The training model uses CenterNet's 2D target detection and uses the DLA-34 backbone network to extract features and output three types of information, including heatmap, 2D center point offset offset_2d and 2D box size size_2d; heatmap is used to predict the target category and the rough 2D box center point coordinates. Each position in the heatmap corresponds to the probability of a category, and the position with the highest probability is the center point of the target; offset_2d is used to predict the offset of the 2D center point coordinates, which is used to fine-tune the rough center point coordinates obtained from the heatmap; size_2d refers to the width (w) of the 2D bounding box of the detected vehicle. 2d ) and height (h 2d ), and then combine the center point coordinates to determine the 2D bounding box of the vehicle.
[0071] Step 3.2: Construct a coordinate tensor map coord_map to store the position information in the image coordinate system, so as to facilitate the subsequent calculation of the center position of the 2D bounding box. According to coord_map and offset_2d in step 3.1, the center position box2d_center of the 2d box is obtained. According to box2d_center and size_2d in step 3.1, the 2D bounding box map box2d_map is obtained to construct the final vehicle 2D bounding box map.
[0072] Step 3.3: Based on the vehicle 2D bounding box map obtained in step 3.2, use RoIAlign to crop the vehicle ROI features from box2d_maps for further feature processing and coordinate normalization. According to the camera calibration information in step 1.3, normalize the coordinate map to make it consistent at different scales and viewing angles. Connect the normalized coordinate map with the ROI feature map to obtain the final ROI feature map for subsequent 3D target detection and tracking.
[0073] Step 3.4: According to the final ROI feature map obtained in step 3.3, the 3D information of the target is predicted. The prediction includes four branches of 3D information: ① heading information; ② offset_3d of the 3D center on the image plane; ③ size_3d (w 3d ,h 3d ,l 3d );④Vehicle depth depth.
[0074] Branch ①: heading information
[0075] Deflection angle information, the ROI feature outputs the deflection angle information through the heading branch, and the MultiBin strategy is used to divide the 360° angle range into 12 Bins, of which 12 components are the probability of classifying the deflection angle into a certain angle range, and the other 12 components correspond to residuals in different ranges. The vehicle's deflection angle can be obtained by adding the category with the maximum probability and its corresponding residual.
[0076] Branch ②: offset_3d of the 3D center on the image plane
[0077] The purpose is to obtain the 3D position of the vehicle. After the offset_3d branch, the output is the offset of the 3D center on the image plane. The vehicle center estimated in the heatmap plus the offset is used to obtain the projection position of the 3dbox center on the image plane. The camera calibration information can be used to obtain its xy coordinates in 3D space, and the estimated vehicle depth z is added to obtain the 3D position of the vehicle.
[0078] Branch ③: 3D bounding box size_3d
[0079] The final size of the vehicle, size_3d outputs the 3D size information of the vehicle (w 3d ,h 3d ,l 3d ), the final size of the vehicle is obtained by combining the average size of the corresponding category of vehicle information with the size_3d information.
[0080] Branch ④: Vehicle depth (Normlized Depth)
[0081] Since the vehicle-side detection camera is installed on the top of the vehicle, its pitch angle is almost parallel to the ground at 0°, while the roadside cameras have different pitch angles when installed. Therefore, the depth calculation is used to introduce the pitch angle by introducing the ground plane equation, thereby ignoring the focal length and pitch angle function of the roadside camera.
[0082] The specific steps of branch ④ are as follows: The imaging principle of the monocular camera is as follows: Figure 3 As shown, according to the triangle similarity relationship, we can get:
[0083]
[0084] z represents the depth of the detected vehicle, f represents the focal length of the monocular camera obtained in step 1, and h 2d represents the imaging height of the detected vehicle obtained in step 3.1, H 3d ′ represents the height of the center point of the detected vehicle, which is obtained from h in step 3.4. 3d Calculated.
[0085] After geometric calculation, we can get:
[0086] H 3d ′=h3d*(cosθ-sinθ*tanδ)*f
[0087] Substituting into the above formula, we get:
[0088]
[0089] By introducing the ground equation αx+βy+γz+d=0, we get the angle θ, that is, θ=arctan(γ / β), and the angle δ is calculated by projection, that is, Where α is the normal vector component in the x direction, representing the tilt of the plane on the x axis, β is the normal vector component in the y direction, representing the tilt of the plane on the y axis, γ is the normal vector component in the z direction, representing the tilt of the plane on the z axis, d represents the offset between the plane and the origin (0, 0, 0), which determines the position of the plane in three-dimensional space, V p is the y-axis pixel coordinate of point P, C y is the principal point coordinate of the y-axis, so the final depth can be expressed as:
[0090]
[0091] Step 3.5: After training is completed, use the validation set and test set to evaluate the trained model, check the detection accuracy of the model, select the optimal detection model for vehicle detection, and output vehicle detection information.
[0092] Step 4: Design a 3D vehicle tracking algorithm. The vehicle 3D position, deflection angle, and size obtained in step 3 are input into the 3D vehicle tracking algorithm.
[0093] The specific implementation steps are:
[0094] Step 4.1: Input the output of step 3 into the 3D vehicle tracking module and use the Kalman filter to predict the vehicle trajectory.
[0095] Step 4.2: The Hungarian algorithm is used to associate and match the detection results of the latest extracted image frame received with the tracked vehicle in the previous frame.
[0096] Step 4.3: For associated targets, use the Kalman filter to update their states to reflect their positions in the current frame. For unassociated targets, start new trajectory tracking.
[0097] In summary, the present invention provides a precise roadside parking detection method based on a high-position camera with monocular three-dimensional vision. After the roadside camera is calibrated, the method captures the video and intercepts each frame for image preprocessing. The processed images form a data set and are input into the improved GUPNet 3D target detection network for training. After training, the optimal target detection model is selected to detect the vehicle, and the detection result is input into the 3D target tracking module for data association, thereby obtaining reliable vehicle position information.
[0098] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A precise roadside parking detection method based on a high-position camera with monocular three-dimensional vision, characterized in that: The steps include: Step 1: Fix the roadside camera posture, perform camera calibration and distortion correction; Step 2: Use a high-position camera to collect raw video, pre-process it and create a vehicle dataset; Step 3: Input the collected vehicle data set into the 3D target detection network model for training to obtain the target detection model, select the optimal detection model to detect the vehicle, and output the 3D information of the vehicle in space; Step 4: Input the vehicle information obtained in step 3 into the 3D vehicle tracking module to achieve real-time tracking.
2. According to claim 1, a high-position camera based on monocular three-dimensional vision accurate roadside parking detection method is characterized in that: The specific implementation steps of step 1 are: Step 1.1: Obtain the camera's internal and external parameters through camera calibration; The internal parameters are focal length, image center, and distortion coefficient; The external parameters are the rotation matrix and the translation matrix; Step 1.2: Distortion correction: correct the radial and tangential distortions of the camera that collects the data; use the internal parameters and distortion coefficients obtained in step 1.1 to correct the distortion of the collected image; use the "cv2.getOptimalNewCameraMatrix" and "cv2.initUndistortRectifyMap" functions in OpenCV to calculate the correction map; apply the correction map and verify the correction effect through visual inspection and accuracy evaluation to determine the quality of the corrected image; Step 1.3: Arrange the camera internal and external parameters obtained in step 1.1 into a 3×4 matrix format and save it as a file for model training.
3. The method for accurate roadside parking detection based on a high-position camera with monocular three-dimensional vision according to claim 1 is characterized in that: The specific implementation steps of step 2 are: Step 2.1: According to the camera calibration and distortion correction in step 1, ensure the quality of the collected data; use a high-position camera to collect vehicle video, capture each frame of the image and save the valid image; Step 2.2: Perform denoising, brightening preprocessing and annotation on the saved images. The annotation format is KITTI format, where the KITTI label contains the vehicle's two-dimensional bounding box coordinate information, three-dimensional bounding box information, and spatial direction information, etc. The annotated images are divided into training set, verification set and test set.
4. The method for accurate roadside parking detection based on a high-position camera with monocular three-dimensional vision according to claim 1 is characterized in that: The specific implementation steps of step 3 are: Step 3.1: Use the improved GUPNet 3D target detection algorithm for detection. Based on the GUPNet algorithm, Mixup3D data enhancement is introduced to generate new training samples by linearly combining two different 3D samples and labels. In terms of depth prediction, the normalized depth module is introduced to eliminate the influence of pitch angle and focal length through coordinate normalization processing; The training model starts with 2D target detection based on CenterNet. The feature extraction network outputs the feat feature map. Based on the feat feature map, it outputs the heatmap, the 2D center point offset offset_2d, and the size of the 2D box size_2d. The heatmap is used to predict the target category and the rough 2D box center point coordinates; offset_2d is used to predict the offset of the 2D center point coordinates; size_2d refers to the width and height of the 2D bounding box of the detected vehicle. Step 3.2: Construct a coordinate tensor map coord_map to store the position information in the image coordinate system, obtain the center position box2d_center of the 2d box according to coord_map and offset_2d, and obtain the 2D bounding box map box2d_map according to box2d_center and size_2d; Step 3.3: Use RoIAlign to extract ROI features from box2d_maps in step 3.2 to obtain the cropped ROI feature map. According to the camera calibration information in step 1.3, the coordinate map is normalized, and the normalized coordinate map is channel-connected with the cropped ROI feature map to obtain the final ROI feature map. According to the camera calibration information, the internal parameters are used to convert the image coordinates to the camera coordinate system, thereby realizing the derivation of the target 3D information; Step 3.4: Based on the final ROI feature map obtained in step 3.3, predict the 3D information including four branches: ① heading information; ② offset_3d of the 3D center on the image plane; ③ size_3d of the 3D bounding box; ④ depth of the vehicle; Step 3.5: After training is completed, select the optimal detection model for vehicle detection.
5. The method for accurate roadside parking detection based on a high-position camera with monocular three-dimensional vision according to claim 1 is characterized in that: The branch ① is the vehicle deflection angle information. The ROI feature outputs the deflection angle information through the heading branch. The MultiBin strategy is used to divide the 360° angle range into 12 Bins, i.e., 24 components. Among them, 12 components are the probabilities of classifying the deflection angle into a certain angle range, and the other 12 components correspond to the residuals of different ranges. The deflection angle of the vehicle can be obtained by adding the category with the maximum probability and its corresponding residual. The branch ② obtains the 3D position of the vehicle. After the offset_3d branch, the output is the offset of the 3D center on the image plane. The vehicle center estimated in the heatmap plus the offset is used to obtain the projection position of the 3Dbox center on the image plane. The camera calibration information is used to obtain its xy coordinates in the 3D space, and the estimated vehicle depth z is added to obtain the 3D position of the vehicle. The branch ③ obtains the final size of the vehicle, and size_3d outputs the offset of the 3D size of the vehicle. The final size of the vehicle is obtained by adding the average size of the corresponding category of vehicle information to the offset; The branch ④ obtains the depth value depth of the vehicle.
6. The method for accurate roadside parking detection based on a high-position camera with monocular three-dimensional vision according to claim 5 is characterized in that: The specific steps of branch ④ are as follows: The imaging principle of the monocular camera is based on the triangle similarity relationship, and the following is obtained: z represents the depth of the detected vehicle, f represents the focal length of the monocular camera obtained in step 1, and h 2d represents the imaging height of the detected vehicle obtained in step 3.1, H 3d ′ represents the height of the center point of the detected vehicle, which is obtained from h in step 3.
4. 3d Calculated; After geometric calculation, we can get: H 3d ′=h 3d *(cosθ-sinθ*tanδ)*f Substituting into the above formula, we get: By introducing the ground equation αx+βy+γz+d=0, we get the angle θ, that is, θ=arctan(γ / β), and the angle δ is calculated by projection, that is, Where α is the normal vector component in the x direction, representing the tilt of the plane on the x axis, β is the normal vector component in the y direction, representing the tilt of the plane on the y axis, γ is the normal vector component in the z direction, representing the tilt of the plane on the z axis, d represents the offset between the plane and the origin (0, 0, 0), which determines the position of the plane in three-dimensional space, V p is the y-axis pixel coordinate of point P, C y is the principal point coordinate of the y-axis, so the final depth is expressed as:
7. The method for accurate roadside parking detection based on a high-position camera with monocular three-dimensional vision according to claim 1 is characterized in that: The specific implementation steps of step 4 are: Step 4.1: Input the output of step 3 into the 3D vehicle tracking module and use the Kalman filter to predict the vehicle trajectory; Step 4.2: Use the Hungarian algorithm to associate and match the detection result of the latest image frame received and extracted with the tracked vehicle in the previous frame; Step 4.3: For associated targets, use the Kalman filter to update their states to reflect their positions in the current frame. For unassociated targets, start new trajectory tracking.