Method and system for managing roadside parking based on heading angle attitude

Abstract

The application discloses a roadside parking management method and system based on a heading angle posture. The method comprises the following steps: performing size transformation, key point data reconstruction and random erasing data enhancement on a vehicle image of each vehicle in sequence to obtain an enhanced vehicle image; inputting the enhanced vehicle image into a backbone network to obtain a backbone feature map; inputting the backbone feature map into a feature aggregation network to obtain an aggregated feature map; inputting the aggregated feature map into a key point prediction network to obtain a predicted posture center point and a plurality of predicted offsets of each vehicle; inputting the aggregated feature map into a vehicle heading angle regression network to obtain a predicted heading angle sine value and a predicted heading angle cosine value of each vehicle; performing key point prediction on each vehicle image extracted from a to-be-tested traffic scene image according to a trained key point detection model to obtain a posture center point, a plurality of offsets, a heading angle sine value and a heading angle cosine value of each to-be-tested vehicle, and performing roadside parking management.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a roadside parking management method and system based on heading angle attitude. Background Technology

[0002] In recent years, high-position video technology has developed rapidly. By installing high-position video cameras on the roadside, vehicle parking can be judged and managed. Through vehicle detection and posture analysis, and then comparing the data with known parking space locations, the system can determine the vehicle's status, such as whether the vehicle is within a parking space, whether it is parked over the line, or whether it is parked in a no-parking zone. Determining whether a vehicle is illegally parked based on its posture has a positive impact on various aspects of urban traffic management and road safety.

[0003] Traditional methods use two-dimensional vehicle bounding boxes to estimate vehicle pose for roadside parking management. However, due to the influence of surrounding traffic conditions, vehicles may be obscured. Relying solely on two-dimensional vehicle bounding boxes to determine vehicle pose cannot accurately determine vehicle location, resulting in low efficiency in roadside parking management. Summary of the Invention

[0004] The purpose of this application is to solve the technical problem of low efficiency in roadside parking management caused by the inability of traditional methods to accurately determine vehicle positions. To achieve the above objective, this application provides a roadside parking management method and system based on heading angle attitude.

[0005] This application provides a roadside parking management method based on heading angle attitude, including:

[0006] A modeling and simulation traffic scene image dataset is obtained. The modeling and simulation traffic scene image dataset includes multiple traffic scene images. Each traffic scene image is labeled with the real information of the two-dimensional detection box of each vehicle, multiple real attitude key points corresponding to the three-dimensional bounding box of each vehicle, the real attitude center point, multiple real offsets of the multiple real attitude key points to the real attitude center point, the real heading angle of each vehicle, the sine value of the real heading angle, and the cosine value of the real heading angle.

[0007] Based on the real information of the two-dimensional detection box, each vehicle in each traffic scene image is extracted to obtain a vehicle image of each vehicle;

[0008] The vehicle images are resized and reconstructed using key point data to obtain a reconstructed vehicle image for each vehicle. The reconstructed vehicle images are then augmented with random erasure data to obtain an augmented vehicle image for each vehicle.

[0009] The enhanced vehicle image is input into the backbone network of the key point detection model for feature extraction to obtain a backbone feature map dataset.

[0010] Each of the backbone feature maps in the backbone feature map dataset is input into the feature aggregation network of the key point detection model for feature fusion to obtain an aggregated feature map dataset.

[0011] Each of the aggregated feature maps in the aggregated feature map dataset is input into the key point prediction network of the key point detection model to predict the location, thereby obtaining the predicted pose center point and multiple predicted offsets for each vehicle.

[0012] Each aggregated feature map in the aggregated feature map dataset is input into the vehicle heading angle regression network of the key point detection model to predict the heading angle, thereby obtaining the sine and cosine values ​​of the predicted heading angle for each vehicle, and obtaining the predicted heading angle for each vehicle based on the sine and cosine values ​​of the predicted heading angle.

[0013] The loss function of the keypoint detection model is constructed based on the predicted heading angle, the predicted attitude center point, the multiple predicted offsets, the true heading angle, the true attitude center point, and the multiple true offsets. The keypoint detection model is then trained and optimized based on the loss function to obtain the trained keypoint detection model.

[0014] Acquire a traffic scene image to be tested, and perform key point prediction on each vehicle image extracted from the traffic scene image to be tested based on the trained key point detection model to obtain the attitude center point, multiple offsets, sine value of heading angle and cosine value of heading angle for each vehicle to be tested.

[0015] Based on the attitude center point and the multiple offsets, multiple attitude key points of each vehicle under test are obtained. Based on the sine and cosine values ​​of the heading angle, the heading angle of each vehicle under test is obtained. Roadside parking management is then performed based on the multiple attitude key points and the heading angle.

[0016] In one embodiment, the step of performing size transformation and key point data reconstruction on the vehicle images to obtain a reconstructed vehicle image for each vehicle, and then performing random erasure data enhancement on the reconstructed vehicle images to obtain an enhanced vehicle image for each vehicle, includes:

[0017] The original length of the vehicle image is set to 256 pixels, and the original width of the vehicle image is transformed according to the length transformation ratio to obtain the new width of the vehicle image.

[0018] The new width of the vehicle image is padded with 0 pixels to 256 pixels to obtain a size-transformed image;

[0019] Based on the transformation ratio between the size of the size-transformed image and the original size of the vehicle image, the multiple real pose key points, the real pose center point, and the multiple real offsets are transformed to obtain the reconstructed vehicle image.

[0020] In one embodiment, the step of constructing a loss function for the keypoint detection model based on the predicted heading angle, the predicted attitude center point, the plurality of predicted offsets, the true heading angle, the true attitude center point, and the plurality of true offsets, and training and optimizing the keypoint detection model based on the loss function to obtain a trained keypoint detection model includes:

[0021] Based on the predicted heading angle and the actual heading angle of each vehicle, a regression loss function for the vehicle heading angle is constructed.

[0022] Based on the predicted attitude center point and the true attitude center point of each vehicle, a regression loss function for the vehicle attitude center point is constructed.

[0023] Based on the multiple predicted offsets and multiple true offsets of each vehicle, a regression loss function for the offset of the vehicle attitude key points is constructed.

[0024] The loss function of the key point detection model is constructed based on the regression loss function of the vehicle heading angle, the regression loss function of the vehicle attitude center point, and the regression loss function of the offset of the vehicle attitude key points.

[0025] In one embodiment, acquiring the traffic scene image to be tested, and performing keypoint prediction on each vehicle image extracted from the traffic scene image to be tested based on the trained keypoint detection model, to obtain the attitude center point, multiple offsets, sine value of the heading angle, and cosine value of the heading angle for each vehicle to be tested, includes:

[0026] Vehicle target detection is performed on the traffic scene image to be tested using a vehicle target detection algorithm to obtain two-dimensional detection box information for each vehicle to be tested.

[0027] The traffic scene image to be tested is extracted based on the two-dimensional detection box information to obtain the vehicle image of each vehicle to be tested;

[0028] The images of the vehicles under test are resized and reconstructed using key point data to obtain a reconstructed vehicle image for each vehicle under test. Random erasure data enhancement is then performed on the reconstructed vehicle images to obtain an enhanced vehicle image for each vehicle under test.

[0029] The image of the enhanced vehicle to be tested is input into the trained key point detection model, and the attitude center point, the multiple offsets, the sine value of the heading angle and the cosine value of the heading angle of each vehicle to be tested are output.

[0030] In one embodiment, obtaining multiple attitude key points for each vehicle under test based on the attitude center point and the multiple offsets, obtaining the heading angle for each vehicle under test based on the sine and cosine values ​​of the heading angle, and performing roadside parking management based on the multiple attitude key points and the heading angle, includes:

[0031] Perform world coordinate transformation on multiple attitude key points of each vehicle under test to obtain the position of each vehicle under test;

[0032] Roadside parking management is performed based on the position of the vehicle under test and the heading angle.

[0033] In one embodiment, this application provides a roadside parking management system based on heading angle attitude, comprising:

[0034] The data acquisition module is used to acquire a modeling and simulation traffic scene image dataset. The modeling and simulation traffic scene image dataset includes multiple traffic scene images. Each traffic scene image is labeled with the real information of the two-dimensional detection box of each vehicle, multiple real attitude key points corresponding to the three-dimensional bounding box of each vehicle, the real attitude center point, multiple real offsets of the multiple real attitude key points to the real attitude center point, the real heading angle of each vehicle, the sine value of the real heading angle, and the cosine value of the real heading angle.

[0035] The image extraction module is used to extract each vehicle in each traffic scene image based on the real information of the two-dimensional detection box, and obtain a vehicle image of each vehicle.

[0036] The data augmentation module is used to perform size transformation and key point data reconstruction on the vehicle images to obtain a reconstructed vehicle image for each vehicle, and to perform random erasure data augmentation on the reconstructed vehicle images to obtain an enhanced vehicle image for each vehicle.

[0037] The backbone network module is used to input the enhanced vehicle image into the backbone network of the key point detection model for feature extraction to obtain a backbone feature map dataset.

[0038] The feature aggregation network module is used to input each of the backbone feature maps in the backbone feature map dataset into the feature aggregation network of the key point detection model for feature fusion to obtain an aggregated feature map dataset.

[0039] The key point prediction network module is used to input each of the aggregated feature maps in the aggregated feature map dataset into the key point prediction network of the key point detection model to predict the position, thereby obtaining the predicted pose center point and multiple predicted offsets for each vehicle.

[0040] The heading angle regression network module is used to input each of the aggregated feature maps in the aggregated feature map dataset into the vehicle heading angle regression network of the key point detection model to predict the heading angle, obtain the sine value and cosine value of the predicted heading angle of each vehicle, and obtain the predicted heading angle of each vehicle based on the sine value and cosine value of the predicted heading angle.

[0041] The model training module is used to construct a loss function for the key point detection model based on the predicted heading angle, the predicted attitude center point, the multiple predicted offsets, the true heading angle, the true attitude center point, and the multiple true offsets, and to train and optimize the key point detection model based on the loss function to obtain a trained key point detection model.

[0042] The detection module is used to acquire the traffic scene image to be tested, and to perform key point prediction on each vehicle image extracted from the traffic scene image to be tested according to the trained key point detection model, so as to obtain the attitude center point, multiple offsets, sine value of heading angle and cosine value of heading angle for each vehicle to be tested.

[0043] The parking management module is used to obtain multiple attitude key points of each vehicle under test based on the attitude center point and the multiple offsets, obtain the heading angle of each vehicle under test based on the sine and cosine values ​​of the heading angle, and perform roadside parking management based on the multiple attitude key points and the heading angle.

[0044] In one embodiment, the data enhancement module includes:

[0045] The size transformation module is used to set the original length of the vehicle image to 256 pixels and transform the original width of the vehicle image according to the length transformation ratio to obtain the new width of the vehicle image.

[0046] A pixel filling module is used to fill the new width of the vehicle image from 0 pixels to 256 pixels to obtain a size-transformed image;

[0047] The key point data reconstruction module is used to transform the plurality of real pose key points, the real pose center point, and the plurality of real offsets according to the transformation ratio between the size of the size-transformed image and the original size of the vehicle image, so as to obtain the reconstructed vehicle image.

[0048] In one embodiment, the model training module includes:

[0049] The first regression loss function module is used to construct a regression loss function for the vehicle heading angle based on the predicted heading angle and the actual heading angle of each vehicle.

[0050] The second regression loss function module is used to construct a regression loss function for the vehicle attitude center point based on the predicted attitude center point and the true attitude center point of each vehicle.

[0051] The third regression loss function module is used to construct a regression loss function for the offset of the vehicle attitude key points based on the multiple predicted offsets and the multiple true offsets of each vehicle.

[0052] The total loss function module is used to construct the loss function of the key point detection model based on the regression loss function of the vehicle heading angle, the regression loss function of the vehicle attitude center point, and the regression loss function of the offset of the vehicle attitude key points.

[0053] In one embodiment, the detection module includes:

[0054] The two-dimensional detection box information acquisition module is used to perform vehicle target detection on the traffic scene image to be tested according to the vehicle target detection algorithm, and obtain the two-dimensional detection box information of each vehicle to be tested.

[0055] The vehicle image acquisition module is used to extract the traffic scene image to be tested based on the two-dimensional detection box information to obtain the vehicle image to be tested for each vehicle to be tested.

[0056] The vehicle image acquisition module is used to perform size transformation and key point data reconstruction on the vehicle image to obtain a reconstructed vehicle image for each vehicle to be tested, and to perform random erasure data enhancement on the reconstructed vehicle image to obtain a vehicle image to be tested for each vehicle to be tested.

[0057] The vehicle attitude acquisition module is used to input the enhanced vehicle image to the trained key point detection model and output the attitude center point, multiple offsets, sine value of the heading angle and cosine value of the heading angle for each vehicle under test.

[0058] In one embodiment, the parking management module includes:

[0059] The coordinate transformation module is used to perform world coordinate transformation on multiple attitude key points of each of the vehicles under test to obtain the position of each vehicle under test;

[0060] The management module is used to manage roadside parking based on the location of the vehicle under test and the heading angle.

[0061] The aforementioned roadside parking management method and system based on heading angle attitude generates virtual data and its labeling information of traffic scenes through simulation modeling technology, which can be applied to real roadside parking scene images. During the virtual data generation process, accurate vehicle 3D bounding boxes can be automatically obtained based on digital vehicle models, eliminating the need for manual annotation and avoiding errors caused by manual annotation. Furthermore, different vehicle models can be used to generate different parking scene image data, and different camera angles can be adjusted to obtain data from different perspectives, greatly reducing data collection time and labor costs.

[0062] Data enhancement through size transformation, keypoint data reconstruction, and random erasure improves the model's robustness in occluded scenarios and prevents overfitting. Enhanced vehicle images, carrying labeled information, are sequentially input into the backbone network, feature aggregation network, keypoint prediction network, and vehicle heading angle regression network for attitude keypoint and heading angle regression prediction, improving the accuracy of vehicle attitude prediction for each vehicle. Each test vehicle image is input into the trained keypoint detection model for prediction, obtaining the corresponding attitude center point, multiple offsets, heading angle sine and cosine values, thus obtaining multiple attitude keypoints and heading angles for each vehicle in the image coordinate system. By converting these multiple attitude keypoints from the image coordinate system to multiple attitude keypoints in the world coordinate system, and combining this with the vehicle's heading angle constraint, the parking position and attitude of the vehicle can be determined by comparing them with the parking space location in the same world coordinate system. This enables vehicle parking judgment and management, identifying whether vehicles are illegally parked or parked beyond the lines. By regressing eight key points of the vehicle and predicting the vehicle's heading angle, the vehicle's attitude can be determined. Even when the vehicle is severely obscured, the vehicle's attitude can still be predicted based on the visible key points, demonstrating strong robustness. Attached Figure Description

[0063] Figure 1 This is a flowchart illustrating the steps of the roadside parking management method based on heading angle attitude provided in this application.

[0064] Figure 2 This is a schematic diagram of eight key points in one embodiment provided in this application.

[0065] Figure 3 This is a schematic diagram of the roadside parking management system based on heading angle attitude provided in this application. Detailed Implementation

[0066] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0067] Please see Figure 1 This application provides a roadside parking management method based on heading angle attitude, including:

[0068] S10, Obtain the modeling and simulation traffic scene image dataset. The modeling and simulation traffic scene image dataset includes multiple traffic scene images. Each traffic scene image is labeled with the real information of the two-dimensional detection box of each vehicle, multiple real pose key points corresponding to the three-dimensional bounding box of each vehicle, the real pose center point, multiple real offsets of multiple real pose key points to the real pose center point, the real heading angle of each vehicle, the sine value of the real heading angle, and the cosine value of the real heading angle.

[0069] S20, based on the real information of the two-dimensional detection box, extract each vehicle in each traffic scene image to obtain the vehicle image of each vehicle;

[0070] S30, the vehicle images are resized and reconstructed using key point data to obtain a reconstructed vehicle image for each vehicle, and the reconstructed vehicle images are then augmented with random erasure data to obtain an augmented vehicle image for each vehicle.

[0071] S40, the enhanced vehicle image is input into the backbone network of the key point detection model for feature extraction to obtain the backbone feature map dataset;

[0072] S50, input each backbone feature map in the backbone feature map dataset into the feature aggregation network of the key point detection model for feature fusion to obtain the aggregated feature map dataset;

[0073] S60, input each aggregated feature map in the aggregated feature map dataset into the key point prediction network of the key point detection model to predict the location, and obtain the predicted pose center point and multiple predicted offsets for each vehicle.

[0074] S70, input each aggregated feature map in the aggregated feature map dataset into the vehicle heading angle regression network of the key point detection model to predict the heading angle, obtain the sine and cosine values ​​of the predicted heading angle for each vehicle, and obtain the predicted heading angle for each vehicle based on the sine and cosine values ​​of the predicted heading angle.

[0075] S80: Construct a loss function for the keypoint detection model based on the predicted heading angle, predicted attitude center point, multiple predicted offsets, true heading angle, true attitude center point, and multiple true offsets. Then, train and optimize the keypoint detection model based on the loss function to obtain the trained keypoint detection model.

[0076] S90, acquire the traffic scene image to be tested, and predict the key points of each vehicle image extracted from the traffic scene image to be tested according to the trained key point detection model, and obtain the attitude center point, multiple offsets, sine value of heading angle and cosine value of heading angle of each vehicle to be tested.

[0077] S100 obtains multiple attitude key points for each vehicle under test based on the attitude center point and multiple offsets, obtains the heading angle for each vehicle under test based on the sine and cosine values ​​of the heading angle, and performs roadside parking management based on multiple attitude key points and heading angles.

[0078] In this embodiment, in S10, a large number of virtual data images of traffic scenes are obtained using modeling and simulation technology. Modeling and simulation technology includes, but is not limited to, using game engines such as Unity and Unreal, and simulation software such as Carla and Arisim to model roadside parking scenes, obtaining virtual roadside parking scene image data. The virtual traffic scene image data obtained through modeling has the same spatial layout as the real traffic scene image data, such as the same camera shooting height and angle, and includes vehicle parking behaviors including the process of a vehicle entering a parking space, the process of a vehicle in a parking space, and the process of a vehicle leaving a parking space, as well as targets such as parking lines and roadside green belts. During the modeling process, when constructing the vehicle model, the true information of the two-dimensional detection box of each vehicle and multiple true posture key points corresponding to the three-dimensional bounding box of each vehicle can be obtained. Based on the true information of the two-dimensional detection box of each vehicle, the coordinate position of the corresponding true posture center point can be obtained, and thus, multiple true offsets from multiple true posture key points to the true posture center point can be obtained. The heading angle can be understood as the vehicle heading angle, the angle between the vehicle's driving direction and the horizontal axis of the coordinate system, with an angle range of [-pi,pi].

[0079] The ground truth information of each vehicle's 2D bounding box includes the x and y coordinates of the top-left and bottom-right corners of the vehicle's bounding box, which are (x1, y1) and (x2, y2) respectively. The ground truth pose center point represents the coordinates of the center point of the 2D bounding box, which can be obtained from the coordinate information of the 2D bounding box. The 3D bounding box information of each vehicle includes the type of ground truth pose keypoints, the coordinates of the ground truth pose keypoints, and the visibility attribute of the ground truth pose keypoints. Multiple ground truth pose keypoints represent the information of the 3D bounding box, representing the 3D contour information of the vehicle.

[0080] Please see Figure 2 In one embodiment, each vehicle corresponds to eight key points, representing the eight vertices of the three-dimensional bounding box. These are four key points where the vehicle is in contact with the ground and four key points where the roof is in the air. Alternatively, based on the vehicle's orientation, the key point on the left side of the vehicle's front, in contact with the ground, is designated as point 1 (bottom-front-1). After clockwise rotation, the other three ground-contact points are designated as bottom-front-2, bottom-back-3, and bottom-back-4. The key point on the left side of the roof, in the front direction, is designated as point 5 (top-front-5). After clockwise rotation, the other three roof-mounted key points are designated as top-front-6, top-back-7, and top-back-8. These eight vertices of the three-dimensional bounding box describing the vehicle's posture represent eight types of key points. The coordinates of the true posture key points are the x and y coordinates in the image coordinate system. The attributes of the true posture key points include visible and invisible, with visible points designated as 1 and invisible points designated as 0. Invisible points can be understood as pose key points that are occluded by the vehicle body in the captured vehicle image, for example... Figure 2 If midpoints 1 and 4 are occluded, they have the attribute of being invisible. Conversely, visible points can be understood as vehicle bodies occluded in vehicle images where pose keypoints have not been captured, for example... Figure 2 If the midpoints 2, 3, 5, 6, 7, and 8 are not obscured, then they have the property of being visible.

[0081] The multiple real offsets from the real pose key points to the real pose center point are compared with the position information of the 2D detection box and the 3D bounding box, making the specific position of the vehicle more accurate and more precise in locating the specific position of each vehicle.

[0082] The label data consists of multiple ground truth bounding boxes, multiple ground truth pose keypoints, multiple ground truth pose center points, multiple ground truth offsets from ground truth pose keypoints to ground truth pose center points, the ground truth heading angle of each vehicle, the sine value of the ground truth heading angle, and the cosine value of the ground truth heading angle. Each traffic scene image containing the label data is used as input to the keypoint detection model for model training and optimization.

[0083] In S20, the ground truth information of each 2D detection box represents the length and width of the vehicle. Using the ground truth information of each vehicle's 2D detection box, each vehicle can be extracted from the traffic scene image, resulting in a vehicle image corresponding to each vehicle. One vehicle corresponds to one vehicle image.

[0084] In S30, the original size of each vehicle image is uniformly adjusted to 256×256, achieving a size transformation. Simultaneously, based on the relationship between the original size and 256×256, coordinate transformations are performed on multiple true pose keypoints, multiple true pose center points, and multiple true offsets from the true pose keypoints to the true pose center points of the vehicle image, achieving keypoint data reconstruction. This proportional transformation yields the reconstructed vehicle image for each vehicle. Data augmentation through random erasure of the reconstructed vehicle images enhances the model's robustness in scenarios with vehicle occlusion, pedestrian occlusion, or vegetation occlusion, preventing overfitting.

[0085] In S40, the keypoint detection model consists of a backbone network, a feature aggregation network, a keypoint prediction network, and a vehicle heading angle regression network. The backbone network is used for image feature extraction. The input image size is H×W×C, where H and W represent the image height and width, respectively, and C represents the number of channels, indicating an RGB three-channel image. Backbone networks include, but are not limited to, ResNet, VGG, and MobileNet.

[0086] In S50, the feature aggregation network is used to aggregate high- and low-level features extracted from different layers of the backbone network, providing more feature representations for subsequent keypoint prediction. Since high-level semantic features are closer to the network output but have lower resolution, while high-resolution features are closer to the input but have fewer semantic features, aggregating features between different layers of the backbone network can achieve the fusion of high- and low-level features, thereby improving the detection accuracy of subsequent keypoint detection tasks.

[0087] In S60, the input to the keypoint prediction network is a feature map after feature aggregation, with dimensions (H / 4)×(W / 4)×C, where C = 256. The keypoint prediction network is used for center point and keypoint feature classification and location localization, outputting the predicted pose center point and predicted offset. The predicted offset corresponds to the true offset, representing the predicted offset from the predicted pose keypoint to the predicted pose center point.

[0088] In S70, the vehicle heading angle refers to the angle between the vehicle's direction of travel and the horizontal axis of the coordinate system, with the angle ranging from [-pi, pi]. Angle regression is performed by calculating the sine and cosine values ​​of the vehicle heading angle. The vehicle heading angle regression network specifically consists of multiple convolutional neural network layers, activation function layers, and normalization layers. The input to the vehicle heading angle regression network is the aggregated feature map, and the output is the predicted sine and cosine values ​​of the heading angle, thereby determining the predicted heading angle for each vehicle.

[0089] In S80, the offsets of the attitude keypoints that make up the vehicle's 3D bounding box relative to the attitude center point are used as the learning metric for the keypoint detection model. The attitude center point and the heading angle of each vehicle are also used as regression objects. By incorporating the attitude center point, multiple attitude keypoints, and heading angle as constraints, the regression difficulty of the keypoint detection task can be reduced, and the detection accuracy can be improved. Model optimization and parameter updates are achieved by constructing a loss function for the keypoint detection model using predicted heading angle, predicted attitude center point, multiple predicted offsets, true heading angle, true attitude center point, and multiple true offsets.

[0090] In the S90, the traffic scene image to be tested includes multiple images of vehicles to be tested. Extracting each vehicle image from the traffic scene image yields the corresponding vehicle image for each vehicle. Inputting these vehicle images into a trained keypoint detection model for keypoint prediction allows for the acquisition of each vehicle's attitude center point, multiple offsets, sine and cosine values ​​of the heading angle, thus providing a comprehensive and accurate prediction of the specific position of each vehicle.

[0091] In S100, the offset is the numerical value of the offset of the attitude key point relative to the attitude center point. Based on the attitude center point and multiple offsets, multiple attitude key points of each vehicle under test can be obtained. The heading angle of each vehicle under test is obtained based on the sine and cosine values ​​of the heading angle.

[0092] The roadside parking management method based on heading angle attitude provided in this application generates virtual data and its labeling information of traffic scenes through simulation modeling technology, which can be applied to real roadside parking scene images. During the virtual data generation process, accurate vehicle 3D bounding boxes can be automatically obtained based on digital vehicle models, eliminating the need for manual annotation and avoiding errors caused by manual annotation. Furthermore, different vehicle models can be used to generate different parking scene image data, and different camera angles can be adjusted to obtain data from different perspectives, greatly reducing data collection time and labor costs. Simultaneously, it can simulate data from different vehicle models, different perspectives, different occlusion conditions, and different parking situations, greatly enriching the data scenarios and enabling effective responses to different situations in real-world scenarios.

[0093] Data enhancement through size transformation, keypoint data reconstruction, and random erasure improves the model's robustness in occluded scenarios and prevents overfitting. Enhanced vehicle images, carrying labeled information, are sequentially input into the backbone network, feature aggregation network, keypoint prediction network, and vehicle heading angle regression network for attitude keypoint and heading angle regression prediction, improving the accuracy of vehicle attitude prediction for each vehicle. Each test vehicle image is input into the trained keypoint detection model for prediction, obtaining the corresponding attitude center point, multiple offsets, heading angle sine and cosine values, thus obtaining multiple attitude keypoints and heading angles for each vehicle in the image coordinate system. By converting these multiple attitude keypoints from the image coordinate system to multiple attitude keypoints in the world coordinate system, and combining this with the vehicle's heading angle constraint, the parking position and attitude of the vehicle can be determined by comparing them with the parking space location in the same world coordinate system. This enables vehicle parking judgment and management, identifying whether vehicles are illegally parked or parked beyond the lines. By regressing eight key points of the vehicle and predicting the vehicle's heading angle, the vehicle's attitude can be determined. Even when the vehicle is severely obscured, the vehicle's attitude can still be predicted based on the visible key points, demonstrating strong robustness.

[0094] In one embodiment, S10, a modeling and simulation traffic scene image dataset is obtained. The modeling and simulation traffic scene image dataset includes multiple traffic scene images. Each traffic scene image is labeled with the true information of the two-dimensional detection box of each vehicle, multiple true pose key points corresponding to the three-dimensional bounding box of each vehicle, the true pose center point, multiple true offsets of the multiple true pose key points from the true pose center point, the true heading angle of each vehicle, the sine value of the true heading angle, and the cosine value of the true heading angle, including:

[0095] S110: Based on the true upper left and lower right corner coordinates of each vehicle, obtain the true attitude center point of the vehicle.

[0096] S120 calculates multiple true offsets from multiple true vehicle pose keypoints to the true vehicle pose center point for each vehicle.

[0097] In this embodiment, the true information of the two-dimensional detection box for each vehicle includes the true coordinates of the top-left corner and the true coordinates of the bottom-right corner of the rectangle for each vehicle. The vehicle attitude center point is C. car The dimension is 1*2, where 2 represents the x-coordinate and y-coordinate. The coordinates of the true attitude center point of each vehicle are calculated as follows:

[0098] x_center = (x1 + x2) / 2;

[0099] y_center = (y1 + y2) / 2.

[0100] The position C of the true attitude center point of each vehicle car Let (x_center, y_center) be a 1*2 dimension.

[0101] The joint offset of each vehicle's true attitude keypoint from its true attitude center point is denoted as J, with dimensions (W / R*H / R*1*2). In one embodiment, L1 distance is used to calculate the offset of each vehicle's true attitude keypoint from its true attitude center point.

[0102] The formula for calculating the true offset from the true pose keypoint to the true pose center point is:

[0103] I1 and I2 represent two vectors, one for each true pose keypoint and the other for the true pose center point, respectively. p represents the dimension of the two vectors, i.e., 1*2 dimensions.

[0104] The true pose keypoint can be represented as the position of the pose center point plus the true offset relative to the center point:

[0105] l k = (x_center, y_center) + J j

[0106] j∈1,2…k.

[0107] In one embodiment, S30, the vehicle images are resized and reconstructed using key point data to obtain a reconstructed vehicle image for each vehicle, and random erasure data augmentation is performed on the reconstructed vehicle images to obtain an enhanced vehicle image for each vehicle, including:

[0108] S310 sets the original length of the vehicle image to 256 pixels and transforms the original width of the vehicle image according to the length transformation ratio to obtain the new width of the vehicle image.

[0109] S320: The new width of the vehicle image is padded with 0 pixels to 256 pixels to obtain a size-transformed image;

[0110] S330 transforms multiple real pose key points, real pose center points, and multiple real offsets according to the transformation ratio between the size of the size-transformed image and the original size of the vehicle image, thereby obtaining a reconstructed vehicle image.

[0111] In this embodiment, each vehicle image is extracted from the traffic scene image based on the actual information of the two-dimensional detection box of each vehicle. The width and height information of each vehicle image can be calculated, which can also be understood as length and width information. The original length of the vehicle image is set to a fixed value of 256 pixels. There is a transformation ratio between the original length and 256 pixels. According to the length change ratio, the original width of the vehicle image is transformed to form a new width. When the new width is less than 256 pixels, the insufficient part is filled with pixel values ​​of 0, and the final size of each vehicle image is 256×256. According to the transformation ratio between the vehicle image and the size-transformed image, corresponding transformations are also performed between multiple real pose key points, real pose center points, and multiple real offsets to obtain the key point coordinate information based on the extracted and resized vehicle image, and obtain the reconstructed vehicle image after key point data reconstruction.

[0112] In one embodiment, the original image size of each vehicle image is 500×300 (length×width). If the length is changed from 500 to 256, the width is changed from 300 to 300×(256 / 500)=153.6. Since the width 153.6 (which can be rounded to the nearest integer 154) is less than 256, the portion between 154 and 256 is padded with 0 pixels, resulting in a transformed image size of 256×256. The original coordinates of one keypoint among multiple real pose keypoints are (200, 100). After reconstructing the vehicle image, the new coordinates of this keypoint are (200 / 500×256=102.4, 100 / 300×256=85.3), which, after rounding, becomes (102, 85). The new coordinates of the remaining real pose keypoints are calculated in the same way. Furthermore, based on the transformed new coordinates, the real pose center point and multiple real offsets can be obtained, thus obtaining the reconstructed vehicle image.

[0113] In one embodiment, S40, the enhanced vehicle image is input into the backbone network of the keypoint detection model for feature extraction to obtain a backbone feature map dataset, including:

[0114] S410 inputs the enhanced vehicle image of each vehicle into the convolutional layer and outputs the first feature map dataset;

[0115] S420, input each first feature map in the first feature map dataset to the normalization layer, and output the second feature map dataset;

[0116] S430, each second feature map in the second feature map dataset is input to the activation function layer, and the third feature map dataset is output, which includes multiple third feature maps.

[0117] In this embodiment, the backbone network employs a convolutional combination of convolutional layers, normalization layers, and activation function layers, performing stacked operations on multiple convolutional combination layers, with downsampling performed once in each convolutional combination operation. The downsampling factor is R. Normalization layers include, but are not limited to, instance normalization layers and adaptive instance normalization layers. Nonlinear activation layers include, but are not limited to, ReLU and Leaky ReLU nonlinear activation functions. In one embodiment, downsampling is performed twice during the backbone network feature extraction stage, i.e., R = 4, to ensure accuracy during subsequent keypoint detection. The input image size of the feature extraction network is H × W × C. After three downsampling processes, the feature map size becomes (H / 4) × (W / 4) × C, where C = 256.

[0118] In one embodiment, S50, each backbone feature map in the backbone feature map dataset is input into the feature aggregation network of the keypoint detection model for feature fusion to obtain an aggregated feature map dataset, including:

[0119] S510, the first feature map, the second feature map and the third feature map are input into the feature aggregation network for feature fusion to obtain the aggregated feature map of the aggregated feature map dataset.

[0120] In this embodiment, a feature aggregation network is used to fuse high- and low-level features extracted from different layers in the backbone network, forming an aggregated feature map. By analyzing the aggregated feature map dataset, the detection accuracy of subsequent keypoint detection tasks is improved.

[0121] In one embodiment, S80, a loss function for the keypoint detection model is constructed based on the predicted heading angle, predicted attitude center point, multiple predicted offsets, the true heading angle, the true attitude center point, and multiple true offsets. The keypoint detection model is then trained and optimized based on the loss function to obtain a trained keypoint detection model, including:

[0122] S810: Based on the predicted and actual heading angles of each vehicle, a regression loss function for the vehicle heading angle is constructed.

[0123] S820 constructs a regression loss function for the vehicle's attitude center point based on the predicted attitude center point and the true attitude center point for each vehicle.

[0124] S830 constructs a regression loss function for the offset of vehicle attitude key points based on multiple predicted offsets and multiple true offsets for each vehicle.

[0125] S840 constructs the loss function of the key point detection model based on the regression loss function of the vehicle heading angle, the regression loss function of the vehicle attitude center point, and the regression loss function of the offset of the vehicle attitude key points.

[0126] In this embodiment, model optimization and parameter updates are performed by constructing a loss function for model training. The loss function of the key point detection model is a multi-task loss function, which includes three parts: a regression loss function for the vehicle heading angle, a regression loss function for the vehicle attitude center point, and a regression loss function for the offset of the vehicle attitude key points.

[0127] The loss function of the keypoint detection model is:

[0128] L=α1L reg +α2L ha +α3L offset ;

[0129] Among them, L reg L ha L offset The regression loss functions for the vehicle attitude center point, the vehicle heading angle, and the offset of the vehicle attitude key points are respectively represented by α1, α2, and α3, which are weighting coefficients. In one embodiment, they can be set to 1.

[0130] In one embodiment, L reg L ha L offest The regression loss function can be the L1 Loss (mean absolute error) function. The L1 Loss (mean absolute error) function can be expressed as:

[0131]

[0132] Here, y1 and y2 represent the two vectors to be calculated, namely the predicted value and the true value, respectively, and m represents the total amount of data. The ultimate goal of the regression loss function is to minimize the absolute difference between the total predicted value and the true value.

[0133] In one embodiment, S90, an image of the traffic scene to be tested is acquired. Based on a trained keypoint detection model, keypoint prediction is performed on each vehicle image extracted from the traffic scene image to be tested, obtaining the attitude center point, multiple offsets, sine value of the heading angle, and cosine value of the heading angle for each vehicle, including:

[0134] S910 performs vehicle target detection on the traffic scene image to be tested according to the vehicle target detection algorithm, and obtains the two-dimensional detection box information of each vehicle to be tested;

[0135] S920 extracts the traffic scene image to be tested based on the two-dimensional detection box information to obtain the image of each vehicle to be tested.

[0136] S930 performs size transformation and key point data reconstruction on the image of the vehicle under test to obtain the reconstructed vehicle image of each vehicle under test, and performs random erasure data augmentation on the reconstructed vehicle image to obtain the augmented vehicle image of each vehicle under test.

[0137] S940 inputs the image of the enhanced vehicle to be tested into the trained key point detection model and outputs the attitude center point, multiple offsets, sine value of the heading angle and cosine value of the heading angle for each vehicle to be tested.

[0138] In this embodiment, the vehicle target detection algorithm can detect each vehicle in the traffic scene image under test, obtain the two-dimensional detection box information of each vehicle, and extract the corresponding vehicle image from the traffic scene image under test based on the two-dimensional detection box information. The description of step S930 is the same as that of S30, except that the object of S930 is the vehicle image under test, which undergoes size transformation, key point data reconstruction, and random erasure to obtain the enhanced vehicle image of each vehicle under test. In one embodiment, the vehicle target detection algorithm includes, but is not limited to, using mainstream target detection algorithms such as YOLO and SSD.

[0139] Therefore, the image of the enhanced vehicle to be tested is input into the trained key point detection model to obtain the attitude center point, multiple offsets, sine value of the heading angle, and cosine value of the heading angle for each vehicle under test. Based on the key point calculation formula in the above embodiment, multiple attitude key points of each vehicle under test are obtained according to the attitude center point and multiple offsets. The heading angle of each vehicle under test is obtained according to the sine value and cosine value of the heading angle.

[0140] Therefore, the roadside parking management method based on heading angle attitude provided in this application is based on a top-down key point detection approach. The first-stage target detection result is obtained through a vehicle target detection algorithm, and the second-stage key point detection task is mainly learned. In this process, by utilizing the regression of vehicle attitude key points and the regression of vehicle heading angle, as well as adding a data augmentation method of random erasure, the accuracy of vehicle key point detection is improved, and it also has good detection results in scenarios with severe occlusion.

[0141] In one embodiment, S100, multiple attitude key points of each vehicle under test are obtained based on the attitude center point and multiple offsets; the heading angle of each vehicle under test is obtained based on the sine and cosine values ​​of the heading angle; and roadside parking management is performed based on the multiple attitude key points and the heading angle, including:

[0142] S101, perform world coordinate transformation on multiple attitude key points of each vehicle under test to obtain the position of each vehicle under test;

[0143] S102, roadside parking management is carried out based on the position and heading angle of the vehicle to be tested.

[0144] In this embodiment, the coordinates of multiple attitude key points are obtained in the image coordinate system. By converting the coordinates of multiple attitude key points in the image coordinate system to coordinates in the world coordinate system, the specific coordinate positions in the same world coordinate system as the parking space can be obtained. The camera position for roadside parking is fixed, and the location of the roadside parking space can be obtained. The coordinates of multiple attitude key points of each vehicle based on the image coordinate system are transformed to obtain the multiple attitude key point positions in the world coordinate system. The parking space position in the same world coordinate system is then judged, and the accuracy of vehicle attitude judgment is further enhanced by combining the constraint of the heading angle. This determines whether the vehicle is parked in the parking space and whether there are illegal parking behaviors such as parking over the line or parking in a no-parking zone, thereby realizing the management and decision-making of roadside parking and realizing roadside parking management.

[0145] Please see Figure 3 In one embodiment, this application provides a roadside parking management system 01 based on heading angle attitude. The roadside parking management system 01 based on heading angle attitude includes a data acquisition module 10, an image extraction module 20, a data augmentation module 30, a backbone network module 40, a feature aggregation network module 50, a key point prediction network module 60, a heading angle regression network module 70, a model training module 80, a detection module 90, and a parking management module 100.

[0146] The data acquisition module 10 is used to acquire a modeling and simulation traffic scene image dataset. This dataset includes multiple traffic scene images. Each image is labeled with the ground truth information of the 2D bounding box for each vehicle, multiple ground truth pose keypoints corresponding to the 3D bounding box of each vehicle, the ground truth pose center point, multiple ground truth offsets from the ground truth pose center point to each keypoint, the ground truth heading angle, the sine value of the ground truth heading angle, and the cosine value of the ground truth heading angle for each vehicle. The image extraction module 20 is used to extract the image of each vehicle from each traffic scene image based on the ground truth information of the 2D bounding box.

[0147] The data augmentation module 30 performs size transformation and keypoint data reconstruction on the vehicle images to obtain a reconstructed vehicle image for each vehicle. It then performs random erasure data augmentation on the reconstructed vehicle images to obtain an enhanced vehicle image for each vehicle. The backbone network module 40 inputs the enhanced vehicle images into the backbone network of the keypoint detection model for feature extraction to obtain a backbone feature map dataset. The feature aggregation network module 50 inputs each backbone feature map from the backbone feature map dataset into the feature aggregation network of the keypoint detection model for feature fusion to obtain an aggregated feature map dataset.

[0148] The keypoint prediction network module 60 is used to input each aggregated feature map in the aggregated feature map dataset into the keypoint prediction network of the keypoint detection model to predict the location of points, thereby obtaining the predicted attitude center point and multiple predicted offsets for each vehicle. The heading angle regression network module 70 is used to input each aggregated feature map in the aggregated feature map dataset into the vehicle heading angle regression network of the keypoint detection model to predict the heading angle, thereby obtaining the predicted heading angle sine and cosine values ​​for each vehicle, and obtaining the predicted heading angle for each vehicle based on the predicted heading angle sine and cosine values.

[0149] The model training module 80 is used to construct a loss function for the keypoint detection model based on the predicted heading angle, predicted attitude center point, multiple predicted offsets, true heading angle, true attitude center point, and multiple true offsets. It then trains and optimizes the keypoint detection model based on the loss function to obtain a trained keypoint detection model. The detection module 90 is used to acquire images of the traffic scene to be tested. Based on the trained keypoint detection model, it predicts keypoints for each vehicle image extracted from the traffic scene image, obtaining the attitude center point, multiple offsets, sine and cosine values ​​of the heading angle for each vehicle. The parking management module 100 is used to obtain multiple attitude keypoints for each vehicle based on the attitude center point and multiple offsets. It also obtains the heading angle for each vehicle based on the sine and cosine values ​​of the heading angle and performs roadside parking management based on the multiple attitude keypoints and the heading angle.

[0150] In this embodiment, the description of the data acquisition module 10 can be referred to the description of S10 in the above embodiment. The description of the image extraction module 20 can be referred to the description of S20 in the above embodiment. The description of the data augmentation module 30 can be referred to the description of S30 in the above embodiment. The description of the backbone network module 40 can be referred to the description of S40 in the above embodiment. The description of the feature aggregation network module 50 can be referred to the description of S50 in the above embodiment. The description of the key point prediction network module 60 can be referred to the description of S60 in the above embodiment. The description of the heading angle regression network module 70 can be referred to the description of S70 in the above embodiment. The description of the model training module 80 can be referred to the description of S80 in the above embodiment. The description of the detection module 90 can be referred to the description of S90 in the above embodiment. The description of the parking management module 100 can be referred to the description of S100 in the above embodiment.

[0151] In one embodiment, the data augmentation module 30 includes a size transformation module, a pixel filling module, and a key point data reconstruction module. The size transformation module sets the original length of the vehicle image to 256 pixels and transforms the original width of the vehicle image according to the length transformation ratio to obtain a new width for the vehicle image. The pixel filling module fills the new width of the vehicle image with 0 pixels to 256 pixels, obtaining a size-transformed image. The key point data reconstruction module transforms multiple real pose key points, real pose center points, and multiple real offsets according to the transformation ratio between the size of the size-transformed image and the original size of the vehicle image, obtaining a reconstructed vehicle image.

[0152] In this embodiment, the description of the size transformation module can be found in the description of S310 in the above embodiment. The description of the pixel filling module can be found in the description of S320 in the above embodiment. The description of the key point data reconstruction module can be found in the description of S330 in the above embodiment.

[0153] In one embodiment, the model training module 80 includes a first regression loss function module, a second regression loss function module, a third regression loss function module, and a total loss function module. The first regression loss function module constructs a regression loss function for the vehicle's heading angle based on the predicted and true heading angles for each vehicle. The second regression loss function module constructs a regression loss function for the vehicle's attitude center point based on the predicted and true attitude center points for each vehicle. The third regression loss function module constructs a regression loss function for the offsets of the vehicle's attitude key points based on multiple predicted and multiple true offsets for each vehicle. The total loss function module constructs the loss function for the key point detection model based on the regression loss function for the vehicle's heading angle, the regression loss function for the vehicle's attitude center point, and the regression loss function for the offsets of the vehicle's attitude key points.

[0154] In this embodiment, the description of the first regression loss function module can be referred to the description of S810 in the above embodiment. The description of the second regression loss function module can be referred to the description of S820 in the above embodiment. The description of the third regression loss function module can be referred to the description of S830 in the above embodiment. The description of the total loss function module can be referred to the description of S840 in the above embodiment.

[0155] In one embodiment, the detection module 90 includes a two-dimensional detection box information acquisition module, a vehicle image acquisition module, an enhanced vehicle image acquisition module, and a vehicle pose acquisition module. The two-dimensional detection box information acquisition module performs vehicle target detection on the traffic scene image under test using a vehicle target detection algorithm, obtaining two-dimensional detection box information for each vehicle under test. The vehicle image acquisition module extracts images from the traffic scene image under test based on the two-dimensional detection box information, obtaining a vehicle image for each vehicle under test. The enhanced vehicle image acquisition module performs size transformation and keypoint data reconstruction on the vehicle images under test, obtaining a reconstructed vehicle image for each vehicle under test, and performs random erasure data augmentation on the reconstructed vehicle images to obtain an enhanced vehicle image for each vehicle under test. The vehicle pose acquisition module inputs the enhanced vehicle images to test into the trained keypoint detection model, outputting the pose center point, multiple offsets, sine of the heading angle, and cosine of the heading angle for each vehicle under test.

[0156] In this embodiment, the description of the two-dimensional detection box information acquisition module can be found in the description of S910 in the above embodiment. The description of the vehicle image acquisition module can be found in the description of S920 in the above embodiment. The description of the enhanced vehicle image acquisition module can be found in the description of S930 in the above embodiment. The description of the vehicle attitude acquisition module can be found in the description of S940 in the above embodiment.

[0157] In one embodiment, the parking management module 100 includes a coordinate transformation module and a management module. The coordinate transformation module performs world coordinate transformation on multiple attitude key points of each vehicle under test to obtain the position of each vehicle. The management module performs roadside parking management based on the position and heading angle of the vehicles under test.

[0158] In this embodiment, the description of the coordinate transformation module can be found in the description of S101 in the above embodiment. The description of the management module can be found in the description of S102 in the above embodiment.

[0159] In the various embodiments described above, the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process may be rearranged without departing from the scope of this disclosure. The appended method claims provide elements of various steps in an exemplary order and are not intended to limit the scope to a specific order or hierarchy.

[0160] Those skilled in the art will also understand that the various illustrative logical blocks, modules, and steps listed in the embodiments of this application can be implemented by electronic hardware, computer software, or a combination of both. To clearly demonstrate the interchangeability of hardware and software, the functions of the various illustrative components, modules, and steps described above have been generally described. Whether such functionality is implemented through hardware or software depends on the specific application and the overall system design requirements. Those skilled in the art can implement the described functions using various methods for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of this application.

[0161] The various illustrative logic blocks or modules described in the embodiments of this application can be implemented or operate the described functions using a general-purpose processor, digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor; alternatively, it can be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented using a combination of computing devices, such as a digital signal processor and a microprocessor, multiple microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0162] The steps of the methods or algorithms described in the embodiments of this application can be directly embedded in hardware, a software module executed by a processor, or a combination of both. The software module can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from and write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and storage medium can be housed in an ASIC, which can be housed in a user terminal. Optionally, the processor and storage medium can also be housed in different components of the user terminal.

[0163] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A roadside parking management method based on heading angle attitude, characterized in that, include: A modeling and simulation traffic scene image dataset is obtained. The modeling and simulation traffic scene image dataset includes multiple traffic scene images. Each traffic scene image is labeled with the real information of the two-dimensional detection box of each vehicle, multiple real attitude key points corresponding to the three-dimensional bounding box of each vehicle, the real attitude center point, multiple real offsets of the multiple real attitude key points to the real attitude center point, the real heading angle of each vehicle, the sine value of the real heading angle, and the cosine value of the real heading angle. Based on the real information of the two-dimensional detection box, each vehicle in each traffic scene image is extracted to obtain a vehicle image of each vehicle; The vehicle images are resized and reconstructed using key point data to obtain a reconstructed vehicle image for each vehicle. The reconstructed vehicle images are then augmented with random erasure data to obtain an augmented vehicle image for each vehicle. The enhanced vehicle image is input into the backbone network of the key point detection model for feature extraction to obtain a backbone feature map dataset. Each of the backbone feature maps in the backbone feature map dataset is input into the feature aggregation network of the key point detection model for feature fusion to obtain an aggregated feature map dataset. Each of the aggregated feature maps in the aggregated feature map dataset is input into the key point prediction network of the key point detection model to predict the location, thereby obtaining the predicted pose center point and multiple predicted offsets for each vehicle. Each aggregated feature map in the aggregated feature map dataset is input into the vehicle heading angle regression network of the key point detection model to predict the heading angle, thereby obtaining the sine and cosine values ​​of the predicted heading angle for each vehicle, and obtaining the predicted heading angle for each vehicle based on the sine and cosine values ​​of the predicted heading angle. The loss function of the keypoint detection model is constructed based on the predicted heading angle, the predicted attitude center point, the multiple predicted offsets, the true heading angle, the true attitude center point, and the multiple true offsets. The keypoint detection model is then trained and optimized based on the loss function to obtain the trained keypoint detection model. Acquire a traffic scene image to be tested, and perform key point prediction on each vehicle image extracted from the traffic scene image to be tested based on the trained key point detection model to obtain the attitude center point, multiple offsets, sine value of heading angle and cosine value of heading angle for each vehicle to be tested. Based on the attitude center point and the multiple offsets, multiple attitude key points of each vehicle under test are obtained. Based on the sine and cosine values ​​of the heading angle, the heading angle of each vehicle under test is obtained. Roadside parking management is performed based on the multiple attitude key points and the heading angle. The process of resizing and reconstructing key point data of the vehicle images to obtain a reconstructed vehicle image for each vehicle, and then performing random erasure data enhancement on the reconstructed vehicle images to obtain an enhanced vehicle image for each vehicle, includes: The original length of the vehicle image is set to 256 pixels, and the original width of the vehicle image is transformed according to the length transformation ratio to obtain the new width of the vehicle image; the new width of the vehicle image is padded with 0 pixels to 256 pixels to obtain a size transformation image. Based on the transformation ratio between the size of the size-transformed image and the original size of the vehicle image, the multiple real pose key points, the real pose center point, and the multiple real offsets are transformed to obtain the reconstructed vehicle image.

2. The roadside parking management method based on heading angle attitude according to claim 1, characterized in that, The step of constructing a loss function for the keypoint detection model based on the predicted heading angle, the predicted attitude center point, the multiple predicted offsets, the true heading angle, the true attitude center point, and the multiple true offsets, and training and optimizing the keypoint detection model based on the loss function to obtain a trained keypoint detection model, includes: Based on the predicted heading angle and the actual heading angle of each vehicle, a regression loss function for the vehicle heading angle is constructed. Based on the predicted attitude center point and the true attitude center point of each vehicle, a regression loss function for the vehicle attitude center point is constructed. Based on the multiple predicted offsets and multiple true offsets of each vehicle, a regression loss function for the offset of the vehicle attitude key points is constructed. The loss function of the key point detection model is constructed based on the regression loss function of the vehicle heading angle, the regression loss function of the vehicle attitude center point, and the regression loss function of the offset of the vehicle attitude key points.

3. The roadside parking management method based on heading angle attitude according to claim 1, characterized in that, The process of acquiring the traffic scene image to be tested involves predicting key points for each vehicle image extracted from the traffic scene image based on the trained key point detection model, thereby obtaining the attitude center point, multiple offsets, sine value of the heading angle, and cosine value of the heading angle for each vehicle. This includes: Vehicle target detection is performed on the traffic scene image to be tested using a vehicle target detection algorithm to obtain two-dimensional detection box information for each vehicle to be tested. The traffic scene image to be tested is extracted based on the two-dimensional detection box information to obtain the vehicle image of each vehicle to be tested; The images of the vehicles under test are resized and reconstructed using key point data to obtain a reconstructed vehicle image for each vehicle under test. Random erasure data enhancement is then performed on the reconstructed vehicle images to obtain an enhanced vehicle image for each vehicle under test. The image of the enhanced vehicle to be tested is input into the trained key point detection model, and the attitude center point, the multiple offsets, the sine value of the heading angle and the cosine value of the heading angle of each vehicle to be tested are output.

4. The roadside parking management method based on heading angle attitude according to claim 1, characterized in that, The process involves obtaining multiple attitude key points for each vehicle under test based on the attitude center point and the multiple offsets, obtaining the heading angle for each vehicle under test based on the sine and cosine values ​​of the heading angle, and performing roadside parking management based on the multiple attitude key points and the heading angle, including: Perform world coordinate transformation on multiple attitude key points of each vehicle under test to obtain the position of each vehicle under test; Roadside parking management is performed based on the position of the vehicle under test and the heading angle.

5. A roadside parking management system based on heading angle attitude, characterized in that, include: The data acquisition module is used to acquire a modeling and simulation traffic scene image dataset. The modeling and simulation traffic scene image dataset includes multiple traffic scene images. Each traffic scene image is labeled with the real information of the two-dimensional detection box of each vehicle, multiple real attitude key points corresponding to the three-dimensional bounding box of each vehicle, the real attitude center point, multiple real offsets of the multiple real attitude key points to the real attitude center point, the real heading angle of each vehicle, the sine value of the real heading angle, and the cosine value of the real heading angle. The image extraction module is used to extract each vehicle in each traffic scene image based on the real information of the two-dimensional detection box, and obtain a vehicle image of each vehicle. The data augmentation module is used to perform size transformation and key point data reconstruction on the vehicle images to obtain a reconstructed vehicle image for each vehicle, and to perform random erasure data augmentation on the reconstructed vehicle images to obtain an enhanced vehicle image for each vehicle. The backbone network module is used to input the enhanced vehicle image into the backbone network of the key point detection model for feature extraction to obtain a backbone feature map dataset. The feature aggregation network module is used to input each of the backbone feature maps in the backbone feature map dataset into the feature aggregation network of the key point detection model for feature fusion to obtain an aggregated feature map dataset. The key point prediction network module is used to input each of the aggregated feature maps in the aggregated feature map dataset into the key point prediction network of the key point detection model to predict the position, thereby obtaining the predicted pose center point and multiple predicted offsets for each vehicle. The heading angle regression network module is used to input each of the aggregated feature maps in the aggregated feature map dataset into the vehicle heading angle regression network of the key point detection model to predict the heading angle, obtain the sine value and cosine value of the predicted heading angle of each vehicle, and obtain the predicted heading angle of each vehicle based on the sine value and cosine value of the predicted heading angle. The model training module is used to construct a loss function for the key point detection model based on the predicted heading angle, the predicted attitude center point, the multiple predicted offsets, the true heading angle, the true attitude center point, and the multiple true offsets, and to train and optimize the key point detection model based on the loss function to obtain a trained key point detection model. The detection module is used to acquire the traffic scene image to be tested, and to perform key point prediction on each vehicle image extracted from the traffic scene image to be tested according to the trained key point detection model, so as to obtain the attitude center point, multiple offsets, sine value of heading angle and cosine value of heading angle for each vehicle to be tested. The parking management module is used to obtain multiple attitude key points of each vehicle under test based on the attitude center point and the multiple offsets, obtain the heading angle of each vehicle under test based on the sine and cosine values ​​of the heading angle, and perform roadside parking management based on the multiple attitude key points and the heading angle. The data enhancement module includes: The size transformation module is used to set the original length of the vehicle image to 256 pixels and transform the original width of the vehicle image according to the length transformation ratio to obtain the new width of the vehicle image. A pixel filling module is used to fill the new width of the vehicle image from 0 pixels to 256 pixels to obtain a size-transformed image; The key point data reconstruction module is used to transform the multiple real pose key points, the real pose center point, and the multiple real offsets according to the transformation ratio between the size of the size-transformed image and the original size of the vehicle image, so as to obtain the reconstructed vehicle image.

6. The roadside parking management system based on heading angle attitude according to claim 5, characterized in that, The model training module includes: The first regression loss function module is used to construct a regression loss function for the vehicle heading angle based on the predicted heading angle and the actual heading angle of each vehicle. The second regression loss function module is used to construct a regression loss function for the vehicle attitude center point based on the predicted attitude center point and the true attitude center point of each vehicle. The third regression loss function module is used to construct a regression loss function for the offset of the vehicle attitude key points based on the multiple predicted offsets and the multiple true offsets of each vehicle. The total loss function module is used to construct the loss function of the key point detection model based on the regression loss function of the vehicle heading angle, the regression loss function of the vehicle attitude center point, and the regression loss function of the offset of the vehicle attitude key points.

7. The roadside parking management system based on heading angle attitude according to claim 5, characterized in that, The detection module includes: The two-dimensional detection box information acquisition module is used to perform vehicle target detection on the traffic scene image to be tested according to the vehicle target detection algorithm, and obtain the two-dimensional detection box information of each vehicle to be tested. The vehicle image acquisition module is used to extract the traffic scene image to be tested based on the two-dimensional detection box information to obtain the vehicle image to be tested for each vehicle to be tested. The vehicle image acquisition module is used to perform size transformation and key point data reconstruction on the vehicle image to obtain a reconstructed vehicle image for each vehicle to be tested, and to perform random erasure data enhancement on the reconstructed vehicle image to obtain a vehicle image to be tested for each vehicle to be tested. The vehicle attitude acquisition module is used to input the enhanced vehicle image to the trained key point detection model and output the attitude center point, multiple offsets, sine value of the heading angle and cosine value of the heading angle for each vehicle under test.

8. The roadside parking management system based on heading angle attitude according to claim 5, characterized in that, The parking management module includes: The coordinate transformation module is used to perform world coordinate transformation on multiple attitude key points of each of the vehicles under test to obtain the position of each vehicle under test; The management module is used to manage roadside parking based on the location of the vehicle under test and the heading angle.

Images