End-to-end roadside parking detection method and device

Abstract

The invention provides an end-to-end roadside parking detection method and device, and relates to the technical field of parking detection, and the method comprises the steps: carrying out the collection of a panoramic image, and obtaining a roadside parking scene image; after preprocessing, carrying out berth coordinate calibration, and generating a berth area coordinate data set; carrying out image feature extraction to generate an image embedding vector; carrying out berth position coding, and generating a berth prompt vector; after splicing, feature interaction fusion is carried out through a multi-head self-attention mechanism to obtain enhanced features of the parking area; performing end-to-end berth occupation state judgment; and in combination with a continuous multi-frame detection result, carrying out time sequence smoothing processing, and outputting a berth stable occupancy state. According to the method and the device, the technical problem that misjudgment is easy to occur in a complex environment due to the fact that a parking detection scheme in the prior art adopts two processing stages of vehicle detection and berth parking judgment and lacks the real-time correlation capability of the vehicle and the berth is solved.

Description

Technical Field

[0001] This invention relates to the field of parking detection technology, specifically to an end-to-end roadside parking detection method and apparatus. Background Technology

[0002] With the rapid pace of urbanization, the number of vehicles in cities is increasing dramatically, leading to a surge in demand for roadside parking. Roadside parking management has become a crucial aspect of urban traffic management. Traditional parking detection solutions employ two processing stages: vehicle detection and parking space determination, to confirm a vehicle's parking status in a specific space. However, these solutions lack real-time correlation between vehicles and parking spaces, posing significant challenges in handling parking disputes and ensuring accurate billing. Consequently, they fail to meet the demands of modern smart cities for efficient and precise parking management. Summary of the Invention

[0003] This application provides an end-to-end roadside parking detection method and apparatus, aiming to solve the technical problem that existing parking detection schemes employ two processing stages—vehicle detection and parking space judgment—lacking the ability to correlate vehicles and parking spaces in real time, which leads to misjudgments in complex environments.

[0004] The first aspect disclosed in this application provides an end-to-end roadside parking detection method, the method comprising: acquiring panoramic images by means of a high-position camera deployed on the roadside to obtain roadside parking scene images, wherein the roadside parking scene images cover multiple parking space areas; preprocessing the roadside parking scene images and then calibrating the parking space coordinates of the multiple parking space areas to generate a parking space area coordinate dataset; extracting image features from the roadside parking scene images to generate image embedding vectors; encoding parking space positions based on the parking space area coordinate dataset to generate parking space prompt vectors; concatenating the image embedding vectors and the parking space prompt vectors, and then performing feature interaction fusion through a multi-head self-attention mechanism to obtain parking space area enhancement features; determining the end-to-end parking space occupancy status based on the parking space area enhancement features using a classifier; and, based on the parking space occupancy status determination results and combined with the detection results of multiple consecutive frames, performing temporal smoothing processing using a predetermined sliding window to output the stable parking space occupancy status.

[0005] The second aspect of this application discloses an end-to-end roadside parking detection device. The device is used in the aforementioned end-to-end roadside parking detection method. The device includes: a panoramic image acquisition module for acquiring panoramic images using a high-position camera deployed on the roadside to obtain roadside parking scene images, wherein the roadside parking scene images cover multiple parking space areas; a parking space coordinate calibration module for preprocessing the roadside parking scene images and then calibrating the parking space coordinates of the multiple parking space areas to generate a parking space area coordinate dataset; and an image feature extraction module for extracting image features from the roadside parking scene images to generate an image embedding vector. The system comprises: a berth location encoding module, used to encode berth locations based on the berth area coordinate dataset and generate berth prompt vectors; a feature interaction fusion module, used to concatenate the image embedding vector and the berth prompt vector, and then perform feature interaction fusion through a multi-head self-attention mechanism to obtain enhanced features for the berth area; an occupancy status judgment module, used to perform end-to-end berth occupancy status judgment based on the enhanced features for the berth area, and a stable occupancy status output module, used to output the stable berth occupancy status based on the berth occupancy status judgment result, combined with the detection results of multiple consecutive frames, using a predetermined sliding window for temporal smoothing processing.

[0006] One or more technical solutions provided in this application have at least the following beneficial effects:

[0007] Panoramic images were acquired using a high-position camera, covering multiple berth areas to ensure comprehensive and efficient image data. A fisheye distortion correction algorithm was employed to eliminate image distortion, ensuring data accuracy. Simultaneously, illumination equalization effectively addressed shadows and low-light conditions at night, further improving robustness in complex environments. Berth coordinate calibration provided an accurate data foundation for subsequent berth location coding, enhancing the mapping between images and physical berths and making berth area identification more precise. Berth coordinates were normalized and transformed into high-dimensional vectors through a learnable location coding layer, providing spatial cues for the Transformer network. Through location coding, berth location information can be dynamically embedded into image features, enhancing the understanding of... The system enhances its focus on berth areas. By segmenting the image into multiple image patches and linearly mapping them to form image embedding vectors, these vectors are concatenated with berth cue vectors and input into a Transformer encoder. Through a multi-head self-attention mechanism, the system achieves interaction and fusion between image and berth location information, effectively capturing long-distance dependencies in the image. A classifier determines the occupancy status of each berth, outputting not only the parking status but also a confidence score, further enhancing the system's accuracy and avoiding misjudgments. A temporal state optimization mechanism is introduced, combining detection results from multiple consecutive frames and smoothing the state through a sliding window mechanism, reducing the impact of instantaneous misjudgments. This improves the stability and accuracy of the detection results and effectively reduces erroneous judgments caused by transient interference.

[0008] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0009] Figure 1 This is a schematic flowchart of the end-to-end roadside parking detection method provided in the embodiments of this application.

[0010] Figure 2 This is a schematic diagram of the end-to-end roadside parking detection device provided in an embodiment of this application.

[0011] Figure labeling: Panoramic image acquisition module 10, image feature extraction module 20, berth coordinate calibration module 30, berth location encoding module 40, feature interaction fusion module 50, occupancy status judgment module 60, stable occupancy status output module 70. Detailed Implementation

[0012] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0013] Example 1, as Figure 1 As shown in the figure, this application provides an end-to-end roadside parking detection method, the method comprising:

[0014] Panoramic images are acquired by deploying high-position cameras along the roadside to obtain roadside parking scene images, which cover multiple parking space areas.

[0015] The deployment of high-position cameras can cover a sufficient number of parking spaces. The cameras acquire images in a panoramic manner, ensuring that multiple parking spaces appear in the image at the same time. This is to obtain the parking status of different parking spaces simultaneously, providing more information for subsequent analysis. To improve detection accuracy, cameras with higher resolution are used, and appropriate lighting conditions are adopted to avoid the influence of occlusion or low light environments.

[0016] After preprocessing the roadside parking scene image, the coordinates of the multiple parking spaces are calibrated to generate a parking space coordinate dataset.

[0017] Because of the fisheye distortion inherent in high-position cameras, straight lines in the image will be distorted. Therefore, distortion correction is needed to restore the true shape of the image. Using standard fisheye correction algorithms, the distorted image is corrected to be closer to the actual scene by calibrating the camera's intrinsic and extrinsic parameters. In some low-light or strong-light environments, such as at night, in bright light or in shadow, the image will be affected, resulting in low contrast or overexposure. To deal with these interferences, the image is processed by illumination equalization to improve the image's visibility.

[0018] To achieve precise correlation between parking detection and physical parking spaces, the parking space areas in the image are calibrated using coordinates. During calibration, the pixel coordinates of the four corner points of each parking space in the image are first determined. Then, using a geometric transformation method, such as perspective transformation, the parking space coordinates in the image are mapped to the actual parking space coordinates. In this way, a correspondence is established between the image coordinates and the real-world parking space locations. The generated parking space area coordinate dataset contains the coordinates of the four corner points of each parking space, serving as input for subsequent processing.

[0019] Image features are extracted from the roadside parking scene image to generate an image embedding vector.

[0020] Effective features are extracted from the collected roadside parking scene images and converted into image embedding vectors suitable for subsequent model processing. Specifically, the collected images are divided into fixed-size blocks, which are the basic units processed by the ViT (VisionTransformer) model. Each image block is flattened into a one-dimensional vector, so that each image block can be represented as a fixed-dimensional vector. The flattened vector of each image block is mapped through a linear projection layer to generate a fixed-dimensional vector. These generated vectors form a sequence, which is the image embedding vector, and serves as the feature representation for the input of the Transformer model.

[0021] Based on the berth area coordinate dataset, berth location encoding is performed to generate berth indication vectors.

[0022] The calibrated berth corner coordinates, i.e., the pixel coordinates of the four corner points of each berth area, are normalized. Normalization aims to convert these coordinates to a uniform scale, avoiding inconsistencies in the encoding process caused by differences in image size across different scenes. The normalized berth coordinates serve as location cues (Prompts), which are then transformed into high-dimensional vectors through a learnable location encoding layer. This location encoding provides the model with spatial positional information, enabling the Transformer model to perceive the specific location of the berth. Finally, a berth cue vector, representing the spatial cue information of the berth, is generated and combined with image features as input for subsequent feature fusion and classification stages.

[0023] After concatenating the image embedding vector with the berth prompt vector, feature interaction fusion is performed through a multi-head self-attention mechanism to obtain enhanced features for the berth area.

[0024] The image embedding vector is concatenated with the berth cue vector to combine the global features of the image with spatial location information, thereby better capturing the state of each berth area in the image. The core mechanism of the Transformer model is self-attention, which can weight different parts of the input to capture global contextual information. The self-attention mechanism calculates the similarity between each berth area and other areas based on the image features and location vector of the current berth area, and dynamically adjusts the weighting values ​​of the features based on these similarities. In this way, the model can focus on key areas or potential changes in parking status and ignore irrelevant parts.

[0025] To address the limitations of a single attention head, a multi-head self-attention mechanism is employed. This mechanism computes multiple attention heads in parallel, each learning different relationships or important features, and then merges them to obtain a richer representation. The multi-head attention mechanism helps the model to more comprehensively understand the interrelationships between different berth areas. After processing by the multi-head self-attention mechanism, the features of each berth area are enhanced. By interacting and fusing image features with berth location information, more accurate and comprehensive berth area features can be generated, supporting subsequent state determinations.

[0026] Based on the enhanced features of the berth area, an end-to-end berth occupancy status is determined using a classifier.

[0027] The enhanced features of the berth area are passed as input to a classifier for classification. The classifier is a fully connected network or neural network layer. By processing the input features, the classifier outputs the berth occupancy status, including whether it is idle or occupied. The classifier can use multi-layer fully connected layers based on deep neural networks or more complex architectures such as convolutional neural networks. In this way, the model can determine the status of each berth based on image and location features.

[0028] Based on the berth occupancy status judgment result and combined with the detection results of multiple consecutive frames, a predetermined sliding window is used for temporal smoothing processing to output the stable berth occupancy status.

[0029] In practical applications, berth occupancy status may fluctuate between consecutive time frames. For example, due to short-term occlusion or detection errors, the berth status may be incorrectly judged. To overcome this fluctuation, a temporal smoothing algorithm is needed to improve the stability of the results. A sliding window works by sliding a window across multiple consecutive frames and generating a smoothed result based on the berth occupancy status within the window. Specifically, assuming there are N frames of detection results, the sliding window determines the berth status of the current frame based on the occupancy status of a certain number of past frames. After temporal smoothing, the final output is a stable berth occupancy status, reflecting the stability of the berth over time. This smoothed status determination effectively avoids misjudgments and detection fluctuations, improving the accuracy and reliability of berth occupancy status.

[0030] Furthermore, the method for preprocessing the roadside parking scene image includes:

[0031] After performing fisheye distortion correction on the roadside parking scene image, illumination equalization processing is performed; based on the illumination equalization result, brightness compensation or contrast enhancement is performed on the shadow areas and low-light areas at night of the roadside parking scene image.

[0032] Fisheye distortion occurs when an image is captured using a fisheye lens. The central area of ​​the image is relatively clear, while the edges exhibit strong curvature or deformation, resulting in distortion of the image's geometry. The core of correction is to use camera intrinsic parameters, distortion coefficients, and transformation matrices to correct the distortion caused by lens characteristics, making the image closer to a view of the real world.

[0033] Illumination equalization is used to enhance image contrast, enabling images to display more details under different brightness conditions, especially in areas that are too bright or too dark. By adjusting the distribution of image pixels, the brightness in the image is made more evenly distributed. Histogram equalization algorithms are used, which enhance image contrast by stretching the range of image brightness values.

[0034] In roadside parking scenarios, shadow areas (such as shadows cast by vehicles) and low-light areas (such as at night or in poorly lit areas) appear as darker regions, potentially leading to lost or unclear image features and affecting parking space status detection. Brightness compensation aims to increase the brightness of darker areas in the image, making them consistent with other areas and preventing overly dark areas from being effectively identified. Contrast enhancement aims to increase the brightness differences between different areas of the image, especially those in low-light or shadowed areas. Through brightness compensation and contrast enhancement, the loss of details in shadow and low-light areas can be effectively avoided, ensuring accurate identification of parking spaces and parking status under various lighting conditions, further improving the stability and reliability of detection.

[0035] Furthermore, based on the berth area coordinate dataset, berth location encoding is performed to generate berth indication vectors, including:

[0036] Based on the berth area coordinate dataset, the pixel position coordinates of the four corner points of the berth area are normalized and then converted into high-dimensional vectors through a learnable position encoding layer to generate the berth prompt vector.

[0037] The berth area coordinate dataset contains the location information of all berth areas in the image, accurately locating the four corner points of each berth and obtaining their pixel coordinates. To eliminate size differences between different images, the pixel positions of the four corner points of the berth area are normalized, converting the pixel coordinates into relative coordinates, for example, scaling the coordinates to the range [0,1]. The normalized coordinates are independent of the image size, enabling better handling of images with different resolutions.

[0038] Position encoding is a technique used in Transformer models to represent the relative positional information of input elements. Since Transformers lack built-in order or spatial information, position encoding is necessary to embed the positional information of each berth into the model's input. Unlike traditional fixed position encoding, a learnable encoding method is used here, meaning the position encoding is continuously optimized through training to better represent spatial positional features. Position encoding transforms the normalized positions of berths into high-dimensional vector representations through a learning layer (such as an embedding layer). These vectors capture the correlation of positions.

[0039] After the location encoding layer, the positions of the four corner points of each berth are converted into a high-dimensional vector. These vectors are called berth cue vectors, which help the model understand the spatial location and structure of each berth, so that image features and location information can be combined more accurately in the feature fusion stage.

[0040] Furthermore, image feature extraction is performed on the roadside parking scene image to generate an image embedding vector, including:

[0041] Using the ViT model, based on a predetermined segmentation size, the roadside parking scene image is segmented into multiple image blocks. After flattening the multiple image blocks, they are mapped to fixed-dimensional vectors through a linear layer to form a sequence input, generating the image embedding vector; wherein, the predetermined segmentation size is 16×16 pixels.

[0042] The ViT model is a self-attention-based model that effectively processes image data. The roadside parking scene image is segmented according to a predetermined size, where the segment size refers to the size of the small blocks into which the image is divided. In this case, the segment size is 16×16 pixels, meaning each small block is 16×16 pixels. Each 16×16 image block is flattened into a one-dimensional vector. The flattening process involves arranging all pixel values ​​in the image block into a long vector in row or column order. To meet the input requirements of ViT, each flattened image block vector is mapped to a fixed dimension through a linear layer. The linear layer compresses or expands the original high-dimensional vector to the predetermined dimension, making it a fixed-dimensional vector. This vector represents the image block for subsequent self-attention processing. After all the segmented image blocks have undergone linear mapping, they form a sequence input. This sequence contains the vector representation of each block in the image, ultimately generating an image embedding vector.

[0043] Furthermore, enhanced features for the berth area are obtained through feature interaction fusion using a multi-head self-attention mechanism. The methods include:

[0044] A cross-attention structure is adopted to interact the image embedding vector and the berth cue vector, dynamically adjust the feature weights, and focus on the features of the berth area. The berth cue vector is used as the query vector Q, and the key vector K and value vector V generated by the image embedding vector are used as the input for attention calculation. The attention weights are obtained by calculating the dot product similarity of Q and K and normalizing it with Softmax. Based on the attention weights, V is weighted and summed to generate an enhanced feature representation of the berth area, thus obtaining the enhanced features of the berth area.

[0045] A cross attention mechanism is used to fuse the features of the image embedding vector and the berth cue vector. The cross attention structure interacts with features from different sources and dynamically adjusts their respective feature weights by calculating their interrelationships in order to focus on the important region, namely the berth region.

[0046] Using the berth prompt vector as the query vector Q helps focus on a specific region in the image. The image embedding vector represents the feature information of the image. By mapping it, the feature representation of each image block can be obtained. The embedding vector of each image block is regarded as the key vector K, which provides a reference for subsequent matching. The value vector V represents the actual image feature information in the image embedding vector and stores the specific image content.

[0047] The relationship between query vector Q and key vector K is evaluated by calculating the similarity of their dot product. This similarity reflects the degree of matching between each query vector and the key vector, thus determining which regions need attention. After calculating the dot product similarity, the data is normalized using the Softmax function to obtain the attention weight for each feature, ensuring that the sum of all weights is 1. This ensures that the model focuses more on important image patches related to the berth area. Based on the calculated attention weights, the value vector V is weighted and summed to generate an enhanced feature representation that focuses on the berth area. This ensures that berth occupancy detection focuses on areas closely related to the berth occupancy status, while ignoring other irrelevant areas or noise.

[0048] Furthermore, the enhanced features of the berth area are represented as follows:

[0049] ;

[0050] Where Q is the Query, a query vector composed of berth suggestion vectors; K is the Key, a key vector calculated from the feature embedding of the image embedding vector; and V is the Value, a value vector representing the actual image feature information in the image embedding vector. Represents the feature dimension of the K vector. It is a square root function. square root Used for scaling The inner product result is used to stabilize the numerical range. This involves normalizing the similarity scores between Q and K into a probability distribution. Enhanced feature representation for focusing on berth areas.

[0051] The dot product of the query vector Q and the key vector K is calculated to obtain a similarity score between each query and key pair. This score reflects the correlation between the query vector and the image patch. To avoid computational instability due to excessively large values, the dot product result is scaled using a square root function. This scaling helps balance the calculation and prevents excessively large values ​​from affecting Softmax normalization. By performing a Softmax operation on the dot product calculation result, the obtained similarity score is converted into a probability distribution. These probability distributions are the attention weights, representing the relative importance of different image patches. Finally, by applying the attention weights to the value vector V and performing a weighted summation of the value vectors, an enhanced feature representation of the focused berth region is obtained. This process extracts the most relevant features from the image, thereby effectively focusing on the region related to the berth occupancy status.

[0052] Furthermore, based on the enhanced features of the berth area, an end-to-end berth occupancy status determination is performed using a classifier, including:

[0053] The classifier is used to perform feature analysis on the enhanced features of the berth area, outputting the berth occupancy status judgment result and simultaneously outputting a confidence score; the confidence score is used to filter false detections based on a predetermined confidence threshold; wherein, the classifier uses the Sigmoid activation function, and the predetermined confidence threshold is set to 0.5.

[0054] A classifier is used to analyze the enhanced features of the berth area to determine the occupancy status of the berths, such as occupied or vacant, and outputs a confidence score, which represents the model's confidence in the judgment result. The classifier uses the Sigmoid activation function, which is the activation function for binary classification problems. The Sigmoid function outputs the classification result as a probability value ranging from 0 to 1. This probability value is the confidence score, representing the probability that the berth is occupied.

[0055] If the confidence score of the classifier is lower than the preset confidence threshold, such as 0.5, the detection result is considered uncertain and has a high error, and can be regarded as a false detection, that is, a misjudgment of occupancy or idleness. Conversely, if the confidence score is higher than the threshold, the detection result is considered reliable and can be used for further decision-making.

[0056] Furthermore, by combining the detection results of multiple consecutive frames, a predetermined sliding window is used for temporal smoothing to output the stable berth occupancy status, including:

[0057] The predetermined sliding window is used to smooth the state of the detection results of the consecutive multiple frames. If three consecutive frames are determined to be occupied, they are marked as stable occupancy. The window length of the predetermined sliding window is greater than or equal to five frames.

[0058] The sliding window technique reduces the impact of noise and abnormal fluctuations by comprehensively judging data from multiple consecutive frames. Here, the predetermined sliding window length is set to be greater than or equal to 5 frames, meaning that at least 5 consecutive detection results will be analyzed to make a final judgment. The rule of determining berth occupancy as occupied after 3 consecutive frames is to ensure the stability of the detection results. Only after multiple consecutive confirmations of berth occupancy will it be marked as a stable occupancy state, avoiding misjudgments of valid occupancy due to short-term changes. The sliding window technique reduces the impact of misjudgments, ultimately outputting a stable occupancy state that reflects the true berth occupancy situation and avoids incorrect occupancy judgments due to occasional detection fluctuations.

[0059] Furthermore, it also includes:

[0060] A sample image dataset is collected, and the berth areas and berth occupancy status of the sample image dataset are labeled. The sample image dataset covers multiple environmental scenarios. The labeled sample image dataset is divided into training set, validation set and test set according to a predetermined ratio for model training. In the loss function design, weighted cross-entropy loss is used to increase the weight of occluded samples. After training, a lightweight deployment method is adopted, and the ViT model is compressed into a MobileViT architecture through knowledge distillation technology.

[0061] A sample image dataset was collected, covering various environmental scenarios to ensure its diversity and comprehensiveness. These scenarios included different weather conditions, day-night cycles, various lighting conditions, and different parking scenarios such as city streets and residential areas. For each image, the location of the parking space was marked, which involved labeling the coordinates of the parking space using methods such as bounding boxes and outlines. Based on the marked parking space areas, the occupancy status of the parking space was further indicated. This is a binary classification problem: occupied or vacant, and the labeling is based on the actual situation.

[0062] The dataset is divided according to a predetermined ratio, for example, 70% for training the model, 15% for adjusting hyperparameters and preventing overfitting during training, and 15% for final evaluation after training. When training the model, especially with imbalanced data, ordinary cross-entropy loss cannot adequately learn the features of minority class samples. In this approach, weighted cross-entropy loss is used to enhance the model's focus on minority class samples. To address situations where berths are occluded or partially occluded in the image, the model needs to pay more attention to these samples. By adding weights to these samples in the loss function, the model can be encouraged to focus more on these difficult-to-detect conditions.

[0063] Knowledge distillation is the process of transferring knowledge from a complex, high-performance model (called the teacher model) to a smaller, lighter model (called the student model). In this step, the ViT model acts as the teacher model. After training, its knowledge is transferred to the lightweight MobileViT architecture through knowledge distillation. This allows the student model to mimic the behavior of the teacher model while maintaining good performance with lower computational resources. During training, the student model not only learns the correct classification results of labeled data but also learns the probability distribution output by the teacher model, thus inheriting the teacher model's capabilities. Despite its smaller size and faster computation speed, it is suitable for deployment on mobile or embedded devices.

[0064] Example 2, based on the same inventive concept as the end-to-end roadside parking detection method in the aforementioned examples, such as... Figure 2 As shown in the figure, this application provides an end-to-end roadside parking detection device, the device comprising:

[0065] The panoramic image acquisition module 10 is used to acquire panoramic images through a high-position camera deployed on the roadside to obtain roadside parking scene images, wherein the roadside parking scene images cover multiple parking space areas; the parking space coordinate calibration module 20 is used to preprocess the roadside parking scene images and then calibrate the parking space coordinates of the multiple parking space areas to generate a parking space area coordinate dataset; the image feature extraction module 30 is used to extract image features from the roadside parking scene images to generate image embedding vectors; and the parking space location encoding module 40 is used to encode the parking space location based on the parking space area coordinate dataset. The system performs berth location encoding to generate berth prompt vectors; the feature interaction fusion module 50 concatenates the image embedding vector with the berth prompt vector and performs feature interaction fusion through a multi-head self-attention mechanism to obtain enhanced features for the berth area; the occupancy status judgment module 60 performs end-to-end berth occupancy status judgment based on the enhanced features for the berth area and uses a classifier; the stable occupancy status output module 70 outputs the stable berth occupancy status based on the berth occupancy status judgment result, combined with the detection results of multiple consecutive frames, and performs temporal smoothing processing using a predetermined sliding window.

[0066] Furthermore, the berth coordinate calibration module 20 is used to perform the following operation steps:

[0067] After performing fisheye distortion correction on the roadside parking scene image, illumination equalization processing is performed; based on the illumination equalization result, brightness compensation or contrast enhancement is performed on the shadow areas and low-light areas at night of the roadside parking scene image.

[0068] Furthermore, the berth location encoding module 40 is used to perform the following operation steps:

[0069] Based on the berth area coordinate dataset, the pixel position coordinates of the four corner points of the berth area are normalized and then converted into high-dimensional vectors through a learnable position encoding layer to generate the berth prompt vector.

[0070] Furthermore, the image feature extraction module 30 is used to perform the following operation steps:

[0071] Using the ViT model, based on a predetermined segmentation size, the roadside parking scene image is segmented into multiple image blocks. After flattening the multiple image blocks, they are mapped to fixed-dimensional vectors through a linear layer to form a sequence input, generating the image embedding vector; wherein, the predetermined segmentation size is 16×16 pixels.

[0072] Furthermore, the feature interaction fusion module 50 is used to perform the following operation steps:

[0073] A cross-attention structure is adopted to interact the image embedding vector and the berth cue vector, dynamically adjust the feature weights, and focus on the features of the berth area. The berth cue vector is used as the query vector Q, and the key vector K and value vector V generated by the image embedding vector are used as the input for attention calculation. The attention weights are obtained by calculating the dot product similarity of Q and K and normalizing it with Softmax. Based on the attention weights, V is weighted and summed to generate an enhanced feature representation of the berth area, thus obtaining the enhanced features of the berth area.

[0074] Furthermore, the enhanced features of the berth area are represented as follows:

[0075] ;

[0076] Where Q is the Query, a query vector composed of berth suggestion vectors; K is the Key, a key vector calculated from the feature embedding of the image embedding vector; and V is the Value, a value vector representing the actual image feature information in the image embedding vector. Represents the feature dimension of the K vector. It is a square root function. square root Used for scaling The inner product result is used to stabilize the numerical range. This involves normalizing the similarity scores between Q and K into a probability distribution. Enhanced feature representation for focusing on berth areas.

[0077] Furthermore, the occupancy status determination module 60 is used to perform the following operation steps:

[0078] The classifier is used to perform feature analysis on the enhanced features of the berth area, outputting the berth occupancy status judgment result and simultaneously outputting a confidence score; the confidence score is used to filter false detections based on a predetermined confidence threshold; wherein, the classifier uses the Sigmoid activation function, and the predetermined confidence threshold is set to 0.5.

[0079] Furthermore, the stable occupancy status output module 70 is used to perform the following operation steps:

[0080] The predetermined sliding window is used to smooth the state of the detection results of the consecutive multiple frames. If three consecutive frames are determined to be occupied, they are marked as stable occupancy. The window length of the predetermined sliding window is greater than or equal to five frames.

[0081] Furthermore, it also includes a model training module for performing the following steps:

[0082] A sample image dataset is collected, and the berth areas and berth occupancy status of the sample image dataset are labeled. The sample image dataset covers multiple environmental scenarios. The labeled sample image dataset is divided into training set, validation set and test set according to a predetermined ratio for model training. In the loss function design, weighted cross-entropy loss is used to increase the weight of occluded samples. After training, a lightweight deployment method is adopted, and the ViT model is compressed into a MobileViT architecture through knowledge distillation technology.

[0083] Through the foregoing detailed description of the end-to-end roadside parking detection method, those skilled in the art can clearly understand the end-to-end roadside parking detection device in this embodiment. Since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and relevant parts can be referred to the method section.

[0084] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. An end-to-end roadside parking detection method, characterized in that, The method includes: Panoramic images are acquired by using high-position cameras deployed along the roadside to obtain images of roadside parking scenes, wherein the roadside parking scene images cover multiple parking space areas; After preprocessing the roadside parking scene image, the parking space coordinates of the multiple parking space areas are calibrated to generate a parking space area coordinate dataset. Image features are extracted from the roadside parking scene image to generate an image embedding vector; Based on the berth area coordinate dataset, berth location encoding is performed to generate berth prompt vectors; After concatenating the image embedding vector with the berth prompt vector, feature interaction fusion is performed through a multi-head self-attention mechanism to obtain enhanced features for the berth area. Based on the enhanced features of the berth area, an end-to-end berth occupancy status is determined using a classifier. Based on the berth occupancy status judgment result and combined with the detection results of multiple consecutive frames, a predetermined sliding window is used for temporal smoothing processing to output the stable berth occupancy status.

2. The end-to-end roadside parking detection method as described in claim 1, characterized in that, The method for preprocessing the roadside parking scene image includes: After performing fisheye distortion correction on the roadside parking scene image, illumination equalization processing is performed. Based on the illumination equalization results, brightness compensation or contrast enhancement is performed on the shadow areas and low-light areas at night in the roadside parking scene image.

3. The end-to-end roadside parking detection method as described in claim 1, characterized in that, Based on the aforementioned berth area coordinate dataset, berth location encoding is performed to generate berth indication vectors, including: Based on the berth area coordinate dataset, the pixel position coordinates of the four corner points of the berth area are normalized and then converted into high-dimensional vectors through a learnable position encoding layer to generate the berth prompt vector.

4. The end-to-end roadside parking detection method as described in claim 1, characterized in that, Image feature extraction is performed on the roadside parking scene image to generate an image embedding vector, including: Using the ViT model, based on a predetermined segmentation size, the roadside parking scene image is segmented into multiple image blocks. After flattening the multiple image blocks, they are mapped to fixed-dimensional vectors through a linear layer to form a sequence input, generating the image embedding vector. The predetermined segmentation size is 16×16 pixels.

5. The end-to-end roadside parking detection method as described in claim 1, characterized in that, Enhanced features for the berth area are obtained through feature interaction fusion using a multi-head self-attention mechanism. The methods include: A cross-attention structure is adopted to interact the image embedding vector and the berth cue vector, dynamically adjust the feature weights, and focus on the features of the berth area. The berth cue vector is used as the query vector Q, and the key vector K and value vector V generated by the image embedding vector are used as the input for attention calculation. The attention weights are obtained by calculating the dot product similarity of Q and K and normalizing it with Softmax. Based on the attention weights, V is weighted and summed to generate an enhanced feature representation of the berth area, thus obtaining the enhanced features of the berth area.

6. The end-to-end roadside parking detection method as described in claim 5, characterized in that, The enhanced features of the berth area are represented as follows: ； Where Q is the Query, a query vector composed of berth suggestion vectors; K is the Key, a key vector calculated from the feature embedding of the image embedding vector; and V is the Value, a value vector representing the actual image feature information in the image embedding vector. Represents the feature dimension of the K vector. It is a square root function. square root Used for scaling The inner product result is used to stabilize the numerical range. This involves normalizing the similarity scores between Q and K into a probability distribution. Enhanced feature representation for focusing on berth areas.

7. The end-to-end roadside parking detection method as described in claim 1, characterized in that, Based on the enhanced features of the berth area, an end-to-end berth occupancy status determination is performed using a classifier, including: The classifier is used to perform feature analysis on the enhanced features of the berth area, outputting the berth occupancy status judgment result and simultaneously outputting the confidence score; False detections are filtered based on the confidence score according to a predetermined confidence threshold; The classifier uses the Sigmoid activation function, and the predetermined confidence threshold is set to 0.

5.

8. The end-to-end roadside parking detection method as described in claim 1, characterized in that, Combining the detection results of multiple consecutive frames, a predetermined sliding window is used for temporal smoothing to output the stable berth occupancy status, including: The predetermined sliding window is used to smooth the state of the detection results of the consecutive multiple frames. If three consecutive frames are determined to be occupied, they are marked as stable occupancy. The window length of the predetermined sliding window is greater than or equal to five frames.

9. The end-to-end roadside parking detection method as described in claim 1, characterized in that, Also includes: Collect a sample image dataset and label the berth areas and berth occupancy status of the sample image dataset. The sample image dataset covers a variety of environmental scenarios. The labeled sample image dataset is divided into training set, validation set and test set according to a predetermined ratio for model training. In the loss function design, weighted cross-entropy loss is used to increase the weight of occluded samples. After training, a lightweight deployment approach is adopted, and the ViT model is compressed into a MobileViT architecture through knowledge distillation technology.

10. An end-to-end roadside parking detection device, characterized in that, For implementing the end-to-end roadside parking detection method according to any one of claims 1-9, the apparatus comprises: A panoramic image acquisition module is used to acquire panoramic images through a high-position camera deployed on the roadside, thereby obtaining roadside parking scene images, wherein the roadside parking scene images cover multiple parking space areas; The parking space coordinate calibration module is used to preprocess the roadside parking scene image and then calibrate the parking space coordinates of the multiple parking space areas to generate a parking space area coordinate dataset. The image feature extraction module is used to extract image features from the roadside parking scene image and generate an image embedding vector; The berth location encoding module is used to encode the berth location based on the berth area coordinate dataset and generate a berth prompt vector. The feature interaction fusion module is used to concatenate the image embedding vector with the berth prompt vector, and then perform feature interaction fusion through a multi-head self-attention mechanism to obtain enhanced features of the berth area. The occupancy status determination module is used to determine the occupancy status of berths end-to-end based on the enhanced features of the berth area and through a classifier. The stable occupancy status output module is used to output the stable occupancy status of the berth based on the berth occupancy status judgment result, combined with the detection results of multiple consecutive frames, and using a predetermined sliding window for time-series smoothing.

Images