A charging parking space occupation identification method based on visual identification

Abstract

The application discloses a kind of based on visual recognition's charging parking stall occupancy identification method, belong to charging parking stall management technical field, this method is by camera collection parking stall image and pre-processing, after identification by target detection model, extract license plate color feature and vehicle depth feature, utilize attention mechanism fusion feature to identify vehicle type;Simultaneously combine continuous multiple frames of images to determine frame-by-frame occupancy rate and construct space-time occupancy track, obtain parking stall occupancy state;Again, identify vehicle charging state by double-path state information, finally fuse vehicle type, charging state and parking stall occupancy state, accurately determine charging parking stall occupancy identification result.The application effectively solves the problem that the prior art is prone to misjudgment, improves the accuracy and efficiency of charging parking stall management.

Description

Technical Field

[0001] This invention belongs to the field of charging parking space management technology, specifically relating to a method for identifying the occupancy of charging parking spaces based on visual recognition. Background Technology

[0002] With the rapid development of the new energy vehicle industry, the number of electric vehicles on the road continues to grow, making the demand for charging facilities increasingly urgent. However, the problem of charging spaces being occupied by non-charging vehicles has long existed, seriously affecting the utilization efficiency of charging facilities and the charging experience of car owners. The phenomena of gasoline vehicles occupying charging spaces and new energy vehicles not leaving promptly after charging not only waste charging resources but also increase the operation and management costs of charging stations.

[0003] In recent years, with the development of deep learning technology, vision-based charging parking space occupancy recognition schemes have gradually become mainstream. However, existing image recognition-based solutions typically use the static overlap area ratio of a single frame image for judgment, failing to consider the complete spatiotemporal process of a vehicle from entering to coming to a complete stop. This makes it impossible to distinguish between a vehicle entering and a vehicle already parked, posing a risk of momentary misjudgment. Furthermore, existing technologies use comparisons between the vehicle's center point coordinates and the marked area of ​​the parking space to determine occupancy. This method may miss occupancy when a vehicle is parked across multiple spaces, as the center point may be within one space. Additionally, in existing technologies, obtaining the charging status either relies on communication with the charging pile management system or on image recognition to detect the charging gun connection status. These two methods operate independently; if one signal is abnormal (e.g., communication failure, image obstruction), the system cannot detect the anomaly and may output an incorrect charging status judgment. Summary of the Invention

[0004] This invention provides a visual recognition-based method for identifying the occupancy of charging parking spaces, which solves the problem of misjudgment that easily occurs in the existing technology during the determination of charging parking spaces.

[0005] This invention provides a visual recognition-based method for identifying the occupancy of charging parking spaces, comprising: Parking space images are captured by cameras deployed in the charging parking space area, and the parking space images are preprocessed to obtain preprocessed parking space images. The preprocessed parking space image is input into a pre-trained target detection model for recognition, and the target detection result is output; the target detection result includes at least vehicle detection bounding boxes, vehicle detection confidence scores, license plate detection bounding boxes, and license plate detection confidence scores; Based on the vehicle detection frame, vehicle detection confidence, license plate detection frame, and license plate detection confidence, the license plate detection frame region and the vehicle detection frame region are determined in the preprocessed parking space image. The license plate color feature encoder is used to extract the license plate color feature corresponding to the license plate detection frame region, and the vehicle feature encoder is used to extract the vehicle depth feature corresponding to the vehicle detection frame region. An attention mechanism is used to obtain the cross-modal association weights between the license plate color features and the vehicle depth features, and the license plate color features and the vehicle depth features are fused according to the cross-modal association weights to obtain a fused feature vector; The fused feature vector is input into a pre-set vehicle type classification network for identification, and the vehicle type in the charging parking space is output. The frame-by-frame occupancy rate is determined based on the target detection results corresponding to multiple consecutive frames of parking space images. At the same time, a spatiotemporal occupancy trajectory is constructed, and the occupancy status of the charging parking space is obtained based on the frame-by-frame occupancy rate and the spatiotemporal occupancy trajectory. The system acquires dual-path status information and identifies the charging status based on the dual-path status information to determine the vehicle charging status. It then performs a fusion judgment based on the vehicle charging status, vehicle type, and charging space occupancy status to determine the charging space occupancy identification result.

[0006] In one possible implementation, parking space images are captured by cameras deployed in the charging parking space area, and the parking space images are preprocessed to obtain preprocessed parking space images, including: Images of the parking spaces are captured by cameras deployed in the charging area; The parking space image is subjected to distortion correction and brightness enhancement processing in sequence to obtain a preprocessed parking space image.

[0007] In one possible implementation, the target detection model employs the YOLOv8 model.

[0008] In one possible implementation, based on the vehicle detection bounding box, vehicle detection confidence score, license plate detection bounding box, and license plate detection confidence score, a license plate detection bounding box region and a vehicle detection bounding box region are determined in the preprocessed parking space image. A license plate color feature encoder is used to extract the license plate color features corresponding to the license plate detection bounding box region, and a vehicle feature encoder is used to extract the vehicle depth features corresponding to the vehicle detection bounding box region. This includes: When the vehicle detection confidence level is greater than a preset first confidence level threshold, the vehicle detection box area is cropped from the preprocessed parking space image. When the license plate detection confidence level is greater than a preset first confidence level threshold, the license plate detection box area is cropped from the preprocessed parking space image. The license plate color feature is extracted from the license plate detection frame area using a license plate color feature encoder, and the vehicle depth feature is extracted from the vehicle detection frame area using a vehicle feature encoder.

[0009] In one possible implementation, an attention mechanism is used to obtain the cross-modal association weights between the license plate color features and the vehicle depth features, and the license plate color features and vehicle depth features are fused according to the cross-modal association weights to obtain a fused feature vector, including: The attention weight of the license plate color feature to the vehicle depth feature is obtained to obtain the first cross-modal weight; the first cross-modal weight represents the degree of attention of the license plate color feature to the features of each region of the vehicle appearance; The attention weights of the vehicle depth features to the license plate color features are obtained to obtain the second cross-modal weights; the second cross-modal weights represent the degree of attention of the vehicle appearance features to the license plate color features; Based on the first cross-modal weights and the second cross-modal weights, the license plate color features and the vehicle depth features are concatenated to obtain a fused feature vector.

[0010] In one possible implementation, the vehicle type classification network is configured as a fully connected classification network or a backpropagation (BP) neural network.

[0011] In one possible implementation, determining the frame-by-frame occupancy rate based on the target detection results corresponding to multiple consecutive frames of parking space images includes: For any given frame of a parking space image, based on the spatial relationship between the parking space detection frame and the charging parking space markings, the ratio of the intersection area of ​​the rectangular area corresponding to the parking space detection frame and each charging parking space area to the area of ​​each charging parking space area is determined, thus obtaining the occupancy rate of each parking space. The occupancy rate is calculated for each consecutive frame of parking space images to obtain the frame-by-frame occupancy rate.

[0012] In one possible implementation, a spatiotemporal occupancy trajectory is constructed, and the occupancy status of the charging parking space is obtained based on the frame-by-frame occupancy rate and the spatiotemporal occupancy trajectory, including: For each preprocessed parking space image, the coordinate sequence of the four corner points of the vehicle detection box in the parking space coordinate system is recorded. The complete coordinate sequence from the moment the vehicle detection box first enters the parking space area to the moment it stops is extracted as the complete spatiotemporal occupancy trajectory of the vehicle. Based on the complete spatiotemporal occupancy trajectory, when the position change of the vehicle detection box in M ​​consecutive frames is less than the position change threshold, it is determined that the vehicle has come to a complete stop; the value of M ranges from 10 to 30 frames, and the value of the position change threshold ranges from 1% to 5% of the diagonal length of the vehicle detection box. When it is determined that the vehicle has come to a complete stop, the coordinates of the vehicle detection box corresponding to the last preprocessed frame of the parking space image are determined as the endpoint position. When the occupancy rate of the destination location exceeds the first preset threshold, the charging space is determined to be occupied; when the occupancy rate of the destination location is between the second preset threshold and the first preset threshold, the charging space is determined to be partially occupied; when the occupancy rate of the destination location and two or more charging spaces simultaneously exceeds the third preset threshold, the charging space is determined to be in a cross-space parking state; all other states are determined to be vacant. The first preset threshold is 60%, the second preset threshold is 30%, and the third preset threshold is 15%.

[0013] In one possible implementation, acquiring dual-path status information and identifying the charging status based on the dual-path status information to determine the vehicle charging status includes: Obtain the real-time electrical parameters of the charging pile at the charging parking space; the real-time electrical parameters of the charging pile include the output power; The parking space image is reacquired and preprocessed to obtain the target parking space image. The target parking space image is then identified to determine the physical connection status between the charging gun and the vehicle in the charging space. The physical connection status is either connected or not connected. When the output power is greater than zero and the physical connection status is connected, the vehicle charging status is determined to be normal charging. When the output power is zero and the physical connection status is connected, the vehicle charging status is determined to be that charging is complete. When the output power is zero and the physical connection status is not connected, the vehicle charging status is determined to be not charging. When the output power is greater than zero and the physical connection status is not connected, the vehicle charging status is determined to be an abnormal device status, and an abnormal device alarm message is generated.

[0014] In one possible implementation, a fusion determination is performed based on the vehicle charging status, vehicle type, and charging space occupancy status to determine the charging space occupancy identification result, including: When the parking space is vacant, output "Parking space vacant"; when the parking space is occupied, the charging status is "normal charging" and the vehicle type is a new energy vehicle, output "normal charging occupied"; when the parking space is occupied, the charging status is "charging complete" and the vehicle type is a new energy vehicle, output "fully charged and not moved," indicating illegal parking; when the parking space is occupied, the charging status is "not charging" and the vehicle type is a new energy vehicle occupied, indicating illegal parking; when the parking space is occupied, the charging status is "not charging" and the vehicle type is a gasoline vehicle, output "gasoline vehicle illegally occupied"; when the parking space is occupied and the charging status is "equipment malfunction," output "equipment malfunction alarm," obtaining the parking space occupancy identification result.

[0015] This invention provides a visual recognition-based method for identifying the occupancy of charging parking spaces. The method acquires and preprocesses parking space images using a camera. After identification by a target detection model, it extracts license plate color features and vehicle depth features, and uses an attention mechanism to fuse these features to identify the vehicle type. Simultaneously, it combines multiple consecutive frames of images to determine the frame-by-frame occupancy rate and construct a spatiotemporal occupancy trajectory to obtain the parking space occupancy status. Then, it identifies the vehicle charging status through dual-path status information, and finally fuses the vehicle type, charging status, and parking space occupancy status to accurately determine the charging parking space occupancy identification result. This invention effectively solves the problem of misjudgment that easily occurs in existing technologies, improving the accuracy and efficiency of charging parking space management. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0017] Figure 1 This is a flowchart of a visual recognition-based charging space occupancy identification method according to an embodiment of the present invention.

[0018] The accompanying drawings have illustrated specific embodiments of the invention, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0019] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0020] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0021] like Figure 1 As shown in the figure, an embodiment of the present invention provides a method for identifying the occupancy of charging parking spaces based on visual recognition, comprising: S101. Acquire parking space images using cameras deployed in the charging parking space area, and preprocess the parking space images to obtain preprocessed parking space images. S102. Input the preprocessed parking space image into a pre-trained target detection model for recognition and output the target detection result; the target detection result includes at least the vehicle detection box, vehicle detection confidence, license plate detection box, and license plate detection confidence. S103. Based on the vehicle detection frame, vehicle detection confidence, license plate detection frame, and license plate detection confidence, determine the license plate detection frame region and the vehicle detection frame region in the preprocessed parking space image, and use a license plate color feature encoder to extract the license plate color feature corresponding to the license plate detection frame region and use a vehicle feature encoder to extract the vehicle depth feature corresponding to the vehicle detection frame region. S104. An attention mechanism is used to obtain the cross-modal association weight between the license plate color feature and the vehicle depth feature, and the license plate color feature and the vehicle depth feature are fused according to the cross-modal association weight to obtain a fused feature vector. S105. Input the fused feature vector into a pre-set vehicle type classification network for identification, and output the vehicle type in the charging parking space. S106. Determine the frame-by-frame occupancy rate based on the target detection results corresponding to multiple consecutive frames of parking space images, construct a spatiotemporal occupancy trajectory, and obtain the occupancy status of the charging parking space based on the frame-by-frame occupancy rate and the spatiotemporal occupancy trajectory. S107. Obtain dual-path status information, identify the charging status based on the dual-path status information, determine the vehicle charging status, and perform a fusion judgment based on the vehicle charging status, vehicle type, and charging space occupancy status to determine the charging space occupancy identification result.

[0022] In one possible design approach, parking space images are captured by cameras deployed in the charging parking space area, and the parking space images are preprocessed to obtain preprocessed parking space images, including: Images of the parking spaces are captured by cameras deployed in the charging area; The parking space image is subjected to distortion correction and brightness enhancement processing in sequence to obtain a preprocessed parking space image.

[0023] Image distortion correction is used to eliminate image distortion caused by camera lens distortion; brightness enhancement is used to improve image quality under low-light conditions.

[0024] Optionally, perspective correction can be used for system calibration to ensure recognition accuracy. This perspective correction may include: during system initialization, solving the initial perspective transformation matrix based on the correspondence between the image coordinates and world coordinates of the four corner points of the parking space; during system operation, monitoring the confidence level of the parking space marking detection results in each frame; when the detection confidence level of the four corner points of the parking space is lower than the first confidence threshold for N consecutive frames, it is determined that the perspective transformation matrix has failed, and a recalibration process is automatically triggered; the recalibration process includes: based on the current frame image, re-detecting the image coordinates of the four corner points of the parking space, combining them with the pre-stored world coordinates, resolving the perspective transformation matrix, and updating the system parameters; the value of N ranges from 5 to 30 frames, and the value of the first confidence threshold ranges from 0.7 to 0.9. Through a dynamic adaptive correction mechanism, the problem of static perspective correction failing to automatically recover after a slight camera displacement in existing technologies is solved. It is worth noting that each charging parking space corner is marked with a corner marker.

[0025] By using a confidence-driven dynamic adaptive perspective correction method, when the camera undergoes a slight displacement due to external factors, the system can automatically detect and recalibrate the perspective transformation matrix, ensuring that the spatial determination accuracy is not affected. This solves the problem of accuracy failure in existing static perspective correction technologies after camera displacement.

[0026] For example, six 2-megapixel high-definition network cameras can be installed on the top of the charging station, each corresponding to one of the six charging spaces. The cameras are installed at a top-down angle, with the lens optical axis at a 15° angle to the ground normal, ensuring that each camera fully covers the corresponding parking space area and a 0.5-meter radius around it.

[0027] The camera captures images of the parking space in real time at a rate of 25 frames per second. The captured raw images are then processed sequentially for distortion correction (using the Zhang Zhengyou calibration method to obtain camera intrinsic parameters and distortion coefficients) and brightness enhancement (using an adaptive histogram equalization algorithm, with a contrast limit threshold of 3.0 and a histogram block size of 8×8).

[0028] Perspective correction employs a confidence-driven dynamic adaptive correction method, with the following specific parameters: During system initialization, calibration boards are placed at the four corners of the parking space, and the image coordinates (pixel coordinates) and world coordinates (physical coordinates in centimeters with the center of the parking space as the origin) of the four corners are recorded. The 3×3 perspective transformation matrix is ​​then solved. During system operation, the confidence of the parking space marking detection results in each frame is monitored. The number of consecutive frames is set to N=15 frames, and the first confidence threshold is 0.8. When the detection confidence of the four corners of the parking space is lower than 0.8 in 15 consecutive frames, the recalibration process is automatically triggered. The recalibration process uses the current frame image as a basis to re-detect the image coordinates of the four corners of the parking space, combines them with the pre-stored world coordinates of the four corners, and re-solves and updates the perspective transformation matrix.

[0029] In one possible design, the object detection model employs the YOLOv8 model. This YOLOv8 model has been trained on a charging parking space dataset containing 80,000 labeled images, covering various lighting and weather conditions such as sunny, cloudy, rainy, and nighttime, as well as diverse scenarios involving different vehicle types and parking postures. The model can simultaneously detect vehicles, license plates, and parking space markings.

[0030] The preprocessed image is input into a pre-trained YOLOv8 model, which simultaneously detects vehicle targets, license plate targets, and parking space marking targets in the image. Therefore, the output of the target detection model includes: vehicle detection boxes and their class confidence scores, license plate detection boxes and their class confidence scores, license plate number recognition results, and parking space marking detection boxes and their confidence scores. The license plate number recognition results are used for subsequent occupancy reminders, while the parking space marking detection boxes, which are essentially corner marker detection boxes, are mainly used to update the initial perspective transformation matrix. The perspective transformation matrix primarily transforms points in the image coordinate system to points in the parking space coordinate system. For the parking space coordinate system, a real coordinate system can be constructed using any point within the charging space or the center power supply as the origin.

[0031] In one possible design approach, based on the vehicle detection bounding box, vehicle detection confidence score, license plate detection bounding box, and license plate detection confidence score, a license plate detection bounding box region and a vehicle detection bounding box region are determined in the preprocessed parking space image. A license plate color feature encoder is then used to extract the license plate color features corresponding to the license plate detection bounding box region, and a vehicle feature encoder is used to extract the vehicle depth features corresponding to the vehicle detection bounding box region. This includes: When the vehicle detection confidence level is greater than a preset first confidence level threshold, the vehicle detection box area is cropped from the preprocessed parking space image. When the license plate detection confidence level is greater than a preset first confidence level threshold, the license plate detection box area is cropped from the preprocessed parking space image. The license plate color feature is extracted from the license plate detection frame area using a license plate color feature encoder, and the vehicle depth feature is extracted from the vehicle detection frame area using a vehicle feature encoder.

[0032] For example, the license plate detection bounding box region (normalized to 64×128 pixels) can be input into a license plate color feature encoder. This encoder can employ a convolutional neural network (such as a lightweight convolutional neural network consisting of 3 convolutional layers and 2 fully connected layers) to output 128-dimensional license plate color features. These license plate color features represent the RGB statistical values ​​of the license plate background color and an abstract feature representation of the color category.

[0033] The vehicle detection bounding box (normalized to 224×224 pixels) is input into a ResNet-50 convolutional neural network. The final fully connected layer is removed, and 2048-dimensional vehicle depth features are extracted. This network has been pre-trained on a dataset containing 50,000 vehicle front-end images. The vehicle depth features are used to represent abstract feature representations of the vehicle's grille region, logo region, and headlight design region. In one possible design approach, an attention mechanism is employed to obtain the cross-modal association weights between the license plate color features and the vehicle depth features. The license plate color features and the vehicle depth features are then fused based on these cross-modal association weights to obtain a fused feature vector, including: The attention weight of the license plate color feature to the vehicle depth feature is obtained to obtain the first cross-modal weight; the first cross-modal weight represents the degree of attention of the license plate color feature to the features of each region of the vehicle appearance; The attention weights of the vehicle depth features to the license plate color features are obtained to obtain the second cross-modal weights; the second cross-modal weights represent the degree of attention of the vehicle appearance features to the license plate color features; Based on the first cross-modal weights and the second cross-modal weights, the license plate color features and the vehicle depth features are concatenated to obtain a fused feature vector.

[0034] For example, the attention weights of the first feature vector (128-dimensional) on the second feature vector (2048-dimensional) can be calculated: A fully connected layer maps the 128-dimensional first feature vector to a 2048-dimensional attention score. After Softmax normalization, this score is multiplied element-wise with the second feature vector to obtain a weighted second feature vector. The attention weights of the second feature vector (2048-dimensional) on the first feature vector (128-dimensional) are then calculated: A fully connected layer maps the 2048-dimensional second feature vector to a 128-dimensional attention score. After Softmax normalization, this score is multiplied element-wise with the first feature vector to obtain a weighted first feature vector. The weighted first feature vector and the weighted second feature vector are then concatenated to generate a 2176-dimensional fused feature vector.

[0035] By fusing license plate color features and vehicle appearance depth features at the feature level through a cross-modal attention mechanism, true multimodal cross-validation is achieved, overcoming the shortcomings of existing technologies such as low efficiency of decision-level serial validation and inability to achieve true cross-validation.

[0036] In one possible design, the vehicle type classification network is configured as a fully connected classification network or a backpropagation (BP) neural network.

[0037] Preferably, a BP neural network can be used to output the vehicle type as either a new energy vehicle or a fuel vehicle.

[0038] In one possible design approach, the frame-by-frame occupancy rate is determined based on the target detection results corresponding to multiple consecutive frames of parking space images, including: For any given frame of parking space image, based on the spatial relationship between the parking space detection frame and the charging parking space markings, the ratio of the intersection area of ​​the rectangular area corresponding to the parking space detection frame and each charging parking space area to the area of ​​each charging parking space area is determined, thus obtaining the occupancy rate of each parking space. The occupancy rate is calculated for each consecutive frame of parking space images to obtain the frame-by-frame occupancy rate.

[0039] The intersection area is calculated using a polygon clipping algorithm, which involves: forming a rectangular polygon from the coordinates of the four corner points of the vehicle detection box, forming a parking space polygon from the pixel coordinates of the parking space markings, and then calculating the area of ​​the intersection polygon of the two polygons.

[0040] In one possible design approach, a spatiotemporal occupancy trajectory is constructed, and the occupancy status of the charging parking space is obtained based on the frame-by-frame occupancy rate and the spatiotemporal occupancy trajectory, including: For each preprocessed parking space image, the coordinate sequence of the four corner points of the vehicle detection box in the parking space coordinate system is recorded. The complete coordinate sequence from the moment the vehicle detection box first enters the parking space area to the moment it stops is extracted as the complete spatiotemporal occupancy trajectory of the vehicle. Based on the complete spatiotemporal occupancy trajectory, when the position change of the vehicle detection box in M ​​consecutive frames is less than the position change threshold, it is determined that the vehicle has come to a complete stop; the value of M ranges from 10 to 30 frames, and the value of the position change threshold ranges from 1% to 5% of the diagonal length of the vehicle detection box. When it is determined that the vehicle has come to a complete stop, the coordinates of the vehicle detection box corresponding to the last preprocessed frame of the parking space image are determined as the endpoint position. When the occupancy rate of the destination location exceeds the first preset threshold, the charging space is determined to be occupied; when the occupancy rate of the destination location is between the second preset threshold and the first preset threshold, the charging space is determined to be partially occupied; when the occupancy rate of the destination location and two or more charging spaces simultaneously exceeds the third preset threshold, the charging space is determined to be in a cross-space parking state; all other states are determined to be vacant. The first preset threshold is 60%, the second preset threshold is 30%, and the third preset threshold is 15%.

[0041] By constructing the spatiotemporal occupancy trajectory of vehicles, the occupancy status is determined based on the complete process from a vehicle entering to coming to a complete stop, effectively distinguishing between the two states of a vehicle entering and one that has already stopped and occupied a parking space, thus avoiding the instantaneous misjudgments caused by the single-frame static determination in existing technologies. The determination of crossing lines and exceeding parking space boundaries is based on the geometric relationship between the vehicle outline and the parking space boundary, without relying on the detection of specific color markings, overcoming the limitations of existing technologies that depend on the detection of specific color markings.

[0042] For example, for the vehicle detection box in each frame of the image, record the coordinates (unit: centimeters) of its four corner points in the perspective-corrected parking space coordinate system.

[0043] Set the number of consecutive frames M = 20 frames (corresponding to 0.8 seconds), and the position change threshold = 2% of the diagonal length of the vehicle detection box. When the position change of the vehicle detection box in 20 consecutive frames (the root mean square value of the changes in the coordinates of the four corner points) is less than 2% of the diagonal length, the vehicle is determined to have come to a stop.

[0044] Extract the entire coordinate sequence from the moment the vehicle detection box first enters any parking space area (with a frame occupancy rate greater than 5%) to the moment it comes to a stop. This coordinate sequence is typically 50 to 200 frames (corresponding to 2 to 8 seconds) and serves as the complete spatiotemporal occupancy trajectory of the vehicle.

[0045] If a perspective transformation matrix triggers recalibration during the spatiotemporal occupancy trajectory recording process, the current vehicle's spatiotemporal occupancy trajectory is abandoned and recording resumes. To ensure the stability of the detection process, the perspective transformation matrix can be recalibrated only if no vehicle is detected in the charging parking space.

[0046] In one possible design approach, dual-path state information is acquired, and charging status is identified based on the dual-path state information to determine the vehicle charging status, including: The system obtains the real-time electrical parameters of the charging piles in the charging spaces; the real-time electrical parameters of the charging piles include the output power; it can communicate with the charging pile management system via TCP / IP protocol and obtain the real-time electrical parameters of the charging piles for the corresponding parking spaces every 2 seconds: output power (unit: kW).

[0047] The parking space image is reacquired and preprocessed to obtain the target parking space image. The target parking space image is then identified to determine the physical connection status between the charging gun and the vehicle in the charging space. The physical connection status is either connected or not connected. Visual recognition technology can be used to check whether the outline of the charging gun plug exists in the vehicle detection frame (e.g., whether the vehicle detection frame intersects with the charging gun plug detection frame or the distance is less than a preset distance threshold), and output whether it is connected or not connected.

[0048] When the output power is greater than zero and the physical connection status is connected, the vehicle charging status is determined to be normal charging. When the output power is zero and the physical connection status is connected, the vehicle charging status is determined to be that charging is complete. When the output power is zero and the physical connection status is not connected, the vehicle charging status is determined to be not charging. When the output power is greater than zero and the physical connection status is not connected, the vehicle charging status is determined to be an abnormal device status, and an abnormal device alarm message is generated.

[0049] By comparing electrical parameters with physical connection status, not only is redundant acquisition of charging status achieved, but also automatic diagnosis of abnormal device status is realized. When there is power output but no connection, an abnormal device alarm is triggered.

[0050] For example, when the output power is greater than 0.1kW and the physical connection status is connected, it is determined to be charging normally; when the output power is less than or equal to 0.1kW and the physical connection status is connected, it is determined to be charging completed; when the output power is less than or equal to 0.1kW and the physical connection status is disconnected, it is determined to be not charging; when the output power is greater than 0.1kW and the physical connection status is disconnected, it is determined to be an abnormal device status, and an abnormal device alarm message is generated: "Equipment in parking space A-03 is abnormal. There is power output but the charging gun is not connected. Please check the charging pile and communication module."

[0051] In one possible design approach, a fusion determination is performed based on the vehicle's charging status, vehicle type, and the occupancy status of the charging space to determine the charging space occupancy identification result, including: When the parking space is vacant, output "Parking space vacant"; when the parking space is occupied, the charging status is "normal charging" and the vehicle type is a new energy vehicle, output "normal charging occupied"; when the parking space is occupied, the charging status is "charging complete" and the vehicle type is a new energy vehicle, output "fully charged and not moved," indicating illegal parking; when the parking space is occupied, the charging status is "not charging" and the vehicle type is a new energy vehicle occupied, indicating illegal parking; when the parking space is occupied, the charging status is "not charging" and the vehicle type is a gasoline vehicle, output "gasoline vehicle illegally occupied"; when the parking space is occupied and the charging status is "equipment malfunction," output "equipment malfunction alarm," obtaining the parking space occupancy identification result.

[0052] Optionally, if the duration of unauthorized occupation exceeds a preset time threshold, the system will search for the user account and / or mobile phone number corresponding to the license plate number detection result, and send a "Pending release of occupation" prompt message through the user account and / or mobile phone number.

[0053] Optionally, cameras in the parking area can be used to detect the number of vehicles moving at low speed or stopped at the entrance of the charging station and in the passageway, which can be used as a reference for the number of vehicles in the queue. Specific detection methods include: Vehicle detection is performed on images captured by cameras to identify all vehicles within the charging station area; For each detected vehicle, its motion state is determined by the position change of consecutive frames. If the position change is less than a preset motion threshold, it is determined to be in a low-speed driving or stopped state. The number of vehicles at the entrance and in the passageway of the charging station that are moving at low speed or stopped is counted as the number of vehicles in the queue.

[0054] When the overall judgment result is that the space is occupied illegally, a timer is started to record the duration of the space occupation.

[0055] Based on the current time period, parking space location, and number of vehicles in the queue, the occupancy tolerance time threshold is dynamically calculated using the following formula: ; Where: T is the occupancy tolerance time threshold, in seconds; This is the base time threshold, in seconds. The value ranges from 300 seconds to 600 seconds; This is a time-period coefficient, dimensionless, used when the current time falls within the preset peak time period. =0.6, otherwise =1.0, the peak hours include 7:00 to 9:00 am and 5:00 to 7:00 pm on weekdays; This is a position coefficient, dimensionless, used when the parking space is located in a preset priority area. =0.7, otherwise =1.0, the priority area includes parking spaces near the entrance of the charging station and parking spaces where the rated power of the charging pile is higher than the average power; The queuing coefficient is dimensionless. =1 / (1+α×Q), where Q is the number of vehicles in the queue, Q is a non-negative integer, and α is a preset proportional coefficient, with a value ranging from 0.1 to 0.3. When the occupancy duration exceeds the tolerance time threshold, a graded alarm message is generated according to the degree of timeout: Let D be the occupancy duration, T be the tolerance time threshold, and the timeout multiple r = D / T: When 1≤r<2, a level 1 alarm is triggered, and the vehicle owner is notified via application push; when 2≤r<3, a level 2 alarm is triggered, and the vehicle owner is notified via application push and SMS; when r≥3, a level 3 alarm is triggered, and the vehicle owner is notified via application push and SMS, and management personnel are notified for on-site handling. The alarm message includes: parking space number, vehicle license plate number, vehicle type, occupancy start time, occupancy duration, occupancy type, and current alarm level.

[0056] By dynamically calculating the tolerance time threshold based on multiple factors (time period, parking space location, number of vehicles in the queue), a differentiated alarm strategy is achieved, enabling rapid response during peak hours and reasonable tolerance during off-peak hours, overcoming the shortcomings of existing fixed threshold strategies. The tiered alarm mechanism ensures that different degrees of parking space hogging are handled differently.

[0057] The system continuously monitors parking space images and determines vehicle movement by observing changes in the vehicle detection box's position across consecutive frames. When the center point of the vehicle detection box moves out of the parking space area and does not return to the area for 15 consecutive frames, the vehicle is considered to have left, and the departure time is recorded. Based on the entry and departure times, the vehicle's occupancy time is calculated, and the parking space status is updated to vacant.

[0058] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. It should be understood that the invention is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for identifying the occupancy of charging parking spaces based on visual recognition, characterized in that, include: Parking space images are captured by cameras deployed in the charging parking space area, and the parking space images are preprocessed to obtain preprocessed parking space images. The preprocessed parking space image is input into a pre-trained object detection model for recognition, and the object detection result is output. The target detection results include at least the vehicle detection bounding box, the vehicle detection confidence score, the license plate detection bounding box, and the license plate detection confidence score. Based on the vehicle detection frame, vehicle detection confidence, license plate detection frame, and license plate detection confidence, the license plate detection frame region and the vehicle detection frame region are determined in the preprocessed parking space image. The license plate color feature encoder is used to extract the license plate color feature corresponding to the license plate detection frame region, and the vehicle feature encoder is used to extract the vehicle depth feature corresponding to the vehicle detection frame region. An attention mechanism is used to obtain the cross-modal association weights between the license plate color features and the vehicle depth features, and the license plate color features and the vehicle depth features are fused according to the cross-modal association weights to obtain a fused feature vector; The fused feature vector is input into a pre-set vehicle type classification network for identification, and the vehicle type in the charging parking space is output. The frame-by-frame occupancy rate is determined based on the target detection results corresponding to multiple consecutive frames of parking space images. At the same time, a spatiotemporal occupancy trajectory is constructed, and the occupancy status of the charging parking space is obtained based on the frame-by-frame occupancy rate and the spatiotemporal occupancy trajectory. The system acquires dual-path status information and identifies the charging status based on the dual-path status information to determine the vehicle charging status. It then performs a fusion judgment based on the vehicle charging status, vehicle type, and charging space occupancy status to determine the charging space occupancy identification result.

2. The charging space occupancy identification method based on visual recognition according to claim 1, characterized in that, Parking space images are captured by cameras deployed in the charging parking space area, and the parking space images are preprocessed to obtain preprocessed parking space images, including: Images of the parking spaces are captured by cameras deployed in the charging area; The parking space image is subjected to distortion correction and brightness enhancement processing in sequence to obtain a preprocessed parking space image.

3. The charging parking space occupancy identification method based on visual recognition according to claim 1, characterized in that, The target detection model used is the YOLOv8 model.

4. The charging space occupancy identification method based on visual recognition according to claim 1, characterized in that, Based on the vehicle detection bounding box, vehicle detection confidence score, license plate detection bounding box, and license plate detection confidence score, a license plate detection bounding box region and a vehicle detection bounding box region are determined in the preprocessed parking space image. A license plate color feature encoder is used to extract the license plate color features corresponding to the license plate detection bounding box region, and a vehicle feature encoder is used to extract the vehicle depth features corresponding to the vehicle detection bounding box region, including: When the vehicle detection confidence level is greater than a preset first confidence level threshold, the vehicle detection box area is cropped from the preprocessed parking space image. When the license plate detection confidence level is greater than a preset first confidence level threshold, the license plate detection box area is cropped from the preprocessed parking space image. The license plate color feature is extracted from the license plate detection frame area using a license plate color feature encoder, and the vehicle depth feature is extracted from the vehicle detection frame area using a vehicle feature encoder.

5. The charging space occupancy identification method based on visual recognition according to claim 1, characterized in that, An attention mechanism is used to obtain the cross-modal association weights between the license plate color features and the vehicle depth features. The license plate color features and the vehicle depth features are then fused based on these cross-modal association weights to obtain a fused feature vector, including: The attention weight of the license plate color feature to the vehicle depth feature is obtained to obtain the first cross-modal weight; the first cross-modal weight represents the degree of attention of the license plate color feature to the features of each region of the vehicle appearance; The attention weights of the vehicle depth features to the license plate color features are obtained to obtain the second cross-modal weights; the second cross-modal weights represent the degree of attention of the vehicle appearance features to the license plate color features; Based on the first cross-modal weights and the second cross-modal weights, the license plate color features and the vehicle depth features are concatenated to obtain a fused feature vector.

6. The charging space occupancy identification method based on visual recognition according to claim 5, characterized in that, The vehicle type classification network is set as a fully connected classification network or a backpropagation neural network.

7. The charging space occupancy identification method based on visual recognition according to claim 1, characterized in that, The frame-by-frame occupancy rate is determined based on the target detection results corresponding to multiple consecutive frames of parking space images, including: For any given frame of parking space image, based on the spatial relationship between the parking space detection frame and the charging parking space markings, the ratio of the intersection area of ​​the rectangular area corresponding to the parking space detection frame and each charging parking space area to the area of ​​each charging parking space area is determined, thus obtaining the occupancy rate of each parking space. The occupancy rate is calculated for each consecutive frame of parking space images to obtain the frame-by-frame occupancy rate.

8. The charging parking space occupancy identification method based on visual recognition according to claim 7, characterized in that, Constructing a spatiotemporal occupancy trajectory, and obtaining the occupancy status of charging spaces based on the frame-by-frame occupancy rate and the spatiotemporal occupancy trajectory, including: For each preprocessed parking space image, the coordinate sequence of the four corner points of the vehicle detection box in the parking space coordinate system is recorded. The complete coordinate sequence from the moment the vehicle detection box first enters the parking space area to the moment it stops is extracted as the complete spatiotemporal occupancy trajectory of the vehicle. Based on the complete spatiotemporal occupancy trajectory, when the position change of the vehicle detection box in M ​​consecutive frames is less than the position change threshold, it is determined that the vehicle has come to a complete stop; the value of M ranges from 10 to 30 frames, and the value of the position change threshold ranges from 1% to 5% of the diagonal length of the vehicle detection box. When it is determined that the vehicle has come to a complete stop, the coordinates of the vehicle detection box corresponding to the last preprocessed frame of the parking space image are determined as the endpoint position. When the occupancy rate of the destination location exceeds the first preset threshold, the charging space is determined to be occupied; when the occupancy rate of the destination location is between the second preset threshold and the first preset threshold, the charging space is determined to be partially occupied; when the occupancy rate of the destination location and two or more charging spaces simultaneously exceeds the third preset threshold, the charging space is determined to be in a cross-space parking state; all other states are determined to be vacant. The first preset threshold is 60%, the second preset threshold is 30%, and the third preset threshold is 15%.

9. The charging parking space occupancy identification method based on visual recognition according to claim 8, characterized in that, Acquire dual-channel status information and identify the charging status based on the dual-channel status information to determine the vehicle charging status, including: Obtain the real-time electrical parameters of the charging pile at the charging parking space; the real-time electrical parameters of the charging pile include the output power; The parking space image is reacquired and preprocessed to obtain the target parking space image. The target parking space image is then identified to determine the physical connection status between the charging gun and the vehicle in the charging space. The physical connection status is either connected or not connected. When the output power is greater than zero and the physical connection status is connected, the vehicle charging status is determined to be normal charging. When the output power is zero and the physical connection status is connected, the vehicle charging status is determined to be that charging is complete. When the output power is zero and the physical connection status is not connected, the vehicle charging status is determined to be not charging. When the output power is greater than zero and the physical connection status is not connected, the vehicle charging status is determined to be an abnormal device status, and an abnormal device alarm message is generated.

10. The charging parking space occupancy identification method based on visual recognition according to claim 9, characterized in that, The charging space occupancy identification result is determined by fusing the vehicle charging status, vehicle type, and charging space occupancy status, including: When the parking space is vacant, output "Parking space vacant"; when the parking space is occupied, the charging status is "normal charging" and the vehicle type is a new energy vehicle, output "normal charging occupied"; when the parking space is occupied, the charging status is "charging complete" and the vehicle type is a new energy vehicle, output "fully charged and not moved," indicating illegal parking; when the parking space is occupied, the charging status is "not charging" and the vehicle type is a new energy vehicle occupied, indicating illegal parking; when the parking space is occupied, the charging status is "not charging" and the vehicle type is a gasoline vehicle, output "gasoline vehicle illegally occupied"; when the parking space is occupied and the charging status is "equipment malfunction," output "equipment malfunction alarm," obtaining the parking space occupancy identification result.

Images