Method, system, device and medium for identifying and warning of charging area entry into a vehicle

Abstract

The application relates to the technical field of data processing, in particular to a charging area entering vehicle identification and early warning method, system, device and medium, wherein the method comprises the following steps: continuously collecting a video stream based on a preset position camera device; outputting vehicle boundary box coordinates through real-time detection of the video stream based on an edge or cloud device; judging whether a vehicle enters a preset intersection area according to the overlap ratio of the boundary box and the intersection area; starting algorithm tracking for the vehicle entering the intersection area, judging whether the vehicle moves towards a core area based on the movement trend of the vehicle; if the vehicle moves towards the core area, identifying the vehicle type of the vehicle; and if the vehicle type is a preset vehicle type, triggering an early warning mechanism. The application can effectively solve the problem of preventing fuel vehicles from occupying charging parking spaces.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a charging area entering vehicle identification and early warning method, system, device and medium. BACKGROUND

[0002] With the rapid development of new energy vehicles, the construction of charging infrastructure is increasingly perfect. However, in the actual operation process, the phenomenon of misentry or occupation of electric vehicle dedicated charging parking spaces by fuel vehicles is relatively common, which seriously affects the effective use of charging resources and the use experience of electric vehicle owners. In view of this problem, the existing technology usually adopts a method based on video monitoring and simple license plate recognition to manage and monitor the vehicles in the charging station. These systems capture vehicle images through cameras and distinguish different types of vehicles using license plate recognition technology. However, the existing license plate recognition system can only collect and statically identify license plate information in the fixed area of the charging station. When the corresponding fuel vehicle is identified, the vehicle is often already in the charging station or even has occupied the corresponding charging parking space, resulting in chaotic vehicle management in the corresponding area of the charging station and difficulty in effectively solving the problem of oil vehicle occupation.

[0003] Therefore, the existing method uses static identification of the vehicle type in the fixed position of the charging pile, which cannot effectively prevent the technical problem of fuel vehicle occupation of the charging parking space, and needs to be solved urgently. SUMMARY

[0004] The main purpose of the present application is to provide a charging area entering vehicle identification and early warning method, system, device and medium, which aims to solve the technical problem that the existing method uses static identification of the vehicle type in the fixed position of the charging pile, which cannot effectively prevent the fuel vehicle from occupying the charging parking space.

[0005] In order to achieve the above-mentioned purpose of the application, the present application provides a charging area entering vehicle identification and early warning method, which comprises:

[0006] The camera device based on the preset position continuously collects video streams;

[0007] The edge or cloud device performs real-time detection on the video stream to output vehicle bounding box coordinates;

[0008] According to the overlap ratio of the bounding box and the preset intersection area, it is judged whether the vehicle enters the intersection area;

[0009] The algorithm tracking is started for the vehicle entering the intersection area, and it is judged whether the vehicle moves towards the core area based on the movement trend of the vehicle;

[0010] If the vehicle moves towards the core area, the vehicle type of the vehicle is identified;

[0011] If the vehicle type is a preset vehicle type, a warning mechanism is triggered.

[0012] Further, before the step of continuously capturing a video stream based on the preset position of the camera device, the method comprises:

[0013] Obtaining virtual map data of the current use scenario from the cloud;

[0014] Identifying geographical information of each functional area in the virtual map data;

[0015] Based on the geographical information, identifying a charging pile parking area, a charging pile parking area access intersection area, and corresponding coordinate ranges, wherein the charging pile parking area access intersection area is the intersection area, and the charging pile parking area is the core area;

[0016] Based on the current camera device installation height and coordinate parameters, calculating an optimal capture angle;

[0017] Based on the optimal capture angle, automatically adjusting the camera to a specified position to obtain a preset position of the camera device.

[0018] Further, before the step of outputting vehicle bounding box coordinates based on real-time detection of the video stream by the edge or cloud device, the method comprises:

[0019] Obtaining position parameters of the camera, and three-dimensional coordinate ranges of the intersection area and the core area;

[0020] According to the camera's intrinsic and extrinsic parameters, calculating a projection matrix for converting a three-dimensional world coordinate system to a two-dimensional image coordinate system;

[0021] Using the calculated projection matrix, converting the three-dimensional coordinates of the intersection area and the core area to two-dimensional image coordinate ranges;

[0022] According to the coordinates of the core area and the intersection area in the two-dimensional image, determining the relative positional relationship between the core area and the intersection area.

[0023] Further, the step of outputting vehicle bounding box coordinates based on real-time detection of the video stream by the edge or cloud device comprises:

[0024] Loading a pre-trained vehicle detection model on the edge or cloud device;

[0025] Using the loaded vehicle detection model to analyze each frame of image, identifying all vehicles, and outputting the bounding box coordinates (x1, y1, x2, y2) of each vehicle, wherein (x1, y1) is the top-left corner coordinate, and (x2, y2) is the bottom-right corner coordinate.

[0026] Further, the step of determining whether the vehicle is moving towards the core area based on the moving trend of the vehicle in the vehicle entering the intersection area starting algorithm tracking comprises:

[0027] If a vehicle is identified to enter the intersection area, a unique ID is assigned to each vehicle entering the intersection area based on a target tracking algorithm, and the trajectory coordinates of the corresponding vehicle are recorded to obtain corresponding trajectory data;

[0028] According to the trajectory data, the speed and acceleration of the vehicle are calculated, and the corresponding travel direction of the vehicle is analyzed;

[0029] The future moving trend of the vehicle is predicted using historical trajectory data combined with the speed and acceleration of the current vehicle and the corresponding travel direction of the vehicle;

[0030] If the prediction result shows that the vehicle is moving towards the core area with a probability exceeding a first preset threshold, it is determined that the vehicle is moving towards the core area.

[0031] Further, the step of identifying the vehicle type of the vehicle if the vehicle is moving towards the core area comprises:

[0032] If the vehicle is moving towards the core area, a license plate recognition algorithm is called to identify the number information and color information of the license plate of the vehicle;

[0033] The vehicle type of the vehicle is determined based on the number information and color information,

[0034] It is determined whether the vehicle type of the vehicle is a first preset vehicle type, wherein the first preset vehicle type is a fuel vehicle.

[0035] Further, the step of triggering a warning mechanism if the vehicle type is a preset vehicle type comprises:

[0036] If the vehicle type is a preset vehicle type, the vehicle information corresponding to the vehicle is obtained;

[0037] The corresponding warning notification is generated combined with the vehicle information and sent to a broadcast device;

[0038] The vehicle is warned through the broadcast device.

[0039] The second aspect of the present application proposes an identification and warning system for vehicles entering a charging area, comprising:

[0040] A data acquisition module is configured to continuously acquire video streams based on a camera device at a preset position;

[0041] The coordinate detection module is configured to detect the video stream in real time based on an edge or a cloud device to output a vehicle bounding box coordinate;

[0042] The entering judgment module is configured to judge whether a vehicle enters a cross region according to an overlap ratio of a bounding box and a preset cross region.

[0043] The trend judgment module is configured to start algorithm tracking for the vehicle entering the cross region, and judge whether the vehicle moves towards a core region based on a moving trend of the vehicle.

[0044] The type identification module is configured to identify a vehicle type of the vehicle if the vehicle moves towards the core region.

[0045] The early warning triggering module is configured to trigger an early warning mechanism if the vehicle type is a preset vehicle type. The third aspect of the present application also provides a device including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in any of the above embodiments when executing the computer program.

[0046] The fourth aspect of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the method in any of the above embodiments when executed by a processor.

[0047] Advantages

[0048] By means of video acquisition, vehicle detection, target tracking and trajectory analysis, intelligent identification and behavior prediction of vehicles entering the charging station are realized. The method can analyze the moving trend of the vehicle in real time before the vehicle enters the core charging region, and judge whether it is a preset type such as a fuel vehicle by combining license plate recognition technology, so as to trigger the early warning mechanism in time when the vehicle is about to enter the core region, effectively preventing non-target vehicles from occupying charging resources. Compared with the existing technology which only relies on static license plate recognition or manual intervention, the present scheme has higher real-time and automation degree, significantly improves the identification accuracy and response efficiency. At the same time, through continuous tracking and direction prediction of the vehicle trajectory, the system can make early warning decisions before the vehicle causes resource occupation, reducing the management lag problem and improving the operation efficiency and service quality of the charging station. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 A flowchart of a charging region entering vehicle identification and early warning method according to an embodiment of the present application;

[0050] Figure 2 A structural schematic block diagram of a charging region entering vehicle identification and early warning system according to an embodiment of the present application;

[0051] Figure 3 Fig. 1 is a structural schematic block diagram of a computer device according to an embodiment of the present application;

[0052] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0053] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application.

[0054] Those skilled in the art can understand that the singular forms "a", "an" and "the" used herein include plural forms unless specifically stated otherwise. It should be further understood that the use of the term "include" in the specification of the present application means that a feature, integer, step, operation, element, module and / or assembly exists, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, modules, assemblies and / or combinations thereof. It should be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there can be an intermediate element. In addition, "connected" or "coupled" used herein can include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any combination of the associated listed items.

[0055] Those skilled in the art can understand that unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have meanings consistent with those in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as such.

[0056] Reference Figure 1 The embodiment of the present application provides a recognition and warning method for a charging area entering a vehicle, which comprises steps S1-S6, specifically:

[0057] S1, a video stream is continuously collected based on a camera device at a preset position;

[0058] S2, a vehicle bounding box coordinate is output based on real-time detection of the video stream by an edge or cloud device;

[0059] S3, whether a vehicle enters a cross region is judged according to an overlap ratio of the bounding box and the cross region;

[0060] S4, start algorithm tracking for the vehicle entering the intersection area, determine whether the vehicle is moving towards the core area based on the movement trend of the vehicle;

[0061] S5, if the vehicle is moving towards the core area, identify the vehicle type of the vehicle;

[0062] S6, if the vehicle type is a preset vehicle type, trigger the early warning mechanism.

[0063] In step S1, in a specific implementation, first, a suitable camera device needs to be selected and its installation position and angle need to be determined to ensure that the intersection area and the core area of the charging station can be covered. Considering the need for all-weather monitoring, the camera should have good low-light performance, waterproof and dustproof capability, and be able to adapt to different climate conditions. Among them, the intersection area refers to the intersection road area passing through the charging station, which is the buffer zone before the vehicle enters the core area; the core area refers to the parking area provided with charging piles. The core area can be within the current monitoring range or can be partially covered by the current monitoring range.

[0064] In terms of hardware, a network camera supporting high-definition resolution (such as 1080P or higher) can be selected to provide clear image quality, which is particularly important for subsequent vehicle detection and license plate recognition. In addition, in order to improve the response speed of the system to sudden events, the camera can be directly connected to the edge computing device, so that preliminary data processing can be performed locally to reduce latency. Of course, according to the actual situation, the video stream can also be transmitted to the cloud server for processing.

[0065] Regarding the collection of video streams, modern cameras usually support H.264 or H.265 encoding formats, which help to reduce bandwidth occupation and storage space requirements while maintaining video quality. The camera will continuously take pictures at a set time interval (such as 30 frames per second) and convert them into digital signals to send to the processing unit.

[0066] For example, in a typical charging station scenario, suppose we want to monitor the entrance of the charging station, which is the intersection point between the intersection area and the outside world. Set up high-definition network cameras facing the entering direction and the leaving direction respectively to ensure that there is no blind area. The cameras are installed at a higher position to obtain a wider field of view while avoiding being blocked by passing vehicles. These cameras are connected to the edge computing device through wired or wireless networks, which is responsible for receiving real-time video streams from the cameras and preparing to pass them to the algorithms in the next step for processing.

[0067] As described in step S2 above, first, a computer vision algorithm is selected to perform the vehicle detection task. For example, a YOLOv11 model is used. YOLO (You Only Look Once) is a real-time object detection algorithm that can provide fast detection speed while maintaining high accuracy, making it very suitable for applications such as this that require real-time processing of large video data. After receiving a video frame, the YOLOv11 model outputs the bounding box coordinates of each detected vehicle in its image, including the upper left corner coordinates (x1, y1) and the lower right corner coordinates (x2, y2), which will be used for subsequent position calculation and area judgment.

[0068] In the specific operation process, when the video stream is transmitted to the edge computing device or cloud server, it first needs to be decoded and converted into a format suitable for algorithm processing. Then, by calling the pre-trained YOLOv11 model, each frame of image is analyzed to identify all vehicles and generate a bounding box for each vehicle.

[0069] As shown in step S3 above, first, the map information of the charging station needs to be integrated with the video monitoring system. This includes identifying the coordinate ranges of the intersection area and the core area on the map. These geographic information can be obtained through GPS or other positioning technology and can be updated in real time to reflect any physical changes. In order to ensure that the video stream obtained from the camera can match the geographic information on the map, the camera needs to be calibrated. This process involves determining the real position, angle and field of view range of the camera relative to the ground. The height and angle can be obtained through the built-in gyroscope or angle and height sensor of the camera. In this way, the screen coordinates captured by the camera can be converted into geographic coordinates. Thus, it is determined whether the corresponding intersection area or core area is in the current video image and the specific coordinate position in the video image.

[0070] After the vehicle bounding box is detected, the system compares its corresponding geographic coordinates with the pre-set intersection area coordinates on the map. The key here is to calculate the overlap ratio between the screen area of the vehicle bounding box and the intersection area. If this ratio exceeds the set threshold, the subsequent target tracking process is triggered.

[0071] As described in step S4 above, when a vehicle first enters the intersection area, its bounding box coordinates, timestamp, ID, and other information are input into the tracker, and its trajectory begins to be recorded; after each frame of video processing is completed, the tracker updates the current vehicle's position coordinates (x, y), speed vector (vx, vy), and angular acceleration change. The trajectory points are smoothed by a Kalman filter to reduce the influence of jitter caused by occlusion or false detection. Then, based on the trajectory points of several consecutive frames, the vehicle's direction vector is calculated and compared with the orientation of the core area obtained from the map. If the angle between the vehicle's direction vector and the direction of the path leading to the core area is less than a preset angle threshold (such as 30 degrees), and its speed vector points to the core area, then it is determined that the vehicle has a clear tendency to enter. Suppose a gasoline vehicle is coming from the intersection area, the system detects its bounding box and confirms its entry into the intersection area, immediately assigns it an ID and starts tracking. As the vehicle advances, the system continuously updates its position and draws a motion trajectory. Through analysis of the trajectory points, it is found that the vehicle is driving along a direction that leads directly to the charging pile, and the speed is stable, at which point the system determines that the vehicle has a clear tendency to enter the core area.

[0072] As described in step S5 above, once the system determines that the vehicle has a tendency to enter the core area, the license plate recognition algorithm is activated. This includes using OCR (Optical Character Recognition) technology to read and analyze license plate information. Different countries and regions have different regulations on license plate colors, for example, blue plates may represent gasoline vehicles, and green plates represent new energy vehicles. By analyzing the color of the license plate, the system can obtain preliminary clues about the type of vehicle. However, considering that there may be exceptions or local differences, relying solely on license plate color is not enough to completely determine the type of vehicle. In order to more accurately identify the type of vehicle, the system can also access the database to query the detailed registration information of the vehicle corresponding to the license plate number. Based on the data of license plate color and registration information, the system will make a final judgment on the type of vehicle. If the license plate shows a gasoline vehicle, the vehicle is marked as a potential violator; if it is an electric vehicle, it is allowed to continue to the charging pile location.

[0073] As described in step S6 above, the identification of the vehicle type has been completed in S5, including license plate color analysis and license plate registration information query. If the system determines that the vehicle belongs to a pre-set specific type (such as a fuel vehicle), it is ready to trigger the early warning mechanism. Warning is directly given to the driver through display screens, voice broadcast devices or light signals installed on site. This way can immediately attract the attention of the driver and guide him to take appropriate action. The violation information is sent to the charging station managers or relevant monitoring center, so that they can respond quickly. This can be achieved in the form of SMS, in-app message push or email, etc. Linkage with parking lot management system or other intelligent traffic facilities, implement more stringent control measures, such as intelligent robot interception, etc. When the vehicle type is confirmed as the pre-set type, the system will automatically generate a warning instruction. According to the pre-configured warning strategy, the appropriate warning method can be selected and executed. For example, the on-site warning light is enabled to flash and the voice prompt is played: "This area is for electric vehicles only, please leave as soon as possible." If remote notification is selected, the relevant information needs to be packaged and sent to the designated recipients through the network. All operation logs will be recorded for subsequent review.

[0074] In an embodiment, before the step of continuously collecting video stream based on the preset position of the camera device, the method comprises:

[0075] S10, obtaining virtual map data of the current use scenario from the cloud;

[0076] S11, identifying the geographical information of each functional area in the virtual map data;

[0077] S12, identifying the charging pile parking area and the charging pile parking area access intersection area and the corresponding coordinate range based on the geographical information, wherein the charging pile parking area access intersection area is the intersection area, and the charging pile parking area is the core area;

[0078] S13, calculating the optimal collection angle based on the current camera device installation height and coordinate parameters;

[0079] S14, automatically adjusting the camera to the specified position based on the optimal collection angle to obtain the preset position of the camera device.

[0080] In this embodiment, in order to ensure that the camera device can accurately cover the key areas of the charging station, before starting to collect the video stream, the system first obtains the virtual map data of the current use scenario from the cloud. This process involves interacting with the cloud database using geographic information system (GIS) technology to download map information containing the detailed layout of the charging station. These data not only include the specific location of the charging pile parking area, but also include the geographical coordinate range of important functional areas such as the access intersection area.

[0081] Next, the system needs to identify the geographical information of each functional area in the virtual map data. By analyzing the markers and metadata on the map, the system can accurately extract the location coordinates of the charging pile parking area and the intersection area of the charging pile parking area access. Special attention is paid to the intersection area, which is the road intersection point that vehicles must pass before entering the core charging area, and the core area itself, which is the parking area with charging piles. This step ensures that subsequent monitoring and early warning mechanisms for vehicle behavior are based on an accurate spatial reference framework.

[0082] After obtaining the above geographical information, the system will calculate the optimal collection angle based on the installation height and coordinate parameters of the current camera equipment. Collect the physical parameters of the camera, including but not limited to installation height (H), horizontal offset (X, Y), and initial pitch angle (θ) and yaw angle (ψ). Using the principles of geometric optics, the field of view of the camera under different tilt angles and focal length settings can be changed by adjusting the pitch and yaw angles of the camera. If the current picture can be presented according to the preset picture ratio, such as the picture area of the intersection area and the common incoming vehicle side and the picture of the core area can present the preset ratio, it is considered that the current angle is the optimal collection angle position. By sending instructions to the camera, automatic adjustment can be achieved to accurately point the camera to the predetermined optimal collection angle position.

[0083] Since all coordinates are based on real-world coordinates provided by the geographic information system, even if the camera moves, as long as its position is recalibrated, the system can continue to accurately perform the judgment. For example, in a charging station scenario, suppose the charging station system has identified the location of the intersection area and the core area based on map data. If one day the camera has to be moved due to construction reasons. In this case, the system will calibrate the shooting angle of the camera based on the height and position of the new camera position, ensuring that it can correctly map the geographical coordinates of the specified area in the real world to the video frame. Then, based on the known geographical positions of the intersection area and the core area, update the corresponding coordinate areas in the current video frame and the changes in the relative orientation between the two in real time.

[0084] This method based on map information rather than fixed picture definition greatly enhances the robustness and adaptability of the system, allowing it to maintain high accuracy and reliability even in complex and changing real-world environments.

[0085] This method not only improves the flexibility and adaptability of the monitoring system, but also ensures that it can maintain efficient operation even when the environment changes, providing continuous and stable monitoring services.

[0086] In an embodiment, the step of real-time detection of the video stream by the edge or cloud device to output the vehicle bounding box coordinates comprises:

[0087] S20, obtaining the position parameters of the camera and the three-dimensional coordinate ranges of the intersection area and the core area;

[0088] S21, calculating the projection matrix for converting the three-dimensional world coordinate system to the two-dimensional image coordinate system according to the intrinsic and extrinsic parameters of the camera;

[0089] S22, converting the three-dimensional coordinates of the intersection area and the core area into two-dimensional image coordinate ranges using the calculated projection matrix;

[0090] S23, determining the relative position relationship between the core area and the intersection area according to the coordinates of the core area and the intersection area in the two-dimensional image.

[0091] In this embodiment, first, the system obtains the position parameters of the camera and the three-dimensional coordinate ranges of the intersection area and the core area. This step involves retrieving detailed information including camera installation height, tilt angle, etc. from the cloud or local database, while determining the geographic coordinates of each key area in the charging station. Specifically, the installation parameters of the camera need to be determined, including but not limited to position (X, Y, Z), height H, pitch angle θ, yaw angle ψ, roll angle φ, etc. In addition, the three-dimensional coordinates of the intersection area need to be extracted from the virtual map, such as the geographic coordinates (longitude, latitude, height) of the four vertices.

[0092] Next, according to the intrinsic parameters (such as focal length Fx, Fy, principal point coordinates Cx, Cy) and extrinsic parameters (such as rotation matrix R and translation vector t) of the camera, the projection matrix P for converting the three-dimensional world coordinate system to the two-dimensional image coordinate system is calculated. The projection matrix P is composed of the intrinsic matrix K and the extrinsic matrix [R|t], where R is the rotation matrix and t is the translation vector. This step uses camera calibration technology to determine the specific parameters of the camera and uses these parameters to build a mathematical model that describes the mapping relationship from the real world space to the image plane. This projection matrix is crucial for achieving high-precision coordinate conversion. With the projection matrix, the system can convert the three-dimensional coordinates of the intersection area and the core area into two-dimensional image coordinate ranges. Specifically, for each vertex of the key area, the three-dimensional coordinates (Xw, Yw, Zw) are converted into the corresponding two-dimensional image coordinates (Ximg, Yimg) by applying the above projection matrix, as follows:

[0093] Finally, based on the coordinates of the core area and the intersection area in the two-dimensional image, the system determines the relative positional relationship between the two areas. This relative positional relationship not only helps to understand the spatial layout of each area in the monitoring picture, but also provides an important basis for subsequent vehicle trajectory tracking and behavior prediction. For example, by analyzing whether a vehicle is approaching or leaving a certain specific area, possible violations can be warned in advance. In practical applications, due to factors such as lens distortion, the results may need to be corrected. A lens distortion model (such as the Brown-Conrady model) can be used to map the coordinates of the intersection area on the virtual map to the two-dimensional image coordinates under the current camera perspective, thereby achieving real-time monitoring and precise positioning.

[0094] In an embodiment, the step of detecting and outputting vehicle bounding box coordinates in real time based on the edge or cloud device includes:

[0095] S30, load a pre-trained vehicle detection model on the edge or cloud device;

[0096] S31, use the loaded vehicle detection model to analyze each frame of image, identify all vehicles, and output the bounding box coordinates (x1, y1, x2, y2) of each vehicle, where (x1, y1) is the top-left corner coordinate and (x2, y2) is the bottom-right corner coordinate.

[0097] In this embodiment, a pre-trained vehicle detection model is loaded on the edge computing device or cloud server. This step is the basis for achieving efficient and accurate vehicle detection. For example, YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), or Faster R-CNN, etc. For example, using YOLOv11, it maintains high accuracy while providing fast detection speed. In order to improve the detection efficiency, the model can be optimized before loading, such as pruning, quantization, etc. operations, to reduce the model size and speed up the inference process. Once the vehicle detection model is successfully loaded on the edge or cloud device, the next step is to analyze each frame of image obtained from the camera in real time, identify all vehicles, and generate a bounding box for each vehicle. Before inputting the image into the model, some preprocessing operations such as scaling, normalization, etc. are usually required to meet the requirements of the model input. Before inputting the image into the model, some preprocessing operations such as scaling, normalization, etc. are usually required to meet the requirements of the model input. By calling the loaded vehicle detection model, the target detection task is performed on each frame of image. The model will output all the objects identified as vehicles in the image and their corresponding bounding box coordinates (x1, y1, x2, y2). The coordinates here represent the position of the vehicle in the image, where (x1, y1) represents the top-left corner coordinates of the bounding box, and (x2, y2) is the bottom-right corner coordinates.

[0098] In another embodiment, since in the actual detection scene, when the vehicle is not in the camera picture The front side or directly opposite, but appears at an angle, using the traditional bounding box may cause the detection box to contain a large number of non-target areas. The output layer and training loss of YOLOv11 are improved to make it able to predict more compact rotated rectangular boxes.

[0099] Specifically, to adapt to the tilted target, the representation method of the bounding box is expanded from the simple two points to include the center point (x, y), the width (w), the height (h), and the rotation angle (angle). Among them, x and y represent the coordinates of the center point of the rectangular box, w and h represent the width and height respectively, and angle represents the rotation angle of the rectangle relative to the horizontal axis. For a tilted vehicle image, first use the model to predict its bounding box in the horizontal state (x, y, w, h), and then estimate the best angle (angle) according to the vehicle's pose. In this way, a minimum enclosing rotated rectangular box is obtained, which fits the actual shape of the target object more closely. The more accurate bounding box enables the tracking algorithm to better follow the target movement, especially in complex scenes or multi-target situations, reducing the possibility of false tracking and facilitating subsequent accurate calculation of the vehicle's movement trend. Therefore, in the output layer of YOLOv11, the center point coordinates (x, y), the width (w), the height (h), the rotation angle (angle), and the confidence score should be adjusted; for the rotated rectangular box, a special angle loss function is also needed to evaluate the difference between the predicted angle and the true angle. The commonly used method is to use mean square error (MSE) or cosine similarity to measure the accuracy of angle prediction.

[0100] The overall loss function Ltotal can be represented as:

[0101] Ltotal=λxyLxy+λwhLwh+λθLθ+λconfLconf

[0102] Where Lxy is the loss of the center point coordinates, Lwh is the loss of the width and height, Lθ is the loss of the angle, Lconf is the loss of the confidence score, and λxy, λwh, λθ, and λconf are the weight coefficients corresponding to the loss terms;

[0103] Through the above improvements to the output layer of YOLOv11 and the training loss, the model can better adapt to the target detection task involving rotated rectangular boxes, thereby improving the accuracy and robustness of the detection results, especially when dealing with tilted or rotated objects. This method can more closely fit the actual shape of the target, thereby improving the effectiveness of subsequent processing steps.

[0104] In an embodiment, the step of starting the algorithm to track the vehicle entering the intersection area based on the movement trend of the vehicle to determine whether the vehicle is moving towards the core area, comprises:

[0105] S40, if a vehicle is identified to enter the intersection area, a unique ID is assigned to each vehicle entering the intersection area based on the target tracking algorithm, and the trajectory coordinates of the corresponding vehicle are recorded to obtain the corresponding trajectory data;

[0106] S41, calculate the speed and acceleration of the vehicle according to the trajectory data, and analyze the corresponding travel direction of the vehicle;

[0107] S42, predict the future movement trend of the vehicle using the historical trajectory data in combination with the speed and acceleration of the current vehicle and the corresponding travel direction of the vehicle;

[0108] S43, if the prediction result shows that the possibility of the vehicle moving towards the core area exceeds a first preset threshold, it is determined that the vehicle is moving towards the core area.

[0109] In the embodiment, when the bounding box output by the detection model first overlaps with the preset "intersection area", it is determined that the vehicle has entered the intersection area. A multi-target tracking algorithm with high precision and robustness such as DeepSORT, ByteTrack or FairMOT is used to assign a unique tracking ID to each vehicle entering the intersection area. With the processing of each frame of video stream, the center point coordinates (x, y) or bounding box position information of the vehicle are continuously recorded to form a continuous time sequence trajectory data. This provides basic data support for subsequent speed, direction and acceleration calculation; assuming that the center point coordinates of the vehicle at time stamps t1 and t2 are (x1, y1) and (x2, y2) respectively, the displacement vector is: ; the vehicle speed can be represented as:

[0110]

[0111] The direction angle θ can be calculated from the displacement vector:

[0112]

[0113] If there are more than three consecutive frames of data, the acceleration can be further estimated:

[0114]

[0115] The direction angle of the vehicle is compared with the direction of the "core area" in the map coordinate system to determine whether the vehicle is advancing towards the area.

[0116] If the current scene belongs to a uniform or uniform acceleration scene, the trajectory and speed of the previous frames are used to directly predict the position of the next period; Then combine the curvature change of the historical trajectory to predict whether the vehicle will approach or enter the core area in the future period of time; According to the spatial relationship (such as distance, angle, intersection) between the predicted trajectory and the core area, the probability or confidence of the vehicle "entering the core area" is calculated. According to the spatial relationship (such as distance, angle, intersection) between the predicted trajectory and the core area, the probability or confidence of the vehicle "entering the core area" is calculated. By introducing target tracking, trajectory analysis, motion state modeling and trend prediction technology, the system realizes the leap from "static identification" to "dynamic perception". This deep understanding of vehicle behavior: improves the accuracy of abnormal behavior identification; Provide solid data support for subsequent warning, scheduling, management and other functions.

[0117] In an embodiment, if the vehicle moves towards the core area, the step of identifying the vehicle type of the vehicle comprises:

[0118] S50, if the vehicle moves towards the core area, call the license plate recognition algorithm to identify the number information and color information of the license plate of the vehicle;

[0119] S51, judging the vehicle type of the vehicle based on the number information and color information,

[0120] S52, judging whether the vehicle type of the vehicle is a first preset vehicle type, wherein the first preset vehicle type is a fuel vehicle.

[0121] This embodiment is an important part of implementing targeted management or early warning for specific types of vehicles, such as fuel vehicles. This process can effectively distinguish different types of vehicles through license plate recognition technology combined with rule judgment, and provide a basis for subsequent behavior decision-making. When it is confirmed in the previous step that the vehicle has a high probability of entering the core area, the system starts the fine analysis process of the vehicle. The OCR (Optical Character Recognition) technology or deep learning model (such as YOLO-LPR, CRNN, EasyOCR, etc.) is used to recognize the license plate area in the detected vehicle bounding box; the output results include: license plate number information (such as "Guangdong B12345"); license plate color information (such as blue, yellow, green, white, black, etc.); for example: blue: small fuel vehicles; gradient green / fully green: new energy vehicles (pure electric or hybrid); yellow: large trucks or trailers; white: police cars, armed police vehicles; black: foreign vehicles or vehicles from Hong Kong and Macao regions. The corresponding color is adjusted according to the real corresponding relationship, and the license plate color is the most direct feature reflecting the energy type of the vehicle; under normal circumstances, if the recognized license plate color is blue, it is initially determined as a fuel vehicle; if the recognized license plate color is gradient green or fully green, it is initially determined as a new energy vehicle. Some regions have specific number rules for new energy vehicle plates (such as "Guangdong AD12345" for pure electric vehicles); the vehicle type can be identified by combining the characteristic number rules of the corresponding region as a supplementary judgment basis to improve the classification accuracy; the system sets the "first preset vehicle type" as a fuel vehicle, which is used to identify non-target vehicles that may violate the rules and enter the core area (such as electric vehicle dedicated charging area). If the vehicle is determined to be a fuel vehicle in the previous step, it is marked as "matching the first preset type"; otherwise, it is marked as "not matching".

[0122] Through license plate recognition + color analysis, fuel vehicles entering a specific area can be quickly and cost-effectively identified to prevent them from occupying new energy resources. The whole process from identification, tracking, prediction to type judgment can be automatically completed without manual supervision, and is suitable for smart park, intelligent parking lot, charging pile management and other scenarios.

[0123] In an embodiment, if the vehicle type is a preset vehicle type, the step of triggering the early warning mechanism includes:

[0124] S60, if the vehicle type is a preset vehicle type, obtaining vehicle information corresponding to the vehicle;

[0125] S61, generating a corresponding early warning notification based on the vehicle information and sending it to a broadcast device;

[0126] S62, warning the vehicle through the broadcast device.

[0127] In this embodiment, the system automatically generates a targeted early warning notification content based on the preset voice template or text template, such as "Attention, the fuel vehicle with license plate Guangdong B12345 is approaching the charging area, please immediately leave the unauthorized area", and pushes the early warning information to the broadcast equipment deployed in the monitoring site in real time through a network communication protocol (such as TCP / IP, HTTP or MQTT), such as smart sound, directional speaker, LED display screen, etc. After receiving the early warning instruction, the broadcast equipment immediately issues a prompt in the form of voice broadcast or rolling subtitles, realizing instant warning and guidance to the target vehicle driver. This process has high automation and low delay characteristics, and can complete identification and early warning response before the vehicle completely enters the core area, thereby effectively preventing non-target vehicles from violating the rules and occupying key resources. In addition, all early warning events can be recorded in the system log to support subsequent backtracking analysis, behavior statistics and strategy optimization. Through this early warning mechanism, the system realizes the whole process closed-loop management from target detection, tracking analysis, type identification to active intervention, significantly improves the intelligent management and control ability of specific areas, and is suitable for various complex application scenarios such as smart park, intelligent parking lot, new energy charging station, etc.

[0128] Reference Figure 2 , is a structure block diagram of a recognition and early warning system of a charging area entering vehicle in an embodiment of the present application, the system comprising:

[0129] The data acquisition module 100 is configured to continuously acquire a video stream based on a preset position camera device;

[0130] The coordinate detection module 200 is configured to output a vehicle bounding box coordinate by detecting the video stream in real time based on an edge or a cloud device;

[0131] The entry judgment module 300 is configured to judge whether a vehicle enters the intersection area according to the overlapping ratio of the bounding box and the preset intersection area;

[0132] The trend judgment module 400 is configured to start an algorithm tracking for the vehicle entering the intersection area, and judge whether the vehicle moves towards the core area based on the moving trend of the vehicle;

[0133] The type identification module 500 is configured to identify the vehicle type of the vehicle if the vehicle moves towards the core area;

[0134] The early warning triggering module 600 is configured to trigger an early warning mechanism if the vehicle type is a preset vehicle type. Further, the system further comprises a position confirmation module, comprising:

[0135] The map acquisition unit is configured to acquire virtual map data of a current use scenario from a cloud;

[0136] A region identification unit is configured to identify geographical information of each functional region in the virtual map data.

[0137] A region division unit is configured to identify a charging pile parking region, a charging pile parking region access intersection region and corresponding coordinate ranges based on the geographical information, wherein the charging pile parking region access intersection region is the intersection region and the charging pile parking region is the core region.

[0138] An optimal collection angle calculation unit is configured to calculate an optimal collection angle based on a current camera installation height and coordinate parameters.

[0139] A camera positioning unit is configured to automatically adjust the camera to a specified position based on the optimal collection angle to obtain a preset position of the camera.

[0140] Further, the system further comprises a three-dimensional mapping module, which comprises:

[0141] A coordinate acquisition unit is configured to acquire position parameters of the camera and three-dimensional coordinate ranges of the intersection region and the core region.

[0142] A matrix calculation unit is configured to calculate a projection matrix for converting a three-dimensional world coordinate system to a two-dimensional image coordinate system based on intrinsic and extrinsic parameters of the camera.

[0143] A coordinate conversion unit is configured to convert the three-dimensional coordinates of the intersection region and the core region into two-dimensional image coordinate ranges using the calculated projection matrix.

[0144] A region positioning unit is configured to determine a relative position relationship between the core region and the intersection region based on coordinates of the core region and the intersection region in the two-dimensional image.

[0145] Further, the coordinate detection module 200 comprises:

[0146] A model loading unit is configured to load a pre-trained vehicle detection model on an edge or cloud device.

[0147] An image analysis unit is configured to analyze each frame of image using the loaded vehicle detection model, identify all vehicles, and output a bounding box coordinate (x1, y1, x2, y2) of each vehicle, wherein (x1, y1) is a top-left corner coordinate and (x2, y2) is a bottom-right corner coordinate.

[0148] Further, the trend judgment module 400 comprises:

[0149] an ID assigning unit configured to assign a unique ID to each vehicle entering the intersection area based on a target tracking algorithm and start recording trajectory coordinates of the corresponding vehicle to obtain corresponding trajectory data if the vehicle is identified to enter the intersection area;

[0150] a trajectory recording unit configured to calculate a speed and an acceleration of the vehicle and analyze a corresponding travel direction of the vehicle according to the trajectory data;

[0151] a state analyzing unit configured to predict a future moving trend of the vehicle using historical trajectory data in combination with the speed and the acceleration of the current vehicle and the corresponding travel direction of the vehicle;

[0152] a trend predicting unit configured to determine that the vehicle moves toward the core area if a prediction result shows that a possibility of the vehicle moving toward the core area exceeds a first preset threshold.

[0153] Further, the type identifying module 500 comprises:

[0154] a license plate recognizing unit configured to identify number information and color information of a license plate of the vehicle by calling a license plate recognition algorithm if the vehicle moves toward the core area;

[0155] a type judging unit configured to judge a vehicle type of the vehicle based on the number information and the color information;

[0156] a type matching unit configured to judge whether the vehicle type of the vehicle is a first preset vehicle type, wherein the first preset vehicle type is a fuel vehicle.

[0157] Further, the early warning triggering module 600 comprises:

[0158] an information extracting unit configured to obtain vehicle information corresponding to the vehicle if the vehicle type is a preset vehicle type;

[0159] a notification generating unit configured to generate a corresponding early warning notification in combination with the vehicle information and send the early warning notification to a broadcasting device;

[0160] a vehicle broadcasting early warning unit configured to early warn the vehicle by the broadcasting device.

[0161] Reference Figure 3 In the embodiments of the present application, a computer device is also provided, which can be a server, and an internal structure of the computer device can be as shown in Figure 3The computer device includes a processor, a memory, a storage medium (non-volatile storage medium), and a network interface connected by a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes the storage medium (non-volatile storage medium) and the memory. The storage medium (non-volatile storage medium) stores an operating system, a computer program, and a database. The memory provides an environment for the operating system and the computer program in the storage medium (non-volatile storage medium) to run. The database of the computer device is used to store data used in the process of the identification and early warning method of the charging area into the vehicle. The network interface of the computer device is used to communicate with the external terminal through the network connection. Further, the computer device can be further provided with an input device, a display screen, and the like. The computer program is executed by the processor to implement an identification and early warning method of a charging area into a vehicle, including the following steps: continuously collecting a video stream based on a camera device at a preset position; outputting a vehicle bounding box coordinate based on real-time detection of the video stream by an edge or cloud device; determining whether a vehicle enters the intersection area according to the overlap ratio of the bounding box and the preset intersection area; starting algorithm tracking for the vehicle entering the intersection area, and determining whether the vehicle moves towards the core area based on the moving trend of the vehicle; if the vehicle moves towards the core area, identifying the vehicle type of the vehicle; and if the vehicle type is a preset vehicle type, triggering an early warning mechanism. Those skilled in the art can understand that Figure 3 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied.

[0162] The computer readable storage medium of the embodiment of the present application stores a computer program, and the computer program is executed by the processor to implement an identification and early warning method of a charging area into a vehicle, including the following steps: continuously collecting a video stream based on a camera device at a preset position; outputting a vehicle bounding box coordinate based on real-time detection of the video stream by an edge or cloud device; determining whether a vehicle enters the intersection area according to the overlap ratio of the bounding box and the preset intersection area; starting algorithm tracking for the vehicle entering the intersection area, and determining whether the vehicle moves towards the core area based on the moving trend of the vehicle; if the vehicle moves towards the core area, identifying the vehicle type of the vehicle; and if the vehicle type is a preset vehicle type, triggering an early warning mechanism. It can be understood that the computer readable storage medium in the embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0163] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, databases, or other media in this application and in examples used herein, unless specifically stated otherwise, can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM), or external cache memory. As an illustration but not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0164] It should be noted that in this document, the terms "comprising", "including", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, device, article, or method that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, device, article, or method. Without more limitations, an element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, device, article, or method that includes the element.

[0165] The above description is only the preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation using the content of the specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A method for identifying and issuing a warning when a vehicle enters a charging area, characterized in that, The method includes: The camera device continuously collects video streams based on preset locations; The video stream is detected in real time using edge or cloud devices, and the vehicle bounding box coordinates are output. Based on the overlap ratio between the bounding box and the preset intersection area, it is determined whether the vehicle has entered the intersection area; The algorithm tracks vehicles entering the intersection area and determines whether the vehicles are moving towards the core area based on their movement trends. If the vehicle moves toward the core area, identify the vehicle type; If the vehicle type is a preset vehicle type, an early warning mechanism is triggered. The step of initiating algorithmic tracking of vehicles entering the intersection area and determining whether the vehicle is moving towards the core area based on the vehicle's movement trend includes: If a vehicle is detected entering the intersection area, a unique ID is assigned to each vehicle entering the intersection area based on the target tracking algorithm, and the trajectory coordinates of the corresponding vehicle are recorded to obtain the corresponding trajectory data. Based on the trajectory data, calculate the vehicle's speed and acceleration, and analyze the vehicle's corresponding direction of travel; The vehicle's future movement trend is predicted by combining historical trajectory data with the current speed and acceleration of the vehicle and the corresponding direction of travel. If the prediction results show that the probability of the vehicle moving toward the core area exceeds a first preset threshold, it is determined that the vehicle is moving toward the core area. The step of identifying the vehicle type if the vehicle moves toward the core area includes: If the vehicle moves toward the core area, the license plate recognition algorithm is invoked to identify the number and color information of the vehicle's license plate; The vehicle type is determined based on the number and color information; Determine whether the vehicle type is a first preset vehicle type, wherein the first preset vehicle type is a fuel vehicle.

2. The method for identifying and warning of vehicles entering a charging area according to claim 1, characterized in that, Before the step of continuously acquiring video streams using a camera device based on a preset position, the following steps are included: Obtain virtual map data for the current usage scenario from the cloud; Identify the geographic information of each functional area in the virtual map data; Based on the geographic information, the charging pile parking area and the intersection area of ​​the charging pile parking area and the corresponding coordinate range are identified, wherein the intersection area of ​​the charging pile parking area and the core area are identified. The optimal acquisition angle is calculated based on the current camera equipment installation height and coordinate parameters; The camera is automatically adjusted to a designated position based on the optimal acquisition angle to obtain the preset position of the camera device.

3. The method for identifying and warning of vehicles entering a charging area according to claim 1, characterized in that, Before the step of real-time detection and output of vehicle bounding box coordinates based on edge or cloud devices of the video stream, the following steps are included: Obtain the camera's position parameters and the 3D coordinate range of the intersection and core areas; Based on the camera's intrinsic and extrinsic parameters, calculate the projection matrix that transforms the three-dimensional world coordinate system to the two-dimensional image coordinate system; Using the calculated projection matrix, the three-dimensional coordinates of the intersection region and the core region are converted into a two-dimensional image coordinate range; The relative positional relationship between the core region and the intersection region is determined based on their coordinates in the two-dimensional image.

4. The method for identifying and warning of vehicles entering a charging area according to claim 1, characterized in that, The step of real-time detection and output of vehicle bounding box coordinates based on edge or cloud devices for the video stream includes: Load a pre-trained vehicle detection model onto an edge or cloud device; The loaded vehicle detection model is used to analyze each frame of the image, identify all vehicles, and output the bounding box coordinates (x1, y1, x2, y2) of each vehicle, where (x1, y1) is the coordinate of the top left corner and (x2, y2) is the coordinate of the bottom right corner.

5. The method for identifying and warning of vehicles entering a charging area according to claim 1, characterized in that, The step of triggering the warning mechanism if the vehicle type is a preset vehicle type includes: If the vehicle type is a preset vehicle type, obtain the vehicle information corresponding to the vehicle; Based on the vehicle information, a corresponding early warning notification is generated and sent to the broadcasting equipment; The vehicle is given a warning via the broadcasting equipment.

6. A vehicle identification and warning system for a charging area, characterized in that, include: The data acquisition module is used to continuously acquire video streams from camera devices located at preset positions. The coordinate detection module is used to detect and output the vehicle bounding box coordinates in real time based on edge or cloud devices in the video stream; The judgment module is used to determine whether a vehicle has entered the intersection area based on the overlap ratio between the bounding box and the preset intersection area. The trend judgment module is used to initiate algorithmic tracking of vehicles entering the intersection area and to determine whether the vehicles are moving towards the core area based on their movement trends. A type recognition module is used to identify the vehicle type if the vehicle moves toward the core area; The warning triggering module is used to trigger the warning mechanism if the vehicle type is a preset vehicle type; The trend judgment module includes: The ID allocation unit is used to assign a unique ID to each vehicle entering the intersection area based on the target tracking algorithm if a vehicle is detected entering the intersection area, and to start recording the trajectory coordinates of the corresponding vehicle to obtain the corresponding trajectory data. The trajectory recording unit is used to calculate the vehicle's speed and acceleration based on the trajectory data, and to analyze the vehicle's corresponding direction of travel. The status analysis unit is used to predict the future movement trend of the vehicle by combining historical trajectory data with the current speed and acceleration of the vehicle and the corresponding direction of travel of the vehicle. The trend prediction unit is used to determine that the vehicle is moving toward the core area if the prediction result shows that the probability of the vehicle moving toward the core area exceeds a first preset threshold. Furthermore, the type recognition module includes: The license plate recognition unit is used to call the license plate recognition algorithm to identify the number and color information of the vehicle's license plate if the vehicle moves toward the core area. A type determination unit is used to determine the vehicle type based on the number information and color information; A type matching unit is used to determine whether the vehicle type of the vehicle is a first preset vehicle type, wherein the first preset vehicle type is a fuel vehicle.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images