Behavioral prompting method and behavioral prompting system

Abstract

The application discloses a behavior supervision method and system, and relates to the technical field of computer vision. The behavior supervision method is applied to helmet wearing behavior detection and comprises the following steps: acquiring a camera collection image of a target area and preprocessing the acquired camera collection image; identifying and positioning a target bounding box from the preprocessed camera collection image based on a target feature of a predefined helmet-wearing cyclist, and performing illegal behavior judgment; the target categories include cyclists, electric vehicles and helmets; based on the illegal behavior judgment result, determining the vehicle identification of a target electric vehicle when it is determined that a cyclist does not wear a helmet in violation of regulations; and based on a preset hierarchical intervention supervision mechanism and the intervention level triggered by the target electric vehicle corresponding to the vehicle identification, outputting a supervision management instruction corresponding to the intervention level. The method is used for accurately identifying the illegal behavior of cyclists not wearing helmets and reducing false detection.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a behavior supervision method and behavior supervision system. Background Technology

[0002] Electric bikes and other electric vehicles are an important mode of transportation for short-distance urban travel, and their riding safety is receiving increasing attention from society. Common methods for encouraging electric bike riders to wear helmets mostly rely on manual patrols and monitoring on key roads, which is not only inefficient but also has blind spots in surveillance.

[0003] While some related technologies combine computer vision to collect images of cyclists wearing helmets and count violations of not wearing helmets, these helmet detection solutions are unable to accurately identify whether cyclists are not wearing helmets and have low detection efficiency. Summary of the Invention

[0004] The main purpose of this application is to provide a behavior supervision method and system, which aims to solve the technical problem of how to accurately identify the violation of cyclists not wearing helmets.

[0005] On the one hand, a behavior supervision method is provided for helmet-wearing behavior detection, the behavior supervision method including: Acquire camera images of the target area and preprocess the acquired camera images; Based on predefined target features of helmet-wearing cyclists, the system identifies and locates target bounding boxes from pre-processed camera images to determine violations. Based on the results of the violation judgment, if it is determined that a rider has violated the rule of not wearing a helmet, the vehicle identification of the target electric vehicle is determined; Based on the pre-set tiered intervention and supervision mechanism and the intervention level triggered by the target electric vehicle with the corresponding vehicle identification, the corresponding supervision and management instructions are output.

[0006] In one embodiment, the step of outputting a supervision and management instruction corresponding to the intervention level based on a pre-set tiered intervention and supervision mechanism and the intervention level triggered by the target electric vehicle with the corresponding vehicle identifier includes: Based on the vehicle identification of the target electric vehicle, obtain the number of violations corresponding to the target electric vehicle; Based on a pre-set tiered intervention and supervision mechanism triggered by the number of violations, the intervention level triggered by the target electric vehicle is determined, and the supervision and management instructions corresponding to the intervention level are output to the terminal device; the terminal device includes user terminal device and charging pile device.

[0007] In one embodiment, the step of determining the intervention level triggered by the target electric vehicle based on a pre-set tiered intervention and supervision mechanism that uses the number of violations as the trigger criterion, and outputting a supervision and management instruction corresponding to the intervention level to the terminal device, includes: Based on a pre-set tiered intervention and supervision mechanism triggered by the number of violations, when the target electric vehicle is determined to have triggered the first intervention level, a violation alarm command corresponding to the first intervention level is output to the terminal device to control the terminal device to display a violation risk notification. When it is determined that the target electric vehicle has triggered the second intervention level, a forced learning instruction corresponding to the second intervention level is output to the charging pile device to control the charging pile device to maintain the locked state within the preset locking time and to control the charging pile device to play a safety education video in the locked state. If the target electric vehicle triggers the third intervention level, the process of disabling the target electric vehicle is triggered, and the charging disabling command corresponding to the third intervention level is output to the charging pile equipment to control the charging pile to refuse the charging request of the corresponding target electric vehicle, and to control the user terminal equipment to display the disabling notification. The disabling notification is used to indicate the reason for disabling, the duration of disabling, and the steps to unblock. The threshold for the number of violations that triggers the first intervention level is less than the threshold for the number of violations that triggers the second intervention level, which in turn triggers the threshold for the number of violations that triggers the third intervention level.

[0008] In one embodiment, the behavior supervision method further includes the following steps: In response to a supervision status update instruction uploaded by a user terminal device, the supervision status update instruction is used to indicate the current supervision status and whether a security commitment letter has been uploaded; If the monitoring status update instruction indicates that the current monitoring status is completed and the safety commitment letter has been uploaded, the process of disabling the target electric vehicle with the corresponding vehicle identifier will end.

[0009] In one embodiment, before executing the step of determining the intervention level triggered by the target electric vehicle based on the pre-set, tiered intervention and supervision mechanism triggered by the number of violations, the method further includes: The system retains screenshots of the scene from the acquired camera images and accumulates the current number of violations for the target electric vehicle corresponding to the vehicle identification number.

[0010] In one embodiment, the step of identifying and locating the target bounding box from the pre-processed camera-captured image based on predefined target features of a helmet-wearing cyclist, and determining the violation behavior, includes: Based on predefined target features of cyclists wearing helmets, target bounding boxes are identified and located from preprocessed camera images. The target bounding boxes include bounding boxes corresponding to cyclists wearing helmets and bounding boxes corresponding to cyclists not wearing helmets. Extract the head detection region from the target bounding box according to the preset extraction area; Based on predefined helmet wearing characteristics, the extracted head detection area is used to determine violations. If the image features of the head detection area match the predefined helmet wearing features, it is determined that the target cyclist is wearing a helmet and no violation has occurred. If the image features of the head detection area do not match the predefined helmet wearing features, it is determined that the target cyclist is not wearing a helmet, thus confirming the violation of not wearing a helmet.

[0011] In one embodiment, determining the vehicle identification of the target electric vehicle based on the violation judgment result, when it is determined that a rider has violated the rule of not wearing a helmet, includes the following steps: Based on the violation judgment results, when it is determined that a rider has violated the rule of not wearing a helmet, the electric vehicle bounding box corresponding to the electric vehicle image is extracted from the pre-processed camera image according to the predefined electric vehicle characteristics. Each electric vehicle bounding box is used to indicate the candidate electric vehicle. Determine the intersection-union ratio (IU) between each extracted electric vehicle bounding box and the target bounding box. If the IU is not less than a set threshold, determine that the candidate electric vehicle indicated by the electric vehicle bounding box is the target electric vehicle corresponding to the rider who is not wearing a helmet. Identify the vehicle identification number of the target electric vehicle.

[0012] In one embodiment, the vehicle identifier includes vehicle binding information, vehicle user identity, and license plate number; the vehicle identifier for determining the target electric vehicle includes: Extract the license plate detection box of the target electric vehicle from the pre-processed camera image, and obtain the license plate number of the target electric vehicle from the license plate detection box to obtain the vehicle identification of the target electric vehicle. And / or, in response to behavioral supervision instructions from the management end, determine the vehicle identification of the target electric vehicle; And / or, based on the stored vehicle image information, determine the vehicle image information and corresponding vehicle identifier that match the target electric vehicle.

[0013] In one embodiment, the behavior supervision method further includes the following steps: With a preset period as the interval, in response to the trigger signal of the current reset period, the number of violations accumulated in the previous reset period is cleared to zero.

[0014] On the other hand, a behavior supervision system is provided, including a memory, a processor, and a behavior supervision program stored in the memory and executable on the processor, wherein the behavior supervision program, when executed by the processor, implements the steps of the behavior supervision method as described above.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: The behavior supervision method of this application is applied to helmet wearing behavior detection. It acquires camera images of the target area and preprocesses the acquired camera images to suppress background noise, highlight the effective features of the target area, reduce missed detections and judgments caused by image quality problems, and improve the reliability of subsequent judgments on violations. Based on predefined target features of helmet-wearing cyclists, the system identifies and locates target bounding boxes from pre-processed camera images to determine violations. It can accurately identify and locate helmet-wearing cyclists in pre-processed camera images, effectively eliminating invalid background interference. The entire detection process is end-to-end, without intermediate stages, and outputs the detection results of helmet-wearing cyclists in a single step. This effectively solves the technical problem of accurately identifying cyclists not wearing helmets, significantly reducing false alarm rates in practical applications and ensuring higher reliability of detection results. It effectively identifies helmet-wearing violations while also improving detection accuracy and efficiency. Based on the violation judgment, when it is determined that a rider has violated the rule of not wearing a helmet, the vehicle identification of the target electric vehicle is determined; based on the pre-set hierarchical intervention and supervision mechanism and the intervention level triggered by the target electric vehicle with the corresponding vehicle identification, the supervision and management instructions of the corresponding intervention level are output; thus, the target electric vehicle with the violation can be identified, and historical violation records can be linked through the vehicle identification. Through the constructed hierarchical intervention and supervision mechanism, a complete vehicle identification and supervision closed loop is formed, and through linkage with terminal devices, the violation can be effectively constrained, gradually restraining riders' bad riding habits and effectively playing a role in control and guidance. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart is provided for one embodiment of the behavior supervision method of this application; Figure 2 A flowchart illustrating an embodiment of step S200 of this application is provided; Figure 3 A flowchart illustrating an embodiment of step S300 of this application is provided; Figure 4 This is a schematic diagram of the interference elimination judgment process in this application; Figure 5 A flowchart illustrating an embodiment of step S400 of this application is provided; Figure 6 A flowchart illustrating an embodiment of step S420 of this application is provided; Figure 7 This is a strategy table for the tiered intervention and supervision mechanism proposed in this application; Figure 8 This is one of the flowcharts illustrating the tiered intervention and supervision mechanism of this application; Figure 9 This is the second flowchart illustrating the tiered intervention and supervision mechanism of this application; Figure 10 A partial flowchart is provided for another embodiment of the behavior supervision method of this application; Figure 11 This is a schematic diagram of the architecture of the behavior supervision system of this application; Figure 12 This is a diagram illustrating the acquisition of user-related data in this application.

[0019] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0021] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0022] Electric bikes and other electric vehicles are an important mode of transportation for short-distance urban travel, and their riding safety is receiving increasing attention. Helmets are crucial safety equipment for protecting the rider's head, and not wearing a helmet is one of the main causes of serious injury or even death in riding accidents. Common methods for encouraging electric bike riders to wear helmets mostly rely on manual patrols and monitoring on key roads, which is not only inefficient but also has blind spots in monitoring.

[0023] While some technologies combine computer vision to collect information on cyclists and electric bikes in images and use this method to count helmet-less violations, these helmet detection solutions only limit the detection target to the helmet or head area. They can only determine whether a helmet is present in the image, but lack precise definition of the detection target, affecting the accuracy of violation identification. In real-world scenarios, various interferences may occur simultaneously within the camera's field of view, leading to a high false detection rate. These interferences include parked electric bikes without riders, idle electric bikes that do not require helmet detection, cyclists who do not require helmet supervision, and ordinary pedestrians unrelated to the cycling scenario and who should not trigger any supervision mechanisms. These interferences cause the system to frequently generate false alarms, making it unable to accurately identify whether a cyclist is not wearing a helmet, resulting in a high false detection rate that seriously affects user experience and system reliability.

[0024] To accurately identify riders' violations of not wearing helmets, embodiments of this application propose a behavior supervision method and system for automatically identifying riders' violations of not wearing helmets in service areas for electric bicycles, electric motorcycles, and other electric vehicles (such as community charging areas, public parking spots, etc.) and implementing effective supervision through linkage with user terminals, charging pile equipment, and other terminal devices.

[0025] In the embodiments of this application, the behavior supervision method is applied to helmet wearing behavior detection. The behavior supervision method includes at least the following steps S100, S200, S300, and S400. For ease of description, the following behavior supervision method is mainly described using processors, control devices with control functions, control systems, and systems with relevant behavior supervision functions as the executing entities. The processor can be, but is not limited to, a microcontroller (MCU), a programmable logic controller (PLC), a digital signal processor (DSP), or a field-programmable gate array (FPGA).

[0026] like Figure 1 As shown, in step S100, the camera captures images of the target area and preprocesses the acquired images.

[0027] For example, a camera device can be used as an image acquisition module to acquire camera images of the target area. The camera device can be a high-definition camera with a resolution of at least 1080P or other camera devices suitable for practical use. The camera device can be installed in locations such as charging areas or parking sheds to cover different areas such as where electric vehicles enter and leave. The camera device needs to be able to acquire full-body images of the rider. Embodiments of this application can deploy a camera device in at least one area to acquire images of the corresponding detection area. Depending on the requirements of the actual application scenario, a camera device can be deployed in a single detection area, using that area as the target area to acquire the corresponding image. Alternatively, camera devices can be deployed in multiple areas, selecting one, two, some, or all of the detection areas as target areas to acquire camera images of the corresponding target areas; this is not limited here.

[0028] Preprocessing the acquired camera images (such as histogram equalization and normalization) can eliminate inconsistencies and interference caused by factors such as lighting changes, shooting angle, and sensor noise in the original camera images. This ensures that the acquired camera images from different time periods and target areas maintain uniformity in dimensions such as brightness, contrast, and color distribution. It can also be used to suppress background noise, highlight the effective features of the target area, reduce missed detections and judgments caused by image quality issues, and improve the reliability of subsequent judgments on violations.

[0029] In some alternative embodiments, an auxiliary lighting module can be provided, which is equipped with an infrared fill light to eliminate environmental interference such as strong light and low light at night, to ensure the image acquisition quality at night and in low-light environments, and to provide stable imaging conditions for the high-definition camera.

[0030] Step S200: Based on the predefined target features of a helmet-wearing cyclist, identify and locate the target bounding box from the pre-processed camera images to determine the violation behavior.

[0031] The predefined target features can include only the visual features of cyclists wearing helmets, or they can include the visual features of cyclists not wearing helmets. Alternatively, they can include visual features obtained by treating the electric vehicle, cyclist, and cyclist's head region as a whole. These visual features are learned through methods such as training the model with labeled samples. By combining helmet detection logic with cyclist detection, and without directly acquiring the cyclist's face to protect personal privacy, the system utilizes an adaptive receptive field to detect surrounding targets. It accurately distinguishes the corresponding target from the background in the captured image, identifies and locates the target, and outputs its bounding box. The embodiments of this application, based on predefined target features of cyclists wearing helmets, can accurately identify and locate cyclists wearing helmets in preprocessed captured images, effectively eliminating invalid background interference. The entire detection process is end-to-end, without intermediate stages, focusing solely on whether a cyclist is wearing a helmet and outputting the detection result for a helmeted cyclist in a single step. From a practical application perspective, this effectively reduces the false alarm rate, ensures higher reliability of the detection results, and significantly improves detection accuracy and efficiency.

[0032] Step S300: Based on the result of the violation judgment, if it is determined that a rider has committed the violation of not wearing a helmet, determine the vehicle identification of the target electric vehicle; Step S400: Based on the pre-set graded intervention and supervision mechanism and the intervention level triggered by the target electric vehicle with the corresponding vehicle identification, output the supervision and management instruction corresponding to the intervention level.

[0033] When a rider is found to be violating the rule of not wearing a helmet, the vehicle identification of the target electric vehicle is determined. By obtaining the electric vehicle identification information associated with the target rider, the violation of not wearing a helmet is matched with the vehicle identification such as the license plate number of the electric vehicle that violated the rule. This can be used to identify the target electric vehicle that violated the rule. By associating the vehicle identification with historical violation records, a graded intervention and supervision mechanism based on the number of violations, frequency of violations, and severity of violations can be established, forming a complete closed loop of vehicle identification and supervision. Through linkage with the charging pile IoT system and other terminal devices, effective constraints on violations can be achieved.

[0034] Unlike existing technologies that limit detection to helmets or head areas and fail to accurately determine whether cyclists are wearing helmets correctly in various real-world riding scenarios, this application combines helmet detection logic with cyclist detection. Without directly acquiring the cyclist's face and protecting privacy, it utilizes an adaptive receptive field to detect surrounding targets, accurately identifying and locating helmet-wearing cyclists in pre-processed camera images. This effectively eliminates invalid interference, solving the technical problem of accurately identifying cyclists not wearing helmets. From a practical application perspective, it effectively reduces false alarm rates, ensuring higher reliability of detection results. It not only effectively identifies helmet-wearing violations but also improves detection accuracy and efficiency, overcoming the lack of specific interference elimination methods in existing technologies. This fundamentally addresses the industry pain point of high false alarm rates in previous technologies, significantly improving the stability and reliability of daily detection work. The embodiments of this application can utilize improved YOLO12 and other cutting-edge target detection algorithms to optimize overall detection performance, integrate multi-head self-attention mechanisms to enhance model feature extraction capabilities, further optimize the recognition of small targets, and significantly improve the detection and recognition accuracy of small objects in complex environments. Compared to related technologies that only provide violation alerts and record information, lacking substantial business constraints and long-term behavioral guidance mechanisms, this application establishes a tiered intervention and supervision mechanism based on actual violation counts. This mechanism includes various measures such as reminders, mandatory learning, and prohibition of charging, forming a complete vehicle identification and supervision loop. Through linkage with terminal devices, it effectively constrains violations, gradually curbing riders' poor riding habits and maximizing the effectiveness of control and guidance. This application also achieves deep collaboration with user terminals, charging pile IoT systems, and other terminal devices, establishing a complete closed-loop riding behavior constraint management system. It also allows for the setting of a fixed monthly statistical cycle, deduplication of valid violation information, and a complete set of standardized statistical management solutions, including violation appeal procedures. This ensures more rigorous and standardized data collection and subsequent handling of riding violations, guaranteeing the orderly implementation of overall supervision.

[0035] In some embodiments, before performing the aforementioned step S100, acquiring a camera image of the target area, and preprocessing the acquired camera image, the following steps are included: The camera device continuously captures video streams to obtain camera images of the target area.

[0036] For example, the camera continuously captures a video stream at a frame rate of not less than 25 frames per second (FPS), decodes the raw video frames in RGB format, and scales them to the input resolution required by the detection model, such as 640×640 pixels.

[0037] When performing preprocessing of the acquired camera image in step S100, for scenarios with uneven lighting, embodiments of this application can perform adaptive histogram equalization processing to limit the contrast of the brightness channel of the acquired camera image, thereby improving the local contrast of the target area. Then, through normalization processing, the image pixel values ​​are normalized to the [0,1] interval, and then standardized according to the mean and standard deviation set during model training.

[0038] In some alternative embodiments, the presence of a moving target in the frame can be quickly determined by frame difference method to determine whether an electric vehicle exists. If a moving target exists, target recognition and reasoning for a rider wearing a helmet are activated. If it is a still frame, i.e. there is no moving target, subsequent target recognition and reasoning are skipped, thereby reducing power consumption.

[0039] Unlike related technologies that limit the detection target to the helmet or head area and cannot accurately determine whether a cyclist is wearing a helmet properly in various actual cycling scenarios, the embodiments of this application clearly define cyclists wearing helmets and cyclists not wearing helmets and conduct detection accordingly. This not only effectively broadens the overall detection target coverage, but also enables accurate identification in different real cycling scenarios, effectively improving the accuracy of helmet wearing status recognition.

[0040] like Figure 2 As shown, in one embodiment, step S200, identifying and locating the target bounding box from the preprocessed camera-captured image based on predefined target features of a helmet-wearing cyclist, and determining the violation behavior, includes: Step S210: Based on the predefined target features of cyclists wearing helmets, identify and locate target bounding boxes from the preprocessed camera images. The target bounding boxes include bounding boxes corresponding to cyclists wearing helmets and bounding boxes corresponding to cyclists not wearing helmets.

[0041] Step S220: Extract the head detection region from the target bounding box according to the preset extraction area.

[0042] Step S230: Based on predefined helmet wearing features, determine the violation behavior in the extracted head detection area.

[0043] For example, when executing step S210, based on the predefined target features of cyclists wearing helmets, the bounding boxes corresponding to cyclists wearing helmets and cyclists not wearing helmets are identified and located from the preprocessed camera images. Then, when executing step S220, the head detection region is extracted from the target bounding boxes according to the predefined extraction area. In step S230, the extracted head detection region is judged for violations according to the predefined helmet wearing features, and is mainly used to detect whether the image features of the extracted head detection region match the predefined helmet wearing features.

[0044] Step S240: If the image features of the head detection area match the predefined helmet wearing features, determine that the target cyclist is wearing a helmet and confirm that no violation has occurred. Step S250: If the image features of the head detection area do not match the predefined helmet wearing features, it is determined that the target cyclist is not wearing a helmet, thus confirming the violation of not wearing a helmet.

[0045] The embodiments of this application can extract head detection regions for target bounding boxes such as those corresponding to cyclists wearing helmets and those corresponding to cyclists not wearing helmets; alternatively, they can extract head detection regions only for the bounding box corresponding to cyclists not wearing helmets. A pre-defined extraction region is estimated based on human proportions to extract the head detection region within the target bounding box. This head detection region can be located in approximately 20% of the top area of ​​the cyclist's bounding box and appropriately expanded to include the entire helmet. During step S230, based on predefined helmet-wearing features, the detection results for the helmet target category are searched within the extracted head detection region. If the image features of the head detection region do not match the predefined helmet-wearing features, it is determined that the target cyclist is not wearing a helmet; otherwise, it is determined that the target cyclist is wearing a helmet.

[0046] In addition, in some other embodiments, the overlap rate between the helmet bounding box and the head detection area can be determined and a confidence level can be judged. If the overlap rate between the helmet bounding box and the head detection area is not less than 0.5 or other set thresholds and the confidence level is not less than 0.6 or other set thresholds, it is determined that the target cyclist is wearing a helmet and no violation has occurred; otherwise, it is determined that the target cyclist is not wearing a helmet and a violation of not wearing a helmet has occurred.

[0047] It should be noted that the same electric vehicle can be ridden by a single person or a tandem. This application's detection of riders wearing helmets and those not wearing helmets can be used to detect the head detection area and corresponding helmet wearing status of all riders on the target electric vehicle. During the violation judgment process of the extracted head detection area in step S230, each rider is associated with at most one electric vehicle to avoid duplicate detection. When judging violations in the extracted head detection area, for fuzzy judgment results where the confidence level between the helmet bounding box and the head detection area is between 0.4 and 0.6, secondary detection can be triggered (cropping and re-inferring the local area of ​​the original camera-captured image at a higher resolution, and then identifying and locating the target bounding box again from the pre-processed camera-captured image) to improve the judgment accuracy.

[0048] like Figure 3 As shown, in one embodiment, step S300, based on the violation judgment result, determines the vehicle identification of the target electric vehicle when it is determined that a rider has violated the rule of not wearing a helmet, including the following steps: Step S310: Based on the violation judgment result, if it is determined that a rider has violated the rule of not wearing a helmet, extract the electric vehicle bounding box corresponding to the electric vehicle image from the pre-processed camera image according to the predefined electric vehicle characteristics. Each electric vehicle bounding box is used to indicate the candidate electric vehicle. Step S320: Determine the intersection-union ratio (IUR) of each extracted electric vehicle bounding box and the target bounding box, and if the IUR is not less than a set threshold, determine that the candidate electric vehicle indicated by the electric vehicle bounding box is the target electric vehicle corresponding to the rider who is not wearing a helmet.

[0049] Step S330: Determine the vehicle identification of the target electric vehicle.

[0050] like Figure 4 As shown in the embodiments of this application, the image frames captured by the camera provide the data foundation for target recognition. An improved YOLO12 model is used to infer the image frames, detect and obtain bounding boxes corresponding to cyclists wearing helmets and those not wearing helmets, and perform an interference elimination judgment process. In the embodiments of this application, after a cyclist is found to be not wearing a helmet (i.e., after helmet status detection is completed), electric vehicle detection begins, indiscriminately detecting electric vehicles in the camera-captured images. Based on the violation judgment result, when it is determined that a cyclist is not wearing a helmet, according to predefined electric vehicle characteristics, the identified electric vehicles are used as candidate electric vehicles. Electric vehicle bounding boxes corresponding to the electric vehicle images are extracted from the preprocessed camera-captured images, and each electric vehicle bounding box is used to indicate the candidate electric vehicle.

[0051] Due to safety requirements for electric vehicle riding, overlapping electric vehicles will not appear in the captured images (pedestrians may pull or hug each other, but this does not affect the detection of candidate electric vehicles and their corresponding bounding boxes in the captured images). The electric vehicle bounding boxes in the acquired images are individual boxes, which are then correlated with the helmet position to identify the target electric vehicle driven by a rider not wearing a helmet. For example, the Intersection over Union (IoU) of the extracted electric vehicle bounding box corresponding to the candidate rider image and the target bounding box corresponding to the target electric vehicle image showing the rider not wearing a helmet is determined to be no less than a set threshold (e.g., 0.3, 0.4, 0.5). If the IoU is less than the set threshold, the candidate electric vehicle corresponding to that bounding box is determined not to be the target electric vehicle, and the vehicle identification process is skipped. If the IoU is no less than the first threshold, the candidate electric vehicle corresponding to that bounding box is determined to be the target electric vehicle, and the vehicle identification of the target electric vehicle is further determined. In this way, the vehicle identification of the target electric vehicle showing the rider not wearing a helmet can be effectively determined, reducing false positives and false negatives and improving the reliability of the recognition results.

[0052] like Figure 4 As shown, in one embodiment, the vehicle identifier includes vehicle binding information, vehicle user identity, and license plate number, and the terminal device includes user terminal device and charging pile device; the vehicle identifier of the target electric vehicle in step S300 can be obtained by at least one or more of the following methods.

[0053] As an example, the license plate detection box of the target electric vehicle is extracted from the pre-processed camera image, and the license plate number of the target electric vehicle is obtained from the license plate detection box to obtain the vehicle identification of the target electric vehicle.

[0054] For example, a license plate detection box corresponding to the target electric vehicle's license plate image is extracted from the preprocessed camera-captured image. This is primarily done by extracting the corresponding license plate detection box from the target electric vehicle's bounding box. Then, when obtaining the target electric vehicle's license plate number from the detection box, an improved rotating bounding box license plate detection algorithm can be used to output the license plate's rotated bounding box (including center coordinates, width, height, and rotation angle). Using the center coordinates of the license plate image in the camera-captured image as the origin, a rotation matrix is ​​constructed according to the rotation angle. An affine transformation is then performed on the acquired license plate image to accurately crop and correct the corresponding region into a horizontal image. The license plate number is then directly identified based on the corrected horizontal license plate image, thus accurately obtaining the target electric vehicle's license plate number. In this way, when the target electric vehicle has a license plate, the target electric vehicle, the rider's violation of not wearing a helmet, and the license plate number can be directly linked, thereby achieving the regulatory and supervisory objectives of this paper.

[0055] As another example, the vehicle identification of the target electric vehicle is determined in response to behavioral prompts from the management side.

[0056] Since not every image of the target electric vehicle captured from the camera shows a license plate, this could be due to the electric vehicle not having a license plate, or issues such as the shooting angle preventing direct detection of the license plate from the captured image. In these cases where a license plate image cannot be directly obtained, a feedback signal is sent to the management system. Community administrators then assess and supervise the situation, and based on the behavioral guidance from the management system, determine the vehicle identification of the target electric vehicle. In this way, even when a license plate image cannot be directly obtained, the management system can assist in acquiring and confirming the license plate number and other vehicle identification of the target electric vehicle exhibiting the violation of not wearing a helmet, and further accumulate the current number of violations for the corresponding vehicle identification.

[0057] As another example, based on stored vehicle image information, vehicle image information and corresponding vehicle identification information matching the target electric vehicle are determined. Vehicle image information is acquired and stored by obtaining user-uploaded three-view drawings of electric vehicles, images of electric vehicles captured by charging stations, etc. The stored vehicle image information includes vehicle identification information such as license plate numbers. After a rider is found to be violating the rule of not wearing a helmet, the captured image of an unlicensed electric vehicle (i.e., the target electric vehicle image in the aforementioned embodiment) is matched with the vehicle image information such as electric vehicle images in the database. The owner of the electric vehicle with the highest matching degree and the corresponding license plate number are output to help community managers quickly locate the user who violated the rule.

[0058] Most of the detection algorithms used in related technologies are suitable for detecting industrial safety helmets, lacking specificity for cycling helmets. This results in poor detection performance when these algorithms are applied to cycling helmets, and a tendency to confuse cycling helmets with industrial helmets. Furthermore, cycling helmets are small targets, and in real-world scenarios, background elements such as parking sheds, streetlights, and buildings are similar in color and shape to the helmets, leading to various interfering elements in the captured images. Changes in the rider's posture, such as looking down, turning to the side, or being obstructed by objects, further complicate head detection. Additionally, lighting conditions such as day / night cycles, backlighting, and shadows significantly affect detection accuracy. Helmets are also easily obscured by hair, scarves, bicycle baskets, and other objects, further increasing the difficulty of identification.

[0059] For example, to effectively identify helmets in camera-captured images, embodiments of this application employ an improved YOLO12 model to implement the aforementioned step S200, which identifies and locates target bounding boxes from preprocessed camera-captured images based on predefined target features of cyclists wearing helmets. YOLO12 (i.e., YOLOv12) is an improved version of YOLO (You Only Look Once) object detection, used to accurately identify and locate target bounding boxes (including bounding boxes corresponding to cyclists wearing helmets and bounding boxes corresponding to cyclists not wearing helmets) from camera-captured images based on an improved attention architecture and collaborative mechanism and predefined target features. The detection algorithm's annotations are the head bounding boxes of cyclists wearing helmets and those not wearing helmets. The object detection algorithm has an adaptively sized receptive field, capable of identifying detection regions. Therefore, after the labeled data reaches a certain scale, it learns the feature of whether a cyclist is wearing a helmet, and will not generate false alarms about not wearing a helmet when ordinary pedestrians pass by. The cyclist helmet detection is an independent algorithm that does not rely on the linkage between the helmet, head, person, and electric vehicle, thus achieving accurate end-to-end target recognition.

[0060] This application employs improved versions of cutting-edge object detection algorithms such as YOLO12, achieving object recognition and localization through pre-built models. The main improvements include: integrating a multi-head self-attention (MHSA) module into the backbone network to enhance small object detection and improve the model's perception of small helmet targets, addressing the issue of missed detections caused by the small proportion of helmet targets in the overall image (typically less than 1% of the image area); adding weights for high-resolution shallow features specifically for helmet detection in the YOLO12 feature pyramid network FPN / PAN, while retaining deep semantic features for overall electric vehicle detection, to optimize multi-scale feature fusion and achieve balanced detection of targets of different sizes; adding a lightweight target spatial association auxiliary branch to the detection head to predict the spatial affiliation between the rider and the electric vehicle; adding data augmentation techniques such as MixUp, Mosaic, and Random Erasing for helmet targets during model training to improve the model's robustness in complex occlusion scenarios; and performing INT8 quantization and channel pruning on the model. Pruning enables inference latency on edge computing devices to meet real-time requirements (such as a target latency of no more than 50ms / frame) and optimizes lightweight deployment.

[0061] In addition, the training dataset of the improved YOLO12 model used in this application contains no less than 80,000 images corresponding to electric vehicle riding scenarios, and covers a variety of scene conditions, including but not limited to different lighting environments such as daytime, nighttime (including infrared supplemental lighting), backlighting, and cloudy days; multiple shooting angles such as front, side, and overhead (30°~60°); various helmet types such as full-face helmets, half-face helmets, and 3 / 4 helmets; complex interference scenarios such as bicycles, ordinary pedestrians, vacant electric vehicles, and multi-person scenes (including tandem riding); and scenarios with different degrees of occlusion, such as helmets being partially occluded by 10% to 50%.

[0062] The specific implementation is as follows: When executing step S100, the acquired processed target area camera image can be an image with a set input specification. Considering the computational cost and feature extraction effect, the embodiment of this application can use a 640×640 resolution image or an image with other input specifications. When executing step S200, the Backbone network extracts features from the preprocessed camera image based on predefined target category features. The improved MHSA multi-head self-attention module enhances the perception of small targets such as helmets and captures global contextual information. The attention-enhanced features are fused through a multi-scale feature fusion structure composed of FPN (Feature Pyramid Network) and PAN (Path Aggregation Network): FPN transmits deep features with strong semantics through a top-down path, while PAN transmits high-resolution shallow features through a bottom-up path. The combination of the two enables the model to output key features of cyclists wearing helmets, shallow high-resolution features for detecting small targets such as helmets, deep semantic features for detecting large targets such as electric vehicles, and mid-level features corresponding to medium-scale targets such as cyclists. The mid-level features take into account both details and semantics. The multi-scale fusion mechanism ensures that bounding boxes of different sizes (including bounding boxes corresponding to cyclists wearing helmets and bounding boxes corresponding to cyclists not wearing helmets) can obtain appropriate feature support and be accurately identified and located, thereby optimizing the recognition effect for small targets such as helmets and significantly improving the detection and recognition accuracy of small target objects in complex environments.

[0063] Unlike related technologies that lack a targeted interference elimination mechanism and suffer from significant false alarms during detection, this solution's implementation can systematically and accurately distinguish between cyclists wearing helmets and those not wearing helmets, effectively eliminating invalid interference from image backgrounds, and performing helmet detection only on cyclists. From a practical application perspective, this effectively reduces the false alarm rate and ensures higher reliability of the detection results.

[0064] The embodiments of this application identify and locate target bounding boxes from pre-processed camera images based on predefined target features of helmet-wearing cyclists to determine violations. These embodiments can identify helmet-wearing and helmetless cyclists with only a single inference, effectively reducing error accumulation caused by multi-level serial target detection and improving overall accuracy. The embodiments of this application are applicable to various complex application scenarios. In some experimental tests, the false detection rate of interfering targets such as empty electric bikes (without riders), cyclists, and ordinary pedestrians is less than 2%. Compared to related technologies that only complete violation alarms and information recording, the embodiments of this application, based on a pre-set hierarchical intervention and supervision mechanism and the intervention level triggered by the target electric bike with corresponding vehicle identification, output corresponding supervision and management instructions. Through the pre-set hierarchical intervention and supervision mechanism, a tiered hierarchical supervision is achieved, deeply integrating detection work with actual violation constraint operations. It can also build a complete and effective behavioral constraint closed loop for high-frequency violators, effectively optimizing the actual supervision results. This application uses improved YOLO12 and other cutting-edge target detection algorithms to optimize the overall detection performance, and can integrate a multi-head self-attention mechanism to enhance the model's feature extraction capabilities. While maintaining high-precision detection capabilities, it also has excellent inference efficiency, making it suitable for deployment in various edge computing devices to meet the requirements for monitoring violations in different scenarios.

[0065] For example, this application can set multiple intervention levels based on the number of violations, and can further use a preset period as the interval, such as a year, half a year, month, week, etc., and use the number of violations within the preset period as the trigger, accumulating the number of violations within each period, and clearing the accumulated number of violations in the previous period before starting the accumulation of the number of violations in the next period; or, it can set multiple intervention levels based on the frequency of violations, and can further determine the frequency of violations by the time interval between the occurrence of multiple violations of the most recently set number, and use the frequency of violations within the preset period as the trigger, and count the frequency of violations within the preset period in each preset period; or, it can set multiple intervention levels based on the severity of violations, and can further distinguish the severity of violations according to no helmet worn, no helmet worn properly, helmet worn is unqualified, no helmet worn and other violations, etc.; the specific settings can be made according to actual conditions and are not limited.

[0066] The embodiments of this application are mainly illustrated by setting multiple intervention levels based on the number of violations as the triggering basis: like Figure 5 As shown, in one embodiment, step S400, based on a pre-set tiered intervention and supervision mechanism and the intervention level triggered by the target electric vehicle with the corresponding vehicle identifier, outputs a supervision and management instruction corresponding to the intervention level, including: Step S410: Obtain the number of violations for the target electric vehicle based on its vehicle identification number. Step S420: Based on a pre-set tiered intervention and supervision mechanism that is triggered by the number of violations, determine the intervention level triggered by the target electric vehicle and output the corresponding supervision and management instructions to the terminal device; the terminal device includes user terminal device and charging pile device.

[0067] Unlike related technologies that only provide alarm reminders after helmet detection and lack differentiated supervision functions that are deeply integrated with actual scenarios, this application's embodiments determine the intervention level triggered by the target electric vehicle and output supervision and management instructions corresponding to the intervention level to the terminal device. The terminal device includes various devices such as user terminal devices, charging pile devices, and other IoT infrastructure, which can optimize the diversity of supervision methods and the actual implementation effectiveness.

[0068] The embodiments of this application implement a tiered and progressive supervision and management mechanism based on the frequency of violations, including reminders, mandatory safety education, and restrictions on charging usage rights. This achieves automatic identification, tiered intervention, and closed-loop supervision of violations, significantly enhancing the practical effectiveness of the violation supervision plan. The system in this application achieves deep data interoperability and business linkage with the charging pile IoT system, constructing a complete and standardized closed-loop management system for cycling behavior constraints. It also incorporates a fixed monthly data statistics cycle or other pre-defined statistical cycles to facilitate the deduplication of valid violation information and the full-process statistical management of violation objection appeals. This makes the statistical work on violations more scientific and standardized, and fully protects the legitimate rights and interests of cyclists.

[0069] In one embodiment, before performing step S420, which determines the intervention level triggered by the target electric vehicle based on a pre-set tiered intervention and supervision mechanism triggered by the number of violations, the method further includes: The system retains screenshots of the scene from the acquired camera images and accumulates the current number of violations for the target electric vehicle corresponding to the vehicle identification number.

[0070] The behavior monitoring method of this application is applied to the behavior monitoring system. The system maintains a helmet-wearing violation calculator for each vehicle identifier (such as vehicle binding information, user ID or other vehicle user identity, license plate number) in the backend database. When it detects that an electric vehicle rider of the vehicle identifier is not wearing a helmet, the violation counter is incremented by 1, and the violation timestamp, camera ID, and on-site screenshots from the captured images are recorded to retain the corresponding violation records, which facilitates later statistics and provides a basis for handling user appeals.

[0071] Electric vehicles generally require regular charging and battery swapping during use. In addition to issuing warnings when violations occur, the embodiments of this application can also be linked with charging pile equipment and other Internet of Things infrastructure. Furthermore, by restricting the charging behavior of electric vehicles with violations or requiring users to learn and improve their driving skills, the application can strongly encourage and regularly intervene in the behavior of riders wearing helmets, and persuade riders to wear helmets for safe riding.

[0072] like Figure 6 , Figure 7 As shown, in one embodiment, step S420, determining the intervention level triggered by the target electric vehicle based on a pre-set tiered intervention and supervision mechanism triggered by the number of violations, and outputting the corresponding supervision and management instruction to the terminal device, includes: Step S421: Based on a pre-set tiered intervention and supervision mechanism triggered by the number of violations, when it is determined that the target electric vehicle has triggered the first intervention level, a violation alarm command corresponding to the first intervention level is output to the terminal device to control the terminal device to display a violation risk notification. Step S422: When it is determined that the target electric vehicle has triggered the second intervention level, output the forced learning instruction corresponding to the second intervention level to the charging pile device to control the charging pile device to maintain the locked state within the preset locking time, and control the charging pile device to play the safety education video in the locked state. Step S423: If it is determined that the target electric vehicle has triggered the third intervention level, the process of disabling the target electric vehicle is triggered, and the charging disabling command corresponding to the third intervention level is output to the charging pile device to control the charging pile to reject the charging request of the corresponding target electric vehicle, and to control the user terminal device to display the disabling notification. The disabling notification is used to indicate the reason for disabling, the duration of disabling, and the steps to unblock. The threshold for the number of violations that triggers the first intervention level is less than the threshold for the number of violations that triggers the second intervention level, which in turn triggers the threshold for the number of violations that triggers the third intervention level.

[0073] For example, the threshold for the number of violations that triggers the first intervention level can be set to 2, the threshold for the number of violations that triggers the second intervention level can be set to 4, and the threshold for the number of violations that triggers the third intervention level can be set to 5. The threshold for the number of violations can be set to other appropriate values ​​or a range of values, depending on the actual scenario and the period for counting violations. The corresponding intervention level is determined when the actual number of violations falls within the corresponding range. No limitation is imposed here.

[0074] like Figure 8 As illustrated, the behavioral supervision method of this application can be applied to a behavioral supervision system that implements this method. This system may include, but is not limited to, a front-end device layer, an edge computing layer, a user terminal, and a platform management terminal, and can be linked with a charging pile IoT system. The edge computing module of the front-end device layer drives the edge computing nodes. This module can be equipped with an embedded processor with neural network acceleration capabilities, such as an ARM SoC or Jetson series with an integrated NPU (Neural Processing Unit), responsible for running object detection inference tasks. When the edge computing module determines through image recognition that a cyclist is not wearing a helmet in the captured image, it collects and uploads the helmet-wearing event (e.g., it may include the vehicle identification tag of the violating electric vehicle, a screenshot, etc.) to the back-end management server of the platform management terminal. This provides a basis for subsequent frequency statistics, determination of the triggered intervention level, and tiered control.

[0075] like Figure 7 , Figure 9 As shown, the backend management server is responsible for backend business logic such as storing violation records, counting user violations, managing supervision strategies, providing learning and reward points for violations, and pushing messages. After receiving helmet-wearing event data reported by the edge computing module, the backend management server accumulates the current violation count for the target electric vehicle corresponding to the vehicle identifier (for example, when using the monthly violation count as the trigger, it increments the monthly violation counter for the target electric vehicle corresponding to that vehicle identifier by 1). When a rider is detected not wearing a helmet, the system queries the backend to obtain the cumulative violation count N for that electric vehicle in the current month or other preset periods (statistical periods), and determines the intervention level triggered for the target electric vehicle based on the violation count N. The charging pile device's controller and user terminal complete corresponding operations based on instructions for different intervention levels, thereby completing corresponding operations at different supervision stages and providing relevant guidance to the user.

[0076] The specific implementation method of this application is as follows: When N≤2, the first intervention level is determined to be triggered, i.e., the first-level supervision is triggered. A violation alarm instruction containing a violation reminder is directly sent to the user's terminal, controlling the display of the violation risk notification on the user's terminal. In the first-level supervision scenario, the user's APP receives and displays the violation reminder and warning notification, providing the user with initial warning education. For example, when the first intervention level is determined to be triggered, a violation reminder and warning notification are pushed to the user's APP. When the number of violations is 1, an APP push notification can be sent to the target rider's corresponding user terminal, or a screen prompt can be sent through the target rider's selected charging station, or a screen prompt can be sent through the corresponding rider's location to select the nearest charging station, to display the violation reminder and warning notification; when the number of violations is 2, the violation reminder and warning notification can be displayed through the user's APP and the charging station. The displayed notification content can further emphasize that multiple violations will trigger mandatory measures, and push helmet purchase channel information. When the violation triggers the first intervention level, the user is informed of the behavior of not wearing a helmet and the safety risks, but there is no mandatory constraint.

[0077] If the system determines that a target electric vehicle has triggered the second or third intervention level, it will execute measures such as mandatory learning or disabling charging, and trigger a disabling process for the target electric vehicle. Taking the second intervention level as an example, the system outputs a mandatory learning instruction corresponding to the second intervention level to the charging pile equipment (mainly the charging pile controller). The mandatory learning instruction format includes, but is not limited to, vehicle identification (such as device ID, license plate number, etc.), measure type (such as mandatory learning), and effective duration. After receiving the mandatory learning instruction, the charging pile controller executes the corresponding operation, which is used to play safety education content on the screen under the mandatory learning measure, and automatically unlocks the device after the learning time is completed. Taking the third intervention level as an example, the system outputs a disabling charging instruction corresponding to the third intervention level to the charging pile equipment (mainly the charging pile controller). The disabling charging instruction format includes, but is not limited to, vehicle identification (such as device ID, license plate number, etc.), measure type (such as disabling charging), and effective duration. After receiving the disabling charging instruction, the charging pile controller executes the corresponding operation, which is used to resolve the vehicle charging request of the target electric vehicle that has triggered the third intervention level under the disabling charging measure, and displays explanatory information and the unblocking channel on the screen. After implementing the corresponding mandatory learning or charging disabling measures, the charging pile controller sends the execution result back to the backend management system to confirm that the measures have taken effect. The system then sends a notification to the user via the user's app, informing them of the current monitoring status, lifting conditions, and methods for lifting the restrictions.

[0078] When N=3 or 4, the second intervention level is triggered, i.e., the second-level supervision is triggered. A mandatory learning command is issued to the charging pile device. In the second-level supervision scenario, after receiving the mandatory learning command, the charging pile controller first sends a lock success status back to the backend management server, then starts playing a 3-5 minute safety education video on the screen. After the learning timer ends, the charging permission is automatically unlocked, and the learning completion and unlock status is sent back to the backend management server again. For example, when the second intervention level is triggered, a lock and mandatory learning command (LEARNING) is issued to the charging pile controller. When there are 3 violations, mandatory learning can be triggered, and a safety education video (preset duration, default 3 minutes) can be displayed on the charging pile. The rider must complete the learning before the charging function in that area can be unlocked. The learning content includes helmet wearing regulations, traffic safety regulations, etc. When there are 4 violations, mandatory learning is triggered, and a safety education video is displayed on the charging pile. The duration of the safety education video is extended (default 5 minutes), and the owner's emergency contact can be notified simultaneously based on the obtained vehicle identification and vehicle user identity and other bound information.

[0079] When N≥5, the third intervention level is triggered, i.e., a level 3 supervision is initiated. A charging disabling command is issued to the charging pile device, and the user's device can be controlled to restrict disabling notifications. In the level 3 supervision scenario, after receiving the charging disabling command, the charging pile controller sends back the effective status, restricting the charging permissions of the corresponding target electric vehicle until it receives an unblocking command from the backend management server, at which point the disabling restriction is lifted, and normal charging service is restored. In the level 3 supervision scenario, the user's APP receives the charging disabling notification and unblocking instructions, guiding the user to complete online safety education and submit a commitment letter on the APP. After submission, the status is synchronized to the backend server as the basis for lifting the charging disabling permission, ultimately achieving a closed-loop user-side process from violation warning to proactive rectification. For example, when the third intervention level is triggered, a disabling command (LOCK) is issued to the charging pile controller, a charging disabling notification and unblocking channel are pushed to the user's APP, and an unblocking command (UNLOCK) is issued after the user completes rectification. When the third intervention level is triggered, the area where the target electric vehicle exhibited violations is designated as the violation area. If the number of violations reaches 5 or more, the system will disable the electric vehicle's access to charging stations within the violation area. The disabling duration is configurable (default is either until the end of the month or a full week, whichever is shorter). Unblocking requires completing a full safety course via the app and submitting a commitment letter. It should be noted that if violations continue to occur after completing the full course and submitting the commitment letter, the violation count will continue to accumulate, eventually leading to a further disabling of the vehicle.

[0080] It should be noted that the learning duration, disabling duration, trigger threshold, and other parameters in the aforementioned embodiments can all be flexibly configured through the backend management system to adapt to the management needs of different application scenarios.

[0081] like Figure 10 As shown, in one embodiment, the behavior supervision method further includes the following steps: Step S510: Respond to the supervision status update instruction uploaded by the user terminal device. The supervision status update instruction is used to indicate the current supervision status and whether the security commitment letter has been uploaded. Step S520: If the supervision status update instruction indicates that the current supervision status is completed and the safety commitment letter has been uploaded, the process of disabling the target electric vehicle with the corresponding vehicle identifier ends.

[0082] When the third-level supervision is triggered, the progress of the user's safety course learning and whether the user has uploaded a safety commitment letter are determined based on the supervision status update instruction uploaded by the user's terminal device. Linking the lifting of the charging ban with the progress of the safety course learning and the upload of the safety commitment letter achieves the purpose of mandatory constraints and standardization, and provides a basis for user appeals, resumption of charging, and whether to impose a charging ban for subsequent violations, making the entire charging ban process more reasonable and traceable.

[0083] In some alternative implementations, cyclists can appeal against incorrect records through a user app. The system can then determine and revoke the corresponding violation record after completing manual or automatic review of the received appeal request.

[0084] In one embodiment, the behavior supervision method further includes the following steps: With a preset period as the interval, in response to the trigger signal of the current reset period, the number of violations accumulated in the previous reset period is cleared to zero.

[0085] The embodiments of this application use a preset period as the interval, such as a year, half a year, month, or week. The number of violations within the preset period is used as the trigger. The number of violations within each period is accumulated, and the accumulated violations from the previous period are cleared before starting the accumulation for the next period. Taking a month as the preset period as an example, the violation count can be reset by the natural month (i.e., cleared to zero on the 1st of each month) to avoid the permanent accumulation of early violations affecting the actual supervision effect and consuming memory. To record valid violation counts and filter duplicate violations, this application can determine repeated violations by the same cyclist in consecutive detection frames as the same cycling event. When a violation is determined, it is counted as only one violation, and deduplication is performed with a minimum violation interval of 5 minutes.

[0086] The specific implementation steps of this application are as follows: At the start of each round of detection, the camera captures real-time images of the charging pile area or other target areas to obtain camera-captured images and outputs a video stream corresponding to the camera-captured images. The edge computing layer, as the edge computing end, decodes the video stream and performs operations such as image format conversion, noise reduction, and frame rate optimization. Finally, the standardized image data is sent to the image preprocessing module of the edge computing layer. The initial processing of the image can be completed without transmitting massive amounts of video data to a remote server. This not only significantly reduces data transmission pressure and processing latency, but also reduces environmental interference through noise reduction, frame rate optimization, and other operations, improving the accuracy and stability of subsequent recognition stages and laying the foundation for efficient operation of the entire process.

[0087] After receiving standardized image data, the image preprocessing module of the edge computing layer performs preprocessing operations such as image enhancement, cropping, and normalization on the camera-captured images. Image enhancement optimizes image quality in complex scenes such as backlighting and low light; cropping focuses on key areas and reduces interference from irrelevant information; and normalization adapts the image data to the input requirements of subsequent models, ensuring accurate recognition. The preprocessed images then enter the target detection stage. Based on predefined target features of a helmeted cyclist, the module identifies and locates the target bounding box from the preprocessed camera-captured images to determine violations. This avoids the omissions inherent in single-target detection, significantly reduces the probability of false positives, and effectively eliminates background interference, reducing unnecessary consumption of system resources and ensuring system stability and efficiency.

[0088] Helmet wearing is a core safety indicator for electric vehicle riders. When the behavioral monitoring method is also linked to the vehicle entrance and exit management system, if the camera footage confirms that the rider is wearing a helmet, the corresponding electric vehicle can be safely allowed to pass, the charging station can remain in normal standby mode, and the current round of detection and processing ends. If the camera footage confirms that the rider is not wearing a helmet, a violation is determined, and the vehicle's unique identifier is obtained. The vehicle identifier can be obtained through image recognition, feedback to relevant community administrators for judgment and supervision, or stored vehicle image information. Subsequently, the backend management server updates the cumulative violation count N, and differentiated monitoring measures are implemented based on the cumulative violation count.

[0089] Taking the threshold for the number of violations that triggers the first intervention level as 2, the threshold for the number of violations that triggers the second intervention level as 4, and the threshold for the number of violations that triggers the third intervention level as 5 as an example, when the cumulative number of violations does not exceed 2, the system will execute the first-level supervision and send a violation warning reminder to the user's terminal and other device terminals. Through a lightweight warning and education method, risk intervention is achieved in the early stage of violations, reducing the probability of serious violations and reducing the risk of violations in a low-cost intervention method.

[0090] When the number of violations reaches 3 or 4, the system will implement a secondary supervision mechanism. The charging station will be temporarily locked and a safety video will be played. The charging station will only be automatically unlocked after the rider completes the video learning. This is used to strengthen the user's safety awareness through mandatory learning, and at the same time, to increase the user's attention to the consequences of violations by temporarily restricting charging access. This approach combines education and guidance with behavioral constraints to reduce the number of repeated violations by users.

[0091] When the number of violations reaches 5 or more, the system will implement a three-level supervision, directly disabling the target electric vehicle's access to charging stations within the controlled area. The user must complete a lengthy online learning session via the app and sign a safety commitment letter before the system restores the vehicle's charging permissions. This effectively creates strong constraints on serious violations such as repeated offenses, maintaining overall safety and order. After the aforementioned supervision measures are completed, the system will end the current round of detection and processing, awaiting the next scenario trigger, forming a complete safety management closed loop and achieving full-process control over the safety of electric vehicle charging stations.

[0092] It should be noted that each round of detection processing can be executed according to a set cycle; it can also be executed after identifying at least some targets such as cyclists, electric vehicles, and helmets from the images captured by the camera; or it can be executed after receiving a charging request uploaded by the terminal device; the specific timing of each round of detection processing can be determined based on actual needs, and is not limited here.

[0093] In some specific embodiments, image acquisition modules such as cameras can be installed at a height of about 2.2m above the ground, with a downward angle of about 20°, covering the entire width of the charging shed entrance passage, so that the camera's field of view corresponds to the detection area.

[0094] When a rider enters the detection area on an electric bike, the image acquisition module captures video streams and other camera images at 25 FPS. After the motion detection module detects the moving target, it activates the inference process.

[0095] An improved YOLO12 model is used to identify and locate target bounding boxes within the same video frame. A head detection region is extracted from the target bounding box according to a predefined extraction area. Based on predefined helmet-wearing features, a violation judgment is made on the extracted head detection region. If the image features of the head detection region match the predefined helmet-wearing features, it is determined that the target cyclist is wearing a helmet, and no violation has occurred. If no violation of not wearing a helmet is found, no enforcement measures are triggered, and the corresponding electric vehicle is allowed to proceed normally and charge.

[0096] In another specific embodiment, in the scenario described above, if the image features of the head detection area do not match the predefined helmet-wearing features, it is determined that the target cyclist is not wearing a helmet, thus confirming a violation of helmet-wearing regulations. Upon confirming a helmet-wearing violation, the system queries backend violation records and updates the violation count based on this violation.

[0097] The threshold for the number of violations that triggers the first intervention level is set to 2, the threshold for the number of violations that triggers the second intervention level is set to 4, and the threshold for the number of violations that triggers the third intervention level is set to 5.

[0098] When the updated violation count triggers the first intervention level (e.g., if the target electric vehicle has 0 violations this month, this record will be updated to 1; or if the target electric vehicle has 1 violation this month, this record will be updated to 2), the system will send a push notification to the user via the app, informing them of the helmet-wearing behavior and safety risks. Charging function remains enabled and is not restricted. The system archives violation records (including but not limited to timestamps, screenshots, and vehicle identification).

[0099] When the updated violation count triggers the second intervention level (e.g., if the target electric vehicle has 2 violations this month, the record will be updated to 3; or if the target electric vehicle has 3 violations this month, the record will be updated to 4), the system sends a mandatory learning command to the charging station. The charging station screen displays a safety education video, including the correct way to wear a helmet and relevant regulations. The video duration is 3 minutes or other preset lock durations. The charging station remains locked until the learning time is completed and automatically unlocks after the preset lock duration is reached. After the user completes the learning, the charging station unlocks and can be used normally; the system records the learning completion timestamp.

[0100] When the updated violation count triggers the third intervention level (e.g., if the target electric vehicle has at least 4 violations this month, the number will be updated to at least 5 after this record), the system sends a charging disabling command to the charging station, disabling the vehicle's charging privileges in this area for a period until the end of the month (7 days if less than 7 days remain). The charging station will refuse the charging request from the violating electric vehicle, and the screen will display a notification stating that the account has been suspended for repeatedly riding without a helmet while charging, and requesting the user to complete a safety course and submit a safety commitment letter through the app to apply for unbanning. The system will also send a notification through the user's app informing them of the reason for the ban, the duration of the ban, and the steps to unban.

[0101] In another specific embodiment, after a rider is found to be not wearing a helmet (i.e., after helmet status detection is completed), the electric vehicle detection phase begins. The improved YOLO12 model indiscriminately detects electric vehicles in the camera-captured images. Based on predefined electric vehicle characteristics, identified electric vehicles are used as candidate electric vehicles. Electric vehicle bounding boxes corresponding to the electric vehicle images are extracted from the pre-processed camera-captured images. Each electric vehicle bounding box is used to indicate a candidate electric vehicle. Besides determining whether a candidate electric vehicle corresponding to a bounding box is the target electric vehicle by judging whether the intersection-union ratio (IoU) between the extracted electric vehicle bounding box corresponding to the candidate rider image and the target bounding box corresponding to the target electric vehicle image showing the rider not wearing a helmet is not less than a set threshold, the system can also determine whether a candidate electric vehicle is in an empty state. When all electric vehicles in the image are determined to be in an empty state, the system does not trigger a reminder process to reduce false alarms for parked vehicles.

[0102] The embodiments of this application provide a behavior supervision system, which includes a memory, a processor, and a behavior supervision program stored in the memory and executable on the processor. When the behavior supervision program is executed by the processor, it implements the steps of the behavior supervision method as described in the above embodiments.

[0103] In the embodiments of this application, the processor may include, but is not limited to, a microcontroller, MCU (microcontroller unit), PLC (programmable logic controller), DSP (digital signal processor), and FPGA (field-programmable gate array). Besides the memory and processor, the behavior supervision system also includes a charging pile linkage interface module, a network interface, a user interface, and a communication bus for connecting and communicating the aforementioned components. The aforementioned interfaces may be wired or wireless. The memory may be high-speed RAM or stable non-volatile memory, such as disk storage. Alternatively, the memory may be a storage device independent of the aforementioned processor, which is not limited here.

[0104] like Figure 11As shown in the embodiments of this application, the behavior supervision system may include, but is not limited to, a front-end device layer, an edge computing layer, a user terminal, and a platform management terminal. The system first uses the front-end device layer to eliminate environmental interference such as strong light and low light at night through auxiliary lighting or infrared supplemental lighting, ensuring image acquisition quality in nighttime and low-light environments and providing stable imaging conditions for the high-definition camera. The high-definition camera acquires real-time images of the charging pile area or other target areas and outputs the corresponding video stream of the captured image. After the video stream is decoded by the edge computing module, it undergoes image format conversion, noise reduction, frame rate optimization, and other operations, and the standardized image data (video frames) is sent to the image preprocessing module of the edge computing layer. The image preprocessing module of the edge computing layer then performs image enhancement, cropping, and normalization on the acquired camera images. The preprocessed images are then used for target detection via an improved YOLO12 algorithm. Based on predefined target features of helmet-wearing cyclists, the module identifies and locates target bounding boxes in the preprocessed images, determining violations. Based on the violation determination, if a cyclist is confirmed to be not wearing a helmet, a screenshot of the violation event is simultaneously generated, along with the vehicle identification of the target electric vehicle (e.g., charging station identification, license plate number, etc.). The violation is then linked to the vehicle identification, and this information is recorded and uploaded to the backend management server. The backend management server processes the violation event data and vehicle identification information from the edge computing layer and can interact with the user interface and platform management layer. It can push violation notifications to user apps or mini-programs and open appeal channels, enabling information exchange with the user interface. The backend management server can also archive violation data to the platform management layer's violation record database and generate differentiated supervision instructions through a tiered supervision strategy engine, facilitating interaction with the platform management layer. The strategy engine generates tiered intervention and supervision strategies based on dimensions such as the number of violations, such as reminders and restrictions on charging permissions. Supervision and management instructions are sent to terminal devices such as the charging pile management system through the charging pile linkage interface module to perform operations such as voice reminders, charging permission restrictions, and locking of charging equipment.

[0105] The edge computing module drives the edge computing nodes. This module can be equipped with an embedded processor with neural network acceleration capabilities, such as an ARM SoC or Jetson series with an integrated NPU (Neural Processing Unit), responsible for running object detection inference tasks. The backend management server is responsible for backend business logic such as storing violation records, counting user violations, managing supervision strategies, providing learning and compensation for violations, and pushing messages. The charging pile linkage interface module can establish a connection with the charging pile controller via wired or wireless communication, supporting the issuance of control commands such as charging start, locking, and unlocking. Specific configurations can be customized based on actual needs and are not limited here.

[0106] The behavioral monitoring system of this application can effectively realize automated identification of violations, user interaction, and full-process control. This application has good scalability; the tiered thresholds for the number of violations, the monitoring content, the learning duration of mandatory learning related to the monitoring content, and the prohibition duration of charging restrictions related to the monitoring content can all be flexibly adjusted through backend configuration. Through an end-to-end detection process, it outputs the detection results for cyclists wearing helmets in a single pass. The charging pile linkage interface supports multiple protocols such as REST API and MQTT, and can be integrated with mainstream community governance, smart property management, and mobility operation platforms. Furthermore, violation records can be aggregated and statistically analyzed across charging areas, enabling unified management of violations by the same user (e.g., the same cyclist, the same license plate number) in different areas. The model used in this application can be updated online and supports remote model updates via OTA (Over-The-Air), eliminating the need for on-site maintenance.

[0107] It should be noted that the user-related data involved in this application (such as vehicle binding information, vehicle user identity, license plate number, entry and exit location information of the detection area, user geographic information, safety commitment documents, and video capture of the detection area) were all obtained with the user's permission or consent. In other words, when this application is applied to specific products or technologies, user permission is required to obtain and process the relevant data, and the processing of the relevant data must comply with the relevant laws, regulations and regulatory standards of the relevant countries and regions.

[0108] like Figure 12 As shown, for example, when it is necessary to obtain a user's current geographical location, a location acquisition prompt can be displayed on the user's terminal (such as the user's APP). After receiving the user's confirmation operation for the location acquisition prompt, the terminal can obtain the user's current geographical location.

[0109] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A behavior supervision method applied to helmet-wearing behavior detection, characterized in that, The methods for encouraging behavior include: Acquire camera images of the target area and preprocess the acquired camera images; Based on predefined target features of helmet-wearing cyclists, the system identifies and locates target bounding boxes from pre-processed camera images to determine violations. Based on the results of the violation judgment, if it is determined that a rider has violated the rule of not wearing a helmet, the vehicle identification of the target electric vehicle is determined; Based on the pre-set tiered intervention and supervision mechanism and the intervention level triggered by the target electric vehicle with the corresponding vehicle identification, the corresponding supervision and management instructions are output.

2. The behavior supervision method as described in claim 1, characterized in that, The intervention level triggered by the target electric vehicle based on the pre-set hierarchical intervention and supervision mechanism and the corresponding vehicle identification number, and the corresponding supervision and management instructions output, include: Based on the vehicle identification of the target electric vehicle, obtain the number of violations corresponding to the target electric vehicle; Based on a pre-set tiered intervention and supervision mechanism triggered by the number of violations, the intervention level triggered by the target electric vehicle is determined, and the supervision and management instructions corresponding to the intervention level are output to the terminal device; the terminal device includes user terminal device and charging pile device.

3. The behavior supervision method as described in claim 2, characterized in that, The pre-set, tiered intervention and supervision mechanism, triggered by the number of violations, determines the intervention level triggered by the target electric vehicle and outputs corresponding supervision and management instructions to the terminal device, including: Based on a pre-set tiered intervention and supervision mechanism triggered by the number of violations, when the target electric vehicle is determined to have triggered the first intervention level, a violation alarm command corresponding to the first intervention level is output to the terminal device to control the terminal device to display a violation risk notification. When it is determined that the target electric vehicle has triggered the second intervention level, a forced learning instruction corresponding to the second intervention level is output to the charging pile device to control the charging pile device to maintain the locked state within the preset locking time and to control the charging pile device to play a safety education video in the locked state. If it is determined that the target electric vehicle has triggered the third intervention level, the process of disabling the target electric vehicle is triggered, and the charging disabling command corresponding to the third intervention level is output to the charging pile equipment to control the charging pile to refuse the charging request of the corresponding target electric vehicle, and to control the user terminal equipment to display the disabling notification. The disabling notification is used to indicate the reason for disabling, the duration of disabling, and the steps to unblock. The threshold for the number of violations that triggers the first intervention level is less than the threshold for the number of violations that triggers the second intervention level, which in turn triggers the threshold for the number of violations that triggers the third intervention level.

4. The behavior supervision method as described in claim 2, characterized in that, The behavioral supervision method also includes the following steps: In response to a supervision status update instruction uploaded by a user terminal device, the supervision status update instruction is used to indicate the current supervision status and whether a security commitment letter has been uploaded; If the monitoring status update instruction indicates that the current monitoring status is completed and the safety commitment letter has been uploaded, the process of disabling the target electric vehicle with the corresponding vehicle identifier will end.

5. The behavior supervision method as described in claim 2, characterized in that, Before executing the step of determining the intervention level triggered by the target electric vehicle using the pre-set, violation-based tiered intervention and supervision mechanism, the following steps are also included: The system retains screenshots of the scene from the acquired camera images and accumulates the current number of violations for the target electric vehicle corresponding to the vehicle identification number.

6. The behavior supervision method as described in claim 1, characterized in that, The process of identifying and locating the target bounding box from pre-processed camera images based on predefined target features of helmet-wearing cyclists, and determining violations, includes: Based on predefined target features of cyclists wearing helmets, target bounding boxes are identified and located from preprocessed camera images. The target bounding boxes include bounding boxes corresponding to cyclists wearing helmets and bounding boxes corresponding to cyclists not wearing helmets. Extract the head detection region from the target bounding box according to the preset extraction area; Based on predefined helmet wearing characteristics, the extracted head detection area is used to determine violations. If the image features of the head detection area match the predefined helmet wearing features, it is determined that the target cyclist is wearing a helmet and no violation has occurred. If the image features of the head detection area do not match the predefined helmet wearing features, it is determined that the target cyclist is not wearing a helmet, thus confirming the violation of not wearing a helmet.

7. The behavior supervision method as described in claim 1, characterized in that, Based on the results of the violation judgment, when it is determined that a rider has violated the rule of not wearing a helmet, the process of identifying the vehicle identification of the target electric vehicle includes the following steps: Based on the violation judgment results, when it is determined that a rider has violated the rule of not wearing a helmet, the electric vehicle bounding box corresponding to the electric vehicle image is extracted from the pre-processed camera image according to the predefined electric vehicle characteristics. Each electric vehicle bounding box is used to indicate the candidate electric vehicle. Determine the intersection-union ratio (IU) between each extracted electric vehicle bounding box and the target bounding box. If the IU is not less than a set threshold, determine that the candidate electric vehicle indicated by the electric vehicle bounding box is the target electric vehicle corresponding to the rider who is not wearing a helmet. Identify the vehicle identification number of the target electric vehicle.

8. The behavior supervision method as described in any one of claims 1 to 7, characterized in that, The vehicle identification includes vehicle binding information, vehicle user identity, and license plate number; The vehicle identifier for determining the target electric vehicle includes: Extract the license plate detection box of the target electric vehicle from the pre-processed camera image, and obtain the license plate number of the target electric vehicle from the license plate detection box to obtain the vehicle identification of the target electric vehicle. And / or, in response to behavioral supervision instructions from the management end, determine the vehicle identification of the target electric vehicle; And / or, based on the stored vehicle image information, determine the vehicle image information and corresponding vehicle identifier that match the target electric vehicle.

9. The behavior supervision method as described in any one of claims 1 to 7, characterized in that, The behavioral supervision method also includes the following steps: With a preset period as the interval, in response to the trigger signal of the current reset period, the number of violations accumulated in the previous reset period is cleared to zero.

10. A behavior supervision system, characterized in that, The method includes a memory, a processor, and a behavior supervision program stored in the memory and executable on the processor, wherein the behavior supervision program, when executed by the processor, implements the steps of the behavior supervision method as described in any one of claims 1 to 9.

Images