Electric bicycle helmet wearing identification and driving safety control method and system

Abstract

The embodiment of the invention relates to the technical field of traffic safety management, and discloses an electric bicycle helmet wearing recognition and driving safety control method and system.The method comprises the steps that firstly, a fisheye camera is used for capturing an image, and the height difference between a driver and passengers is analyzed; when the height difference exceeds a threshold value, the target shooting angle range of the video camera is dynamically calculated, then the target shooting angle range of the video camera is converted into a pitching angle control parameter of the holder, and then the holder is driven to adjust the posture so as to accurately capture high-quality video data of the head area of the driver and passengers; and then the wearing state of the helmet is identified through a deep learning model, and finally, a vehicle control system is linked to execute a safety strategy (such as starting limitation or alarm) according to an identification result. Therefore, the problem of head shielding or view offset caused by height difference is solved through dynamic view angle adjustment, and the reliability and accuracy of image acquisition are improved, so that the accuracy of helmet wearing recognition is improved, and the accuracy of electric bicycle safety control is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic safety management, and particularly to a method and system for identifying the wearing of e-bike helmets and controlling driving safety. Background Art

[0002] In recent years, with the popularization of e-bikes, their safe driving issues have attracted much attention. According to statistics, not wearing a helmet is one of the important reasons for the casualties of riders in e-bike traffic accidents. Therefore, regulations in various countries generally require e-bike drivers and passengers to wear helmets. However, traditional supervision methods rely on manual inspections or fixed camera recognition, which have problems such as poor real-time performance, limited coverage, and insufficient adaptability to complex scenarios. Especially in the scenario of two-person riding, the height difference between the driver and the passenger may cause the head area to be blocked or exceed the camera's field of view, further reducing the detection accuracy, thus leading to a decrease or even failure in the accuracy of e-bike safety control. Summary of the Invention

[0003] The main objective of the present invention is to provide a method and system for identifying the wearing of e-bike helmets and controlling driving safety, aiming to solve the technical problem of low detection accuracy of helmet wearing in the prior art, which leads to the failure of safety control.

[0004] To achieve the above objective, in a first aspect, an embodiment of the present application provides a method for identifying the wearing of e-bike helmets and controlling driving safety, which is applied to an e-bike safety control system. The e-bike safety control system includes a pan-tilt and a fisheye camera disposed at the position of the handlebar, and a video camera is provided on the pan-tilt. The method includes: Obtaining a target image by capturing an image with the fisheye camera, and determining the height information of the driver and the passenger based on the target image. The height information includes at least the height difference between the driver and the passenger; In the case where the height difference between the driver and the passenger is greater than or equal to a height threshold, determining a target shooting angle range of the video camera based on the height information; Determining a target pitch angle range of the pan-tilt based on the target shooting angle range of the video camera; Controlling the pan-tilt to move within the target pitch angle range to obtain video data of the driver and the passenger to obtain target video data; Inputting the target video data into a pre-trained helmet wearing recognition model to obtain a helmet wearing recognition result; Performing e-bike driving safety control based on the helmet wearing recognition result.

[0005] In a possible implementation manner, the determining the height information of the driver and the passenger based on the target image includes: Perform image semantic recognition on the target image to obtain the highest point position information of the driver and passengers in the target image; Determine the blind area height information based on the highest point position information of the driver and passengers in the target image to obtain the first blind area height and the second blind area height. The first blind area height represents the shooting blind area height of the fisheye camera for the driver, and the second blind area height represents the shooting blind area height of the fisheye camera for the passengers; Determine the height information of the driver and passengers based on the initial calibration height, the first blind area height, and the second blind area height of the fisheye camera.

[0006] In a possible implementation, the determining the target shooting angle range of the video camera according to the height information includes: Take the highest point position of the higher one of the driver and passengers as the highest point of the shooting position, and take the position of the e-bike seat as the lowest point of the shooting position; Determine the range between the highest point and the lowest point of the shooting position as the target shooting angle range of the video camera.

[0007] In a possible implementation, the determining the target pitching angle range of the pan-tilt head according to the target shooting angle range of the video camera includes: Establish a conversion relationship according to the relative position relationship between the video camera and the pan-tilt head. The conversion relationship is used to represent the corresponding relationship between the viewing angle change of the video camera and the pitching angle of the pan-tilt head; Determine the target pitching angle range according to the conversion relationship and the target shooting angle range.

[0008] In a possible implementation, the inputting the target video data into a pre-trained helmet wearing recognition model to obtain a helmet wearing recognition result includes: Obtain the network connection status of the wireless communication gateway, where the wireless communication gateway is used to send the target video data to the cloud server; In the case of abnormal network connection status of the wireless communication gateway, input the target video data into the helmet wearing recognition model in the edge server to perform helmet wearing recognition to obtain a helmet wearing recognition result; In the case of normal network connection status of the wireless communication gateway, input the target video data into the helmet wearing recognition model in the cloud server to perform helmet wearing recognition to obtain a helmet wearing recognition result, where the cloud server regularly updates the helmet wearing recognition model of the edge server.

[0009] In a possible implementation, inputting the target video data into a pre-trained helmet-wearing recognition model to obtain a helmet-wearing recognition result includes: Extracting consecutive video frames from the target video data; Performing head feature recognition on the extracted video frames to obtain head video frames; Using a deep learning model to extract head contour features from the head video frames to obtain an initial inner contour and an initial outer contour; Performing circumferential extension alignment on the initial inner contour and the initial outer contour to obtain a target inner contour and a target outer contour; Determining that the average distance value between the target inner contour and the target outer contour is greater than or equal to a preset distance value, and determining that the target user is wearing a helmet; Determining that the average distance value between the target inner contour and the target outer contour is less than the preset distance value, and determining that the target user is not wearing a helmet.

[0010] In a possible implementation, controlling the pan-tilt head to move within the target pitch angle range to obtain video data of the driver and passengers to obtain target video data includes: Obtaining the driving speed of the electric bicycle, and determining the pitch movement speed of the pan-tilt head according to the driving speed of the electric bicycle, where the pitch movement speed of the pan-tilt head is positively correlated with the driving speed of the electric bicycle; Controlling the pan-tilt head to move at the pitch movement speed and within the target pitch angle range to obtain video data of the driver and passengers to obtain target video data.

[0011] In a possible implementation, controlling the pan-tilt head to move within the target pitch angle range to obtain video data of the driver and passengers to obtain target video data includes: Obtaining the driving speed of the electric bicycle, and determining the pitch movement frequency of the pan-tilt head according to the driving speed of the electric bicycle, where the pitch movement frequency of the pan-tilt head is positively correlated with the driving speed of the electric bicycle; Controlling the pan-tilt head to move at the pitch movement frequency and within the target pitch angle range to obtain video data of the driver and passengers to obtain target video data.

[0012] In a possible implementation, performing electric bicycle driving safety control according to the helmet-wearing recognition result includes: Before the electric bicycle starts, if it is determined that the helmet-wearing recognition result is in the state of not wearing a helmet, controlling the electric bicycle to be forced to be in a stopped state; During the driving of an electric bicycle, if it is determined that the helmet wearing recognition result is in the state of helmet removal or helmet slipping, the electric bicycle is controlled to decelerate and a warning signal is triggered.

[0013] In a second aspect, an electric bicycle safety control system is further provided in an embodiment of the present application, including: a pan-tilt and a fisheye camera disposed at the position of the electric bicycle handlebar, and a video camera is disposed on the pan-tilt; and a memory and a processor, the memory is used to store program codes; the processor is used to call the program codes to execute the method as described in the first aspect.

[0014] Different from the prior art, an electric bicycle helmet wearing recognition and driving safety control method provided in an embodiment of the present application is implemented by the collaborative work of a pan-tilt, a fisheye camera and a video camera at the handlebar. First, the fisheye camera captures an image and analyzes the height difference between the driver and the passenger; when the height difference exceeds a threshold, the target shooting angle range of the video camera is dynamically calculated, and then the target shooting angle range of the video camera is converted into the pitch angle control parameter of the pan-tilt, and then the pan-tilt is driven to adjust its posture to accurately capture high-quality video data of the head area of the driver and the passenger; then the helmet wearing state is recognized through a deep learning model, and finally, according to the recognition result, the vehicle control system is linked to execute safety strategies (such as restricting startup or alarming, etc.). In this way, the problem of head occlusion or vision deviation caused by height difference is solved through dynamic perspective adjustment, the reliability and accuracy of image acquisition are improved, thereby improving the accuracy of helmet wearing recognition, and further improving the accuracy of electric bicycle safety control. Description of the Drawings

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on the structures shown in these drawings without creative efforts.

[0016] Figure 1 It is a schematic diagram of the application scenario of electric bicycle helmet wearing recognition in some embodiments of the present application; Figure 2 It is a schematic flowchart of an electric bicycle helmet wearing recognition and driving safety control method in some embodiments of the present application; Figure 3 It is a schematic flowchart of step S500 of an electric bicycle helmet wearing recognition and driving safety control method in some embodiments of the present application; Figure 4 It is a schematic diagram of the hardware structure of an electric bicycle safety control system in some embodiments of the present application.

[0017] The realization of the object, functional features and advantages of the present invention will be further described with reference to the embodiments and the accompanying drawings. Detailed Embodiments

[0018] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0019] It should be noted that all directional indications (such as up, down, left, right, front, back...) in the embodiments of the present invention are only used to explain the relative positional relationship and movement conditions between components in a specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications will also change accordingly.

[0020] In addition, the descriptions involving "first", "second", etc. in the present invention are only for descriptive purposes, and cannot be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In addition, "and / or" throughout the text includes three scenarios. Taking A and / or B as an example, it includes the technical solution of A, the technical solution of B, and the technical solution that A and B are satisfied at the same time. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of those of ordinary skill in the art to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0021] In recent years, with the popularization of electric bicycles, their safe driving issues have attracted much attention. According to statistics, not wearing a helmet is one of the important reasons for the casualties of riders in electric bicycle traffic accidents. Therefore, regulations in various countries generally require electric bicycle drivers and passengers to wear helmets. However, traditional supervision means rely on manual inspections or fixed camera recognition, which have problems such as poor real-time performance, limited coverage, and insufficient adaptability to complex scenarios. Especially in the scenario of two-person riding, the height difference between the driver and the passenger may cause the head area to be blocked or exceed the camera's field of view, further reducing the detection accuracy, resulting in a decrease or even failure in the accuracy of electric bicycle safety control.

[0022] Such as Figure 1As shown in the figure, the electric bicycle safety control system in the embodiment of the present application includes a pan-tilt (not shown) disposed at the handlebar position and a fish-eye camera 100. A video camera (not shown) is disposed on the pan-tilt. Among them, the fish-eye camera 100 can be disposed at the left handle position, and the video camera is disposed at the right handle position. The installation position of the fish-eye camera 100 is fixed and is used to capture images of the electric bicycle and its surrounding environment. For example, the fish-eye camera 100 is installed in the direction of the driver's seat to capture images of the driver or the passenger.

[0023] It can be understood that since the viewing angle range of the fish-eye camera 100 is fixed and there are differences in the heights of the driver and the passenger, when the driver and the passenger are completely within the viewing angle range of the fish-eye camera after taking their seats, the fish-eye camera can directly sense the height information of the driver and the passenger. For example, the height of the user can be directly obtained based on the captured image of the fish-eye camera. When the driver or the passenger is not completely within the viewing angle range of the fish-eye camera after taking their seats, that is, there is a visual blind area for the fish-eye camera, the fish-eye camera cannot directly sense the height information of the driver and the passenger.

[0024] That is to say, when the height difference between the driver and the passenger is large, the height difference between the FOVs of the fish-eye camera 100 is relatively small compared to the height difference between the driver and the passenger. Therefore, the fish-eye camera 100 cannot fully capture the head images of the driver and the passenger. For example, Figure 1 In the figure, the line of sight S1 is the upper line of sight of the fish-eye camera 100, and the line of sight S2 is the lower line of sight of the fish-eye camera 100.

[0025] Such as Figures 1 - 3 As shown in the figure, the following takes the electric bicycle safety control system executing the electric bicycle helmet wearing recognition and driving safety control method as an example for description. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here. Please refer to the appendix Figure 2 The method includes the following steps S100 - step S600: Step S100: Obtain the captured image of the fish-eye camera to obtain a target image, and determine the height information of the driver and the passenger according to the target image. The height information at least includes the height difference between the driver and the passenger; In the embodiment of the present application, the fish-eye camera is used to capture images of the driver and the passenger to obtain a target image; and by analyzing the target image, the driver and the passenger are identified, and their height information is calculated, especially the height difference between the driver and the passenger.

[0026] In one embodiment, the step of determining the height information of the driver and the passenger according to the target image includes: Perform image semantic recognition on the target image to obtain the highest point position information of the driver and passengers in the target image; Determine the blind area height information based on the highest point position information of the driver and passengers in the target image to obtain the first blind area height and the second blind area height. The first blind area height represents the shooting blind area height of the fish-eye camera for the driver, and the second blind area height represents the shooting blind area height of the fish-eye camera for the passengers; Determine the height information of the driver and passengers based on the initial calibration height, the first blind area height, and the second blind area height of the fish-eye camera.

[0027] Specifically, in the embodiment of the present application, first perform semantic analysis on the target image to identify the highest point positions of the driver and passengers (such as Figure 1 the shoulder G1, the forehead G2, etc. in the figure). Then, based on the identified highest point position information, combined with big data or ergonomic data, estimate the unphotographed height part (i.e., the blind area height) caused by the blind area of the fish-eye camera. Finally, combined with the initial calibration height of the fish-eye camera and the estimated result of the blind area height, calculate the actual height information of the driver and passengers through calculation.

[0028] Exemplarily, first, an instance segmentation model based on deep learning (such as Mask R-CNN or YOLACT) can be used to segment the areas of the driver and passengers in the target image. In the segmented area, through a human key point detection model (such as OpenPose or HRNet), identify key points such as the shoulders, ears, or the top of the head of the target person, and select the key point with the highest vertical coordinate (i.e., closest to the top of the image) in the image as the "highest point position". Then, according to the detected highest point position type (such as the ear), query the pre-set ergonomic database to determine the blind area height (for example, take the average distance from the ear to the top of the head of an adult as the blind area height). During the process, a height estimation model (such as proportional calculation based on the key point spacing) can be introduced, combined with parameters such as the detected shoulder width and torso length, to dynamically adjust the blind area height and reduce the error caused by individual body type differences. Then, according to the axis coordinate (vertical direction) in the camera coordinate system, combined with the camera installation height, calculate the actual ground height of the highest point position, so as to obtain the actual height information of the driver and passengers. According to the actual height information of the driver and passengers, the height difference between the two can be determined.

[0029] Step S200: When the height difference between the driver and the passengers is greater than or equal to the height threshold, determine the target shooting angle range of the video camera according to the height information; It can be understood that when the height difference between the driver and the passenger is small, it indicates that their head positions are roughly on the same horizontal plane. In this case, simply adjusting the shooting angle of the video camera to this common horizontal position can ensure that the head images of both are captured. However, if the height difference between the driver and the passenger is significant, it means that their head positions are not on the same horizontal plane. At this time, due to the limitation of a single perspective, the video camera may not be able to capture the head images of both simultaneously. To solve this problem, the embodiment of the present application dynamically adjusts the pitching angle of the video camera to ensure that the heads of the driver and the passenger can be comprehensively captured.

[0030] In one embodiment, the step of determining the target shooting angle range of the video camera according to the height information includes: Taking the highest point position of the height value of the higher one of the driver and the passenger as the highest point of the shooting position, and taking the position of the electric bicycle seat as the lowest point of the shooting position; Determining the range between the highest point and the lowest point of the shooting position as the target shooting angle range of the video camera.

[0031] Specifically, select the highest point of the head of the higher one of the driver and the passenger as the upper limit of the shooting range to ensure that the camera can cover this height. For example, according to the actual heights of the driver and the passenger calculated in step S100, directly compare the two values, and select the highest point coordinate corresponding to the larger value as the highest point of the shooting position. Since the seat is the fixed support point of the driver and the passenger and its height is relatively stable, the position of the electric bicycle seat can be used as the lower limit of the shooting range. After obtaining the upper limit and the lower limit of the shooting range, the target shooting angle range of the video camera can be determined according to the upper and lower limits. In this way, by dynamically adjusting the shooting angle of the video camera, it can be ensured that the head images of the driver and the passenger are comprehensively and accurately captured.

[0032] Step S300: Determine the target pitching angle range of the pan-tilt according to the target shooting angle range of the video camera; In one embodiment, the step S300 of determining the target pitching angle range of the pan-tilt according to the target shooting angle range of the video camera includes: S310: Establish a conversion relationship according to the relative position relationship between the video camera and the pan-tilt, and the conversion relationship is used to represent the corresponding relationship between the perspective change of the video camera and the pitching angle of the pan-tilt; S320: Determine the target pitching angle range according to the conversion relationship and the target shooting angle range.

[0033] Specifically, first, a mathematical mapping model (such as a transformation matrix) between the camera view angle change and the pan-tilt angle of the pan-tilt head is established according to the installation position relationship between the video camera and the pan-tilt head. Then, the target pan-tilt angle range is determined based on the mathematical mapping model and the target shooting angle range. For example, a global coordinate system is established with the origin at the rotation center of the pan-tilt head. Then, a local coordinate system is established with the origin at the optical center of the camera and the direction consistent with the camera view direction. A rotation matrix is used to describe the rotation of the camera relative to the global coordinate system. The change in the camera view angle (usually the change in the field of view angle) is converted into the change in the pan-tilt angle of the pan-tilt head through geometric transformation (such as a transformation matrix). The calculation of the specific mapping transformation model is prior art and will not be described in detail here.

[0034] Exemplarily, when the pan-tilt angle of the pan-tilt head changes by 60°, the view angle of the video camera changes by 30°. At this time, the conversion relationship between the two is θ = 0.5β (θ is the change in the view angle of the video camera, and β is the change in the pan-tilt angle of the pan-tilt head).

[0035] Step S400: Control the pan-tilt head to move within the target pan-tilt angle range to obtain video data of the driver and passengers, thereby obtaining target video data; After obtaining the target pan-tilt angle range of the pan-tilt head, control the pan-tilt head to move dynamically within the target pan-tilt angle range to obtain video data of the driver and passengers, thereby obtaining target video data. During the movement of the pan-tilt head, continuously monitor the capture quality of the video data. If problems such as view angle deviation, jitter, or other issues are found, the control parameters of the pan-tilt head need to be adjusted in a timely manner. In this way, the video data of the driver and passengers can be captured comprehensively and accurately, providing a high-quality video data source for subsequent video analysis or monitoring applications.

[0036] It can be understood that the faster the electric bicycle travels, the greater the danger. Therefore, it is necessary to collect video data of the driver and passengers at a higher frequency or more efficiently. To improve the frequency or efficiency of video collection, in one embodiment, the step S400: Control the pan-tilt head to move within the target pan-tilt angle range to obtain video data of the driver and passengers, thereby obtaining target video data, includes: Step S410: Obtain the traveling speed of the electric bicycle and determine the pan-tilt movement speed of the pan-tilt head according to the traveling speed of the electric bicycle. Among them, the pan-tilt movement speed of the pan-tilt head is positively correlated with the traveling speed of the electric bicycle; Step S420: Control the pan-tilt head to move within the pan-tilt movement speed and the target pan-tilt angle range to obtain video data of the driver and passengers, thereby obtaining target video data.

[0037] Specifically, the current driving speed of the electric bicycle is obtained in real time and accurately by using the speed sensor built in the electric bicycle or by communicating with the vehicle control system. According to the driving speed of the electric bicycle obtained in real time, the pitching motion speed of the pan-tilt head is determined through a preset algorithm or mapping relationship. This algorithm or mapping relationship should ensure that the pitching motion speed of the pan-tilt head is positively correlated with the driving speed of the electric bicycle, that is, the faster the electric bicycle travels, the faster the pan-tilt head needs to pitch to capture key video data more quickly. After determining the pitching motion speed of the pan-tilt head, the motion trajectory of the pan-tilt head is dynamically adjusted in combination with the previously set target pitching angle range. The pan-tilt head will move at this speed and within this angle range, and at the same time, the video acquisition device is started to capture the video data of the driver and passengers in real time to obtain the target video data.

[0038] In another embodiment, step S400: controlling the pan-tilt head to move within the target pitching angle range to obtain the video data of the driver and passengers to obtain the target video data includes: Step S430: obtaining the driving speed of the electric bicycle and determining the pitching motion frequency of the pan-tilt head according to the driving speed of the electric bicycle, wherein the pitching motion frequency of the pan-tilt head is positively correlated with the driving speed of the electric bicycle; Step S440: controlling the pan-tilt head to move at the pitching motion frequency and within the target pitching angle range to obtain the video data of the driver and passengers to obtain the target video data.

[0039] Specifically, the current driving speed of the electric bicycle can also be obtained in real time and accurately by using the speed sensor built in the electric bicycle or by communicating with the vehicle control system. According to the driving speed of the electric bicycle obtained in real time, the pitching motion frequency of the pan-tilt head is determined through a preset algorithm or mapping relationship. This algorithm or mapping relationship should ensure that the pitching motion frequency of the pan-tilt head is positively correlated with the driving speed of the electric bicycle, that is, the faster the electric bicycle travels, the higher the frequency at which the pan-tilt head needs to pitch (for example, after the vehicle speed increases, the video data is obtained by shooting once every 3 minutes is converted to shooting once every 1 minute) to capture key video data more frequently. As the pitching motion frequency of the pan-tilt head increases, the update speed of the video data accelerates, enabling the monitoring system to more real-time reflect the states of the driver and passengers. This is crucial for timely detecting and handling potential safety hazards and helps improve the overall monitoring level of road traffic safety. After determining the pitching motion frequency of the pan-tilt head, the motion trajectory of the pan-tilt head is dynamically adjusted in combination with the previously set target pitching angle range. The pan-tilt head will move at this frequency and within this angle range, and at the same time, the video acquisition device is started to capture the video data of the driver and passengers in real time to obtain the target video data.

[0040] Step S500: Input the target video data into a pre-trained helmet-wearing recognition model to obtain a helmet-wearing recognition result; In one embodiment, Step S500: Input the target video data into a pre-trained helmet-wearing recognition model to obtain a helmet-wearing recognition result, including: Step S510: Obtain the network connection status of the wireless communication gateway, where the wireless communication gateway is used to send the target video data to the cloud server; Step S520: In the case where the network connection status of the wireless communication gateway is abnormal, input the target video data into the helmet-wearing recognition model in the edge server to perform helmet-wearing recognition to obtain a helmet-wearing recognition result; Step S530: In the case where the network connection status of the wireless communication gateway is normal, input the target video data into the helmet-wearing recognition model in the cloud server to perform helmet-wearing recognition to obtain a helmet-wearing recognition result, where the cloud server regularly updates the helmet-wearing recognition model in the edge server.

[0041] Specifically, before performing helmet-wearing recognition, first detect the network connection status of the wireless communication gateway. For example, check whether the connection between the gateway and the Internet or the cloud server is stable and whether the latency is within an acceptable range. If the network connection status of the wireless communication gateway is abnormal (such as unstable connection, disconnection, etc.), then input the target video data into the helmet-wearing recognition model in the edge server for recognition. The edge server is usually deployed locally or at the network edge and can process data in real time, reducing the dependence on the cloud server and network latency. If the network connection status of the wireless communication gateway is normal, then input the target video data into the helmet-wearing recognition model in the cloud server for recognition. The cloud server usually has more powerful computing capabilities and larger storage space, and can provide more accurate and efficient recognition services. In addition, the cloud server is also responsible for regularly updating the helmet-wearing recognition model in the edge server to ensure the accuracy and timeliness of the model.

[0042] In other embodiments, step S500: inputting the target video data into a pre-trained helmet-wearing recognition model to obtain a helmet-wearing recognition result, includes: extracting consecutive video frames from the target video data; performing head feature recognition on the extracted video frames to obtain head video frames; using a deep learning model to extract head contour features from the head video frames to obtain an initial inner contour and an initial outer contour; performing circumferential extension alignment on the initial inner contour and the initial outer contour to obtain a target inner contour and a target outer contour; determining that the average distance value between the target inner contour and the target outer contour is greater than or equal to a preset distance value, determining that the target user wears a helmet; determining that the average distance value between the target inner contour and the target outer contour is less than the preset distance value, determining that the target user does not wear a helmet.

[0043] Specifically, consecutive video frames can be first extracted from the target video data. These video frames are the basis for subsequent processing. Then, head feature recognition is performed on the extracted video frames to obtain video frames containing the head region (head video frames). Then, a deep learning model (such as a convolutional neural network) is used to extract head contour features from the head video frames. This includes an initial inner contour (usually representing features inside the head, such as facial contours) and an initial outer contour (usually representing features outside the head, such as the edge of the helmet). Then, circumferential extension alignment is performed on the initial inner contour and the initial outer contour (that is, circumferential filling is performed on the initial inner contour and the initial outer contour) to eliminate problems such as incomplete contours caused by factors such as perspective changes and head rotations. Then, the average distance value between the target inner contour and the target outer contour is calculated. This distance value reflects the spatial relationship between the head and the helmet. If the average distance value is greater than or equal to the preset distance value (this value is usually determined according to factors such as the size, shape, and wearing method of the helmet), it is determined that the target user wears a helmet. If the average distance value is less than the preset distance value, it is determined that the target user does not wear a helmet, which may be due to helmet removal or helmet slipping.

[0044] Step S600: performing electric bicycle driving safety control according to the helmet-wearing recognition result.

[0045] Finally, specifically performing electric bicycle driving safety control according to the helmet-wearing recognition result. For example, before starting the electric bicycle, if it is determined that the helmet-wearing recognition result is the state of not wearing a helmet (any one of the driver and passengers does not wear a helmet), the electric bicycle can be controlled to be forcibly in a stopped state, that is, the vehicle is prohibited from starting; during the driving of the electric bicycle, if it is determined that the helmet-wearing recognition result is the state of helmet removal or helmet slipping (any one of the driver and passengers removes or slips the helmet), the electric bicycle can be timely controlled to decelerate and trigger a warning signal, such as a voice reminder for the driver to pull over to the side of the road and put on the helmet.

[0046] Based on this, an electric bicycle helmet wearing recognition and driving safety control method provided by an embodiment of the present application is implemented through the collaborative work of a pan-tilt, a fisheye camera, and a video camera at the handlebar. First, the fisheye camera captures an image and analyzes the height difference between the driver and the passenger; when the height difference exceeds a threshold, the target shooting angle range of the video camera is dynamically calculated, and then the target shooting angle range of the video camera is converted into the pitch angle control parameter of the pan-tilt, and then the pan-tilt is driven to adjust its posture to accurately capture high-quality video data of the head area of the driver and the passenger; then, the helmet wearing state is recognized through a deep learning model, and finally, according to the recognition result, the vehicle control system is linked to execute safety strategies (such as restricting startup or alarming, etc.). In this way, the problem of head occlusion or vision deviation caused by height differences is solved through dynamic perspective adjustment, improving the reliability and accuracy of image acquisition, thereby improving the accuracy of helmet wearing recognition and further improving the accuracy of electric bicycle safety control.

[0047] As Figure 4 shown, Figure 4 FIG. is a schematic diagram of the hardware structure of an electric bicycle safety control system in some embodiments of the present application. The electric bicycle safety control system provided by the embodiment of the present application further includes a memory 1000 and a processor 2000. Among them, the memory 1000 is used to store computer-readable instructions, and the processor 2000 is used to call the computer-readable instructions to execute the electric bicycle helmet wearing recognition and driving safety control method as described above.

[0048] Among them, the processor 2000 is used to provide computing and control capabilities to control the electric bicycle safety control system to execute corresponding tasks. For example, the processor 2000 controls the electric bicycle safety control system to execute the electric bicycle helmet wearing recognition and driving safety control method in any of the above method embodiments. The method includes: obtaining a target image by capturing an image of the fisheye camera, and determining the height information of the driver and the passenger according to the target image. The height information at least includes the height difference between the driver and the passenger; in the case where the height difference between the driver and the passenger is greater than or equal to a height threshold, determining the target shooting angle range of the video camera according to the height information; determining the target pitch angle range of the pan-tilt according to the target shooting angle range of the video camera; controlling the pan-tilt to move within the target pitch angle range to obtain video data of the driver and the passenger to obtain target video data; inputting the target video data into a pre-trained helmet wearing recognition model to obtain a helmet wearing recognition result; and performing electric bicycle driving safety control according to the helmet wearing recognition result.

[0049] The processor 2000 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), a hardware chip, or any combination thereof; it may also be a Digital Signal Processing (DSP), an Application Specific Integrated Circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The above PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0050] The memory 1000, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the electric bicycle helmet wearing recognition and driving safety control method in the embodiments of the present application. By running the non-transitory software programs, instructions, and modules stored in the memory 1000, the processor 2000 can implement the electric bicycle helmet wearing recognition and driving safety control method in any of the above method embodiments.

[0051] Specifically, the memory 1000 may include volatile memory (VM), such as random access memory (RAM); the memory 1000 may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, a hard disk drive (HDD), or a solid-state drive (SSD), or other non-transitory solid-state storage devices; the memory 1000 may further include a combination of the above types of memories.

[0052] In summary, the electric bicycle safety control system of the present application adopts the technical solution of any of the above embodiments of the electric bicycle helmet wearing recognition and driving safety control method. Therefore, it has at least the beneficial effects brought by the technical solutions of the above embodiments, which will not be elaborated here one by one.

[0053] The embodiments of the present application also provide a computer-readable storage medium, such as a memory including program codes, and the above program codes can be executed by a processor to complete the electric bicycle helmet wearing recognition and driving safety control method in the above embodiments. For example, the computer-readable storage medium can be a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CDROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0054] The embodiments of the present application also provide a computer program product, which includes one or more program codes, and the program codes are stored in a computer-readable storage medium. The processor of the electric bicycle safety control system reads the program codes from the computer-readable storage medium, and the processor executes the program codes to complete the steps of the electric bicycle helmet wearing recognition and driving safety control method provided in the above embodiments.

[0055] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above embodiments can be completed by hardware, or can be completed by hardware related to program codes through a program. The program can be stored in a computer-readable storage medium, and the above-mentioned storage medium can be a read-only memory, a magnetic disk, or an optical disc, etc.

[0056] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0057] Through the description of the above embodiments, those of ordinary skill in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Those of ordinary skill in the art can understand that all or part of the processes of implementing the method in the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. Among them, the storage medium can be a magnetic disk, an optical disc, a Read-Only Memory (ROM), or a Random Access Memory (RAM), etc.

[0058] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structural transformation made under the inventive concept of the present invention by using the content of the specification and drawings of the present invention, or any direct / indirect application in other related technical fields is included in the patent protection scope of the present invention.

Claims

1. A motorcycle helmet wearing recognition and driving safety control method, applied to a motorcycle safety control system, characterized in that: The motorcycle safety control system comprises a pan-tilt and a fisheye camera arranged at the handlebar position, wherein a video camera is arranged on the pan-tilt. The method comprises: Acquire the image captured by the fisheye camera to obtain a target image, and determine the height information of the driver and the passenger according to the target image, wherein the height information at least includes the height difference between the driver and the passenger; When the height difference between the driver and the passenger is greater than or equal to a height threshold, determining a target shooting angle range of the video camera according to the height information; Determining a target pitch angle range of the pan / tilt head according to a target shooting angle range of the video camera; Controlling the pan / tilt head to move within the target pitch angle range to acquire the video data of the driver and passengers to obtain target video data; Inputting the target video data into a pre-trained helmet wearing recognition model to obtain a helmet wearing recognition result; Motorcycle driving safety control is performed according to the helmet wearing recognition result.

2. The motorcycle helmet wearing identification and driving safety control method according to claim 1, characterized in that: The step of determining the height information of the driver and the passenger according to the target image includes: Performing image semantic recognition on the target image to obtain the highest point position information of the driver and passengers in the target image; Determine blind spot height information according to the highest point position information of the driver and the passenger in the target image to obtain a first blind spot height and a second blind spot height, wherein the first blind spot height represents the blind spot height of the fisheye camera for photographing the driver, and the second blind spot height represents the blind spot height of the fisheye camera for photographing the passenger; The height information of the driver and the passengers is determined according to the initial calibration height of the fisheye camera, the first blind spot height and the second blind spot height.

3. The motorcycle helmet wearing identification and driving safety control method according to claim 1, characterized in that: Determining the target shooting angle range of the video camera according to the height information includes: The highest point of the height of the taller one of the driver and the passenger is used as the highest point of the shooting position, and the position of the motorcycle seat is used as the lowest point of the shooting position; The range between the highest point of the shooting position and the lowest point of the shooting position is determined as the target shooting angle range of the video camera.

4. The motorcycle helmet wearing identification and driving safety control method according to claim 1, characterized in that: The step of determining the target pitch angle range of the pan / tilt head according to the target shooting angle range of the video camera comprises: Establishing a conversion relationship according to the relative position relationship between the video camera and the pan-tilt head, wherein the conversion relationship is used to characterize the corresponding relationship between the viewing angle change of the video camera and the pitch angle of the pan-tilt head; The target pitch angle range is determined according to the conversion relationship and the target shooting angle range.

5. The motorcycle helmet wearing identification and driving safety control method according to claim 1, characterized in that: The step of inputting the target video data into a pre-trained helmet wearing recognition model to obtain a helmet wearing recognition result includes: Acquiring a network connection status of a wireless communication gateway, wherein the wireless communication gateway is used to send the target video data to a cloud server; In the case where the network connection state of the wireless communication gateway is abnormal, the target video data is input into a helmet wearing recognition model in an edge server to perform helmet wearing recognition to obtain a helmet wearing recognition result; When the network connection status of the wireless communication gateway is normal, the target video data is input into the helmet wearing recognition model in the cloud server to perform helmet wearing recognition to obtain a helmet wearing recognition result, wherein the cloud server periodically updates the helmet wearing recognition model of the edge server.

6. The motorcycle helmet wearing identification and driving safety control method according to claim 1, characterized in that: The step of inputting the target video data into a pre-trained helmet wearing recognition model to obtain a helmet wearing recognition result includes: Extracting continuous video frames from the target video data; Performing head feature recognition on the extracted video frame to obtain a head video frame; Using a deep learning model to extract head contour features from the head video frame to obtain an initial inner contour and an initial outer contour; Circumferentially extending and aligning the initial inner contour and the initial outer contour to obtain a target inner contour and a target outer contour; Determining that the average distance value between the target inner contour and the target outer contour is greater than or equal to a preset distance value, and determining that the target user is wearing a helmet; It is determined that the average distance value between the target inner contour and the target outer contour is less than a preset distance value, and it is determined that the target user is not wearing a helmet.

7. The motorcycle helmet wearing identification and driving safety control method according to claim 1, characterized in that: The step of controlling the pan / tilt head to move within the target pitch angle range to acquire the video data of the driver and passengers to obtain target video data includes: Acquire the running speed of the motorcycle, and determine the pitching speed of the gimbal according to the running speed of the motorcycle, wherein the pitching speed of the gimbal is positively correlated with the running speed of the motorcycle; The pan / tilt head is controlled to move at the pitch motion speed and within the target pitch angle range to acquire the video data of the driver and passengers to obtain target video data.

8. The motorcycle helmet wearing identification and driving safety control method according to claim 1, characterized in that: The step of controlling the pan / tilt head to move within the target pitch angle range to acquire the video data of the driver and passengers to obtain target video data includes: Acquire the running speed of the motorcycle, and determine the pitching frequency of the gimbal according to the running speed of the motorcycle, wherein the pitching frequency of the gimbal is positively correlated with the running speed of the motorcycle; The pan / tilt platform is controlled to move at the pitch motion frequency and within the target pitch angle range to acquire the video data of the driver and passengers to obtain target video data.

9. The motorcycle helmet wearing identification and driving safety control method according to claim 1, characterized in that: The method of performing motorcycle driving safety control according to the helmet wearing recognition result includes: Before the motorcycle is started, if the helmet wearing recognition result is determined to be a state where the helmet is not worn, the motorcycle is forced to be parked; During the driving of the motorcycle, if it is determined that the helmet wearing recognition result is a helmet removal or helmet slipping state, the motorcycle is controlled to slow down and a warning signal is triggered.

10. A motorcycle safety control system, characterized in that: include: A pan head and a fisheye camera are arranged at the handlebars of the motorcycle, wherein the pan head is provided with a video camera; as well as, A memory and a processor, wherein the memory is used to store program codes; The processor is used to call the program code to execute the method according to any one of claims 1 to 9.

Images