A method and system for dynamic detection of driver gestures based on kinematic features

Abstract

This invention provides a method and system for dynamic detection of driver gestures based on kinematic features, comprising: extracting a preset number of frame image sequences containing a complete set of gestures from a video stream of the driver's operation process; performing hand detection on each frame and determining the coordinates of the hand's center point; calculating gesture motion parameters and determining the turning point of the hand movement based on the hand center point coordinates of two adjacent frames; determining whether a valid hand movement exists based on the gesture motion parameters and intermediate frames of the image sequence; if a valid hand movement is determined to exist, further determining the direction of hand movement based on the turning point and intermediate frames, and recording the movement direction sequence using a sliding window method; comparing the movement direction sequence with the direction sequence of the gesture rules to determine the compliance of the gesture; and issuing an early warning when the determination result is a violation. This invention can dynamically detect driver gestures.

Description

Technical Field

[0001] This invention relates to the field of rail transit operation safety monitoring technology, and in particular to a method and system for dynamic detection of driver gestures based on kinematic characteristics. Background Technology

[0002] In the railway transportation sector, the safety of train operation has always been a core prerequisite for the industry's development, directly affecting passenger safety and property security, public interests, and the stable operation of the railway transportation system. With the continuous expansion of the railway network, the increase in train speeds, and the sustained growth in passenger traffic, increasingly stringent requirements are being placed on the standardization and precision of train driving operations.

[0003] As the direct executor of train driving operations, the compliance of train drivers' driving behavior is a crucial link in ensuring the safe operation of trains. Through long-term railway transportation practice, the industry has developed a series of standardized operating procedures for train drivers. Among these, hand gestures, as an important way for drivers to focus on confirming the train's status and operational procedures during driving, have been incorporated into the standardized operating system. These gestures entirely serve the driver's self-verification of driving-related statuses. For example, in critical scenarios such as train arrival at and departure from stations, and train start-up, drivers must use specific gestures to point to corresponding positions on the dashboard to confirm the relevant status. These gestures are clearly defined operational indications within the industry, and their core function is to ensure the driver's accurate grasp of the operational status. Their standardization directly affects the accuracy of operational confirmation, thus impacting the safety and efficiency of train operations.

[0004] Existing technologies mainly utilize image data processing and specific network model training to achieve gesture recognition. Their working principle is based on image processing technology to perform various transformations on images to expand the dataset, and then use the powerful object detection capabilities of the YOLOv5 network to recognize gestures.

[0005] However, this existing technology has some drawbacks and limitations: image-based recognition is highly dependent on ambient lighting conditions. Image quality is easily affected by excessively strong or dim lighting, or when the vehicle is in a complex lighting environment, reducing the accuracy of gesture recognition by the YOLOv5 network. Furthermore, this method focuses on processing static images and struggles to capture continuous, dynamic gesture changes in real time, failing to meet the need for immediate and accurate gesture recognition during driving. Summary of the Invention

[0006] To address the technical problems existing in the prior art, the present invention provides a method and system for dynamic detection of driver gestures based on kinematic features, the technical solution of which is as follows:

[0007] On the one hand, a method for dynamic detection of driver gestures based on kinematic features is provided, the method comprising:

[0008] S1. Capture the video stream of the driver's operation process through the camera, and extract a continuous preset number of frame image sequences from the video stream, including a complete set of gestures;

[0009] S2. Perform hand detection on each captured frame and determine the coordinates of the hand's center point;

[0010] S3. Based on the coordinates of the hand center point in two adjacent frames, calculate the gesture motion parameters and determine the turning point of the hand motion. The gesture parameters include the speed and acceleration of the hand motion, and the turning point represents the position of the frame where the hand motion state undergoes a key change.

[0011] S4. Determine whether a valid hand movement exists based on the gesture motion parameters and the intermediate frames of the preset number of frame image sequences. If a valid hand movement exists, continue to determine the hand movement direction based on the turning point and the intermediate frames, and record the movement direction sequence using a sliding window method.

[0012] S5. Compare the motion direction sequence with the direction sequence of the gesture rules to determine the compliance of the gesture.

[0013] S6. Issue a warning when the judgment result is a violation.

[0014] Optionally, S2 specifically includes:

[0015] Hand detection is performed on each frame of the image. The bounding box of the hand region is obtained using an object detection algorithm. The bounding box is represented by pixel coordinates, including the x-coordinate of the top-left vertex. y-coordinate of the top left vertex The bounding box has a width w and a height h.

[0016] The coordinates of the hand's center point are calculated based on the bounding box, where the x-axis coordinate is... The y-axis coordinate is This allows us to obtain the coordinates of the hand's center point in each frame. Finally, the set of coordinates of the hand's center point is obtained: , , ..., Where n is the preset number of frames, and These represent the coordinates of the center point of the hand in the i-th frame.

[0017] Optionally, S3 specifically includes:

[0018] S31. Calculate the motion speed between adjacent frames based on the coordinates of the hand center point in two adjacent frames;

[0019] Assuming the time interval between two adjacent frames is The speed from frame i to frame i+1 is determined by the capture frame rate. The calculation formula is:

[0020] i = 1, 2, ..., n-1

[0021] Select the maximum value from all calculated velocity values. ;

[0022] S32. Calculate the acceleration of the velocity change based on the difference in velocity between two adjacent frames;

[0023] Acceleration from frame i to frame i+1 The calculation formula is: , i = 1, 2, ..., n-1;

[0024] S33. From all acceleration values, find the point where the acceleration is 0, and use it as the turning point of the hand movement. This represents the position of the frame corresponding to the key change in the hand movement state, denoted as... k is the frame index corresponding to this point: when the acceleration is positive, the hand is accelerating; when the acceleration is negative, the hand is decelerating; and the instant when the acceleration is 0 is the critical point for the conversion between these two trends - if it accelerated before, it will start to decelerate afterward, and if it decelerated before, it will start to accelerate afterward. Therefore, this point is the dividing point for the hand movement to switch from the acceleration phase to the deceleration phase or vice versa.

[0025] Optionally, in step S4, determining whether a valid hand movement exists based on the gesture motion parameters and the intermediate frames of the preset number of frame image sequences specifically includes:

[0026] Set a speed threshold Compare the maximum speed and :like If the action is deemed invalid, the process terminates; Then proceed to the next step of judgment;

[0027] Select the middle frame of the image sequence, denoted as the m-th frame, and calculate the straight-line distance between the middle frame and the first and last frames of the sequence. and Set a distance threshold ,like and If the motion trajectory is deemed unclear and there is no effective action, the process terminates; or If so, it is determined that a valid action exists.

[0028] Optionally, in step S4, the direction of hand movement is further determined based on the turning point and the intermediate frame, specifically including:

[0029] S41. Detect the head region in the image and extract the x-axis coordinate of the head center point, denoted as... ;

[0030] S42. Calculate the x-axis difference between the key points and the center point of the head;

[0031] Calculate the turning point The difference between the x-axis coordinate and the x-axis coordinate of the head center point: ;

[0032] Since the turning point only reflects the positional state at a certain instant and cannot reflect the continuity and trend of the movement, the intermediate frame is added for auxiliary judgment to verify whether the direction of the turning point is continuous, and the difference between the x-axis coordinate of the intermediate frame and the x-axis coordinate of the head center point is calculated: ;

[0033] S43. Determine the direction of hand movement based on the difference;

[0034] Preset rightward threshold and left threshold The left and right critical points for direction determination, if and This indicates that the x-coordinates of the turning point and the intermediate frame are both larger than the x-coordinate of the head center point, and exceed the rightward threshold. If the current direction of movement is determined to be to the right; and This indicates that the absolute value of the difference between the x-coordinate of the turning point and the middle frame and the x-coordinate of the head center point is within the threshold range, thus indicating movement towards the center; if and This indicates that the x-coordinates of the turning point and the intermediate frame are both smaller than the x-coordinate of the head center point, and also smaller than the leftward threshold. If so, it is determined to be moving to the left.

[0035] Optionally, step S4 uses a sliding window method to record the motion direction sequence, specifically including:

[0036] Set the window to a frame length that is less than the number of frames of a complete set of gestures;

[0037] When sliding, the window starts from the beginning frame of the hand center point coordinate sequence and moves in a continuous frame sequence in a frame-by-frame manner. Each time it slides, the window moves forward one frame, forming a series of continuous and overlapping window intervals.

[0038] Each window interval independently determines the gesture direction and obtains the corresponding direction result. These results are then filtered: the direction result of the current window interval is compared with the direction result of the previous window interval. If they are different, it means the end of a direction has been reached, and the direction result of the current window interval is included in the final gesture direction rule sequence. If they are the same, it means the movement is continuing in one direction, and it is not included. This process generates a complete set of gesture-related direction sequences. ,in The direction is left, right, or center.

[0039] Optionally, S5 specifically includes:

[0040] S51, Customized rule construction;

[0041] Based on rail transit industry standards and the specific regulations for driver hand gestures on different lines and vehicle types, the directional sequence of hand gesture rules is defined. ;

[0042] S52, Compliance Verification;

[0043] Read the The sequence of motion directions With the The sequences are compared one by one. If the two sequences are the same, they are considered normal; otherwise, they are considered to be in violation.

[0044] On the other hand, a driver gesture dynamic detection system based on kinematic features is provided, the system comprising:

[0045] The acquisition and capture module is used to acquire video streams during the driver's operation through a camera, and to capture a continuous sequence of a preset number of frames of images from the video stream, including a complete set of gestures.

[0046] The detection and determination module is used to detect hands in each captured frame of image and determine the coordinates of the hand's center point;

[0047] The calculation and determination module is used to calculate gesture motion parameters and determine the turning point of hand movement based on the coordinates of the hand center point in two adjacent frames. The gesture parameters include the velocity and acceleration of hand movement, and the turning point represents the position corresponding to the frame where the hand movement state undergoes a key change.

[0048] The judgment and recording module is used to determine whether a valid hand movement exists based on the gesture motion parameters and the intermediate frames of the preset number of frame image sequences. When a valid hand movement is determined to exist, the module continues to determine the hand movement direction based on the turning point and the intermediate frames, and records the movement direction sequence using a sliding window method.

[0049] The judgment module is used to compare the motion direction sequence with the direction sequence of the gesture rules to determine the compliance of the gesture.

[0050] The early warning module is used to issue an early warning when the judgment result is a violation.

[0051] On the other hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the above-described method for dynamic detection of driver gestures based on kinematic features.

[0052] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, the at least one instruction being loaded and executed by a processor to implement the above-described method for dynamic detection of driver gestures based on kinematic features.

[0053] The beneficial effects of the technical solution provided by this invention include at least the following:

[0054] The driver gesture dynamic detection method proposed in this invention, based on kinematic features, achieves multi-dimensional technological improvements in the field of rail transit operation safety monitoring through a series of innovative technical solutions, bringing significant beneficial effects. It employs video data acquisition combined with an advanced target detection model to extract hand coordinates. Compared to traditional techniques relying on single image feature extraction algorithms, this method significantly reduces sensitivity to complex environmental factors (coordinate extraction is less affected by lighting), effectively solving the problem of environmental factors affecting the stability of detection results in existing technologies. This ensures the stable and reliable operation of the detection system in various complex operating environments. Through professional analysis and processing of continuous multi-frame hand center point position sequences, it overcomes the limitation of existing technologies that can only perform static judgments, accurately identifying the gesture movement state and direction. Compared to existing technologies, it significantly improves the accuracy of judging whether driver gestures conform to standardized operating procedures, effectively avoiding safety hazards caused by misjudgment of gesture direction. This invention closely integrates… By combining the customized requirements of the rail transit industry for driver gestures, this invention determines whether the driver's current gestures are compliant based on the detection results. Through flexible customization settings, it can be adapted to various application scenarios, overcoming the predicament that existing general gesture recognition solutions cannot meet the special needs of rail transit. This greatly improves the flexibility and versatility of the detection system in the rail transit field, fully meeting the diverse operational standard detection needs of the industry. In addition, this invention achieves real-time and accurate monitoring of driver's work behavior, enabling timely detection of non-standard behaviors in driver gesture operations and rapid response. Compared with inefficient methods such as traditional manual inspections and video spot checks, it can provide early warnings at the first moment of a problem, facilitating timely intervention by management personnel and effectively reducing the probability of safety accidents. At the same time, it provides reliable data support and technical means for intelligent safety management of rail transit, promoting the development of industry safety management from passive handling to proactive prevention and intelligent management, and improving the overall operation and management level of rail transit. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a flowchart of a driver gesture dynamic detection method based on kinematic features provided in an embodiment of the present invention;

[0057] Figure 2 This is a schematic diagram of driver gestures provided in an embodiment of the present invention;

[0058] Figure 3This is a block diagram of a driver gesture dynamic detection system based on kinematic features provided in an embodiment of the present invention;

[0059] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0060] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0061] The purpose of this invention is to provide a highly efficient, accurate, and adaptable method for dynamic detection of driver gestures. This method collects video data from train drivers during their operations, utilizes an advanced target detection model to accurately extract the driver's hand coordinates, and obtains a set of hand coordinates. Further, it performs in-depth analysis on the sequence of hand center point positions across multiple consecutive frames, calculating velocity and acceleration parameters to accurately identify the motion state and turning points of the gesture, thereby determining the direction (left, center, right, etc.). Simultaneously, considering the customized requirements for driver gestures in the rail transit industry, it determines whether the driver's current gesture is compliant based on the detection results, achieving real-time and accurate monitoring of driver behavior. The core of this invention is the analysis of the extracted hand coordinate sequence, focusing on the kinematic characteristics of the hand's trajectory. Even in poor lighting conditions, as long as the target detection model can stably extract the hand center point coordinates (coordinate extraction is less affected by lighting), subsequent kinematic analysis based on the coordinate sequence can still effectively determine the gesture direction. This invention effectively overcomes the shortcomings of existing technologies in terms of environmental adaptability, dynamic analysis capabilities, and meeting customized needs, providing strong support for improving the safety and reliability of rail transit operations and playing a significant role in promoting the intelligent development of rail transit.

[0062] This invention provides a method for dynamic detection of driver gestures based on kinematic features. This method can be implemented by an electronic device, which can be a terminal or a server. Figure 1 The diagram shown is a flowchart of the method. The processing flow may include the following steps:

[0063] S1. Capture the video stream of the driver's operation process through the camera, and extract a continuous preset number of frame image sequences from the video stream, including a complete set of gestures;

[0064] Based on experience and statistical data, it is known that the time it takes for a driver to complete a set of gestures will not vary too much. For example, with a frame rate of 20, it takes between 70 and 90 frames for a driver to complete a set of gestures in three directions. Therefore, in this embodiment of the invention, the preset number of frames can be set to 100 frames.

[0065] S2. Perform hand detection on each captured frame and determine the coordinates of the hand's center point;

[0066] Optionally, S2 specifically includes:

[0067] Hand detection is performed on each frame of the image. The bounding box of the hand region is obtained using an object detection algorithm (such as YOLO). The bounding box is represented by pixel coordinates, including the x-coordinate of the top-left vertex. y-coordinate of the top left vertex The bounding box has a width w and a height h.

[0068] The coordinates of the hand's center point are calculated based on the bounding box, where the x-axis coordinate is... The y-axis coordinate is This allows us to obtain the coordinates of the hand's center point in each frame. Finally, the set of coordinates of the hand's center point is obtained: , , ..., Where n is the preset number of frames, and These represent the coordinates of the center point of the hand in the i-th frame.

[0069] S3. Based on the coordinates of the hand center point in two adjacent frames, calculate the gesture motion parameters and determine the turning point of the hand motion. The gesture parameters include the speed and acceleration of the hand motion, and the turning point represents the position of the frame where the hand motion state undergoes a key change.

[0070] Optionally, S3 specifically includes:

[0071] S31. Calculate the motion speed between adjacent frames based on the coordinates of the hand center point in two adjacent frames;

[0072] Assuming the time interval between two adjacent frames is (Unit: seconds), determined by the capture frame rate (e.g., if the frame rate is 30 frames / second). The speed from frame i to frame i+1 (Unit: pixels / second) The calculation formula is:

[0073] i = 1, 2, ..., n-1

[0074] Select the maximum value from all calculated velocity values. (Represents the fastest speed of hand movement in this image sequence);

[0075] S32. Calculate the acceleration of the velocity change based on the difference in velocity between two adjacent frames;

[0076] Acceleration from frame i to frame i+1 (Unit: pixels / second²) The calculation formula is: , i = 1, 2, ..., n-1;

[0077] S33. From all acceleration values, find the point where the acceleration is 0 (or close to 0, allowing for small errors). This point serves as the turning point of the hand movement, representing the position of the frame corresponding to the key change in the hand movement state. This is denoted as... k is the frame index corresponding to this point: when the acceleration is positive, the hand is accelerating; when the acceleration is negative, the hand is decelerating; and the instant when the acceleration is 0 is the critical point for the conversion between these two trends - if it accelerated before, it will start to decelerate afterward, and if it decelerated before, it will start to accelerate afterward. Therefore, this point is the dividing point for the hand movement to switch from the acceleration phase to the deceleration phase or vice versa.

[0078] S4. Determine whether a valid hand movement exists based on the gesture motion parameters and the intermediate frames of the preset number of frame image sequences. If a valid hand movement exists, continue to determine the hand movement direction based on the turning point and the intermediate frames, and record the movement direction sequence using a sliding window method.

[0079] Optionally, in step S4, determining whether a valid hand movement exists based on the gesture motion parameters and the intermediate frames of the preset number of frame image sequences specifically includes:

[0080] Set a speed threshold Compare the maximum speed and :like If the action is deemed invalid, the process terminates; Then proceed to the next step of judgment;

[0081] Select the middle frame of the image sequence, denoted as the m-th frame, and calculate the straight-line distance between the middle frame and the first and last frames of the sequence. and Set a distance threshold ,like and If the motion trajectory is deemed unclear and there is no effective action, the process terminates; or If so, it is determined that a valid action exists.

[0082] like Figure 2 As shown, the driver needs to use a set of hand gestures to point to different areas of the dashboard in front of him. For example, he needs to point to the right area first, then to the left area, and then to the middle area. However, when pointing in each direction, the driver needs to start from an upright position in front of him (as shown in the upper part of the figure), point in a certain direction and then return to the upright position in front of him before pointing in another direction. In this embodiment of the invention, the direction of hand movement can be determined based on the turning point and the intermediate frame.

[0083] Optionally, in step S4, the direction of hand movement is further determined based on the turning point and the intermediate frame, specifically including:

[0084] S41. Detect the head region in the image (using the same method as for the hand region), and extract the x-axis coordinate of the head center point, denoted as... ;

[0085] S42. Calculate the x-axis difference between the key points and the center point of the head;

[0086] Calculate the turning point The difference between the x-axis coordinate and the x-axis coordinate of the head center point: ;

[0087] Since the turning point only reflects the positional state at a certain instant and cannot reflect the continuity and trend of the movement, the intermediate frame is added for auxiliary judgment to verify whether the direction of the turning point is continuous, and the difference between the x-axis coordinate of the intermediate frame and the x-axis coordinate of the head center point is calculated: ;

[0088] S43. Determine the direction of hand movement based on the difference;

[0089] Preset rightward threshold (Positive numbers) and left threshold (Negative numbers) represent the left and right critical points for direction determination. and This indicates that the x-coordinates of the turning point and the intermediate frame are both larger than the x-coordinate of the head center point, and exceed the rightward threshold. If the current direction of movement is determined to be to the right; and This indicates that the absolute value of the difference between the x-coordinate of the turning point and the middle frame and the x-coordinate of the head center point is within the threshold range, thus indicating movement towards the center; if and This indicates that the x-coordinates of the turning point and the intermediate frame are both smaller than the x-coordinate of the head center point, and also smaller than the leftward threshold. If so, it is determined to be moving to the left.

[0090] Optionally, step S4 uses a sliding window method to record the motion direction sequence, specifically including:

[0091] Set the window to a frame length that is less than the number of frames of a complete set of gestures;

[0092] When sliding, the window starts from the beginning frame of the hand center point coordinate sequence and moves in a frame-by-frame manner in a continuous frame sequence. Each time it slides, the window moves forward one frame, forming a series of continuous and overlapping window intervals (for example, the initial position of the window covers frames 1 to n, after sliding once it covers frames 2 to n+1, and after sliding again it covers frames 3 to n+2, and so on, until the window slides to the end of the sequence).

[0093] Each window interval independently determines the gesture direction and obtains the corresponding direction result. These results are then filtered: the direction result of the current window interval is compared with the direction result of the previous window interval. If they are different, it means the end of a direction has been reached, and the direction result of the current window interval is included in the final gesture direction rule sequence. If they are the same, it means the movement is continuing in one direction, and it is not included. This process generates a complete set of gesture-related direction sequences. ,in The direction is left, right, or center.

[0094] S5. Compare the motion direction sequence with the direction sequence of the gesture rules to determine the compliance of the gesture.

[0095] Optionally, S5 specifically includes:

[0096] S51, Customized rule construction;

[0097] Based on rail transit industry standards and the specific regulations for driver hand gestures on different lines and vehicle types, the directional sequence of hand gesture rules is defined. (like );

[0098] S52, Compliance Verification;

[0099] Read the The sequence of motion directions With the A step-by-step comparison is performed. If two sequences are identical, the sequence is considered normal; otherwise, it is considered irregular (e.g., ...). , (If so, it will be considered a violation).

[0100] S6. Issue a warning when the judgment result is a violation.

[0101] When a violation is determined, an immediate warning is issued. The warning message is displayed on the vehicle's screen with a striking color and icon to remind the driver that the current operation is in violation. At the same time, a buzzer is triggered to emit a sharp sound, attracting the driver's attention from both visual and auditory perspectives, ensuring that the driver can promptly detect and correct the improper operation.

[0102] While issuing warnings, this embodiment of the invention can also upload violation records (including specific information about the violation gestures, the time of occurrence, etc.) to the dispatch center in real time. The dispatch center can store, analyze and manage this data to evaluate and train drivers' work behavior in the future, and also provide strong data support for the safety management of rail transit, realizing comprehensive monitoring and management of train operation safety.

[0103] This invention discloses a dynamic driver gesture detection method based on kinematic features. It extracts hand coordinates using video data and a target detection model, reducing sensitivity to complex environments and accurately identifying hands even under varying lighting conditions and clothing differences, ensuring stable system operation. By analyzing the velocity and acceleration of the hand center point position sequence across a preset number of frames, it determines the motion state and direction, accurately reconstructing the gesture trajectory and improving the accuracy of gesture compliance judgment. Furthermore, by adjusting the judgment rules to adapt to gesture operations on different lines and vehicle types, it flexibly customizes detection standards to meet diverse operational requirements, tailoring the method to the customized requirements of rail transit. Real-time and accurate monitoring of driver behavior and rapid response to non-standard gestures provide timely warnings compared to traditional methods, supporting intelligent safety management and reducing the probability of accidents.

[0104] like Figure 3 As shown, this embodiment of the invention also provides a driver gesture dynamic detection system based on kinematic features, the system comprising:

[0105] The acquisition and capture module 310 is used to acquire video streams during the driver's operation through a camera, and capture a continuous preset number of frame image sequences, including a complete set of gestures, from the video stream.

[0106] The detection and determination module 320 is used to detect the hand in each captured frame of the image and determine the coordinates of the center point of the hand.

[0107] The calculation and determination module 330 is used to calculate gesture motion parameters and determine the turning point of hand movement based on the coordinates of the hand center point in two adjacent frames. The gesture parameters include the speed and acceleration of hand movement, and the turning point represents the position corresponding to the frame where the hand movement state undergoes a key change.

[0108] The judgment and recording module 340 is used to determine whether a valid hand movement exists based on the gesture motion parameters and the intermediate frames of the preset number of frame image sequences. When a valid hand movement is determined to exist, the module continues to determine the hand movement direction based on the turning point and the intermediate frames, and records the movement direction sequence using a sliding window method.

[0109] The determination module 350 is used to compare the motion direction sequence with the direction sequence of the gesture rules to determine the compliance of the gesture.

[0110] The 360 ​​early warning module is used to issue an early warning when the judgment result is a violation.

[0111] The driver gesture dynamic detection system based on kinematic features provided in this embodiment of the invention has a functional structure that corresponds to the driver gesture dynamic detection method based on kinematic features provided in this embodiment of the invention, and will not be described again here.

[0112] Figure 4 This is a schematic diagram of the structure of an electronic device 400 provided in an embodiment of the present invention. The electronic device 400 may vary considerably due to different configurations or performance. It may include one or more central processing units (CPUs) 401 and one or more memories 402. The memory 302 stores at least one instruction, which is loaded and executed by the processor 401 to implement the steps of the above-described dynamic detection method for driver gestures based on kinematic features.

[0113] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions that can be executed by a processor in a terminal to perform the aforementioned driver gesture dynamic detection method based on kinematic features. For example, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0114] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0115] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for dynamic detection of driver gestures based on kinematic features, characterized in that, The method includes: S1. Capture the video stream of the driver's operation process through the camera, and extract a continuous preset number of frame image sequences from the video stream, including a complete set of gestures; S2. Perform hand detection on each captured frame and determine the coordinates of the hand's center point; S3. Based on the coordinates of the hand center point in two adjacent frames, calculate the gesture motion parameters and determine the turning point of the hand movement. The gesture motion parameters include the speed and acceleration of the hand movement. The turning point represents the position of the frame where the hand movement state undergoes a key change. S4. Determine whether a valid hand movement exists based on the gesture motion parameters and the intermediate frames of the preset number of frame image sequences. If a valid hand movement exists, continue to determine the hand movement direction based on the turning point and the intermediate frames, and record the movement direction sequence using a sliding window method. S5. Compare the motion direction sequence with the direction sequence of the gesture rules to determine the compliance of the gesture. S6. Issue a warning when the judgment result is a violation; S3 specifically includes: S31. Calculate the motion speed between adjacent frames based on the coordinates of the hand center point in two adjacent frames; Assuming the time interval between two adjacent frames is The speed from frame i to frame i+1 is determined by the capture frame rate. The calculation formula is: ,i=1,2,…,n-1; Select the maximum value from all calculated velocity values. ; S32. Calculate the acceleration of the velocity change based on the difference in velocity between two adjacent frames; Acceleration from frame i to frame i+1 The calculation formula is: , i = 1, 2, ..., n-1; S33. From all acceleration values, find the point where the acceleration is 0, and use it as the turning point of the hand movement. This represents the position of the frame corresponding to the key change in the hand movement state, denoted as... k is the frame index corresponding to this point: when the acceleration is positive, the hand is accelerating; when the acceleration is negative, the hand is decelerating; and the instant when the acceleration is 0 is the critical point for the conversion between these two trends - if it accelerated before, it will start to decelerate afterward, and if it decelerated before, it will start to accelerate afterward. Therefore, this point is the dividing point for the hand movement to switch from the acceleration phase to the deceleration phase or vice versa. In step S4, the direction of hand movement is further determined based on the turning point and the intermediate frame, specifically including: S41. Detect the head region in the image and extract the x-axis coordinate of the head center point, denoted as... ; S42. Calculate the x-axis difference between the key points and the center point of the head; Calculate the turning point The difference between the x-axis coordinate and the x-axis coordinate of the head center point: ; Since turning points only reflect the positional state at a certain instant and cannot reflect the continuity and trend of movement, an intermediate frame of the image sequence is added, denoted as the m-th frame, for auxiliary judgment to verify whether the direction of the turning point is continuous. The difference between the x-axis coordinate of the intermediate frame and the x-axis coordinate of the head center point is calculated. ; S43. Determine the direction of hand movement based on the difference; Preset rightward threshold and left threshold The left and right critical points for direction determination, if and This indicates that the x-coordinates of the turning point and the intermediate frame are both larger than the x-coordinate of the head center point, and exceed the rightward threshold. If the current direction of movement is determined to be to the right; and This indicates that the absolute value of the difference between the x-coordinate of the turning point and the middle frame and the x-coordinate of the head center point is within the threshold range, thus indicating movement towards the center; if and This indicates that the x-coordinates of the turning point and the intermediate frame are both smaller than the x-coordinate of the head center point, and also smaller than the leftward threshold. If so, it is determined to be moving to the left.

2. The method according to claim 1, characterized in that, S2 specifically includes: Hand detection is performed on each frame of the image. The bounding box of the hand region is obtained using an object detection algorithm. The bounding box is represented by pixel coordinates, including the x-coordinate of the top-left vertex. y-coordinate of the top left vertex The bounding box has a width w and a height h. The coordinates of the hand's center point are calculated based on the bounding box, where the x-axis coordinate is... The y-axis coordinate is This allows us to obtain the coordinates of the hand's center point in each frame. Finally, the set of coordinates of the hand's center point is obtained: , , ..., Where n is the preset number of frames, and These represent the coordinates of the center point of the hand in the i-th frame.

3. The method according to claim 2, characterized in that, In step S4, determining whether a valid hand movement exists based on the gesture motion parameters and the intermediate frames of the preset number of frame image sequences specifically includes: Set a speed threshold Compare the maximum speed and :like If the action is deemed invalid, the process terminates; Then proceed to the next step of judgment; Calculate the straight-line distance between the intermediate frame and the first and last frames of the sequence. and Set a distance threshold ,like and If the motion trajectory is deemed unclear and there is no effective action, the process terminates; or If so, it is determined that a valid action exists.

4. The method according to claim 1, characterized in that, The motion direction sequence is recorded using a sliding window method in S4, specifically including: Set the window to a frame length that is less than the number of frames of a complete set of gestures; When sliding, the window starts from the beginning frame of the hand center point coordinate sequence and moves in a continuous frame sequence in a frame-by-frame manner. Each time it slides, the window moves forward one frame, forming a series of continuous and overlapping window intervals. Each window interval independently determines the gesture direction and obtains the corresponding direction result. These results are then filtered: the direction result of the current window interval is compared with the direction result of the previous window interval. If they are different, it means the end of a direction has been reached, and the direction result of the current window interval is included in the final gesture direction rule sequence. If they are the same, it means the movement is continuing in one direction, and it is not included. This process generates a complete set of gesture-related direction sequences. ,in The direction is left, right, or center.

5. The method according to claim 4, characterized in that, S5 specifically includes: S51, Customized rule construction; Based on rail transit industry standards and the specific regulations for driver hand gestures on different lines and vehicle types, the directional sequence of hand gesture rules is defined. ; S52, Compliance Verification; Read the The sequence of motion directions With the The sequences are compared one by one. If the two sequences are the same, they are considered normal; otherwise, they are considered to be in violation.

6. A driver gesture dynamic detection system based on kinematic features, characterized in that, The system includes: The acquisition and capture module is used to acquire video streams during the driver's operation through a camera, and to capture a continuous sequence of a preset number of frames of images from the video stream, including a complete set of gestures. The detection and determination module is used to detect hands in each captured frame of image and determine the coordinates of the hand's center point; The calculation and determination module is used to calculate the gesture motion parameters and determine the turning point of the hand movement based on the coordinates of the hand center point in two adjacent frames. The gesture motion parameters include the velocity and acceleration of the hand movement, and the turning point represents the position corresponding to the frame where the hand movement state undergoes a key change. The judgment and recording module is used to determine whether a valid hand movement exists based on the gesture motion parameters and the intermediate frames of the preset number of frame image sequences. When a valid hand movement is determined to exist, the module continues to determine the hand movement direction based on the turning point and the intermediate frames, and records the movement direction sequence using a sliding window method. The judgment module is used to compare the motion direction sequence with the direction sequence of the gesture rules to determine the compliance of the gesture. The early warning module is used to issue an early warning when the judgment result is a violation; The calculation and determination module is specifically used for: S31. Calculate the motion speed between adjacent frames based on the coordinates of the hand center point in two adjacent frames; Assuming the time interval between two adjacent frames is The speed from frame i to frame i+1 is determined by the capture frame rate. The calculation formula is: ,i=1,2,…,n-1; Select the maximum value from all calculated velocity values. ; S32. Calculate the acceleration of the velocity change based on the difference in velocity between two adjacent frames; Acceleration from frame i to frame i+1 The calculation formula is: , i = 1, 2, ..., n-1; S33. From all acceleration values, find the point where the acceleration is 0, and use it as the turning point of the hand movement. This represents the position of the frame corresponding to the key change in the hand movement state, denoted as... k is the frame index corresponding to this point: when the acceleration is positive, the hand is accelerating; when the acceleration is negative, the hand is decelerating; and the instant when the acceleration is 0 is the critical point for the conversion between these two trends - if it accelerated before, it will start to decelerate afterward, and if it decelerated before, it will start to accelerate afterward. Therefore, this point is the dividing point for the hand movement to switch from the acceleration phase to the deceleration phase or vice versa. The judgment and recording module is specifically used for: S41. Detect the head region in the image and extract the x-axis coordinate of the head center point, denoted as... ; S42. Calculate the x-axis difference between the key points and the center point of the head; Calculate the turning point The difference between the x-axis coordinate and the x-axis coordinate of the head center point: ; Since turning points only reflect the positional state at a certain instant and cannot reflect the continuity and trend of movement, an intermediate frame of the image sequence is added, denoted as the m-th frame, for auxiliary judgment to verify whether the direction of the turning point is continuous. The difference between the x-axis coordinate of the intermediate frame and the x-axis coordinate of the head center point is calculated. ; S43. Determine the direction of hand movement based on the difference; Preset rightward threshold and left threshold The left and right critical points for direction determination, if and This indicates that the x-coordinates of the turning point and the intermediate frame are both larger than the x-coordinate of the head center point, and exceed the rightward threshold. If the current direction of movement is determined to be to the right; and This indicates that the absolute value of the difference between the x-coordinate of the turning point and the middle frame and the x-coordinate of the head center point is within the threshold range, thus indicating movement towards the center; if and This indicates that the x-coordinates of the turning point and the intermediate frame are both smaller than the x-coordinate of the head center point, and also smaller than the leftward threshold. If so, it is determined to be moving to the left.

7. An electronic device comprising a processor and a memory, wherein the memory stores at least one instruction, characterized in that, The at least one instruction is loaded and executed by the processor to implement the driver gesture dynamic detection method based on kinematic features as described in any one of claims 1-5.

8. A computer-readable storage medium storing at least one instruction, characterized in that, The at least one instruction is loaded and executed by the processor to implement the driver gesture dynamic detection method based on kinematic features as described in any one of claims 1-5.

Images