Monocular vision snakelike running action detection method, device, equipment and medium

Abstract

The invention provides a monocular vision snakelike running action detection method and device, equipment and a medium. The method comprises the steps that a real-time video stream of a snakelike running test site is collected through a single camera; performing area cutting on the video frame based on the site calibration point to obtain a human body detection area and a rod body detection area; loading and running a deep learning model, performing asynchronous reasoning on the video frame, and obtaining a human body detection frame, a human body posture key point, a rod body detection frame and a rod body state classification result; continuously tracking the detected human body target based on a human body tracking algorithm, and obtaining a motion track of an effective tester; according to the spatial relationship among the human body position, the posture key point and the rod array, the motion stage is judged in real time; in the movement process, detecting at least one illegal behavior in real time; according to the exercise stage and the violation detection result, automatic timing is carried out, a structured report containing exercise performance and violation information is generated, and full-process automatic detection can be achieved only through a single camera.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a monocular vision method, apparatus, device, and medium for detecting serpentine running motion. Background Technology

[0002] The serpentine run is a widely used speed and agility test in physical fitness testing, sports training, and student physical fitness evaluation. This test typically requires the test taker to run back and forth in an "S" shaped pattern between designated poles, navigating around the poles in sequence without touching or missing them, and completing the movement within the valid starting and finishing zones. To ensure the objectivity and fairness of the test results, accurate assessments are needed of the test taker's running path, the order of navigating the poles, the direction of navigating, whether they touched or fell a pole, whether they missed a pole, whether they crossed the boundary, whether they false-started, and the overall continuity of their movements.

[0003] Currently, the detection and judgment of serpentine running movements mostly rely on the following types of technologies: (1) Manual observation and manual timing methods.

[0004] In traditional physical fitness tests, judges visually assess whether test takers have followed the prescribed route and record scores using manual timers. This method is inherently subjective and easily influenced by factors such as observation angle, reaction time, and venue conditions, leading to misjudgments, omissions, or timing errors. It is therefore unsuitable for meeting the objective requirements of large-scale testing or rigorous examinations.

[0005] (2) Action determination technology based on wearable sensors (such as IMU).

[0006] Some existing technologies involve installing inertial sensors on the test subject's body or shoes to determine the continuity of their actions or movements by measuring data such as acceleration and angular velocity. However, wearable devices can affect the naturalness of running, increase the complexity of the test, and present problems such as device loss, improper wearing, and difficulties in managing multiple test participants, thus limiting their practical application scenarios.

[0007] (3) Motion path recovery system based on binocular vision or multi-camera spatial positioning.

[0008] Some research and commercial products already utilize binocular cameras, multi-camera arrays, or laser measurement devices for spatial 3D reconstruction, achieving high-precision path determination. However, such systems are typically expensive to implement, complex to deploy, and require precise on-site calibration, making them unsuitable for flexible deployment in ordinary schools, sports fields, or mobile testing scenarios.

[0009] (4) Motion detection method based on ordinary monocular vision.

[0010] Some existing literature proposes using monocular cameras combined with human detection and target tracking technologies to analyze running. However, since monocular vision cannot directly provide depth information, existing solutions struggle to accurately determine pole positions, pole-circling sequences, circling directions, timing of actions, and pole-falling states. Furthermore, most solutions fail to simultaneously address fine-grained evaluation needs such as pole-missing detection, out-of-bounds detection, false start detection, and action area validity assessment, resulting in insufficient overall functional coverage.

[0011] In summary, the existing technology still has the following shortcomings: 1. Relying on manual judgment results in large errors and low efficiency, making it unsuitable for standardized exams and large-scale tests.

[0012] 2. Wearable devices affect natural movements, are complex to manage, and cannot guarantee the fairness of testing.

[0013] 3. Dual-camera or multi-camera solutions are costly and difficult to deploy, limiting their widespread application.

[0014] 4. Existing monocular vision methods are insufficient in judging several key actions, such as pole array detection, pole direction, missing poles, falling poles, going out of bounds, and false starts.

[0015] 5. Lack of unified logical management for the entire motion process, including state machine control, real-time tracking, and multi-model collaborative reasoning capabilities.

[0016] 6. Timing, violation judgment, and motion recognition are not integrated into a single system, making it impossible to generate complete scores and analysis reports that can be automatically output.

[0017] Therefore, there is an urgent need for a technical solution that can be deployed with a regular single camera, has multi-model collaborative reasoning capabilities, and can detect the details of serpentine running movements in real time and make judgments on multiple types of violations, so as to improve the objectivity, fairness and automation of physical fitness tests. Summary of the Invention

[0018] The technical problem to be solved by the present invention is to provide a monocular vision serpentine running motion detection method, device, equipment and medium, which can complete motion recognition, path judgment, violation detection and score output of the entire serpentine running test process without the need for additional sensors, binocular equipment and complex deployment, thereby effectively solving the problems of insufficient accuracy, incomplete functions, high deployment cost and poor practicality of the existing technology.

[0019] In a first aspect, the present invention provides a monocular vision method for detecting serpentine running motions, comprising the following steps: Step 1: Capture real-time video streams of the serpentine test track using a single camera; Step 2: Based on the field landmarks, perform region cropping on the video frames to obtain the human body detection region and the pole detection region; Step 3: Load and run the deep learning model to perform asynchronous inference on the video frames and obtain human body detection boxes, human pose key points, pole detection boxes, and pole state classification results. Step 4: Continuously track the detected human targets based on the human body tracking algorithm to obtain the movement trajectory of the valid test personnel; Step 5: Based on the spatial relationship between the human body position, key posture points and the pole array, determine the movement stage in real time. The movement stage includes the preparation state, the waiting for the start voice to end state, the pole detection state, the reaching the end state and the end state. Step 6: During the movement, based on the human body detection frame, the human body posture key points, the pole detection frame, and the pole state classification results, detect at least one violation in real time; Step 7: Based on the exercise stage and violation detection results, automatically time the exercise and generate a structured report containing exercise performance and violation information.

[0020] Secondly, the present invention provides a monocular vision serpentine running motion detection device, comprising: The video acquisition module captures real-time video streams from the serpentine test track using a single camera; The cropping module performs region cropping on video frames based on field landmarks to obtain human detection regions and pole detection regions. The model loading and data acquisition module loads and runs the deep learning model, performs asynchronous inference on video frames, and obtains human body detection boxes, human pose key points, pole detection boxes, and pole state classification results. The motion trajectory acquisition module continuously tracks detected human targets based on a human tracking algorithm to acquire the motion trajectory of valid test personnel; The status acquisition module determines the movement stage in real time based on the spatial relationship between the human body position, key posture points and the pole array. The movement stages include the preparation state, the waiting for the start voice to end state, the pole detection state, the reaching the destination state and the end state. The violation detection module detects at least one violation in real time during the movement, based on the human body detection frame, the human body posture key points, the pole detection frame, and the pole state classification results. The exercise detection report module automatically times the exercise and generates a structured report containing exercise performance and violation information based on the exercise stage and violation detection results.

[0021] Thirdly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect.

[0022] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0023] One or more technical solutions provided by this invention have at least the following technical effects or advantages: 1. Utilize monocular vision to replace manual observation and complex hardware: Acquire images through a single camera, build a lightweight visual detection framework, realize automatic recognition and analysis of the entire serpentine running process, and avoid problems such as high cost of binocular systems and interference from wearable devices on movement.

[0024] 2. Compensating for the lack of spatial information in monocular vision through multi-model collaborative reasoning: Introducing multiple visual models such as human detection, key point recognition, pole detection, and pole fall classification, and combining them with site location information, the system can simultaneously grasp the spatial relationship between the athlete and the pole array under monocular conditions.

[0025] 3. This invention establishes state machine logic for processes such as starting, navigating around poles, returning, and arriving, enabling the system to automatically determine the action state according to the rules of a real serpentine run, thus overcoming the problem of incomplete recognition of action stages in existing algorithms.

[0026] 4. This invention uses geometric relationships such as joint position, left and right position of ankle relative to pole, and vertical position change of character frame relative to pole baseline, combined with continuous frame temporal features, to accurately judge behaviors such as missing pole, wrong direction detour, and incomplete pole detour.

[0027] 5. Integrating action event rules to achieve "automatic identification of multiple types of violations": such as falling poles, missing poles, crossing boundaries, false starts, invalid starting zones, invalid arrival zones, personnel intrusion, and anti-cheating measures are all automatically triggered through the rule system to achieve comprehensive violation detection and avoid human misjudgment and omissions.

[0028] 6. This invention integrates detection, tracking, judgment, and timing into an automated business process through unified process control, and finally outputs results and analysis data, achieving a high degree of automation and standardization in testing.

[0029] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0030] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0031] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the device in Embodiment 2 of the present invention. Detailed Implementation

[0032] The overall concept of the technical solution in this application is as follows: An automatic detection method for serpentine running motion based on monocular vision includes the following steps: 1. System Deployment and Initialization (1) Hardware deployment: A single camera is set up at a height of 2.5-3.5m on the side of the serpentine track to ensure that the lens covers the entire track (including the starting area, 7 poles, and the finish area) without visual obstruction; the camera is connected to a computing device with an Intel Core i7 or higher CPU, an NVIDIA GeForce RTX 3060 or higher GPU, and ≥16GB of memory.

[0033] (2) Software environment configuration: Install Ubuntu 18.04 / 20.04 LTS or Windows 10 / 11 Professional Edition operating system, configure basic dependency libraries such as OpenCV 4.5.5 and above, CUDA 11.3 and above, cuDNN 8.2 and above, TensorRT 8.4 and above; load deep learning models, including human detection model (YOLOv5s), pose recognition model (RTMPose 26 key points + optional 133-point high-precision model), pole detection model (YOLOv5n), pole classification model (ResNet18), face recognition model (ArcFace) and tracking algorithm (BYTETracker).

[0034] (3) Field calibration: Collect the coordinates of the four corner points and the seven pole base calibration points of the field using calibration tools, establish the field coordinate system, clarify the starting area, the finish area and the pole distribution boundary, and save the calibration data to the system.

[0035] (4) System parameter initialization: Read the field location data and motion start parameters (video path, task ID, lane number), configure the motion mode (default mode / automatic mode / free mode / immediate mode), enable the violation detection switch (missing pole / falling pole / false start / out of bounds / anti-proxy test, etc.); initialize the camera acquisition parameters (resolution: 720P in timing mode only, 2160P in full mode, frame rate 25fps), number of frame buffers (2 frames), and violation video storage directory.

[0036] 2. Video Acquisition and Preprocessing (1) Video stream acquisition: Start a single camera to acquire video frames of the entire snake run test process in real time, and ensure data continuity by calculating the cached frame data of the computing device.

[0037] (2) Region clipping optimization: Based on the field calibration data, the acquired frames are clipped to divide the human body detection area and the pole detection area. The pole detection area is further divided into the near pole (first 3 poles) detection area and the far pole (last 4 poles) detection area to reduce invalid area calculation and improve detection efficiency.

[0038] (3) Frame filtering: In timed mode only, the "4 frames skip 1 frame" strategy is adopted to balance detection efficiency and real-time performance; in full mode, all acquisition frames are retained to ensure detection accuracy.

[0039] 3. Core Testing Process (1) Human body detection and tracking: Human targets are detected in video frames using the YOLOv5s model, and human bounding boxes are output. Based on the BYTETracker tracker, a unique tracking ID is assigned to the detected human body to continuously track the human body in motion and avoid occlusion or target loss between frames. Select valid human targets within the calibration area (excluding off-site personnel), prioritize retaining the human bounding box with the largest area as the test object, and update the real-time location information of the human body.

[0040] (2) Human posture key point recognition: For the tracked human target, 26 or 133 pose key points (including ankles, knees, shoulders, fingertips, etc.) are extracted using the RTMPose model, and the coordinates of the key points are output in real time. Verify the validity of the posture and remove invalid posture data with missing or abnormal key points to provide a basis for subsequent motion phase identification and violation judgment.

[0041] (3) Pole array detection and state recognition: The YOLOv5n model is used to synchronously detect pole targets in video frames and output the pole bounding boxes. Based on the site calibration data, the pole detection frame is bound to the calibration coordinates of the 7 poles, and the base coordinates, axis (X-axis left and right, Y-axis up and down) and left and right distribution attributes of each pole are initialized; The ResNet18 pole classification model is started at 1-second intervals to perform asynchronous inference on the pole region, calculate the similarity between the current features and historical features of the pole, and accumulate the number of pole collapse judgments. When the accumulated number exceeds 2, it is judged as a pole collapse.

[0042] 4. Motion state machine and key phase determination Establish a dedicated state machine for the serpentine running pattern, implementing state transitions according to the following timing sequence to identify key movement phases: (1) Ready state (PREPARE): Default mode: Wait for external manual start command, and at the same time detect whether the tester has crossed the line. If the tester crosses the line, a "crossing the line" voice broadcast will be triggered. Automatic mode: When the tester is detected to have entered the starting area and stayed there for more than 10 frames without stepping on the line, the start process is automatically triggered. Free Mode: When the test subject enters the starting area and starts running autonomously, switch to the pole detection mode; Immediate mode: The startup process is triggered directly after the system starts, without waiting for personnel to prepare.

[0043] (2) Waiting for the voice to end (WAIT_START_SOUND_END): Play the "Ready to Run" countdown audio and record the timestamp when the audio ends; The system detects whether the test subject has started running prematurely. If a starting action is detected before the voice command ends, it is considered a false start. After the voice recording ends, the validity of the starting zone is checked. If the tester's ankles are outside the valid starting zone, it is considered an invalid start and a violation. If no violation is found, switch to the pole-looping detection state and record the start time of the movement.

[0044] (3) HOOP_CHECK status: Based on the spatial relationship between key points of the human body and the axis of the pole, determine the direction of the pole (forward / backward) and the state of the pole (before / during / after the pole), record the start and end times of the pole, and calculate the time taken to circle a single pole; Real-time detection of violations such as pole tilting, going out of bounds, unauthorized personnel intrusion, and cheating on exams, triggering corresponding voice broadcasts and evidence preservation; The system detects whether the test subject has crossed the finish line; if so, it switches to the finish line status.

[0045] (4) Arrival at the destination (ARRIVE_SUCCESS): The validity of the arrival zone is checked. If the tester's ankle does not enter the valid finish line area, it is judged as an invalid arrival violation. Verify the routing status of all poles. If any pole has not completed its routing, it is considered a missed pole violation. To prevent cheating during exams, a second verification process is implemented to confirm that the facial information of the test taker matches that of the registration personnel. If there are no violations, record the end time of the exercise and calculate the total time and statistics of the pole-circling statistics (fastest / slowest / average pole-circling time, average speed).

[0046] (5) Finished state: Save keyframe images of the starting moment, the finishing moment, and various violations; Generate a structured analysis report, including data such as violation status, pole maneuvering time statistics, and average speed; Output the final test results (valid score / violation type) and trigger the end voice broadcast.

[0047] 5. Automatic identification of violations Based on human posture, pole status, time-series data, and regional location, the following eight types of violations are determined in real time. The specific determination logic is as follows: (1) False start violation: When the starting action (ankle or bottom of the body crossing the starting line) is detected during the waiting start voice end phase, and the current time is earlier than the voice end timestamp, the "false start violation" voice broadcast is immediately triggered, the violation image and video are saved, the score is set to 0, and the exercise is terminated.

[0048] (2) Missing pole violation: After reaching the finish line, if there is a pole that is not marked as "End of detour", the "Missing pole violation" voice broadcast will be triggered, the violation video will be saved, and the score will be set to 0.

[0049] (3) Falling pole violation: If the pole classification model has accumulated more than 2 falling pole judgments, the "falling pole" voice broadcast will be triggered in real time, the violation image will be saved, and the movement will be terminated.

[0050] (4) Out of bounds violation: When the center point of the bottom of the human body (the midpoint of the lower edge of the human body frame) exceeds the left and right boundaries of the field, the voice broadcast of "human out of bounds violation" is triggered, the violation image and video are saved, and the movement is terminated.

[0051] (5) Invalid starting zone violation: When the tester’s ankles are outside the valid starting zone (starting pole X coordinate < left and right ankle X coordinates), the voice broadcast “Not in valid starting zone” is triggered, the score is set to 0, and the exercise is terminated.

[0052] (6) Invalid arrival zone violation: When the tester reaches the finish line, the ankle does not enter the valid arrival zone (starting pole X coordinate > left ankle X coordinate), triggering the voice broadcast of "not in the valid finish zone", the score is set to 0, and the exercise is terminated.

[0053] (7) Prevention of cheating during the test: In the middle of the pole detection stage (when the second pole is reached), face verification is initiated. If face extraction fails, Seeta face validity detection fails, or face recognition does not match the registered person (cumulative failure count > 0), the voice broadcast of "Face secondary verification failed" is triggered, and the score is set to 0.

[0054] (8) Personnel intrusion violation: The number of people in the venue is monitored throughout the process. If more than 1 person is present, a voice broadcast of "Personnel intrusion violation" will be triggered. The broadcast will only be given once per second until the excess person leaves.

[0055] 6. Timing and Result Output (1) Real-time timing: Starting from the effective start time of the movement, the timing status is refreshed every 100ms to update the movement duration in real time; the start and end times of the circle around each pole are recorded, and the time spent circling a single pole is calculated.

[0056] (2) Calculation of results: After the exercise, if there is no violation, calculate the total time (end time - start time), the fastest time to circle the pole, the slowest time to circle the pole, the average time to circle the pole (total time to circle the pole / number of poles) and the average speed (total length of the field 60 meters / total time); if there is a violation, the score is set to 0.

[0057] (3) Evidence preservation: Save key frame images (JPG format) of the starting moment, the ending moment, and various violation moments, and save violation video clips (MP4 format). The storage path is the storage directory configured during initialization.

[0058] (4) Report generation and output: Generate a structured JSON format analysis report, including information such as test lanes, exercise duration, violation status, pole-driving statistics, and average speed; output the final test results (valid score / violation type / violation description) through the computing device, supporting subsequent traceability and analysis.

[0059] 7. Resource Release After the exercise is completed (normal completion / timeout / violation termination), release resources such as deep learning model instances, camera acquisition streams, and trackers, clear frame buffers and temporary data, ensure stable system operation, and avoid memory leaks.

[0060] By adopting the monocular vision serpentine running detection scheme of this invention, the system can achieve automated recognition, refined action judgment, and multi-type violation detection of the entire serpentine running process without the need for wearable devices, multiple cameras, or complex deployment. This significantly reduces subjective errors caused by manual judgment and improves the consistency and stability of the judgment. Through multi-model collaborative reasoning and geometric temporal logic judgment, it can accurately identify key events such as pole-circling direction, missing poles, pole-falling, boundary crossing, false start, and invalid areas, making the test more objective and fair. At the same time, the system can automatically time, output complete results and action analysis reports, improve physical fitness testing efficiency, reduce personnel operating costs, and significantly enhance the feasibility and reliability of large-scale examination and training scenarios for serpentine running projects.

[0061] Example 1 like Figure 1 As shown, this embodiment provides a monocular vision method for detecting serpentine running motion, including the following steps: Step 1: Capture real-time video streams of the serpentine test track using a single camera; Step 2: Based on the field landmarks, perform region cropping on the video frames to obtain the human body detection region and the pole detection region; Step 3: Load and run the deep learning model to perform asynchronous inference on the video frames and obtain human body detection boxes, human pose key points, pole detection boxes, and pole state classification results. Step 4: Continuously track the detected human targets based on the human body tracking algorithm to obtain the movement trajectory of the valid test personnel; Step 5: Based on the spatial relationship between the human body position, key posture points and the pole array, determine the movement stage in real time. The movement stage includes the preparation state, the waiting for the start voice to end state, the pole detection state, the reaching the end state and the end state. Step 6: During the movement, based on the human body detection frame, the human body posture key points, the pole detection frame, and the pole state classification results, detect at least one violation in real time; Step 7: Based on the exercise stage and violation detection results, automatically time the exercise and generate a structured report containing exercise performance and violation information.

[0062] In this embodiment, preferably, loading and running the deep learning model includes: Load the YOLOv5s model for target detection of humans and poles; Load the RTMPose model for human pose key point recognition; Load the ResNet18 model for binary classification of the pole's state to determine whether the pole is upright or tilted; Load the ArcFace model for facial recognition to achieve secondary verification to prevent cheating during exams; The BYTETracker algorithm is loaded for continuous human body tracking.

[0063] In this embodiment, preferably, the region cropping of the video frame based on field landmark positioning includes: Based on the boundary points obtained from the venue markings, calculate and cut out the human detection area covering the starting area, the pole-crossing area, and the finish area; Based on the pole array calibration points, the detection area of ​​the distant poles is calculated and cut out.

[0064] In this embodiment, preferably, step 5 specifically comprises: Based on the spatial relationship between the human body position, key posture points and the pole array, the movement stage is determined in real time. The movement stage includes the preparation state, the waiting for the start voice end state, the pole detection state, the end point state, and the end state. In the ready state, the system detects whether personnel have entered the preparation area or are waiting for an external start command, based on the configured motion mode, and also detects whether they have stepped on the line. While waiting for the startup voice to finish playing, determine whether the startup voice has finished playing and detect if there is any preemptive behavior. In the pole-looping detection state, the system detects pole-looping actions, pole tilting, human body crossing the boundary, and personnel intrusion in real time, and determines whether the endpoint has been reached. Upon reaching the endpoint, a final determination of the missing person and a secondary verification to prevent cheating are conducted. In the terminated state, business processing is stopped and resources are prepared to be released.

[0065] In this embodiment, preferably, the real-time detection of at least one violation includes at least one of the following: False start violation detection: The person was determined to have started running before the start-up voice command ended; Pole tilting violation detection: The pole tilting status is determined cumulatively through a pole classification model; Missing pole violation detection: After the exercise is completed, determine whether any poles were not marked as having completed the detour; Boundary violation detection: Determines whether the center point of the bottom of the human body exceeds the boundary formed by the fixed points of the field. Invalid starting zone violation detection: Determines whether the key point of the ankle exceeds the boundary of the starting zone marked by the starting pole at the start of the race; Invalid finish zone violation detection: Determines whether the key ankle point has not entered the finish zone boundary marked by the starting pole upon arrival; Anti-proxy test detection: Verify the identity of test takers using a facial recognition model during the pole-driving process; Personnel intrusion detection: Determines whether the number of human targets within the site is greater than one.

[0066] In this embodiment, preferably, the pole reversal violation detection specifically includes: For each detected pole region, an image is captured and submitted to the pole classification model for asynchronous inference; Obtain the classification result. If the result is "tilted", then accumulate the number of times the pole has been tilted. If the cumulative number of times the same pole is tilted exceeds a preset threshold, it is considered a violation of the pole tilting rule.

[0067] In this embodiment, preferably, the generation of the structured report includes: Record the start and end times of the exercise, and calculate the total time as the exercise score; Statistics on violations including false starts, pole tilting, missing poles, going out of bounds, invalid starts, and invalid finishes; Calculate the fastest time to circle the pole, the slowest time to circle the pole, the average time to circle the pole, and the average speed; Save images of the starting moment, the finishing moment, and images and videos of each violation.

[0068] Based on the same inventive concept, this application also provides an apparatus corresponding to the method in Embodiment 1, as detailed in Embodiment 2.

[0069] Example 2 like Figure 2 As shown, this embodiment provides a monocular vision serpentine running motion detection device, including: The video acquisition module captures real-time video streams from the serpentine test track using a single camera; The cropping module performs region cropping on video frames based on field landmarks to obtain human detection regions and pole detection regions. The model loading and data acquisition module loads and runs the deep learning model, performs asynchronous inference on video frames, and obtains human body detection boxes, human pose key points, pole detection boxes, and pole state classification results. The motion trajectory acquisition module continuously tracks detected human targets based on a human tracking algorithm to acquire the motion trajectory of valid test personnel; The status acquisition module determines the movement stage in real time based on the spatial relationship between the human body position, key posture points and the pole array. The movement stages include the preparation state, the waiting for the start voice to end state, the pole detection state, the reaching the destination state and the end state. The violation detection module detects at least one violation in real time during the movement, based on the human body detection frame, the human body posture key points, the pole detection frame, and the pole state classification results. The exercise detection report module automatically times the exercise and generates a structured report containing exercise performance and violation information based on the exercise stage and violation detection results.

[0070] In this embodiment, preferably, loading and running the deep learning model includes: Load the YOLOv5s model for target detection of humans and poles; Load the RTMPose model for human pose key point recognition; Load the ResNet18 model for binary classification of the pole's state to determine whether the pole is upright or tilted; Load the ArcFace model for facial recognition to achieve secondary verification to prevent cheating during exams; The BYTETracker algorithm is loaded for continuous human body tracking.

[0071] In this embodiment, preferably, the region cropping of the video frame based on field landmark positioning includes: Based on the boundary points obtained from the venue markings, calculate and cut out the human detection area covering the starting area, the pole-crossing area, and the finish area; Based on the pole array calibration points, the detection area of ​​the distant poles is calculated and cut out.

[0072] In this embodiment, preferably, the status acquisition module specifically comprises: Based on the spatial relationship between the human body position, key posture points and the pole array, the movement stage is determined in real time. The movement stage includes the preparation state, the waiting for the start voice end state, the pole detection state, the end point state, and the end state. In the ready state, the system detects whether personnel have entered the preparation area or are waiting for an external start command, based on the configured motion mode, and also detects whether they have stepped on the line. While waiting for the startup voice to finish playing, determine whether the startup voice has finished playing and detect if there is any preemptive behavior. In the pole-looping detection state, the system detects pole-looping actions, pole tilting, human body crossing the boundary, and personnel intrusion in real time, and determines whether the endpoint has been reached. Upon reaching the endpoint, a final determination of the missing person and a secondary verification to prevent cheating are conducted. In the terminated state, business processing is stopped and resources are prepared to be released.

[0073] In this embodiment, preferably, the real-time detection of at least one violation includes at least one of the following: False start violation detection: The person was determined to have started running before the start-up voice command ended; Pole tilting violation detection: The pole tilting status is determined cumulatively through a pole classification model; Missing pole violation detection: After the exercise is completed, determine whether any poles were not marked as having completed the detour; Boundary violation detection: Determines whether the center point of the bottom of the human body exceeds the boundary formed by the fixed points of the field. Invalid starting zone violation detection: Determines whether the key point of the ankle exceeds the boundary of the starting zone marked by the starting pole at the start of the race; Invalid finish zone violation detection: Determines whether the key ankle point has not entered the finish zone boundary marked by the starting pole upon arrival; Anti-proxy test detection: Verify the identity of test takers using a facial recognition model during the pole-driving process; Personnel intrusion detection: Determines whether the number of human targets within the site is greater than one.

[0074] In this embodiment, preferably, the pole reversal violation detection specifically includes: For each detected pole region, an image is captured and submitted to the pole classification model for asynchronous inference; Obtain the classification result. If the result is "tilted", then accumulate the number of times the pole has been tilted. If the cumulative number of times the same pole is tilted exceeds a preset threshold, it is considered a violation of the pole tilting rule.

[0075] In this embodiment, preferably, the generation of the structured report includes: Record the start and end times of the exercise, and calculate the total time as the exercise score; Statistics on violations including false starts, pole tilting, missing poles, going out of bounds, invalid starts, and invalid finishes; Calculate the fastest time to circle the pole, the slowest time to circle the pole, the average time to circle the pole, and the average speed; Save images of the starting moment, the finishing moment, and images and videos of each violation.

[0076] Since the apparatus described in Embodiment 2 of the present invention is an apparatus used to implement the method of Embodiment 1 of the present invention, those skilled in the art can understand the specific structure and variations of the apparatus based on the method described in Embodiment 1 of the present invention, and therefore will not be described again here. All apparatuses used in the method of Embodiment 1 of the present invention fall within the scope of protection of the present invention.

[0077] Based on the same inventive concept, this application provides an electronic device embodiment corresponding to Embodiment 1, as detailed in Embodiment 3.

[0078] Example 3 This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it can implement any of the implementation methods in Embodiment 1.

[0079] Since the electronic device described in this embodiment is the device used to implement the method in Embodiment 1 of this application, those skilled in the art can understand the specific implementation method and various variations of the electronic device in this embodiment based on the method described in Embodiment 1 of this application. Therefore, how the electronic device implements the method in the embodiment of this application will not be described in detail here. Any device used by those skilled in the art to implement the method in the embodiment of this application falls within the scope of protection of this application.

[0080] Based on the same inventive concept, this application provides a storage medium corresponding to Embodiment 1, as detailed in Embodiment 4.

[0081] Example 4 This embodiment provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it can implement any of the implementation methods in Embodiment 1.

[0082] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0083] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0084] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0085] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0086] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A monocular vision method for detecting serpentine running motion, characterized in that: Includes the following steps: Step 1: Capture real-time video streams of the serpentine test track using a single camera; Step 2: Based on the field landmarks, perform region cropping on the video frames to obtain the human body detection region and the pole detection region; Step 3: Load and run the deep learning model to perform asynchronous inference on the video frames and obtain human body detection boxes, human pose key points, pole detection boxes, and pole state classification results. Step 4: Continuously track the detected human targets based on the human body tracking algorithm to obtain the movement trajectory of the valid test personnel; Step 5: Based on the spatial relationship between the human body position, key posture points and the pole array, determine the movement stage in real time. The movement stage includes the preparation state, the waiting for the start voice to end state, the pole detection state, the reaching the end state and the end state. Step 6: During the movement, based on the human body detection frame, the human body posture key points, the pole detection frame, and the pole state classification results, detect at least one violation in real time; Step 7: Based on the exercise stage and violation detection results, automatically time the exercise and generate a structured report containing exercise performance and violation information.

2. The monocular vision serpentine running motion detection method according to claim 1, characterized in that: The loading and running of the deep learning model includes: Load the YOLOv5s model for target detection of humans and poles; Load the RTMPose model for human pose key point recognition; Load the ResNet18 model for binary classification of the pole's state to determine whether the pole is upright or tilted; Load the ArcFace model for facial recognition to achieve secondary verification to prevent cheating during exams; The BYTETracker algorithm is loaded for continuous human body tracking.

3. The monocular vision serpentine running motion detection method according to claim 1, characterized in that: The method of region cropping of video frames based on field landmark positioning includes: Based on the boundary points obtained from the venue markings, calculate and cut out the human detection area covering the starting area, the pole-crossing area, and the finish area; Based on the pole array calibration points, the detection area of ​​the distant poles is calculated and cut out.

4. The monocular vision serpentine running motion detection method according to claim 1, characterized in that: Step 5 specifically involves: Based on the spatial relationship between the human body position, key posture points and the pole array, the movement stage is determined in real time. The movement stage includes the preparation state, the waiting for the start voice end state, the pole detection state, the end point state, and the end state. In the ready state, the system detects whether personnel have entered the preparation area or are waiting for an external start command, based on the configured motion mode, and also detects whether they have stepped on the line. While waiting for the startup voice to finish playing, determine whether the startup voice has finished playing and detect if there is any preemptive behavior. In the pole-looping detection state, the system detects pole-looping actions, pole tilting, human body crossing the boundary, and personnel intrusion in real time, and determines whether the endpoint has been reached. Upon reaching the endpoint, a final determination of the missing person and a secondary verification to prevent cheating are conducted. In the terminated state, business processing is stopped and resources are prepared to be released.

5. The monocular vision serpentine running motion detection method according to claim 1, characterized in that: The real-time detection of at least one violation includes at least one of the following: False start violation detection: The person was determined to have started running before the activation voice command ended; Pole tilting violation detection: The pole tilting status is determined cumulatively through a pole classification model; Missing pole violation detection: After the exercise is completed, determine whether any poles were not marked as having completed the detour; Boundary violation detection: Determines whether the center point of the bottom of the human body exceeds the boundary formed by the fixed points of the field. Invalid starting zone violation detection: Determines whether the key point of the ankle exceeds the boundary of the starting zone marked by the starting pole at the start of the race; Invalid finish zone violation detection: Determines whether the key ankle point has not entered the finish zone boundary marked by the starting pole upon arrival; Anti-proxy test detection: Verify the identity of test takers using a facial recognition model during the pole-driving process; Personnel intrusion detection: Determines whether the number of human targets within the site is greater than one.

6. The monocular vision serpentine running motion detection method according to claim 1, characterized in that: The specific detection of pole reversal violations includes: For each detected pole region, an image is captured and submitted to the pole classification model for asynchronous inference; Obtain the classification result. If the result is "tilted", then accumulate the number of times the pole has been tilted. If the cumulative number of times the same pole is tilted exceeds a preset threshold, it is considered a violation of the pole tilting rule.

7. The monocular vision serpentine running motion detection method according to claim 1, characterized in that: The generation of the structured report includes: Record the start and end times of the exercise, and calculate the total time as the exercise score; Statistics on violations including false starts, pole tilting, missing poles, going out of bounds, invalid starts, and invalid finishes; Calculate the fastest time to circle the pole, the slowest time to circle the pole, the average time to circle the pole, and the average speed; Save images of the starting moment, the finishing moment, and images and videos of each violation.

8. A monocular vision serpentine running motion detection device, characterized in that: include: The video acquisition module captures real-time video streams from the serpentine test track using a single camera; The cropping module performs region cropping on video frames based on field landmarks to obtain human detection regions and pole detection regions. The model loading and data acquisition module loads and runs the deep learning model, performs asynchronous inference on video frames, and obtains human body detection boxes, human pose key points, pole detection boxes, and pole state classification results. The motion trajectory acquisition module continuously tracks detected human targets based on a human tracking algorithm to acquire the motion trajectory of valid test personnel; The status acquisition module determines the movement stage in real time based on the spatial relationship between the human body position, key posture points and the pole array. The movement stages include the preparation state, the waiting for the start voice to end state, the pole detection state, the reaching the destination state and the end state. The violation detection module detects at least one violation in real time during the movement, based on the human body detection frame, the human body posture key points, the pole detection frame, and the pole state classification results. The exercise detection report module automatically times the exercise and generates a structured report containing exercise performance and violation information based on the exercise stage and violation detection results.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images