Upward challenge visual-environment co-perception and edge deployment method and system

Abstract

The present application relates to the technical field of computer vision and intelligent edge computing, and is a vision-environment collaborative perception and edge deployment method for pull-up examination, comprising the following steps: collecting a video stream through a monocular RGB camera fixed in front of the side of an examination site, and performing multi-scale illumination compensation on each frame of image; the multi-scale illumination compensation comprises three stages. The vision-environment collaborative perception and edge deployment method and system for pull-up examination introduce a multi-scale illumination compensation mechanism, combine CLAHE global contrast enhancement, adaptive gamma correction and shadow detection compensation, effectively solve the problem of image quality degradation caused by direct sunlight or shadow blocking in the examination room, significantly improve the stability of feature extraction, and at the same time, the improved YOLOv8 detection network embeds a channel attention mechanism and adds a line constraint prediction head, thereby strengthening the modeling ability of the geometric relationship between human key points and the reference line of the single bar.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent edge computing technology, specifically to a visual-environment collaborative perception and edge deployment method and system for pull-up assessment. Background Technology

[0002] In traditional pull-up fitness assessments, manual counting or simple video playback for post-assessment is typically relied upon. This method not only consumes a large amount of human resources but is also prone to counting errors due to human fatigue or perspective deviation, making it difficult to meet the real-time and accuracy requirements of large-scale, high-concurrency scenarios. With the development of computer vision technology, automated assessment solutions based on monocular cameras have gradually emerged. However, in actual complex examination environments, drastic changes in natural lighting conditions (such as direct strong light or shadow occlusion) can severely interfere with image quality, leading to a decrease in the accuracy of human body key point detection. In addition, examinees often occlude each other's limbs or overlap with the protection personnel during the exercise. Existing multi-object tracking algorithms often fail when dealing with long-term occlusion and identity re-identification, resulting in missed detections or miscounts. Furthermore, the pull-up movement has obvious periodic characteristics. Without state machine logic that incorporates geometric constraints (such as the height of the chin over the bar and the angle of arm extension), it is difficult to accurately distinguish between valid movements and invalid swings based solely on posture estimation, resulting in low reliability of the assessment results.

[0003] On the other hand, while deploying high-precision deep learning models directly on cloud servers can guarantee computing power, it also presents problems such as high network latency, high bandwidth costs, and risks of data privacy leaks, making it difficult to meet the needs of real-time feedback on-site. When deployed at the edge, the computing resources and power consumption budget of embedded devices (such as NPUs) are limited, and the inference speed of unoptimized full-precision models is slow, failing to meet real-time requirements. Simple model compression methods often lead to a significant drop in mAP, affecting the accuracy of assessments. In addition, assessment data involves personal privacy and the fairness of grades, and traditional data storage methods lack effective integrity verification mechanisms, making them susceptible to tampering. Therefore, there is an urgent need for technical solutions that can synergistically optimize aspects such as lighting robustness, occlusion handling, accurate action judgment, and efficient and secure deployment at the edge, in order to solve the problems of poor stability, high deployment costs, and insufficient security and reliability of existing technologies in complex environments. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a visual-environment collaborative perception and edge deployment method and system for pull-up assessment. It has advantages such as strong robustness and adaptability to complex environments, and solves the problems of low efficiency, large subjective error, and difficulty in handling large-scale real-time assessments.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a visual-environment collaborative perception and edge deployment method for pull-up assessment, comprising the following steps: Step S1: Acquire video stream using a monocular RGB camera fixed to the side front of the assessment site, and perform multi-scale illumination compensation on each frame; the multi-scale illumination compensation includes three stages: Phase 1: Convert the image from RGB space to HSV space, extract the V channel luminance map, and use contrast-limited adaptive histogram equalization (CLAHE) to process the V channel. Set the block grid size to 8×8, the clipping threshold to clipLimit=2.0, and output the enhanced luminance map V'_CLAHE(x,y). Phase 2: Perform adaptive gamma correction on V'_CLAHE(x,y). The correction formula is γ(x,y)=1.0+α×(0.5-μ_L(x,y)), where μ_L(x,y) is the average brightness value within the local window of the pixel, and α is the adaptive coefficient. Phase 3: Construct a shadow mask S(p), with the determination formula S(p)=[L(p)<τ(p)]∧[||C(p)−B_c(p)||2<ε_c], where L(p) is the luminance component of pixel p, τ(p) is the local adaptive threshold, B_c(p) is the average chromaticity of neighboring non-shadow pixels, and ε_c is the chromaticity consistency threshold; perform gamma enhancement on shadow pixels that satisfy S(p)=1 to restore details; Step S2: Input the illumination-compensated image into the enhanced YOLOv8 detection network for target detection and pose estimation; the enhanced YOLOv8 detection network is improved on the basis of standard YOLOv8 by: embedding a channel attention mechanism in the C2f module to realize feature recalibration; adding a line constraint prediction head to predict the geometric relationship between human key points and the horizontal bar reference line; and using the task-aware gradient modulation (TAGM) mechanism to dynamically balance the training gradient weights of the three task heads for detection, pose estimation and line constraint. Step S3: Perform kinematic analysis based on the limb key points in the detection results, and execute the pull-up validity judgment through a finite state machine (FSM); the judgment includes three criteria: Rule 1: Calculate the vertical distance d_chin between the key point of the nose tip and the single-line reference line. When d_chin is greater than the preset first threshold, it is determined that the mandible has crossed the line. Rule 2: Calculate the elbow angle θ_elbow from key points of the shoulder, elbow and wrist. When θ_elbow is greater than or equal to the preset second threshold, the arm is considered to be straight. Criterion 3: Calculate the horizontal displacement σ_hip of the hip center relative to the initial static position. When σ_hip is less than the preset third threshold, it is determined that there is no excessive oscillation. The finite state machine sequentially goes through the states of Idle, Ascent, Hold, and Descent. Only actions that complete the complete state transition sequence are counted as valid counts. Step S4: The improved DeepSORT multi-target tracking framework is used to continuously track the detected target. The occlusion scene is handled by a five-step hierarchical re-association strategy, and a lightweight pose classifier is used to distinguish between examinees and protection personnel. Only the examinee category is counted. Step S5: Deploy the optimized detection model to the NPU of the edge computing device, and perform a three-stage progressive compression process of mixed precision quantization, knowledge distillation, and structured pruning in sequence; Step S6: Monitor the device temperature in real time through the temperature-sensing adaptive inference (TAI) framework, and dynamically adjust the inference strategy according to the temperature threshold, including reducing the input resolution, switching to pure INT8 inference, or activating the depth-adaptive skipping mechanism. Step S7: Perform end-to-end encrypted storage on all assessment data. The encryption key is managed through a five-stage lifecycle using TrustZone, and the data integrity is verified using a Merkle tree hash chain.

[0006] Furthermore, in step S1, the formula for calculating the local adaptive threshold τ(p) is τ(p) = μ_L,local × (1-β·σ_L,local), where μ_L,local and σ_L,local are the mean and standard deviation of the brightness within the local window, respectively, and β = 1.5; the gamma enhancement parameter γ_shadow used for shadow compensation is 0.6.

[0007] Furthermore, in step S2, the loss function of the line constraint head is defined as follows: ,in Key point The perpendicular distance to the single-bar reference line l_ref N represents the distance predicted by the network, and N is the number of visible keypoints.

[0008] Furthermore, in step S3, the first threshold δ_chin = 0.5cm is derived from the average distance from the tip of the nose to the chin of an adult Asian male after head tilt compensation and detection error correction; the second threshold is 160°, allowing natural overextension; the third threshold δ_swing = 3.0cm is determined by multiplying the 95th percentile of natural human body sway by a safety factor of 1.25.

[0009] Furthermore, in step S4, the five-step hierarchical reassociation strategy consists of: IoU matching, appearance feature cosine similarity matching, pose PCK similarity matching, Kalman trajectory prediction matching, and pose classification; wherein the IoU matching threshold is 0.5, the appearance matching threshold is 0.75, and the pose geometry matching threshold is 0.6; for long-term occlusion exceeding 30 frames, the potential trajectory is maintained for linear motion prediction, with a maximum maintenance of 60 frames.

[0010] Furthermore, in step S5, hybrid precision quantization performs INT8 quantization on the standard layers of the backbone network and retains FP16 precision on the sensitive layers; knowledge distillation uses full-precision YOLOv8-m as the teacher network and YOLOv8-s as the student network, employing soft-target distillation; structured pruning performs L1 norm channel pruning on the bottleneck layer of CSPDarknet with a sparsity ratio of 0.4; after comprehensive compression, the model size is compressed from 186.5MB to 12.8MB, while maintaining mAP above 96.2%.

[0011] Furthermore, in step S6, the degradation strategy of the TAI framework is as follows: when the device temperature T>45°C, the input resolution is reduced from 640×640 to 480×480; when T>50°C, the FP16 fallback path is closed, and all layers execute INT8 inference; when T>55°C, the skip-layer mechanism is activated, and forward propagation is performed once every other backbone block, and the model is switched to a lightweight model with 6.5M parameters.

[0012] Furthermore, in step S7, the TrustZone five-stage lifecycle management includes: Phase 1: Within the secure world, a hardware true random number generator (TRNG) is invoked to generate a 128-bit master key; Phase 2: Derive the session key from the master key using the HKDF-SM3 algorithm, binding the session identifier and timestamp during the derivation process; Phase 3: The master key is sealed and stored through the Trusted Platform Module (TPM) and bound to the PCR state of the platform configuration register; Phase 4: Session keys are automatically rotated every 4 hours, or immediately upon detection of tampering. Phase 5: After the session ends, the key buffer is released after being overwritten with cryptographically random data three times.

[0013] Furthermore, the data integrity verification adopts a Merkle tree SM3 hash chain, with leaf nodes being the hashes of single-frame detection results, and the root hash signature being stored together with the assessment data, supporting offline integrity verification.

[0014] This application further proposes a visual-environment collaborative perception and edge deployment method for pull-up assessment, including an input layer, a perception layer, an evaluation layer, and an edge intelligence layer connected in sequence: The input layer includes a monocular RGB camera fixed to the front side of the assessment site, with an installation height of 2.0~2.5m, a horizontal distance of 2.5~3.5m from the horizontal bar, a field of view covering the area from 0.5m above the horizontal bar to the ground, and a frame rate of not less than 30FPS; The perception layer includes a multi-scale illumination compensation module, an enhanced YOLOv8 detection network, and a multi-target tracking module. The multi-scale illumination compensation module sequentially includes a CLAHE global contrast enhancement unit, an adaptive gamma correction unit, and a shadow detection compensation unit. The enhanced YOLOv8 detection network includes a C2f backbone network with embedded channel attention, a PANet feature fusion neck, a detection head, a pose estimation head, a line constraint head, and a task-aware gradient modulation (TAGM) unit. The multi-target tracking module includes an improved DeepSORT tracker, a ResNet-18 re-identification network, and a five-step hierarchical re-association unit. The evaluation layer includes a limb kinematics analysis module and a pull-up validity judgment module; the validity judgment module uses a finite state machine to sequentially execute three criteria: chin over bar judgment, arm extension judgment, and body swing control. The edge intelligence layer includes a model optimization module, a temperature-sensing adaptive inference (TAI) module, and a security encryption module. The model optimization module includes an INT8 / FP16 hybrid quantization unit, a knowledge distillation unit, and a structured pruning unit. The TAI module includes a temperature monitoring unit, a decision tree scheduling unit, and a multi-level degradation execution unit. The security encryption module includes an SM4-CBC encryption unit, a Merkle tree integrity verification unit, and a TrustZone key management unit.

[0015] Compared with the prior art, the technical solution of this application has the following beneficial effects: 1. This visual-environment collaborative perception and edge deployment method and system for pull-up assessment effectively solves the problem of image quality degradation caused by direct strong light or shadow occlusion in the examination room by introducing a multi-scale illumination compensation mechanism, combined with CLAHE global contrast enhancement, adaptive gamma correction and shadow detection compensation, significantly improving the stability of feature extraction. At the same time, the improved YOLOv8 detection network embeds a channel attention mechanism and adds a line constraint prediction head, which enhances the modeling ability of the geometric relationship between human key points and the horizontal bar reference line. With the multi-target tracking framework of the five-step hierarchical re-association strategy, it can accurately handle the identity switching in scenarios where limb occlusion occurs and the examinee and the protection personnel overlap. In addition, the finite state machine decision logic based on the three criteria of chin over the bar, arm extension and body swing control ensures that only valid actions that complete the complete motion sequence are counted, thus achieving high-confidence automatic pull-up assessment in complex dynamic environments.

[0016] 2. The visual-environment collaborative perception and edge deployment method and system for pull-up assessment utilizes a three-stage progressive compression technique involving hybrid precision quantization, knowledge distillation, and structured pruning to significantly reduce the model size to 12.8MB. While maintaining an mAP accuracy of over 96.2%, it greatly reduces the storage and computational burden on edge devices. Combined with a temperature-aware adaptive inference (TAI) framework, the system can dynamically adjust the input resolution, inference precision, or activate a skip-layer mechanism based on the real-time temperature of the device, effectively preventing performance degradation or hardware damage caused by high temperatures and ensuring the stability of long-term assessments. In terms of data security, it relies on the TrustZone secure world to implement five-stage lifecycle management of keys and SM4 encrypted storage, and uses a Merkle tree hash chain to verify data integrity, completely eliminating the risk of assessment data tampering and meeting the strict requirements for data privacy and fairness in sensitive fields such as public security and sports. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the overall architecture of the system of the present invention; Figure 2 The flowchart shows the three-stage processing of the multi-scale illumination compensation module. Figure 3 This is a schematic diagram of the enhanced YOLOv8 network architecture; Figure 4 The state transition diagram of the finite state machine for determining the validity of a pull-up; Figure 5 A flowchart of the five-step hierarchical re-association strategy; Figure 6 This is a schematic diagram of a Temperature-Sensitive Adaptive Inference (TAI) decision tree. Figure 7 This is a sequence diagram for TrustZone's five-stage key lifecycle management. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Please see Figure 1-7 The visual-environment collaborative perception and edge deployment method for pull-up assessment includes the following steps: Step S1: Acquire video stream using a monocular RGB camera fixed to the side front of the assessment site, and perform multi-scale illumination compensation on each frame; multi-scale illumination compensation includes three stages: Phase 1: Convert the image from RGB space to HSV space, extract the V channel luminance map, and use contrast-limited adaptive histogram equalization (CLAHE) to process the V channel. Set the block grid size to 8×8, the clipping threshold to clipLimit=2.0, and output the enhanced luminance map V'_CLAHE(x,y). Phase 2: Perform adaptive gamma correction on V'_CLAHE(x,y). The correction formula is γ(x,y)=1.0+α×(0.5-μ_L(x,y)), where μ_L(x,y) is the average brightness value within the local window of the pixel, and α is the adaptive coefficient. Phase 3: Construct a shadow mask S(p), with the determination formula S(p)=[L(p)<τ(p)]∧[||C(p)-B_c(p)||2<ε_c], where L(p) is the luminance component of pixel p, τ(p) is the local adaptive threshold, B_c(p) is the average chromaticity of neighboring non-shadow pixels, and ε_c is the chromaticity consistency threshold; perform gamma enhancement on shadow pixels that satisfy S(p)=1 to restore details; Step S2: Input the illumination-compensated image into the enhanced YOLOv8 detection network for target detection and pose estimation; the enhanced YOLOv8 detection network is improved on the basis of standard YOLOv8, including: embedding a channel attention mechanism in the C2f module to realize feature recalibration; adding a line constraint prediction head to predict the geometric relationship between human keypoints and the horizontal bar reference line; and using the task-aware gradient modulation (TAGM) mechanism to dynamically balance the training gradient weights of the three task heads for detection, pose estimation and line constraint. Step S3: Perform kinematic analysis based on the limb key points in the detection results, and execute the pull-up validity judgment through a finite state machine (FSM); the judgment includes three criteria: Rule 1: Calculate the vertical distance d_chin between the key point of the nose tip and the single-line reference line. When d_chin is greater than the preset first threshold, it is determined that the mandible has crossed the line. Rule 2: Calculate the elbow angle θ_elbow from key points of the shoulder, elbow and wrist. When θ_elbow is greater than or equal to the preset second threshold, the arm is considered to be straight. Criterion 3: Calculate the horizontal displacement σ_hip of the hip center relative to the initial static position. When σ_hip is less than the preset third threshold, it is determined that there is no excessive oscillation. The finite state machine sequentially passes through the states of Idle, Ascent, Hold, and Descent. Only actions that complete the complete state transition sequence are counted as valid actions. Step S4: The improved DeepSORT multi-target tracking framework is used to continuously track the detected target. The occlusion scene is handled by a five-step hierarchical re-association strategy, and a lightweight pose classifier is used to distinguish between examinees and protection personnel. Only the examinee category is counted. Step S5: Deploy the optimized detection model to the NPU of the edge computing device, and perform a three-stage progressive compression process of mixed precision quantization, knowledge distillation, and structured pruning in sequence; Step S6: Monitor the device temperature in real time through the temperature-sensing adaptive inference (TAI) framework, and dynamically adjust the inference strategy according to the temperature threshold, including reducing the input resolution, switching to pure INT8 inference, or activating the depth-adaptive skipping mechanism. Step S7: Perform end-to-end encrypted storage on all assessment data. The encryption key is managed through a five-stage lifecycle using TrustZone, and the data integrity is verified using a Merkle tree hash chain.

[0020] By acquiring video streams and performing illumination compensation preprocessing, a deep learning model is used to obtain key points of the human body and the position of the horizontal bar. The validity of the action is determined by combining kinematic constraints and finite state machines. Multi-target tracking is used to distinguish examinees. The compressed and optimized model is deployed to edge devices. At the same time, a temperature sensing mechanism is used to ensure the stability of inference and the data is encrypted and stored.

[0021] In step S1, the formula for calculating the local adaptive threshold τ(p) is τ(p) = μ_L,local × (1-β·σ_L,local), where μ_L,local and σ_L,local are the mean and standard deviation of the brightness within the local window, respectively, and β = 1.5; the gamma enhancement parameter γ_shadow used for shadow compensation is 0.6.

[0022] The illumination compensation process of this method specifically includes converting the image to a luminance-chrominance space, enhancing the contrast of the luminance component, performing adaptive gamma correction in combination with local statistical characteristics, and identifying and compensating for shadow areas to solve the problem of image quality degradation caused by strong light or shadows.

[0023] In step S2, the loss function of the line constraint head is defined as follows: ,in Key point The perpendicular distance to the single-bar reference line l_ref N represents the distance predicted by the network, and N is the number of visible keypoints.

[0024] The target detection and pose estimation model used in this method is based on an improved YOLO architecture. It enhances feature recalibration by embedding an attention mechanism in the backbone network, adds a line constraint prediction head to predict the geometric relationship between key points and the single-bar reference line, and adopts a task-aware gradient modulation mechanism to balance the training weights of detection, pose estimation and line constraint tasks.

[0025] In step S3, the first threshold δ_chin = 0.5cm is derived from the average distance from the tip of the nose to the chin of an adult Asian male after head tilt compensation and detection error correction; the second threshold is 160°, allowing natural overextension; the third threshold δ_swing = 3.0cm is determined by multiplying the 95th percentile of natural human body sway by a safety factor of 1.25.

[0026] The validity determination of this method includes three criteria: the vertical distance between the tip of the nose and the bar determines whether the chin passes the bar, the elbow angle determines whether the arm is straight, and the hip displacement determines whether there is excessive swinging. The method also uses a finite state machine to sequentially pass through the static, ascending, peak hold and descending states, and only actions that complete the complete state transition sequence are counted as valid attempts.

[0027] In step S4, the five-step hierarchical reassociation strategy consists of: IoU matching, appearance feature cosine similarity matching, pose PCK similarity matching, Kalman trajectory prediction matching, and pose classification. The IoU matching threshold is 0.5, the appearance matching threshold is 0.75, and the pose geometric matching threshold is 0.6. For long-term occlusion exceeding 30 frames, the potential trajectory is maintained for linear motion prediction, with a maximum maintenance of 60 frames.

[0028] Multi-object tracking employs an improved DeepSORT framework, which uses a five-step hierarchical re-association strategy to handle occluded scenes: IoU matching, appearance feature similarity matching, pose similarity matching, trajectory prediction matching, and pose classification to solve the problems of identity confusion and tracking loss.

[0029] In step S5, hybrid precision quantization performs INT8 quantization on the standard layers of the backbone network and retains FP16 precision on the sensitive layers; knowledge distillation uses full-precision YOLOv8-m as the teacher network and YOLOv8-s as the student network, employing soft-target distillation; structured pruning performs L1 norm channel pruning on the bottleneck layer of CSPDarknet with a sparsity ratio of 0.4; after comprehensive compression, the model size is reduced from 186.5MB to 12.8MB, while maintaining mAP above 96.2%.

[0030] Model compression techniques include mixed precision quantization, knowledge distillation, and structured pruning. Mixed precision quantization performs INT8 quantization on the standard layers of the backbone network while retaining FP16 precision on sensitive layers. Knowledge distillation uses a high-precision model to guide a low-precision student network. Structured pruning performs channel pruning based on the L1 norm on bottleneck layers.

[0031] In step S6, the degradation strategy of the TAI framework is as follows: when the device temperature T>45°C, the input resolution is reduced from 640×640 to 480×480; when T>50°C, the FP16 fallback path is closed and all layers execute INT8 inference; when T>55°C, the skip-layer mechanism is activated, forward propagation is performed once every other backbone block, and the model is switched to a lightweight model with 6.5M parameters.

[0032] The temperature sensing mechanism includes real-time monitoring of edge device temperature and dynamic adjustment of the inference strategy based on thresholds. When the temperature exceeds the first threshold, the input resolution is reduced; when it exceeds the second threshold, it switches to pure INT8 inference; and when it exceeds the third threshold, a depth-adaptive skip-layer mechanism is activated and a lightweight model is switched to prevent performance degradation caused by high temperatures. Secure data storage includes generating a master key in a secure world, generating a session key by binding a session identifier and a timestamp through a key derivation algorithm, and sealing and storing the master key in a trusted platform module; the session key is rotated periodically or immediately upon detection of tampering events, and the test data is encrypted using the SM4 algorithm and its integrity is verified using a Merkle tree hash chain.

[0033] In step S7, TrustZone's five-stage lifecycle management includes: Phase 1: Within the secure world, a hardware true random number generator (TRNG) is invoked to generate a 128-bit master key; Phase 2: Derive the session key from the master key using the HKDF-SM3 algorithm, binding the session identifier and timestamp during the derivation process; Phase 3: The master key is sealed and stored through the Trusted Platform Module (TPM) and bound to the PCR state of the platform configuration register; Phase 4: Session keys are automatically rotated every 4 hours, or immediately upon detection of tampering. Phase 5: After the session ends, the key buffer is released after being overwritten with cryptographically random data three times.

[0034] The data integrity verification uses a Merkle tree SM3 hash chain, with leaf nodes representing the hash of a single frame detection result and the root hash signature stored along with the assessment data, supporting offline integrity verification.

[0035] The camera used in this method is mounted on the side front of the assessment site, with a field of view covering the area above the horizontal bar and the ground. The frame rate meets the real-time requirements to ensure that the entire pull-up movement can be captured.

[0036] This application further proposes a visual-environment collaborative perception and edge deployment system for pull-up assessment, including a visual-environment collaborative perception and edge deployment method for pull-up assessment, comprising an input layer, a perception layer, an evaluation layer, and an edge intelligence layer connected in sequence: The input layer includes a monocular RGB camera fixed to the front side of the assessment site, with an installation height of 2.0~2.5m, a horizontal distance of 2.5~3.5m from the horizontal bar, a field of view covering the area from 0.5m above the horizontal bar to the ground, and a frame rate of not less than 30FPS; The perception layer includes a multi-scale illumination compensation module, an enhanced YOLOv8 detection network, and a multi-target tracking module. The multi-scale illumination compensation module includes, in sequence, a CLAHE global contrast enhancement unit, an adaptive gamma correction unit, and a shadow detection compensation unit. The enhanced YOLOv8 detection network includes a C2f backbone network with embedded channel attention, a PANet feature fusion neck, a detection head, a pose estimation head, a line constraint head, and a task-aware gradient modulation (TAGM) unit. The multi-target tracking module includes an improved DeepSORT tracker, a ResNet-18 re-identification network, and a five-step hierarchical re-association unit. The evaluation layer includes a limb kinematics analysis module and a pull-up validity judgment module; the validity judgment module uses a finite state machine to sequentially execute three criteria: chin over bar judgment, arm extension judgment, and body swing control. The edge intelligence layer includes a model optimization module, a temperature-sensing adaptive inference (TAI) module, and a security encryption module. The model optimization module includes an INT8 / FP16 hybrid quantization unit, a knowledge distillation unit, and a structured pruning unit. The TAI module includes a temperature monitoring unit, a decision tree scheduling unit, and a multi-level degradation execution unit. The security encryption module includes an SM4-CBC encryption unit, a Merkle tree integrity verification unit, and a TrustZone key management unit.

[0037] It should be noted that the system includes an input layer, a perception layer, an evaluation layer, and an edge intelligence layer connected in sequence; the input layer is a monocular RGB camera; the perception layer includes a multi-scale illumination compensation module, an enhanced YOLOv8 detection network, and a multi-target tracking module; the evaluation layer includes a limb kinematics analysis module and a pull-up validity judgment module; the edge intelligence layer includes a model optimization module, a temperature-sensing adaptive inference module, and a security encryption module, realizing fully automated evaluation from data acquisition to secure storage.

[0038] Example 1: System Hardware Deployment This system will be deployed at an outdoor pull-up testing area in a training base. The area will be equipped with one set of horizontal bars (2.2m high) and a testing area of ​​3m x 4m. The system hardware includes: (1) Hikvision DS-2CD3T46WD-I3 network camera, resolution 2560×1440, frame rate 30FPS, installed 3.0m in front of the site, height 2.2m, downward angle 15°, field of view completely covers the bar and the area below.

[0039] (2) The edge computing device uses Huawei Atlas200IDKA2 (Ascend 310P, 8TOPSINT8, 8GBLPDDR4), which is connected to the camera via gigabit Ethernet port and runs the complete inference pipeline locally.

[0040] (3) The backup power supply is a 500Wh lithium battery pack, which supports the equipment to operate continuously for 48 hours without mains power.

[0041] (4) The equipment enclosure has an IP65 protection rating and an operating temperature range of -30°C to +60°C, meeting the deployment requirements in the field.

[0042] Example 2: Verification of Multi-Scale Illumination Compensation Effect A comparative experiment was conducted under two extreme conditions: a sunny summer noon (light intensity 85,000 lux) and a cloudy dusk (light intensity 1,200 lux). Condition A (no illumination compensation): The original image was directly input into the detection network. The mAP decreased from 97.8% to 89.1% (-8.7%) on sunny days and to 85.3% (-12.5%) on dusk.

[0043] Condition B (CLAHE only): mAP recovers to 94.5% on sunny days and 91.2% at dusk, but there are still a large number of missed detections in the shaded areas.

[0044] Condition C (Three-stage MSIC of this invention): mAP reached 97.2% on sunny days and 96.8% at dusk, with a difference of <1% from the indoor benchmark (97.8%). The recall rate for detecting shadowed areas improved from 71.3% to 95.6%.

[0045] Example 3: TAGM Multi-Task Training Effect Comparison of equal-weight joint training and TAGM dynamic modulation training on the PED dataset: Equal-weight training: mAP 98.1% for detection task, RMSE 0.42cm for pose estimation task, and accuracy 87.5% for line constraint task. The gradient norm ||g_det|| for detection task is 3.2 times that of pose task, leading to insufficient convergence in pose and line constraint tasks.

[0046] TAGM training (λ=0.3, β=0.9): mAP for detection task 97.8% (-0.3%), RMSE for pose estimation task 0.30cm (improvement of 28.6%), and accuracy for line constraint task 94.1% (improvement of 6.6%). The gradient norm ratios for the three tasks stabilized in the range of 1.0–1.5, achieving balanced convergence.

[0047] Example 4: Temperature Adaptive Inference Verification Full-temperature range testing was conducted in a high and low temperature test chamber, with a 30FPS video stream input, and the FPS and power consumption were recorded. -30°C: Initial startup power is 3.2W, stabilizes at 3.5W after 5 minutes, 30FPS, battery life is 52 hours.

[0048] +25°C (room temperature): 3.5W, 30FPS, 48 hours of battery life.

[0049] +45°C: TAI starts with a level 1 downgrade, reducing the resolution to 480×480, 3.8W, 25FPS, and battery life to 44 hours.

[0050] +55°C: TAI initiates level 3 downgrade, switches to lightweight model, 2.1W, 15FPS, accuracy loss of 1.5%, battery life of 80 hours.

[0051] +60°C (for 2 hours): The system did not crash or lose data, and the assessment data was completely saved and passed the SM3 integrity check.

[0052] Example 5: End-to-End Secure Encryption Verification Commission a third-party security assessment agency to conduct the graded protection assessment: (1) Key security: The master key resides in TrustZone security world throughout the process, and the plaintext key was not extracted during memory forensics; the session key rotation interval is 4 hours, and the time complexity of brute-force attack is 2^128.

[0053] (2) Data integrity: 1,000 assessment records were tested for tampering. The Merkle root hash check detection rate was 100%, and the tampering was located at the frame level.

[0054] (3) Cryptographic compliance: The SM4 implementation has passed the State Cryptography Administration's commercial cryptographic product certification (all GMSSL test vectors have passed) and meets the GMB / T0002-2012 specification.

[0055] (4) Offline availability: After running offline for 72 hours, all assessment functions were normal, data encryption and storage were normal, and data export was 100% complete after reconnecting to the network.

[0056] Example 1: Feasibility and Environmental Adaptability of System Hardware Deployment Proof of core functionality: This system has high availability in harsh outdoor environments.

[0057] Summary of evidence: By configuring high-resolution cameras, edge computing boxes, and high-capacity backup power supplies, the system's long-term battery life (48 hours+) in environments without mains power was verified; the IP65 protection rating and wide temperature range (-30°C to +60°C) demonstrate that it meets the physical deployment standards of field military or training bases.

[0058] Example 2: Proof of the effectiveness of the multi-scale illumination compensation module The core proof is that the three-stage MSIC algorithm proposed in this invention significantly outperforms traditional methods under extreme lighting conditions.

[0059] Summary of evidence: Comparative experiments show that without compensation and using only CLAHE, the mAP decreased by 8.7% and 12.5% ​​in sunny and twilight conditions, respectively, with severe missed detections in shadow areas. However, after applying this invention, even under conditions of huge differences in light intensity (85000 lux vs 1200 lux), the mAP remained stable at around 97%, and the shadow recall rate increased significantly from 71.3% to 95.6%, demonstrating the decisive improvement effect of this module on visual perception quality.

[0060] Example 3: Proof of the superiority of the TAGM multi-task dynamic modulation strategy The core of the proof is that it solves the problem of unbalanced convergence caused by gradient conflicts in multi-task learning.

[0061] Summary of evidence: Compared to the insufficient convergence of pose and line constraint tasks caused by equal-weight training (RMSE 0.42cm), the TAGM strategy improves pose estimation accuracy by 28.6% (RMSE reduced to 0.30cm) and line constraint accuracy by 6.6% by controlling the gradient norm ratio between 1.0 and 1.5, while only slightly reducing detection accuracy by 0.3%, demonstrating its effectiveness in achieving balanced and efficient convergence across multiple tasks.

[0062] Example 4: Proof of the robustness and energy-saving effect of the temperature-adaptive inference mechanism Proof of core functionality: The TAI mechanism can effectively cope with high-temperature challenges, ensuring that the equipment does not shut down under high temperatures and extends its battery life.

[0063] Summary of evidence: Tests show that the system automatically triggers three-level degradation (reduced resolution and switching to lightweight model) at +55°C, reducing power consumption from 3.5W to 2.1W and significantly extending battery life from 48 hours to 80 hours. Furthermore, it did not crash during continuous operation at +60°C, demonstrating the mechanism's intelligent scheduling capability that balances performance, security, and energy efficiency on resource-constrained edge devices.

[0064] Example 5: Security Compliance Proof of an End-to-End Secure Encryption System Proof of core functionality: The system has passed national graded protection and commercial cryptography security assessments.

[0065] Summary of evidence: Third-party evaluation results show that the master key resides in the TrustZone security environment and cannot be extracted; the data tampering detection rate reaches 100% and can be located down to the frame level; the SM4 / SM3 algorithm complies with the certification specifications of the State Cryptography Administration; and it maintains functional integrity and data consistency during 72 hours of offline operation without network access, comprehensively proving the system's data security, integrity, and anti-attack capabilities.

[0066] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0067] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A visual-environment collaborative perception and edge deployment method for pull-up assessment, characterized in that, Includes the following steps: Step S1: Acquire video stream using a monocular RGB camera fixed to the side front of the assessment site, and perform multi-scale illumination compensation on each frame; the multi-scale illumination compensation includes three stages: Phase 1: Convert the image from RGB space to HSV space, extract the V channel luminance map, and use contrast-limited adaptive histogram equalization (CLAHE) to process the V channel. Set the block grid size to 8×8, the clipping threshold to clipLimit=2.0, and output the enhanced luminance map V'_CLAHE(x,y). Phase 2: Perform adaptive gamma correction on V'_CLAHE(x, y), the correction formula is as follows: , where μ_L(x, y) is the average brightness value within the local window of the pixel, and α is the adaptive coefficient; Phase 3: Construct the shadow mask S(p), the determination formula is as follows Where L(p) is the luminance component of pixel p, τ(p) is the local adaptive threshold, B_c(p) is the average chromaticity of the neighboring non-shaded pixels, and ε_c is the chromaticity consistency threshold; gamma enhancement is performed on the shaded pixels that satisfy S(p)=1 to restore details; Step S2: Input the illumination-compensated image into the enhanced YOLOv8 detection network for target detection and pose estimation; the enhanced YOLOv8 detection network is improved on the basis of standard YOLOv8 by: embedding a channel attention mechanism in the C2f module to realize feature recalibration; adding a line constraint prediction head to predict the geometric relationship between human key points and the horizontal bar reference line; and using the task-aware gradient modulation (TAGM) mechanism to dynamically balance the training gradient weights of the three task heads for detection, pose estimation and line constraint. Step S3: Perform kinematic analysis based on the limb key points in the detection results, and execute the pull-up validity judgment through a finite state machine (FSM); the judgment includes three criteria: Rule 1: Calculate the vertical distance d_chin between the key point of the nose tip and the single-line reference line. When d_chin is greater than the preset first threshold, it is determined that the mandible has crossed the line. Rule 2: Calculate the elbow angle θ_elbow from key points of the shoulder, elbow and wrist. When θ_elbow is greater than or equal to the preset second threshold, the arm is considered to be straight. Criterion 3: Calculate the horizontal displacement σ_hip of the hip center relative to the initial static position. When σ_hip is less than the preset third threshold, it is determined that there is no excessive oscillation. The finite state machine sequentially goes through the states of Idle, Ascent, Hold, and Descent. Only actions that complete the complete state transition sequence are counted as valid counts. Step S4: The improved DeepSORT multi-target tracking framework is used to continuously track the detected target. The occlusion scene is handled by a five-step hierarchical re-association strategy, and a lightweight pose classifier is used to distinguish between examinees and protection personnel. Only the examinee category is counted. Step S5: Deploy the optimized detection model to the NPU of the edge computing device, and perform a three-stage progressive compression process of mixed precision quantization, knowledge distillation, and structured pruning in sequence; Step S6: Monitor the device temperature in real time through the temperature-sensing adaptive inference (TAI) framework, and dynamically adjust the inference strategy according to the temperature threshold, including reducing the input resolution, switching to pure INT8 inference, or activating the depth-adaptive skipping mechanism. Step S7: Perform end-to-end encrypted storage on all assessment data. The encryption key is managed through a five-stage lifecycle using TrustZone, and the data integrity is verified using a Merkle tree hash chain.

2. The visual-environment collaborative perception and edge deployment method for pull-up assessment according to claim 1, characterized in that: In step S1, the formula for calculating the local adaptive threshold τ(p) is τ(p)=μ_L,local×(1-β·σ_L,local), where μ_L,local and σ_L,local are the mean and standard deviation of the brightness within the local window, respectively, and β=1.5; the gamma enhancement parameter γ_shadow used for shadow compensation is 0.

6.

3. The visual-environment collaborative perception and edge deployment method for pull-up assessment according to claim 1, characterized in that: In step S2, the loss function of the line constraint head is defined as follows: ,in Key point The perpendicular distance to the single-bar reference line l_ref N represents the distance predicted by the network, and N is the number of visible keypoints.

4. The visual-environment collaborative perception and edge deployment method for pull-up assessment according to claim 1, characterized in that: In step S3, the first threshold δ_chin = 0.5cm is derived from the average distance from the tip of the nose to the chin of an adult Asian male after head tilt compensation and detection error correction; the second threshold is 160°, allowing natural overextension. The third threshold δ_swing = 3.0 cm is determined by multiplying the 95th percentile value of natural human body sway by a safety factor of 1.

25.

5. The visual-environment collaborative perception and edge deployment method for pull-up assessment according to claim 1, characterized in that: In step S4, the five-step hierarchical reassociation strategy consists of: IoU matching, appearance feature cosine similarity matching, pose PCK similarity matching, Kalman trajectory prediction matching, and pose classification; wherein the IoU matching threshold is 0.5, the appearance matching threshold is 0.75, and the pose geometry matching threshold is 0.6; for long-term occlusion exceeding 30 frames, the potential trajectory is maintained for linear motion prediction, with a maximum maintenance of 60 frames.

6. The visual-environment collaborative perception and edge deployment method for pull-up assessment according to claim 1, characterized in that: In step S5, hybrid precision quantization performs INT8 quantization on the standard layers of the backbone network and retains FP16 precision on the sensitive layers; knowledge distillation uses full-precision YOLOv8-m as the teacher network and YOLOv8-s as the student network, employing soft-target distillation; structured pruning performs L1 norm channel pruning on the bottleneck layer of CSPDarknet with a sparsity ratio of 0.4; after comprehensive compression, the model size is reduced from 186.5MB to 12.8MB, while maintaining mAP above 96.2%.

7. The visual-environment collaborative perception and edge deployment method for pull-up assessment according to claim 1, characterized in that: In step S6, the degradation strategy of the TAI framework is as follows: when the device temperature T>45°C, the input resolution is reduced from 640×640 to 480×480; when T>50°C, the FP16 fallback path is closed and all layers execute INT8 inference; when T>55°C, the skip-layer mechanism is activated, forward propagation is performed once every other backbone block, and the model is switched to a lightweight model with 6.5M parameters.

8. The visual-environment collaborative perception and edge deployment method for pull-up assessment according to claim 1, characterized in that: In step S7, TrustZone's five-stage lifecycle management includes: Phase 1: Within the secure world, a hardware true random number generator (TRNG) is invoked to generate a 128-bit master key; Phase 2: Derive the session key from the master key using the HKDF-SM3 algorithm, binding the session identifier and timestamp during the derivation process; Phase 3: The master key is sealed and stored through the Trusted Platform Module (TPM) and bound to the PCR state of the platform configuration register; Phase 4: Session keys are automatically rotated every 4 hours, or immediately upon detection of tampering. Phase 5: After the session ends, the key buffer is released after being overwritten with cryptographically random data three times.

9. The visual-environment collaborative perception and edge deployment method for pull-up assessment according to claim 1, characterized in that: The data integrity verification adopts a Merkle tree SM3 hash chain, with leaf nodes being the hashes of single-frame detection results, and the root hash signature being stored together with the assessment data, supporting offline integrity verification.

10. A visual-environment collaborative perception and edge deployment system for pull-up assessment, comprising the visual-environment collaborative perception and edge deployment method for pull-up assessment as described in any one of claims 1-9, characterized in that: It includes the input layer, perception layer, evaluation layer, and edge intelligence layer connected in sequence: The input layer includes a monocular RGB camera fixed to the front side of the assessment site, with an installation height of 2.0~2.5m, a horizontal distance of 2.5~3.5m from the horizontal bar, a field of view covering the area from 0.5m above the horizontal bar to the ground, and a frame rate of not less than 30FPS; The perception layer includes a multi-scale illumination compensation module, an enhanced YOLOv8 detection network, and a multi-target tracking module. The multi-scale illumination compensation module sequentially includes a CLAHE global contrast enhancement unit, an adaptive gamma correction unit, and a shadow detection compensation unit. The enhanced YOLOv8 detection network includes a C2f backbone network with embedded channel attention, a PANet feature fusion neck, a detection head, a pose estimation head, a line constraint head, and a task-aware gradient modulation (TAGM) unit. The multi-target tracking module includes an improved DeepSORT tracker, a ResNet-18 re-identification network, and a five-step hierarchical re-association unit. The evaluation layer includes a limb kinematics analysis module and a pull-up validity judgment module; the validity judgment module uses a finite state machine to sequentially execute three criteria: chin over bar judgment, arm extension judgment, and body swing control. The edge intelligence layer includes a model optimization module, a temperature-sensing adaptive inference (TAI) module, and a security encryption module. The model optimization module includes an INT8 / FP16 hybrid quantization unit, a knowledge distillation unit, and a structured pruning unit. The TAI module includes a temperature monitoring unit, a decision tree scheduling unit, and a multi-level degradation execution unit. The security encryption module includes an SM4-CBC encryption unit, a Merkle tree integrity verification unit, and a TrustZone key management unit.

Images