A method, system, and electronic device for human posture detection in buried environments.

Abstract

This invention relates to the field of intelligent recognition technology, specifically to a method, system, and electronic device for human posture detection in buried scenarios. The method includes: collaboratively acquiring video frames of the buried scene using a low-light image acquisition device and an adaptively brightening lighting device; scaling the video frames to a standard size and adaptively brightening them to generate a pre-processed image; inputting this image into a student model trained using a multi-teacher collaborative distillation framework, and post-processing to optimize and output the coordinates of human key points, the coordinates of the detection box, and their respective confidence scores; extracting three types of geometric feature vectors—joint angles, bone length ratios, and spatial distribution of key points—and inputting them into a classification network to obtain the posture detection result. This invention effectively overcomes interference from low-light and occlusion scenarios, accurately identifies some buried postures, meets the requirements for portable deployment and real-time response, provides efficient data support for post-disaster relief, improves rescue efficiency, and reduces the risk of misjudgment and missed detection.

Description

Technical Field

[0001] This invention relates to the field of intelligent recognition technology, and in particular to a method, system and electronic device for human posture detection in buried scenarios. Background Technology

[0002] Current earthquake rescue equipment only features cameras in terms of visual technology, lacking posture recognition algorithms to identify specific human postures. This traditional rescue equipment relies solely on images transmitted from the camera, demanding rescuers visually assess the condition of trapped individuals. This approach is not only inefficient but also prone to misjudgments and missed detections due to the complexity of the environment. In contrast, while existing human posture recognition technology is maturely applied in fields such as medical rehabilitation, human-computer interaction, and abnormal behavior detection, it has not yet been effectively integrated into the rescue equipment and technology systems for post-earthquake burial scenarios. This application gap exists, and even if directly applied, significant drawbacks exist due to the specific nature of the scenario, as detailed below:

[0003] Existing technologies are mostly designed for conventional scenes with ample lighting and clean backgrounds. However, post-earthquake burial scenarios typically present challenges such as low light levels, dust, and cluttered backgrounds. This makes it difficult for existing pose recognition algorithms' image preprocessing modules to effectively improve image quality, leading to missed or false detections in key point detection and a significant drop in pose classification accuracy. Existing applications such as abnormal behavior detection can only recognize the conventional poses of the entire human body. However, in post-earthquake burial scenarios, trapped individuals are often partially buried. Existing algorithms have not built recognition models for the special poses of partially buried individuals, making it impossible to accurately determine their pose status and meet rescue and positioning needs. Post-earthquake sites suffer from damaged infrastructure and a lack of stable power and network support. Existing pose recognition systems often rely on high-performance servers or fixed monitoring equipment, making lightweight and portable deployment difficult. Furthermore, existing algorithms are not optimized for embedded platforms. Even if deployed, they cannot achieve the real-time detection frame rate required for rescue operations, failing to provide timely support for emergency rescue. Summary of the Invention

[0004] The purpose of this invention is to provide a method, system, and electronic device for human posture detection in buried scenarios, to solve the application gap of existing human posture recognition algorithms in rescue scenarios, as well as the problems of insufficient adaptability of such algorithms in such application scenarios, lack of partial buried posture recognition capabilities, and mismatch between portable deployment and real-time performance.

[0005] The technical solution of this invention is: a human posture detection method for buried scenarios, comprising:

[0006] Video frames of the burial scene were captured using low-light image acquisition equipment and adaptively brightness-adjusting lighting equipment.

[0007] The video frame images are reduced to a standard size, and the image brightness is adaptively adjusted to obtain a preprocessed image;

[0008] The preprocessed image is input into the trained student model. After post-processing optimization, the coordinates of human key points, the coordinates of the detection box, and their respective confidence scores are output. The trained student model is obtained by training a multi-teacher collaborative distillation framework. The multi-teacher collaborative distillation framework is based on teacher model groups and a multi-dimensional fusion loss function. The student model parameters are updated through iterative distillation training.

[0009] Geometric feature vectors are extracted based on the coordinates of human key points, and then input into a classification network to obtain posture detection results. The geometric feature vectors include joint angle features, bone length ratio features, and key point spatial distribution features.

[0010] Preferably, the teacher model group includes a main teacher model and at least one auxiliary teacher model. The main teacher model outputs hard labels, soft labels, and intermediate layer features, while the auxiliary teacher model outputs soft labels and intermediate layer features. The hard labels are the coordinates of human key points, the coordinates of the detection box, and their respective confidence scores. The soft labels are the KL divergence of the pose classification probability distribution. The intermediate layer features are the feature maps output by the intermediate layer of the model.

[0011] Preferably, the master teacher model is an improved YOLOv8-Pose model, which includes a Backbone layer, a Neck layer, and a Head layer; wherein the C2f module of the Backbone layer contains a CBAM attention module, and the PAN module of the Neck layer is equipped with an adaptive weight fusion layer.

[0012] Preferably, the adaptive weight fusion layer has built-in scene recognition logic, which determines the current scene state by analyzing the brightness value of the preprocessed image and the occlusion ratio of human key points, thereby dynamically adjusting the multi-scale feature fusion weights in the PAN module.

[0013] Preferably, the multi-dimensional fusion loss function includes hard label loss, soft label loss, and feature distillation loss;

[0014] The hard label loss is the confidence-weighted mean square error between the human keypoint coordinates and the detection box width and height output by the student model and the master teacher model. It is calculated by weighting and summing each error term using the confidence scores of the human keypoints output by the master teacher model, and then dividing by the number of human keypoints. The formula for calculating the hard label loss is as follows:

[0015] ,

[0016] in, For hard label loss; N is the number of human keypoints; The confidence score of the human body key points in the main teacher model; x and y are the coordinates of the human body key points; w and h are the width and height of the detection box; the superscript s represents the student model and t represents the main teacher model;

[0017] The soft label loss is the mean KL divergence of the pose classification probability distributions of the student model and the main teacher model, and the student model and the auxiliary teacher model; the formula for calculating the soft label loss is:

[0018] ;

[0019] in, For soft label loss; K represents the total number of main teacher models and auxiliary teacher models; C refers to the number of pose types; This represents the probability that the student model classifies the c-th pose. This represents the probability that the k-th teacher model classifies the c-th pose;

[0020] The feature distillation loss is the cosine similarity loss of the feature maps output by the intermediate layers of the student model and the main teacher model; the formula for calculating the feature distillation loss is:

[0021] ,

[0022] in, The characteristic distillation loss is M; M refers to the number of characteristic layers. Here is the feature map of the m-th layer of the student model. The feature map of the m-th layer of the main teacher model.

[0023] Preferably, the method for extracting geometric feature vectors based on the coordinates of the human body key points is as follows: extracting joint angle features, bone length ratio features and key point spatial distribution features, and splicing them together to form a geometric feature vector;

[0024] The joint angle features include the absolute angles of key human joints to the vertical or horizontal direction and the relative angles between adjacent joints; the bone length ratio features include the lengths and ratios of human skeletal segments; and the key point spatial distribution features are used to reflect the spatial distribution features of the overall human posture contour.

[0025] Preferably, the post-processing optimization includes confidence filtering and key point smoothing;

[0026] The confidence filtering involves removing key human body points with a confidence level lower than the adaptive confidence threshold.

[0027] The keypoint smoothing mentioned above uses Kalman filtering to correct the abrupt changes in human body keypoints between video frames.

[0028] On the other hand, this application also discloses a human posture detection system for burial scenarios, including:

[0029] The data acquisition module is used to acquire video frames of the burial scene using low-light image acquisition equipment and adaptive brightness adjustment lighting equipment;

[0030] The data preprocessing module is used to reduce the image size of video frames to a standard size and adaptively adjust the image brightness to obtain a preprocessed image;

[0031] The key point detection module is used to input the preprocessed image into the trained student model, and after post-processing optimization, outputs the coordinates of human key points, the coordinates of the detection box, and their respective confidence scores. The trained student model is obtained by training through a multi-teacher collaborative distillation framework, which is based on a teacher model group and a multi-dimensional fusion loss function, and updates the student model parameters through iterative distillation training.

[0032] The pose classification module is used to extract geometric feature vectors based on the coordinates of human key points, and input the geometric feature vectors into the classification network to obtain the pose detection results; the geometric feature vectors include joint angle features, bone length ratio features and key point spatial distribution features.

[0033] The visualization module is used to intuitively present human posture detection results, display multi-dimensional information, and support interactive operations.

[0034] Preferably, the human posture detection system for buried scenarios is deployed on an embedded terminal;

[0035] The embedded terminal includes a core computing unit, a data acquisition unit, a lighting control unit, and an extended sensing unit, wherein the core computing unit is communicatively connected to the data acquisition unit, the lighting control unit, and the extended sensing unit.

[0036] The core computing unit is a processor that supports multi-task parallel scheduling and mixed-precision computing, and has the ability to cache data, infer algorithms, and coordinate the control of each unit.

[0037] The data acquisition unit uses a low-light camera as the image acquisition device and reserves a dual-modal acquisition expansion interface for connecting an external infrared camera to adapt to extreme scenarios.

[0038] The lighting control unit includes at least one high color rendering supplementary light device as a lighting device, and achieves adaptive start / stop and supplementary light intensity adjustment through the light sensing feedback control of the core computing unit.

[0039] The extended sensing unit supports the access of multiple types of sensors and collects environmental or personnel data in the buried scene according to the type of sensor accessed.

[0040] On the other hand, this application further discloses an electronic device, including:

[0041] One or more processors;

[0042] Memory, used to store one or more programs;

[0043] When the one or more programs are executed by the one or more processors, the one or more processors implement a human posture detection method for buried scenarios as described in any one of the above.

[0044] Compared with the prior art, the advantages of the present invention are:

[0045] (1) This invention addresses the core deficiency of existing pose recognition technologies in low-light scenarios through multi-dimensional collaborative design. In the image acquisition and preprocessing process, the collaborative work of lighting equipment and image acquisition equipment, combined with an adaptive brightness adjustment method, effectively reduces the interference of low light and uneven illumination on image quality. In the key point detection stage, the main teacher model adopts an improved YOLOv8-Pose architecture. By embedding an attention module, it enhances the feature differentiation between the human body area and backgrounds such as ruins and dust. At the same time, it uses an adaptive weight fusion layer to dynamically analyze the image brightness and key point occlusion ratio, flexibly adjusts the multi-scale feature fusion weights, and enables the model to accurately capture key points in low-light scenarios. In occluded scenarios, it relies on high-level semantic features to indirectly infer missing key points, significantly reducing the false detection and missed detection rates, and ensuring the detection stability in complex environments.

[0046] (2) This invention adopts a multi-teacher collaborative distillation framework. Through collaborative training of the main teacher model and the auxiliary teacher model, combined with a multi-dimensional fusion design of hard label loss, soft label loss and feature distillation loss, it not only transfers the high-precision features of the main teacher model to the student model, but also significantly reduces the number of parameters and computational load of the student model through the lightweight feature adaptation of the auxiliary teacher model. At the same time, the adaptive confidence filtering and Kalman filtering smoothing in the post-processing optimization stage further improve the continuity and reliability of video stream detection. This design enables the model to meet the lightweight deployment requirements of embedded terminals, adapt to mobile rescue scenarios without stable power supply and network support after disasters, and maintain high detection accuracy. The overall system processing latency can match the real-time response requirements of the rescue site, solving the contradiction between accuracy and speed and the problem of insufficient deployment flexibility in the prior art.

[0047] (3) This invention constructs feature vectors by extracting three types of geometric features: joint angles, bone length ratios, and key point spatial distribution. Among them, joint angle features capture local posture details, bone length ratio features eliminate the influence of differences in human body scale, and key point spatial distribution features reflect the overall posture contour. These three types of feature vectors enable the classification network to accurately identify postures such as curled-up and side-lying when partially buried. Combined with confidence filtering and Kalman filtering in the post-processing stage, it effectively corrects key point jumps between frames and interference from low-quality key points, significantly reducing the misjudgment rate of posture classification. In addition, the system integrates a visualization module and extended sensor interfaces, which can intuitively present key information such as skeletal framework, posture labels, and number of people. At the same time, it supports the access of environmental status sensors and life detection sensors, realizing the collaborative output of posture data and multi-dimensional information of the rescue scene. It provides comprehensive data support for rescuers to quickly locate trapped people, judge the degree of entrapment, and formulate targeted rescue plans. Compared with the traditional rescue mode that relies on manual observation, it greatly improves rescue efficiency and reduces the risk of misjudgment and omission caused by visual fatigue and environmental interference. Attached Figure Description

[0048] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0049] Figure 1 This is a flowchart of a human posture detection method for buried scenarios according to the present invention;

[0050] Figure 2 This is a structural diagram of the teacher model group and student model described in this invention;

[0051] Figure 3 This is a schematic diagram of the multi-teacher collaborative distillation framework described in this invention;

[0052] Figure 4 This is a structural diagram of the classification network described in this invention;

[0053] Figure 5 This is a structural diagram of a human posture detection system for buried environments as described in this invention. Detailed Implementation

[0054] The present invention will be further described in detail below with reference to specific embodiments:

[0055] The core application of this invention is in the rescue site after disasters such as earthquakes and landslides. It is adapted to the non-contact posture detection scenario of rescuers in the ruins area. This scenario is generally characterized by complex conditions such as low light, limb obstruction, cluttered background, and lack of stable power supply and network support. The core requirement of rescue is to quickly and accurately obtain the posture and number information of trapped people, so as to provide data support for the formulation of safe and efficient rescue plans.

[0056] This invention uses a multi-teacher collaborative distillation framework to train a lightweight student model as its core, extracting three complementary geometric feature vectors: joint angles, bone length ratios, and key point spatial distribution. It also provides customized adaptation for burial scenarios across the entire process of data acquisition, preprocessing, detection, and classification. This addresses the core issues in existing technologies, such as insufficient adaptability to burial scenarios, inaccurate identification of specific burial postures, and difficulties in lightweight model deployment and real-time response. The invention achieves real-time and accurate detection of human postures in burial scenarios, effective identification of specific burial postures, and portable deployment on embedded terminals. This provides rescuers with intuitive and reliable information on the postures and numbers of trapped individuals, supporting the efficient development of rescue plans.

[0057] like Figure 1 As shown, a human posture detection method for buried scenarios includes:

[0058] S1. Acquire video frames of the burial scene using low-light image acquisition equipment and adaptively adjustable lighting equipment.

[0059] Specifically, low-light image acquisition equipment refers to devices used to capture clear images in low-light environments. They are mainly used in places requiring nighttime or low-light monitoring, such as banks, schools, military bases, and river monitoring. Their performance is measured in lux (Lux) and can be categorized into standard levels such as low-light (0.1 Lux), moonlight (0.01 Lux), and starlight (0.001 Lux and below).

[0060] An adaptive brightness adjustment lighting device refers to an auxiliary imaging device that dynamically adjusts its working state and supplementary light power based on the ambient light intensity. It is designed to adapt to the complex conditions of low illumination, uneven lighting, and instantaneous light changes in buried scenes, providing precise supplementary lighting support for low-illuminance image acquisition equipment.

[0061] S2. Reduce the image size of the video frame to a standard size and adaptively adjust the image brightness to obtain a preprocessed image.

[0062] Specifically, video frames have high image resolution. Directly using the original images as input to the model for inference would lead to a surge in computation and prolonged inference time, failing to meet the real-time response requirements of rescue scenarios. Furthermore, redundant background information in high-resolution images can interfere with human feature extraction and reduce the accuracy of key point detection. Therefore, it is necessary to first downscale and standardize the video frames to support the rapid inference of the subsequent lightweight model, while also adapting to the input specifications of the model in the multi-teacher collaborative distillation framework.

[0063] In addition, although the image in step S1 was acquired after supplementing the lighting with lighting equipment, the low illumination, uneven lighting, and dust-induced fogging problems that are common in burial scenes still need to be further optimized for image quality through adaptive brightness adjustment.

[0064] In one implementation, the video frame image size is 1920×1080, and it is scaled to 640×640 using a bilinear interpolation algorithm. In other implementations, bicubic interpolation or nearest neighbor interpolation algorithms can also be used to scale the image. After obtaining the standard-sized image, a Gamma correction strategy is used to process the standard-sized image. The γ parameter is dynamically adjusted by analyzing the average brightness value of the standard-sized image, and the value of the γ parameter ranges from 0.2 to 0.8.

[0065] S3. Input the preprocessed image into the trained student model, and output the coordinates of human key points, the coordinates of the detection box, and their respective confidence scores after post-processing optimization. The trained student model is obtained by training through a multi-teacher collaborative distillation framework. The multi-teacher collaborative distillation framework is based on teacher model groups and a multi-dimensional fusion loss function, and updates the student model parameters through iterative distillation training.

[0066] Specifically, the student model refers to the target model trained by the multi-teacher collaborative distillation framework. It is designed to meet the deployment requirements of embedded terminals in buried scenarios, reducing the number of parameters and computation to meet real-time response while ensuring the detection effect of human key points in low-light and occluded environments.

[0067] The multi-teacher collaborative distillation framework is a model optimization architecture that guides the training of lightweight models using high-precision models. Its core logic is to leverage the high-quality output of the teacher model group and guide the student model to learn effective features through a multi-dimensional loss function, thus avoiding accuracy loss due to lightweighting. The multi-dimensional loss function is a composite loss function that integrates multiple loss types, used to constrain the training process of the student model from different dimensions. This ensures that the output of the student model is consistent with the guidance information of the teacher model group, balancing the accuracy of detection results with the generalization ability of the model.

[0068] Iterative distillation training refers to a training method that updates the parameters of the student model in stages. Through multiple iterations, the student model gradually learns the high-quality features of the teacher model group, continuously optimizing the detection performance.

[0069] In one implementation, a teacher model group is composed of a model with high pose detection accuracy and a lightweight model with few parameters and fast inference speed, and pre-trained for the human pose detection task. A multi-dimensional fusion loss function is set, the parameters of the teacher model group are fixed, and the human pose dataset is input into the teacher model group and the student model respectively to obtain the output results of the teacher model group and the student model. The loss value is calculated by the multi-dimensional fusion loss function, and the parameters of the student model are updated by the backpropagation algorithm. The above process is repeated for multiple rounds of iterative training until the loss value converges to a preset threshold, thus completing the training of the student model.

[0070] After the student model is trained, the preprocessed images obtained in step S2 are input into the trained student model in batches. The student model outputs the coordinates of human key points, the coordinates of the detection box, and their respective confidence scores. In addition, the output of the student model needs to be post-processed and optimized. Through noise filtering and coordinate correction methods, invalid key point information caused by scene interference is removed, and small deviations in the coordinates of the detection box are corrected. This makes the output coordinates of human key points and the detection box information more consistent with the actual human target, improves the reliability of the data, and lays the foundation for subsequent geometric feature vector extraction.

[0071] S4. Extract geometric feature vectors based on the coordinates of human key points, input the geometric feature vectors into the classification network, and obtain the pose detection results; among them, the geometric feature vectors include joint angle features, bone length ratio features, and key point spatial distribution features.

[0072] Specifically, geometric feature vectors are data features that integrate joint angle features, bone length ratio features, and key point spatial distribution features. Their core function is to transform discrete human body key point coordinates into structured features that can accurately represent human posture. In cases where some human body key points may be missing due to limb burial or have special postures in buried scenarios, they provide highly discriminative input data for classification networks.

[0073] In one implementation, three types of geometric features are extracted based on the coordinates of human key points. The extracted joint angle features, bone length ratio features, and key point spatial distribution features are then concatenated to form a geometric feature vector with fixed dimensions. The geometric feature vector is then normalized to avoid the numerical scale of a certain type of feature being too large and affecting the model training and inference effect.

[0074] The normalized geometric feature vector is input into the classification network. In this embodiment, the classification network adopts an architecture combining fully connected layers and convolutional layers. Through feature mapping, activation function processing, and classification output, the input geometric feature vector is used to determine the pose category. The classification network outputs the probability distribution of various poses through the Softmax activation function, and selects the pose category with the highest probability that exceeds the preset confidence threshold as the final detection result. The number of categories in the output layer is adapted to the rescue needs of the burial scenario, covering the common pose types of trapped personnel.

[0075] In summary, this invention achieves multiple core technological effects through a collaborative design across the entire process of data acquisition, preprocessing, student model detection, and classification network recognition. Firstly, by employing low-light acquisition and adaptive supplementary lighting and brightness adjustment, it overcomes the imaging challenges of low light and dust interference in buried scenarios, laying a high-quality image foundation for subsequent detection. Secondly, relying on a multi-teacher collaborative distillation framework and standardized image size, it balances detection accuracy and computational efficiency within a lightweight student model, adapting to portable deployment on embedded terminals and meeting the needs of real-time rescue response. Thirdly, through the construction of three types of geometric feature vectors and classification network inference, it accurately identifies various trapped postures, including some special postures of those buried, overcoming the limitations of conventional algorithms. Finally, it achieves human posture detection in buried scenarios, significantly reducing the risk of human error and omission, improving investigation efficiency and the targeting of rescue plans, and providing core support for the scientific and safe nature of post-disaster rescue.

[0076] Based on the aforementioned human pose detection method for buried scenarios, to further elaborate on the scheme of this application, this application further describes that the teacher model group includes a main teacher model and at least one auxiliary teacher model. The main teacher model outputs hard labels, soft labels, and intermediate layer features, while the auxiliary teacher model outputs soft labels and intermediate layer features. Among them, the hard labels are the coordinates of human key points, the coordinates of the detection box, and their respective confidence scores, and the soft labels are the KL divergence of the pose classification probability distribution. The intermediate layer features are the feature maps output by the intermediate layer of the model.

[0077] Specifically, the main teacher model is a model with high-precision posture detection capabilities, responsible for providing accurate positioning information and deep features; the auxiliary teacher model focuses on generalization and lightweight characteristics, used to supplement the main teacher model's scene adaptation capabilities. The two together form a teacher model group, establishing a multi-dimensional guidance system to avoid the guidance bias of a single model.

[0078] like Figure 2 As shown, the master teacher model is an improved YOLOv8-Pose model, which includes a Backbone layer, a Neck layer, and a Head layer. The C2f module of the Backbone layer contains a CBAM attention module, and the PAN module of the Neck layer has an adaptive weight fusion layer.

[0079] The adaptive weight fusion layer incorporates scene recognition logic, which determines the current scene state by analyzing the brightness value and the occlusion ratio of human key points in the preprocessed image, thereby dynamically adjusting the multi-scale feature fusion weights in the PAN module.

[0080] Specifically, the YOLOv8-Pose model is a network model designed specifically for pose estimation. Its overall architecture achieves end-to-end processing from image input to keypoint and pose information output through the coordinated operation of the Backbone layer, Neck layer, and Head layer. The core roles of each layer are clearly defined: the Backbone layer, as the backbone network layer, is responsible for extracting multi-scale features from the input image layer by layer; the Neck layer, as the feature fusion layer, compensates for the information gaps of different levels of features by splicing and fusing multi-scale features; and the Head layer, as the output head, completes the prediction output of keypoint coordinates, detection boxes, and pose classification results.

[0081] The improvement of the YOLOv8-Pose model in this invention is mainly aimed at addressing the problems of inaccurate feature extraction and poor scene adaptability caused by the complex environment of disaster sites, by adding targeted optimization structures to the key layers.

[0082] In one implementation, the teacher group model includes a main teacher model and two auxiliary teacher models. The main teacher model adopts the YOLOv8-Pose model, fully utilizing its three-stage core structure of Backbone, Neck, and Head layers to ensure high-precision keypoint detection capabilities. The Backbone layer consists of four C2f modules and one SPPF module. Through layer-by-layer downsampling and feature aggregation, it achieves multi-scale feature extraction from local human keypoints to the global contour, providing rich feature support for subsequent detection. The Neck layer uses a PAN feature pyramid attention network, fusing low-level spatial detail features and high-level semantic features through upsampling, downsampling, and cross-layer feature concatenation to solve the adaptation problem of human keypoint detection at different scales. The Head layer is a decoupled detection head that separates the classification and regression tasks, focusing on human target recognition and keypoint coordinate localization respectively, reducing interference between tasks and improving detection accuracy and optimization efficiency. In this implementation, the main teacher model outputs 17 human keypoint coordinates and detection box information, along with their respective confidence scores. The detection box information includes the coordinates of the two diagonals of the detection box.

[0083] To adapt to the complexity of post-disaster relief scenarios, targeted feature enhancement designs were implemented in the model, specifically as follows: A CBAM convolutional block attention module was embedded in the C2f module of the backbone layer. Through the synergistic effect of channel attention and spatial attention, the distinguishability of human region features from the background was enhanced. Channel attention comprehensively captures the feature importance of the channel dimension through global average pooling and max pooling. Channel weights are generated through two fully connected layers and a sigmoid activation function, highlighting the contribution of feature channels related to the human skeleton and suppressing interference from irrelevant background channels. Spatial attention models the spatial information of the feature map through 7×7 convolutions, and merges the results of global average pooling and max pooling to generate a spatial weight distribution, accurately locating the human region and effectively filtering background noise such as rubble and collapsed objects.

[0084] Meanwhile, an adaptive weight fusion layer is added to the PAN module of the Neck layer. This layer has built-in scene recognition logic, which can dynamically adjust the multi-scale feature weights by analyzing parameters such as the brightness value and key point occlusion ratio of the input image. For example, in low-light scenes, the weights of mid-to-low-level features are automatically increased to enhance the capture of key point details; in occluded scenes, the weights of high-level semantic features are increased. Based on the prior rules of the topological relationship between the overall human body contour and key points, the indirect inference of occluded key points is realized, ensuring the robustness of detection in complex scenes.

[0085] like Figure 2 As shown, the auxiliary teacher model includes two lightweight networks, MobileNet-Pose and ShuffleNet-Pose. The core design goal is a lightweight architecture and efficient feature extraction, which complements the main teacher model in terms of accuracy and speed.

[0086] MobileNet-Pose employs a complete architecture consisting of a Backbone layer, a Neck layer, and a Head layer, balancing lightweight design with strong feature representation capabilities. The Backbone layer uses the MobileNetV3 model, with its core employing depthwise separable convolutions instead of traditional convolutions. This decomposes standard convolutions into depthwise and pointwise convolutions, significantly reducing the number of parameters and computational cost while retaining sufficient feature extraction capabilities. The Neck layer uses a simplified Lite-PAN structure, removing redundant convolutional layers from the original PAN network and retaining only two key feature fusion operations. This reduces computational complexity while maintaining feature associations across multiple scales through cross-layer feature transfer. The Head layer is a lightweight decoupling head that uses 1×1 convolutions for channel dimensionality reduction, ensuring the number of output channels is completely consistent with the master teacher model. This achieves precise alignment of features and labels during distillation, providing a foundation for knowledge transfer.

[0087] The overall network framework of ShuffleNet-Pose is consistent with MobileNet-Pose, only replacing the backbone layer with the ShuffleNetV2 model. Its core relies on channel rearrangement technology to optimize the feature extraction process. This technique reduces the dependency between feature channels and memory access overhead by grouping and rearranging feature channels, avoiding computational waste caused by channel redundancy in traditional convolution. Under the same parameter scale, inference speed is improved by more than 30% compared to MobileNet-Pose, further enhancing the advantages of the auxiliary teacher model in efficient feature extraction and providing fast feature mapping supervision signals for multi-teacher collaborative distillation.

[0088] like Figure 2 As shown, the student model follows the ShuffleNet-Pose model, with an added feature distillation layer between the Neck layer and the Head layer for subsequent calculation of feature distillation loss.

[0089] This invention designs a multi-dimensional fusion loss function in the process of multi-teacher collaborative distillation, including hard label loss, soft label loss and feature distillation loss;

[0090] The hard label loss is the confidence-weighted mean square error between the human keypoint coordinates and the detection box width and height output by the student model and the master teacher model. The calculation method is to sum the weighted error terms by the confidence of the human keypoints output by the master teacher model and divide by the number of human keypoints to obtain the hard label loss.

[0091] The soft label loss is the mean KL divergence of the pose classification probability distributions of the student model and the main teacher model, and the student model and the assistant teacher model.

[0092] Feature distillation loss is the cosine similarity loss of the feature maps output by the intermediate layers of the student model and the master teacher model.

[0093] Specifically, a multi-teacher collaborative distillation framework, such as Figure 3 As shown; during training, the teacher model group and the student model calculate three types of losses and fuse them using a multi-loss fusion component; among them, the hard label loss... The calculation formula is as follows:

[0094] ,

[0095] Where N is 17, it refers to the number of key points on the human body; The confidence score of the human body key points in the main teacher model is given by x and y, which are the coordinates of the human body key points, and w and h are the width and height of the detection box. The superscript s represents the student model and t represents the main teacher model.

[0096] Soft tag loss The calculation formula is as follows:

[0097] ;

[0098] Where k takes values ​​of 1, 2, and 3, they represent the main teacher model, the MobileNet-Pose model, and the ShuffleNet-Pose model, respectively; c takes values ​​from 1 to 4, corresponding to the four poses.

[0099] Characteristic distillation loss The calculation formula is as follows:

[0100] ;

[0101] Where M equals 3, it refers to the number of feature layers; Here is the feature map of the m-th layer of the student model. The feature map of the m-th layer of the main teacher model.

[0102] The multi-loss fusion component fuses the above three types of losses to obtain a multi-dimensional fusion loss function:

[0103] ;

[0104] in, , , The weights for the three types of loss are 0.4, 0.3, and 0.3, respectively, in this embodiment. This weight allocation is used to balance the accuracy and speed of the student model obtained after training. In other embodiments, the values ​​can be further optimized through experiments.

[0105] The student model is trained iteratively using a multi-dimensional fusion loss function. The training strategies in this implementation include single distillation by the main teacher, multi-teacher collaborative distillation, and post-disaster scenario fine-tuning. Single distillation by the main teacher allows the student model to learn the high-precision features of the main teacher. Multi-teacher collaborative distillation aims to fuse the lightweight features of the auxiliary teachers to optimize inference speed. Post-disaster scenario fine-tuning incorporates weighted training with a custom dataset to further improve the recognition adaptability in buried scenarios.

[0106] The preprocessed image is input into the trained student model to obtain the output results including human keypoint coordinates, detection box information and confidence scores. Post-processing optimization is also required, including confidence filtering and keypoint smoothing. Confidence filtering removes human keypoints with confidence scores lower than the adaptive confidence threshold. Keypoint smoothing uses Kalman filtering to correct the jump phenomenon of human keypoints between video frames.

[0107] Specifically, while the student model's output of human keypoint coordinates, bounding box information, and confidence scores already possesses basic detection capabilities, disaster sites present complex factors such as rubble obstruction, dust interference, and camera shake, which can easily lead to two types of problems: First, some output keypoints are pseudo-keypoints generated by background interference, resulting in low confidence scores. Directly using them for feature extraction can lead to pose classification bias. Second, the keypoint coordinates between video frames may experience instantaneous changes due to camera movement or slight limb movements of trapped individuals, disrupting pose continuity and affecting the stable assessment of the trapped individuals' condition. Therefore, post-processing optimization is needed to improve the accuracy, stability, and reliability of the detection results, providing high-quality foundational data for subsequent geometric feature vector extraction and pose classification.

[0108] The core of confidence filtering is to select valid key points based on confidence thresholds. The adaptive confidence threshold is a dynamic adjustment mechanism designed for differences in lighting at disaster sites. Confidence is essentially a quantitative assessment of the reliability of the detection results by the model. False key points or key points that are blurred due to occlusion usually have low confidence. Threshold filtering can directly remove such invalid data. The adaptive adjustment is because in low-light scenes, blurred image details may lead to generally low confidence of valid key points. If a fixed threshold is used, valid points may be mistakenly removed. In normal lighting scenes, the threshold can be appropriately increased to strictly filter, ensuring that the filtering results do not miss valid key points while eliminating interference.

[0109] Keypoint smoothing, the core logic of the Kalman filter algorithm, corrects abrupt changes by utilizing the motion correlation of keypoints between frames. It should be noted that the changes in keypoint coordinates of the same human body in a video stream are continuous, and abrupt changes are mostly outliers caused by instantaneous interference. Kalman filtering, through an iterative process of prediction and updating, predicts the reasonable position of the keypoints in the current frame based on the keypoint coordinates of the previous frame, and then corrects the predicted values ​​by combining the detection results of the current frame, finally outputting smooth keypoint coordinates, thus eliminating the impact of instantaneous changes on the continuity of posture.

[0110] The post-processed and optimized coordinates of human keypoints are discrete spatial location data. Directly inputting them into a classification network cannot effectively represent the structured features of human posture. Therefore, it is necessary to extract geometric feature vectors based on these keypoint coordinates, transforming the discrete coordinate information into structured data that accurately reflects human posture features before inputting it into a classification network for posture determination. Therefore, this invention provides a method for extracting geometric feature vectors based on human keypoint coordinates.

[0111] Extract joint angle features, bone length ratio features, and key point spatial distribution features, and concatenate them to form a geometric feature vector;

[0112] Among them, joint angle features include the absolute angles of key human joints to the vertical or horizontal direction and the relative angles between adjacent joints; bone length ratio features include the lengths and proportional relationships of human skeletal segments; and key point spatial distribution features are used to reflect the spatial distribution characteristics of the overall human posture contour.

[0113] Specifically, the 17 key points on the human body include the nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle.

[0114] Joint angle features are used to calculate the absolute and relative angles of key human joints, covering the core joints for posture differentiation in post-disaster rescue scenarios. The calculation method is to convert the coordinates of key human points into angle values ​​through the vector dot product formula, and normalize them to the interval [0,π] to eliminate the influence of human body scale and shooting angle. The absolute angles include the angles between the hip joint, knee joint, elbow joint, shoulder joint, and neck and the vertical and horizontal directions, totaling 10 features. The relative angles include the angles between the connecting line segments between adjacent joints, such as the angle between the line connecting the shoulder, elbow, and wrist, totaling 22 features.

[0115] Skeletal length proportion features are used to calculate the length and proportion of key skeletal segments, avoiding the influence of individual length on human height. The calculation method is to normalize all length values ​​by dividing them by the total height of the human body (top of head - bottom of feet) to a range of [0,1]. Skeletal length includes trunk length (mid-shoulder to mid-hip), upper limb length (shoulder to wrist), and lower limb length (hip to ankle), totaling 8 dimensions. Proportional features include the ratio of upper limb to trunk, the ratio of lower limb to trunk, and the ratio of left to right limb length, totaling 12 dimensions.

[0116] The spatial distribution features of key points include the aspect ratio of the human body, the centroid coordinates of key points, and the dispersion of key points of the limbs, totaling 12 features.

[0117] In addition, due to the loss of key points caused by post-disaster occlusion, missing value processing is required. This implementation uses adjacent joint interpolation to fill in the missing coordinates. For example, when the wrist key point is missing, it can be calculated by linear interpolation of the shoulder and elbow coordinates. All feature values ​​are standardized to a distribution with a mean of 0 and a variance of 1 using Z-Score to eliminate dimensional differences and improve network convergence efficiency. Finally, the three types of features are concatenated in the order of angle, proportion, and spatial distribution to form a 64-dimensional fixed-length feature vector.

[0118] In one implementation, the classification network is a lightweight hybrid network of fully connected and convolutional networks adapted for embedded devices, balancing feature fitting ability and inference speed. The network structure is as follows: Figure 4As shown, the feature mapping layer uses 1×1 convolutions instead of traditional fully connected layers to increase the dimensionality of the input feature vector, enhancing the non-linear expressive power of features while reducing the number of parameters. The hidden layer uses the Leaky-ReLU activation function to effectively avoid neuron death and improve the fitting ability of features in low-light and occluded scenes. The batch normalization layer accelerates network convergence, and the Dropout layer suppresses overfitting, adapting to scenarios with limited sample sizes in disaster relief datasets. The output layer outputs a 4-dimensional probability distribution, corresponding to standing, curled-up, lying down, and four other postures. The Softmax activation function ensures that the sum of the output probabilities is 1, facilitating subsequent decision-making. To further adapt to the computing power of embedded devices, the total number of network parameters is controlled within 100,000, and INT8 quantization is performed after training.

[0119] This invention also provides a human posture detection system for buried environments, such as... Figure 5 As shown, it includes:

[0120] The data acquisition module is used to acquire video frames of the burial scene using low-light image acquisition equipment and adaptive brightness adjustment lighting equipment;

[0121] The data preprocessing module is used to reduce the image size of video frames to a standard size and adaptively adjust the image brightness to obtain a preprocessed image;

[0122] The key point detection module is used to input the preprocessed image into the trained student model, and after post-processing optimization, outputs the coordinates of human key points, the coordinates of the detection box, and their respective confidence scores. The trained student model is obtained by training through a multi-teacher collaborative distillation framework, which is based on a teacher model group and a multi-dimensional fusion loss function, and updates the student model parameters through iterative distillation training.

[0123] The pose classification module is used to extract geometric feature vectors based on the coordinates of human key points, and input the geometric feature vectors into the classification network to obtain the pose detection results; the geometric feature vectors include joint angle features, bone length ratio features and key point spatial distribution features.

[0124] The visualization module is used to intuitively present human posture detection results, display multi-dimensional information, and support interactive operations.

[0125] In one implementation, the visualization module is designed with a fully functional visualization display system, which includes two major modules: real-time overlay display of images and local interactive interface, taking into account both the need for rapid viewing of the rescue scene and the need for refined operation in the local scenario.

[0126] The visualization module automatically connects key human body points to form a skeletal framework, clearly outlining the human posture contour. It also employs an independent color space encoding strategy, assigning unique colors to different posture categories, making the classification results intuitive and easily identifiable. Furthermore, it utilizes a hybrid OpenCV and PIL rendering technology to achieve clear display of Chinese text, supports automatic matching of various system fonts, and adapts to different operating environments.

[0127] In addition to visualizing the video processing results, the interface offers a wealth of operational and text result display functions: it supports three operating modes—image loading, video playback, and real-time camera switching—to adapt to the recognition needs of different data sources; it provides model selection and parameter setting functions, allowing flexible adjustment of core parameters such as confidence level to meet the recognition accuracy requirements of different scenarios; the interface simultaneously displays the original image, the recognition result image, and the detection table, with the detection table detailing key information such as recognition ID, pose category, Chinese label, confidence level, target location, and size for detailed viewing; it supports filtering and displaying recognition results by pose category, and also features people counting, image and text result saving functions to meet the needs of local debugging, data processing, and analysis. When lighting is insufficient, it also supports manually turning the lighting system on and off.

[0128] The present invention provides a human posture detection system for buried scenarios, which is deployed on an embedded terminal. The embedded terminal includes a core computing unit, a data acquisition unit, a lighting control unit, and an extended sensing unit. The core computing unit is communicatively connected to the data acquisition unit, the lighting control unit, and the extended sensing unit.

[0129] The core computing unit is a processor that supports multi-task parallel scheduling and mixed-precision computing, and has the ability to cache data, infer algorithms, and coordinate the control of various units.

[0130] The data acquisition unit uses a low-light camera as the image acquisition device and reserves a dual-modal acquisition expansion interface for connecting an external infrared camera to adapt to extreme scenarios.

[0131] The lighting control unit includes at least one high color rendering supplementary light device as a lighting device, and achieves adaptive start / stop and supplementary light intensity adjustment through the light sensing feedback control of the core computing unit.

[0132] The extended sensing unit supports the access of multiple types of sensors, and collects environmental or personnel data in the buried scene according to the type of sensor accessed.

[0133] In one implementation, the core computing unit, serving as the system's computing power core and control center, utilizes a Jetson Orin NX 16GB processor. This processor boasts 100 TOPS (INT8) computing power, supports FP16 / FP32 mixed-precision computing, and meets the data processing requirements of core algorithm inference and extended functions. It is also equipped with 16GB of LPDDR5 memory (102.4GB / s bandwidth) to ensure data caching and efficient scheduling across multiple modules. It supports two MIPI CSI-2 interfaces, USB 3.2, GPIO, and other interfaces, enabling seamless integration with various hardware modules and expansion devices. Typical power consumption is 15-18W, and with passive cooling and an integrated fan, it ensures stable operation under full load for extended periods.

[0134] The data acquisition unit uses an IMX385 low-light camera with a 1 / 2.8-inch Sony CMOS sensor, 2 megapixels, supports 30fps output and 0.0005Lux starlight-level acquisition, suitable for mid-range scenes of 3-15 meters. It has a reserved MIPI CSI-2 expansion interface for connecting an external infrared camera, forming a dual-modal acquisition system (visible light and infrared), suitable for extreme scenes such as no light or dense smoke. It comes standard with a 5.1mm fixed-focus lens, but supports wide-angle lens replacement.

[0135] The lighting control unit consists of two 3W high color rendering white LEDs, with a total luminous flux of 600-720lm, an illumination angle of 60°, and a supplementary lighting coverage of ≥5 meters. It is controlled by GPIO levels in the core computing unit, automatically starting and stopping according to ambient light levels to meet the image acquisition needs of low-light scenes.

[0136] The extended sensing unit has reserved sensor interfaces, such as smoke concentration sensors and life detection sensors, which can be connected to the core computing unit via USB 3.2 or GPIO interfaces to identify changes in the rescue environment and life signals of trapped personnel. The results are output in conjunction with the attitude recognition results to enrich the information on the rescue environment and the status of trapped personnel, thereby improving the targeting of rescue efforts.

[0137] The present invention also provides an electronic device, comprising:

[0138] One or more processors;

[0139] Memory, used to store one or more programs;

[0140] When the one or more programs are executed by the one or more processors, the one or more processors implement a human posture detection method for buried scenarios as described in any one of the above.

[0141] The above embodiments are merely illustrative of the technical concept and features of the present invention, intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly, and should not be construed as limiting the scope of protection of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects. The scope of the present invention is defined by the appended claims rather than the foregoing description, and thus all changes falling within the meaning and scope of the equivalents of the claims are intended to be included within the present invention.

Claims

1. A human posture detection method for a landfill scenario, characterized in that, include: Video frames of the burial scene were captured using low-light image acquisition equipment and adaptively brightness-adjusting lighting equipment. The video frame images are reduced to a standard size, and the image brightness is adaptively adjusted to obtain a preprocessed image; The preprocessed image is input into the trained student model. After post-processing optimization, the coordinates of human key points, the coordinates of the detection box, and their respective confidence scores are output. The trained student model is obtained by training through a multi-teacher collaborative distillation framework. The multi-teacher collaborative distillation framework is based on teacher model groups and a multi-dimensional fusion loss function, and updates the student model parameters through iterative distillation training. Geometric feature vectors are extracted based on the coordinates of human key points, and then input into a classification network to obtain pose detection results. The geometric feature vectors include joint angle features, bone length ratio features, and key point spatial distribution features. The teacher model group includes a main teacher model and at least one auxiliary teacher model. The main teacher model outputs hard labels, soft labels, and intermediate layer features, while the auxiliary teacher model outputs soft labels and intermediate layer features. The hard labels are the coordinates of human key points, the coordinates of the detection boxes, and their respective confidence scores. The soft labels are the KL divergence of the pose classification probability distribution. The intermediate layer features are the feature maps output by the intermediate layer of the model. The multi-dimensional fusion loss function includes hard label loss, soft label loss, and feature distillation loss; The hard label loss is the confidence-weighted mean square error between the human keypoint coordinates and the detection box width and height output by the student model and the master teacher model. It is calculated by weighting and summing each error term using the confidence scores of the human keypoints output by the master teacher model, and then dividing by the number of human keypoints. The formula for calculating the hard label loss is as follows: ， wherein, is the hard label loss; N is the number of human key points; is the human key point confidence of the main teacher model; x, y are the human key point coordinates; w and h are the width and height of the detection box; superscript s represents the student model, and t represents the main teacher model. The soft label loss is the mean KL divergence of the pose classification probability distributions of the student model and the main teacher model, and the student model and the auxiliary teacher model; the formula for calculating the soft label loss is: ； wherein, is the soft label loss; K represents the total number of main teacher models and auxiliary teacher models; C refers to the number of posture types; represents the probability of the student model classifying the cth posture; represents the probability of the kth teacher model classifying the cth posture. The feature distillation loss is the cosine similarity loss of the feature maps output by the intermediate layers of the student model and the main teacher model; the formula for calculating the feature distillation loss is: ， in, The characteristic distillation loss is M; M refers to the number of characteristic layers. Here is the feature map of the m-th layer of the student model. The feature map of the m-th layer of the main teacher model.

2. The human posture detection method for buried scenarios according to claim 1, characterized in that, The master teacher model is an improved YOLOv8-Pose model, which includes a Backbone layer, a Neck layer, and a Head layer. The C2f module of the Backbone layer contains a CBAM attention module, and the PAN module of the Neck layer has an adaptive weight fusion layer.

3. The human posture detection method for buried scenarios according to claim 2, characterized in that, The adaptive weight fusion layer has built-in scene recognition logic. It determines the current scene state by analyzing the brightness value of the preprocessed image and the occlusion ratio of human key points, thereby dynamically adjusting the multi-scale feature fusion weights in the PAN module.

4. The human posture detection method for buried scenarios according to claim 1, characterized in that, The method for extracting geometric feature vectors based on the coordinates of human key points is as follows: extracting joint angle features, bone length ratio features and key point spatial distribution features, and splicing them together to form a geometric feature vector; The joint angle features include the absolute angles of key human joints to the vertical or horizontal direction and the relative angles between adjacent joints; the bone length ratio features include the lengths and ratios of human skeletal segments; and the key point spatial distribution features are used to reflect the spatial distribution features of the overall human posture contour.

5. The human posture detection method for buried scenarios according to claim 1, characterized in that, The post-processing optimization includes confidence filtering and keypoint smoothing; The confidence filtering involves removing key human body points with a confidence level lower than the adaptive confidence threshold. The keypoint smoothing mentioned above uses Kalman filtering to correct the abrupt changes in human body keypoints between video frames.

6. A human posture detection system for burial scenarios, based on the human posture detection method for burial scenarios according to any one of claims 1-5, characterized in that, include: The data acquisition module is used to acquire video frames of the burial scene using low-light image acquisition equipment and adaptive brightness adjustment lighting equipment; The data preprocessing module is used to reduce the image size of video frames to a standard size and adaptively adjust the image brightness to obtain a preprocessed image; The key point detection module is used to input the preprocessed image into the trained student model, and after post-processing optimization, outputs the coordinates of human key points, the coordinates of the detection box, and their respective confidence scores. The trained student model is obtained by training through a multi-teacher collaborative distillation framework, which is based on a teacher model group and a multi-dimensional fusion loss function, and updates the student model parameters through iterative distillation training. The pose classification module is used to extract geometric feature vectors based on the coordinates of human key points, and input the geometric feature vectors into the classification network to obtain the pose detection results; the geometric feature vectors include joint angle features, bone length ratio features and key point spatial distribution features. The visualization module is used to intuitively present human posture detection results, display multi-dimensional information, and support interactive operations.

7. A human posture detection system for buried environments according to claim 6, characterized in that, The aforementioned human posture detection system for buried scenarios is deployed on an embedded terminal; The embedded terminal includes a core computing unit, a data acquisition unit, a lighting control unit, and an extended sensing unit, wherein the core computing unit is communicatively connected to the data acquisition unit, the lighting control unit, and the extended sensing unit. The core computing unit is a processor that supports multi-task parallel scheduling and mixed-precision computing, and has the ability to cache data, infer algorithms, and coordinate the control of each unit. The data acquisition unit uses a low-light camera as the image acquisition device and reserves a dual-modal acquisition expansion interface for connecting an external infrared camera to adapt to extreme scenarios. The lighting control unit includes at least one high color rendering supplementary light device as a lighting device, and achieves adaptive start / stop and supplementary light intensity adjustment through the light sensing feedback control of the core computing unit. The extended sensing unit supports the access of multiple types of sensors and collects environmental or personnel data in the buried scene according to the type of sensor accessed.

8. An electronic device, characterized in that: The electronic device includes: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a human posture detection method for buried scenarios as described in any one of claims 1-5.

Images