Multi-angle cpr training video detection method based on machine vision

Abstract

The application discloses a multi-angle CPR training video detection based on machine vision, and has the characteristics of establishing an LCSG-YOLO model, preparing work, designing a KCHT matching algorithm and formulating a scoring standard.Compared with the prior art, the application has the advantages that: through pure machine vision technology, the key targets in cardiopulmonary resuscitation are accurately detected, and appropriate frequency of cardiopulmonary compression and artificial respiration is ensured, especially in a complex dynamic environment, efficient and accurate evaluation is ensured, a customized data set for CPR evaluation is constructed, an LCSG-YOLO model is trained based on the data set, and the system automatically evaluates the CPR operation according to the preset standard to determine whether the CPR operation meets the standard through video uploading, without relying on sensors, thereby providing scientific support for CPR quality improvement and training and promoting the popularization of intelligent evaluation application.

Description

Technical Field

[0001] This invention relates to the field of CPR assessment system technology, specifically a multi-angle CPR training video detection method based on machine vision. Background Technology

[0002] Cardiac arrest (CA) is a life-threatening emergency, and the survival rate of out-of-hospital patients highly depends on the timely and effective implementation of cardiopulmonary resuscitation (CPR). However, current CPR skills assessments mainly rely on simulators and human scoring, which often leads to inconsistent, inaccurate, and inefficient assessment results.

[0003] To improve the accuracy and efficiency of CPR skills assessment, the medical community and the field of artificial intelligence have begun to collaborate. Existing research has explored various AI-based methods, including object detection, motion capture, and even systems like ChatGPT for scoring. For example, Zhang et al. developed a wearable device based on binocular vision to monitor wrist movements for real-time assessment of CPR quality. Similarly, Tang et al. utilized a 3D motion capture system to assess CPR quality, highlighting the potential of precise motion data. Furthermore, Wang et al. applied AI and the ChatGPT-4 model to analyze CPR training videos, achieving automated scoring comparable to expert evaluation. Despite these advancements, many methods still rely on expensive and bulky 3D capture devices or sensors. This study aims to overcome these limitations by proposing a sensor-free CPR assessment method.

[0004] The YOLO (You Only Look Once) algorithm is widely recognized for its real-time performance and accuracy, making it a popular choice for object detection tasks. The latest version, YOLOv11, optimizes its architecture by reducing the number of parameters, thereby improving both speed and detection accuracy. This makes YOLO particularly suitable for dynamic environments where additional sensors are not required. Furthermore, the Kalman-Hungarian (KH) matching algorithm improves target tracking and assignment, further enhancing detection accuracy and system efficiency.

[0005] To address the challenge of reducing reliance on sensors and developing cost-effective, flexible, and practically applicable CPR assessment techniques, this research proposes a novel machine vision-based CPR assessment system. This system combines the YOLO11 model and the KH matching algorithm, designing an LCSG-YOLO model and KCHT matching algorithm, and establishing a novel machine vision-based CPR detection and evaluation standard. This provides an innovative solution for cost-effective and accurate CPR assessment and lays the foundation for practical applications. Summary of the Invention

[0006] The technical problem to be solved by this invention is to overcome the above-mentioned technical defects and provide a multi-angle CPR training video detection method based on machine vision.

[0007] 1. To solve the above problems, the technical solution of the present invention is: a multi-angle CPR training video detection method based on machine vision, characterized in that: it includes establishing an LCGS-YOLO model, preparatory work, KCHT matching algorithm design and setting of scoring criteria;

[0008] The establishment of the LCGS-YOLO model includes the design of the local condition sharing feature sharing layer, the design of the GCSCA module, and the design of the GCSCA optical module.

[0009] The local condition sharing feature sharing layer is designed to enhance the local and global information in the input feature map by integrating multi-scale context features. The local condition sharing feature sharing layer uses adaptive pooling technology to extract global and local context and adopts learnable weights. The input feature map x undergoes adaptive pooling at two scales.

[0010]

[0011] Channel reduction is performed using 1x1 convolutional layers to generate shared feature representations. It integrates contextual information at different scales and normalizes it using the Sigmoid activation function to generate a channel attention map;

[0012] ;

[0013] Bilinear interpolation is used to upsample the local conditional shared feature sharing layer to match its spatial dimension with the input feature map x.

[0014] Input feature map x and shared feature map The elements are multiplied one by one and weighted by learnable weights to generate the output of the local conditional sharing feature sharing layer.

[0015] In the CSCA module design, the GCSCA (Global Context and Spatial Context Attention) module employs a self-attention mechanism to capture global and spatial context information;

[0016] The GSCCA optical module design simplifies and enhances the GSCCA module, creating the GSCCA Light module;

[0017] The preparatory work includes creating a dedicated dataset and establishing a grayscale axis standard for CPR training videos;

[0018] The KCHT matching algorithm design includes target dynamic prediction, observation update, application of the Hungarian algorithm, and confidence correction mechanism;

[0019] Target dynamic prediction: The KCHT algorithm uses a Kalman filter to dynamically predict the target's position, velocity, and other state information, generating a prediction for each target;

[0020] Observation update: The KC filter updates the target state using the YOLO detection results, compares the predicted position with the detected position, and adjusts the state to make the prediction consistent with the actual target state;

[0021] Application of the Hungarian algorithm: The Hungarian algorithm is used for temporal target matching, which associates targets in consecutive frames through optimal matching;

[0022] Confidence correction mechanism: When confusion or inconsistency occurs, the KCHT algorithm applies weights to correct the predictions;

[0023] The established scoring criteria include the Cohen's Kappa coefficient and an assessment of classification accuracy.

[0024] Furthermore, the GCSCA (Global Context and Spatial Context Attention) module is a self-attention mechanism that captures global and spatial context information, enhancing the model's feature representation capabilities and robustness.

[0025] The GCSCA module contains the following components:

[0026] Channel attention: Adaptively adjusts the weight of each channel, highlighting the features of key channels and enhancing attention to local regions and important targets;

[0027] ;

[0028]

[0029] Spatial attention: Spatial attention applies weights to spatial features;

[0030] ;

[0031]

[0032] Global context convolutional layer: Aggregates global information to enhance the model's global awareness;

[0033]

[0034] Feature fusion and residual linking: A feature fusion and residual linking strategy is adopted.

[0035] Furthermore, the creation of the dedicated dataset involved selecting 40% of the collected videos, totaling 105 segments. Based on the complexity of the actions and key events, 20 to 30 keyframes were extracted from each video. This process ultimately produced 2,156 labeled frames, which were divided into a training set of 1,946 frames and a validation set of 210 frames.

[0036] The establishment of the grayscale standard for CPR training videos: Grayscale values ​​of chest compressions and artificial respiration corresponding to key labels were extracted from the CPR research dataset. These values ​​were used to create a 10-level grayscale scale to facilitate the identification and counting of chest compressions and the detection of compression depth.

[0037] Furthermore, the Cohen's Kappa coefficient is a statistical indicator used to assess the difference between classification prediction accuracy and randomness. It is suitable for evaluating the consistency between two evaluators or classifiers on classification tasks, even if these classification tasks involve randomness.

[0038] ;

[0039] ;

[0040] ;

[0041] The assessment of classification accuracy is expressed as the percentage of samples that pass classification and those that fail classification out of all samples. Accuracy is calculated using the following formula:

[0042] .

[0043] The machine vision-based multi-angle CPR video detection method is characterized by comprising the following components:

[0044] Small object optimization: The LCSG-YOLO model improves the detection sensitivity of key cardiopulmonary resuscitation areas by improving the convolutional layers and integrating local contextual information;

[0045] KCHT matching algorithm: The KCHT algorithm combines Kalman filtering and Hungarian matching algorithm, and introduces confidence index and time series features to achieve target pairing and tracking in multi-angle environments;

[0046] Cardiopulmonary resuscitation (CPR) frequency detection: Based on time-based averaging, grayscale and jitter feature analysis, and fast Fourier transform for frequency domain analysis, the CPR frequency is accurately detected.

[0047] Cardiopulmonary resuscitation depth detection: Compression depth is determined using signal smoothing, period detection, peak-to-valley difference measurement, and average depth analysis methods.

[0048] Multi-angle fusion: In multi-angle scenes, a dynamic weight allocation mechanism adjusts weights based on the duration of the viewpoint and the video frame rate;

[0049] Cardiopulmonary resuscitation (CPR) training video module: Integrates and analyzes the results, assesses indicators including the number of compressions, frequency, and depth, and automatically determines whether each indicator meets international CPR standards.

[0050] The advantages of this invention compared to existing technologies are:

[0051] 1. Propose the LCSG-YOLO model: Through pure machine vision technology, accurately detect the key targets in cardiopulmonary resuscitation—appropriate frequency of cardiopulmonary compressions and artificial respiration, especially in complex dynamic environments, to ensure efficient and accurate assessment.

[0052] 2. Create a dedicated dataset: A custom dataset for CPR evaluation was built, and the LCSG-YOLO model was trained based on this dataset.

[0053] 3. Improved KCHT matching algorithm: Based on the traditional KH matching algorithm, the target matching process is optimized by combining confidence and time series information, which significantly improves the matching accuracy and robustness of the LCSG-YOLO model.

[0054] 4. Establish scoring criteria: Based on the AHA CPR and ECC guidelines, CPR assessment criteria applicable to automatic scoring were developed, providing a scientific basis for the intelligent assessment system.

[0055] 5. Design a complete CPR evaluation process: Through video upload, the system automatically evaluates CPR operations according to preset standards to determine whether they meet the standards. It does not rely on sensors, providing scientific support for CPR quality improvement and training, and promoting the popularization of intelligent evaluation applications. Attached Figure Description

[0056] Figure 1 This is a placement diagram of the multi-view fluoroscopic device for cardiopulmonary resuscitation according to the present invention.

[0057] Figure 2 This is the multi-angle CPR training diagram of the present invention.

[0058] Figure 3 This is the grayscale value scale of the present invention.

[0059] Figure 4 This is a flowchart of the multi-angle detection process of the present invention.

[0060] Figure 5This is a schematic diagram of the LCSG-YOLO mode of the present invention.

[0061] Figure 6 This is the input feature map x of the present invention.

[0062] Figure 7 This is a design diagram of the GCSCA module of the present invention.

[0063] Figure 8 This is the CPR training video process of the present invention.

[0064] Figure 9 This is the jittered grayscale data image of the present invention.

[0065] Figure 10 This is a depth detection data diagram of the present invention.

[0066] Figure 11 This is a diagram showing the CPR training video results of the present invention.

[0067] Figure 12 This is a diagram showing the test results of the multi-view CPR training video technology of the present invention.

[0068] Figure 13 This is the evaluation confusion matrix diagram of the present invention.

[0069] Figure 14 This is the frequency error distribution diagram of the present invention.

[0070] Figure 15 This is a depth detection distribution map of the present invention. Detailed Implementation

[0071] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings. Identical components are indicated by the same reference numerals.

[0072] It should be noted that the terms “front,” “back,” “left,” “right,” “up,” and “down” used in the following description refer to the directions shown in the attached diagram, while the terms “inside” and “outside” refer to the directions toward or away from the geometric center of a specific component, respectively.

[0073] To make the content of this invention easier to understand, the technical solutions in the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings.

[0074] like Figure 6 As shown

[0075] The establishment of the LCGS-YOLO model includes the design of the local condition sharing feature sharing layer, the design of the GCSCA module, and the design of the GCSCA optical module;

[0076] The local conditional shared feature sharing layer enhances the local and global information in the input feature map by integrating multi-scale contextual features. This module uses adaptive pooling to extract global and local context and adopts learnable weights to improve target focus and detection accuracy at different spatial scales. The input feature map x undergoes two scales of adaptive pooling: global pooling and mesoscale pooling, resulting in global pooling feature map C1 and mesoscale pooling feature map C2, respectively.

[0077]

[0078] Channel reduction is performed using 1x1 convolutional layers to generate shared feature representations. Then, contextual information at different scales is fused and normalized using the Sigmoid activation function to generate a channel attention map;

[0079]

[0080] Bilinear interpolation is used to upsample the shared feature map so that its spatial dimensions match those of the input feature map x;

[0081]

[0082] Input feature map x and shared feature map Element-wise multiplication is performed, and the results are weighted using learnable weights to generate the output of the module.

[0083]

[0084] like Figure 7 As shown

[0085] The GCSCA (Global Content and Spatial Content Attention) module employs a self-attention mechanism designed to capture global and spatial context information, thereby enhancing the model's feature representation capabilities and robustness. This module can extract global information and spatial details from local regions, improving the ability to identify key features in complex scenes, and thus enhancing the accuracy of object detection in multi-object and challenging environments.

[0086] This technical design includes the following components:

[0087] Channel attention: By adaptively adjusting the weights of each channel, the model can highlight the features of key channels and enhance attention to local regions and important targets, especially in complex contexts.

[0088] ;

[0089]

[0090] It is a channel weight adjustment parameter; It is a feature map after channel attention enhancement;

[0091] Spatial attention: By weighting spatial features, this model can accurately locate target regions, isolate noise, and improve detection accuracy;

[0092] ;

[0093]

[0094] Global context convolutional layer: Aggregates global information, enhances the model's global perception ability, helps the model maintain consistency in multiple targets and complex backgrounds, and improves target recognition accuracy;

[0095]

[0096] Feature fusion and residual linking: Feature fusion and residual linking strategies are adopted to ensure effective information propagation, avoid gradient vanishing, and improve the stability and robustness of the model;

[0097]

[0098] To improve computational efficiency and reduce model complexity, the GCSCA module was simplified and enhanced, resulting in the creation of the GCSCA Light module. This module is particularly suitable for the neck region of object detection networks, providing effective feature enhancement with limited computational resources.

[0099] like Figures 1 to 3 As shown

[0100] Preparations include creating a dedicated dataset and establishing grayscale axis standards for CPR training videos;

[0101] Due to the limited availability of publicly accessible CPR datasets, this study collected data from 265 participants in a controlled CPR training environment. Each training room was equipped with advanced CPR teaching equipment and three cameras were set up, positioned at the top left, bottom, and top right corners to capture CPR operations from multiple angles. Data was collected in four CPR training rooms, each with a roughly consistent camera configuration and only minor positional adjustments. Since no publicly available CPR datasets were available, keyframe images extracted from CPR videos were analyzed using a video parser and then manually labeled with five labels, as shown in Table 1.

[0102]

[0103] To create a dedicated dataset, 40% of the collected videos, totaling 105 segments, were selected. Based on the complexity of the actions and key events, such as the start and end of chest compressions and artificial respiration, 20 to 30 keyframes were extracted from each video. This process ultimately produced 2,156 labeled frames, which were divided into a training set of 1,946 frames and a validation set of 210 frames.

[0104] Gray values ​​of chest compressions and artificial respiration corresponding to key labels were extracted from the dataset. These values ​​were used to create a 10-level gray scale to facilitate the identification and counting of chest compressions and the detection of compression depth.

[0105] During the action recognition phase, the grayscale value of the target area is compared with a constructed grayscale scale. When the grayscale value falls within the scale's range of 4 to 7, the action is classified as effective chest compression or artificial respiration. This method improves the accuracy of action recognition and counting, provides a reliable mechanism for assessing compression depth, and thus ensures compliance with CPR standards and improves the overall accuracy of the assessment.

[0106] like Figure 8 As shown

[0107] The KCHT matching algorithm combines the Kalman filter with the Hungarian algorithm (KH matching) and introduces a confidence mechanism. This mechanism uses temporal features to dynamically predict changes in the target's position in consecutive video frames. By applying the recursive principle of the Kalman filter, the algorithm fuses target state estimates (such as position and velocity) with observation data, and improves matching accuracy through prediction updates and confidence correction.

[0108] The Kalman filter is extended to a Kalman belief filter (KC filter), which combines label detection and updates the target state equation based on Kalman prediction, thereby further improving the accuracy of state estimation. The main steps of the algorithm are as follows:

[0109] Target dynamic prediction is a method in which the KCHT algorithm uses a Kalman filter to dynamically predict the position, velocity, and other state information of a target, generating a reasonable prediction for each target.

[0110] Observation Update: The KC filter updates the target state using YOLO detection results. The state is adjusted to match the predicted and detected positions.

[0111] Application of the Hungarian algorithm: The Hungarian algorithm is used for temporal target matching, which associates targets in consecutive frames through optimal matching;

[0112] Confidence correction mechanism: When confusion or inconsistency occurs, the KCHT algorithm applies weights to correct the predictions;

[0113] In the CPR training video module, CPR quality assessment is enhanced by analyzing features such as grayscale values, jitter, and duration. Grayscale values ​​reflect the depth of the press and the stability of the action; jitter detection is used to identify non-standard actions, such as unstable or excessively fast presses; and duration ensures adherence to CPR standards, confirming both the standardization of the procedure and the integrity of the operation. This multi-dimensional analysis makes the assessment more detailed and accurate.

[0114] like Figures 9 to 10 As shown

[0115] The effectiveness of chest compressions and artificial respiration during cardiopulmonary resuscitation (CPR) is evaluated through three analyses: time analysis, grayscale change and jitter analysis, and frequency domain analysis.

[0116] First, temporal characteristics are detected by calculating the standardized average time of CPR and artificial respiration. For each label (i.e. CPR and artificial respiration) in each viewpoint, the frequency of actions is determined by calculating the duration of each label and comparing it with the standardized average time.

[0117] ;

[0118]

[0119] "total" represents the total duration, in seconds or minutes; "Tavg" represents the average duration of a set of standard CPR movements; "Ntotal" represents the total number of repetitions; and "BPM" represents the number of movements per minute.

[0120] Gray-scale analysis: First, the LCSG-YOLO and KCHT matching algorithms are used to accurately locate the CPR area and extract the gray-scale value of the area. By comparing this value with a pre-set standard gray-scale axis, the effectiveness of the compression action can be evaluated. If the detected gray-scale value falls within the standard range, the action is considered effective, whether it is compression or artificial respiration; otherwise, the action is considered invalid and excluded from the statistics.

[0121] Jitter Analysis: Pressing motions typically produce minute motion jitters. To assess this, the jitter value of a “marker point” located at the center of the pressing area was analyzed to detect vibrations and jitters generated during the motion.

[0122] Fast Fourier Transform Analysis: In order to study the frequency characteristics of CPR actions in greater depth, Fast Fourier Transform was used to perform frequency domain analysis on the grayscale and jitter signals of cardiopulmonary compression and artificial respiration.

[0123] Time domain signal: First, extract the time domain signals of cardiopulmonary compression and artificial respiration as input signals;

[0124] Frequency domain transformation: Converting a time-domain signal into a frequency-domain signal using a fast Fourier transform and calculating its frequency characteristics;

[0125] Frequency feature map: Based on the FFT results, a frequency feature map is generated to capture the peaks and valleys in the frequency domain signal, which represent each cycle of CPR and artificial respiration, respectively.

[0126] Periodic analysis: By analyzing the number of peaks and troughs in the frequency domain, the timing and frequency of each cardiopulmonary compression and artificial respiration cycle can be accurately determined;

[0127]

[0128] X(n) is the time-domain signal, X(f) is the frequency-domain signal, f is the frequency band, and N is the length of the signal.

[0129] Depth detection calculates the compression depth by analyzing the amplitude of the jitter signal generated during each press. The CPR depth detection module determines the depth of each press based on the jitter signal at the center of the target area;

[0130] Signal smoothing and cycle detection: The jitter signal is smoothed during compression depth detection. Then, a peak detection algorithm is used to identify the peaks and troughs within each compression cycle. By locating these peaks and troughs, the start and end points of each compression cycle can be determined, providing key data points for calculating compression depth.

[0131] Compression depth calculation: Compression depth refers to the difference between the peak and trough values ​​within a compression cycle;

[0132] After YOLO11n recognition, filtering, and matching algorithms, the confidence level of YOLO11n in recognizing the tag and the confidence level after filtering and matching algorithms are extracted.

[0133]

[0134] Peaki represents the peak value of the i-th compression cycle, and Trougi represents the trough value of that cycle. This allows for the precise calculation of the depth of each compression cycle.

[0135] Calculated based on the average pressing depth:

[0136] The depth of all compression cycles can be calculated using the following formula:

[0137]

[0138] n represents the total number of compression cycles, while The calculated value representing the total pressing depth

[0139] By averaging the depth over multiple cycles, the noise that may occur in a single cycle can be minimized, resulting in a more stable and accurate calculation of the pressure depth.

[0140] like Figure 11 As shown

[0141] To address challenges such as varying illumination, complex experimental environments, potential occlusion, or viewpoint information misalignment, a dynamic weighting mechanism is introduced. This mechanism ensures that contributions from information from different viewpoints are appropriately weighted, thereby improving the accuracy of CPR analysis.

[0142] In the analysis of CPR videos, eligibility criteria were determined based on a multi-view dynamic weighted fusion of the LCSG-YOLO and KCHT matching algorithms, along with a comprehensive consideration of parameters such as time, frequency, and compression depth. Since variations in lighting, camera angle, and distance in the experimental setup could introduce errors in detecting chest compressions and artificial respiration, this study designed a eligibility criteria process that combines compression frequency and depth and uses machine learning standards to evaluate CPR training videos. The following is an overview of the process and criteria used in this study to determine whether a CPR test was passed:

[0143] CPR frequency: The compression frequency for each viewpoint is calculated by combining a time count weight of 0.5 and a jitter grayscale weight of 0.5 to arrive at the final CPR frequency.

[0144] CPR compression count: Similar to frequency, the number of compressions per viewpoint is also determined by combining a time count weight of 0.5 and a jitter grayscale weight of 0.5, thus calculating the final number of compressions;

[0145] Compression depth: Compression depth from three perspectives is integrated through a dynamic weighting mechanism;

[0146] Effective cardiopulmonary compression rate: The compression rate must be between 100 and 120 compressions per minute;

[0147] Minimum compression depth: The compression depth must be at least 5 centimeters;

[0148] Compliance assessment: Based on the above standards, the CPR procedure is evaluated to determine whether it complies with the established standards.

[0149] like Figures 12 to 14 As shown

[0150] The scoring criteria include the Cohen's Kappa coefficient and the assessment of classification accuracy;

[0151] Cohen's Kappa coefficient is a statistical metric used to assess the difference between classification prediction accuracy and randomness. It is particularly useful for evaluating the consistency between two evaluators or classifiers on a classification task, even if the task involves randomness.

[0152] ;

[0153] ;

[0154]

[0155] The assessment of classification accuracy is expressed as the percentage of samples that pass classification and those that fail classification out of all samples. Accuracy is calculated using the following formula:

[0156]

[0157] This study compared the detection performance of LCSG-YOLO with the YOLO series models YOLO11 and YOLOv8 (see Table 2-A). The results show that LCSG-YOLO outperforms other models in key metrics, especially in handling complex occlusion scenarios. Although its mAP50 performance is comparable to YOLO11, LCSG-YOLO achieves a slight improvement in mAP50-95, reaching 0.681. In terms of inference speed, LCSG-YOLO runs at 1.3 milliseconds. Figure 12 Significantly faster than YOLOv8 and YOLO11, LCSG-YOLO is better suited for real-time applications. Specifically, it demonstrates higher accuracy in detecting CPR and artificial respiration areas, as well as DC and DH occlusion. It significantly improves the detection capability of complex occlusions and the precise localization of compression areas. The enhanced KCHT matching algorithm shows significant performance improvements in all categories, with an accuracy rate exceeding 97% in each category. In particular, the accuracy rates of key metrics—CPR and artificial respiration—have improved from 97.8% and 90.8% to 98.5% and 98.7%, respectively (see Table 2-B). These improvements lay a solid foundation for accurately identifying compression frequency and depth, thereby ensuring the quality and accuracy of CPR operations in complex scenarios.

[0158] Table 2. Results of Multi-View CPR Practical Video Test

[0159] ;

[0160]

[0161] The final data were compared with data from CPR teaching aids to evaluate the performance of CPR training videos in two-view and three-view configurations, as shown in Table 2-C. Figure 13 and Figure 14 The results showed that in the regression analysis, the three-view configuration outperformed the two-view configuration, with mean squared error and mean absolute error of 276.08 and 8.59, respectively. In addition, the three-view configuration also had a higher Kappa coefficient (0.63), precision (0.82), and F1 score (0.84). At the same time, the area under the receiver operating characteristic curve (ROC-AUC) of 0.82 also exceeded that of the two-view configuration (285.42 and 11.14). Although the recall of the two-view configuration was slightly higher (0.86), the three-view configuration was superior in overall performance.

[0162] like Figure 15 As shown

[0163] There is a negative correlation between compression frequency and compression depth. In the low-frequency range (0-100 compressions / minute), most compressions are standard and more effective. However, in the high-frequency range (120 compressions / minute and above), compression depth decreases, although some compressions still meet the standard. This variation is influenced by individual differences in rescuers' skills and environmental factors such as lighting and camera angles. Furthermore, changes in perspective have a negligible impact on compression depth; there is no significant difference between the two-view and three-view groups. Overall, lower compression frequencies tend to produce deeper compressions, while higher frequencies may result in insufficient compression depth due to time constraints.

[0164] like Figure 4 As shown

[0165] A machine vision-based multi-angle CPR training video detection method, which includes the following components:

[0166] Small object optimization: The LCSG-YOLO model improves the detection sensitivity of key cardiopulmonary resuscitation areas, such as chest compressions and artificial respiration, by improving the convolutional layers and integrating local contextual information. These improvements reduce missed detections and false alarms, ensuring reliable identification of small or occluded objects.

[0167] KCHT matching algorithm: The KCHT algorithm combines Kalman filtering and Hungarian matching algorithm, and introduces confidence index and temporal features to ensure efficient target matching and tracking in multi-angle environments. Kalman filtering can refine target trajectory, reduce noise, and improve matching accuracy in dynamic scenes.

[0168] Cardiopulmonary resuscitation (CPR) frequency detection: The CPR frequency is accurately detected through time-based averaging, grayscale and jitter feature analysis, and frequency domain analysis using fast Fourier transform. These techniques can accurately identify and calculate the frequency of chest compressions and artificial respiration.

[0169] Cardiopulmonary resuscitation depth assessment: Compression depth is determined using methods such as signal smoothing, period detection, peak-to-trough difference measurement, and average depth analysis. These methods ensure compliance with cardiopulmonary resuscitation standards and provide reliable depth assessment.

[0170] Multi-angle fusion: In multi-angle scenarios, the dynamic weight allocation mechanism adjusts the weights according to the duration of the viewpoint and the video frame rate, ensuring accurate recognition of CPR actions even when angle information is limited.

[0171] Cardiopulmonary resuscitation (CPR) training video module: Integrates and analyzes the results, assesses indicators such as the number of compressions, frequency, and depth, and automatically determines whether each indicator meets international CPR standards, providing a comprehensive assessment of operational compliance.

[0172] By combining the YOLO11 model with the KH matching algorithm, the LCSG-YOLO model and the KCHT matching algorithm are proposed.

[0173] The LCSG-YOLO model is an improvement on YOLO11, specifically optimized for key actions and target detection in CPR training videos. It has local context sensitivity, enhances feature extraction, and improves the detection accuracy of key cardiopulmonary resuscitation areas.

[0174] The KCHT matching algorithm is an improvement on the KH matching algorithm, featuring keypoint detection and context-level tracking, aiming to more accurately track and analyze motion trajectories in CPR training videos.

[0175] This approach provides a cost-effective and flexible standard for CPR training videos and assessments, offering a practical and innovative solution for CPR training and evaluation.

[0176] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. A multi-angle CPR training video detection method based on machine vision, characterized in that: This includes establishing the LCGS-YOLO model, preparatory work, designing the KCHT matching algorithm, and developing scoring criteria; The establishment of the LCGS-YOLO model includes the design of the local condition sharing feature sharing layer, the design of the GCSCA module, and the design of the GCSCA optical module. The local condition sharing feature sharing layer is designed to enhance the local and global information in the input feature map by integrating multi-scale context features. The local condition sharing feature sharing layer uses adaptive pooling technology to extract global and local context and adopts learnable weights. The input feature map x undergoes adaptive pooling at two scales. ； Channel reduction is performed using 1x1 convolutional layers to generate shared feature representations. It integrates contextual information at different scales and normalizes it using the Sigmoid activation function to generate a channel attention map; ； Bilinear interpolation is used to upsample the local conditional shared feature sharing layer to match its spatial dimension with the input feature map x. ； Input feature map x and shared feature map The elements are multiplied one by one and weighted by learnable weights to generate the output of the local conditional sharing feature sharing layer. ； In the CSCA module design, the GCSCA (Global Context and Spatial Context Attention) module employs a self-attention mechanism to capture global and spatial context information; The GSCCA optical module design simplifies and enhances the GSCCA module, creating the GSCCA Light module; The preparatory work includes creating a dedicated dataset and establishing a grayscale axis standard for CPR training videos; The KCHT matching algorithm design includes target dynamic prediction, observation update, application of the Hungarian algorithm, and confidence correction mechanism; Target dynamic prediction: The KCHT algorithm uses a Kalman filter to dynamically predict the target's position, velocity, and other state information, generating a prediction for each target; Observation update: The KC filter updates the target state using the YOLO detection results, compares the predicted position with the detected position, and adjusts the state to make the prediction consistent with the actual target state; Application of the Hungarian algorithm: The Hungarian algorithm is used for temporal target matching, which associates targets in consecutive frames through optimal matching; Confidence correction mechanism: When confusion or inconsistency occurs, the KCHT algorithm applies weights to correct the predictions; The established scoring criteria include the Cohen's Kappa coefficient and an assessment of classification accuracy.

2. The multi-angle CPR training video detection method based on machine vision according to claim 1, characterized in that: The GCSCA (Global Context and Spatial Context Attention) module is a self-attention mechanism that captures global and spatial context information, enhancing the model's feature representation capabilities and robustness. The GCSCA module contains the following components: Channel attention: Adaptively adjusts the weight of each channel, highlighting the features of key channels and enhancing attention to local regions and important targets; ； ； Spatial attention: Spatial attention weights spatial features; ； ； Global context convolutional layer: Aggregates global information to enhance the model's global awareness; ； Feature fusion and residual linking: A feature fusion and residual linking strategy is adopted.

3. The multi-angle CPR training video detection method based on machine vision according to claim 1, characterized in that: The creation of the dedicated dataset involved selecting 40% of the collected videos, totaling 105 segments. Based on the complexity of the actions and key events, 20 to 30 keyframes were extracted from each video. This process ultimately produced 2,156 labeled frames, which were divided into a training set of 1,946 frames and a validation set of 210 frames. The establishment of the grayscale standard for CPR training videos: Grayscale values ​​of chest compressions and artificial respiration corresponding to key labels were extracted from the CPR research dataset. These values ​​were used to create a 10-level grayscale scale to facilitate the identification and counting of chest compressions and the detection of compression depth.

4. The multi-angle CPR training video detection method based on machine vision according to claim 1, characterized in that: The Cohen's Kappa coefficient is a statistical indicator used to assess the difference between classification prediction accuracy and randomness. It is suitable for evaluating the consistency between two evaluators or classifiers on classification tasks, even if these classification tasks involve randomness. ； ； ； The assessment of classification accuracy is expressed as the percentage of samples that pass classification and those that fail classification out of all samples. Accuracy is calculated using the following formula: 。 5. A multi-angle CPR video detection method based on machine vision, characterized in that: The machine vision-based multi-angle CPR training video detection method as described in claim 1 comprises the following components: Small object optimization: The LCSG-YOLO model improves the detection sensitivity of key cardiopulmonary resuscitation areas by improving the convolutional layers and integrating local contextual information; KCHT matching algorithm: The KCHT algorithm combines Kalman filtering and Hungarian matching algorithm, and introduces confidence index and time series features to achieve target pairing and tracking in multi-angle environments; Cardiopulmonary resuscitation (CPR) frequency detection: Based on time-based averaging, grayscale and jitter feature analysis, and fast Fourier transform for frequency domain analysis, the CPR frequency is accurately detected. Cardiopulmonary resuscitation depth detection: Compression depth is determined using signal smoothing, period detection, peak-to-valley difference measurement, and average depth analysis methods. Multi-angle fusion: In multi-angle scenes, a dynamic weight allocation mechanism adjusts weights based on the duration of the viewpoint and the video frame rate; Cardiopulmonary resuscitation (CPR) training video module: Integrates and analyzes the results, assesses indicators including the number of compressions, frequency, and depth, and automatically determines whether each indicator meets international CPR standards.

Images