CPR real-time guiding method and system, first-aid device and readable storage medium

Abstract

The invention discloses a CPR real-time guiding method and system, a first-aid device and a readable storage medium. The method comprises the steps that a human body posture point physical model and a marker physical model based on a marker worn by a CPR rescuer are established; a CPR rescuer wears the marker to carry out CPR external chest compression rescue, and the external chest compression process and a video image of the marker are visually collected; on the basis of the marker physical model, the marker is used as a visual reference point, visual acquisition distortion of the marker is calibrated, and mapping projection of an image scale and a real scale is carried out through pre-calibration information of the marker; the CPR external chest compression index is calculated on the basis of the combined real-time tracking result. The system comprises a visual collector, a processor and a prompter. By means of deployment and application of the scheme, a rescuer can be guided and taught more accurately and timely during CPR external chest compression, the operation level of the rescuer is enhanced, the rescuer can apply standard CPR external chest compression to a patient, and the life of the patient is saved in time.

Description

Technical Field

[0001] The present invention relates to the technical field of cardiopulmonary resuscitation, and in particular to a CPR real-time guidance method, system, first aid device, and readable storage medium. Background Art

[0002] The statements in this section merely provide background technical information related to the present application and do not necessarily constitute prior art.

[0003] Chest compression is of irreplaceable importance in cardiopulmonary resuscitation. In the case of cardiac arrest, chest compression is the only available means to maintain cardiac circulation and ensure that oxygen and nutrients are delivered to critical organs such as the brain and heart. Through chest compression, a certain hemodynamics can be maintained, the survival window of cardiac arrest patients can be extended, and precious time can be bought for other first aid measures and further medical intervention. In addition, chest compression can also promote the heart to re-establish autonomous heart rhythm and improve the patient's survival rate. Therefore, chest compression is regarded as a vital step in the first aid process of cardiac arrest, and its timely and effective implementation can greatly improve the patient's prognosis and survival rate.

[0004] Using machine vision technology to assist chest compression is a potential method. By integrating cameras and image processing systems, it can effectively combine posture recognition technology to capture the rescuer's behavior trajectory, including the compression posture, frequency and number of chest compressions. The application of this technology can provide personalized guidance for the automatic external defibrillator (AED) system, through voice prompts and screen display of specific operating steps, thereby improving the accuracy and efficiency of rescue.

[0005] The depth of chest compression is a key parameter that will directly affect the effect of cardiopulmonary resuscitation. However, traditional visual technology methods have some limitations, which affect their application in AED systems. Among them, the most important influencing factor is the inability to obtain the conversion from pixel scale to real physical scale, which makes it impossible to directly estimate the depth of chest compression by the rescuer through conventional methods, and thus it is impossible to provide accurate CPR guidance, which affects the prognosis and survival rate.

[0006] In view of this, how to solve the problem in the existing machine vision-assisted chest compression technology that the rescuer's chest compression depth cannot be directly estimated by conventional methods has become a topic to be studied and solved by the present invention. Summary of the invention

[0007] The purpose of the present invention is to provide a CPR real-time guidance method, system, first aid device, and readable storage medium.

[0008] To achieve the above object, a first aspect of the present invention provides a CPR real-time guidance method, the method comprising:

[0009] Establish a physical model of human body posture points and a physical model of markers based on the markers worn by CPR rescuers;

[0010] The CPR rescuer wears a marker while performing CPR chest compressions, and visually captures video images of the chest compression process and the marker;

[0011] Detect human body posture points based on the physical model of human body posture points;

[0012] Based on the physical model of the marker, the marker is used as a visual reference point to obtain pre-calibration information, and the visual acquisition distortion of the marker is calibrated. The image scale and the real scale are mapped and projected through the pre-calibration information of the marker.

[0013] The rapidly changing dynamic trajectory changes of the human body and the dynamic trajectory changes of the markers are tracked separately, the movement trajectory of the CPR rescuer is tracked in real time according to the human body posture points and the visual reference points of the markers, and the CPR chest compression indicators are calculated based on the joint real-time tracking results. The CPR chest compression indicators include at least compression depth and one or more of the following: compression posture, compression frequency, and number of compressions.

[0014] The second aspect of the present invention provides a CPR real-time guidance system, the CPR real-time guidance system includes a visual collector, a processor, and a prompter;

[0015] The visual collector is used to collect video images of the rescuer's chest compression process and the identifier worn by the rescuer;

[0016] The processor is used to obtain video images of the rescuer's chest compression process and the marker worn by the rescuer collected by the visual collector, and detect the human body posture points based on the physical model of the human body posture points; use the marker as a visual reference point based on the physical model of the marker, obtain pre-calibration information, and calibrate the visual acquisition distortion of the marker, and perform mapping projection of the image scale and the real scale through the pre-calibration information of the marker; track the fast-changing dynamic trajectory changes of the human body and the dynamic trajectory changes of the marker respectively, and track the movement trajectory of the CPR rescuer in real time according to the human body posture points and the visual reference points of the marker, and calculate the CPR chest compression index based on the joint real-time tracking results. The CPR chest compression index includes at least one or more of the following: compression posture, compression frequency, number of compressions, and compression depth;

[0017] The prompter is used to provide text prompts, picture prompts or voice prompts for the CPR chest compression indicators.

[0018] A third aspect of the present invention provides an emergency treatment device, which includes the CPR real-time guidance system as described in the second aspect of the present invention.

[0019] A fourth aspect of the present invention provides a readable storage medium having a control program stored thereon. When the control program is executed by a processor, the processor executes the steps of the method described in the first aspect.

[0020] The relevant contents of the present invention are explained as follows:

[0021] 1. Through the implementation of the above technical solutions of the present invention, the existing machine vision-assisted chest compression technology cannot obtain the conversion from pixel scale to real physical scale, which makes it impossible to directly estimate the depth of chest compression by the rescuer through conventional methods, and thus it is impossible to provide accurate CPR guidance, which affects the prognosis and survival rate. Therefore, an innovative CPR real-time guidance method, system, first aid device, and readable storage medium are designed; in the CPR real-time guidance method, a specially designed physical model of a marker based on the marker worn by the CPR rescuer is used. When performing CPR chest compression rescue, the rescuer can wear the marker, and the video image of the chest compression process and the marker is collected by vision. In the application of the marker, the marker is used as a visual reference point based on the physical model of the marker. , obtain pre-calibration information, and calibrate the visual acquisition distortion of the marker. The image scale and the real scale are mapped and projected through the pre-calibration information of the marker, so as to obtain the proportional coefficient of the pixel and the real scale transformation, and combine the recognition and tracking technology of the marker and the human skeleton point to further enhance the user positioning and the detection of moving targets. The combination of the two provides a new solution for the accuracy and reliability of the chest compression auxiliary system, mainly to obtain more accurate compression depth indicators and other CPR chest compression indicators, so that the rescuer can accurately perform the standard chest compression procedure according to the real-time prompts and feedback of the system, so that the CPR guidance process is more in line with medical standards, thereby improving the accuracy and efficiency of rescue, and also providing support for the patient's good prognosis.

[0022] 2. In the first aspect of the above technical solution, in the process of establishing the human body posture point physical model, a pre-trained weighted skeleton point detection model is used to detect the human body posture points, and the human body posture point physical model at least includes the joints in the rescuer's head, shoulders, elbows, palms, torso and legs. In this way, the joints of the key moving parts of the human body when the rescuer performs CPR chest compressions are covered, thereby improving the accuracy of the calculation of CPR chest compression indicators.

[0023] 3. In the first aspect of the above technical solution, in the process of establishing a physical model of a marker based on the marker worn by the CPR rescuer, a rectangular frame is used to establish a rectangular physical model of the pixels of the armband, and the area covered by the rectangular frame is the position of the armband when it is worn. The object recognition network is implemented using a deep convolutional network structure, and the position of the rescuer is accurately tracked after the marker is trained and identified. Through the implementation of this step, the marker is intended to provide a reliable visual reference point to help the machine vision system locate the rescuer more accurately; after the marker is trained and identified, the accurate tracking of the rescuer's position is ensured.

[0024] 4. In the first aspect of the above technical solution, the worn marker is identified as a rectangular frame box in the image, and the characteristics of the matrix frame can be represented by the center of the region matrix (x, y) and the width and height of the region rectangle (w, h), that is, box = (x, y, w, h);

[0025] In the process of calibrating the visual acquisition distortion of the marker, the rectangular physical model is reprojected using affine transformation. Affine transformation includes rotation and misalignment in the x-direction and y-direction to correct the visual distortion caused by the angle, so that the position and features of the marker can be accurately captured in the video image. The reason for adopting this step is that in the actual video acquisition process, due to the spatial angle between the camera and the human body, distortion may occur, affecting the visual recognition of the armband. To solve this problem, the rectangular physical model is reprojected using affine transformation. The main affine transformations involved include rotation and misalignment in the x-direction and misalignment in the y-direction to correct the visual distortion caused by the angle, thereby ensuring that the position and features of the armband are accurately captured in the video.

[0026] 5. In the first aspect of the above technical solution, in the process of mapping the image scale and the real scale through the pre-calibration information of the marker, the actual displacement distance of the marker is estimated by using the pre-calibration information of the marker and the principle of similar triangles, and the relationship between the actual height of the marker and the pixel height in the image is established. The relationship between the actual horizontal and vertical displacements of the marker and the horizontal and vertical displacements of the pixels in the image is established, so as to map the image scale and the real scale of the marker. Through the above steps, the pixel height of the calibration object in the image can be accurately obtained, and the displacement is calculated in combination with the known actual height, so that the displacement distance of the object in the real space can be effectively estimated.

[0027] 6. In the first aspect of the above technical solution, the fast-changing dynamic trajectory changes of the human body and the dynamic trajectory changes of the marker are tracked separately based on the hidden Markov hypothesis, the forward correlation of the action in the CPR process is established using the Markov hypothesis, and the constant speed model is used as the transition function to represent the chain propagation process of the behavior trajectory. Considering that in the CPR scenario, the recognition focus is on the cyclic arm compression movement. Therefore, the information across frames shows a high degree of correlation, and the action mainly occurs in a specific local area, and the position of the human body is relatively fixed. This feature contributes to the targeted enhancement of the local recognition accuracy of the posture recognition model specially customized for the CPR environment.

[0028] 7. In the first aspect of the above technical solution, in the process of tracking the fast-changing dynamic trajectory changes of the human body and the dynamic trajectory changes of the marker based on the hidden Markov hypothesis, the following steps are included:

[0029] The Markov hypothesis is used to establish the forward correlation of actions during CPR. The human posture model and object recognition model jointly represent the implicit compression state characteristics of the rescuer, which are:

[0030] The feature vector obtained from the physical model of human body posture points is used as the hidden state feature Where i represents the number of human body posture point vectors, and the human body posture point vectors include: human upper arm feature vector, human lower arm feature vector, human trunk feature vector, elbow bending angle, and shoulder and trunk bending angle;

[0031] Obtain the basic features of the recognition frame from the physical model of the marker Where j represents the number of marker vectors, including: (x, y, w, h, dx, dy, dw, dh), the center of the region matrix (x, y), the width and height of the region rectangle (w, h), and the change speed of each component in the image coordinate system;

[0032] Use the constant velocity model as the transition function to represent the chain propagation process of the behavior trajectory:

[0033]

[0034] in, and denote the system noise of human pose points and markers, respectively, expressed as zero-mean Gaussian noise with manually adjusted variance, and They represent the human body posture point feature vector and the marker feature vector at time t+1 (the next moment after the current time t), and They represent the human body posture point feature vector and the marker feature vector at time t-1 (the previous moment before the current time t), and They respectively represent the human body posture point feature vector and the marker feature vector at the current time t.

[0035] This step is adopted because the action of CPR is a continuous change in direction. This change is expressed as a chain propagation process. The current motion trajectory is only related to the motion hidden state of the previous moments, ensuring the smoothness of the motion trajectory.

[0036] 8. In the first aspect of the above technical solution, a joint real-time tracking of the CPR rescuer's motion trajectory for human posture points and markers is established based on Kalman filtering, and the joint representation of the human posture point state space and the marker state space is defined using Kalman filtering, and the state of the joint representation is updated and verified. In this way, the expression of the CPR rescuer's motion trajectory is more accurate, providing better guarantee for the subsequent calculation of chest compression quality indicators, and thus providing more accurate, standard and timely guidance for CPR real-time guidance.

[0037] 9. In the second aspect of the above technical solution, in the process of establishing a joint real-time tracking of the motion trajectory of the CPR rescuer for the human body posture points and the markers based on Kalman filtering, the following steps are included:

[0038] The Kalman filter is used to define the joint representation of the human body posture point state space and the marker state space, including the following:

[0039] Define the joint latent state space: It is a joint representation of the state space of human body posture points and the state space of markers;

[0040] Define the joint observation state space: The observation state is defined in an explicit way, so the representation dimension of the observation state space is the same as that of the implicit state vector space;

[0041] Define the prior joint process noise covariance:

[0042] Define the projection matrix from the observation state space to the hidden state space For the explicit transformation from observation space to state space, and based on the constant velocity model, it is expressed as Where I represents a diagonal matrix with diagonal elements of 1, and 0 represents a 0 matrix;

[0043] Define the state transition matrix Based on the isokinetic model, it can be expressed as Where T is represented by a diagonal matrix with diagonal elements dt, and dt represents the rate of change;

[0044] The state of the joint representation is updated and verified. The state update process of the Kalman filter is as follows:

[0045] Initialization parameters, for the state space Use the constant velocity model to estimate the change process of the motion trajectory and initialize each element of the state space to 0;

[0046] Estimating the prior state space in represents the prior estimate of the current state space, represents the prior estimate of the previous state space;

[0047] and estimate the prior covariance where F T represents the transpose operation of the state transfer matrix F;

[0048] Update Kalman gain in and Represent the prior covariance of the previous state space and the current state space, respectively. and They represent the projection matrix from the observed state space to the implicit state space and its transpose operation, is the prior joint process noise covariance;

[0049] Update the posterior state space Where ∈ represents the current observation error, represents the posterior estimate of the current state space, O k Represents the actual observation value of the current state, and the predicted observation value of the current state is estimated by the prior estimate of the current state space mentioned above and the projection matrix from the observed state space to the implicit state space Calculate, K represents the Kalman gain matrix;

[0050] And update the posterior covariance Where I represents the identity matrix, K is the aforementioned Kalman gain matrix, is the projection matrix from the aforementioned observed state space to the implicit state space, is the prior covariance of the current state.

[0051] 10. In the first aspect of the above technical solution, in the CPR chest compression index:

[0052] For pressing posture: evaluate and define the standard pressing posture. The evaluation criteria are: first, the angle θ1 between the arm and the horizontal plane, especially 85°≤θ1≤90°; second, the average arm angle θ2 (the angle between the upper arm and the lower arm) is estimated when the human body posture points are detected. If the average arm angle is 160°≤θ1≤180°, the arm is considered to be straight. If both meet the standards, it is considered to be a standard pressing posture;

[0053] For compression frequency: evaluate and define the standard compression frequency, use the human body posture point physical model to identify and record the rescuer's key bone points, which include the shoulder, arm and palm positions during compression. These bone points start to move downward together, and then the downward displacement continues until it reaches the lowest point and returns to the original height; the moment when the continuous downward displacement starts is marked as t1 and t2, and Δt represents the interval between these two moments, which constitutes a compression cycle, measured in seconds. The compression frequency is defined as 60seconds / Δt, which represents the number of compressions per minute. The evaluation standard stipulates that the compression frequency is in the range of 100 to 120 times per minute;

[0054] For the number of compressions: in the process of identifying the compression cycle of recording the compression frequency, each compression cycle is also counted and recorded as the number of compressions;

[0055] For compression depth: evaluate and define the standard compression depth, use the physical model of human posture points to identify and record the positions of the rescuer's shoulders, arms and palms during compression, and after the operator's palms are aligned with the simulated individual, the moment when the center points of the shoulders, arms and palms begin to move downward is marked as the highest point. As the movement proceeds downward to the lowest point and is about to start moving upward, this moment is recorded as the lowest point. The displacement is projected and scaled using the depth of field scale in the physical model of the marker and is defined as the difference ΔA between the highest point and the lowest point. The standard for compression depth assessment is 5cm≤ΔA≤6cm.

[0056] 11. In the present invention, unless otherwise clearly defined and specified, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0057] 12. In the present invention, the orientation or position relationship indicated by the terms "center", "upper", "lower", "axial", "bottom", "inner", "outer", etc. is based on the orientation or position assembly relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application.

[0058] 13. In addition, the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0059] 14. Also, the term “and / or” in this application means three parallel schemes, taking “obtaining the current measurement result information of the current user’s blood pressure and / or pulse” as an example, including the scheme of “obtaining the current measurement result information of the current user’s blood pressure”, or the scheme of “obtaining the current measurement result information of the current user’s pulse”, or the scheme of “obtaining the current measurement result information of the current user’s blood pressure and pulse” that are satisfied at the same time.

[0060] Due to the application of the above scheme, the present invention has the following advantages and effects compared with the prior art:

[0061] In the scheme of the present invention, in the CPR real-time guidance method, a specially designed physical model of a marker based on a marker worn by a CPR rescuer is used. When performing CPR chest compression rescue, the rescuer can wear the marker, and visually capture the video image of the chest compression process and the marker. In the application of the marker, the marker is used as a visual reference point based on the physical model of the marker to obtain pre-calibration information, and the visual acquisition distortion of the marker is calibrated. The image scale and the real scale are mapped and projected through the pre-calibration information of the marker, so as to obtain the proportional coefficient of the pixel and the real scale transformation, and combined with the recognition and tracking technology of the marker and the human skeleton point, the user positioning and the detection of the moving target are further enhanced. The combination of the two provides a new solution for the accuracy and reliability of the chest compression auxiliary system, mainly to obtain more accurate compression depth indicators and other CPR chest compression indicators, so that the rescuer can accurately perform the standard chest compression procedure according to the real-time prompts and feedback of the system, so that the CPR guidance process is more in line with medical standards, thereby improving the accuracy and efficiency of the rescue, and also providing assistance for the good prognosis of the patient. Through the deployment and application of the CPR real-time guidance system and first aid equipment, rescuers can receive more accurate and timely guidance and teaching when performing CPR chest compressions, thereby improving their operational level and enabling them to perform standard CPR chest compressions on patients and save their lives in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Attached Figure 1 Schematic diagram of the process of the CPR real-time guidance method in an embodiment of the present invention;

[0063] Attached Figure 2 A schematic diagram of chest compressions tested using a simulated family member in an embodiment of the present invention;

[0064] Attached Figure 3 It is a schematic diagram of re-projecting a rectangular physical model using affine transformation in an embodiment of the present invention;

[0065] Attached Figure 4 The figure is a flowchart of an embodiment of the present invention in actual operation. DETAILED DESCRIPTION

[0066] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present application, so the present application is not limited by the specific embodiments disclosed below.

[0067] The present invention aims to address the problem that the existing machine vision-assisted chest compression technology cannot obtain the conversion from pixel scale to real physical scale, which makes it impossible to directly estimate the rescuer's chest compression depth through conventional methods, and thus cannot provide accurate CPR guidance, affecting the prognosis and survival rate. The present invention innovatively designs a CPR real-time guidance method, system, first aid device, and readable storage medium.

[0068] Embodiment 1: Embodiment 1 of the present invention proposes a CPR real-time guidance method, the method comprising:

[0069] Establish a physical model of human body posture points and a physical model of markers based on the markers worn by CPR rescuers;

[0070] The CPR rescuer wears a marker while performing CPR chest compressions, and visually captures video images of the chest compression process and the marker;

[0071] Detect human body posture points based on the physical model of human body posture points;

[0072] Based on the physical model of the marker, the marker is used as a visual reference point to obtain pre-calibration information, and the visual acquisition distortion of the marker is calibrated. The image scale and the real scale are mapped and projected through the pre-calibration information of the marker.

[0073] The rapidly changing dynamic trajectory changes of the human body and the dynamic trajectory changes of the markers are tracked separately, the movement trajectory of the CPR rescuer is tracked in real time according to the human body posture points and the visual reference points of the markers, and the CPR chest compression indicators are calculated based on the joint real-time tracking results. The CPR chest compression indicators include at least compression depth and one or more of the following: compression posture, compression frequency, and number of compressions.

[0074] In the CPR real-time guidance method, through a specially designed physical model of a marker based on a marker worn by a CPR rescuer, the rescuer can wear a marker when performing CPR chest compression rescue, and visually capture the video image of the chest compression process and the marker. In the application of the marker, the marker is used as a visual reference point based on the physical model of the marker to obtain pre-calibration information, and the visual acquisition distortion of the marker is calibrated. The image scale and the real scale are mapped and projected through the pre-calibration information of the marker, so as to obtain the proportional coefficient of the pixel and the real scale transformation, and combined with the recognition and tracking technology of the marker and the human skeleton point, the user positioning and the detection of the moving target are further enhanced. The combination of the two provides a new solution for the accuracy and reliability of the chest compression auxiliary system, mainly to obtain more accurate compression depth indicators and other CPR chest compression indicators, so that the rescuer can accurately perform the standard chest compression procedure according to the real-time prompts and feedback of the system, so that the CPR guidance process is more in line with medical standards, thereby improving the accuracy and efficiency of the rescue, and also providing support for the good prognosis of the patient.

[0075] In one implementation of the first embodiment of the present invention, in the process of establishing a physical model of human posture points, a pre-trained weighted skeleton point detection model is used to detect human posture points, and the physical model of human posture points at least includes joints in the rescuer's head, shoulders, elbows, palms, torso and legs. In this way, the joints of the key moving parts of the human body when the rescuer performs CPR chest compressions are covered, thereby improving the accuracy of the calculation of CPR chest compression indicators.

[0076] In another implementation of Example 1 of the present invention, in the process of establishing a physical model of a marker based on a marker worn by a CPR rescuer, a rectangular frame is used to establish a rectangular physical model of the pixels of the armband, and the area covered by the rectangular frame is the position of the armband when it is worn. A deep convolutional network structure is used to implement an object recognition network, and the position of the rescuer is accurately tracked after the marker is trained and identified. Through the implementation of this step, the marker is intended to provide a reliable visual reference point to help the machine vision system locate the rescuer more accurately; the marker is trained and identified to ensure accurate tracking of the rescuer's position.

[0077] Specifically, the worn marker is identified as a rectangular box in the image, and the features of the matrix box can be represented by the center of the region matrix (x, y) and the width and height of the region rectangle (w, h), that is, box = (x, y, w, h);

[0078] In the process of calibrating the visual acquisition distortion of the marker, the rectangular physical model is reprojected using affine transformation. Affine transformation includes rotation and misalignment in the x-direction and y-direction to correct the visual distortion caused by the angle, so that the position and features of the marker can be accurately captured in the video image. The reason for adopting this step is that in the actual video acquisition process, due to the spatial angle between the camera and the human body, distortion may occur, affecting the visual recognition of the armband. To solve this problem, the rectangular physical model is reprojected using affine transformation. The main affine transformations involved include rotation and misalignment in the x-direction and misalignment in the y-direction to correct the visual distortion caused by the angle, thereby ensuring that the position and features of the armband are accurately captured in the video.

[0079] Furthermore, in the process of mapping the image scale and the real scale through the pre-calibration information of the marker, the actual displacement distance of the marker is estimated by using the pre-calibration information of the marker and the principle of similar triangles, and the relationship between the actual height of the marker and the pixel height in the image is established. The relationship between the actual horizontal and vertical displacements of the marker and the horizontal and vertical displacements of the pixels in the image is established, so as to map the image scale and the real scale of the marker. Through the above steps, the pixel height of the calibration object in the image can be accurately obtained, and the displacement is calculated in combination with the known actual height, so that the displacement distance of the object in the real space can be effectively estimated.

[0080] In one implementation of Example 1 of the present invention, the fast-changing dynamic trajectory changes of the human body and the dynamic trajectory changes of the marker are tracked separately based on the hidden Markov hypothesis, the forward correlation of the action in the CPR process is established using the Markov hypothesis, and the constant speed model is used as the transition function to represent the chain propagation process of the behavior trajectory. Considering that in the CPR scenario, the recognition focus is on the cyclic arm compression movement. Therefore, the information across frames shows a high degree of correlation, and the action mainly occurs in a specific local area, and the position of the human body is relatively fixed. This feature helps to enhance the local recognition accuracy of the posture recognition model specially customized for the CPR environment.

[0081] Specifically, in the process of tracking the fast-changing dynamic trajectory changes of the human body and the dynamic trajectory changes of the marker based on the hidden Markov hypothesis, the following steps are included:

[0082] The Markov hypothesis is used to establish the forward correlation of actions during CPR. The human posture model and object recognition model jointly represent the implicit compression state characteristics of the rescuer, which are:

[0083] The feature vector obtained from the physical model of human body posture points is used as the hidden state feature Where i represents the number of human body posture point vectors, and the human body posture point vectors include: human upper arm feature vector, human lower arm feature vector, human trunk feature vector, elbow bending angle, and shoulder and trunk bending angle;

[0084] Obtain the basic features of the recognition frame from the physical model of the marker Among them, represents the quantity represented by the marker vector, including: (x, y, w, h, dx, dy, dw, dh), the center of the region matrix (x, y), the width and height of the region rectangle (w, h), and the change speed of each component in the image coordinate system;

[0085] Use the constant velocity model as the transition function to represent the chain propagation process of the behavior trajectory:

[0086]

[0087] in, and denote the system noise of human pose points and markers, respectively, expressed as zero-mean Gaussian noise with manually adjusted variance, and They represent the human body posture point feature vector and the marker feature vector at time t+1 (the next moment after the current time t), and They represent the human body posture point feature vector and the marker feature vector at time t-1 (the previous moment before the current time t), and They respectively represent the human body posture point feature vector and the marker feature vector at the current time t.

[0088] This step is adopted because the action of CPR is a continuous change in direction. This change is expressed as a chain propagation process. The current motion trajectory is only related to the motion hidden state of the previous moments, ensuring the smoothness of the motion trajectory.

[0089] In another implementation of the first embodiment of the present invention, a joint real-time tracking of the motion trajectory of the CPR rescuer for human posture points and markers is established based on Kalman filtering, and a joint representation of the human posture point state space and the marker state space is defined using Kalman filtering, and the state of the joint representation is updated and verified. In this way, the expression of the motion trajectory of the CPR rescuer is made more accurate, providing better guarantee for the subsequent calculation of the chest compression quality index, thereby providing more accurate, standard and timely guidance for CPR real-time guidance.

[0090] Specifically, in the process of establishing a joint real-time tracking of the motion trajectory of the CPR rescuer based on the human body posture points and the markers based on the Kalman filter, the following steps are included:

[0091] The Kalman filter is used to define the joint representation of the human body posture point state space and the marker state space, including the following:

[0092] Define the joint latent state space: It is a joint representation of the state space of human body posture points and the state space of markers;

[0093] Define the joint observation state space: The observation state is defined in an explicit way, so the representation dimension of the observation state space is the same as that of the implicit state vector space;

[0094] Define the prior joint process noise covariance:

[0095] Define the projection matrix from the observation state space to the hidden state space For the explicit transformation from observation space to state space, and based on the constant velocity model, it is expressed as Where I represents a diagonal matrix with diagonal elements of 1, and 0 represents a 0 matrix;

[0096] Define the state transition matrix Based on the isokinetic model, it can be expressed as Where T is represented by a diagonal matrix with diagonal elements dt, and dt represents the rate of change;

[0097] The state of the joint representation is updated and verified. The state update process of the Kalman filter is as follows:

[0098] Initialization parameters, for the state space Use the constant velocity model to estimate the change process of the motion trajectory and initialize each element of the state space to 0;

[0099] Estimating the prior state space in represents the prior estimate of the current state space, represents the prior estimate of the previous state space;

[0100] and estimate the prior covariance where F T represents the transpose operation of the state transfer matrix F;

[0101] Update Kalman gain in and Represent the prior covariance of the previous state space and the current state space, respectively. and They represent the projection matrix from the observed state space to the implicit state space and its transpose operation, is the prior joint process noise covariance;

[0102] Update the posterior state space Where ∈ represents the current observation error, represents the posterior estimate of the current state space, O k Represents the actual observation value of the current state, and the predicted observation value of the current state is estimated by the prior estimate of the current state space mentioned above and the projection matrix from the observed state space to the implicit state space Calculate, K represents the Kalman gain matrix;

[0103] And update the posterior covariance Where I represents the identity matrix, K is the aforementioned Kalman gain matrix, is the projection matrix from the aforementioned observed state space to the implicit state space, is the prior covariance of the current state.

[0104] In another implementation of Example 1 of the present invention, in the CPR chest compression index:

[0105] For pressing posture: evaluate and define standard pressing posture. The evaluation criteria are: first, the angle θ1 between the arm and the horizontal plane, especially 85°≤θ1≤90°; second, the average arm angle when detecting human body posture points.

[0106] (The angle between the upper arm and the lower arm) 2 is estimated. If the average arm angle is 160°≤θ1≤180°, the arm is considered to be straight. If both meet the standard, it is considered to be a standard pressing posture;

[0107] For compression frequency: evaluate and define the standard compression frequency, use the human body posture point physical model to identify and record the rescuer's key bone points, which include the shoulder, arm and palm positions during compression. These bone points start to move downward together, and then the downward displacement continues until it reaches the lowest point and returns to the original height; the moment when the continuous downward displacement starts is marked as t1 and t2, and Δt represents the interval between these two moments, which constitutes a compression cycle, measured in seconds. The compression frequency is defined as 60seconds / Δt, which represents the number of compressions per minute. The evaluation standard stipulates that the compression frequency is in the range of 100 to 120 times per minute;

[0108] For the number of compressions: in the process of identifying the compression cycle of recording the compression frequency, each compression cycle is also counted and recorded as the number of compressions;

[0109] For compression depth: evaluate and define the standard compression depth, use the physical model of human posture points to identify and record the positions of the rescuer's shoulders, arms and palms during compression, and after the operator's palms are aligned with the simulated individual, the moment when the center points of the shoulders, arms and palms begin to move downward is marked as the highest point. As the movement proceeds downward to the lowest point and is about to start moving upward, this moment is recorded as the lowest point. The displacement is projected and scaled using the depth of field scale in the physical model of the marker and is defined as the difference ΔA between the highest point and the lowest point. The standard for compression depth assessment is 5cm≤ΔA≤6cm.

[0110] Embodiment 2, Embodiment 2 of the present invention discloses a CPR real-time guidance system, the CPR real-time guidance system includes a visual collector, a processor, and a prompter;

[0111] The visual collector is used to collect video images of the rescuer's chest compression process and the identifier worn by the rescuer;

[0112] The processor is used to obtain video images of the rescuer's chest compression process and the marker worn by the rescuer collected by the visual collector, and detect the human body posture points based on the physical model of the human body posture points; use the marker as a visual reference point based on the physical model of the marker, obtain pre-calibration information, and calibrate the visual acquisition distortion of the marker, and perform mapping projection of the image scale and the real scale through the pre-calibration information of the marker; track the fast-changing dynamic trajectory changes of the human body and the dynamic trajectory changes of the marker respectively, and track the movement trajectory of the CPR rescuer in real time according to the human body posture points and the visual reference points of the marker, and calculate the CPR chest compression index based on the joint real-time tracking results. The CPR chest compression index includes at least one or more of the following: compression posture, compression frequency, number of compressions, and compression depth;

[0113] The prompter is used to provide text prompts, picture prompts or voice prompts for the CPR chest compression indicators.

[0114] Embodiment 3 of the present invention discloses a first aid device, which includes the CPR real-time guidance system as described in Embodiment 2 of the present invention, that is, it includes a visual collector, a processor, and a prompter, and its structure and principle are not described in detail here.

[0115] Embodiment 4: Embodiment 4 of the present invention further discloses a readable storage medium, on which a control program is stored. When the control program is executed by a processor, the processor executes the steps of the method described in Embodiment 1.

[0116] Those skilled in the art can understand that the operation of the main control unit in the above-mentioned embodiment method can be completed by instructing the relevant hardware (blood pressure detection unit) through a program, and the program is stored in a readable storage medium, including a number of instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform the operation performed by the main control unit in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0117] The following is an example of the actual application process of the above embodiment of the present invention.

[0118] As attached Figure 4 As shown, the use process of the CPR real-time guidance method in the embodiment of the present invention includes:

[0119] 1. Use the posture detection model to establish a physical model of human posture points;

[0120] 2. Use the object recognition model to establish the physical model of the marker, and use affine transformation to calibrate the angle;

[0121] 3. Map the image scale and the real scale through the pre-calibration information of the marker;

[0122] 4. Based on the hidden Markov hypothesis, the dynamic trajectory changes of the human body and the dynamic trajectory changes of the markers are tracked respectively;

[0123] 5. Based on Kalman filtering, the movement trajectory of the CPR rescuer is tracked according to the human posture points and markers;

[0124] 6. Based on the combined tracking results, the overall CPR chest compression indicators are calculated, including compression posture, compression depth, compression frequency, and number of compressions.

[0125] For more details, please refer to the following:

[0126] 1. Use the posture detection model to establish a physical model of human posture points:

[0127] A public pre-trained weighted skeleton point detection model is used to detect key points of the human body. For public models with different key point detections, such as 15 points, 18 points, and 33 points, it is ensured that the posture recognition model for chest compression recognition contains at least the joint points in the rescuer's head, shoulder, elbow, palm, torso, and leg areas.

[0128] When calculating the angle between the main trunks of the human body, the cosine angle is used, and the calculation method is as follows:

[0129]

[0130] in Represents two vectors that form an angle.

[0131] 2. Use the object recognition model to build a physical model of the marker, and use affine transformation to calibrate the angle:

[0132] In order to enhance the machine vision recognition and tracking of the rescuer's location, a physical accessory marker is designed, specifically a special pattern armband worn on the lower or upper arm. The marker is intended to provide a reliable visual reference point to help the machine vision system locate the rescuer more accurately.

[0133] The pixels of the armband are modeled using a rectangular box, and the area covered is the position of the armband when it is worn. The object recognition network is implemented by using a deep convolutional network structure. The markers are trained and recognized to ensure accurate tracking of the rescuer's position.

[0134] The worn marker is identified as a rectangular box in the image. The features of the matrix box can be represented by the center of the region matrix (x, y) and the width and height of the region rectangle (w, h), that is, box = (x, y, w, h).

[0135] In addition, in the actual video acquisition process, due to the spatial angle between the camera and the human body, distortion may occur, affecting the visual recognition of the armband. To solve this problem, the rectangular physical model is reprojected using affine transformation. The main affine transformations involved include rotation and shearing in the x direction and shearing in the y direction, such as Figure 3 shown.

[0136] After affine transformation calibration, visual distortion caused by angles is corrected to ensure that the position and features of the armband are accurately captured in the video. The application of markers further enhances the machine vision system's ability to accurately identify and track the rescuer's position, providing strong support for the performance improvement of the chest compression auxiliary system.

[0137] 3. Mapping the image scale and the real scale through the pre-calibration information of the marker:

[0138] In the field of computer vision and image processing, the actual displacement distance of an object can be estimated by using the pre-calibrated information of the marker and the principle of similar triangles. The above affine transformation can accurately obtain the pixel height of the calibration object in the image, and calculate the displacement in combination with the known actual height.

[0139] Assume that the actual height of the calibration object is H, and the pixel height in the image is h. It is known that the focal length of the camera is f, and the distance between the object and the camera is D. According to the principle of similar triangles, the relationship between the actual height of the object and the pixel height in the image can be expressed as:

[0140]

[0141] Therefore, the distance D between the object and the camera can be expressed as:

[0142]

[0143] After obtaining the distance d, the actual horizontal and vertical displacements can be calculated by the horizontal displacement Δx and vertical displacement Δy of the pixel point and the scaling.

[0144] The horizontal displacement and vertical displacement are proportional to the scale between them. The horizontal displacement ΔX and vertical displacement ΔY of an object in real space can be calculated by the following formula:

[0145]

[0146] The combined displacement of an object in the horizontal and vertical directions can be synthesized into the actual displacement ΔD by the Pythagorean theorem:

[0147]

[0148] By combining the known size of the calibration object and the principle of similar triangles, the displacement distance of the object in the actual space can be effectively estimated through the camera intrinsic parameters and pixel displacement.

[0149] 4. Based on the Hidden Markov Hypothesis, the dynamic trajectory changes of the human body and the dynamic trajectory changes of the markers are tracked separately:

[0150] Chest compressions are a series of rapid, cyclical movements in which the rescuer holds his arms vertically with his shoulders above his palms and applies pressure to the victim's chest using his shoulders, elbows, and wrists simultaneously.

[0151] When using machine vision, especially for bone point detection, executing the bone recognition algorithm on consecutive frames with rapidly changing motion trajectories may lead to cumulative errors in evaluating the CPR compression motion trajectory due to problems such as motion blur and error propagation in the heat map features.

[0152] Considering that in the CPR scenario, the recognition focus is on the cyclic arm compression movement. Therefore, the information across frames shows a high degree of correlation, and the action mainly occurs in a specific local area, and the position of the human body is relatively fixed. This feature helps to specifically enhance the local recognition accuracy of the posture recognition model customized for the CPR environment.

[0153] The Markov hypothesis is used to establish the forward correlation of actions during CPR, and the isokinetic model is used as a transition function to control the evolution of human posture points over time. The main basic elements of Markov are defined as follows:

[0154] (1) Implicit state: used to represent the implicit compression state feature representation of the rescuer. Specifically, it is represented by the human posture model and the object recognition model, which are:

[0155] The human body posture point model obtains the feature vector as the hidden state feature Where i represents the number of human body posture point vectors, including: human upper arm feature vector, human lower arm feature vector, human trunk feature vector, elbow bending angle, and shoulder and trunk bending angle.

[0156] Basic features of the recognition box for marker model tracking Where j represents the quantity represented by the marker vector, including: (x, y, w, h, dx, dy, dw, dh), the center of the region matrix (x, y), the width and height of the region rectangle (w, h), and the change speed of each component in the image coordinate system.

[0157] (1) Transition function: The transition function controls how the hidden state evolves over time, and uses the constant velocity model as the transition function to represent the chain propagation process of the behavior trajectory.

[0158]

[0159] in, and They represent the system noise of human posture points and markers, respectively, and are expressed as zero-mean Gaussian noise with manually adjusted variance. In the above assumptions, the action of CPR is a continuous change in direction, and it is considered that this change is manifested as a chain propagation process. The current motion trajectory is only related to the motion hidden state of the previous moments, ensuring the smoothness of the motion trajectory.

[0160] 5. Based on Kalman filtering, the movement trajectory of the CPR rescuer is tracked according to the human posture points and markers;

[0161] Use Kalman filtering to establish joint real-time tracking of human posture points and markers as follows:

[0162] (1) Define the joint implicit state space: It is a joint representation of the human body posture point state space and the marker state space.

[0163] (2) Define the joint observation state space: The observation state is defined in an explicit way, so the representation dimension of the observation state space is the same as that of the implicit state vector space.

[0164] (3) Define the prior joint process noise covariance:

[0165] (4) Define the projection matrix from the observed state space to the implicit state space For the explicit transformation from observation space to state space, and based on the constant velocity model, it is expressed as Where I represents a diagonal matrix with diagonal elements of 1, and 0 represents a 0 matrix.

[0166] (5) Define the state transfer matrix Based on the isokinetic model, it can be expressed as Where T is represented as a diagonal matrix with diagonal elements dt, dt represents the rate of change, which is related to the FPS of image acquisition.

[0167] The state update process of Kalman filter is as follows:

[0168] (1) Initialization parameters

[0169] For the state space The constant velocity model is used to estimate the change process of the motion trajectory, so each element of the state space is initialized to 0.

[0170] (2) Estimation of prior state space

[0171]

[0172] in represents the prior estimate of the current spatial state, represents the prior estimate of the previous state space.

[0173] (3) and estimate the prior covariance

[0174]

[0175] where F T Represents the transpose operation of the state transfer matrix F.

[0176] (4) Update Kalman gain

[0177]

[0178] (5) Update the posterior state space

[0179]

[0180] Where ∈ represents the current observation error, represents the posterior estimate of the current state space.

[0181] (6) and update the posterior covariance

[0182]

[0183] 6. Calculate the overall CPR chest compression indicators based on the joint tracking results, including compression posture, compression depth, compression frequency, and number of compressions;

[0184] Compression posture: The standard CPR procedure requires the rescuer to position both arms vertically with the shoulders above the palms and apply pressure to the patient's chest using the shoulders, elbows, and wrists simultaneously. The operator's shoulder skeleton, arm skeleton points, and the position of the palm immersed in the patient's chest were tracked using a human posture model. The evaluation criteria were determined based on the angle θ1 perpendicular to the horizontal plane, specifically 85°≤θ1≤90°. Because effective CPR requires deep and repetitive compression movements, the arms should be straight, i.e., 85°≤θ1≤90°, to avoid premature fatigue, which is detrimental to performance. In posture estimation, the arms were considered straight if the average arm angle θ2 was above 170°, with a standard deviation below 5° to account for anatomical differences between participants.

[0185] Compression frequency: The human posture model is used to identify and record the operator's key skeletal points, including the position of the shoulder, arm, and palm when pressing. The moment when these skeletal points begin to move downward together is designated as t1. Subsequently, the downward displacement continues until it reaches the lowest point and returns to the original height. The moments when the downward displacement begins are marked as t1 and t2. Δt represents the interval between these two moments, which constitutes a compression cycle, measured in seconds. The compression frequency is defined as H60 / Δt, which represents the number of compressions per minute. The evaluation criteria stipulate that the compression frequency should be in the range of 100 to 120 times per minute.

[0186] Number of presses: In the above-mentioned recognition process of the press cycle, in addition to recording the press frequency, each press cycle is also counted.

[0187] Compression depth: The position of the operator's shoulder, arm, and palm during compression is identified and recorded using a human posture model. After the operator's palm is aligned with the simulated individual, the moment when the center point of the shoulder, arm, and palm begins to move downward is marked as the starting point. As the movement progresses downward to the lowest point and is about to begin moving upward, this moment is recorded as the lowest point. The displacement is projected and scaled using a depth of field scale and is defined as the difference ΔA between the highest and lowest points. The standard for CPR compression depth assessment has been established as 5cm≤ΔA≤6cm.

[0188] When the rescuer performs actual CPR chest compressions, he or she may refer to the following: Figure 4 The process shown is carried out.

[0189] refer to Figure 2 As shown, a simulated dummy is used for testing. The simulated dummy is equipped with pressure sensors, displacement sensors and other sensors for recording CPR chest compression indicators in the embodiment, which can measure the pressure and compression depth applied to the chest in real time, and simultaneously calculate the number of compressions and compression frequency. By comparing the measured values ​​of the simulated dummy sensors with the results of the method of the present invention, the comparison results are as follows:

[0190] For the compression depth index, the mean error was less than 5% (P<0.01);

[0191] For the index of compression depth times, the average number of errors was less than 1 (P<0.01);

[0192] For compression frequency, the mean error was less than 10% (P < 0.01);

[0193] Regarding pressing posture, our discrimination results were compared with the expert observation and evaluation, and the average error was less than 1 time / person (P<0.01).

[0194] Through the implementation of the above embodiments, through the deployment and application of the CPR real-time guidance system and first aid equipment, rescuers can receive more accurate and timely guidance and teaching when performing CPR chest compressions, thereby enhancing the rescuer's operational level and enabling the rescuer to apply standard CPR chest compressions to the patient, thereby saving the patient's life in time, thereby achieving the purpose of the present invention.

[0195] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A CPR real-time guidance method, characterized in that: The method comprises: Establish a physical model of human body posture points and a physical model of markers based on the markers worn by CPR rescuers; The CPR rescuer wears a marker while performing CPR chest compressions, and visually captures video images of the chest compression process and the marker; Detect human body posture points based on the physical model of human body posture points; Based on the physical model of the marker, the marker is used as a visual reference point to obtain pre-calibration information, and the visual acquisition distortion of the marker is calibrated. The image scale and the real scale are mapped and projected through the pre-calibration information of the marker. Track the fast-changing dynamic trajectory of the human body and the dynamic trajectory of the marker respectively, and track the movement trajectory of the CPR rescuer in real time based on the human body posture points and the visual reference points of the marker; The CPR chest compression index is calculated overall based on the combined real-time tracking results. The CPR chest compression index includes at least compression depth and one or more of the following: compression posture, compression frequency, and number of compressions.

2. The CPR real-time guidance method according to claim 1, characterized in that: In the process of establishing a physical model of human body posture points, a pre-trained weighted skeleton point detection model is used to detect human body posture points, and the physical model of human body posture points at least includes joint points in the rescuer's head, shoulders, elbows, palms, torso and legs.

3. The CPR real-time guidance method according to claim 2, characterized in that: In the process of establishing a physical model of the marker based on the marker worn by the CPR rescuer, a rectangular frame is used to establish a rectangular physical model of the pixels of the armband. The area covered by the rectangular frame is the position of the armband when it is worn. A deep convolutional network structure is used to implement the object recognition network. After training and identifying the marker, the rescuer's position is accurately tracked.

4. The CPR real-time guidance method according to claim 3, characterized in that: The worn marker is identified as a rectangular box in the image. The features of the matrix box can be represented by the center of the region matrix (x, y) and the width and height of the region rectangle (w, h), that is, box = (x, y, w, h); In the process of calibrating the visual acquisition distortion of the marker, the rectangular physical model is re-projected using affine transformation. Affine transformation includes rotation and misalignment in the x-direction and the y-direction to correct the visual distortion caused by the angle, so that the position and features of the marker can be accurately captured in the video image.

5. The CPR real-time guidance method according to claim 4, characterized in that: In the process of mapping the image scale and the actual scale through the pre-calibration information of the marker, the actual displacement distance of the marker is estimated by the pre-calibration information of the marker and the principle of similar triangles, and the relationship between the actual height of the marker and the pixel height in the image is established. The relationship between the actual horizontal and vertical displacements of the marker and the horizontal and vertical displacements of the pixels in the image is established, so as to map the image scale and the actual scale of the marker.

6. The CPR real-time guidance method according to claim 1, characterized in that: Based on the hidden Markov hypothesis, the fast-changing dynamic trajectory changes of the human body and the dynamic trajectory changes of the markers are tracked separately. The forward correlation of the actions in the CPR process is established using the Markov hypothesis, and the isokinetic model is used as the transition function to represent the chain propagation process of the behavior trajectory.

7. The CPR real-time guidance method according to claim 6, characterized in that: In the process of tracking the fast-changing dynamic trajectory changes of the human body and the dynamic trajectory changes of the marker based on the hidden Markov hypothesis, the following steps are included: The Markov hypothesis is used to establish the forward correlation of actions during CPR. The human posture model and object recognition model jointly represent the implicit compression state characteristics of the rescuer, which are: The feature vector obtained from the physical model of human body posture points is used as the hidden state feature Where i represents the number of human body posture point vectors, and the human body posture point vectors include: human upper arm feature vector, human lower arm feature vector, human trunk feature vector, elbow bending angle, and shoulder and trunk bending angle; Obtain the basic features of the recognition frame from the physical model of the marker Among them, represents the quantity represented by the marker vector, including: (x, y, w, h, dx, dy, dw, dh), the center of the region matrix (x, y), the width and height of the region rectangle (w, h), and the change speed of each component in the image coordinate system; Use the constant velocity model as the transition function to represent the chain propagation process of the behavior trajectory: in, and denote the system noise of human pose points and markers, respectively, expressed as zero-mean Gaussian noise with manually adjusted variance, and They represent the human body posture point feature vector and the marker feature vector at time t+1 respectively, and They represent the human body posture point feature vector and the marker feature vector at time t-1 respectively, and They respectively represent the human body posture point feature vector and the marker feature vector at the current time t.

8. The CPR real-time guidance method according to claim 1, characterized in that: Based on Kalman filtering, a joint real-time tracking system for human posture points and markers is established to track the motion trajectory of the CPR rescuer. Kalman filtering is used to define the joint representation of the human posture point state space and the marker state space, and the state of the joint representation is updated and verified.

9. The CPR real-time guidance method according to claim 8, characterized in that: The process of establishing a joint real-time tracking of the CPR rescuer's motion trajectory for human body posture points and markers based on Kalman filtering includes the following steps: The Kalman filter is used to define the joint representation of the human body posture point state space and the marker state space, including the following: Define the joint latent state space: It is a joint representation of the state space of human body posture points and the state space of markers; Define the joint observation state space: The observation state is defined in an explicit way, so the representation dimension of the observation state space is the same as that of the implicit state vector space; Define the prior joint process noise covariance: Define the projection matrix from the observation state space to the hidden state space For the explicit transformation from observation space to state space, and based on the constant velocity model, it is expressed as Where I represents a diagonal matrix with diagonal elements of 1, and 0 represents a 0 matrix; Define the state transition matrix Based on the isokinetic model, it can be expressed as Where T is represented by a diagonal matrix with diagonal elements dt, and dt represents the rate of change; The state of the joint representation is updated and verified. The state update process of the Kalman filter is as follows: Initialization parameters, for the state space Use the constant velocity model to estimate the change process of the motion trajectory and initialize each element of the state space to 0; Estimating the prior state space in represents the prior estimate of the current state space, represents the prior estimate of the previous state space; and estimate the prior covariance Wherein, T represents the transpose operation of the state transfer matrix F; Update Kalman gain in and Represent the prior covariance of the previous state space and the current state space, respectively. and They represent the projection matrix from the observed state space to the implicit state space and its transpose operation, is the prior joint process noise covariance; Update the posterior state space Where ∈ represents the current observation error, represents the posterior estimate of the current state space, O k Represents the actual observation value of the current state, and the predicted observation value of the current state is estimated by the prior estimate of the current state space mentioned above and the projection matrix from the observed state space to the implicit state space Calculate, K represents the Kalman gain matrix; And update the posterior covariance Where I represents the identity matrix, K is the aforementioned Kalman gain matrix, is the projection matrix from the aforementioned observed state space to the implicit state space, is the prior covariance of the current state.

10. The CPR real-time guidance method according to any one of claims 1 to 9, characterized in that: In the CPR chest compression indicators: For pressing posture: evaluate and define the standard pressing posture. The evaluation criteria are: first, the angle θ1 between the arm and the horizontal plane, especially 85°≤θ1≤90°; second, the average arm angle θ2 (the angle between the upper arm and the lower arm) is estimated when the human body posture points are detected. If the average arm angle is 160°≤θ1≤180°, the arm is considered to be straight. If both meet the standards, it is considered to be a standard pressing posture; For compression frequency: evaluate and define the standard compression frequency, use the human body posture point physical model to identify and record the rescuer's key bone points, which include the shoulder, arm and palm positions during compression. These bone points start to move downward together, and then the downward displacement continues until it reaches the lowest point and returns to the original height; the moment when the continuous downward displacement starts is marked as t1 and t2, and Δt represents the interval between these two moments, which constitutes a compression cycle, measured in seconds. The compression frequency is defined as 60seconds / Δt, which represents the number of compressions per minute. The evaluation standard stipulates that the compression frequency is in the range of 100 to 120 times per minute; For the number of compressions: during the recognition process of the compression cycle of recording the compression frequency, each compression cycle is also counted and recorded as the number of compressions; For compression depth: evaluate and define the standard compression depth, use the human posture point physical model to identify and record the position of the rescuer's shoulder, arm and palm during compression, after the operator's palm is aligned with the simulated individual, the moment when the center point of the shoulder, arm and palm begins to move downward is marked as the highest point, and as the movement moves downward to the lowest point and is about to start moving upward, this moment is recorded as the lowest point. The displacement is projected and scaled using the depth of field scale in the physical model of the marker, and is defined as the difference ΔA between the highest point and the lowest point. The standard for compression depth assessment is 5cm≤ΔA≤6cmm.

11. A CPR real-time guidance system, used for CPR real-time guidance in the method according to any one of claims 1 to 10, characterized in that: The CPR real-time guidance system includes a visual collector, a processor, and a prompter; The visual collector is used to collect video images of the rescuer's chest compression process and the identifier worn by the rescuer; The processor is used to obtain the video images of the rescuer's chest compression process and the identifier worn by the rescuer collected by the visual collector, and detect the human body posture points based on the physical model of the human body posture points; Based on the physical model of the marker, the marker is used as a visual reference point to obtain pre-calibration information, and the visual acquisition distortion of the marker is calibrated, and the image scale and the real scale are mapped and projected through the pre-calibration information of the marker; the fast-changing dynamic trajectory changes of the human body and the dynamic trajectory changes of the marker are tracked respectively, and the motion trajectory of the CPR rescuer is tracked in real time according to the human posture point and the visual reference point of the marker, and the CPR chest compression index is calculated based on the joint real-time tracking result. The CPR chest compression index includes at least one or more of the following: compression posture, compression frequency, number of compressions, and compression depth; The prompter is used to provide text prompts, picture prompts or voice prompts for the CPR chest compression indicators.

12. A first aid device, characterized in that: The first aid device comprises the CPR real-time guidance system as claimed in claim 11.

13. A readable storage medium, characterized in that: The readable storage medium stores a control program, and when the control program is executed by a processor, the processor executes the steps of the method according to any one of claims 1 to 10.

Images