Cardiac resuscitation assessment method, device and program product

Abstract

The application provides a heart resuscitation evaluation method, device and program product, which can be applied to the cross field of medical first-aid training and artificial intelligence technology. The method comprises the following steps: in the case that a difference between a first compression parameter and a second compression parameter for heart resuscitation on a simulation object meets a preset condition, correcting key point coordinates in a position movement sequence corresponding to each of a plurality of key points to obtain a corrected position movement sequence, wherein the first compression parameter is collected by a pressure sensor, the second compression parameter and the position movement sequence are collected by a visual sensor, and the pressure sensor is arranged inside a chest cavity of the simulation object; determining a visual sign parameter according to the corrected position movement sequence of each of the plurality of key points, wherein the visual sign parameter represents the accuracy of a heart resuscitation posture; and determining an evaluation result according to evaluation values of the visual sign parameter, the first compression parameter and a gas flow parameter determined based on a gas sensor.

Description

Technical Field

[0001] This application relates to the intersection of medical emergency training and artificial intelligence technology, specifically to a cardiac resuscitation assessment method, device, and procedure product. Background Technology

[0002] Cardiac resuscitation (CPR) is an effective means of rescuing patients with cardiac arrest and can greatly improve patient survival rates. However, the accuracy of CPR assessment results using related techniques is relatively low. Summary of the Invention

[0003] In view of the above problems, this application provides a method, device, and procedure for assessing cardiac resuscitation. It also provides an intelligent cardiac resuscitation assessment device and storage medium.

[0004] According to a first aspect of this application, a method for assessing cardiac resuscitation is provided, comprising: correcting the coordinates of key points in a position movement sequence corresponding to multiple key points, wherein the difference between a first compression parameter and a second compression parameter for cardiac resuscitation of a simulated subject satisfies a preset condition, to obtain a corrected position movement sequence, wherein the first compression parameter is acquired by a pressure sensor, and the second compression parameter and the position movement sequence are acquired by a vision sensor, the pressure sensor being disposed inside the chest cavity of the simulated subject; determining visual sign parameters based on the corrected position movement sequences of the multiple key points, wherein the visual sign parameters characterize the accuracy of the cardiac resuscitation posture; and determining an assessment result based on the assessment values ​​of the visual sign parameters, the first compression parameter, and a gas flow parameter determined based on a gas sensor.

[0005] According to embodiments of this application, multiple key points include wrist key points and other key points, the other key points including at least one of the following: elbow key points, shoulder key points, or hip key points; wherein, correcting the coordinates of key points in the position movement sequence to obtain a corrected position movement sequence includes: determining a compensation displacement based on the difference between a first compression parameter and a second compression parameter; determining a compensation vector based on a wrist motion vector and a compensation displacement, wherein the wrist motion vector represents the dominant direction vector of the wrist motion trajectory during cardiac resuscitation; correcting the coordinates of key points in the position movement sequence corresponding to the wrist key points based on the compensation vector to obtain a corrected wrist position movement sequence; and processing the corrected wrist position movement sequence and the position movement sequences of other key points using a posture prior constraint strategy to obtain a corrected position movement sequence for other key points, wherein the posture prior constraint strategy is used to constrain the length of human skeletal segments to be within a predetermined length range.

[0006] According to an embodiment of this application, the cardiac resuscitation assessment method further includes: determining a wrist mean sequence based on the wrist position movement sequence corresponding to the wrist key points and the mean of the wrist position movement sequence; and processing the covariance matrix determined based on the wrist mean sequence using an eigenvalue decomposition algorithm to obtain a wrist motion vector.

[0007] According to embodiments of this application, a posture prior constraint strategy is used to process the wrist correction position movement sequence and other keypoint position movement sequences to obtain a correction position movement sequence for other keypoints. This includes: processing the wrist correction position movement sequence and the elbow keypoint position movement sequence using a posture prior constraint strategy to obtain an elbow correction position movement sequence, wherein the posture prior constraint strategy includes ensuring the forearm length is the same before and after correction, and the forearm length represents the distance between the wrist keypoint and the elbow keypoint; processing the elbow correction position movement sequence and the shoulder keypoint position movement sequence using a posture prior constraint strategy to obtain a shoulder correction position movement sequence, wherein the posture prior constraint strategy includes ensuring the upper arm length is the same before and after correction, and the upper arm length represents the distance between the shoulder keypoint and the elbow keypoint; and processing the shoulder correction position movement sequence and the hip keypoint position movement sequence using a posture prior constraint strategy to obtain a hip correction position movement sequence, wherein the posture prior constraint strategy includes ensuring the back length is the same before and after correction, and the back length represents the distance between the shoulder keypoint and the hip keypoint.

[0008] According to an embodiment of this application, the first pressing parameter includes a first pressing depth and a first pressing frequency, and the second pressing parameter includes a second pressing depth and a second pressing frequency; the preset condition includes at least one of the following: a first difference between the first pressing depth and the second pressing depth is greater than or equal to a first preset value; a second difference between the first pressing frequency and the second pressing frequency is greater than or equal to a second preset value; a first difference between the first pressing depth and the second pressing depth is greater than or equal to the first preset value, and within a preset period, the number of times the first difference is greater than or equal to the first preset value is greater than or equal to a third preset value; or a second difference between the first pressing frequency and the second pressing frequency is greater than or equal to the second preset value, and within a preset period, the number of times the second difference is greater than or equal to the second preset value is greater than or equal to a fourth preset value.

[0009] According to an embodiment of this application, the cardiac resuscitation assessment method further includes: determining a wrist movement distance sequence based on a wrist position movement sequence, the mean of the wrist position movement sequence, and a wrist movement vector; and determining a second compression depth based on the difference between adjacent peaks and troughs in the wrist movement distance sequence.

[0010] According to embodiments of this application, visual sign parameters include at least one of hip joint stability parameters or elbow extension; wherein, determining visual sign parameters based on the respective corrected position movement sequences of multiple key points includes at least one of the following: determining a hip mean sequence based on the hip corrected position movement sequence and the mean of the hip corrected position movement sequence; processing the covariance matrix determined based on the hip mean sequence using an eigenvalue decomposition algorithm to determine the hip motion vector; determining the hip joint stability parameters based on the hip corrected position movement sequence, the mean of the hip corrected position movement sequence, the hip motion vector, and the number of key point coordinates in the hip corrected position movement sequence; or determining elbow extension based on the corrected position movement sequences corresponding to the wrist key point, elbow key point, and shoulder key point, respectively.

[0011] According to an embodiment of this application, an evaluation result is determined based on the evaluation values ​​of visual vital signs parameters, a first pressing parameter, and a gas flow rate parameter determined based on a gas sensor. This includes: determining a visual evaluation value based on the visual vital signs parameters and their corresponding weights; determining a pressing evaluation value based on the first pressing parameter and its corresponding weights; determining a gas evaluation value based on the gas flow rate parameter and its corresponding weights; and fusing the visual evaluation value, the pressing evaluation value, and the gas evaluation value to obtain the evaluation result.

[0012] A second aspect of this application provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0013] A third aspect of this application also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0014] A fourth aspect of this application provides a cardiac resuscitation assessment device, comprising: a correction module, configured to correct the coordinates of key points in a position movement sequence corresponding to multiple key points, provided that the difference between a first compression parameter and a second compression parameter for cardiac resuscitation on a simulated subject meets preset conditions, thereby obtaining a corrected position movement sequence, wherein the first compression parameter is acquired by a pressure sensor, and the second compression parameter and the position movement sequence are acquired by a vision sensor, the pressure sensor being disposed inside the chest cavity of the simulated subject; a visual sign module, configured to determine visual sign parameters based on the corrected position movement sequence of the multiple key points, wherein the visual sign parameters characterize the accuracy of the cardiac resuscitation posture; and an assessment result determination module, configured to determine an assessment result based on the assessment values ​​of the visual sign parameters, the first compression parameter, and a gas flow parameter determined based on a gas sensor.

[0015] A fifth aspect of this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0016] According to an embodiment of this application, during cardiac resuscitation on a simulated object, using the first compression parameter collected by the pressure sensor as a reference, and if the difference between the first compression parameter and the second compression parameter collected by the vision sensor meets a preset condition, the coordinates of key points in the position movement sequence corresponding to multiple key points can be corrected to obtain a more accurate corrected position movement sequence. Based on the corrected position movement sequence of multiple key points, visual signs parameters characterizing the accuracy of the cardiac resuscitation posture can be determined. By fusing the evaluation values ​​of the visual signs parameters, the first compression parameter, and the gas flow parameter determined by the gas sensor, an evaluation result can be obtained. By fusing the data collected by the pressure sensor and the vision sensor, dynamic correction can be performed using high-confidence information from another mode when the data of a single mode is disturbed, thereby improving the accuracy and robustness of the evaluation result. Attached Figure Description

[0017] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments of this application with reference to the accompanying drawings.

[0018] Figure 1 The diagram illustrates an application scenario of the cardiac resuscitation assessment method, device, and procedure product according to embodiments of this application.

[0019] Figure 2 A flowchart of a cardiac resuscitation assessment method according to an embodiment of this application is shown.

[0020] Figure 3 A diagram illustrating sensor placement operation according to an embodiment of this application is shown.

[0021] Figure 4 A schematic diagram of a graphical user interface according to an embodiment of this application is shown.

[0022] Figure 5 A flowchart illustrating the overall process of cardiac resuscitation assessment according to an embodiment of this application is shown.

[0023] Figure 6 A flowchart of data processing and cardiac resuscitation assessment according to an embodiment of this application is shown.

[0024] Figure 7 A structural block diagram of a cardiac resuscitation assessment device according to an embodiment of this application is shown.

[0025] Figure 8A block diagram of an electronic device suitable for implementing a cardiac resuscitation assessment method according to an embodiment of this application is shown. Detailed Implementation

[0026] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0027] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0028] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0029] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0030] According to incomplete statistics, more than one million people suffer from cardiac arrest each year, posing a significant threat to public health. Cardiopulmonary resuscitation (CPR) is an effective means of rescuing cardiac arrest patients and can greatly improve patient survival rates. Therefore, CPR training is crucial. The main CPR training programs and their shortcomings are as follows:

[0031] 1. Trainees practice on simplified mannequins, which lack feedback and quantitative assessment. Its main drawbacks are as follows:

[0032] (1) Lack of objective and quantitative assessment: It relies entirely on the coach's subjective observation and experience judgment, and cannot accurately measure and provide real-time feedback on key information such as pressure position, pressure frequency, and pressure depth;

[0033] (2) Training data cannot be analyzed and recorded: It is impossible to form a personal training file, making it difficult to track the trainees' progress and weaknesses;

[0034] (3) The training costs are high and it relies on senior coaches.

[0035] 2. Trainees practice on a mannequin equipped with electronic feedback. The mannequin features pressure sensors and audio prompts, displaying key parameters such as compression depth and frequency on a screen in real time, and ultimately providing an evaluation score. Its main drawbacks are as follows:

[0036] (1) Single feedback dimension: The core sensor of the electronic feedback device—the pressure sensor array—is determined by its physical principle to be able to measure only a limited number of parameters directly related to "deformation". However, high-quality CPR operation not only requires the strength and rhythm to meet the standards, but also includes a series of key spatial geometry and ergonomic specifications. Therefore, the feedback information it provides is one-sided and incomplete, and it is impossible to build a three-dimensional skill assessment model that conforms to clinical guidelines.

[0037] (2) Visual assessment methods have orientation-dependent errors: Visual aids typically assume that the pressing direction is perpendicular to the surface of the mannequin or parallel to the camera's optical axis. In actual, non-ideal training environments, trainees' heights, positions, and camera angles vary greatly. This orientation presupposition introduces systematic measurement biases, which seriously affect the accuracy of assessments of key parameters such as pressing depth and rebound.

[0038] (3) Multimodal data are not deeply integrated and mutually corrected: Pressure sensor and vision sensor data are often processed independently and compared simply, lacking a mechanism for data alignment and consistency verification under a unified three-dimensional spatial reference. When single-modal data is disturbed (such as visual occlusion), the system cannot use the high confidence information of another modality for dynamic correction, resulting in poor robustness of the evaluation results.

[0039] (4) Lack of intelligent analysis and personalized guidance: The stress feedback is mechanical and instantaneous, and it is impossible to comprehensively score the duration of different cardiopulmonary resuscitation stages, nor can it provide personalized improvement suggestions based on historical data;

[0040] (5) Poor interactivity and boring experience: lack of visual guidance and immersive training scenarios, resulting in low student participation.

[0041] Therefore, there is an urgent need for an assessment method that can comprehensively, objectively, and automatically evaluate the quality of CPR procedures.

[0042] In view of this, this application provides a method, device, and program product for assessing cardiac resuscitation. The method includes: correcting the coordinates of key points in a positional movement sequence corresponding to multiple key points, based on a preset condition that the difference between a first compression parameter and a second compression parameter for cardiac resuscitation on a simulated subject satisfies the given conditions, to obtain a corrected positional movement sequence. The first compression parameter is acquired by a pressure sensor, and the second compression parameter and the positional movement sequence are acquired by a vision sensor, the pressure sensor being positioned inside the chest cavity of the simulated subject; determining visual vital signs parameters based on the corrected positional movement sequences of the multiple key points, wherein the visual vital signs parameters characterize the accuracy of the cardiac resuscitation posture; and determining an assessment result based on the assessment values ​​of the visual vital signs parameters, the first compression parameter, and a gas flow rate parameter determined based on a gas sensor.

[0043] It should be noted that the cardiac resuscitation assessment method and cardiac resuscitation assessment device provided in this application can be used in the field of artificial intelligence, or in any field other than artificial intelligence, such as the field of medical emergency training. Therefore, the application field of the cardiac resuscitation assessment method and cardiac resuscitation assessment device provided in this application is not limited.

[0044] In the technical solution of this application, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0045] In scenarios involving automated decision-making using personal information, the methods, devices, and systems provided in this application all offer users corresponding entry points for choosing to agree to or reject the automated decision-making results. If the user chooses to reject, the process proceeds to the expert decision-making stage. Here, "automated decision-making" refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests, or economic, health, and credit status through computer programs, and then making a decision. Here, "expert decision-making" refers to the activity of making decisions by personnel who specialize in a particular field, possess specialized experience, knowledge, and skills, and have reached a certain level of professional expertise.

[0046] Figure 1 The diagram illustrates an application scenario of the cardiac resuscitation assessment method, device, and procedure product according to embodiments of this application.

[0047] like Figure 1As shown, the application scenario according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0048] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication terminal applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email terminals, social media platform software, etc. (for example only).

[0049] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0050] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0051] It should be noted that the cardiac resuscitation assessment method provided in this application embodiment can generally be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103. Accordingly, the cardiac resuscitation assessment device provided in this application embodiment can generally be installed in the first terminal device 101, the second terminal device 102, or the third terminal device 103.

[0052] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0053] Figure 2 A flowchart of a cardiac resuscitation assessment method according to an embodiment of this application is shown.

[0054] like Figure 2As shown, the cardiac resuscitation assessment method of this embodiment includes operations S210 to S230, and the cardiac resuscitation assessment method can be performed by an electronic device.

[0055] In operation S210, if the difference between the first compression parameter and the second compression parameter for performing cardiac resuscitation on the simulated object meets the preset conditions, the coordinates of the key points in the position movement sequence corresponding to each of the multiple key points are corrected to obtain the corrected position movement sequence.

[0056] In operation S220, visual sign parameters are determined based on the correction position movement sequence of multiple key points.

[0057] In operation S230, the evaluation result is determined based on the evaluation values ​​of the visual sign parameters, the first pressing parameters, and the gas flow parameters determined based on the gas sensor.

[0058] The first compression parameters are acquired by a pressure sensor placed inside the chest cavity of the simulated subject. The pressure sensor can be an array of multiple flexible force-sensitive resistors, with a sampling frequency set to meet certain conditions, such as a sampling frequency of not less than 100Hz, but not limited to this. The embodiments of this application do not limit the sampling frequency. The pressure sensor is used to acquire the pressure distribution and changing trends during chest compressions on the simulated subject in real time, thereby calculating the compression depth, compression frequency, and determining whether the chest cavity rebound is sufficient.

[0059] The visual sensor can be a depth sensor, which can be placed on the side of the simulated object to capture complete posture images of the trainee during CPR. The visual sensor can acquire image data during CPR. It can simultaneously acquire red-green-blue (RGB) images and depth maps based on the visual sensor to detect key points of the human body and the simulated object in each frame, and then analyze and calculate key information such as compression position, compression frequency, compression depth, and whether the compression posture is correct. A preset sampling frequency can be set for the visual sensor, for example, a frame rate of not less than 25Hz, but it is not limited to this; the embodiments of this application do not limit the sampling frequency.

[0060] The gas flow parameters are determined based on a gas sensor integrated into the mouth and nose of the simulated object. The gas sensor can be a thermal micro-flow sensor, used to collect gas flow signals in real time during the ventilation process, thereby calculating the duration, tidal volume, and ventilation frequency of the artificial respiration procedure. A corresponding sampling frequency can be set for the gas sensor, for example, a sampling frequency of not less than a frame rate of 50Hz, but it is not limited to this; the embodiments of this application do not limit the sampling frequency.

[0061] It is important to note that all sensors are equipped with a timestamp synchronization mechanism, ensuring strict temporal alignment of multi-source data. Asynchronous and heterogeneous data acquired by vision, pressure, and airflow sensors can be timestamped, filtered, and normalized to form a unified standard multimodal data stream. In the timestamp alignment step, data with different sampling frequencies (30Hz for vision data, 100Hz for pressure data, and 50Hz for airflow data) can be unified to a common time axis through spline interpolation, setting a predetermined sampling period T, for example... Low-pass filters can be used to remove high-frequency jitter interference from the data acquired by the vision sensor, pressure sensor, and airflow sensor. Zero-point calibration and amplitude normalization are performed on the signals from each channel acquired by the vision sensor, pressure sensor, and airflow sensor to form a standard format multimodal data stream.

[0062] The image data acquired by the vision sensor is processed. Based on the image data stream (multiple RGB images and their corresponding depth maps), a convolutional neural network (CNN) model can be used to detect key points of the human upper limbs and torso, obtaining two-dimensional pixel coordinates. , Represents the x-coordinate of the key point on the torso. This represents the ordinate of the keypoint i on the torso. It combines the camera intrinsic parameter matrix Q with the depth map. The two-dimensional pixels are back-projected into three-dimensional spatial points in the camera coordinate system. As shown in formula (1).

[0063] (1);

[0064] Where T denotes transpose and R denotes the set of real numbers. It is a scaling factor that maps depth map values ​​to the actual physical depth, ensuring that the coordinates of the back-projected 3D points are consistent with the scale of the camera intrinsics. This is achieved when the depth map and the intrinsic matrix have the same units. The above formula (1) can be simplified to the following formula (2).

[0065] (2);

[0066] Obtain the spatial 3D coordinates of each key point Several key points may include: left wrist (LW), right wrist (RW), left elbow (LE), right elbow (RE), left shoulder (LS), right shoulder (RS), left hip (LH), and right hip (RH), but are not limited thereto, and the embodiments of this application do not limit this.

[0067] The position movement sequence is represented as follows For example, when the wrist is the key point, the position movement sequence can represent the K coordinate changes of the wrist key point during the compression process. During simulated CPR, using the first compression parameter collected by the pressure sensor as a benchmark, if the difference between the first compression parameter and the second compression parameter collected by the vision sensor meets preset conditions, it indicates a large deviation in the data collected by these two sensors. This allows for correction of the key point coordinates in the position movement sequences corresponding to multiple key points, resulting in a corrected position movement sequence.

[0068] Visual vital signs parameters characterize the accuracy of cardiac resuscitation posture. Cardiac resuscitation posture can include the degree of elbow extension, body posture stability, etc. Visual vital signs parameters can be determined based on the corrected positional movement sequence corresponding to multiple key points after correction.

[0069] Gas flow parameters may include, but are not limited to, blowing duration, tidal volume, or blowing frequency. The embodiments of this application do not limit the gas flow parameters. The evaluation result can be determined based on the evaluation values ​​corresponding to the visual signs parameters, the first compression parameter, and the gas flow parameters.

[0070] Figure 3 A diagram illustrating sensor placement operation according to an embodiment of this application is shown.

[0071] like Figure 3 As shown, the vision sensor can be placed on the side of the simulation object, in front of the trainee's right side, to fully capture the trainee's operating posture image. The pressure sensor can be placed inside the simulation object's chest cavity, and the airflow sensor can be integrated into the simulation object's airway inlet.

[0072] Figure 4 A schematic diagram of a graphical user interface according to an embodiment of this application is shown.

[0073] like Figure 4 As shown, audiovisual feedback on CPR operations can be provided to trainees based on the aforementioned cardiac resuscitation assessment results. The recognition information and assessment results are displayed on the screen in real time, including dynamic compression depth and frequency curves or anatomical diagrams of key human points, and non-standard areas can be highlighted. The overall score and sub-scores can be updated in real time. Based on the assessment results, pre-recorded guidance voice messages can also be played in real time, such as "Compression too shallow," "Please ensure full chest recoil," or "Please straighten your arms," ​​etc., but this is not limited to these, and the embodiments of this application do not impose such limitations.

[0074] The assessment results, various assessment values, real-time pressure curves, and human key point skeleton diagrams are rendered onto the screen in real time. When any indicator exceeds the standard range, it is highlighted on the corresponding human key point skeleton diagram, and a corresponding voice prompt is triggered.

[0075] According to an embodiment of this application, during cardiac resuscitation on a simulated object, using the first compression parameter collected by the pressure sensor as a reference, and if the difference between the first compression parameter and the second compression parameter collected by the vision sensor meets a preset condition, the coordinates of key points in the position movement sequence corresponding to multiple key points can be corrected to obtain a more accurate corrected position movement sequence. Based on the corrected position movement sequence of multiple key points, visual signs parameters characterizing the accuracy of the cardiac resuscitation posture can be determined. By fusing the evaluation values ​​of the visual signs parameters, the first compression parameter, and the gas flow parameter determined by the gas sensor, an evaluation result can be obtained. By fusing the data collected by the pressure sensor and the vision sensor, dynamic correction can be performed using high-confidence information from another mode when the data of a single mode is disturbed, thereby improving the accuracy and robustness of the evaluation result.

[0076] According to an embodiment of this application, the first pressing parameter includes a first pressing depth and a first pressing frequency, and the second pressing parameter includes a second pressing depth and a second pressing frequency; the preset condition includes at least one of the following: a first difference between the first pressing depth and the second pressing depth is greater than or equal to a first preset value; a second difference between the first pressing frequency and the second pressing frequency is greater than or equal to a second preset value; a first difference between the first pressing depth and the second pressing depth is greater than or equal to the first preset value, and within a preset period, the number of times the first difference is greater than or equal to the first preset value is greater than or equal to a third preset value; or a second difference between the first pressing frequency and the second pressing frequency is greater than or equal to the second preset value, and within a preset period, the number of times the second difference is greater than or equal to the second preset value is greater than or equal to a fourth preset value.

[0077] The first pressing parameter may include a first pressing depth and a first pressing frequency, and the second pressing parameter may include a second pressing depth and a second pressing frequency.

[0078] To determine whether there is positioning deviation or depth drift in the keypoint coordinates obtained from the vision sensor, the pressure depth and frequency of the nth press can be calculated independently from image data and pressure data. The second press depth can be determined based on the image data. Second press frequency The initial compression depth can be determined based on pressure data. and the first pressing frequency If the difference between the first and second compression parameters during simulated CPR meets preset conditions, the coordinates of key points in the positional movement sequences corresponding to multiple key points can be corrected. The preset conditions may include at least one of the following:

[0079] First difference between the first compression depth and the second compression depth If the value is greater than or equal to the first preset value, it means that the coordinates of the key points in the position movement sequence need to be corrected.

[0080] The second difference between the first and second pressing frequencies If the value is greater than or equal to the second preset value, it means that the coordinates of the key points in the position movement sequence need to be corrected.

[0081] First difference between the first compression depth and the second compression depth The number of times the first difference is greater than or equal to the first preset value, and within a preset period, the first difference is greater than or equal to the first preset value. If the value is greater than or equal to the third preset value, it means that the coordinates of key points in the position movement sequence need to be corrected.

[0082] The second difference between the first and second pressing frequencies The number of times the second difference is greater than or equal to the second preset value within a preset period. If the value is greater than or equal to the fourth preset value, it indicates that the coordinates of key points in the position movement sequence need to be corrected. See formulas (3), (4), (5), and (6).

[0083] (3);

[0084] (4);

[0085] (5);

[0086] (6);

[0087] in, Indicates the first difference. Indicates the second difference. This indicates the number of times the first difference is greater than or equal to the first preset value. This indicates the number of times the second difference is greater than or equal to the first preset value.

[0088] The first, second, third, and fourth preset values ​​can be set based on actual conditions. For example, the first preset value can be set to... (Depth tolerance), the second preset value can be set to The frequency tolerance is times per minute. The third or fourth preset value can be set to the same or different numbers, for example, all three times, but it is not limited thereto. The embodiments of this application do not specifically limit the above threshold settings.

[0089] By utilizing the output of a highly reliable pressure sensor, the three-dimensional coordinates of key points in the original detection are corrected in reverse, thus forming a closed-loop structure of "perception-fusion-feedback-optimization".

[0090] According to the embodiments of this application, the difference between the first pressing parameter and the second pressing parameter can be used to determine whether there is a positioning deviation or depth drift in the key point coordinates. By setting preset conditions, the key point coordinates can be verified. If the preset conditions are met, an anomaly flag can be triggered and the coordinate modification process can be started, thereby improving the accuracy and robustness of the key point detection results.

[0091] According to an embodiment of this application, the cardiac resuscitation assessment method further includes: determining a wrist mean sequence based on the wrist position movement sequence corresponding to the wrist key points and the mean of the wrist position movement sequence; and processing the covariance matrix determined based on the wrist mean sequence using an eigenvalue decomposition algorithm to obtain a wrist motion vector.

[0092] This application takes into account that, in the actual pressing process, the pressing direction is not completely parallel to the camera plane, and the movement trajectory of the human body's key points often has obvious tilt components. If the movement direction is still assumed to be a certain coordinate axis, it will lead to systematic estimation bias.

[0093] The sequence of wrist movement corresponding to key points within the nth pressing cycle. Mean of wrist position movement sequence As shown in formula (7).

[0094] (7);

[0095] Where K represents the number of times the wrist key points move.

[0096] Based on the wrist position movement sequence corresponding to the wrist key points and the mean of the wrist position movement sequence, determine the wrist average sequence. As shown in formula (8).

[0097] (8);

[0098] Based on the wrist mean sequence, the covariance matrix C can be determined as shown in formula (9).

[0099] (9);

[0100] The covariance matrix can be processed using eigenvalue decomposition algorithms. By decomposing the eigenvalues, the principal directions can be extracted, and the covariance matrix can be further processed. Perform eigenvalue decomposition, as shown in formula (10).

[0101] (10);

[0102] Where v represents the eigenvector and λ represents the eigenvalue.

[0103] Can be taken The unit eigenvector corresponding to the largest eigenvalue is determined as the wrist motion vector Vmain, as shown in formula (11).

[0104] (11).

[0105] According to the embodiments of this application, by autonomously identifying the main motion direction with the most significant displacement change from the motion curve of the key points of the wrist, and projecting and calculating the compression parameters along this direction, the dependence on the preset compression direction is eliminated, the problem of parameter distortion caused by the deviation between the actual compression direction and the preset direction is solved, and adaptive and precise motion projection is realized, which improves the realism and accuracy of the calculation of key parameters such as compression depth and frequency, and makes the visual evaluation results more consistent with the actual rescue action.

[0106] According to an embodiment of this application, the cardiac resuscitation assessment method further includes: determining a wrist movement distance sequence based on a wrist position movement sequence, the mean of the wrist position movement sequence, and a wrist movement vector; and determining a second compression depth based on the difference between adjacent peaks and troughs in the wrist movement distance sequence.

[0107] Based on the wrist position movement sequence, the mean of the wrist position movement sequence, and the wrist motion vector, the wrist movement distance sequence is determined. Specifically, the wrist position movement sequence is... Projected onto the main direction of motion The wrist movement distance sequence was obtained. , which is a one-dimensional displacement sequence, represents the effective displacement calculated by projection along the main direction, as shown in formula (12).

[0108] (12);

[0109] This sequence This represents the positional changes of keypoints along their actual motion path in the projected sequence. After baseline correction and smoothing filtering, a peak detection algorithm can be used to identify the set of maxima. and the set of minimum points .

[0110] The second pressing depth is determined based on the difference between adjacent peaks and troughs in the wrist movement distance sequence, as shown in formula (13).

[0111] (13);

[0112] The difference between the peak and trough of a single press is the second press depth.

[0113] For example, the set of maximum points The set of points is [10, 8, 9, 7]. Given the sequence [2, 1, 1, 3], we can deduce that the second pressing depth sequence is [8, 7, 8, 4], which is the difference between the values ​​of the same sequence in the maximum and minimum value sets.

[0114] According to the embodiments of this application, the wrist movement distance sequence can be determined based on the wrist motion vector and the wrist position movement sequence, which solves the dependence on the preset pressing direction and the parameter distortion problem caused by the deviation between the actual pressing direction and the preset direction, and the calculated second pressing depth is more accurate.

[0115] According to embodiments of this application, multiple key points include wrist key points and other key points, the other key points including at least one of the following: elbow key points, shoulder key points, or hip key points; wherein, correcting the coordinates of key points in the position movement sequence to obtain a corrected position movement sequence includes: determining a compensation displacement based on the difference between a first compression parameter and a second compression parameter; determining a compensation vector based on a wrist motion vector and a compensation displacement, wherein the wrist motion vector represents the dominant direction vector of the wrist motion trajectory during cardiac resuscitation; correcting the coordinates of key points in the position movement sequence corresponding to the wrist key points based on the compensation vector to obtain a corrected wrist position movement sequence; and processing the corrected wrist position movement sequence and the position movement sequences of other key points using a posture prior constraint strategy to obtain a corrected position movement sequence for other key points, wherein the posture prior constraint strategy is used to constrain the length of human skeletal segments to be within a predetermined length range.

[0116] Multiple key points may include wrist key points and other key points, which may include at least one of the following: elbow key points, shoulder key points, or hip key points.

[0117] Unidirectional compensation cannot adapt to tilted pressing scenarios, so a vector projection compensation strategy can be adopted to decompose the correction amount into the X, Y, and Z axes according to the main motion direction.

[0118] The compensation displacement is determined based on the difference between the first and second pressing parameters. As shown in formula (14).

[0119] (14);

[0120] Based on wrist motion vectors and compensation displacement Determine the compensation vector As shown in formula (15).

[0121] (15);

[0122] in, The compensation vector representing the x-coordinate. The compensation vector representing the y-coordinate. The compensation vector representing the z-coordinate.

[0123] The compensation amount is distributed to the three coordinate axes according to the proportion of the main motion direction.

[0124] For each point in the wrist key point sequence corresponding to the nth press The following correction is performed: based on the compensation vector pairs, the coordinates of key points in the position movement sequence corresponding to the wrist key points are adjusted. After correction, the wrist correction position movement sequence is obtained. As shown in formula (16).

[0125] (16);

[0126] in This is a smoothing coefficient to prevent overshoot.

[0127] By using the attitude prior constraint strategy to process the wrist correction position movement sequence and the position movement sequence of other key points, the correction position movement sequence of other key points can be obtained. The attitude prior constraint strategy is used to constrain the length of human bone segments to be within a predetermined length range.

[0128] According to embodiments of this application, a "perception-verification-correction" integrated closed-loop optimization method is formed by constructing a cross-modal verification based on pressure sensor data and generating a compensation vector along the main motion direction to correct the key point coordinates in reverse. This not only achieves deep fusion and mutual verification of multi-sensor data, improving the overall fault tolerance of the system, but more importantly, it overcomes the limitation of pure visual methods being susceptible to interference in complex environments by dynamically calibrating visual vital signs parameters through high-confidence physical signals. Thus, while ensuring the comprehensiveness of the evaluation results, it enhances the effectiveness and robustness.

[0129] According to embodiments of this application, a posture prior constraint strategy is used to process the wrist correction position movement sequence and other keypoint position movement sequences to obtain a correction position movement sequence for other keypoints. This includes: processing the wrist correction position movement sequence and the elbow keypoint position movement sequence using a posture prior constraint strategy to obtain an elbow correction position movement sequence, wherein the posture prior constraint strategy includes ensuring the forearm length is the same before and after correction, and the forearm length represents the distance between the wrist keypoint and the elbow keypoint; processing the elbow correction position movement sequence and the shoulder keypoint position movement sequence using a posture prior constraint strategy to obtain a shoulder correction position movement sequence, wherein the posture prior constraint strategy includes ensuring the upper arm length is the same before and after correction, and the upper arm length represents the distance between the shoulder keypoint and the elbow keypoint; and processing the shoulder correction position movement sequence and the hip keypoint position movement sequence using a posture prior constraint strategy to obtain a hip correction position movement sequence, wherein the posture prior constraint strategy includes ensuring the back length is the same before and after correction, and the back length represents the distance between the shoulder keypoint and the hip keypoint.

[0130] By using the attitude prior constraint strategy to process the wrist correction position movement sequence and the elbow key point position movement sequence, the elbow correction position movement sequence can be obtained. The attitude prior constraint strategy can include that the forearm length is the same before and after correction, as shown in formulas (17) and (18). The forearm length represents the distance between the wrist key point and the elbow key point.

[0131] (17);

[0132] (18);

[0133] in, This indicates the sequence of right wrist correction position movement. This indicates the sequence of right elbow correction position movement. Represents the left wrist correction position movement sequence, This indicates the sequence of left elbow correction position shifts. Indicates forearm length, This indicates the allowable length error value.

[0134] By using the attitude prior constraint strategy to process the elbow correction position movement sequence and the shoulder key point position movement sequence, the shoulder correction position movement sequence can be obtained. The attitude prior constraint strategy can include the upper arm length being the same before and after correction, as shown in formulas (19) and (20). The upper arm length represents the distance between the shoulder key point and the elbow key point.

[0135] (19);

[0136] (20);

[0137] in, This indicates the sequence of right elbow correction position movement. This indicates the sequence of right shoulder correction position movement. This indicates the sequence of left elbow correction position shifts. This indicates the sequence of movements for correcting the position of the left shoulder. Indicates upper arm length. This indicates the allowable length error value.

[0138] By using the posture prior constraint strategy to process the shoulder correction position movement sequence and the hip key point position movement sequence, the hip correction position movement sequence can be obtained. The posture prior constraint strategy can include the back length being the same before and after correction, as shown in formulas (21) and (22). The back length represents the distance between the shoulder key point and the hip key point.

[0139] (twenty one);

[0140] (twenty two);

[0141] in, This indicates the sequence of right shoulder correction position movement. This indicates the right hip correction position shift sequence. This indicates the sequence of movements for correcting the position of the left shoulder. This indicates the left hip correction position shift sequence. Indicates back length. This indicates the allowable length error value.

[0142] The corrected keypoint coordinates can be written back to the pose sequence for pose prediction and parameter calculation in subsequent frames. Simultaneously, the corrected parameters are recorded for long-term model self-learning.

[0143] According to the embodiments of this application, by processing the wrist correction position movement sequence and other key point position movement sequences through the posture prior constraint strategy, the correction position movement sequence of the corresponding key points can be obtained, which facilitates the calculation of visual characteristic parameters in subsequent frames, improves the accuracy of the evaluation results, and effectively solves the key point recognition deviation caused by factors such as partial occlusion, limb crossing, and illumination changes.

[0144] According to embodiments of this application, visual sign parameters include at least one of hip joint stability parameters or elbow extension; wherein, determining visual sign parameters based on the respective corrected position movement sequences of multiple key points includes at least one of the following: determining a hip mean sequence based on the hip corrected position movement sequence and the mean of the hip corrected position movement sequence; processing the covariance matrix determined based on the hip mean sequence using an eigenvalue decomposition algorithm to determine the hip motion vector; determining the hip joint stability parameters based on the hip corrected position movement sequence, the mean of the hip corrected position movement sequence, the hip motion vector, and the number of key point coordinates in the hip corrected position movement sequence; or determining elbow extension based on the corrected position movement sequences corresponding to the wrist key point, elbow key point, and shoulder key point, respectively.

[0145] Visual signs parameters may include at least one of hip stability parameters or elbow extension.

[0146] Based on the hip correction position movement sequence and its mean, a hip mean value sequence can be determined. The covariance matrix determined from the hip mean value sequence is then processed using an eigenvalue decomposition algorithm to determine the hip motion vector. The process is the same as obtaining the wrist motion vector, and will not be repeated here.

[0147] The hip position movement sequence can be determined based on the hip correction position movement sequence, the mean of the hip correction position movement sequence, and the hip motion vector. As shown in formula (23).

[0148] (twenty three);

[0149] in, Represents the hip motion vector. This indicates the hip correction position movement sequence. This represents the mean of the hip correction position shift sequence.

[0150] Based on hip position movement sequence Mean of hip correction position shift sequence The number of key point coordinates N in the hip correction position movement sequence can be used to determine the hip joint stability parameters. As shown in formula (24).

[0151] (twenty four);

[0152] If its standard deviation If the threshold is exceeded, it is determined that the rescuer did not use their hips to exert force and there is a problem of waist collapse or swaying.

[0153] The elbow extension is determined based on the correction position movement sequence corresponding to the key points of the wrist, elbow, and shoulder. The angle between the lines connecting the key points of the wrist, elbow, and shoulder can be calculated using the law of cosines. As shown in formula (25). (25);

[0154] If the angle between the connecting lines is greater than the preset angle, the arm can be considered to be basically straight.

[0155] Visual vital signs parameters can also include the pressure point. Based on the principle of human key points, 3D spatial points of key points of the simulated object can be obtained, with the core key points being the coordinates of the left nipple (LN) and the right nipple (RN).

[0156] Based on whether the coordinates of key wrist points are within the effective pressure area, the pressure position parameters are determined. The effective pressure area can be a circular area with a diameter of approximately 5cm, centered at the midpoint of the line connecting the two nipples. Let the center point of this area be... coordinates As shown in formula (26).

[0157] (26);

[0158] in, This indicates the sequence of movement of the left nipple. This indicates the sequence of movement of the right nipple.

[0159] By comparing the relative positions of key points on the left and right wrists of the human body with the compression area on the chest of the simulated human in each frame, it can be determined whether the compression position is correct.

[0160] According to embodiments of this application, by using a corrected positional movement sequence to calculate hip joint stability parameters and elbow extension, visual signs parameters are made more accurate, which helps to improve the accuracy of assessment results.

[0161] According to an embodiment of this application, an evaluation result is determined based on the evaluation values ​​of visual vital signs parameters, a first pressing parameter, and a gas flow rate parameter determined based on a gas sensor. This includes: determining a visual evaluation value based on the visual vital signs parameters and their corresponding weights; determining a pressing evaluation value based on the first pressing parameter and its corresponding weights; determining a gas evaluation value based on the gas flow rate parameter and its corresponding weights; and fusing the visual evaluation value, the pressing evaluation value, and the gas evaluation value to obtain the evaluation result.

[0162] The first compression parameter can also include the chest cavity recoil rate. After amplification and ADC conversion, a digital signal can be obtained. The pre-calibrated nonlinear mapping function The pressure value can be converted into a displacement value, as shown in formula (27).

[0163] (27);

[0164] in, This represents the press data in time period t. Backpropagation regression modeling is used to ensure that the error is less than the preset error throughout the entire pressing range.

[0165] Can be The same peak detection algorithm used in image processing was employed to obtain the compression depth sequence. and pressing frequency .

[0166] If the minimum pressure between two consecutive compressions is close to zero, it can be considered that the chest cavity has fully rebounded, and the full rebound rate can be statistically analyzed.

[0167] The compression cycle can be determined based on the time difference between adjacent troughs (i.e., minimum values). As shown in formula (28).

[0168] (28);

[0169] in, This represents the time corresponding to the (n+1)th trough. This represents the time corresponding to the nth trough.

[0170] The pressing frequency can be determined based on the number of presses per unit time, as shown in formula (29).

[0171] (29);

[0172] in, This represents the average period, and N represents the total number of periods. Indicates the frequency of pressing.

[0173] The airflow sensor outputs gas flow rate. Through testing Is it higher than the background noise (i.e.) ), thereby determining the start and end times of the blowing. and .

[0174] Based on blowing and deadline Determine the duration of blowing. As shown in formula (30).

[0175] (30);

[0176] Through real-time gas flow The tidal volume is obtained by integrating over the duration. As shown in formula (31), the tidal volume is used to determine whether the requirements are met.

[0177] (31)

[0178] By measuring the duration, the number of breaths per unit time can be obtained, thus yielding the blowing frequency. As shown in formula (32).

[0179] (32);

[0180] In addition, the CPR process can be divided into different stages, and an action recognition algorithm can be used to determine the current stage of the video stream, outputting a score based on the cumulative number of repetitions at each stage. Referring to CPR action standards, scores can also be output for the degree of standardization of different actions at each stage. The stage ratio can be determined based on the ratio of chest compressions to artificial respiration.

[0181] For each trainee performing CPR, a scoring vector can be constructed based on the cardiac resuscitation operation, as shown in formula (33).

[0182] (33);

[0183] in, This is the first pressing depth. The first pressing frequency, This is the pressing position. The degree of elbow extension. For hip joint stability, For thoracic cavity rebound rate, For the stage proportion, This refers to tidal volume.

[0184] Based on the values ​​obtained above, the evaluation value of each item can be determined, as shown in formula (34).

[0185] (34);

[0186] in These are measured values. For the target value, This represents the allowable error range.

[0187] A visual assessment value is determined based on visual vital sign parameters and their corresponding weights; a compression assessment value is determined based on a first compression parameter and its corresponding weight; a gas assessment value is determined based on a gas flow rate parameter and its corresponding weight; and the visual, compression, and gas assessment values ​​are then combined to obtain the final assessment result. As shown in formula (35).

[0188] (35);

[0189] in, The weights are assigned to each item, and these weights can be obtained through machine learning training.

[0190] According to the embodiments of this application, by integrating multimodal data such as visual signs parameters, first compression parameters, and gas flow parameters, an objective quantitative assessment of the entire process of chest compressions and artificial respiration in CPR operations is achieved in multiple dimensions (compression position, compression depth, compression frequency, compression posture, chest recoil, tidal volume, etc.), overcoming the drawbacks of single assessment.

[0191] Figure 5 A flowchart illustrating the overall process of cardiac resuscitation assessment according to an embodiment of this application is shown.

[0192] like Figure 5 As shown, the multimodal data collected by the RGB-D depth vision sensor, pressure sensor, and gas sensor can first be preprocessed to obtain the operator's posture image, depth information, pressure data, and gas flow rate data. Preprocessing operations include, but are not limited to, timestamp alignment, data filtering, and normalization. Based on the gas flow rate data, the ventilation flow rate and ventilation frequency can be determined. Based on the ventilation flow rate and ventilation frequency, the blowing volume and blowing frequency can be determined respectively. Based on the pressure data, peak values ​​can be measured to obtain the compression depth. Based on the number of peak values ​​in the pressure data, the compression frequency can be determined. Based on the pressure values, chest recoil can be determined.

[0193] The operation posture image can be processed based on the motion posture model to obtain human body key points and simulated human key points. Based on the movement distance and frequency of the left and right wrist key points, the compression depth and compression frequency can be determined. The compression depth and compression frequency determined by the pressure sensor are compared with those obtained from the wrist key points (i.e., verification data). If the difference in compression depth or compression frequency exceeds its corresponding preset threshold, it indicates an error in the key point coordinates. The key point coordinates can be corrected based on the compression depth difference, resulting in a corrected position movement sequence for each key point. Using the corrected position movement sequence of the hip key points, hip joint stability can be determined. Using the corrected position movement sequences of the elbow, wrist, and shoulder key points, elbow extension can be determined. The compression position is further determined based on whether the wrist key points are within the preset range of the simulated object.

[0194] The preprocessed images of the operational postures can be processed based on an action recognition model to determine the CPR stage of the video stream, and then determine the stage ratio. The stage ratio can include the ratio of chest compressions to artificial respiration.

[0195] Each of the above parameter values ​​is used to calculate its own score. By combining these scores, the machine learning model can output a comprehensive score (evaluation result).

[0196] Figure 6 A structural block diagram of a cardiac resuscitation assessment device according to an embodiment of this application is shown.

[0197] like Figure 6As shown, the cardiac resuscitation assessment device may include a data acquisition module, a data processing module, an intelligent assessment module, and a real-time feedback and guidance module. The data processing module may include multimodal data fusion, key indicator extraction, and data verification and correction. The data acquisition module can collect data from visual sensors, gas sensors, and pressure sensors. Multimodal data fusion may involve preprocessing the collected data, including timestamp alignment, data filtering, and normalization, to form a unified standard multimodal data stream. Key indicator extraction from the multimodal data stream may include, but is not limited to, compression depth, compression frequency, ventilation volume, and ventilation frequency, etc., and is not listed here. Data verification and correction are performed based on the compression depth and compression frequency acquired by the visual and pressure sensors. If the verification passes, the assessment can be directly performed based on the obtained key indicators to obtain the cardiac resuscitation assessment result. Real-time feedback and guidance can be provided based on the cardiac resuscitation assessment result. If the verification fails, the coordinates of key points can be corrected to obtain a corrected position movement sequence. The relevant key indicators are re-determined based on the corrected position movement sequence. An assessment is conducted based on the re-obtained key indicators to obtain the cardiac resuscitation assessment results. Real-time feedback and guidance can be provided based on these results.

[0198] Figure 7 A structural block diagram of a cardiac resuscitation assessment device according to an embodiment of this application is shown.

[0199] like Figure 7 As shown, the cardiac resuscitation assessment device of this embodiment includes a correction module 710, a visual signs module 720, and an assessment result determination module 730.

[0200] The correction module 710 is used to correct the coordinates of key points in the position movement sequence corresponding to multiple key points when the difference between the first compression parameter and the second compression parameter for performing cardiac resuscitation on the simulated object meets the preset conditions, so as to obtain the corrected position movement sequence. The first compression parameter is acquired by a pressure sensor, and the second compression parameter and the position movement sequence are acquired by a vision sensor. The pressure sensor is set inside the chest cavity of the simulated object.

[0201] The visual signs module 720 is used to determine visual signs parameters based on the corrected positional movement sequence of multiple key points, wherein the visual signs parameters characterize the accuracy of the cardiac resuscitation posture.

[0202] The evaluation result determination module 730 is used to determine the evaluation result based on the evaluation values ​​of the visual sign parameters, the first compression parameter, and the gas flow parameter determined based on the gas sensor.

[0203] According to an embodiment of this application, during cardiac resuscitation on a simulated object, using the first compression parameter collected by the pressure sensor as a reference, and if the difference between the first compression parameter and the second compression parameter collected by the vision sensor meets a preset condition, the coordinates of key points in the position movement sequence corresponding to multiple key points can be corrected to obtain a more accurate corrected position movement sequence. Based on the corrected position movement sequence of multiple key points, visual signs parameters characterizing the accuracy of the cardiac resuscitation posture can be determined. By fusing the evaluation values ​​of the visual signs parameters, the first compression parameter, and the gas flow parameter determined by the gas sensor, an evaluation result can be obtained. By fusing the data collected by the pressure sensor and the vision sensor, dynamic correction can be performed using high-confidence information from another mode when the data of a single mode is disturbed, thereby improving the accuracy and robustness of the evaluation result.

[0204] Multiple key points include wrist key points and other key points, which include at least one of the following: elbow key points, shoulder key points, or hip key points.

[0205] The correction module 710 includes: a compensation displacement determination unit, a compensation vector determination unit, a correction unit, and an attitude prior unit.

[0206] The compensation displacement determination unit is used to determine the compensation displacement based on the difference between the first pressing parameter and the second pressing parameter.

[0207] The compensation vector determination unit is used to determine the compensation vector based on the wrist motion vector and the compensation displacement, wherein the wrist motion vector represents the dominant direction vector of the wrist motion trajectory during cardiac resuscitation.

[0208] The correction unit is used to correct the coordinates of key points in the position movement sequence corresponding to the wrist key points based on the compensation vector, so as to obtain the corrected wrist position movement sequence.

[0209] The attitude prior unit is used to process the wrist correction position movement sequence and other key point position movement sequences using the attitude prior constraint strategy to obtain the correction position movement sequence of other key points. The attitude prior constraint strategy is used to constrain the length of human bone segments to be within a predetermined length range.

[0210] The cardiac resuscitation assessment device in this embodiment also includes a wrist mean module and a wrist motion vector module.

[0211] The wrist averaging module is used to determine the wrist averaging sequence based on the wrist position movement sequence corresponding to the wrist key points and the mean of the wrist position movement sequence.

[0212] The wrist motion vector module is used to process the covariance matrix determined based on the wrist mean sequence using an eigenvalue decomposition algorithm to obtain the wrist motion vector.

[0213] The posture prior unit includes: elbow correction subunit, shoulder correction subunit and hip correction subunit.

[0214] The elbow correction subunit is used to process the wrist correction position movement sequence and the elbow key point position movement sequence using the attitude prior constraint strategy to obtain the elbow correction position movement sequence. The attitude prior constraint strategy includes that the forearm length is the same before and after correction, and the forearm length represents the distance between the wrist key point and the elbow key point.

[0215] The shoulder correction subunit is used to process the elbow correction position movement sequence and the shoulder key point position movement sequence using the attitude prior constraint strategy to obtain the shoulder correction position movement sequence. The attitude prior constraint strategy includes ensuring that the upper arm length is the same before and after correction, and the upper arm length represents the distance between the shoulder key point and the elbow key point.

[0216] The hip correction subunit is used to process the shoulder correction position movement sequence and the hip key point position movement sequence using a posture prior constraint strategy to obtain the hip correction position movement sequence. The posture prior constraint strategy includes ensuring that the back length is the same before and after correction, and the back length represents the distance between the shoulder key point and the hip key point.

[0217] The first pressing parameters include a first pressing depth and a first pressing frequency, and the second pressing parameters include a second pressing depth and a second pressing frequency.

[0218] The preset conditions include at least one of the following: A first difference between a first pressing depth and a second pressing depth is greater than or equal to a first preset value. A second difference between a first pressing frequency and a second pressing frequency is greater than or equal to a second preset value. A first difference between a first pressing depth and a second pressing depth is greater than or equal to a first preset value, and within a preset period, the number of times the first difference is greater than or equal to the first preset value is greater than or equal to a third preset value. Or, a second difference between a first pressing frequency and a second pressing frequency is greater than or equal to a second preset value, and within a preset period, the number of times the second difference is greater than or equal to the second preset value is greater than or equal to a fourth preset value.

[0219] The cardiac resuscitation assessment device of this embodiment also includes: a wrist movement module and a second compression depth module.

[0220] The wrist movement module is used to determine the wrist movement distance sequence based on the wrist position movement sequence, the mean of the wrist position movement sequence, and the wrist movement vector.

[0221] The second compression depth module is used to determine the second compression depth based on the difference between adjacent peaks and troughs in the wrist movement distance sequence.

[0222] Visual signs parameters include at least one of hip stability parameters or elbow extension.

[0223] The visual signs module 720 includes: a hip mean unit and an elbow extension determination unit.

[0224] The hip mean unit is used to determine the hip mean sequence based on the hip correction position movement sequence and its mean. The covariance matrix determined based on the hip mean sequence is processed using an eigenvalue decomposition algorithm to determine the hip motion vector. Hip joint stability parameters are determined based on the hip correction position movement sequence, its mean, the hip motion vector, and the number of keypoint coordinates in the hip correction position movement sequence.

[0225] The elbow extension determination unit is used to determine the elbow extension based on the correction position movement sequence corresponding to the wrist key point, elbow key point, and shoulder key point.

[0226] The evaluation result determination module 730 includes: a visual evaluation unit, a pressure evaluation unit, and a gas evaluation unit.

[0227] The visual assessment unit is used to determine the visual assessment value based on visual sign parameters and the weights corresponding to those parameters.

[0228] The pressure assessment unit is used to determine the pressure assessment value based on the first pressure parameter and the weight corresponding to the first pressure parameter.

[0229] The gas assessment unit is used to determine the gas assessment value based on the gas flow rate parameter and the weight corresponding to the gas flow rate parameter.

[0230] The evaluation results are obtained by integrating visual assessment values, pressure assessment values, and gas assessment values.

[0231] According to embodiments of this application, any plurality of modules among the correction module 710, visual characteristics module 720, and evaluation result determination module 730 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this application, at least one of the correction module 710, visual characteristics module 720, and evaluation result determination module 730 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any appropriate combination of any of these three implementation methods. Alternatively, at least one of the correction module 710, visual characteristics module 720, and evaluation result determination module 730 can be at least partially implemented as a computer program module, which, when run, can perform corresponding functions.

[0232] Figure 8 A block diagram of an electronic device suitable for implementing a cardiac resuscitation assessment method according to an embodiment of this application is shown.

[0233] like Figure 8 As shown, an electronic device 800 according to an embodiment of this application includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a ROM 802 (Read-Only Memory) or a program loaded from a storage portion 808 into a RAM 803 (Random Access Memory). The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0234] RAM 803 stores various programs and data required for the operation of electronic device 800. Processor 801, ROM 802, and RAM 803 are interconnected via bus 804. Processor 801 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 802 and / or RAM 803. It should be noted that the programs may also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0235] According to embodiments of this application, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to a bus 804. The electronic device 800 may also include one or more of the following components connected to the input / output (I / O) interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 810 as needed so that computer programs read from it can be installed into the storage section 808 as needed.

[0236] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0237] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include ROM 802 and / or RAM 803 and / or one or more memories other than ROM 802 and RAM 803 described above.

[0238] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the recommended methods provided in the embodiments of this application.

[0239] When the computer program is executed by the processor 801, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0240] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 809, and / or installed from a removable medium 811. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0241] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, it performs the functions defined in the system of this application embodiment. According to the embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0242] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0243] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0244] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

[0245] The embodiments of this application have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of this application. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Without departing from the scope of this application, those skilled in the art can make various substitutions and modifications, all of which should fall within the scope of this application.

Claims

1. A method for assessing cardiac resuscitation, characterized in that, include: If the difference between the first compression parameter and the second compression parameter for performing cardiac resuscitation on the simulated object meets a preset condition, the coordinates of key points in the position movement sequence corresponding to each of multiple key points are corrected to obtain a corrected position movement sequence. The first compression parameter is acquired by a pressure sensor, and the second compression parameter and the position movement sequence are acquired by a vision sensor. The pressure sensor is located inside the chest cavity of the simulated object. The multiple key points include wrist key points and other key points. The other key points include at least one of the following: elbow key point, shoulder key point, or hip key point. Visual signs parameters are determined based on the corrected positional movement sequences of the various key points, wherein the visual signs parameters characterize the accuracy of the cardiac resuscitation posture. The evaluation result is determined based on the evaluation values ​​of the visual signs parameters, the first compression parameter, and the gas flow rate parameter determined by the gas sensor. The process of correcting the coordinates of key points in the position movement sequence to obtain a corrected position movement sequence includes: The compensation displacement is determined based on the difference between the first pressing parameter and the second pressing parameter; The compensation vector is determined based on the wrist motion vector and the compensation displacement, wherein the wrist motion vector represents the dominant direction vector of the wrist motion trajectory during cardiac resuscitation. The wrist motion vector is determined based on the following steps: determining a wrist mean sequence based on the wrist position movement sequence corresponding to the wrist key points and the mean of the wrist position movement sequence; and processing the covariance matrix determined based on the wrist mean sequence using an eigenvalue decomposition algorithm to obtain the wrist motion vector. Based on the compensation vector, the coordinates of key points in the position movement sequence corresponding to the wrist key points are corrected to obtain the wrist correction position movement sequence. The wrist correction position movement sequence and the position movement sequence of other key points are processed using a posture prior constraint strategy to obtain the correction position movement sequence of the other key points. The posture prior constraint strategy is used to constrain the length of human skeletal segments to be within a predetermined length range.

2. The method according to claim 1, characterized in that, The wrist correction position movement sequence and the position movement sequences of other key points are processed using a posture prior constraint strategy to obtain the correction position movement sequence of the other key points, including: The wrist correction position movement sequence and the elbow key point position movement sequence are processed using the attitude prior constraint strategy to obtain the elbow correction position movement sequence. The attitude prior constraint strategy includes that the forearm length is the same before and after correction, and the forearm length represents the distance between the wrist key point and the elbow key point. The elbow correction position movement sequence and the shoulder key point position movement sequence are processed using the posture prior constraint strategy to obtain the shoulder correction position movement sequence. The posture prior constraint strategy includes ensuring that the upper arm length is the same before and after correction, and the upper arm length represents the distance between the shoulder key point and the elbow key point. The posture prior constraint strategy is used to process the shoulder correction position movement sequence and the hip key point position movement sequence to obtain the hip correction position movement sequence. The posture prior constraint strategy includes ensuring that the back length is the same before and after correction. The back length represents the distance between the shoulder key point and the hip key point.

3. The method according to any one of claims 1 to 2, characterized in that, The first pressing parameter includes a first pressing depth and a first pressing frequency, and the second pressing parameter includes a second pressing depth and a second pressing frequency; The preset conditions include at least one of the following: The first difference between the first pressing depth and the second pressing depth is greater than or equal to a first preset value; The second difference between the first pressing frequency and the second pressing frequency is greater than or equal to a second preset value; The first difference between the first pressing depth and the second pressing depth is greater than or equal to the first preset value, and within a preset period, the number of times the first difference is greater than or equal to the first preset value is greater than or equal to a third preset value; or The second difference between the first pressing frequency and the second pressing frequency is greater than or equal to the second preset value, and in the preset period, the number of times the second difference is greater than or equal to the second preset value is greater than or equal to the fourth preset value.

4. The method according to claim 1 or 2, characterized in that, The method further includes: Based on the wrist position movement sequence, the mean of the wrist position movement sequence, and the wrist movement vector, a wrist movement distance sequence is determined; The second compression depth is determined based on the difference between adjacent peaks and troughs in the wrist movement distance sequence.

5. The method according to any one of claims 1 to 2, characterized in that, The visual signs parameters include at least one of hip stability parameters or elbow extension. The visual sign parameters are determined based on the corrected position movement sequence of each of the multiple key points, including at least one of the following: Based on the hip correction position movement sequence and its mean, a hip mean value sequence is determined; the covariance matrix determined based on the hip mean value sequence is processed using an eigenvalue decomposition algorithm to determine the hip motion vector; hip joint stability parameters are determined based on the hip correction position movement sequence, its mean, the hip motion vector, and the number of key point coordinates in the hip correction position movement sequence; or The elbow extension is determined based on the correction position movement sequence corresponding to the wrist key point, elbow key point, and shoulder key point.

6. The method according to any one of claims 1 to 2, characterized in that, The evaluation result is determined based on the evaluation values ​​of the visual signs parameters, the first compression parameter, and the gas flow rate parameter determined based on the gas sensor, including: The visual assessment value is determined based on the visual sign parameters and the weights corresponding to the visual sign parameters; The pressure evaluation value is determined based on the first pressure parameter and the weight corresponding to the first pressure parameter; The gas evaluation value is determined based on the gas flow rate parameter and the weight corresponding to the gas flow rate parameter; The evaluation result is obtained by integrating the visual evaluation value, the pressure evaluation value, and the gas evaluation value.

7. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 6.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images