Smart AED-based first-aid auxiliary method, system and apparatus and readable storage medium

Abstract

Disclosed in the present application are a smart AED-based first-aid auxiliary method, system and apparatus and a readable storage medium. The method comprises: on the basis of body posture key point detection and positioning features of a rescuer and hemodynamic parameter features of a patient, according to a CPR action standard identifier, training and establishing a CPR mapping relationship, and outputting a CPR mapping model; before implementation of CPR, identifying whether a first-aid condition satisfies a standard; and using the trained CPR mapping model to output CPR action standardization feedback, so as to perform CPR action standardization guidance to guide implementation of a CPR process. The system comprises a smart AED apparatus, a first-aid platform server and a first-aid medical staff terminal. The present solution combines deep learning, construction of important models before and after the implementation of the CPR and smart AED-based first-aid assistance with artificial intelligence assistant guidance during entire first-aid processes, so as to reduce requirements on the professional degree of rescuers under emergency conditions, thus ensuring that CPR first aid steps can be quickly and effectively performed on patients even in emergency situations without the presence of professional personnel.

Description

Intelligent AED first aid assistance method and system, device, and readable storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on September 27, 2023, with application number 202311256171.6 and invention name “Intelligent AED emergency assistance method and system, device, and readable storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of cardiopulmonary resuscitation technology, and in particular to an intelligent AED first aid assistance method and system, device, and readable storage medium with artificial intelligence-assisted guidance for the entire first aid process. Background Art

[0003] The statements in this section merely provide background technical information related to this application and do not necessarily constitute prior art.

[0004] In the early stages of cardiac arrest, 85% to 90% of patients experience ventricular fibrillation. The most effective treatment for ventricular fibrillation is early defibrillation with an AED (Automated External Defibrillator). Defibrillation is most effective within 3-5 minutes of onset. For every minute of delay, survival decreases by 7% to 10%. Approximately 540,000 people die from cardiac arrest in my country each year. Placing AEDs in public places is a crucial measure to reduce sudden death from cardiac arrest. External defibrillation with an AED is a crucial step in CPR (Cardio Pulmonary Resuscitation). The person using an AED is called the AED CPR performer (hereinafter referred to as the performer), and the person receiving external defibrillation with an AED is called the AED CPR recipient (hereinafter referred to as the patient).

[0005] Currently, AEDs have been widely used in public places outside hospitals for timely rescue of patients suffering from sudden cardiac arrest. The main structure of an AED is divided into power storage, main control unit, discharge unit, electrodes, etc. Since AED equipment is a professional device related to the life safety of patients, it usually requires implementers who have undergone frequent basic life support training and professional medical staff to perform first aid operations. However, when patients with sudden cardiac arrest appear in public places, not all places have professional first aid personnel and medical staff to operate AEDs. Therefore, there is a need for methods and devices that can provide first aid assistance during the operation of AEDs to provide corresponding first aid assistance to implementers of AEDs and CPR. The required first aid assistance must at least include providing guidance and instructions.

[0006] In the process of implementing this application, the applicant discovered that the existing technology for providing guidance and instructions for AED use still has at least the following problems:

[0007] 1. Currently, in the process of combining CPR with AEDs, only AEDs with CPR sensors have corresponding CPR feedback. AEDs without CPR sensors are far from sufficient in providing guidance and instructions during the CPR process.

[0008] 2. Currently available CPR assistance technologies, especially artificial intelligence technologies using images, mostly focus only on the guidance and instructions of the CPR process, but are still insufficient in the guidance and environmental judgment of the CPR preparation and defibrillation stages in combination with the use of AEDs.

[0009] 3. Currently, existing auxiliary methods for assisting the CPR implementation stage mainly focus on using image recognition technology to identify the posture of the implementer, which is used to solve the standardization of CPR implementation action requirements, but ignore the CPR recipient, that is, the patient's feedback during CPR. Improper or excessive CPR may cause secondary injuries to the patient.

[0010] Therefore, how to solve the above-mentioned problems of AED during cardiopulmonary resuscitation has become the subject to be studied and solved in this application.

[0011] Summary of the Invention

[0012] The purpose of this application is to provide an intelligent AED first aid assistance method and system, device, and readable storage medium.

[0013] To achieve the above objectives, the first aspect of the present application proposes an intelligent AED first aid assistance method, which comprises the following steps:

[0014] S100, training and establishing a CPR mapping model, using the implementer's human posture key point detection positioning features and the patient's hemodynamic parameter features, training and establishing a CPR mapping relationship according to CPR action standard identification, and outputting the CPR mapping model;

[0015] S200, before CPR is implemented, identify whether the emergency conditions meet the standards. After the intelligent AED device is activated, separate the foreground and background before CPR is implemented, identify the environmental background and determine whether the current emergency environment meets the emergency standards. Identify the human body foreground and determine whether the patient's body position characteristics meet the emergency standards.

[0016] S300, provide CPR action standard guidance and guidance when CPR is implemented. When CPR is implemented, use the human posture recognition model to locate the implementer's key points and obtain the implementer's human posture key point detection and positioning features, establish a human skin reflection model and provide real-time feedback on the patient's CPR feedback based on the patient's hemodynamic parameter characteristics. Combined with the implementer's human posture key point detection and positioning features and the patient's hemodynamic parameter characteristics, use the trained CPR mapping model to output CPR action standard feedback to provide CPR action standard guidance and guide the implementation of the CPR process.

[0017] The second aspect of the present application proposes an intelligent AED emergency assistance system, which is used for the intelligent AED emergency assistance in the method described in the first aspect. The innovation of the system is that it includes: an intelligent AED device, an emergency platform server, and an emergency medical terminal;

[0018] The intelligent AED device is a device for external defibrillation equipped with an AI recognition module. The AI ​​recognition module is equipped with a CPR mapping model that combines the key points of the implementer's body posture detection and positioning features with the patient's hemodynamic parameter features, and is trained and output after establishing a CPR mapping relationship based on CPR action standard identification. The AI ​​recognition module is used to identify whether the emergency conditions meet the standards before CPR is implemented and provide guidance and instruction on CPR action standards during CPR implementation.

[0019] The first aid platform server communicates with the intelligent AED device via long-distance wireless communication. The first aid platform server is used to receive data uploaded by the intelligent AED device, store and transfer the data required by the emergency doctor to the first aid medical terminal;

[0020] The emergency medical terminal is used to display the received data and conduct real-time audio and video calls with the intelligent AED device.

[0021] The third aspect of the present application provides an intelligent AED first aid assistance device, which is used for the intelligent AED first aid assistance in the method described in the first aspect. The intelligent AED first aid assistance device includes a main control module, a wake-up detection module, a video acquisition module, an audio acquisition module, a power supply module, a communication module, an AI recognition module, a storage module, a voice broadcast module, a video display module, and an early warning module.

[0022] The main control module is used for information collection, storage, analysis and decision-making;

[0023] The wake-up detection module is used to automatically wake up the AED data collection when the intelligent AED device is in use;

[0024] The audio acquisition module is used to collect real-time audio on site;

[0025] The video acquisition module is used to collect real-time video on site;

[0026] The power supply module is used to supply power to other modules;

[0027] The AI ​​recognition module is used to identify the electrode installation position, interference factors in the emergency environment, perform CPR quality analysis, identify the patient's physiological data and output error correction result information;

[0028] The storage module is used to store AED operation information, treatment information, detection information, and AI recognition feedback information;

[0029] The communication module is responsible for communicating with the emergency platform server and the emergency medical terminal;

[0030] The voice broadcast module is used to broadcast the voice prompts and the audio of the video call;

[0031] The video display module is used to display the video prompt and the video call;

[0032] The early warning module is responsible for issuing an alarm when the medical emergency terminal fails to respond to an emergency event.

[0033] The fourth aspect of the present application proposes a readable storage medium, on which a control program is stored. When the control program is executed by the AI ​​recognition module, the AI ​​recognition module executes the steps of the intelligent AED first aid assistance method described in the first aspect.

[0034] The relevant contents of this application are explained as follows:

[0035] 1. Through the implementation of the above-mentioned technical solution of the present application, before deploying the intelligent AED device, the positioning features of the implementer's human posture key points and the patient's hemodynamic parameter features are first detected, and the CPR mapping relationship is trained and established according to the CPR action standard identification, so as to provide a CPR mapping model for subsequent CPR implementation. In the process of training and establishing the CPR mapping model, both the implementer's CPR action specifications and the patient's feedback during CPR are taken into consideration, so as to provide correct and good CPR action standard guidance and guidance for subsequent CPR implementation. This is a deep learning process, and the accuracy and recognition speed can be gradually improved as the amount of training and the number of applications of the intelligent AED emergency assistance method increase. Before CPR is implemented, the foreground and background are separated, and recognition can be performed quickly. The environmental background is identified and whether the current emergency environment meets the emergency standards is determined. The human foreground is identified and whether the patient's body position characteristics meet the emergency standards is determined, and corresponding guidance and prompts are provided to ensure that the CPR environment is safe and the patient's posture state meets the emergency requirements, providing guidance for the implementation of CPR. Provide necessary technical identification and guidance to avoid secondary injury to the patient and to avoid reduced or ineffective CPR effectiveness; when CPR is implemented, the human posture recognition model is used to locate the implementer's key points and obtain the implementer's human posture key point detection and positioning features. This provides corresponding feature parameters for whether the rescuer's operating posture, CPR compression position, CPR compression posture, CPR compression depth, CPR compression frequency, CPR compression number, etc. are standard. A human skin reflex model is established and real-time feedback is provided on the patient's CPR feedback based on the patient's hemodynamic parameter characteristics. This provides corresponding feature parameters for the patient's human heart activity and blood flow activity status, which are very important when CPR is implemented. Combined with the implementer's human posture key point detection and positioning features and the patient's hemodynamic parameter characteristics, real-time feedback can also allow third parties other than the implementer to intervene to better provide additional guidance and cooperation based on the patient's human heart activity and blood flow activity status. When using the trained CPR mapping model to output CPR action standard feedback, CPR action standard guidance is provided to guide the implementation of the CPR process. The implementation of the above plan is a combination of deep learning, the construction of important models before and after CPR implementation, and intelligent AED emergency assistance with artificial intelligence-assisted guidance for the entire emergency process. It can reduce the professional requirements for implementers in emergency situations, so that implementers who have undergone less frequent basic life support training can also quickly perform CPR on patients. This ensures that CPR can be performed on patients quickly and effectively in emergencies without professional personnel, thereby improving patient survival rates and emergency effects, and providing accurate and complete treatment information for subsequent treatment, thereby further ensuring the patient's life safety and health protection.

[0036] Due to the application of the above scheme, this application has the following advantages and effects compared with the prior art:

[0037] 1. Through the implementation of the technical solution of the present application, before deploying the intelligent AED device, the positioning features of the implementer's human posture key points and the patient's hemodynamic parameter features are first detected, and the CPR mapping relationship is trained and established according to the CPR action standard identification, so as to provide a CPR mapping model for subsequent CPR implementation. In the process of training and establishing the CPR mapping model, both the implementer's CPR action specifications and the patient's feedback during CPR are taken into consideration, so as to provide correct and good CPR action specification guidance and guidance for subsequent CPR implementation. This is a deep learning process, and the accuracy and recognition speed can be gradually improved according to the increase in training volume and the increase in the number of applications of intelligent AED first aid auxiliary methods.

[0038] 2. Through the implementation of the technical solution of this application, the foreground and background are separated before CPR is implemented, and identification can be carried out quickly. The environmental background is identified and it is judged whether the current first aid environment meets the first aid standards. The human foreground is identified and it is judged whether the patient's body position characteristics meet the first aid standards, and corresponding guidance and prompts are provided to ensure that the CPR environment is safe and the patient's posture can meet the first aid requirements, providing necessary technical identification and guidance for the implementation of CPR, avoiding secondary injuries to patients, and avoiding reduced or ineffective CPR effects.

[0039] 3. Through the implementation of the technical solution of the present application, when CPR is implemented, the human posture recognition model is used to locate the key points of the implementer and obtain the implementer's human posture key point detection and positioning features. This provides corresponding feature parameters for whether the rescuer's operating posture, CPR compression position, CPR compression posture, CPR compression depth, CPR compression frequency, CPR compression times, etc. are standard. A human skin reflex model is established and the patient's CPR feedback is provided in real time based on the patient's hemodynamic parameter characteristics. This provides corresponding feature parameters for the patient's human heart activity and blood flow activity status, which are very important when CPR is implemented. In combination with the implementer's human posture key point detection and positioning features and the patient's hemodynamic parameter characteristics, real-time feedback can also allow third parties other than the implementer to intervene, so as to better provide additional guidance and cooperation based on the patient's human heart activity and blood flow activity status, and output CPR action standard feedback using the trained CPR mapping model to provide standardized guidance on CPR actions and guide the implementation of the CPR process.

[0040] 4. In summary, the implementation of the above plan is an intelligent AED first aid assistance that combines deep learning, the construction of important models before and after CPR implementation, and artificial intelligence-assisted guidance for the entire first aid process. It can reduce the professional requirements for implementers in emergency situations, so that implementers who have undergone less frequent basic life support training can also quickly perform CPR first aid on patients, thereby ensuring that CPR first aid can be performed on patients quickly and effectively in emergencies and without professional personnel. At the same time, it also combines the patient's real-time physical feedback throughout the process to improve the patient's survival rate and first aid effect, and provide accurate and complete treatment information for subsequent treatment, so as to further ensure the patient's life safety and health protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] FIG1 is a schematic diagram of a flow chart of an intelligent AED first aid assistance method according to an embodiment of the present application;

[0042] FIG2 is a schematic diagram of a detailed flow chart of an intelligent AED first aid assistance method according to an embodiment of the present application;

[0043] FIG3 is a schematic diagram of an intelligent AED emergency assistance system according to an embodiment of the present application;

[0044] FIG4 is a schematic diagram of an intelligent AED emergency assistance device according to an embodiment of the present application;

[0045] FIG5 is a system workflow diagram of an embodiment of the present application;

[0046] FIG6 is a schematic diagram of a color camera collecting human rPPG signals;

[0047] FIG7 is a schematic diagram of a dichroic reflection model;

[0048] Figure 8 shows the ECG and PPG waveforms. DETAILED DESCRIPTION

[0049] To make the above-mentioned objects, features, and advantages of the present application more clearly understood, the specific embodiments of the present application are described in detail below with reference to the accompanying drawings. The following description sets forth many specific details to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar improvements without violating the scope of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0050] The terms used herein are for describing specific embodiments only and are not intended to be limiting of the present invention. Singular forms such as "a," "the," "this," "this," and "the" as used herein also include plural forms.

[0051] The terms “first”, “second”, etc. used in this document do not specifically refer to an order or sequence, nor are they used to limit this case. They are only used to distinguish components or operations described with the same technical terms.

[0052] As used herein, “connected” or “positioned” may refer to two or more components or devices being in direct or indirect physical contact with each other, or may refer to two or more components or devices operating or moving with each other.

[0053] The terms “include”, “including”, “have”, etc. used in this document are open-ended terms, meaning including but not limited to.

[0054] Unless otherwise noted, the terms used herein generally have their ordinary meanings in the art, in the context of the present invention, and in the specific context. Certain terms used to describe the present invention are discussed below or elsewhere in this specification to provide additional guidance to those skilled in the art regarding the description of the present invention.

[0055] This application aims to provide intelligent AED emergency assistance with artificial intelligence-assisted guidance throughout the entire emergency process. It combines deep learning and the construction of important models before and after CPR implementation. It can reduce the professional requirements for implementers in emergency situations, so that implementers who have undergone less frequent basic life support training can also quickly perform CPR on patients. This ensures that CPR can be performed on patients quickly and effectively in emergencies without professional personnel. At the same time, it also combines the patient's real-time physical feedback throughout the process to improve the patient's survival rate and emergency effect.

[0056] Embodiment 1: Embodiment 1 of the present application proposes an intelligent AED first aid assistance method, which includes the following steps:

[0057] S100, training and establishing a CPR mapping model, using the implementer's human posture key point detection positioning features and the patient's hemodynamic parameter features, training and establishing a CPR mapping relationship according to CPR action standard identification, and outputting the CPR mapping model;

[0058] S200, before CPR is implemented, identify whether the emergency conditions meet the standards. After the intelligent AED device is activated, separate the foreground and background before CPR is implemented, identify the environmental background and determine whether the current emergency environment meets the emergency standards. Identify the human body foreground and determine whether the patient's body position characteristics meet the emergency standards.

[0059] S300, provide CPR action standard guidance and guidance when CPR is implemented. When CPR is implemented, use the human posture recognition model to locate the implementer's key points and obtain the implementer's human posture key point detection and positioning features, establish a human skin reflection model and provide real-time feedback on the patient's CPR feedback based on the patient's hemodynamic parameter characteristics. Combined with the implementer's human posture key point detection and positioning features and the patient's hemodynamic parameter characteristics, use the trained CPR mapping model to output CPR action standard feedback to provide CPR action standard guidance and guide the implementation of the CPR process.

[0060] In the above-mentioned S100 training step of establishing a CPR mapping model, before deploying the intelligent AED device, the implementer's human posture key points are used to detect and locate the features and the patient's hemodynamic parameter features, and the CPR mapping relationship is trained and established according to the CPR action standard identification, so as to provide a CPR mapping model for subsequent CPR implementation. In the process of training and establishing the CPR mapping model, both the implementer's CPR action specifications and the patient's feedback under CPR are taken into consideration, so as to provide correct and good CPR action specification guidance and guidance for subsequent CPR implementation. This is a deep learning process, and the accuracy and recognition speed can be gradually improved according to the increase in training volume and the increase in the number of applications of intelligent AED first aid auxiliary methods.

[0061] In the above-mentioned S200 step of identifying whether the first aid conditions meet the standards before CPR is implemented, the foreground and background are separated before CPR is implemented, which can quickly identify and judge whether the current first aid environment meets the first aid standards based on the environmental background identification, and whether the patient's body position characteristics meet the first aid standards based on the human foreground identification, and provide corresponding guidance and prompts to ensure that the CPR environment is safe and the patient's posture can meet the first aid requirements, provide necessary technical identification and guidance for the implementation of CPR, avoid secondary injuries to patients, and avoid reduced or ineffective CPR effects.

[0062] In the above-mentioned S300 step of providing standardized guidance and instruction for CPR actions during CPR implementation, when CPR is implemented, the human posture recognition model is used to locate the implementer's key points and obtain the implementer's human posture key point detection and positioning features. This provides corresponding feature parameters for whether the rescuer's operating posture, CPR compression position, CPR compression posture, CPR compression depth, CPR compression frequency, CPR compression times, etc. are standard. A human skin reflex model is established and real-time feedback is provided on the patient's CPR feedback based on the patient's hemodynamic parameter features. This provides corresponding feature parameters for the patient's human heart activity and blood flow activity status, which are very important when CPR is implemented. In combination with the implementer's human posture key point detection and positioning features and the patient's hemodynamic parameter features, real-time feedback can also allow third parties other than the implementer to intervene to better provide additional guidance and cooperation based on the patient's human heart activity and blood flow activity status. When using the trained CPR mapping model to output CPR action standard feedback, CPR action standard guidance is provided to guide the implementation of the CPR process.

[0063] In Example 1 of the present application, the first aid assistance method also includes starting the remote AI first aid function after the intelligent AED device is enabled. The intelligent AED device connects to the remote doctor through the first aid platform server, and uploads the foreground data and background data before CPR implementation, as well as the implementer data and patient data during CPR implementation to the first aid platform server. The remote doctor participates in the CPR first aid process with audio and video before and during CPR implementation. The remote doctor remotely guides the implementer on site to perform effective CPR and correctly use AED and other operations, so as to make up for the fact that the first aid scene is relatively urgent and the implementer cannot take into account all situations. As a professional remote doctor who fully participates in CPR first aid, he can provide reference and relevant opinions for the subsequent treatment of patients who have undergone CPR first aid, and further ensure that the patient's personal health can be well treated.

[0064] Specifically, after the intelligent AED device is enabled, it automatically wakes up the AED data collection, and performs AI identification of foreground data, background data, implementer data, and patient data. The AI ​​identification result corresponds to the CPR mapping model for identification and output, and provides voice guidance and screen display during the CPR action standard guidance, so that remote doctors at a remote location can synchronously see a series of first aid data such as the first aid environment, first aid conditions, implementer's actions, patient feedback, etc. at the first aid scene, which is more intuitive and fast, allowing remote doctors to understand the relevant first aid process at the first time, so that remote doctors can participate in CPR first aid more comprehensively. At the same time, these identifications and outputs can also enable implementers at the first aid scene to receive more comprehensive, accurate, and rapid CPR guidance, thereby providing patients with more professional AED first aid.

[0065] The following further describes Example 1 of the present application with one of the preferred embodiments in more detail.

[0066] The intelligent AED first aid assistance method of this preferred embodiment includes the following detailed steps as shown in Figure 2.

[0067] S100, training and establishing a CPR mapping model, using the implementer's body posture key point detection positioning features and the patient's hemodynamic parameter features, training and establishing a CPR mapping relationship according to CPR action standard identification, and outputting the CPR mapping model.

[0068] The step of training and establishing a CPR mapping model in S100 specifically includes the following steps:

[0069] S110, collecting training data sets, collecting video images of the implementer and the patient during training, and collecting CPR action standard identification;

[0070] S120, training video image preprocessing, grayscale and filtering preprocessing of the collected video image;

[0071] S130, obtaining the implementer's human body posture key point detection and positioning features during training, performing the implementer's human body posture key point detection and positioning, and obtaining the implementer's human body posture key point detection and positioning features;

[0072] S140, establishing a human skin reflection model of the patient during training, using optical rPPG technology to establish the human skin reflection model during training;

[0073] S150, obtaining hemodynamic parameter characteristics of the patient during training, and further calculating and obtaining the hemodynamic parameter characteristics of the patient during training using a human skin reflection model;

[0074] S160: Training and establishing a CPR mapping relationship, using the implementer's body posture key point detection positioning features and the patient's hemodynamic parameter features during training, and training and establishing a CPR mapping relationship according to the CPR action standard identification;

[0075] S170: Output CPR mapping model parameters. Output CPR mapping model parameters according to the CPR mapping relationship.

[0076] By implementing the above training process to establish a CPR mapping model, the problem of ignoring the patient's feedback during CPR during the standardization of CPR implementation action requirements is solved, so that the output CPR mapping model parameters combine the implementer's human posture key point detection and positioning characteristics and the patient's hemodynamic parameter characteristics, and the human skin reflection model during training is established using optical rPPG technology, so that the training results are more in line with the various situations that may occur to the implementer and the patient during actual CPR first aid implementation, thereby making the final CPR mapping model more reasonable and accurate, and providing better intelligent AED first aid assistance during CPR first aid.

[0077] S200, before implementing CPR, identify whether the first aid conditions meet the standards. After activating the intelligent AED device, separate the foreground and background before implementing CPR. Identify the environmental background and judge whether the current first aid environment meets the first aid standards. Identify the human foreground and judge whether the patient's body position characteristics meet the first aid standards.

[0078] The steps to identify whether first aid conditions meet the standards before implementing S200CPR include the following:

[0079] S210, collecting video images before CPR is implemented, collecting video images of the environment where the AED device is located and video images of the patient's body before CPR is implemented;

[0080] S220, pre-processing the video image before CPR implementation, gray-scaling and filtering the collected video image of the environment where the AED device is located and the video image of the patient's body;

[0081] S230, performing foreground and background separation before CPR implementation, separating the environmental background from the foreground human body; in the step of performing foreground and background separation before CPR implementation in S230, using a deep low-rank model to separate the foreground and background; extracting the background using the residual component, and comparing the background residual changes during defibrillation; automatically describing the patient's body position characteristics using the low-rank component;

[0082] S240: Determine whether the current emergency environment meets the emergency standards. Calculate the environmental interference feedback indicator before CPR implementation based on the environmental background to determine whether the environmental background meets the emergency standards. In the step of determining whether the current emergency environment meets the emergency standards in S240, determine whether the output camera distance is appropriate and whether the environment is safe and dry based on the environmental interference feedback indicator.

[0083] S250, determine whether the patient's body position characteristics meet the first aid standards, extract the patient's body position characteristics of the foreground body, and determine whether the patient meets the first aid standards based on the patient's body position characteristics; in the step of S250 determining whether the patient's body position characteristics meet the first aid standards, determine whether the patient's posture meets the requirements, whether the clothing meets the requirements, and whether the electrode placement is correct based on the patient's body position characteristics.

[0084] S260: Output the identification of the foreground and background before CPR implementation.

[0085] By implementing the above pre-CPR process, the implementer can intelligently determine whether the first aid conditions meet the standards before performing CPR, ensuring the safety of the first aid environment for the implementer, preventing the patient from suffering secondary injuries, and providing correct intelligent assistance to the implementer performing CPR in emergency situations, thereby reducing the implementer's burden. At the same time, the above pre-CPR process can more quickly and accurately separate the foreground and background and judge and identify them separately, with high accuracy, and can eliminate many interference factors to further ensure the safety of the first aid environment.

[0086] By implementing the above process before CPR, the precision, accuracy and efficiency of foreground and background separation are further enhanced, and a more reasonable and rapid means is adopted to identify whether the emergency conditions meet the standards before CPR is implemented.

[0087] S300, provide CPR action standard guidance and guidance when CPR is implemented. When CPR is implemented, use the human posture recognition model to locate the implementer's key points and obtain the implementer's human posture key point detection and positioning features, establish a human skin reflection model and provide real-time feedback on the patient's CPR feedback based on the patient's hemodynamic parameter characteristics. Combined with the implementer's human posture key point detection and positioning features and the patient's hemodynamic parameter characteristics, use the trained CPR mapping model to output CPR action standard feedback to provide CPR action standard guidance and guide the implementation of the CPR process.

[0088] The steps for providing guidance and instruction on CPR techniques during S300CPR implementation include:

[0089] S310, collecting video images during CPR implementation, collecting video images of the implementer and the patient during CPR implementation;

[0090] S320, pre-processing the video images during CPR implementation, gray-scaling and filtering the collected video images of the implementer and the patient;

[0091] S330, obtaining key point detection and positioning features of the implementer's human body posture when performing CPR, performing key point detection and positioning of the implementer's human body posture, and obtaining key point detection and positioning features of the implementer's human body posture when performing CPR;

[0092] S340, establishing a human skin reflection model of the patient during CPR implementation, using optical rPPG technology to establish a human skin reflection model during CPR implementation;

[0093] S350, obtaining hemodynamic parameter characteristics of the patient during CPR implementation, and further calculating and obtaining the hemodynamic parameter characteristics of the patient during CPR implementation using a human skin reflection model;

[0094] S360, calculating CPR action standard feedback, using the trained CPR mapping model to calculate CPR action standard feedback based on the key points of the implementer's human posture and the patient's hemodynamic parameter characteristics during CPR implementation;

[0095] S370: Output CPR action standard feedback, feed back the CPR action standard to the implementer who is performing CPR, provide CPR action standard guidance, and guide the implementation of the CPR process.

[0096] Through the above specific implementation of providing CPR action standard guidance and guidance during CPR implementation, we can better obtain the implementer's actions and the patient's reactions during actual CPR implementation, and combine this with the CPR mapping model to obtain more reasonable and accurate feedback, so as to provide correct and rapid CPR action standard guidance, reduce the professional requirements of the implementer, and ensure that the patient can be rescued correctly.

[0097] The establishment of the human skin reflectance model in steps S100 and S300 is based on remote photoplethysmography. Using a multi-wavelength RGB camera within a camera, the camera detects subtle color changes caused by pulses on the human skin surface, enabling non-contact monitoring of cardiac activity and hemodynamic parameters to establish a human skin reflectance model. The principle of using rPPG technology to establish a skin reflectance optical model is to use the light source formed by diffuse reflection from the skin, which is received by the camera through its spectral and color channel sensitivity. This color change is caused by changes in skin deformation / specular reflection caused by the RPC operator's operation and subtle color changes caused by pulses. This color change, known as the skin diffuse reflectance spectrum, reflects the state of human blood flow due to varying degrees of absorption and reflection by the skin's epidermal tissue and internal bone. Using artificial intelligence methods to analyze the received light can provide feedback on human hemodynamic parameters.

[0098] Embodiment 2, as shown in FIG3 , embodiment 2 of the present application proposes an intelligent AED emergency assistance system, which includes: an intelligent AED device, an emergency platform server, and an emergency medical terminal;

[0099] The intelligent AED device is a device for external defibrillation equipped with an AI recognition module. The AI ​​recognition module is equipped with a CPR mapping model that combines the key points of the implementer's body posture detection and positioning features with the patient's hemodynamic parameter features, and is trained and output after establishing a CPR mapping relationship based on CPR action standard identification. The AI ​​recognition module is used to identify whether the emergency conditions meet the standards before CPR is implemented and provide guidance and instruction on CPR action standards during CPR implementation.

[0100] The first aid platform server communicates with the intelligent AED device via long-distance wireless communication. The first aid platform server is used to receive data uploaded by the intelligent AED device, store and transfer the data required by the emergency doctor to the first aid medical terminal;

[0101] The emergency medical terminal is used to display the received data and conduct real-time audio and video calls with the intelligent AED device.

[0102] In the second embodiment of the present application, long-distance wireless methods include but are not limited to cellular networks, Wi-Fi, and other methods. The emergency medical terminal can be a mobile terminal, such as a smartphone or tablet computer, with real-time network communication and video call functions, or a laptop computer or a PC equipped with a camera and microphone, used to remotely guide on-site personnel to perform effective CPR and use AEDs. An alarm device is also provided on the emergency medical terminal to alarm when the emergency medical terminal fails to respond to an emergency incident.

[0103] Embodiment 3, as shown in FIG4 , embodiment 3 of the present application proposes an intelligent AED first aid auxiliary device, which is used for the intelligent AED first aid auxiliary in the method described in embodiment 1 of the present application. The intelligent AED first aid auxiliary device includes a main control module, a wake-up detection module, a video acquisition module, an audio acquisition module, a power supply module, a communication module, an AI recognition module, a storage module, a voice broadcast module, a video display module, and an early warning module;

[0104] The main control module is used for information collection, storage, analysis and decision-making;

[0105] The wake-up detection module is used to automatically wake up the AED data collection when the intelligent AED device is in use;

[0106] The audio acquisition module is used to collect real-time audio on site;

[0107] The video acquisition module is used to collect real-time video on site;

[0108] The power supply module is used to supply power to other modules;

[0109] The AI ​​recognition module is used to identify the electrode installation position, interference factors in the emergency environment, perform CPR quality analysis, identify the patient's physiological data and output error correction result information;

[0110] The storage module is used to store AED operation information, treatment information, detection information, and AI recognition feedback information;

[0111] The communication module is responsible for communicating with the emergency platform server;

[0112] The voice broadcast module is used to broadcast the voice prompts and the audio of the video call;

[0113] The video display module is used to display the video prompt and the video call;

[0114] The early warning module is responsible for issuing an alarm when the medical emergency terminal fails to respond to an emergency event.

[0115] The practical application of the intelligent AED emergency assistance system of the present application is described below with reference to the system workflow shown in FIG5 .

[0116] The system workflow of the intelligent AED emergency assistance system is as follows:

[0117] (1) After turning on the intelligent AED emergency assistance device, it automatically wakes up the AED data collection and starts the remote AI emergency function.

[0118] (2) After the intelligent first aid assistance function is turned on, the AI ​​recognition module of the intelligent AED first aid assistance device is automatically turned on, and multi-angle real-time images of the first aid scene are obtained through the 360-degree movable camera to collect relevant data for AI image recognition.

[0119] (3) After the intelligent AED emergency auxiliary device is activated, the emergency event is transmitted to the emergency medical terminal through the emergency platform server, and the doctor is notified through text messages, system message reminders, etc. The alarm device provides double insurance for the doctor to quickly respond to the emergency event.

[0120] (4) The AI ​​recognition module determines whether the patient currently needs CPR. If CPR is not required, AED defibrillation is performed directly.

[0121] (5) When the patient needs CPR, the AI ​​recognition module first identifies whether the current emergency environment meets the standards.

[0122] (6) After confirming that the current emergency environment meets the emergency standards, the AI ​​recognition module identifies whether the patient's electrodes are placed correctly.

[0123] (7) During the CPR process, the AI ​​recognition module will combine multiple algorithm models to identify whether the implementer's CPR operation meets the standards.

[0124] (8) The voice broadcast module and video display module will provide voice and video prompts for the error correction feedback after the AI ​​algorithm model data is processed.

[0125] (9) During the first aid process, the intelligent AED first aid auxiliary device establishes a video call with the first aid medical terminal through the first aid platform server.

[0126] (10) Remote doctors can see the results of AI data processing and analysis in real time on the screen of the emergency medical center and make correct decisions.

[0127] (11) The storage module stores AED operation information, treatment information, detection information, and AI recognition feedback information in real time.

[0128] (12) The communication module uploads the collected AED operation data, monitoring data, on-site real-time audio and video data, data fed back by the AI ​​recognition module, and positioning data to the emergency platform server in real time and forwards it to the emergency medical terminal.

[0129] (13) The emergency medical terminal displays the audio and video information, positioning information, AED data and information fed back by the AI ​​recognition module sent by the AED intelligent module in real time, and sends audio and video data to the AED through the emergency platform server.

[0130] (14) The intelligent module communicates with the emergency platform server through the communication module, obtains the audio and video data sent by the emergency platform server, and broadcasts it through the voice broadcast module and displays it through the video display module.

[0131] The following is a detailed introduction to the AI ​​recognition module in the intelligent AED device.

[0132] The CPR process combined with AED mainly involves identifying whether the current emergency environment meets the standards and whether the patient's electrodes are placed correctly, as well as identifying whether the CPR operation meets the standards, as corresponding to processes (5) and (6).

[0133] The main functions of the AI ​​recognition module include:

[0134] 1. In the pre-CPR stage, a deep low-rank model is used to separate the foreground and background, extract the foreground patient's body contours respectively, and automatically describe the body position features with low-rank components. Based on these features, it is output whether the patient's posture meets the requirements, whether the clothing and electrode placement are correct; the background is extracted with the residual component, and the background residual changes during defibrillation are compared. The environmental interference feedback mark is calculated, and based on the mark, it is output whether the camera distance is appropriate and whether the environment is safe and dry.

[0135] 2. During the real-time CPR phase, the operator's key points are located using a human posture keypoint model, and a human skin reflectance model is established using optical rPPG technology. This provides real-time feedback on the patient's hemodynamic parameters, including CO, BVR, and SV. Combining the operator's key point location and the patient's hemodynamic parameters as features, the pre-trained mapping model outputs CPR action guidelines to guide the implementation of the CPR process.

[0136] Therefore, the functions of the AI ​​recognition module are divided into AI module 1 and AI module 2.

[0137] The specific functions provided by AI module 1 and AI module 2 are shown in Table 1:

[0138] Table 1

[0139] The implementation method of AI module 1 technology in the AI ​​recognition module is achieved through a foreground-background separation model.

[0140] The use of an AED during CPR requires standardization of the patient's condition and the wearing of the device, as well as ensuring that the patient is not affected by the external environment during electrode defibrillation. Using AI-based image foreground and background separation technology, the foreground human body and background environment are separated. The foreground image represents the patient's upper body outline to locate key points on the body, determine whether clothing is worn, and determine the body's lying position. The background image represents the patient's body and the external environment in which the AED is located, performing matching calculations to eliminate interference from the environment during defibrillation.

[0141] The auxiliary process of the CPR preparation phase based on the depth image foreground and background separation technology can be referred to as shown in Table 2 as follows:

[0142] Table 2

[0143] For S230, the foreground and background of the continuously collected video are separated and extracted based on the low-rank decomposition theory, as follows: X = Az + E Formula (1)

[0144] Here, X represents the captured video image matrix. Each image must be converted into a (0, 255) grayscale matrix. Az represents the foreground portion obtained after low-rank decomposition and is the product of the low-rank representation matrix and the low-rank coefficient matrix. z and A have the following meanings: z represents the low-rank representation matrix. In image and video applications, the low-rank representation matrix is ​​used to represent feature images. In this scenario, feature images are directly related to the positioning points of the human body. A represents the low-rank coefficient matrix, which is used to represent the dynamic coefficient changes of the foreground. E represents the background portion described by the foreground human body outline minus the captured video image matrix.

[0145] By implementing the above specific process before CPR implementation, the environmental interference feedback mark before CPR implementation can be accurately and effectively calculated, and the patient's body position characteristics of the foreground body can be extracted to provide support for intelligent AED first aid assistance.

[0146] The images and videos collected in reality represent the manifold structure of the three-dimensional space. In order to further enhance the nonlinear characteristics of the low-rank decomposition, we nonlinearize Equation (1) and use the Frobenius norm to constrain the low rank of the matrix, converting the real-time matrix decomposition solution process into an extreme value problem under the Frobenius norm for solution:

[0147] Where Π is a nonlinear function. Usually, we use a deep network to represent and solve the nonlinearity of formula (2). Specifically, a multi-layer neural network is used to perform a low-rank decomposition model as follows:

[0148] Among them, x t Represents the image grayscale matrix at a certain time t; in the forward propagation process of the network, the process of establishing the human body contour is g (K+1) (g (K) (...g (1) (z i ,Θ (1) ),Θ (K) ),Θ (K+1) ). In this representation, g (k) (·) represents the activation function of a single-layer neural network, k=1,2,...K+1 represents the depth of the neural network, Θ (k) Represents the parameter set of a single-layer neural network, as follows: Θ (k) ={W (k) ,b (k) Formula (4)

[0149] Where W (k) ,b (k)Represent the weight and bias parameters of a single-layer neural network respectively. Therefore, the nonlinear representation of a single-layer neural network in formula (3) is specifically: g (k) (z,Θ (k) )=g (k) (W (k) z+b) Formula (5)

[0150] In formula (2), during the forward propagation of the network, the background residual at a certain time t can be calculated as: E t =x t -g (K+1) (g (K) (...g (1) (z t ,Θ (1) ),Θ (K) ),Θ (K+1) ) Formula (6)

[0151] For S240, we use the scale-invariant feature transform (SIFT) computer vision algorithm model, which is used to detect and describe local features in images. It searches for extreme points in the spatial scale and extracts their position, scale, and rotation invariants to represent the key point results of the human body and the device.

[0152] Aligning S250, the overall average background can be represented by the average of the overall background E, that is, We calculate the background E at each time t t With average background The difference between and is used to determine whether there is interference at the current moment, as follows:

[0153] If the difference ε is greater than the threshold set by the system, we judge that there is environmental interference in the background at this time, otherwise there is no environmental interference.

[0154] By implementing the above specific processes before CPR implementation, the environmental interference feedback mark before CPR implementation can be calculated more accurately and effectively, and the patient's body position characteristics of the foreground body can be extracted to provide greater support for intelligent AED emergency assistance.

[0155] For the low-rank model, the iterative process of the foreground and background separation algorithm is mainly the ε process of updating the discrimination threshold according to the user's use, as follows:

[0156] 1. In actual use, the model obtains the foreground and background separation results through the following inference process: Az = g (K+1)(g (K) (...g (1) (z i ,Θ (1) ),Θ (K) ),Θ (K+1) ) E=Xg (K+1) (g (K) (...g (1) (z t ,Θ (1) ),Θ (K) ),Θ (K+1) )

[0157] 2. The user follows the system prompts, and the video frame corresponding to the actual operation initiated by the user is

[0158] 3. The current moment background calculated based on the corresponding moment and average background They are:

[0159] 4. Update the discrimination threshold based on the current background and the average background difference:

[0160] The implementation method of AI module 2 technology in the AI ​​recognition module is achieved by combining CPR guidance with the human skin feedback optical model.

[0161] Existing assistance methods for the CPR implementation phase primarily focus on using image recognition technology to identify the practitioner's posture, addressing the standardization of CPR action requirements. For example, human posture key point detection technology is widely used in intelligent image-assisted CPR applications. Widely used models include the Pose Landmark Model for human body key point location and detection, and residual networks such as ResNet-18 and ResNet-50. These models are used to extract key point features and analyze the operator's movements.

[0162] However, the above methods ignore the CPR recipient, that is, the patient's feedback during CPR. Inappropriate or excessive CPR may cause secondary harm to the patient. To address the above issues, the solution proposed in this application proposes to establish a skin reflection optical model based on rPPG technology, which is used to improve the current human posture key point detection model by combining patient hemodynamic parameter feedback.

[0163] The main process of implementing the AI ​​module 2 function is divided into training process and online inference process, which are as follows:

[0164] The training process (i.e., training to establish the S100CPR mapping model) can be referred to Table 3:

[0165] Table 3

[0166] The online inference process (i.e., providing CPR action standard guidance and guidance when training and establishing S300 CPR implementation) can be referred to Table 4:

[0167] Table 4

[0168] Regarding steps S130 and S330 in the training and online inference process: the Pose Landmark Model is used to monitor and locate 33 key points of the operator's body.

[0169] For steps S140 and S340 in the training and online inference processes: Based on remote photoplethysmography (rPPG), we use a multi-wavelength RGB camera in the camera to detect subtle color changes caused by pulses on the surface of human skin, thereby achieving non-contact monitoring of human cardiac activity and hemodynamic parameters.

[0170] The principle of using rPPG technology to build a skin reflectance optical model is to use the light source formed by the skin's diffuse reflection, and the camera receives its spectrum and color channel sensitivity. This color is due to the skin deformation / specular reflection changes caused by the RPC operator's operation and the subtle color changes caused by the pulse. This color change, namely the skin diffuse reflectance spectrum, is caused by the varying degrees of absorption and reflection by the skin's epidermis and internal bone, and can reflect the state of human blood flow. Using artificial intelligence methods to analyze the received light, feedback on human hemodynamic parameters can be obtained.

[0171] Figure 6 illustrates the use of a color camera to capture rPPG signals. Consider a light source illuminating a patch of human skin containing pulsating blood and a remote color camera recording this image. The light source is assumed to have a constant spectral composition but varying intensity (the intensity observed at the camera depends on the distance from the light source to the skin tissue and the camera sensor). The skin measured by the camera has a specific color (the skin color measured by the camera is a combination of the light source (e.g., intensity and spectrum), the inherent skin color, and the sensitivity of the camera's color channels). This color varies over time due to variations in intensity / specular reflection caused by motion and subtle color variations caused by pulses. These temporal variations are proportional to the brightness intensity level.

[0172] Therefore, in establishing the skin reflection optical model in this application, a dichromatic reflection model is used. As shown in Figure 7, specular reflection is the specular light reflected from the skin surface and does not contain pulsating information. Therefore, its spectral composition is comparable to that of the light source. Body movement affects the geometric structure between the light source, the skin surface, and the camera, and is time-dependent. Diffuse reflection is related to the absorption and scattering of light in skin tissue. The hemoglobin and melanin content in skin tissue determines chromaticity.

[0173] Based on the dichromatic reflectance model, the reflectance of each skin pixel in the recorded image sequence can be defined as a time-varying function in the RGB channels: k (t)=I(t)·(V s (t)+V d (t))+V n (t) Formula (8)

[0174] Among them, C k (t) represents the pixel value of the kth skin pixel at time t; I(t) represents the light intensity, which is determined by the brightness change of the light source itself and the distance between the light source, skin tissue and camera, and is affected by V s (t) and V d (t) Modulation of two reflections, where V s (t) represents specular reflection, which means the mirror-like reflected light emitted by the skin; V d (t) represents diffuse reflection, which represents the light scattered and absorbed by skin tissue; and V n (t) represents the quantization noise of the camera.

[0175] Specular Reflection V s It comes from the skin surface, similar to the light reflection from a mirror. Since most of the light is reflected by the skin surface, only a small part of the light enters the skin tissue. Therefore, the mirror reflection is the main part of the two reflection components, and it does not contain any pulse information. However, due to the deformation of the patient's skin caused by CPR operation, the angle and distance between the light, skin surface and camera change, so the mirror reflection also contains a variable component. Therefore, the mirror reflection V s It can be defined as: V s (t) = U s (s0+s(t)) Formula (9)

[0176] Where U s The unit color vector representing the specular reflection light spectrum captured by the camera, i.e., the contribution of the three colors red, green, and blue to the specular reflection light intensity; S0 and s(t) represent the light intensity of the DC and AC parts of the specular reflection, respectively.

[0177] Diffuse Vd It is related to the light absorbed and reflected by the skin tissue and is the main component of the pulse signal. The effect of skin tissue on light reflection is mainly related to the pigmentation of the skin epidermis, such as melanin and carotene, and the concentration of hemoglobin in the blood. Among them, melanin and carotene affect the intrinsic reflection of the skin and remain unchanged over time. The concentration of hemoglobin changes periodically with the activity of the heart. Therefore, the intensity of light reflected by hemoglobin will also change periodically, thus reflecting the pulse information. Therefore, the diffuse reflection V d (t) can be defined as: V d (t) = U d ·d0+U p p(t) Formula (10)

[0178] Where U d U represents the unit color vector of the skin epidermis, whose direction is mainly determined by pigmentation and is related to the depth of skin color; d0 represents the intensity of inherent diffuse reflection. Since blood absorbs different colors of light differently, U p It represents the contribution of different color channels in the pulse signal, and p(t) represents the intensity of the pulse signal.

[0179] It can be seen that the mirror reflection V s and diffuse V d Both contain a static part that does not change with time and a dynamic part that changes with time. Combining the static parts of the two reflections, it can be expressed as: U c c0=U s ·s0+U d ·d0 Formula (11)

[0180] Where U c The unit color vector representing the intrinsic reflectance of the skin; c0 represents the light intensity of the intrinsic reflectance.

[0181] Similarly, the light intensity I(t) in equation (8) can also be divided into the intrinsic light intensity I0 and the time-varying light intensity I0·i(t). I0 and I0·i(t) have the same direction and can be expressed as: I(t)=I0·(1+i(t)) Equation (12)

[0182] Substituting equations (10) and (11) into the model represented by equation (8), the skin reflection model of the CPR patient is obtained as follows: k (t)=I0·(1+i(t))·(U c c0+U s ·s(t)+p(t))+V n(t) Formula (13)

[0183] Most of the existing rPPG methods construct the time domain signal by spatially averaging the pixels in the ROI area of ​​each frame. The spatial averaging of sufficient pixels can effectively reduce the quantization noise of the camera (i.e., V n (t)). At the same time, compared with the DC component captured by the camera, the intensity of the AC component is very small. Therefore, the product terms of the AC component, such as i(t)·s(t) and i(t)·p(t), can be ignored. Considering the above two points and expanding Equation (13), we can obtain: C k (t)≈I0·U c ·c0+I0·U c c0 i(t) + I0 U s ·s(t)+I0·U p p(t) Formula (14)

[0184] Under the skin reflection optical model, the observed value C k (t) becomes a linear mixture of a DC component and three AC source signals i(t), s(t) and p(t). They are: I0·U representing the DC component c c0, represents the AC component of light intensity I0·U c c0 i(t), specular reflection component I0 U s ·s(t), and the pulse signal component I0·U p ·p(t).

[0185] For steps S150 and S350 during training and online inference: We first calculate the one-dimensional pulse signal of rPPG by weighted averaging the k skin pixels in equation (14):

[0186] From formula (14), we can see that C(t) is a one-dimensional signal related to time, which mainly contains two parts of information. One part comes from the influence of illumination and light source, including I0·U c ·c0、I0·U c ·c0·i(t) and I0·U s ·s(t); the other part comes from the pulse separation signal I0·U p p(t), the pulse component signal, represents the ventricular blood supply process. The heart's periodic contraction and relaxation pump blood from the heart into the arteries and then back into the heart through the veins, forming an organic cycle. The pulse wave is formed by the heartbeat propagating along the arteries and blood flow to the periphery. The ventricular relaxation and contraction correspond to the diastolic peak and absorption peak of the signal. Therefore, C(t) is directly related to hemodynamic parameters. Typical hemodynamic parameters can be calculated using the following methods:

[0187] Heart rate: The highest peak value of the signal represents the absorption peak of the one-dimensional pulse signal in equation (15), corresponding to the ventricular contraction process. Similar to the way of calculating the RRI of ECG signals, the time interval between two absorption peaks (PPI) can be calculated to calculate the heart rate.

[0188] HRV: Heart rate variability (HRV) refers to the slight variations in the timing between successive sinus rhythm cycles. Under physiological conditions, HRV is primarily due to the autonomic activity of the sinoatrial node, which typically varies by tens of milliseconds between beats. However, HRV during CPR directly reflects changes in the patient's baroreflex and the regulation of respiratory activity. The number of beats per minute (i.e., pulse rate) and heart rate are consistent. Therefore, the peak-to-peak interval (PPI) of the pulse wave signal can be used instead of the RRI (R wave interval) in the ECG signal to calculate HRV.

[0189] SV: Stroke volume refers to the amount of blood ejected from one ventricle per heartbeat. During CPR, SV is directly related to the operator's compressions. We use the pulse circuit analysis method to calculate the pulse stroke volume as follows: SV ≈ ∫dC′(t) / dt Equation (16)

[0190] The above equation, C'(t), represents one complete cycle of the C(t) signal. The stroke volume (SV) is estimated by integrating the signal change from end-diastole to end-systole, which is the systolic portion of the C(t) signal following the systolic peak until aortic valve closure.

[0191] CO: CO refers to the amount of blood pumped out by the left or right ventricle per minute, which is the product of stroke volume and heart rate. It is important to monitor during CPR.

[0192] Referring to the ECG and PPG waveforms shown in Figure 8, the peak (R wave) of the ECG comes from ventricular contraction, while the peak (main wave) of the PPG comes from vasoconstriction. The pulse transit time (PTT) reflects the time it takes for blood to travel from the heart to the measurement site. Heart rate variability (HRV) can be calculated by calculating the interval between R waves (RRI) in the ECG signal or the interval between main peaks in the PPG.

[0193] Regarding steps S160 and S360 in the training and online inference process: the process of guiding the CPR action standard based on the implementer's rescue action and the patient's blood flow status feedback is the process of establishing a mapping relationship between the implementer's body key point positioning and the patient's hemodynamic parameters as features and the CPR action implementation standard identifier as feedback, as follows: Y = f(X)

[0194] The input feature X consists of two sets of features and composition, Represents the concatenation of matrices and vectors. and They represent the key points of the operator's body and the hemodynamic parameters of the patient. Y represents the numerical identifier of the CPR action implementation standard.

[0195] This mapping relationship can be determined through a large number of experiments, combined with observation and statistics; it can also be calculated using machine learning and deep learning methods. The CPR action standard identification used for training needs to be manually marked.

[0196] Below we use the multivariate regression model to establish the above mapping relationship: Y=W·X+b

[0197] In the above formula, W and b represent the parameter weights of the model respectively. The model parameters in the embodiment of this application mainly include:

[0198] During the training process, we mainly use X calculated from step S130 to step S150 and the CPR action standard identifier Y contained in the training data set to train and obtain model parameters W and b;

[0199] In the real-time inference process, we obtain the feedback feature set X representing the implementer's rescue action and the patient's blood flow state, as well as the trained model parameters W and b in real time through steps S330 to S350, and calculate the CPR action standard identifier Y through formula (18).

[0200] At the same time, this application also proposes a solution for iterating the algorithm of the CPR mapping relationship model. The algorithm iteration process of the CPR mapping relationship model is as follows:

[0201] 1. During actual use, the model obtains the CPR action standard identifier Y through the following inference process: Y = W·X + b

[0202] 2. The user's actual operation is converted into a numerical value according to the system prompts

[0203] 3. The model will As the target output, the collected image data X is used as the model input to update the model parameters.

[0204] 4. We transform the model parameter update problem into a least squares problem for solution. The specific model updates the original model parameters W and b through the following formula to achieve the update of the model.

[0205] in(·) T Performs the matrix transpose operation.

[0206] By iterating the foreground and background separation algorithm and the CPR mapping relationship model algorithm, the data collected during the actual use of the intelligent AED device and the corresponding actions performed by the implementer can be further iterated and updated to the relevant model algorithm, so that the intelligent guidance function of the intelligent AED device can be more in line with the actual operation of the implementer, and the relevant model algorithm itself can be updated from time to time. With the promotion and use of the present application solution, it can become more intelligent.

[0207] The implementation of the above embodiments has achieved the purpose of this application. It combines deep learning, the construction of important models before and after CPR implementation, and intelligent AED emergency assistance with artificial intelligence-assisted guidance for the entire emergency process. It can reduce the professional requirements for implementers in emergency situations, so that implementers who have undergone less frequent basic life support training can also quickly perform CPR on patients, thereby ensuring that CPR can be performed on patients quickly and effectively in emergency situations without professional personnel, improving patient survival rates and emergency effects, and providing accurate and complete treatment information for subsequent treatment, thereby further ensuring the patient's life safety and health protection.

[0208] The above embodiments are intended only to illustrate the technical concepts and features of this application. Their purpose is to enable those familiar with the art to understand the content of this application and implement it accordingly. They are not intended to limit the scope of protection of this application. Any equivalent changes or modifications made in accordance with the spirit of this application shall be included in the scope of protection of this application.

Claims

1. An intelligent AED first aid assistance method, wherein: The first aid assistance method comprises the following steps: S100, training and establishing a CPR mapping model, using the implementer's human body posture key point detection positioning features and the patient's hemodynamic parameter features, training and establishing a CPR mapping relationship according to the CPR action standard identification, and outputting the CPR mapping model; S200, before CPR is implemented, identify whether the emergency conditions meet the standards. After the intelligent AED device is activated, separate the foreground and background before CPR is implemented, identify the environmental background and judge whether the current emergency environment meets the emergency standards, and identify the human body foreground and judge whether the patient's body position characteristics meet the emergency standards; S300, provide CPR action standard guidance and direction when implementing CPR. When implementing CPR, use the human posture recognition model to locate the implementer's key points and obtain the implementer's human posture key point detection and positioning features, establish a human skin reflection model and provide real-time feedback on the patient's CPR feedback based on the patient's hemodynamic parameter characteristics. Combined with the implementer's human posture key point detection and positioning features and the patient's hemodynamic parameter characteristics, use the trained CPR mapping model to output CPR action standard feedback to provide CPR action standard guidance and guide the implementation of the CPR process.

2. The intelligent AED first aid assistance method according to claim 1, wherein: The first aid assistance method also includes starting a remote AI first aid function after activating the intelligent AED device. The intelligent AED device connects to a remote doctor through a first aid platform server, and uploads foreground data and background data before CPR implementation, as well as implementer data and patient data during CPR implementation to the first aid platform server. The remote doctor participates in CPR first aid through audio and video throughout the entire process before and during CPR implementation.

3. The intelligent AED first aid assistance method according to claim 2, wherein: After the intelligent AED device is enabled, it automatically wakes up the AED data collection, and performs AI identification of foreground data, background data, implementer data, and patient data. It outputs the identification based on the AI ​​identification result corresponding to the CPR mapping model, and provides voice guidance and screen display during the CPR action standard guidance.

4. The intelligent AED first aid assistance method according to claim 1, wherein: The steps to identify whether the first aid conditions meet the standards before implementing S200CPR include the following steps: S210, collecting video images before CPR is implemented, collecting video images of the environment where the AED device is located and video images of the patient's body before CPR is implemented; S220, preprocessing the video image before CPR implementation, graying and filtering the collected video image of the environment where the AED device is located and the video image of the patient's body; S230, performing foreground and background separation before CPR implementation, separating the environmental background from the foreground human body; S240, determining whether the current emergency environment meets the emergency standards, calculating the environmental interference feedback mark before CPR implementation according to the environmental background, and determining whether the environmental background meets the emergency standards; S250, judging whether the patient's body position characteristics meet the first aid standard, extracting the patient's body position characteristics of the foreground body, and judging whether the patient meets the first aid standard according to the patient's body position characteristics; S260: Output the judgment mark of the foreground and background before CPR is implemented.

5. The intelligent AED first aid assistance method according to claim 4, wherein: In the step of performing foreground and background separation before CPR implementation in S230, a deep low-rank model is used to separate the foreground and background; the background is extracted by residual components, and the background residual changes during defibrillation are compared; the patient's body position characteristics are automatically recorded by low-rank components; In the step of determining whether the current emergency environment meets the emergency standard at S240, the output is determined based on the environmental interference feedback flag. Whether the camera distance is appropriate and the environment is safe and dry; In the step S250 of determining whether the patient's body position characteristics meet the first aid standards, it is determined whether the patient's posture meets the requirements, whether the clothing meets the requirements, and whether the electrode placement is correct based on the patient's body position characteristics.

6. The intelligent AED first aid assistance method according to claim 4, wherein: In the step of acquiring a video image before CPR is performed at S210, input X=x1,x2,...x t , where X represents the acquired video image matrix; In the step of preprocessing the video image before the CPR is implemented, X=x1, x2, ... x t Each frame in is converted into a grayscale matrix of (0, 255) and filtered preprocessed; In the step of separating the foreground and the background before the CPR is implemented in S230, X=x1,x2,...x t The foreground and background are extracted using low-rank decomposition theory, X = Az + E; Az represents the foreground part obtained after low-rank decomposition, and the foreground part Az = Az1, Az2, ... Az t ; E represents the background part described by the foreground human body contour minus the acquired video image matrix, and the background part E=E1,E2,...E t , extract the background with the residual component and compare the changes of background residual during defibrillation; In the step S240 of determining whether the current emergency environment meets the emergency standard, the background E at each time t t Match with the overall background E to determine whether there is interference in the background environment; In the step S250 of determining whether the patient's body position characteristics meet the first aid standard, the feature image representation Z=z1,z2,...z of the human body contour is obtained by using texture extraction and human body positioning point technology. t Position key points and extract features; In the step of outputting the foreground and background judgment marks before CPR implementation at S260, the foreground human key point positioning and clothing judgment are output, and the background environment interference judgment mark is output.

7. The intelligent AED first aid assistance method according to claim 6, wherein: In the step of foreground and background separation before CPR implementation in S230, the formula X=Az+E is nonlinearized, and the Frobenius norm is used to constrain the low rank of the matrix, and the real-time matrix decomposition solution process is converted into an extreme value problem under the Frobenius norm for solution: Among them, Π represents a nonlinear function; Then use the deep network to represent and solve the nonlinearity of the above formula, specifically a multi-layer neural network for low-rank decomposition model, as follows: Among them, x t Represents the image grayscale matrix at a certain time t; In the forward propagation process of the multi-layer neural network, the process of establishing the human body contour is g (K+1) (g (K) (... (1) (z i ,Θ (1) ),Θ (K) ),Θ (K+1) ), where g (k) (·) represents the activation function of a single-layer neural network, k=1,2,...K+1 represents the depth of the neural network, Θ (k) Represents the parameter set of a single-layer neural network, as follows: I (k) ={W (k) ,b (k) }, Where W (k) Represent the weights of a single-layer neural network, b (k) Represents the bias parameter of a single-layer neural network. The nonlinear representation of a single-layer neural network in a multi-layer neural network is specifically: g (k) (z,θ (k) ) = g (k) (W (k) z+b); In the forward propagation process of the multi-layer neural network, the background residual at a certain time t can be calculated as: E t =x t - g (K+1) (g (K) (...g (1) (z t ,I (1) ),I (K) ),I (K+1) ); In the step S240 of determining whether the current emergency environment meets the emergency standard, the overall average background is represented by the average of the overall background E, that is, the average background By calculating the background E at each time t t With average background The difference between the two is used to determine whether there is interference at the current moment, as follows: If the difference ε is greater than the threshold set by the system, we judge that there is environmental interference in the background at this time, otherwise there is no environmental interference; In the step S250 of determining whether the patient's body position characteristics meet the first aid standards, a scale-invariant feature transformation computer vision algorithm model is used to detect and describe local features in the image, find extreme points in the spatial scale, and extract their position, scale, and rotation invariants to represent the key point results of the human body and the device worn, so as to obtain the patient's body position characteristics.

8. The intelligent AED first aid assistance method according to claim 7, wherein: In the step of determining whether the current emergency environment meets the emergency standard at S240, the determination threshold is updated according to the use of the intelligent AED device. The process is as follows: The deep low-rank model obtains the foreground human body part through the following inference process: Az=g (K+1) (g (K) (...g (1) (z i ,Θ (1) ),Θ (K) ),Θ (K+1) ); The deep low-rank model obtains the following results for the environmental background through the following inference process: E=Xg (K+1) (g (K) (...g (1) (z t ,I (1) ),I (K) ),I (K+1) ); Get the video frame corresponding to the actual operation started by the implementer Calculate the current background according to the time corresponding to the video frame Calculate the average background According to the current background and average background To update the discrimination threshold:

9. The intelligent AED first aid assistance method according to claim 1, wherein: The steps of training and establishing a CPR mapping model in S100 include the following steps: S110, collecting training data sets, collecting video images of implementers and patients during training, and collecting CPR action standard identification; S120, preprocessing the training video image, graying and filtering the collected video image; S130, obtaining the implementer's human body posture key point detection and positioning features during training, performing the implementer's human body posture key point detection and positioning, and obtaining the implementer's human body posture key point detection and positioning features; S140, establishing a human skin reflection model of the patient during training, using optical rPPG technology to establish a human skin reflection model during training; S150, obtaining the hemodynamic parameter characteristics of the patient during training, and further calculating and obtaining the hemodynamic parameter characteristics of the patient during training using a human skin reflection model; S160, training and establishing a CPR mapping relationship, using the implementer's human body posture key point detection positioning features and the patient's hemodynamic parameter features during training, and training and establishing a CPR mapping relationship according to the CPR action standard identification; S170: Output CPR mapping model parameters. Output CPR mapping model parameters according to the CPR mapping relationship.

10. The intelligent AED first aid assistance method according to claim 9, wherein: The steps to provide guidance and instruction on CPR movements during the implementation of S300CPR include the following: S310, collecting video images when CPR is being implemented, collecting video images of the implementer and the patient during CPR implementation; S320, preprocessing the video images during CPR implementation, graying and filtering the collected video images of the implementer and the patient; S330, obtaining key point detection and positioning features of the implementer's human body posture when CPR is implemented, performing key point detection and positioning of the implementer's human body posture, and obtaining key point detection and positioning features of the implementer's human body posture when CPR is implemented; S340, establishing a human skin reflection model of the patient during CPR implementation, using optical rPPG technology to establish a human skin reflection model during CPR implementation; S350, obtaining the hemodynamic parameter characteristics of the patient during CPR implementation, and further calculating and obtaining the hemodynamic parameter characteristics of the patient during CPR implementation using a human skin reflection model; S360, calculating CPR action standard feedback, using the implementer's human body posture key point detection positioning features and the patient's hemodynamic parameter features during CPR implementation, and using the trained CPR mapping model to calculate CPR action standard feedback; S370, outputting CPR action standard feedback, feeding back the CPR action standard to the implementer who is performing CPR, providing CPR action standard guidance, and guiding the implementation of the CPR process.

11. The intelligent AED first aid assistance method according to claim 10, wherein: The establishment of the human skin reflection model is based on remote photoelectric volumetric pulse wave. By using the multi-wavelength RGB camera in the camera, the subtle color changes caused by pulses on the human skin surface are detected, and non-contact monitoring of human cardiac activity and hemodynamic parameters is achieved to establish the human skin reflection model.

12. The intelligent AED first aid assistance method according to claim 11, wherein: To establish the human skin reflection model, based on the two-color reflection model, the reflection of each skin pixel in the recorded image sequence is defined as a time-varying function in the RGB channel: k (t)=I(t)·(V s (t)+V d (t))+V n (t), where C k (t) represents the pixel value of the kth skin pixel at time t, I(t) represents the light intensity, V s (t) represents specular reflection, V d (t) represents diffuse reflection, V n (t) represents the quantization noise of the camera; Specular Reflection V s Defined as: V s (t) = U s (s0+s(t)), where U s represents the unit color vector of the specular reflection light spectrum captured by the camera, S0 represents the light intensity of the DC part of the specular reflection, and s(t) represents the light intensity of the AC part of the specular reflection; Diffuse V d (t) is defined as: V d (t) = U d ·d0+U p p(t), where U d represents the unit color vector of the skin epidermis, d0 represents the intensity of the inherent diffuse reflection, U p Represents the contribution of different color channels in the pulse signal, and p(t) represents the intensity of the pulse signal; Specular Reflection V s and diffuse reflection V d Both contain a static part that does not change with time and a dynamic part that changes with time. The static parts of the two reflections are combined and expressed as: U c c0=U s ·s0+U d ·d0, where U c The unit color vector representing the intrinsic reflection of the skin; c0 represents the illumination intensity of the intrinsic reflection; The light intensity I(t) is also divided into the intrinsic light intensity I0 and the light intensity that changes with time I0·i(t). I0 and I0·i(t) have the same direction, expressed as: I(t) = I0·(1+i(t)); The patient's human skin reflection model is obtained as follows: C k (t) = I0·(1+i(t))·(U c ·c0+U s ·s(t)+p(t))+V n (t).

13. The intelligent AED first aid assistance method according to claim 12, wherein: The time domain signal is constructed by spatially averaging the pixels in the ROI area of ​​each frame, and the spatial average of enough pixels is obtained to effectively reduce the camera's quantization noise, thereby ignoring the camera's quantization noise V n (t), at the same time, compared with the DC component captured by the camera, the intensity of the AC component is very small, and the product term of the AC component is also ignored. In this way, the obtained human skin reflection model is expanded to obtain: C k (t)≈I0·U c ·c0+I0·U c ·c0·i(t)+I0·U s ·s(t)+I0·U p p(t); Under the unfolded human skin reflection optical model, the observed value C k (t) becomes a linear mixture of a DC component and three AC source signals i(t), s(t) and p(t), which are: I0·U representing the DC component c c0, represents the AC component of light intensity I0·U c c0 i(t), which represents the specular component I0 U s ·s(t), and I0·U representing the pulse signal component p ·p(t).

14. The intelligent AED first aid assistance method according to claim 13, wherein: In the step of acquiring the patient's hemodynamic parameter characteristics, the k skin pixels in the expanded human skin reflection model are first weighted averaged to calculate the one-dimensional pulse signal of the remote photoplethysmogram: The calculation of various patient hemodynamic parameter characteristics includes: Heart rate: The highest peak value of a one-dimensional pulse signal represents the absorption peak of the signal. The time interval between two absorption peaks in a one-dimensional pulse signal is the peak-to-peak interval PPI in the pulse wave signal. The peak-to-peak interval PPI in the pulse wave signal is calculated to calculate the heart rate. HRV: HRV stands for heart rate variability. The peak-to-peak interval PPI in the pulse wave signal is used instead of the RRI in the ECG signal to calculate HRV. SV: SV represents the stroke volume, and the pulse stroke volume is calculated using the pulse circuit analysis method. SV≈∫dC′(t) / dt, where C′(t) represents one complete cycle of the C(t) signal. CO: CO indicates the cardiac output pumped by the left or right ventricle per minute, which is the product of stroke volume and heart rate.

15. The intelligent AED first aid assistance method according to claim 14, wherein: The step of training and establishing the CPR mapping relationship or the step of calculating the CPR action standard feedback is a process of combining the implementer's human body posture key point detection and positioning characteristics and the patient's hemodynamic parameter characteristics, and using the CPR action standard as feedback to form a CPR mapping relationship. The process is as follows: Input features: Y=f(X); The feature set X consists of two sets of features and composition, represents the concatenation of matrix vectors, It represents the key point detection and positioning features of the implementer's human body posture. Indicates the patient's hemodynamic parameter characteristics, and Y indicates the CPR action standard identifier; The CPR mapping relationship is established by a large number of experiments combined with observation and statistical determination, or by calculation using machine learning and deep learning methods, wherein the CPR action standard identifier Y used for training is manually labeled and a training data set is established; Then, the CPR mapping relationship is established through the multivariate regression model: Y=W·X+b; In the above formula, W represents the feature weight of the CPR mapping model, and b represents the bias term of the CPR mapping model; In the step of training and establishing a CPR mapping model in S100, training is performed to obtain model parameters W and b according to the calculated feature set X and the CPR action standard identifier Y included in the training data set; In the step of providing guidance and instruction for CPR action standards during the implementation of S300CPR, the CPR action standard identifier Y is calculated by using the feature set X obtained through real-time calculation and the trained model parameters W and b.

16. The intelligent AED first aid assistance method according to claim 15, wherein: In the step of calculating the CPR action standard feedback in S360, the model parameters W and b are updated according to the use of the intelligent AED device, and the process is as follows: Get the CPR action standard identifier Y; Obtain the actual operation performed by the implementer according to the CPR action specification and convert it into CPR numerical identifier Indicated by CPR value As the target output, the collected feature set X is used as the model input to update the model parameters W and b; Update the original model parameters W and b by: in(·) T Performs the matrix transpose operation.

17. An intelligent AED emergency assistance system, used for the intelligent AED emergency assistance in the method according to any one of claims 1 to 16, wherein: The system includes: intelligent AED device, emergency platform server, emergency medical terminal; The intelligent AED device is a device for external defibrillation with an AI recognition module, in which a CPR mapping model is deployed after combining the implementer's human body posture key point detection and positioning features and the patient's hemodynamic parameter features, training according to the CPR action standard identification and establishing the CPR mapping relationship. The AI ​​recognition module is used to identify whether the emergency conditions meet the standards before CPR is implemented, and provide CPR action standard guidance and guidance during CPR implementation; The first aid platform server communicates with the intelligent AED device via long-distance wireless communication. The first aid platform server is used to receive data uploaded by the intelligent AED device, store and transfer the data required by the emergency doctor to the emergency medical terminal; The emergency medical terminal is used to display the received data and conduct real-time audio and video communication with the intelligent AED device.

18. An intelligent AED first aid assistance device, used for the intelligent AED first aid assistance in the method according to any one of claims 1 to 16, wherein: The device includes a main control module, a wake-up detection module, a video acquisition module, an audio acquisition module, a power module, a communication module, an AI recognition module, a storage module, a voice broadcast module, a video display module, and an early warning module; The main control module is used for information concentration, storage, analysis and decision-making; The wake-up detection module is used to automatically wake up the AED data collection when the intelligent AED device is in use; The audio acquisition module is used to collect real-time audio on site; The video acquisition module is used to acquire real-time video on site; The power module is used to supply power to other modules; The AI ​​recognition module is used to identify the electrode installation position, the interference factors of the emergency environment, perform CPR quality analysis, identify the patient's physiological data and output error correction result information; The storage module is used to store AED operation information, treatment information, detection information, and AI recognition feedback information; The communication module is responsible for communicating with the emergency platform server; The voice broadcast module is used to broadcast the audio of the voice prompt and the video call; The video display module is used to display the video prompt and the video call; The early warning module is responsible for alarming when the medical emergency terminal fails to respond to the emergency event.

19. A readable storage medium, wherein: The readable storage medium stores a control program, and when the control program is executed by the AI ​​recognition module, the AI ​​recognition module executes the steps of the method according to any one of claims 1 to 16.

Images