Atrial fibrillation detection method and system based on face video

Abstract

This application discloses a method and system for atrial fibrillation (AF) detection based on facial videos. The method includes: acquiring multiple sets of facial videos of different human subjects; extracting photoplethysmography (PPG) signals from the facial videos; extracting heart rate variability (HRV) features corresponding to the human subject based on the facial PPG signals; determining an AF judgment result matching the HRV features based on the detected ECG signals corresponding to the human subject, and adding corresponding AF markers to the video frames of the AF judgment result; classifying and training the multiple sets of facial videos with AF markers using a multilayer perceptron to obtain an AF detection model; and using the AF detection model to perform feature analysis on the facial videos corresponding to the subject, and outputting the AF detection result corresponding to the subject. This application's solution, through artificial intelligence analysis of facial videos, provides timely warnings of potential AF in patients, thus providing a reference for clinical treatment.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method and system for atrial fibrillation detection based on facial video. Background Technology

[0002] Atrial fibrillation, or atrial fibrillation, is the most common sustained cardiac arrhythmia. Generally, atrial fibrillation is detected by performing an electrocardiogram (ECG) on the patient, analyzing the rhythm variations in the ECG waveform to determine its presence. However, ECG testing can be time-consuming, making it difficult to provide diagnostic evidence during clinical treatment and thus hindering the timely detection of potential medical crises. Summary of the Invention

[0003] The purpose of this application is to overcome the above-mentioned problems or at least partially solve or mitigate them.

[0004] According to one aspect of this application, a method for atrial fibrillation detection based on facial video is provided, comprising:

[0005] Collect multiple sets of facial videos of different human subjects;

[0006] Based on the aforementioned face video, the face photoplethysmography (PPG) signal is extracted.

[0007] Heart rate variability features corresponding to the human object are extracted based on the face PPG signal;

[0008] Based on the ECG signal detected corresponding to the human object, an atrial fibrillation judgment result matching the heart rate variability characteristics is determined, and the atrial fibrillation judgment result is used to add a corresponding atrial fibrillation marker to the video frame;

[0009] A multilayer perceptron was used to classify and train multiple sets of face videos with the atrial fibrillation marker added to them to obtain an atrial fibrillation detection model.

[0010] The atrial fibrillation detection model is used to perform feature analysis on the face video corresponding to the test subject, and outputs the atrial fibrillation detection result corresponding to the test subject.

[0011] Optionally, the step of extracting the face photoplethysmography (PPG) signal from the face video includes:

[0012] The MNNKIT face detector is used to identify the position of the face of the tested human body in each video frame of the face video, and the positions of multiple feature points corresponding to multiple face feature points in each video frame are obtained.

[0013] The motion coordinates of the human face in each video frame are obtained based on the positions of the multiple feature points.

[0014] The forehead region, left face region, right face region, and nose region of the face are determined based on the locations of the multiple facial features.

[0015] The mean value of the HSV color space of the forehead region, left face region, right face region and nose region is obtained as the HSV color value. Based on the HSV color value, the range of the skin color space in the HSV channel is designed according to the variance principle of three times. A skin mask is generated according to the range.

[0016] The spatial mean values ​​of the forehead region, left face region, right face region and nose region are obtained based on the skin mask to form a time-series signal C;

[0017] The motion trajectory of the human face is generated based on the motion coordinates of the human face in each of the video frames, and is used as the interference signal N for motion noise;

[0018] A matrix S is generated based on the time-series signal C and the interference signal N, where S = [C, N];

[0019] The face PPG signal is obtained from matrix S using the least squares method.

[0020] Optionally, the motion coordinates of the face along the X-axis are the average of the X-coordinates of the plurality of feature points; the motion coordinates of the face along the Y-axis are the average of the Y-coordinates of the plurality of feature points.

[0021] Optionally, the step of obtaining the spatial mean values ​​of the forehead region, left cheek region, right cheek region, and nose region based on the skin mask to form a time-series signal C includes:

[0022] The spatial mean values ​​of the forehead region, left face region, right face region and nose region are obtained based on the skin mask to form a 3*T temporal signal, where T is the length of the video frame;

[0023] The timing signal C is time-domain normalized and band-pass filtered, with a filtering range of [0.5Hz, 4Hz].

[0024] Optionally, the step of extracting the heart rate variability features corresponding to the human object based on the face PPG signal includes:

[0025] Based on the obtained face PPG signal, the R-wave position of the face PPG signal is detected according to the peak value, and the corresponding RR interval is obtained based on the R-wave position;

[0026] The heart rate variability features corresponding to the human body are extracted by analyzing the time-domain, frequency-domain, and waveform characteristics of the RR interval.

[0027] Optionally, the acquisition of multiple sets of facial videos of different human subjects includes:

[0028] In an environment that meets preset requirements, multiple sets of facial videos of different human subjects are captured using video capture equipment;

[0029] The preset requirements include that the signal-to-noise ratio of the acquired face video signal is not less than 0.5 dB; and the distance between the video acquisition device and the human object is not more than 2 m.

[0030] According to another aspect of this application, an atrial fibrillation detection system based on facial video is provided, comprising:

[0031] The acquisition module is configured to acquire multiple sets of facial videos of different human subjects;

[0032] The signal extraction module is configured to extract the face photoplethysmography (PPG) signal based on the face video.

[0033] The feature extraction module is configured to extract the heart rate variability features corresponding to the human object based on the face PPG signal;

[0034] The tagging module is configured to determine an atrial fibrillation judgment result that matches the heart rate variability feature based on the detected ECG signal corresponding to the human object, and add the corresponding atrial fibrillation tag to the video frame based on the atrial fibrillation judgment result;

[0035] The model building module is configured to classify and train multiple sets of face videos with the atrial fibrillation marker added using a multilayer perceptron to obtain an atrial fibrillation detection model.

[0036] The detection module is configured to use the atrial fibrillation detection model to perform feature analysis on the face video of the subject to be tested, and output the atrial fibrillation detection result of the subject to be tested.

[0037] A computing device according to this application includes a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor executes the computer program to implement the atrial fibrillation detection method based on face video as described in any of the preceding claims.

[0038] According to another aspect of this application, a computer-readable storage medium, preferably a non-volatile readable storage medium, is provided, wherein a computer program is stored therein, which, when executed by a processor, implements the atrial fibrillation detection method based on face video as described in any of the preceding claims.

[0039] According to another aspect of this application, a computer program product is provided, comprising computer-readable code that, when executed by a computer device, causes the computer device to perform the atrial fibrillation detection method based on face video as described in any of the preceding claims.

[0040] The atrial fibrillation detection method and system based on facial video of this application obtains heart rate variability characteristics by performing signal analysis on facial video in advance, and adds a matching relationship between the atrial fibrillation judgment results that match the heart rate variability characteristics determined by ECG signal to obtain an atrial fibrillation detection model capable of atrial fibrillation detection. This realizes artificial intelligence analysis of facial video and can provide timely warning of the patient's possible atrial fibrillation before waiting in line for detection and ECG analysis, thus providing a reference for the patient's clinical treatment.

[0041] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments of this application in conjunction with the accompanying drawings. Attached Figure Description

[0042] The following sections will describe some specific embodiments of this application in detail by way of example and not limitation, with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar parts or components. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0043] Figure 1 This is a schematic flowchart of an atrial fibrillation detection method based on face video according to an embodiment of this application;

[0044] Figure 2 This is a schematic diagram of the process for extracting face PPG signals according to an embodiment of this application;

[0045] Figure 3 This is a schematic diagram of the structure of an atrial fibrillation detection system based on face video according to an embodiment of this application;

[0046] Figure 4 This is a schematic diagram of a computing device structure according to an embodiment of this application; Figure 5 This is a schematic diagram of a computer-readable storage medium according to an embodiment of this application. Detailed Implementation

[0047] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments of this application in conjunction with the accompanying drawings.

[0048] Figure 1 This is a schematic flowchart of an atrial fibrillation detection method based on face video according to an embodiment of this application, as shown below. Figure 1 As shown, the atrial fibrillation detection method based on face video in this application embodiment may include at least the following steps S101 to S104.

[0049] S101 collects multiple sets of facial videos of different human subjects.

[0050] In this embodiment, the human subject can be any user subject with or without atrial fibrillation, and the acquired multiple sets of face videos can be face videos containing faces of the same or different subjects with or without atrial fibrillation. Optionally, when acquiring multiple sets of face videos of different human subjects in step S101 above, the video acquisition device can be used to acquire multiple sets of face videos of different human subjects in an environment that meets preset requirements; wherein, the preset requirements include that the signal-to-noise ratio of the acquired face video signal is not less than 0.5 dB; and the distance between the video acquisition device and the human subject does not exceed 2 m.

[0051] In other words, when capturing facial videos, the distance between the human subject and the video capture device should not exceed 2 meters. The video frame rate should be no less than 20 frames per second, and each set of facial videos should be at least 30 seconds long. Facial video capture must be performed in an environment that meets preset requirements. In this embodiment, the preset requirement is a minimum lighting requirement; that is, it must be performed based on the minimum lighting conditions. In this embodiment, the minimum lighting condition refers to the signal-to-noise ratio (SNR) of the acquired facial video signal being no less than 0.5 dB, thereby ensuring that the acquired facial video can accurately convey the facial state. The formula for calculating the facial video signal-to-noise ratio (SNR) is as follows:

[0052]

[0053]

[0054] in, The power spectrum was calculated using the Welch method to obtain the PPG signal of the face during the initial 8 seconds. Here, f is the frequency per minute, and U(f) is a binary window that is 1 when f is between 30 and 240, corresponding to the heart rate frequency. The method for acquiring the PPG signal is described below.

[0055] S102, extract the face photoplethysmography (PPG) signal based on the face video.

[0056] PPG (Photoplethysmography) is an infrared non-destructive testing technique used to detect human heart rate during exercise. Since motion noise has a consistent modulation effect on color channels, this embodiment uses facial motion signals to observe the signal and recover the face PPG signal from the face video. In this embodiment, as... Figure 2 As shown, extracting face PPG signals from face videos can include: S102-1 to S102-8.

[0057] S102-1, using the MNNKIT face detector to identify the position of the face of the tested human body in each video frame of the face video, and obtaining the positions of multiple feature points corresponding to multiple face feature points in each video frame.

[0058] Optionally, the positions Pi(t) of 108 facial feature points in the face video can be obtained.

[0059] S102-2, Obtain the motion coordinates of the human face in each video frame based on the positions of the multiple feature points. The motion coordinates of the face along the X-axis are the average of the X-coordinates of the multiple feature points; the motion coordinates of the face along the Y-axis are the average of the Y-coordinates of the multiple feature points.

[0060] Specifically, for each frame, the motion coordinates of the face in the current frame can be obtained based on 108 facial feature points. The motion coordinates of the face on the X-axis are the average of the X-axis coordinates of the 108 feature points in the current frame, and the motion coordinates of the face on the Y-axis are the average of the Y-axis coordinates of the 108 feature points in the current frame. Thus, the motion coordinates of the face in each frame of the face video frame can be obtained.

[0061] S102-3, determine the forehead region, left face region, right face region and nose region of the face based on the multiple facial feature locations.

[0062] That is, based on the location of 108 facial feature points, four regions are determined in each video frame of the face video: the forehead region, the left face region, the right face region, and the nose region.

[0063] S102-4, obtain the mean values ​​of the HSV color spaces for the forehead region, left cheek region, right cheek region, and nose region, respectively, as HSV color values; design the range of the skin color space in the HSV channel according to the variance principle of three times based on the HSV color values, and generate a skin mask based on the range. The formulas for calculating the spatial mean values ​​of different regions and different color spaces and the skin mask formula are as follows:

[0064]

[0065]

[0066]

[0067] in That is, different areas of the face, such as the cheeks and forehead. The area of ​​the region, Let x and y be the horizontal and vertical coordinates of the face, and y be its position in the video. The face region in the i-th region of the HSV color channel at frame t. The pixel values ​​of this area.

[0068] S102-5, a timing signal C is formed based on the spatial mean of the RGB color channels of the skin in the forehead region, left cheek region, right cheek region, and nose region using the skin mask. The formula for calculating the spatial mean is as follows:

[0069]

[0070] in For the face region of the i-th region in the t-th frame of the RGB color channel, the values ​​in frame T are... The timing signal C is formed.

[0071] Optionally, the spatial mean values ​​of the forehead region, left face region, right face region and nose region are obtained according to the skin mask to form a 3*T time-series signal, where T is the length of the video frame; the time-series signal C is time-domain normalized and band-pass filtered, with a filtering range of [0.5Hz, 4Hz].

[0072] S102-6, Generate the motion trajectory of the face based on the motion coordinates of the human face in each of the video frames, and use it as the interference signal N for motion noise.

[0073] The motion coordinates corresponding to each frame of the face in the face video frame have been obtained in step S102-2 above. The motion coordinates can be spliced ​​according to the time sequence in the face video to obtain the motion trajectory of the face, which can be used as an interference signal for motion noise.

[0074] S102-7, Generate matrix S based on the timing signal C and the interference signal N, S=[C,N]. Matrix S is the connection matrix between matrix C and matrix N.

[0075] Performing maximum-minimum normalization on the time-series signal in C and the motion noise in N yields the following results: .

[0076] S102-8, the face PPG signal is solved from matrix S using the least squares method.

[0077] In this embodiment, the following formula can be solved.

[0078]

[0079] in PPG signals for facial recognition are in demand. The signal is obtained approximated, and W is the projection matrix.

[0080] Solving for the given information yields:

[0081]

[0082] in These are the weighted values ​​for different channels. The variance of the face PPG signal.

[0083] Finally, the weighted vector of five channels—the RGB three-color channels and the motion noise of the horizontal and vertical axes—is calculated. The absorption capacity of the blood is determined by the RGB three channels, while the corresponding value of the motion noise channel is 0.

[0084] S103, extract the heart rate variability features corresponding to the human object based on the face PPG signal.

[0085] After obtaining the face PPG signal, the heart rate variability features corresponding to each human subject can be extracted. Optionally, this may specifically include:

[0086] S103-1, Based on the obtained face PPG signal, the R-wave position of the face PPG signal is detected according to the peak value, and the corresponding RR interval is obtained based on the R-wave position.

[0087] S103-2, Analysis of the time domain, frequency domain and waveform characteristics of the RR interval, thereby extracting the heart rate variability characteristics corresponding to the human object.

[0088] Temporal features include the standard deviation of the RR interval (SDNN), mean, median, and RMSSD and PNN50.

[0089]

[0090] Where N represents the number of RR intervals, .

[0091]

[0092] PNN50 represents the percentage of adjacent RR intervals that differ by more than 50 ms.

[0093] Frequency domain characteristics: Based on the PPG signal, Fourier transform or fast Fourier transform is used to convert the time-series signal into a frequency domain signal.

[0094] Frequency domain characteristics include low-frequency LF, high-frequency HF, and LF / HF, among which,

[0095] LF low frequency: 0.04Hz <f<0.15Hz

[0096] HF (High Frequency): 0.15Hz <f<0.4Hz

[0097] Waveform characteristics: height and width of a single cardiac cycle, and height of the replay wave.

[0098] Nonlinear characteristics: Calculate the sample entropy and approximate entropy of the RR interval.

[0099] The time-domain features, frequency-domain features, and waveform features obtained above can be used as heart rate variability features corresponding to human subjects.

[0100] S104, based on the ECG signal detected corresponding to the human object, determine the atrial fibrillation judgment result that matches the heart rate variability feature, and add the corresponding atrial fibrillation mark to the video frame based on the atrial fibrillation judgment result.

[0101] Simultaneously, the ECG signals detected for each individual subject can be used to determine the atrial fibrillation (AF) assessment result, matching the heart rate variability characteristics of the same subject; that is, the assessment result of whether AF has occurred. The AF assessment result can be obtained from the judgment of ECG signals by a doctor or other professional. Based on this, the AF assessment result can be used as an AF marker for the corresponding human video, which can include a marker indicating whether AF has occurred or not.

[0102] S105, an atrial fibrillation detection model is obtained by classifying and training multiple sets of face videos with the added atrial fibrillation marker using a multilayer perceptron.

[0103] At this point, for any given human video, the corresponding heart rate variability features and atrial fibrillation markers have been learned. Using the aforementioned data, a multilayer perceptron can be used to classify and train multiple sets of face videos with the added atrial fibrillation markers, thus obtaining an atrial fibrillation detection model.

[0104] A multilayer perceptron (MLP), also known as an artificial neural network (ANN), has multiple hidden layers in addition to the input and output layers. The simplest MLP contains only one hidden layer, i.e., a three-layer structure. In this embodiment, a multilayer perceptron is trained using human videos labeled with atrial fibrillation to obtain an artificial intelligence model that can learn features from the human videos to output corresponding atrial fibrillation labels.

[0105] S106, the atrial fibrillation detection model is used to perform feature analysis on the face video of the subject to be tested, and the atrial fibrillation detection result of the subject to be tested is output.

[0106] The trained atrial fibrillation detection model can be deployed to a server for subsequent use. In practical applications, assuming there is any subject to be tested and a corresponding facial video has been captured, when a user transmits the facial video containing the face PPG signal via a mobile device such as a smartphone, the atrial fibrillation detection model can be used directly for feature learning, and the detection result of whether atrial fibrillation has occurred can be returned to the mobile device for display by the user or doctor.

[0107] Furthermore, feedback from doctors on the returned test results can be obtained. This feedback allows for further learning and adjustment of the atrial fibrillation detection model, continuously optimizing the model and making the test results more accurate.

[0108] The atrial fibrillation detection method and system based on facial video provided in this application obtains heart rate variability characteristics by performing signal analysis on facial video in advance, and adds a matching relationship to the atrial fibrillation judgment results that match the heart rate variability characteristics based on ECG signals, so as to obtain an atrial fibrillation detection model capable of atrial fibrillation detection. This realizes artificial intelligence analysis of facial video, provides timely warnings of patients who may have atrial fibrillation, and thus provides a reference for patients in clinical treatment.

[0109] Based on the same inventive concept, embodiments of this application also provide an atrial fibrillation detection system based on facial video, such as... Figure 3 As shown, the atrial fibrillation detection system based on face video in this embodiment may include:

[0110] Acquisition module 310 is configured to acquire multiple sets of facial videos of different human subjects;

[0111] Signal extraction module 320 is configured to extract face photoplethysmography (PPG) signal based on the face video.

[0112] The feature extraction module 330 is configured to extract the heart rate variability features corresponding to the human object based on the face PPG signal;

[0113] The tagging module 340 is configured to determine an atrial fibrillation judgment result that matches the heart rate variability feature based on the detected ECG signal corresponding to the human object, and add a corresponding atrial fibrillation tag to the video frame based on the atrial fibrillation judgment result;

[0114] The model building module 350 is configured to classify and train multiple sets of face videos with the atrial fibrillation marker added through a multilayer perceptron to obtain an atrial fibrillation detection model.

[0115] The detection module 360 ​​is configured to use the atrial fibrillation detection model to perform feature analysis based on the face video of the subject to be tested, and output the atrial fibrillation detection result of the subject to be tested.

[0116] In an optional embodiment of this application, the signal extraction module 320 may also be configured as follows:

[0117] The MNNKIT face detector is used to identify the position of the subject's face in each video frame of the face video, and the positions of multiple feature points corresponding to multiple facial feature points in each video frame are obtained; the motion coordinates of the subject's face in each video frame are obtained based on the multiple feature point positions; the forehead region, left face region, right face region, and nose region of the face are determined based on the multiple facial feature positions; the average value of the HSV color space of the forehead region, left face region, right face region, and nose region is obtained as the HSV color value, and then... Based on the HSV color values, the range of the skin color space in the HSV channel is designed according to the three-fold variance principle, and a skin mask is generated based on the range. The spatial mean values ​​of the forehead region, left face region, right face region, and nose region are obtained based on the skin mask to form a temporal signal C. The motion trajectory of the human face is generated based on the motion coordinates of the human face in each of the video frames, which serves as the interference signal N for motion noise. A matrix S is generated based on the temporal signal C and the interference signal N, where S = [C, N]. The face PPG signal is solved from the matrix S using the least squares method.

[0118] In an optional embodiment of this application, the feature extraction module 330 may also be configured as follows:

[0119] Based on the obtained face PPG signal, the R-wave position of the face PPG signal is detected according to the peak value, and the corresponding RR interval is obtained based on the R-wave position;

[0120] The heart rate variability features corresponding to the human body are extracted by analyzing the time-domain, frequency-domain, and waveform characteristics of the RR interval.

[0121] This application also provides a computing device, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor executes the computer program to implement the atrial fibrillation detection method based on face video as described in the above embodiments.

[0122] This application also provides a computer-readable storage medium, preferably a non-volatile readable storage medium, which stores a computer program that, when executed by a processor, implements the atrial fibrillation detection method based on face video as described in the above embodiments.

[0123] This application also provides a computer program product, including computer-readable code, which, when executed by a computer device, causes the computer device to perform the atrial fibrillation detection method based on face video described in the above embodiments.

[0124] This application also provides a computing device, referring to... Figure 4 The computing device includes a memory 1120, a processor 1110, and a computer program stored in the memory 1120 and executable by the processor 1110. The computer program is stored in a space 1130 for program code in the memory 1120. When executed by the processor 1110, the computer program implements a method step 1131 according to this application for performing any of the method steps 1131.

[0125] This application also provides a computer-readable storage medium. (See attached embodiments.) Figure 5 The computer-readable storage medium includes a storage unit for program code, the storage unit being provided with a program 1131' for performing the method steps according to the present application, the program being executed by a processor.

[0126] This application also provides a computer program product containing instructions. When the computer program product is run on a computer, it causes the computer to perform the method steps according to this application.

[0127] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When a computer loads and executes the computer program instructions, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0128] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0129] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing a processor. The program can be stored in a computer-readable storage medium, which is a non-transitory medium, such as random access memory, read-only memory, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof.

[0130] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for atrial fibrillation detection based on facial video, comprising: Collect multiple sets of facial videos of different human subjects; Based on the aforementioned face video, the face photoplethysmography (PPG) signal is extracted. Heart rate variability features corresponding to the human object are extracted based on the face PPG signal; Based on the ECG signal detected corresponding to the human object, an atrial fibrillation judgment result matching the heart rate variability characteristics is determined, and the atrial fibrillation judgment result is used to add a corresponding atrial fibrillation marker to the video frame; A multilayer perceptron was used to classify and train multiple sets of face videos with the atrial fibrillation marker added to them to obtain an atrial fibrillation detection model. The atrial fibrillation detection model is used to perform feature analysis on the face video corresponding to the test subject, and outputs the atrial fibrillation detection result corresponding to the test subject.

2. The method according to claim 1, characterized in that, Extracting face photoplethysmography (PPG) signals from face videos includes: The MNNKIT face detector is used to identify the position of the human face in each video frame of the face video, and the positions of multiple feature points corresponding to multiple face feature points in each video frame are obtained. The motion coordinates of the human face in each video frame are obtained based on the positions of the multiple feature points. The forehead region, left face region, right face region, and nose region of the face are determined based on the locations of the multiple facial features. The mean value of the HSV color space of the forehead region, left face region, right face region and nose region is obtained as the HSV color value. Based on the HSV color value, the range of the skin color space in the HSV channel is designed according to the variance principle of three times. A skin mask is generated according to the range. A temporal signal C is formed based on the spatial mean of the RGB color channels of the skin in the forehead region, left face region, right face region and nose region according to the skin mask; The motion trajectory of the human face is generated based on the motion coordinates of the human face in each of the video frames, and is used as the interference signal N for motion noise; A matrix S is generated based on the time-series signal C and the interference signal N, where S = [C, N]; The face PPG signal is obtained from matrix S using the least squares method.

3. The method according to claim 2, characterized in that, The motion coordinates of the face along the X-axis are the average of the X-coordinates of the multiple feature points; the motion coordinates of the face along the Y-axis are the average of the Y-coordinates of the multiple feature points.

4. The method according to claim 2, characterized in that, The step of forming a time-series signal C based on the spatial mean of the RGB color channels of the skin in the forehead region, left cheek region, right cheek region, and nose region according to the skin mask includes: A 3*T temporal signal is formed based on the spatial mean of the RGB color channels of the skin in the forehead region, left face region, right face region and nose region according to the skin mask, where T is the length of the video frame; The timing signal C is time-domain normalized and band-pass filtered, with a filtering range of [0.5Hz, 4Hz].

5. The method according to claim 1, characterized in that, The step of extracting the heart rate variability features corresponding to the human object based on the face PPG signal includes: Based on the obtained face PPG signal, the R-wave position of the face PPG signal is detected according to the peak value, and the corresponding RR interval is obtained based on the R-wave position; The heart rate variability features corresponding to the human body are extracted by analyzing the time-domain, frequency-domain, and waveform characteristics of the RR interval.

6. The method according to any one of claims 1-5, characterized in that, The collection of multiple sets of facial videos of different human subjects includes: In an environment that meets preset requirements, multiple sets of facial videos of different human subjects are captured using video capture equipment; The preset requirements include that the signal-to-noise ratio of the acquired face video signal is not less than 0.5 dB; and the distance between the video acquisition device and the human object is not more than 2 m.

7. An atrial fibrillation detection system based on facial video, comprising: The acquisition module is configured to acquire multiple sets of facial videos of different human subjects; The signal extraction module is configured to extract the face photoplethysmography (PPG) signal based on the face video. The feature extraction module is configured to extract the heart rate variability features corresponding to the human object based on the face PPG signal; The tagging module is configured to determine an atrial fibrillation judgment result that matches the heart rate variability feature based on the detected ECG signal corresponding to the human object, and add the corresponding atrial fibrillation tag to the video frame based on the atrial fibrillation judgment result; The model building module is configured to classify and train multiple sets of face videos with the atrial fibrillation marker added using a multilayer perceptron to obtain an atrial fibrillation detection model. The detection module is configured to use the atrial fibrillation detection model to perform feature analysis on the face video of the subject to be tested, and output the atrial fibrillation detection result of the subject to be tested.

8. A computing device comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein, When the processor executes the computer program, it implements the atrial fibrillation detection method based on face video as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program therein, said computer program, when executed by a processor, implementing the atrial fibrillation detection method based on face video as described in any one of claims 1-6.

10. A computer program product comprising computer-readable code, which, when executed by a computer device, causes the computer device to perform the atrial fibrillation detection method based on face video as described in any one of claims 1-6.

Images