Method, device, computer equipment and storage medium for remote monitoring of physiological signals

Abstract

The present invention relates to the field of signal processing technology, and discloses a method, device, computer equipment and storage medium for monitoring physiological signals at a distance, the method comprising: acquiring a first video image of multiple consecutive frames, and converting the first video image of the multiple consecutive frames into a two-dimensional matrix signal; the first video image is a video image whose acquisition distance exceeds a first preset distance; performing low-rank sparse matrix decomposition on the two-dimensional matrix signal to determine a sparse matrix obtained by the decomposition; extracting a region of interest corresponding to the skin of the object under test from the sparse matrix, determining a target PPG signal according to changes in the region of interest, and determining the physiological signal of the object under test according to the target PPG signal. The present invention can effectively obtain the changing signal in the video from the sparse matrix by means of low-rank sparse matrix decomposition, and can also significantly improve the signal-to-noise ratio in the scenario of long-distance measurement.

Description

Technical Field

[0001] The present invention relates to the field of signal processing technology, and in particular to a method, device, computer equipment and storage medium for remote monitoring of physiological signals. Background Art

[0002] Traditional heart rate and blood pressure measurement methods usually rely on direct contact or close-range monitoring, which limits their application in remote monitoring scenarios. However, in the field of health care, in the application scenario of all-weather health monitoring of the elderly, real-time monitoring is required through monitoring images. With the development of video surveillance technology, it has become possible to monitor physiological signals using video images. However, long-distance monitoring faces problems such as signal attenuation and noise interference, resulting in reduced measurement accuracy. Summary of the invention

[0003] In view of this, the present invention provides a method, an apparatus, a computer device and a storage medium for remotely monitoring physiological signals to solve the problem of low accuracy in remotely monitoring physiological signals.

[0004] In a first aspect, the present invention provides a method for remotely monitoring physiological signals, comprising:

[0005] Acquire a first video image of a plurality of consecutive frames, and convert the first video image of the plurality of consecutive frames into a two-dimensional matrix signal; the first video image is a video image whose acquisition distance exceeds a first preset distance;

[0006] Performing low-rank sparse matrix decomposition on the two-dimensional matrix signal to determine a sparse matrix obtained by the decomposition;

[0007] Extracting a region of interest corresponding to the skin of the subject under test from the sparse matrix, and determining a target PPG signal according to a change in the region of interest;

[0008] The physiological signal of the subject is determined according to the target PPG signal.

[0009] In some optional implementations, converting the first video images of the continuous multiple frames into a two-dimensional matrix signal includes:

[0010] Flatten each frame of the first video image into a one-dimensional vector;

[0011] The one-dimensional vectors corresponding to the multiple frames of the first video image are combined into a two-dimensional matrix signal X 2D , and X 2D ∈R (m·n)×T ; wherein m and n are the height and width of the first video image respectively, and T is the number of frames of the first video image in the plurality of consecutive frames of the first video image.

[0012] In some optional implementations, performing low-rank sparse matrix decomposition on the two-dimensional matrix signal to determine a sparse matrix obtained by the decomposition includes:

[0013] Performing low-rank sparse matrix decomposition on the two-dimensional matrix signal according to the objective function, decomposing the two-dimensional matrix signal into a low-rank matrix and a sparse matrix;

[0014] The objective function is used to minimize the rank measure of the low-rank matrix, minimize the sparsity measure of the sparse matrix, and minimize the difference between the sparse matrix and a reference signal constructed based on the periodic characteristics of the PPG signal.

[0015] In some optional implementations, the process of constructing the reference signal includes:

[0016] Acquire a second video image within a plurality of PPG signal cycles; the second video image is a video image acquired at a distance less than a second preset distance; the second preset distance does not exceed the first preset distance;

[0017] Performing sparse coding on the second video image to obtain a sparse dictionary including multiple groups of basis vectors;

[0018] For each of the first video images, perform sparse representation using the sparse dictionary to determine a corresponding sparse coefficient;

[0019] A periodic reference signal is determined according to the sparse dictionary and the sparse coefficients of each of the first video images.

[0020] In some optional implementations, the objective function is:

[0021]

[0022] Wherein, L represents the low-rank matrix, S represents the sparse matrix, S p is the reference signal; |L| * is the nuclear norm of the low-rank matrix L, |S| 1 is the sparse measure of the sparse matrix S, is the sparse matrix S and the reference signal S p The difference between them, λ and μ are adjustment parameters.

[0023] In some optional implementations, extracting a region of interest corresponding to the skin of the subject from the sparse matrix, and determining a target PPG signal according to a change in the region of interest includes:

[0024] Performing three-dimensional feature extraction on the sparse matrix to extract spatiotemporal features corresponding to the sparse matrix;

[0025] Determine the time dimension attention weight and the space dimension attention weight corresponding to the sparse matrix, and fuse the spatiotemporal features, the time dimension attention weight and the space dimension attention weight to generate a target PPG signal.

[0026] In some optional implementations, determining the time dimension attention weight and the space dimension attention weight corresponding to the sparse matrix, and fusing the spatiotemporal features, the time dimension attention weight, and the space dimension attention weight to generate a target PPG signal includes:

[0027] According to the sparse matrix, the attention weight corresponding to each frame of the first video image in the time dimension is determined to generate a time dimension attention weight Att containing the attention weights of the first video images of each frame Temporal ;

[0028] According to the sparse matrix, the attention weight corresponding to each pixel point in the spatial dimension is determined to generate a spatial dimension attention weight Att containing the attention weights of each pixel point Spatial ;

[0029] The time dimension attention weight is expanded into the space dimension to obtain the first attention weight matrix Att′ Temporal ; Expand the spatial dimension attention weight by time dimension to obtain the second attention weight matrix Att′ Spatial ;

[0030] Fusion of the spatiotemporal features F base , the first attention weight matrix Att′ Temporal And the second attention weight matrix Att′ Spatial , generate the target PPG signal, and:

[0031] Signal PPG =f(F base ⊙Att′ Temporal ⊙Att′ Spatial );

[0032] Among them, F base ∈R M×N×T×C ;Att Tempora1 ∈R 1×1×T×1 , Att′ Temporal ∈R M×N×T×C , and the first attention weight matrix Att′ Temporal The value a in x,y,t,c For Att Temporal (1,1,t,1);Att Spatial ∈R M×N×1×1 , Att 'Spatial ∈R M ×N×T×C , and the second attention weight matrix Att′ Spatial The value b in x,y,t,c For Att spatial (x,y,1,1); f() is the feature extraction function, ⊙ represents the Hadamard product;

[0033] T is the number of frames of the first video image in the first video image of the continuous multiple frames, M and N are respectively the height and width of the spatiotemporal feature, C is the number of feature channels of the spatiotemporal feature; x, y, t, c are respectively the indexes of the height, width, video frame, and feature channel in the matrix.

[0034] In a second aspect, the present invention provides a device for remotely monitoring physiological signals, comprising:

[0035] An acquisition module, used for acquiring a plurality of consecutive frames of first video images, and converting the plurality of consecutive frames of first video images into two-dimensional matrix signals; the first video images are video images whose acquisition distance exceeds a first preset distance;

[0036] A decomposition module, used for performing low-rank sparse matrix decomposition on the two-dimensional matrix signal to determine a sparse matrix obtained by the decomposition;

[0037] An extraction module, used to extract a region of interest corresponding to the skin of the subject under test from the sparse matrix, and determine a target PPG signal according to a change in the region of interest;

[0038] A processing module is used to determine the physiological signal of the subject according to the target PPG signal.

[0039] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method for remotely monitoring physiological signals of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0040] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method for remotely monitoring physiological signals of the first aspect or any corresponding embodiment thereof.

[0041] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the method for remotely monitoring physiological signals of the first aspect or any corresponding embodiment thereof.

[0042] The present invention utilizes continuous multi-frame first video images collected at a long distance, converts them into two-dimensional matrix signals, and performs low-rank sparse matrix decomposition on the two-dimensional matrix signals, so as to extract a sparse matrix that can represent signal changes, and then extract the PPG signal of the region of interest, and realize real-time calculation of physiological signals. By means of low-rank sparse matrix decomposition, the changing signals in the video can be effectively obtained from the sparse matrix, and the physiological signals related to heart rate and blood pressure can be taken out. In the scenario of long-distance measurement, the signal-to-noise ratio can be significantly improved, signal enhancement can be achieved, and the accuracy and reliability of the measurement can be significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 is a schematic flow chart of a method for remotely monitoring physiological signals according to an embodiment of the present invention;

[0045] Figure 2 is a flow chart of another method for remotely monitoring physiological signals according to an embodiment of the present invention;

[0046] Figure 3 is a schematic diagram of a structure of a prediction model according to an embodiment of the present invention;

[0047] Figure 4 is a structural block diagram of a device for remotely monitoring physiological signals according to an embodiment of the present invention;

[0048] Figure 5 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0050] The principle of PPG (Photoplethysmography) signal is that the change in blood volume in blood vessels causes the change in light reflected from the skin surface. In long-distance non-cooperative monitoring scenarios, the PPG signal does not change significantly, the signal-to-noise ratio is low, it is easily affected by various external lights, the measurement accuracy is low, and it is difficult to accurately monitor the elderly's heart rate, blood pressure and other physiological signals.

[0051] An embodiment of the present invention provides a method for remote monitoring of physiological signals. For video images collected at a long distance, a low-rank sparse matrix decomposition is performed on them to extract a sparse matrix that can represent signal changes. The physiological signals are calculated based on the sparse matrix, which can improve the signal-to-noise ratio and achieve signal enhancement, thereby improving the accuracy and reliability of remote physiological monitoring.

[0052] According to an embodiment of the present invention, an embodiment of a method for remotely monitoring physiological signals is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0053] In this embodiment, a method for remotely monitoring physiological signals is provided, which can be applied to devices with processing capabilities, such as mobile terminals, personal computers, servers, etc.; for example, the device can be a monitoring device for monitoring physiological signals of the elderly and other human bodies. Figure 1 is a flow chart of a method for remotely monitoring physiological signals according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps.

[0054] Step S101, acquiring a plurality of consecutive frames of first video images, and converting the plurality of consecutive frames of first video images into a two-dimensional matrix signal; the first video image is a video image whose acquisition distance exceeds a first preset distance.

[0055] In this embodiment, when monitoring the physiological signals of the elderly and other subjects at a long distance, the raw data that can be collected is generally the video collected at a long distance, that is, the collection distance when collecting the video generally exceeds a certain preset distance, that is, the first preset distance; for example, the first preset distance is 1 meter, or it can be a centimeter-level threshold, such as 20cm. And, for the convenience of subsequent description, the video image obtained from the long-distance video is called the first video image.

[0056] For example, the long-distance video may be a surveillance video collected by a video surveillance device, and multiple frames of video images are extracted from the surveillance video, and these video images can be used as the first video image. For example, the video surveillance device may be a camera used to monitor the health status of the elderly in a nursing home or other health care institution.

[0057] Among them, the first video image of each frame itself is a two-dimensional matrix data, and the first video images of multiple consecutive frames are a three-dimensional matrix as a whole; to facilitate the subsequent low-rank sparse matrix decomposition, the three-dimensional matrix is ​​converted into an equivalent two-dimensional matrix signal.

[0058] Step S102, performing low-rank sparse matrix decomposition on the two-dimensional matrix signal to determine a sparse matrix obtained by the decomposition.

[0059] In this embodiment, a low-rank sparse matrix decomposition (Low-Rank Sparse Matrix Decomposition, LRSMD) is performed on a two-dimensional matrix signal formed by multiple frames of first video images, so that the two-dimensional matrix signal can be decomposed into a low-rank matrix and a sparse matrix.

[0060] Among them, a low-rank matrix refers to a matrix with a lower rank, and a sparse matrix refers to a matrix containing a large number of zero elements. Performing low-rank sparse matrix decomposition on a two-dimensional matrix signal is essentially to divide the two-dimensional matrix signal into two matrices. By minimizing the rank of one of the matrices and minimizing the number of non-zero elements in the other matrix, the corresponding low-rank matrix and sparse matrix can be decomposed.

[0061] Step S103, extracting a region of interest corresponding to the skin of the subject under test from the sparse matrix, and determining a target PPG signal according to changes in the region of interest.

[0062] Since the sparse matrix can represent the changes in the first video image, and the PPG signal is also generated based on the changes in the reflected light, the sparse matrix mainly contains information related to the PPG signal. In this embodiment, the physiological signal of the object under test is calculated based on the sparse matrix obtained by decomposition.

[0063] The measured object refers to the object whose physiological signals need to be monitored, which may be, for example, an elderly person in a nursing home, a patient in a rehabilitation hospital, etc. It can be understood that in order to monitor the measured object, the measured object also needs to exist in the first video image, that is, the first video image is also an image of the space where the measured object is located.

[0064] The PPG signal is a signal reflected from the skin surface, so in this embodiment, the area corresponding to the skin of the object being measured is used as the region of interest for extracting the PPG signal. Generally, the face area of ​​the object being measured can be used as the region of interest.

[0065] In this embodiment, after determining the change of the region of interest (e.g., the face region) within a period of multiple consecutive frames, the corresponding PPG signal can be extracted based on the signal change of the region of interest. The PPG signal is the PPG signal of the object under test, and for the convenience of distinction and description, it is referred to as the target PPG signal. For example, the signal of the region of interest can be further processed by wavelet transform to extract the PPG signal.

[0066] Step S104, determining the physiological signal of the subject according to the target PPG signal.

[0067] The target PPG signal of the measured object is a periodic signal. Based on the target PPG signal, the heartbeat, blood pressure and other physiological signals of the measured object can be calculated, thereby realizing the monitoring video collected at a long distance and monitoring the physiological signals of the measured object in real time.

[0068] The method for remote monitoring of physiological signals provided in this embodiment utilizes the continuous multi-frame first video images collected at a long distance, converts them into two-dimensional matrix signals, and performs low-rank sparse matrix decomposition on the two-dimensional matrix signals, so as to extract a sparse matrix that can represent signal changes, and then extract the PPG signal of the region of interest, and realize real-time calculation of physiological signals. By means of low-rank sparse matrix decomposition, the changing signals in the video can be effectively obtained from the sparse matrix, and the physiological signals related to heart rate and blood pressure can be taken out. In the scenario of remote measurement, the signal-to-noise ratio can be significantly improved, signal enhancement can be achieved, and the accuracy and reliability of the measurement can be significantly improved.

[0069] In this embodiment, a method for remotely monitoring physiological signals is provided, which can be applied to devices with processing capabilities, such as mobile terminals, personal computers, servers, etc.; for example, the device can be a monitoring device for monitoring physiological signals of the elderly and other human bodies. Figure 2 is a flow chart of a method for remotely monitoring physiological signals according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps.

[0070] Step S201, acquiring a plurality of consecutive frames of first video images, and converting the plurality of consecutive frames of first video images into a two-dimensional matrix signal; the first video image is a video image whose acquisition distance exceeds a first preset distance.

[0071] For details, please see Figure 1Step S101 of the illustrated embodiment will not be described in detail here.

[0072] In some optional implementations, the above step S201 "converting a plurality of consecutive frames of first video images into a two-dimensional matrix signal" specifically includes steps A1 to A2.

[0073] Step A1: flatten each frame of the first video image into a one-dimensional vector.

[0074] Step A2: compose a two-dimensional matrix signal X from one-dimensional vectors corresponding to multiple frames of the first video image 2D , and X 2D ∈R (m ·2)×T ; Wherein, m and n are the height and width of the first video image respectively, and T is the number of frames of the first video image in the continuous multiple frames of the first video image.

[0075] In this embodiment, T consecutive frames of images are obtained from the video as the first video image, that is, T is the number of frames of the first video image; and each frame of the first video image has the same size, that is, has a certain height and width. In this embodiment, m and n represent the height and width of the first video image respectively. Therefore, if X represents the continuous first video image of multiple frames extracted from the video, then X is a three-dimensional matrix, and X∈R m×n×T .

[0076] For each frame of the first video image in the three-dimensional matrix X, the m×n first video image (two-dimensional matrix) is flattened into a one-dimensional vector, which includes m×n elements. All the one-dimensional vectors are combined into a two-dimensional matrix to obtain the two-dimensional matrix signal X 2D , and X 2D ∈R (m·n)×T .

[0077] In addition, if the two-dimensional matrix signal X 2D By performing low-rank sparse matrix decomposition, we can obtain a low-rank matrix L and a sparse matrix S, then the sparse matrix S also satisfies: S∈R (m·n)×T ; When the region of interest corresponding to the skin of the subject is extracted from the sparse matrix S later, the sparse matrix S can be restored to a plurality of consecutive frames of sparse images (two-dimensional matrix), and the region of interest corresponding to the skin of the subject is extracted from each frame of the sparse image, and then the PPG signal is extracted.

[0078] In order to avoid a large change in the position of the region of interest in different frames due to the movement of the measured object, generally, when the measured object is relatively still, a plurality of continuous first video images are collected to ensure the accuracy of subsequent measurements.

[0079] Step S202, performing low-rank sparse matrix decomposition on the two-dimensional matrix signal to determine a sparse matrix obtained by the decomposition.

[0080] Specifically, the above step S202 of "performing low-rank sparse matrix decomposition on the two-dimensional matrix signal to determine a sparse matrix obtained by the decomposition" includes step S2021.

[0081] Step S2021, performing low-rank sparse matrix decomposition on the two-dimensional matrix signal according to the objective function, decomposing the two-dimensional matrix signal into a low-rank matrix and a sparse matrix. The objective function is used to minimize the rank metric of the low-rank matrix, minimize the sparsity metric of the sparse matrix, and minimize the difference between the sparse matrix and a reference signal constructed based on the periodic characteristics of the PPG signal.

[0082] In this embodiment, in order to decompose the two-dimensional matrix signal X by low-rank sparse matrix 2D Decomposed into a low-rank matrix L and a sparse matrix S, an objective function for low-rank sparse matrix decomposition is set, and the objective function mainly includes three parts.

[0083] As described above, the objective function is used to minimize the rank metric of the low-rank matrix L, minimize the sparse metric of the sparse matrix S, and minimize the sparse matrix S and the reference signal S constructed based on the periodic characteristics of the PPG signal. p The difference between.

[0084] Among them, in the low-rank sparse matrix decomposition, by minimizing the rank metric of the low-rank matrix L and minimizing the sparse metric of the sparse matrix S, it can be ensured that the rank of the decomposed low-rank matrix L is relatively low, and the sparse matrix S is relatively sparse, that is, the sparse matrix S contains more zero elements.

[0085] In addition, because the PPG signal follows periodic changes, a weighted term of the periodic signal is also introduced into the optimization objective function, that is, a reference signal S with periodic characteristics is constructed based on the periodic characteristics of the PPG signal. p , by minimizing the sparse matrix S and the reference signal S p The difference between them can introduce a periodic constraint into the sparse matrix S, so that the periodic PPG signal can be more accurately extracted based on the sparse matrix S in the future.

[0086] Optionally, the objective function may specifically be:

[0087]

[0088] Among them, L represents a low-rank matrix, S represents a sparse matrix, and S p is the reference signal; |L| * is the nuclear norm of the low-rank matrix L, |S| 1is the sparse measure of the sparse matrix S, is the sparse matrix S and the reference signal S p The difference between them, λ and μ are adjustment parameters.

[0089] In this embodiment, the rank is represented by the nuclear norm, that is, the nuclear norm of the low-rank matrix L |L| * represents the rank of the low-rank matrix L, for example, |L| * =∑σ i (L), σ i (L) represents the singular value of the low-rank matrix L. In addition, since it is not easy to minimize the solution using the L0 norm, the L1 norm of the sparse matrix S |S| is used in this embodiment. 1 represents its sparseness measure.

[0090] In addition, the weighted term of the periodic signal in the objective function is Among them is the sparse matrix S and the reference signal S p The difference between them is used to introduce a periodic constraint. The difference can be specifically the square difference between the two.

[0091] Among them, the adjustment parameter λ is used to balance the low-rank term |L| * and the sparse term λ|S| 1 The adjustment parameter μ is used to balance the influence of the periodic constraint and other terms.

[0092] By optimizing and solving the above objective function, a low-rank matrix L and a sparse matrix S can be obtained, and the sparse matrix S mainly contains information related to the physiological signal. For example, an algorithm such as the alternating direction method of multipliers (ADMM) can be used to solve the optimization problem, and other algorithms can also be used, which is not limited in this embodiment.

[0093] Optionally, construct the reference signal S p The process may specifically include the following steps B1 to B4.

[0094] Step B1, acquiring a second video image within a plurality of PPG signal cycles; the second video image is a video image acquired at a distance less than a second preset distance; and the second preset distance does not exceed the first preset distance.

[0095] Step B2: sparsely encode the second video image to obtain a sparse dictionary containing multiple groups of basis vectors.

[0096] In this embodiment, a periodic signal, namely a reference signal S, is extracted from a video collected at a close distance. p , to ensure that the reference signal S p accuracy.

[0097] The video collected at a close distance can be obtained in advance, and video images within multiple PPG signal cycles, i.e., the second video image, can be extracted based on the video; the period size of the PPG signal can be determined based on the video collected at a close distance, and a time window can be defined, and the length of the time window is equal to one cycle of the PPG signal. In each time window, the value of the PPG signal is projected into a low-dimensional subspace, which is defined by the periodic characteristics of the signal, and the sparse dictionary D of the sparse representation is further determined.

[0098] Specifically, for videos with a collection distance less than a second preset distance, video images within multiple PPG signal cycles, i.e., second video images, can be extracted therefrom; the second preset distance does not exceed the first preset distance, for example, the second preset distance can be 5 cm, or even 0, that is, the second video image is acquired in a contact manner so as to more accurately represent the PPG signal.

[0099] For these second video images, sparse coding is performed, so that multiple groups of basis vectors of sparse representation can be determined, and a corresponding sparse dictionary D can be formed. Among them, these second video images can be used as training sets, and dictionary training can be performed to obtain a sparse dictionary D; the sparse dictionary D includes K basis vectors, which are used to sparsely represent any signal in the training set. Among them, the dictionary training process can use algorithms such as K-SVD (K-Singular Value Decomposition) and OMP (Orthogonal Matching Pursuit), which are not described in detail in this embodiment.

[0100] Step B3: for each first video image, a sparse dictionary is used to perform sparse representation and determine a corresponding sparse coefficient.

[0101] Step B4: determining a periodic reference signal according to the sparse dictionary and the sparse coefficients of each first video image.

[0102] In this embodiment, the sparse dictionary D can be predetermined. In long-distance measurement, for each first video image x i , can be sparsely represented using a sparse dictionary, so that the first video image x i The corresponding sparse coefficient c i .

[0103] For example, the size of the sparse dictionary D is (m·n)×K, where K is the number of atoms in the sparse dictionary D, that is, the number of basis vectors; for the first video image x of size m×n i , and use the sparse dictionary D to represent it sparsely, then the K×1 sparse coefficient c can be determined i .

[0104] The number of frames of the first video image is T, so T sparse coefficients c can be determined i , and the coefficient matrix C=|c 1 c 2 …c T |, then based on the sparse dictionary D and the sparse coefficients c of each first video image i The coefficient matrix C formed can determine the required reference signal S p , and S p =DC.

[0105] In this embodiment, a sparse dictionary D is pre-determined using close-range samples, and then the sparse dictionary D is used for sparse representation, so that a reference signal S having similar periodic characteristics to the PPG signal can be reconstructed more accurately. p Based on the reference signal S p A periodic constraint is added to the sparse matrix S, so that a periodic target PPG signal can be more accurately extracted based on the sparse matrix S later.

[0106] Step S203, extracting a region of interest corresponding to the skin of the subject under test from the sparse matrix, and determining a target PPG signal according to changes in the region of interest.

[0107] For details, please see Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0108] In some optional embodiments, the above step S203 "extracting a region of interest corresponding to the skin of the subject under test from the sparse matrix, and determining a target PPG signal according to changes in the region of interest" may specifically include the following steps C1 to C2.

[0109] Step C1, performing three-dimensional feature extraction on the sparse matrix to extract the spatiotemporal features corresponding to the sparse matrix.

[0110] In this embodiment, the sparse matrix contains information of multiple frames of the first video image collected at different times, that is, it contains information of the time dimension; and the sparse matrix also contains information of each pixel point of the video image, that is, it also contains information of the spatial dimension. Therefore, by performing three-dimensional feature extraction on the sparse matrix, spatiotemporal features that include both time features and spatial features can be extracted.

[0111] Among them, for the decomposed sparse matrix S∈R (m·n)×T , each column of the sparse matrix can be restored to a frame of image, and a three-dimensional sparse image sequence S′ is obtained, and S∈R m×n×T Based on the sparse image sequence S′, the corresponding spatiotemporal features are extracted.

[0112] For example, a three-dimensional (3D) convolutional neural network can be used to perform three-dimensional feature extraction to extract spatiotemporal features. Specifically, the spatiotemporal features F base The process can be shown as follows:

[0113] F base =3DCNN 1 (S) (1)

[0114] Among them, F base ∈R M×N×T×C , 3DCNN 1 () represents the three-dimensional convolution operation implemented by the first three-dimensional convolutional neural network. M and N are the height and width of the spatiotemporal feature, T is the number of frames of the first video image in the continuous multi-frame first video image, and C is the number of feature channels of the spatiotemporal feature. It can be understood that the above formula (1) can also directly generate the spatiotemporal feature F for the sparse image sequence S′ base .

[0115] M and N may be the same as the height m and width n of the first video image, or, when extracting features from a sparse matrix, downsampling is generally performed, and M and N may be determined based on a downsampling ratio d. Specifically, M=m / d, N=n / d.

[0116] Step C2, determine the time dimension attention weight and space dimension attention weight corresponding to the sparse matrix, and fuse the spatiotemporal features, the time dimension attention weight and the space dimension attention weight to generate the target PPG signal.

[0117] In this embodiment, the attention weights corresponding to the sparse matrix S are extracted from the time dimension to obtain the time dimension attention weights; similarly, the attention weights corresponding to the sparse matrix S are also extracted from the space dimension to obtain the space dimension attention weights.

[0118] By fusing the spatiotemporal features, the time dimension attention weights and the space dimension attention weights, for example by performing weighted processing, etc., a PPG signal that conforms to the current spatiotemporal characteristics, namely the target PPG signal, can be finally obtained.

[0119] In this embodiment, by extracting the attention weights of the time dimension and the space dimension, the features of the spatial and temporal positions that are more important for accurately predicting physiological indicators can be enhanced based on these attention weights. This can effectively cope with the noise caused by facial movement and light changes that may occur in long-distance scenes, and extract more accurate PPG signals, so that the physiological indicators predicted based on the PPG signals are more accurate.

[0120] Optionally, the above-mentioned step C2 "determine the time dimension attention weight and the space dimension attention weight corresponding to the sparse matrix, and fuse the spatiotemporal features, the time dimension attention weight and the space dimension attention weight to generate a target PPG signal" may specifically include steps C21 to C24.

[0121] Step C21, determining the attention weight corresponding to each frame of the first video image in the time dimension according to the sparse matrix, so as to generate a time dimension attention weight Att containing the attention weights of each frame of the first video image Temporal .

[0122] Step C22, according to the sparse matrix, determine the attention weight corresponding to each pixel point in the spatial dimension to generate a spatial dimension attention weight Att containing the attention weights of each pixel point Spatial .

[0123] Step C23, expand the time dimension attention weight by spatial dimension to obtain the first attention weight matrix Att′ Temporal ; Expand the spatial dimension attention weight by time dimension to obtain the second attention weight matrix Att′ Spatial .

[0124] Step C24, fusion of spatiotemporal features F base , the first attention weight matrix Att′ Temporal and the second attention weight matrix Att′ Spatial , generate the target PPG signal, and:

[0125] Signal PPG =f(F base ⊙Att′ Temporal ⊙Att′ Spatial );

[0126] Among them, F base ∈R M×N×T×C ;Att Temporal ∈R 1×1×T×1 , Att′ Temporal ∈R M×N×T×C , and the first attention weight matrix Att′ Temporal The value a in x,y,t,c For Att Temporal (1,1,t,1);Att Spatial ∈R M×N×1×1 , Att′ Spatial ∈R M ×N×T×C , and the second attention weight matrix Att′ Spatial The value b in x,y,t,c For Att Spatial(x,y,1,1); f() is the feature extraction function, and ⊙ represents the Hadamard product.

[0127] Moreover, T is the number of frames of the first video image in the first video image of the continuous multiple frames, M and N are the height and width of the spatiotemporal feature respectively, and C is the number of feature channels of the spatiotemporal feature; x, y, t, and c are the indexes of the height, width, video frame, and feature channel in the matrix respectively.

[0128] In this embodiment, in the time dimension, the sparse matrix S contains data related to each frame of the first video image. The corresponding attention weight can be determined based on the data of each frame. Combined with the attention weight corresponding to the T frame data, the overall time dimension attention weight Att can be obtained. Temporal Similarly, in the spatial dimension, the video image contains multiple pixels, and the sparse matrix S contains data related to each pixel. Therefore, the corresponding attention weight can be determined based on the data of each pixel, and finally the overall spatial dimension attention weight Att is obtained. Spatial .

[0129] Specifically, the time dimension attention weight Att can be extracted based on Point-wise convolution and 3D convolutional neural network. Temporal For example, extracting the time dimension attention weight Att Temporal The process can be shown as follows:

[0130] Att Temporal =3DCNN 2 (S) (2)

[0131] Similarly, the spatial dimension attention weight Att can also be extracted based on the 3D convolutional neural network Spatial , the process can be shown as follows:

[0132] Att Spatial =3DCNN 3 (S) (3)

[0133] Among them, 3DCNN 2 () represents the three-dimensional convolution operation implemented by the second three-dimensional convolutional neural network, 3DCNN 3 () represents the 3D convolution operation implemented by the third 3D convolutional neural network. The two 3D convolutional neural networks have the same input dimension but different output dimensions, one extracts temporal attention and the other extracts spatial attention.

[0134] Specifically, Att Temporal ∈R 1×1×T×1 , which can represent the attention weight on the time dimension T (also the number of frames); and Att sPatial ∈RM×N×1×1 , which can represent the calculated attention weight on the spatial dimension M×N.

[0135] As mentioned above, the spatiotemporal feature F base ∈R M×N×T×C In order to be able to integrate with the spatiotemporal features, the time dimension attention weight Att Temporal , spatial dimension attention weight Att Spatial are expanded to the same scale as the spatiotemporal feature, that is, the expanded attention weight ∈R M×N×T×C For ease of description, the time dimension attention weight Att Temporal The matrix obtained after expansion is called the first attention weight matrix Att′ Temporal , the spatial dimension attention weight Att Spatial The matrix obtained after expansion is called the second attention weight matrix Att′ Spatial .

[0136] Let x, y, t, and c represent the height, width, video frame, and feature channel index in the matrix respectively, then the first attention weight matrix Att′ Temporal In the example, the element with height x, width y, video frame t, and feature channel c is represented by a x,y,t,c indicates that, and, a x,y,t,c =Att Temporal (1,1,t,1). Among them, Att Temporal (1,1,t,1) represents the attention weight of the first video image in the t-th frame in the time dimension attention weight (the weight in the time dimension), so the first attention weight matrix Att′ Temporal The value a in x,y,t,c , only related to the frame number t.

[0137] Similarly, for the second attention weight matrix Att′ Spatial In the example, the element with height x, width y, video frame t, and feature channel c is represented by b. x,y,t,c indicates that, and b x,y,t,c =Att Spatial (x,y,1,1); where Att Spatial (x, y, 1, 1) represents the attention weight (weight on the spatial dimension) corresponding to the pixel point with height x and width y in the spatial dimension attention weight, so the second attention weight matrix Att′ Spatial The value b in x,y,t,c , only related to height x and width y.

[0138] After expansion, the first attention weight matrix Att′ Temporal , the second attention weight matrix Att′Spatial , and the spatiotemporal characteristics F base The scale is the same, that is, all three are ∈R M×N×T×C .

[0139] In this embodiment, the Hadamard product is used to calculate the spatiotemporal feature F base , the first attention weight matrix Att′ Temporal and the second attention weight matrix Att′ Spatial The fusion is not only computationally simple, but also can highlight important features related to time and space and suppress minor features.

[0140] The feature extraction function f() can also be implemented by a three-dimensional convolutional neural network. For example, the generated target PPG signal can be shown in the following formula (4):

[0141] Signal PPG =3DCNN 4 (F base ⊙Att Temporal ⊙Att Spatial ) (4)

[0142] Among them, 3DCNN 4 () represents the three-dimensional convolution operation implemented using the fourth three-dimensional convolutional neural network.

[0143] A prediction model for extracting a target PPG signal can be pre-built, and the prediction model includes multiple three-dimensional convolutional neural networks to extract the desired target PPG signal Signal from the sparse matrix S PPG . Figure 3 A schematic diagram of the prediction model is shown, in which the 3DCNN 1 , 3DCNN 2 , 3DCNN 3 , 3DCNN 4 They represent the corresponding three-dimensional convolutional neural networks respectively.

[0144] In this embodiment, the region of interest can be identified when extracting the spatiotemporal features of the sparse matrix, or the region of interest related to the physiological signal can be highlighted based on the time dimension attention weight and the space dimension attention weight, thereby automatically identifying the region of interest. In addition, one or more regions can be used as the region of interest, and the PPG signal of each region of interest can be predicted.

[0145] For example, the regions of interest include the face region and the extremity region. During training, the PPG signals of the two regions are selected in advance based on experience, and then directly calculated based on the above formula (4).

[0146] This embodiment integrates temporal attention and spatial attention, determines attention in the temporal dimension, realizes information sharing between adjacent frames, and enhances the model's ability to capture dynamic changes in PPG signals; while the spatial attention mechanism weights data containing important physiological signals, thereby enhancing the signal extraction effect.

[0147] Step S204, determining the physiological signal of the subject according to the target PPG signal.

[0148] For details, please see Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0149] For example, a multi-task two-dimensional (2D) convolutional neural network can be used to estimate physiological signals, and the network output includes heart rate HR, respiratory rate RR, diastolic blood pressure DBP, and systolic blood pressure SBP. For example, the target PPG signal is Signal PPG , the obtained physiological signal can be expressed as follows:

[0150] HR,RR,DBP,SBP=2DCNN 1 (PPG Facial ,PPG Acral );

[0151] Among them, 2DCNN 1 () represents the convolution operation implemented by the two-dimensional convolutional neural network. Based on the predicted physiological signals, the health status of the subject can be remotely monitored.

[0152] The method for remote monitoring of physiological signals provided in this embodiment can effectively obtain the changing signals in the video from the sparse matrix through the method of low-rank sparse matrix decomposition, and extract the physiological signals related to heart rate and blood pressure, which significantly improves the signal-to-noise ratio, thereby realizing accurate remote PPG measurement; this method solves the limitations of traditional methods in remote monitoring, has good adaptability and robustness in complex environments, is conducive to realizing remote medical monitoring, and provides a more convenient solution for users' health monitoring, and has important application value and market prospects.

[0153] In this embodiment, a remote monitoring physiological signal device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0154] This embodiment provides a device for remotely monitoring physiological signals, such as Figure 4 As shown, including:

[0155] An acquisition module 401 is used to acquire a plurality of consecutive frames of first video images and convert the plurality of consecutive frames of first video images into a two-dimensional matrix signal; the first video image is a video image whose acquisition distance exceeds a first preset distance;

[0156] A decomposition module 402 is used to perform low-rank sparse matrix decomposition on the two-dimensional matrix signal to determine a sparse matrix obtained by the decomposition;

[0157] An extraction module 403 is used to extract a region of interest corresponding to the skin of the subject under test from the sparse matrix, and determine a target PPG signal according to a change in the region of interest;

[0158] The processing module 404 is used to determine the physiological signal of the subject according to the target PPG signal.

[0159] In some optional implementations, the acquisition module 401 converts the first video images of the continuous multiple frames into a two-dimensional matrix signal, including:

[0160] Flatten each frame of the first video image into a one-dimensional vector;

[0161] The one-dimensional vectors corresponding to the multiple frames of the first video image are combined into a two-dimensional matrix signal X 2D , and X 2D ∈R (m·n)×T ; wherein m and n are the height and width of the first video image respectively, and T is the number of frames of the first video image in the plurality of consecutive frames of the first video image.

[0162] In some optional implementations, the decomposition module 402 performs low-rank sparse matrix decomposition on the two-dimensional matrix signal to determine a sparse matrix obtained by the decomposition, including:

[0163] Performing low-rank sparse matrix decomposition on the two-dimensional matrix signal according to the objective function, decomposing the two-dimensional matrix signal into a low-rank matrix and a sparse matrix;

[0164] The objective function is used to minimize the rank measure of the low-rank matrix, minimize the sparsity measure of the sparse matrix, and minimize the difference between the sparse matrix and a reference signal constructed based on the periodic characteristics of the PPG signal.

[0165] In some optional implementations, the process of constructing the reference signal includes:

[0166] Acquire a second video image within a plurality of PPG signal cycles; the second video image is a video image acquired at a distance less than a second preset distance; the second preset distance does not exceed the first preset distance;

[0167] Performing sparse coding on the second video image to obtain a sparse dictionary including multiple groups of basis vectors;

[0168] For each of the first video images, perform sparse representation using the sparse dictionary to determine a corresponding sparse coefficient;

[0169] A periodic reference signal is determined according to the sparse dictionary and the sparse coefficients of each of the first video images.

[0170] In some optional implementations, the objective function is:

[0171]

[0172] Wherein, L represents the low-rank matrix, S represents the sparse matrix, S p is the reference signal; |L| * is the nuclear norm of the low-rank matrix L, |S| 1 is the sparse measure of the sparse matrix S, is the sparse matrix S and the reference signal S p The difference between them, λ and μ are adjustment parameters.

[0173] In some optional implementations, the extraction module 403 extracts a region of interest corresponding to the skin of the subject from the sparse matrix, and determines a target PPG signal according to a change in the region of interest, including:

[0174] Performing three-dimensional feature extraction on the sparse matrix to extract spatiotemporal features corresponding to the sparse matrix;

[0175] Determine the time dimension attention weight and the space dimension attention weight corresponding to the sparse matrix, and fuse the spatiotemporal features, the time dimension attention weight and the space dimension attention weight to generate a target PPG signal.

[0176] In some optional implementations, the extraction module 403 determines the time dimension attention weight and the space dimension attention weight corresponding to the sparse matrix, and fuses the spatiotemporal features, the time dimension attention weight and the space dimension attention weight to generate a target PPG signal, including:

[0177] According to the sparse matrix, the attention weight corresponding to each frame of the first video image in the time dimension is determined to generate a time dimension attention weight Att containing the attention weights of the first video images of each frame Temporal ;

[0178] According to the sparse matrix, the attention weight corresponding to each pixel point in the spatial dimension is determined to generate a spatial dimension attention weight Att containing the attention weights of each pixel point Spatial ;

[0179] The time dimension attention weight is expanded into the space dimension to obtain the first attention weight matrix Att′ Temporal ; Expand the spatial dimension attention weight by time dimension to obtain the second attention weight matrix Att′ Spatial ;

[0180] Fusion of the spatiotemporal features F base , the first attention weight matrix Att′ Temporal And the second attention weight matrix Att′ Spatial , generate the target PPG signal, and:

[0181] Signal PPG =f(F base ⊙Att′ Temporal ⊙Att′ Spatial );

[0182] Among them, F base ∈R M×N×T×C ;Att Tempora1 ∈R 1×1×T×1 , Att′ Temporal ∈R M×N×T×C , and the first attention weight matrix Att′ Temporal The value a in x,y,t,c For Att Temporal (1,1,t,1);Att Spatial ∈R M×N×1×1 , Att ' Spatial ∈R M ×N×T×C , and the second attention weight matrix Att′ Spatial The value b in x,y,t,c For Att spatial (x,y,1,1); f() is the feature extraction function, ⊙ represents the Hadamard product;

[0183] T is the number of frames of the first video image in the first video image of the continuous multiple frames, M and N are respectively the height and width of the spatiotemporal feature, C is the number of feature channels of the spatiotemporal feature; x, y, t, c are respectively the indexes of the height, width, video frame, and feature channel in the matrix.

[0184] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0185] The remote physiological signal monitoring device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, including a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0186] The embodiment of the present invention also provides a computer device having the above Figure 4 The device for remotely monitoring physiological signals is shown.

[0187] See also Figure 5 , Figure 5 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 5 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.

[0188] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0189] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0190] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0191] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0192] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0193] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0194] The computer device may further include a communication interface for the computer device to communicate with other devices or a communication network.

[0195] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0196] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0197] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations should all be included in the protection scope of the present invention.

Claims

1. A method for remotely monitoring physiological signals, characterized in that: The method comprises: Acquire a first video image of a plurality of consecutive frames, and convert the first video image of the plurality of consecutive frames into a two-dimensional matrix signal; the first video image is a video image whose acquisition distance exceeds a first preset distance; Performing low-rank sparse matrix decomposition on the two-dimensional matrix signal to determine a sparse matrix obtained by the decomposition; Extracting a region of interest corresponding to the skin of the subject under test from the sparse matrix, and determining a target PPG signal according to a change in the region of interest; The physiological signal of the subject is determined according to the target PPG signal.

2. The method according to claim 1, characterized in that The step of converting the first video images of the continuous multiple frames into a two-dimensional matrix signal comprises: Flatten each frame of the first video image into a one-dimensional vector; The one-dimensional vectors corresponding to the multiple frames of the first video image are combined into a two-dimensional matrix signal X 2D , and X 2D ∈R (m·n)×T ; wherein m and n are the height and width of the first video image respectively, and T is the number of frames of the first video image in the plurality of consecutive frames of the first video image.

3. The method according to claim 1, characterized in that The step of performing low-rank sparse matrix decomposition on the two-dimensional matrix signal to determine a sparse matrix obtained by the decomposition includes: Performing low-rank sparse matrix decomposition on the two-dimensional matrix signal according to the objective function, decomposing the two-dimensional matrix signal into a low-rank matrix and a sparse matrix; The objective function is used to minimize the rank measure of the low-rank matrix, minimize the sparsity measure of the sparse matrix, and minimize the difference between the sparse matrix and a reference signal constructed based on the periodic characteristics of the PPG signal.

4. The method according to claim 3, characterized in that The process of constructing the reference signal includes: Acquire a second video image within a plurality of PPG signal cycles; the second video image is a video image acquired at a distance less than a second preset distance; the second preset distance does not exceed the first preset distance; Performing sparse coding on the second video image to obtain a sparse dictionary including multiple groups of basis vectors; For each of the first video images, perform sparse representation using the sparse dictionary to determine a corresponding sparse coefficient; A periodic reference signal is determined according to the sparse dictionary and the sparse coefficients of each of the first video images.

5. The method according to claim 3, characterized in that: The objective function is: Wherein, L represents the low-rank matrix, S represents the sparse matrix, S p is the reference signal; |L| * is the nuclear norm of the low-rank matrix L, |S|1 is the sparse measure of the sparse matrix S, is the sparse matrix S and the reference signal S p The difference between them, λ and μ are adjustment parameters.

6. The method according to any one of claims 1 to 5, characterized in that The step of extracting a region of interest corresponding to the skin of the subject under test from the sparse matrix and determining a target PPG signal according to a change in the region of interest includes: Performing three-dimensional feature extraction on the sparse matrix to extract spatiotemporal features corresponding to the sparse matrix; Determine the time dimension attention weight and the space dimension attention weight corresponding to the sparse matrix, and fuse the spatiotemporal features, the time dimension attention weight and the space dimension attention weight to generate a target PPG signal.

7. The method according to claim 6, characterized in that The determining the time dimension attention weight and the space dimension attention weight corresponding to the sparse matrix, and fusing the spatiotemporal features, the time dimension attention weight and the space dimension attention weight to generate a target PPG signal includes: According to the sparse matrix, the attention weight corresponding to each frame of the first video image in the time dimension is determined to generate a time dimension attention weight Att containing the attention weights of the first video images of each frame Temporal ; According to the sparse matrix, the attention weight corresponding to each pixel point in the spatial dimension is determined to generate a spatial dimension attention weight Att containing the attention weights of each pixel point Spatial ; The time dimension attention weight is expanded into the space dimension to obtain the first attention weight matrix Att′ Temporal ; Expand the spatial dimension attention weight by time dimension to obtain the second attention weight matrix Att′ Spatial ; Fusion of the spatiotemporal features F base , the first attention weight matrix Att′ Temporal And the second attention weight matrix Att′ Spatial , generate the target PPG signal, and: Signal PPG =f(F base ⊙Att′ Temporal ⊙Att′ Spatial ); Among them, F base ∈R M×N×T×C ;Att Temporal ∈R 1×1×T×1 , Att′ Temporal ∈R M×N×T×C , and the first attention weight matrix Att′ Temporal The value a in x，y，t，c For Att Temporal (1, 1, t, 1); Att Spatial ∈R M×N×1×1 , Att′ Spatial ∈R M ×N×T×C , and the second attention weight matrix Att′ Spatial The value b in x，y，t，c For Att Spatial (x, y, 1, 1); f() is the feature extraction function, ⊙ represents the Hadamard product; T is the number of frames of the first video image in the continuous multiple frames of the first video image, M and N are the height and width of the spatiotemporal feature, C is the number of feature channels of the spatiotemporal feature; x, y, t, c are the indexes of height, width, video frame, and feature channel in the matrix, respectively.

8. A device for remotely monitoring physiological signals, characterized in that: The device comprises: An acquisition module, used for acquiring a plurality of consecutive frames of first video images, and converting the plurality of consecutive frames of first video images into two-dimensional matrix signals; the first video images are video images whose acquisition distance exceeds a first preset distance; A decomposition module, used for performing low-rank sparse matrix decomposition on the two-dimensional matrix signal to determine a sparse matrix obtained by the decomposition; An extraction module, used to extract a region of interest corresponding to the skin of the subject under test from the sparse matrix, and determine a target PPG signal according to a change in the region of interest; A processing module is used to determine the physiological signal of the object under test according to the target PPG signal.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for remotely monitoring physiological signals according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for remotely monitoring physiological signals according to any one of claims 1 to 7.

Images