Physiological signal detection method, device, terminal equipment and storage medium based on Moore effect

Abstract

The present application relates to the field of biomedical engineering technology. The present application discloses a physiological signal detection method, apparatus, terminal device, and storage medium based on the moiré effect, which can improve the accuracy of detecting physiological signals. The physiological signal detection method based on the moiré effect comprises obtaining a target video of the chest and / or abdomen of a target user, wherein each frame of the original image in the target video contains moiré patterns, and the moiré patterns are used to amplify the pixel displacement caused by the physiological movement of the target user; performing moiré pattern enhancement processing on each frame of the original image to obtain multiple frames of target images; and performing signal extraction processing on the multiple frames of target images to obtain physiological signals.

Description

Technical Field

[0001] The present application relates to the field of biomedical engineering technology. More specifically, the present application relates to a physiological signal detection method, apparatus, terminal device, and storage medium based on the moiré effect. Background Art

[0002] Traditional physiological signal detection methods use a camera to capture a target video of the face, arms, legs, or heart. An initial region of interest (ROI) is obtained from the target video. This initial region of interest is then reconstructed using a pre-defined algorithm (such as pixel clustering, pixel value distribution probability density function, or deep learning algorithm) to generate a final ROI. Signal extraction and processing are then performed on this final ROI to obtain the physiological signal. Improvements to traditional methods have mostly focused on the pre-defined algorithms. However, these algorithms are often affected by data quality factors, and poor data quality can easily lead to deviations between the final physiological signal and the actual physiological signal. Therefore, the accuracy of physiological signal detection in existing technologies needs to be improved. Summary of the Invention

[0003] The purpose of the embodiments of the present application is to provide a physiological signal detection method, apparatus, terminal device, and storage medium based on the moiré effect, which can improve the accuracy of physiological signal detection. The embodiments of the present application are mainly achieved through the following technical solutions:

[0004] A first aspect of an embodiment of the present application provides a physiological signal detection method based on the moiré effect, comprising:

[0005] Acquire a target video of the chest and / or abdomen of a target user, wherein each frame of the original image in the target video contains moiré patterns, and the moiré patterns are used to amplify pixel displacement caused by physiological movement of the target user;

[0006] Each frame of the original image is subjected to moiré enhancement processing to obtain multiple frames of target images;

[0007] Signal extraction processing is performed on the multiple frames of target images to obtain physiological signals.

[0008] According to one embodiment of the present application, the step of obtaining a target video of the chest and / or abdomen of a target user includes:

[0009] Setting a preset condition, wherein the preset condition is used to generate moiré patterns in each frame of the original image;

[0010] A target video of the chest and / or abdomen of the target user is acquired based on the preset condition.

[0011] According to one embodiment of the present application, the steps of performing moiré enhancement processing on each frame of the original image to obtain multiple frames of target images include:

[0012] Each frame of the original image is grayscaled to obtain a grayscale image corresponding to each frame of the original image;

[0013] Each frame of grayscale image is filtered to obtain a target image corresponding to each frame of grayscale image, so as to enhance the moiré pattern of each frame of original image.

[0014] According to one embodiment of the present application, the step of performing signal extraction processing on the multiple frames of target images to obtain physiological signals includes:

[0015] A convolutional neural network is used to identify a region of interest in each frame of the target image, and a first target region of interest corresponding to each frame of the target image is obtained;

[0016] A motion detection algorithm is used to perform signal extraction processing on all first target regions of interest to obtain the physiological signals.

[0017] According to one embodiment of the present application, the step of performing signal extraction processing on all first target regions of interest using a motion detection algorithm to obtain the physiological signal includes:

[0018] Using a motion detection algorithm to estimate first target features of each pixel in each first target region of interest in multiple predetermined directions between consecutive frames;

[0019] Accumulating and calculating all first target features of each pixel point in each predetermined direction between consecutive frames to obtain a first displacement signal corresponding to each pixel point in each predetermined direction between consecutive frames;

[0020] calculating an average value of all first displacement signals in each predetermined direction to obtain a physiological motion signal corresponding to each predetermined direction;

[0021] All physiological motion signals in predetermined directions are fused and processed to obtain the physiological signal.

[0022] According to one embodiment of the present application, the plurality of predetermined directions are horizontal directions and vertical directions.

[0023] According to one embodiment of the present application, the step of performing signal extraction processing on the multiple frames of target images to obtain physiological signals includes:

[0024] A multi-scale downsampling analysis is performed on each frame of the target image using a multi-scale analysis method to obtain multiple sub-images of different resolutions corresponding to each frame of the target image;

[0025] A convolutional neural network is used to identify a region of interest in each frame sub-image, and a second target region of interest corresponding to each frame sub-image is obtained;

[0026] Using a motion detection algorithm, signal extraction processing is performed on all second target regions of interest at each resolution to obtain sub-signals corresponding to each resolution;

[0027] All sub-signals are fused to obtain the physiological signal.

[0028] A second aspect of an embodiment of the present application provides a physiological signal detection apparatus based on the moiré effect, comprising: a terminal device and a sensor, the terminal device comprising a processor and a memory, the memory being configured to store a computer program, the processor being configured to call and execute the computer program stored in the memory to perform the steps of the physiological signal detection method based on the moiré effect provided in the first aspect of the embodiment of the present application;

[0029] The sensor is used to capture a target video of the chest and / or abdomen of a target user and transmit the target video to the processor.

[0030] According to a third aspect of an embodiment of the present application, a terminal device is provided, comprising: a processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory, and executing the steps of the physiological signal detection method based on the moiré effect provided in the first aspect of the embodiment of the present application.

[0031] In a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store a computer program, and the computer program enables a computer to execute the steps of the physiological signal detection method based on the moiré effect provided in the first aspect of the embodiment of the present application.

[0032] The beneficial effects of the embodiments of the present application include:

[0033] The embodiment of the present application introduces moiré patterns into the video captured by the sensor from the perspective of the quality of the processing object, amplifies the pixel displacement caused by the physiological movement of the target user, enhances the visibility of the physiological movement characteristics, and thus improves the accuracy of detecting physiological signals. Specifically, the present application obtains a target video of the chest and / or abdomen of the target user, wherein each frame of the original image in the target video contains moiré patterns, and the moiré patterns are used to amplify the pixel displacement caused by the physiological movement of the target user; performs moiré enhancement processing on each frame of the original image to obtain multiple frames of target images; and performs signal extraction processing on the multiple frames of target images to obtain physiological signals. Compared with the prior art, the embodiment of the present application takes the quality of the processing object (i.e., data quality) as an improvement point, effectively avoiding the situation where the physiological signal deviates from the actual existence due to poor data quality, thereby improving the accuracy of detecting physiological signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the conventional technology, the following briefly introduces the drawings required for use in the embodiments or the conventional technology descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0035] Figure 1 A flowchart of a physiological signal detection method based on moiré effect in some embodiments of the present application;

[0036] Figure 2 An image of a target surface having moiré patterns in this application;

[0037] Figure 3 For this application Figure 2 The corresponding frequency domain representation;

[0038] Figure 4 An image of the target surface in this application without obvious moiré patterns;

[0039] Figure 5 For this application Figure 4 The corresponding frequency domain representation;

[0040] Figure 6 The signal-to-noise ratio heat map of the respiratory signal extracted from each local area of ​​the target surface when moiré patterns are formed under five different respiratory motion amplitudes in this application;

[0041] Figure 7 This is a principle block diagram of a physiological signal detection device based on the moiré effect in some embodiments of the present application;

[0042] Figure 81 is a principle block diagram of another embodiment of the physiological signal detection device based on the moiré effect of the present application;

[0043] Figure 9 1 is a principle block diagram of a physiological signal detection device based on the moiré effect in some further embodiments of the present application;

[0044] Figure 10 This is a principle block diagram of the terminal device of the present application in some embodiments. DETAILED DESCRIPTION

[0045] To make the above-mentioned objects, features, and advantages of the present application more clearly understood, the specific embodiments of the present application are described in detail below with reference to the accompanying drawings. The following description sets forth many specific details to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar improvements without violating the scope of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0046] It should be noted that the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of such features. In the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0047] The terms "exemplary" or "for example" are used to indicate an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0048] The terms "comprises," "comprising," or any other variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, product, or apparatus.

[0049] The term "moiré" refers to periodic, low-frequency interference fringes formed by the superposition of two sets of patterns with similar frequencies. During the digital imaging process, noticeable moiré patterns can also occur when the sensor (i.e., camera) is undersampled. Although these fringes are often considered image noise in photography and can seriously affect the visual quality of the image, controllable moiré patterns are of great value in some fields, such as image information hiding and macro measurement. Currently, moiré patterns have not been applied to sensor physiological testing. Due to the characteristics of modern textile technology, items such as clothing, bed sheets, and quilt covers used in daily life and hospital settings all have fabrics with high-density and regularly arranged textures, which create ideal conditions for the generation of moiré patterns.

[0050] Unless otherwise defined, all technical and scientific terms used in the specification of this application have the same meaning as commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used in the specification of this application includes any and all combinations of one or more of the relevant listed items.

[0051] The specific implementation of this application is further described below with reference to the accompanying drawings.

[0052] refer to Figure 1 FIG. 1 is a flow chart of a physiological signal detection method based on the moiré effect provided in the first aspect of the embodiment of the present application. Figure 1 In the embodiment, the physiological signal detection method based on the moiré effect includes:

[0053] S1. Acquire a target video of the chest and / or abdomen of a target user, wherein each frame of the original image in the target video contains moiré patterns, and the moiré patterns are used to amplify pixel displacement caused by physiological movements of the target user.

[0054] The target video is acquired by a sensor of a physiological signal detection device based on the Moore effect based on preset conditions. The sensor is a camera.

[0055] Furthermore, the step of obtaining a target video of the chest and / or abdomen of the target user includes:

[0056] S11 . Setting preset conditions, where the preset conditions are used to generate moiré patterns in each frame of the original image.

[0057] The preset conditions are used to adjust the parameters of the sensor so that the sensor can produce obvious moiré patterns when recording the target video.

[0058] Specifically, the calculation formula of the preset condition is:

[0059] ;

[0060] ;

[0061] ;

[0062] in, is the period of the generated moiré pattern, is the spatial frequency of the image formed by the optical lens of the sensor when a fabric with a periodic structure on the surface of the target user's chest and / or abdomen (such as clothing worn or a bed sheet covered) passes through the optical lens of the sensor; Image Side refers to the direction in which light rays converge to form an image after passing through the lens of the sensor. is the spatial frequency of the sensor; is the empirical threshold, usually 6-50 pixels; is the focal length of the sensor's lens; is the magnification of the sensor's lens; The spacing between the stripes on the surface of the target user's photographed area (i.e., the target surface, such as the bed sheet or clothing covering the target user's photographed area); is the distance between the target user's photographed part (i.e., the photographed target) and the camera; Object Side, which is the direction from which light enters the sensor. is the downsampling factor of the sensor in sampling mode; is the pixel pitch of the sensor; Refers to parameters of the sensor, such as the effective aperture of the sensor lens, the conjugation relationship between the object and image spaces, the radius of curvature of the sensor, and / or the center of curvature of the sensor. It should be understood that the parameters of the sensor are not limited to the effective aperture, the conjugation relationship between the object and image spaces, the radius of curvature, and / or the center of curvature described above, but may also be other parameters, which can be specifically set by those skilled in the art based on actual needs.

[0063] It should be understood that the embodiment of the present application can increase or decrease the focal length of the lens of the sensor. To change the sensor lens magnification , thereby reducing or increasing the spatial frequency of the image of the target user's chest and / or abdomen through the optical lens of the sensor , and set the downsampling multiple of the sensor in sampling mode Or the downsampling factor in image post-processing (multi-scale analysis) to adjust .

[0064] S12. Acquire a target video of the chest and / or abdomen of the target user based on the preset condition.

[0065] Furthermore, the fabric used by the target user in each frame of the original image is a plain fabric. In other embodiments, the fabric used by the target user in each frame of the original image may also be a patterned fabric, which can be specifically configured according to the actual needs of those skilled in the art.

[0066] The embodiment of the present application fully utilizes the high-density regular structure of the fabric itself, and has good convenience and scalability.

[0067] S2. Perform moiré enhancement processing on each frame of the original image to obtain multiple frames of target images.

[0068] Furthermore, step S2 includes:

[0069] S21 , performing grayscale processing on each frame of the original image to obtain a grayscale image corresponding to each frame of the original image.

[0070] For example, a first preset algorithm can be used to grayscale each frame of the original image. The first preset algorithm can be a function for color space conversion in OpenCV (OpenCV is a cross-platform computer vision and machine learning software library). In other embodiments, the first preset algorithm can also be other algorithms, and the specific configuration can be determined by those skilled in the art based on actual needs.

[0071] S22 , performing filtering processing on each frame of grayscale image to obtain a target image corresponding to each frame of grayscale image, so as to enhance the moiré pattern of each frame of original image.

[0072] For example, a second preset algorithm can be used to filter each grayscale image frame. The second preset algorithm can be a Gaussian filter function, a median filter function, or a bilateral filter function in OpenCV. In other embodiments, the second preset algorithm can also be other algorithms, which can be specifically set by those skilled in the art based on actual needs.

[0073] The settings of steps S21 and S22 can reduce noise interference in the image and enhance the visibility of moiré patterns in the image.

[0074] S3. Perform signal extraction processing on the multiple frames of target images to obtain physiological signals.

[0075] Furthermore, step S3 includes:

[0076] S31. Use a convolutional neural network to identify a region of interest in each frame of the target image, and obtain a first target region of interest corresponding to each frame of the target image.

[0077] The convolutional neural network can be R-CNN (Regions with CNN features), Fast R-CNN (Fast Regions with CNN features), or SSD (Single Shot MultiBox Detector). The convolutional neural network is not limited to the aforementioned ones and can also be other ones. The specific configuration can be determined by those skilled in the art based on actual needs.

[0078] In this embodiment of the present application, the chest and / or abdomen of the target user is used as the region of interest.

[0079] S32: Perform signal extraction processing on all first target regions of interest using a motion detection algorithm to obtain the physiological signals.

[0080] The motion detection algorithm is an optical flow method, a background subtraction method, or a frame difference method. In the embodiment of the present application, the optical flow method is used. In other embodiments, the motion detection algorithm is not limited to the optical flow method, the background subtraction method, or the frame difference method, and can be specifically set by those skilled in the art according to actual needs.

[0081] Furthermore, step S32 includes:

[0082] S321 , using a motion detection algorithm to estimate first target features of each pixel point in each first target region of interest in multiple predetermined directions between consecutive frames.

[0083] In the embodiment of the present application, the plurality of predetermined directions are horizontal and vertical directions. In other embodiments, those skilled in the art can set other directions according to actual needs.

[0084] The first target feature is a velocity component. In other embodiments, those skilled in the art may set it to other features according to actual needs, such as an acceleration component.

[0085] S322: Accumulate and calculate all first target features of each pixel point in each predetermined direction between consecutive frames to obtain a first displacement signal corresponding to each pixel point in each predetermined direction between consecutive frames.

[0086] S323: Calculate the average value of all first displacement signals in each predetermined direction to obtain a physiological motion signal corresponding to each predetermined direction.

[0087] S324: Fusing the physiological motion signals of all predetermined directions to obtain the physiological signal.

[0088] Specifically, a signal fusion method is used to fuse all physiological motion signals in predetermined directions. The signal fusion method may be a principal component analysis method.

[0089] Furthermore, in the embodiment of the present application, after fusing the physiological motion signals of all predetermined directions, the fused signals are filtered to obtain the physiological signals.

[0090] The physiological signal may be a respiratory signal or a cardiac oscillation signal. In other embodiments, the physiological signal is not limited to the respiratory signal or cardiac oscillation signal, and those skilled in the art may set the specific type of the signal according to actual needs.

[0091] From the perspective of the quality of the processing object, the embodiment of the present application introduces moiré patterns into the video captured by the sensor, amplifies the pixel displacement caused by the physiological movement of the target user, enhances the visibility of the physiological movement characteristics, and thus improves the accuracy of detecting physiological signals.

[0092] Due to the amplification effect of moiré patterns, the embodiments of the present application can expand the effective physiological detection area, extract physiological signals in a larger range, and reduce the dependence of traditional methods on specific areas with obvious gradient changes.

[0093] The application of moiré patterns can improve the sensitivity of physiological signal extraction and is suitable for application scenarios such as weak motion (such as shallow breathing, heart vibration), occlusion by thick coverings, and textureless surfaces.

[0094] In addition, the embodiments of the present application enhance the visibility of physiological motion signals from the signal source end, which can reduce the computing power burden of traditional detection methods and at the same time achieve a greater improvement in physiological detection results. In some embodiments, step S3 includes:

[0095] S41 , performing multi-scale downsampling analysis on each frame of the target image using a multi-scale analysis method to obtain a plurality of sub-images of different resolutions corresponding to each frame of the target image.

[0096] The multi-scale analysis method is an image pyramid method. In other embodiments, the multi-scale analysis method can be a deep learning algorithm, which can be specifically configured by those skilled in the art according to actual needs.

[0097] The use of the multi-scale analysis method can also significantly amplify the pixel displacement caused by physiological motion (including breathing and heart vibration), and can achieve accurate and robust physiological motion detection in extremely challenging application scenarios such as low-texture surfaces, weak motion, and occlusion by thick coverings.

[0098] S42. Use a convolutional neural network to identify a region of interest in each frame sub-image, and obtain a second target region of interest corresponding to each frame sub-image.

[0099] S43: Perform signal extraction processing on all second target regions of interest at each resolution using a motion detection algorithm to obtain sub-signals corresponding to each resolution.

[0100] Furthermore, step S43 includes:

[0101] S431 , using a motion detection algorithm to estimate second target features of each pixel in each second target region of interest at each resolution in multiple predetermined directions between consecutive frames.

[0102] The second target characteristic is a velocity component.

[0103] S432. Accumulate and calculate all second target features of each pixel point in each second target region of interest at each resolution in each predetermined direction between consecutive frames to obtain a second displacement signal corresponding to each pixel point in each second target region of interest at each resolution in each predetermined direction between consecutive frames.

[0104] S433: Calculate the average value of all second displacement signals in each predetermined direction at each resolution to obtain a sub-signal corresponding to each predetermined direction at each resolution. S44: Fusion process all sub-signals to obtain the physiological signal.

[0105] The settings of steps S41 to S44 can adjust the prominence of the moiré pattern and the motion amplification factor, thereby improving the extraction quality of the physiological signal.

[0106] The present application embodiment takes the respiratory signal detection as an example. When detecting the respiratory signal, any frame image of the target video is an image of the target surface with moiré patterns. Figure 2 As shown, the image of the target surface with moiré patterns is converted into a frequency domain representation, which can be referred to Figure 3 In order to facilitate the comparison of images with and without moiré patterns, the present embodiment also provides an image of the target surface without obvious moiré patterns (refer to Figure 4 ), and converting the image of the target surface without obvious moiré into a frequency domain representation (refer to Figure 5 shown).

[0107] The present embodiment also provides a heat map of the signal-to-noise ratio of the respiratory signal extracted from each local area when the target surface has or does not have moiré patterns under five different respiratory motion amplitudes (amplitude 5mm (mm is millimeter), amplitude 3mm, amplitude 2mm, amplitude 1mm and amplitude 0.5mm, frequency 20bpm (bpm is Beats Per Minute, beats per minute)). Figure 6 As shown. Figure 6 In the , each signal-to-noise ratio heat map has an effective breathing detection area (mean absolute error MAE of breathing rate estimation < 3bpm), which can be referred to Figure 6 Areas covered in medium yellow, green, and light blue.

[0108] In some embodiments, after step S3, the physiological signal detection method based on the moiré effect further comprises:

[0109] S5. Perform feature extraction processing on the physiological signal using a feature extraction algorithm to obtain physiological parameters.

[0110] The feature extraction algorithm may be a peak detection method or a deep learning model (such as an LSTM (Long Short-Term Memory) network). In other embodiments, the feature extraction algorithm may also be other algorithms, which may be determined by those skilled in the art based on actual needs.

[0111] The physiological parameter is respiratory rate or heart rate. In other embodiments, the physiological parameter may also include other parameters, which can be determined by those skilled in the art according to actual needs.

[0112] Since the physiological signal has a high accuracy, the extraction accuracy of the physiological parameters can be improved.

[0113] refer to Figure 7 FIG. 1 is a block diagram showing the principle of a physiological signal detection device based on the moiré effect provided in the second aspect of the embodiment of the present application. Figure 7 In the embodiment, the physiological signal detection device 100 based on the moiré effect includes: a terminal device 101 and a sensor 102, the terminal device includes a processor and a memory, the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to perform the steps of the physiological signal detection method based on the moiré effect provided in the first aspect of the embodiment of the present application;

[0114] The sensor 102 is configured to capture a target video of the chest and / or abdomen of a target user and transmit the target video to the processor.

[0115] The sensor 102 is a camera. The sensor 102 can be understood as Figure 8 The camera module in the apparatus not only has a high frame rate to ensure continuous detection, but also has a high spatial sampling frequency. In other embodiments, multiple sensors can be used to overcome the limitations of a single sensing technology. These multiple sensors can include infrared cameras and depth cameras, for example. The use of multiple sensors allows the entire device to more stably capture physiological motion signals, improving the robustness of signal detection.

[0116] The computer program includes an image preprocessing module and a physiological respiratory motion extraction module. The image preprocessing module is used to perform moiré enhancement processing on each frame of the original image to obtain multiple frames of target images. The physiological respiratory motion extraction module is used to use a convolutional neural network to identify the region of interest in each frame of the target image to obtain the first target region of interest corresponding to each frame of the target image; use a motion detection algorithm to perform signal extraction processing on all first target regions of interest to obtain the physiological signal. Alternatively, the physiological respiratory motion extraction module is used to use a convolutional neural network to identify the region of interest in each frame of the sub-image to obtain the second target region of interest corresponding to each frame of the sub-image; use a motion detection algorithm to perform signal extraction processing on all second target regions of interest of each resolution to obtain sub-signals corresponding to each resolution; and fuse all sub-signals to obtain the physiological signal. The image preprocessing module and the physiological respiratory motion extraction module can refer to Figure 8 shown.

[0117] The embodiments of the present application do not require the target user to wear / cover clothing with specific patterns (such as grids, QR codes, etc.) or use active light sources to project light spots on the target surface, thereby simplifying the design of the detection device and improving the applicability and versatility of the detection device.

[0118] In some embodiments, the computer program further includes a multi-scale analysis module, which is configured to perform multi-scale downsampling analysis on each frame of the target image using a multi-scale analysis method to obtain a plurality of sub-images of different resolutions corresponding to each frame of the target image. The multi-scale analysis module can refer to Figure 9 shown.

[0119] The multi-scale analysis module can optimize the performance of the detection device.

[0120] The third aspect of the embodiment of the present application provides a terminal device, the principle block diagram of the terminal device can be as follows: Figure 10As shown. The terminal device includes a processor, a memory, a network interface, a display screen and a temperature sensor connected via a system bus. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a physiological signal detection method based on the Moore effect is implemented. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the temperature sensor is pre-set inside the terminal device to detect the operating temperature of the internal device.

[0121] Those skilled in the art will understand that Figure 10 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the terminal device to which the solution of the present invention is applied. The specific terminal device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0122] In some embodiments, an embodiment of the present application provides a terminal device, comprising a processor and a memory, the memory being configured to store a computer program, the processor being configured to call and execute the computer program stored in the memory to perform the steps of the physiological signal detection method based on the moiré effect provided in the first aspect of the embodiment of the present application. A fourth aspect of the embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium being configured to store a computer program that causes a computer to perform the steps of the physiological signal detection method based on the moiré effect provided in the first aspect of the embodiment of the present application.

[0123] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the various embodiments provided herein may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0124] The technical features of the above embodiments can be combined without changing the basic principles of this application. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of patent protection for the present application shall be determined by the appended claims.

Claims

1. A physiological signal detection method based on the moiré effect, characterized in that: include: Acquire a target video of the chest and / or abdomen of a target user, wherein each frame of the original image in the target video contains moiré patterns, and the moiré patterns are used to amplify pixel displacement caused by physiological movement of the target user; Each frame of the original image is subjected to moiré enhancement processing to obtain multiple frames of target images; performing signal extraction processing on the multiple frames of target images to obtain physiological signals; The step of acquiring a target video of the chest and / or abdomen of the target user includes: setting a preset condition, the preset condition being used to generate moiré patterns in each frame of the original image, the preset condition being further used to adjust sensor parameters so that the sensor generates moiré patterns when recording the target video; and acquiring the target video of the chest and / or abdomen of the target user based on the preset condition; The calculation formula of the preset condition is: ; ; ;in, is the period of the generated moiré pattern, is the spatial frequency of the image formed by the fabric with periodic structure on the chest and / or abdomen surface of the target user through the optical lens of the sensor; Refers to the image side; is the spatial frequency of the sensor; is the empirical threshold, 6-50 pixels; is the focal length of the sensor's lens; is the magnification of the sensor's lens; is the spacing of the surface stripe texture of the target user's photographed part; is the distance between the target user’s photographed part and the camera; Refers to things; is the downsampling factor of the sensor in sampling mode; is the pixel pitch of the sensor; Refers to parameters of the sensor, including the effective aperture of the sensor lens, the conjugate relationship between the object space and the image space, the curvature radius of the sensor and / or the curvature center of the sensor.

2. The physiological signal detection method based on the moiré effect according to claim 1, characterized in that: Each frame of the original image is subjected to moiré enhancement processing, and the steps of obtaining multiple frames of target images include: Each frame of the original image is grayscaled to obtain a grayscale image corresponding to each frame of the original image; Each frame of grayscale image is filtered to obtain a target image corresponding to each frame of grayscale image, so as to enhance the moiré pattern of each frame of original image.

3. The physiological signal detection method based on the moiré effect according to claim 1, characterized in that: The step of performing signal extraction processing on the multiple frames of target images to obtain physiological signals includes: A convolutional neural network is used to identify a region of interest in each frame of the target image, and a first target region of interest corresponding to each frame of the target image is obtained; A motion detection algorithm is used to perform signal extraction processing on all first target regions of interest to obtain the physiological signals.

4. The physiological signal detection method based on the moiré effect according to claim 3, characterized in that: The steps of performing signal extraction processing on all first target regions of interest using a motion detection algorithm to obtain the physiological signals include: Using a motion detection algorithm to estimate first target features of each pixel in each first target region of interest in multiple predetermined directions between consecutive frames; Accumulating and calculating all first target features of each pixel point in each predetermined direction between consecutive frames to obtain a first displacement signal corresponding to each pixel point in each predetermined direction between consecutive frames; calculating an average value of all first displacement signals in each predetermined direction to obtain a physiological motion signal corresponding to each predetermined direction; All physiological motion signals in predetermined directions are fused and processed to obtain the physiological signal.

5. The physiological signal detection method based on the moiré effect according to claim 4, characterized in that: The plurality of predetermined directions are a horizontal direction and a vertical direction.

6. The physiological signal detection method based on the moiré effect according to claim 1, characterized in that: The step of performing signal extraction processing on the multiple frames of target images to obtain physiological signals includes: A multi-scale downsampling analysis is performed on each frame of the target image using a multi-scale analysis method to obtain multiple sub-images of different resolutions corresponding to each frame of the target image; A convolutional neural network is used to identify a region of interest in each frame sub-image, and a second target region of interest corresponding to each frame sub-image is obtained; Using a motion detection algorithm, signal extraction processing is performed on all second target regions of interest at each resolution to obtain sub-signals corresponding to each resolution; All sub-signals are fused to obtain the physiological signal.

7. A physiological signal detection device based on the moiré effect, characterized in that: include: A terminal device and a sensor, the terminal device comprising a processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to perform the steps of the physiological signal detection method based on the moiré effect according to any one of claims 1 to 6; The sensor is used to capture a target video of the chest and / or abdomen of a target user and transmit the target video to the processor.

8. A terminal device, characterized in that: include: A processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute the steps of the physiological signal detection method based on the moiré effect as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that Used to store a computer program, wherein the computer program enables a computer to execute the steps of the physiological signal detection method based on the moiré effect according to any one of claims 1 to 6.

Images