Image processing method and device for remote heart rate measurement

Abstract

The invention discloses an image processing method and device for remote heart rate measurement. The method comprises the steps that to-be-processed image information is acquired; the to-be-processed image information comprises a plurality of to-be-processed images; pre-processing the to-be-processed image information to obtain target image processing information; the target image processing information comprises a plurality of target processing images; and performing feature analysis processing on the target image processing information to obtain target signal processing result information.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method and device for remote heart rate measurement. Background Art

[0002] Heart rate (HR) is a parameter that reflects the physiological state of the human body and is an important indicator for detecting and preventing various diseases in clinical practice. It is an important basis for assessing the physiological health status of an individual. Traditional instruments such as ECG sensors measure heart rate by directly contacting the patient's skin. Although these contact-based monitoring devices can measure HR continuously and non-invasively, their contact-based characteristics limit their application scenarios. For example, in complex environments such as battlefields or natural disaster sites, contact devices cannot be used for patients with skin burns. In addition, in the case of epidemic prevention and control or the presence of infectious diseases, the use of contact devices poses a risk of cross-infection. Therefore, non-contact HR measurement using facial videos has become a hot topic in recent years. Most of the existing non-contact HR measurement methods are based on remote photoplethysmography (rPPG) technology. This technology uses changes in light reflected from facial skin to reflect changes in cardiovascular blood volume and pulse. It uses a digital camera to capture these changes, with ambient light as the illumination source. In early studies, researchers mainly extracted rPPG signals through traditional signal processing methods. However, due to the interference of noise, these traditional methods cannot cope with complex and changeable real-world scenarios and it is difficult to obtain reliable measurement results. Therefore, an image processing method and device for remote heart rate measurement are provided to improve the recovery of degraded remote photoplethysmogram signals from low-resolution images, improve the signal extraction quality, and effectively reduce noise interference, thereby significantly improving the accuracy of remote measurement. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide an image processing method and device for remote heart rate measurement, which is conducive to improving the recovery of degraded remote photoelectric volume pulse wave signals from low-resolution images, improving the signal extraction quality, and then effectively reducing noise interference, thereby significantly improving the accuracy of remote measurement.

[0004] In order to solve the above technical problem, the first aspect of the embodiment of the present invention discloses an image processing method, which includes:

[0005] Acquire image information to be processed; the image information to be processed includes a plurality of images to be processed;

[0006] Pre-processing the image information to be processed to obtain target image processing information; the target image processing information includes a plurality of target processing images;

[0007] Perform feature analysis on the target image processing information to obtain target signal processing result information.

[0008] A second aspect of an embodiment of the present invention discloses an image processing device, the device comprising:

[0009] An acquisition module, used for acquiring image information to be processed; the image information to be processed includes a plurality of images to be processed;

[0010] A first processing module is used to pre-process the image information to be processed to obtain target image processing information; the target image processing information includes a plurality of target processing images;

[0011] The second processing module is used to perform feature analysis processing on the target image processing information to obtain target signal processing result information.

[0012] A third aspect of the present invention discloses another image processing device, the device comprising:

[0013] A memory storing executable program code;

[0014] a processor coupled to the memory;

[0015] The processor calls the executable program code stored in the memory to execute part or all of the steps in the image processing method disclosed in the first aspect of the embodiment of the present invention.

[0016] The fourth aspect of the present invention discloses a computer-readable storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all of the steps in the image processing method disclosed in the first aspect of the embodiment of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 is a schematic diagram of a scene of an image processing system provided by an embodiment of the present invention;

[0019] Figure 2 It is a flowchart of an image processing method disclosed in an embodiment of the present invention;

[0020] Figure 3 is a structural schematic diagram of an image processing device disclosed in an embodiment of the present invention;

[0021] Figure 4 is a structural schematic diagram of another image processing device disclosed in an embodiment of the present invention;

[0022] Figure 5 It is a structural schematic diagram of a target feature processing model disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or equipment.

[0025] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0026] In this application, the word "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any technician in the field to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0027] It should be noted that since the method of the embodiment of the present application is executed in a computer device, the processing objects of each computer device exist in the form of data or information. For example, time is actually time information. It can be understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data for processing by the computer device. The details will not be repeated here.

[0028] The embodiments of the present application provide an image processing method, apparatus, computer device, and computer-readable storage medium, which are described in detail below.

[0029] See also Figure 1 , Figure 1 Schematic diagram of a scene of an image processing system provided in an embodiment of the present application. The image processing system may include a computer device 100, in which an image processing device is integrated, such as Figure 1 Computer equipment in.

[0030] In the embodiment of the present application, the computer device 100 is mainly used to obtain image information to be processed; the image information to be processed includes a plurality of images to be processed;

[0031] Pre-processing the image information to be processed to obtain target image processing information; the target image processing information includes a plurality of target processing images;

[0032] Perform feature analysis on the target image processing information to obtain target signal processing result information.

[0033] It can improve the recovery of degraded remote photoplethysmography signals from low-resolution images, improve the quality of signal extraction, and effectively reduce noise interference, thereby significantly improving the accuracy of remote measurements.

[0034] In the embodiment of the present application, the computer device 100 may be an independent server, or a server network or server cluster composed of servers. For example, the computer device 100 described in the embodiment of the present application includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets or a cloud server composed of a plurality of servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0035] It is understandable that the computer device 100 used in the embodiments of the present application may be a device including both receiving and transmitting hardware, that is, a device having receiving and transmitting hardware capable of performing two-way communication on a two-way communication link. Such a device may include: a cellular or other communication device having a single-line display or a multi-line display or a cellular or other communication device without a multi-line display. The specific computer device 100 may be a desktop terminal or a mobile terminal, and the computer device 100 may also be one of a mobile phone, a tablet computer, a laptop computer, etc.

[0036] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only one application scenario of the present application solution and does not constitute a limitation on the application scenario of the present application solution. Other application environments may also include Figure 1 More or less computer equipment as shown in Figure 1 Only one computer device is shown in the figure. It can be understood that the image processing system can also include one or more other services, which are not specifically limited here.

[0037] In addition, if Figure 1 As shown, the image processing system may further include a memory 200 for storing data, such as image data, position information, and the like.

[0038] It should be noted that Figure 1 The scene diagram of the image processing system shown is only an example. The image processing system and scene described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. Ordinary technicians in this field can know that with the evolution of the image processing system and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0039] The present invention discloses an image processing method and device for remote heart rate measurement, which is beneficial to recovering degraded remote photoplethysmogram signals from low-resolution images, improving signal extraction quality, and effectively reducing noise interference, thereby significantly improving the accuracy of remote measurement. The following are detailed descriptions.

[0040] Embodiment 1

[0041] See also Figure 2 , Figure 2 is a flow chart of an image processing method disclosed in an embodiment of the present invention. Figure 2 The described image processing method is applied to a management system, such as a local server or a cloud server for management, and the embodiments of the present invention do not limit this. Figure 2As shown, the image processing method may include the following operations:

[0042] 101. Obtain image information to be processed.

[0043] In the embodiment of the present invention, the image information to be processed includes a plurality of images to be processed.

[0044] 102. Pre-process the image information to be processed to obtain target image processing information.

[0045] In the embodiment of the present invention, the target image processing information includes a plurality of target processed images.

[0046] 103. Perform feature analysis on the target image processing information to obtain target signal processing result information.

[0047] It should be noted that the target feature processing model of the present application is obtained by training based on a multi-channel adversarial learning method. Furthermore, the above-mentioned training based on the multi-channel adversarial learning method is to add a data stream channel in parallel during model training, that is, to add a training subsidiary model from the first feature extraction module → the second feature extraction module → the third feature extraction module → the grid interpolation module → the dimension conversion module → the perception network module → the reshaping module in parallel, and input the two data streams into the feature fusion module at the same time to perform adversarial learning network training on the model, that is, during the model training process, the data processing path from the first feature extraction module → the second feature extraction module → the third feature extraction module → the grid interpolation module → the dimension conversion module → the perception network module → the reshaping module is more sensitive to the local area and subtle changes of the rPPG signal to the image mapping, thereby enhancing the recovery of low-quality signals, enhancing the robustness to changes in image resolution, and suppressing the irrelevant noise interference introduced by the traditional super-resolution method. Furthermore, since the feature fusion module receives two data streams simultaneously during training, the performance of the feature fusion module in low-resolution image experiments can be improved during training, and the quality gap between low-quality rPPG signals can be reduced. In addition, even under normal high-resolution image conditions, the proposed model also improves the performance of the feature fusion module.

[0048] It should be noted that the image processing method of the present application can be used for remote photoplethysmography (rPPG) heart rate monitoring technology. The problem of rPPG signal degradation in low-resolution images under long-distance conditions can be solved by using a target feature processing model trained by a multi-channel adversarial learning method, aiming to restore the degraded rPPG signal in the low-resolution image and minimize the quality gap of the rPPG signal in the low-resolution image.

[0049] It should be noted that the above-mentioned images to be processed are distributed in time series. Furthermore, the number of the images is not less than 160, which is not limited in the embodiment of the present invention.

[0050] It can be seen that implementing the image processing method described in the embodiment of the present invention is conducive to improving the recovery of degraded remote photoplethysmography signals from low-resolution images, improving the signal extraction quality, and then effectively reducing noise interference, thereby significantly improving the accuracy of remote measurement.

[0051] In an optional embodiment, the feature analysis processing is performed on the target image processing information to obtain the target signal processing result information, including:

[0052] Using the target feature processing model to perform feature extraction processing on the target image processing information, the target signal feature information is obtained;

[0053] The target signal characteristic information is converted and processed to obtain the target signal processing result information.

[0054] It can be seen that implementing the image processing method described in the embodiment of the present invention is conducive to improving the recovery of degraded remote photoplethysmography signals from low-resolution images, improving the signal extraction quality, and then effectively reducing noise interference, thereby significantly improving the accuracy of remote measurement.

[0055] In another optional embodiment, the target signal characteristic information is converted to obtain target signal processing result information, including:

[0056] Performing fast Fourier transform on the target signal characteristic information to obtain frequency domain signal characteristic information; the frequency domain signal characteristic information represents a number of frequency domain signal characteristic value information; the frequency domain signal characteristic value information includes signal value and signal frequency;

[0057] In the frequency domain signal characteristic information, the frequency domain signal characteristic value information corresponding to the maximum signal value is determined as the target frequency domain signal characteristic value information;

[0058] Determine the target signal frequency by using the signal frequency corresponding to the target frequency domain signal characteristic value information;

[0059] Get the signal gain value;

[0060] The signal gain value and the target signal frequency are multiplied to obtain target signal processing result information.

[0061] It should be noted that the fast Fourier transform of the target signal characteristic information is to convert the signal characteristic information from the time domain to the frequency domain through the fast Fourier transform. Further, the frequency domain signal characteristic value information corresponding to the highest peak (largest signal value) in the frequency domain is found in the corresponding frequency domain signal characteristic information, and its frequency is determined as the target signal frequency.

[0062] Further, the signal gain value may be set by the user as required, or may be a default value given by the system, which is not limited in the embodiment of the present invention. Further, its value may be any one of 58, 59, 60, and 61, which is not limited in the embodiment of the present invention.

[0063] It can be seen that implementing the image processing method described in the embodiment of the present invention is conducive to improving the recovery of degraded remote photoplethysmography signals from low-resolution images, improving the signal extraction quality, and then effectively reducing noise interference, thereby significantly improving the accuracy of remote measurement.

[0064] In yet another optional embodiment, Figure 5 As shown, the target feature processing model includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a grid interpolation module, a dimension conversion module, a perception network module, a reshaping module and a feature fusion module; wherein,

[0065] The input end of the first feature extraction module is configured as the model input of the target feature processing model, the output end of the first feature extraction module is connected to the input end of the second feature extraction module; the output end of the second feature extraction module is connected to the output end of the third feature extraction module; the output end of the third feature extraction module is connected to the input end of the grid interpolation module; the output end of the grid interpolation module is connected to the input end of the dimension conversion module; the output end of the dimension conversion module is connected to the input end of the perception network module; the output end of the perception network module is connected to the input end of the reshaping module; the output end of the reshaping module is connected to the input end of the feature fusion module; the output end of the feature fusion module is configured as the model output of the target feature processing model.

[0066] It should be noted that the model structures of the first feature extraction module, the second feature extraction module, and the third feature extraction module are consistent, and the embodiment of the present invention does not limit them. Further, it mainly expands the receptive field by stacking convolutional layers to extract rich signal features, that is, accurately captures the semantic information of the rPPG signal in the low-resolution face image, and splices it in the time dimension to obtain a more comprehensive feature representation, which is not limited in the embodiment of the present invention.

[0067] It should be noted that the above-mentioned grid interpolation module is constructed based on Fixed Grid Interpolation to map the extracted features to a fixed grid and mine the semantic information of the rPPG signal, which is not limited in the embodiment of the present invention.

[0068] It should be noted that the above-mentioned dimensional conversion module is constructed based on the Flatten layer, which mainly pulls the input multi-dimensional data into one dimension, which can be intuitively understood as "flattening" the data, and the embodiment of the present invention does not limit this.

[0069] It should be noted that the above-mentioned perception network module is constructed based on a multi-layer perceptron, the number of layers of which can be between 2 and 10, and the number of neurons in each layer can be between 30 and 100, which is not limited in the embodiment of the present invention.

[0070] It should be noted that the above reshaping module is constructed based on the reshape function, which is mainly used to give a new shape to the array without changing the data, and the embodiment of the present invention does not limit this.

[0071] It can be seen that implementing the image processing method described in the embodiment of the present invention is conducive to improving the recovery of degraded remote photoplethysmography signals from low-resolution images, improving the signal extraction quality, and then effectively reducing noise interference, thereby significantly improving the accuracy of remote measurement.

[0072] In another optional embodiment, the first feature extraction module includes a first convolution unit, a first normalization unit, a first activation unit, and a first pooling unit; wherein,

[0073] The input end of the first convolution unit is configured as the model input of the target feature processing model, the output end of the first convolution unit is connected to the input end of the first normalization unit; the output end of the first normalization unit is connected to the input end of the first activation unit; the output end of the first activation unit is connected to the input end of the first pooling unit; the input end of the first pooling unit is connected to the second feature extraction module.

[0074] It should be noted that the kernel size of the above-mentioned first convolution unit is 3x3x3, and it is a depth convolution block, which is not limited in this embodiment of the present invention.

[0075] It should be noted that the above-mentioned first normalization unit is constructed based on the batch normalization layer, which is not limited in the embodiment of the present invention.

[0076] It should be noted that the above-mentioned first activation unit is constructed based on the relu activation function, which is not limited in the embodiment of the present invention.

[0077] It should be noted that the first pooling unit is constructed based on the Max Pool 3D normalization layer, and its parameter specification can be [1, 2, 2], which is not limited in the embodiment of the present invention.

[0078] It can be seen that implementing the image processing method described in the embodiment of the present invention is conducive to improving the recovery of degraded remote photoplethysmography signals from low-resolution images, improving the signal extraction quality, and then effectively reducing noise interference, thereby significantly improving the accuracy of remote measurement.

[0079] In an optional embodiment, the feature fusion module includes a second convolution unit, a third convolution unit, a fourth convolution unit, a fifth convolution unit, a sixth convolution unit, a seventh convolution unit, a word segmentation unit, a multi-head attention unit, a first fusion unit, a second fusion unit, a second normalization unit, a third normalization unit and a prediction unit; wherein,

[0080] The input end of the second convolution unit is connected to the output end of the reshaping module, and the output end of the second convolution unit is connected to the input end of the third convolution unit; the output end of the third convolution unit is connected to the input end of the fourth convolution unit; the output end of the fourth convolution unit is respectively connected to the input end of the word segmentation unit and the input end of the first fusion unit; the output end of the word segmentation unit is connected to the input end of the multi-head attention unit; the output end of the multi-head attention unit is connected to the input end of the first fusion unit; the output end of the first fusion unit is connected to the input end of the second normalization unit; the output end of the second normalization unit is respectively connected to the input end of the second fusion unit and the input end of the fifth convolution unit; the output end of the fifth convolution unit is connected to the input end of the sixth convolution unit; the output end of the sixth convolution unit is connected to the input end of the seventh convolution unit; the output end of the seventh convolution unit is connected to the input end of the second fusion unit; the output end of the second fusion unit is connected to the input end of the third normalization unit; the output end of the third normalization unit is connected to the input end of the prediction unit; the output end of the prediction unit is configured as the model output of the target feature processing model.

[0081] It should be noted that the above-mentioned feature fusion module can restore the degraded rPPG signal in the low-resolution image, thereby minimizing the quality gap of the rPPG signal in the low-resolution image.

[0082] It should be noted that the kernel size of the second convolution unit is 1x5x5; the kernel sizes of the third and fourth convolution units are both 3x3x3; the kernel size of the fifth convolution unit is 1x1x1; the kernel size of the sixth convolution unit is 3x3x3, and it is a depth convolution block; the kernel size of the seventh convolution unit is 1x1x1, which is not limited in the embodiment of the present invention.

[0083] It should be noted that the above-mentioned word segmentation unit is constructed based on the tube word segmenter, which can divide the data into spatiotemporal tube tokens, which is not limited in the embodiment of the present invention.

[0084] It should be noted that the above-mentioned multi-head attention unit is constructed based on the self-attention mechanism, which is not limited in the embodiments of the present invention.

[0085] It should be noted that the first fusion unit and the second fusion unit are constructed based on the Add operation, which is not limited in the embodiment of the present invention.

[0086] It should be noted that the second normalization unit and the third normalization unit are constructed based on the batch normalization layer, which is not limited in the embodiment of the present invention.

[0087] It should be noted that the above prediction unit is constructed based on the predictor operation, which mainly performs temporal upsampling, spatial averaging and projection on the features output by the above module to obtain one-dimensional target signal feature information, which is not limited in the embodiment of the present invention.

[0088] It can be seen that implementing the image processing method described in the embodiment of the present invention is conducive to improving the recovery of degraded remote photoplethysmography signals from low-resolution images, improving the signal extraction quality, and then effectively reducing noise interference, thereby significantly improving the accuracy of remote measurement.

[0089] In another optional embodiment, pre-processing the image information to be processed to obtain target image processing information includes:

[0090] For any image to be processed in the image information to be processed, performing image detection and segmentation processing on the image to be processed to obtain a target detection and segmentation image corresponding to the image to be processed;

[0091] The image specifications of the target detection segmentation image are randomly adjusted within the image specification range to obtain a target processed image corresponding to the image to be processed.

[0092] It should be noted that the above image specification range is that the image pixels are between 32×32 and 128×128, which is not limited in the embodiment of the present invention. Further, the above random adjustment is to randomly adjust the image size to between 32×32 and 128×128, which is not limited in the embodiment of the present invention.

[0093] It should be noted that the image detection and segmentation processing of the image to be processed can be based on a trained artificial intelligence model to detect and segment the face area in the image. Further, the trained artificial intelligence model can be a model based on Cascade-CNN, a model based on R-CNN and Faster-RCNN, and a model based on a feature pyramid network, which is not limited in the embodiments of the present invention.

[0094] It can be seen that implementing the image processing method described in the embodiment of the present invention is conducive to improving the recovery of degraded remote photoplethysmography signals from low-resolution images, improving the signal extraction quality, and then effectively reducing noise interference, thereby significantly improving the accuracy of remote measurement.

[0095] Embodiment 2

[0096] See also Figure 3 , Figure 3is a schematic diagram of the structure of an image processing device disclosed in an embodiment of the present invention. Figure 3 The described device can be applied to a management system, such as a local server or a cloud server for management, etc., which is not limited in the embodiments of the present invention. Figure 3 As shown, the device may include:

[0097] The acquisition module 201 is used to acquire image information to be processed; the image information to be processed includes a plurality of images to be processed;

[0098] The first processing module 202 is used to perform pre-processing on the image information to be processed to obtain target image processing information; the target image processing information includes a plurality of target processing images;

[0099] The second processing module 203 is used to perform feature analysis processing on the target image processing information to obtain target signal processing result information.

[0100] It can be seen that implementation Figure 3 The described image processing device is beneficial to recovering degraded remote photoplethysmography signals from low-resolution images, improving the signal extraction quality, and thus effectively reducing noise interference, thereby significantly improving the accuracy of remote measurement.

[0101] In another optional embodiment, Figure 3 As shown, the first processing module 202 performs feature analysis processing on the target image processing information to obtain target signal processing result information, including:

[0102] Using the target feature processing model to perform feature extraction processing on the target image processing information, the target signal feature information is obtained;

[0103] The target signal characteristic information is converted and processed to obtain the target signal processing result information.

[0104] It can be seen that implementation Figure 3 The described image processing device is beneficial to recovering degraded remote photoplethysmography signals from low-resolution images, improving the signal extraction quality, and thus effectively reducing noise interference, thereby significantly improving the accuracy of remote measurement.

[0105] In yet another optional embodiment, Figure 3 As shown, the first processing module 202 converts the target signal characteristic information to obtain target signal processing result information, including:

[0106] Performing fast Fourier transform on the target signal characteristic information to obtain frequency domain signal characteristic information; the frequency domain signal characteristic information represents a number of frequency domain signal characteristic value information; the frequency domain signal characteristic value information includes signal value and signal frequency;

[0107] In the frequency domain signal characteristic information, the frequency domain signal characteristic value information corresponding to the maximum signal value is determined as the target frequency domain signal characteristic value information;

[0108] Determine the target signal frequency by using the signal frequency corresponding to the target frequency domain signal characteristic value information;

[0109] Get the signal gain value;

[0110] The signal gain value and the target signal frequency are multiplied to obtain target signal processing result information.

[0111] It can be seen that implementation Figure 3 The described image processing device is beneficial to recovering degraded remote photoplethysmography signals from low-resolution images, improving the signal extraction quality, and thus effectively reducing noise interference, thereby significantly improving the accuracy of remote measurement.

[0112] In yet another optional embodiment, Figure 3 As shown, the target feature processing model includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a grid interpolation module, a dimension conversion module, a perception network module, a reshaping module and a feature fusion module; wherein,

[0113] The input end of the first feature extraction module is configured as the model input of the target feature processing model, the output end of the first feature extraction module is connected to the input end of the second feature extraction module; the output end of the second feature extraction module is connected to the output end of the third feature extraction module; the output end of the third feature extraction module is connected to the input end of the grid interpolation module; the output end of the grid interpolation module is connected to the input end of the dimension conversion module; the output end of the dimension conversion module is connected to the input end of the perception network module; the output end of the perception network module is connected to the input end of the reshaping module; the output end of the reshaping module is connected to the input end of the feature fusion module; the output end of the feature fusion module is configured as the model output of the target feature processing model.

[0114] It can be seen that implementation Figure 3 The described image processing device is beneficial to recovering degraded remote photoplethysmography signals from low-resolution images, improving the signal extraction quality, and thus effectively reducing noise interference, thereby significantly improving the accuracy of remote measurement.

[0115] In yet another optional embodiment, Figure 3 As shown, the first feature extraction module includes a first convolution unit, a first normalization unit, a first activation unit, and a first pooling unit; wherein,

[0116] The input end of the first convolution unit is configured as the model input of the target feature processing model, the output end of the first convolution unit is connected to the input end of the first normalization unit; the output end of the first normalization unit is connected to the input end of the first activation unit; the output end of the first activation unit is connected to the input end of the first pooling unit; the input end of the first pooling unit is connected to the second feature extraction module.

[0117] It can be seen that implementation Figure 3 The described image processing device is beneficial to recovering degraded remote photoplethysmography signals from low-resolution images, improving the signal extraction quality, and thus effectively reducing noise interference, thereby significantly improving the accuracy of remote measurement.

[0118] In yet another optional embodiment, Figure 3 As shown, the feature fusion module includes a second convolution unit, a third convolution unit, a fourth convolution unit, a fifth convolution unit, a sixth convolution unit, a seventh convolution unit, a word segmentation unit, a multi-head attention unit, a first fusion unit, a second fusion unit, a second normalization unit, a third normalization unit and a prediction unit; wherein,

[0119] The input end of the second convolution unit is connected to the output end of the reshaping module, and the output end of the second convolution unit is connected to the input end of the third convolution unit; the output end of the third convolution unit is connected to the input end of the fourth convolution unit; the output end of the fourth convolution unit is respectively connected to the input end of the word segmentation unit and the input end of the first fusion unit; the output end of the word segmentation unit is connected to the input end of the multi-head attention unit; the output end of the multi-head attention unit is connected to the input end of the first fusion unit; the output end of the first fusion unit is connected to the input end of the second normalization unit; the output end of the second normalization unit is respectively connected to the input end of the second fusion unit and the input end of the fifth convolution unit; the output end of the fifth convolution unit is connected to the input end of the sixth convolution unit; the output end of the sixth convolution unit is connected to the input end of the seventh convolution unit; the output end of the seventh convolution unit is connected to the input end of the second fusion unit; the output end of the second fusion unit is connected to the input end of the third normalization unit; the output end of the third normalization unit is connected to the input end of the prediction unit; the output end of the prediction unit is configured as the model output of the target feature processing model.

[0120] It can be seen that implementation Figure 3 The described image processing device is beneficial to recovering degraded remote photoplethysmography signals from low-resolution images, improving the signal extraction quality, and thus effectively reducing noise interference, thereby significantly improving the accuracy of remote measurement.

[0121] In yet another optional embodiment, Figure 3 As shown, the second processing module 203 performs pre-processing on the image information to be processed to obtain target image processing information, including:

[0122] For any image to be processed in the image information to be processed, performing image detection and segmentation processing on the image to be processed to obtain a target detection and segmentation image corresponding to the image to be processed;

[0123] The image specifications of the target detection segmentation image are randomly adjusted within the image specification range to obtain a target processed image corresponding to the image to be processed.

[0124] It can be seen that implementation Figure 3 The described image processing device is beneficial to recovering degraded remote photoplethysmography signals from low-resolution images, improving the signal extraction quality, and thus effectively reducing noise interference, thereby significantly improving the accuracy of remote measurement.

[0125] Embodiment 3

[0126] See also Figure 4 , Figure 4 is a schematic diagram of the structure of another image processing device disclosed in an embodiment of the present invention. Figure 4 The described device can be applied to a management system, such as a local server or a cloud server for management, etc., which is not limited in the embodiments of the present invention. Figure 4 As shown, the device may include:

[0127] A memory 301 storing executable program codes;

[0128] a processor 302 coupled to the memory 301;

[0129] The processor 302 calls the executable program code stored in the memory 301 to execute the steps in the image processing method described in the first embodiment.

[0130] Embodiment 4

[0131] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the image processing method described in the first embodiment.

[0132] Embodiment 5

[0133] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps of the image processing method described in the first embodiment.

[0134] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0135] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0136] Finally, it should be noted that the image processing method and device for remote heart rate measurement disclosed in the embodiments of the present invention only disclose the preferred embodiments of the present invention, which are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire image information to be processed; the image information to be processed includes a plurality of images to be processed; Pre-processing the image information to be processed to obtain target image processing information; the target image processing information includes a plurality of target processing images; Perform feature analysis on the target image processing information to obtain target signal processing result information.

2. The image processing method according to claim 1, characterized in that: The performing feature analysis processing on the target image processing information to obtain target signal processing result information includes: Using a target feature processing model to perform feature extraction processing on the target image processing information to obtain target signal feature information; The target signal characteristic information is converted to obtain target signal processing result information.

3. The image processing method according to claim 2, characterized in that: The converting and processing the target signal characteristic information to obtain target signal processing result information includes: Performing a fast Fourier transform on the target signal characteristic information to obtain frequency domain signal characteristic information; the frequency domain signal characteristic information represents a number of frequency domain signal characteristic value information; the frequency domain signal characteristic value information includes a signal value and a signal frequency; In the frequency domain signal characteristic information, the frequency domain signal characteristic value information corresponding to the largest signal value is determined as the target frequency domain signal characteristic value information; Determine the target signal frequency by using the signal frequency corresponding to the target frequency domain signal characteristic value information; Get the signal gain value; The signal gain value and the target signal frequency are multiplied to obtain target signal processing result information.

4. The image processing method according to claim 2, characterized in that: The target feature processing model includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a grid interpolation module, a dimension conversion module, a perception network module, a reshaping module and a feature fusion module; wherein, The input end of the first feature extraction module is configured as the model input of the target feature processing model, the output end of the first feature extraction module is connected to the input end of the second feature extraction module; the output end of the second feature extraction module is connected to the output end of the third feature extraction module; the output end of the third feature extraction module is connected to the input end of the grid interpolation module; the output end of the grid interpolation module is connected to the input end of the dimension conversion module; the output end of the dimension conversion module is connected to the input end of the perception network module; the output end of the perception network module is connected to the input end of the reshaping module; the output end of the reshaping module is connected to the input end of the feature fusion module; the output end of the feature fusion module is configured as the model output of the target feature processing model.

5. The image processing method according to claim 4, characterized in that: The first feature extraction module includes a first convolution unit, a first normalization unit, a first activation unit, and a first pooling unit; wherein, The input end of the first convolution unit is configured as the model input of the target feature processing model, the output end of the first convolution unit is connected to the input end of the first normalization unit; the output end of the first normalization unit is connected to the input end of the first activation unit; the output end of the first activation unit is connected to the input end of the first pooling unit; the first pooling unit is connected to the input end of the second feature extraction module.

6. The image processing method according to claim 4, characterized in that: The feature fusion module includes a second convolution unit, a third convolution unit, a fourth convolution unit, a fifth convolution unit, a sixth convolution unit, a seventh convolution unit, a word segmentation unit, a multi-head attention unit, a first fusion unit, a second fusion unit, a second normalization unit, a third normalization unit and a prediction unit; wherein, The input end of the second convolution unit is connected to the output end of the reshaping module, and the output end of the second convolution unit is connected to the input end of the third convolution unit; the output end of the third convolution unit is connected to the input end of the fourth convolution unit; the output end of the fourth convolution unit is respectively connected to the input end of the word segmentation unit and the input end of the first fusion unit; the output end of the word segmentation unit is connected to the input end of the multi-head attention unit; the output end of the multi-head attention unit is connected to the input end of the first fusion unit; the output end of the first fusion unit is connected to the input end of the second normalization unit; the output end of the second normalization unit is respectively connected to the input end of the second fusion unit and the input end of the fifth convolution unit; the output end of the fifth convolution unit is connected to the input end of the sixth convolution unit; the output end of the sixth convolution unit is connected to the input end of the seventh convolution unit; the output end of the seventh convolution unit is connected to the input end of the second fusion unit; the output end of the second fusion unit is connected to the input end of the third normalization unit; the output end of the third normalization unit is connected to the input end of the prediction unit; the output end of the prediction unit is configured as the model output of the target feature processing model.

7. The image processing method according to claim 1, characterized in that: The pre-processing of the image information to be processed to obtain target image processing information includes: For any of the to-be-processed images in the to-be-processed image information, performing image detection and segmentation processing on the to-be-processed image to obtain a target detection and segmentation image corresponding to the to-be-processed image; The image specifications of the target detection segmentation image are randomly adjusted within the image specification range to obtain a target processed image corresponding to the image to be processed.

8. An image processing device, characterized in that: The device comprises: An acquisition module, used for acquiring image information to be processed; the image information to be processed includes a plurality of images to be processed; A first processing module is used to pre-process the image information to be processed to obtain target image processing information; the target image processing information includes a plurality of target processing images; The second processing module is used to perform feature analysis processing on the target image processing information to obtain target signal processing result information.

9. An image processing device, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the image processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the image processing method according to any one of claims 1 to 7.

Images