Denoising method for PPG frequency domain signal, and device and storage medium

Abstract

Provided in the present application are a denoising method for a PPG frequency domain signal, and a device and a storage medium. In the method, by using a transformer encoder model and taking an ACC signal and an IR signal as denoising reference signals, a data-driven method is used to perform learning, so as to construct a PPG spectrum denoising model; and then in practical application, by using the constructed PPG spectrum denoising model, a PPG frequency domain signal is denoised by means of the collected ACC signal and IR signal, so that on the basis of a PPG signal, a heart rate can be accurately measured even in a motion state.

Description

PPG frequency domain signal denoising method, device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on August 17, 2023, with application number 202311051681.X and invention name “PPG frequency domain signal denoising method, device and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of communication technology, and in particular to a denoising method, device and storage medium for PPG frequency domain signals. Background Art

[0003] Photoplethysmography (PPG) is a noninvasive method that uses photoelectric technology to detect changes in blood volume in living tissue. Because the intensity of reflected light is related to blood flow velocity, which is also affected by the cyclical heart rate, PPG signals can be used to measure blood oxygen saturation (SPO2), heart rate, blood pressure, and other parameters.

[0004] However, since PPG signals are particularly sensitive to motion, i.e., easily interfered by motion artifacts, the correlation prediction results based on PPG signals are not accurate enough in a motion state.

[0005] Summary of the Invention

[0006] In order to solve the above technical problems, the present application provides a PPG frequency domain signal denoising method, device and storage medium, which aims to remove noise in the PPG spectrum characteristics, so that accurate measurement based on the PPG signal can be performed even in motion.

[0007] In a first aspect, the present application provides a method for denoising a PPG frequency domain signal. The method comprises: obtaining a photoplethysmography (PPG) signal, an acceleration (ACC) signal, and an infrared (IR) signal collected by a wearable device; denoising the spectral features of the PPG signal based on a PPG spectrum denoising model, spectral features in the ACC signal, and spectral features in the IR signal to obtain a denoised PPG frequency domain signal; wherein the PPG spectrum denoising model is a model obtained by training a transformer encoder model using the spectral features in the PPG sample signal, the spectral features in the ACC sample signal, and the spectral features in the IR sample signal as inputs and a PPG spectrum gold standard as a supervisory signal.

[0008] Therefore, by using the transformer encoder model, taking the ACC signal and IR signal as the denoising reference signals, a data-driven approach is adopted to learn and construct a PPG spectrum denoising model. Then, in practical applications, the constructed PPG spectrum denoising model is used to denoise the PPG frequency domain signal through the collected ACC signal and IR signal, so that the heart rate can be accurately measured based on the PPG signal even in exercise.

[0009] According to the first aspect, obtaining the PPG signal, the ACC signal, and the IR signal collected by the wearable device includes: obtaining the PPG signal, the ACC signal, and the IR signal collected by the wearable device in the same state and the same period.

[0010] Therefore, by constraining the denoising reference signal selected when denoising the PPG frequency domain signal, that is, the ACC signal and the IR signal are collected in the same period as the PPG signal and when the wearable device is in the same state, the effectiveness of the denoising result of the PPG frequency domain signal is guaranteed, and the denoised PPG frequency domain signal is guaranteed to meet the spectrum characteristics under ideal conditions (noise-free), so that the subsequent heart rate prediction based on the PPG spectrum characteristics is more accurate.

[0011] According to the first aspect, or any implementation of the first aspect above, the PPG signal, the ACC signal, and the IR signal are PPG signals, ACC signals, and IR signals in the time domain; according to the PPG spectrum denoising model, the spectrum features in the ACC signal, and the spectrum features in the IR signal, the spectrum features of the PPG signal are denoised to obtain a denoised PPG frequency domain signal, including: converting the acquired PPG signal, ACC signal, and IR signal in the time domain from the time domain to the frequency domain to obtain a PPG frequency domain signal and an ACC frequency domain signal in the frequency domain. and IR frequency domain signals; for each frame of PPG frequency domain signal, extract the spectral features in the PPG frequency domain signal; for each frame of ACC frequency domain signal, extract the spectral features of the ACC frequency domain signal on the x-axis, y-axis and z-axis respectively; for each frame of IR frequency domain signal, extract the spectral features in the IR frequency domain signal; input the spectral features of the PPG signal, the spectral features in the IR frequency domain signal and the spectral features of the ACC frequency domain signal on the x-axis, y-axis and z-axis into the PPG spectrum denoising model; use the output result of the PPG spectrum denoising model as the denoised PPG frequency domain signal.

[0012] Therefore, by converting the PPG signals, ACC signals and IR signals collected by the wearable device from time domain to frequency domain, the spectral characteristics of different signals can be obtained, and then the PPG frequency domain signals can be denoised based on the PPG spectrum denoising model.

[0013] According to the first aspect, or any implementation manner of the first aspect above, the PPG signal, ACC signal, and IR signal acquired in the time domain are converted from the time domain to the frequency domain, including: performing windowing, slicing, and framing processing on the PPG signal, ACC signal, and IR signal acquired in the time domain to obtain a continuous framed signal, where the framed signal includes the PPG signal, ACC signal, and IR signal in the time domain; for each frame of the framed signal, converting the PPG signal, ACC signal, and IR signal in the time domain included in the framed signal from the time domain to the frequency domain.

[0014] Therefore, by performing windowing, slicing and framing processing on the PPG signal, ACC signal and IR signal collected by the wearable device, each frame signal includes the PPG signal, ACC signal and IR signal, thereby ensuring that the PPG frequency domain signal can be denoised based on the spectral characteristics of the ACC signal and the spectral characteristics of the IR signal in each frame, and further ensuring that the spectral characteristics of the denoised PPG frequency domain signal can be closer to the spectral characteristics of the PPG frequency domain signal under ideal conditions (noise-free).

[0015] According to the first aspect, or any implementation of the first aspect above, the PPG signal, ACC signal, and IR signal acquired in the time domain are windowed, sliced, and framed, including: using a window size of 8 seconds and a window movement step of 1 second to set the windowed, slicing, and framing PPG signal, ACC signal, and IR signal acquired in the time domain.

[0016] According to the first aspect, or any implementation of the first aspect above, the device type and resource configuration of the wearable device that collects the PPG sample signal, the ACC sample signal, and the IR sample signal are the same as those of the wearable device that collects the PPG signal, the ACC signal, and the IR signal.

[0017] Therefore, by ensuring that the source of sample data used in training the PPG spectrum denoising model is the same as the source of data in the actual application scenario, the accuracy of the denoising results can be better guaranteed.

[0018] According to the first aspect, or any implementation of the first aspect above, the PPG spectrum gold standard is determined based on the heart rate gold standard collected by the heart rate tester; wherein, the heart rate gold standard is the heart rate with absolute accuracy collected by the heart rate tester when the wearable device collects the PPG sample signal, the ACC sample signal and the IR sample signal.

[0019] According to the first aspect, or any implementation of the first aspect above, the PPG spectrum gold label is obtained by processing the heart rate gold label using a filtering construction method.

[0020] According to the first aspect, or any implementation of the first aspect above, the PPG spectrum gold label is obtained by processing the heart rate gold label using a heuristic construction method.

[0021] According to the first aspect, or any implementation of the first aspect above, the PPG spectrum denoising model is obtained based on training of a transformer encoder model.

[0022] According to the first aspect, or any implementation method of the first aspect above, the PPG spectrum denoising model is obtained based on the training of two transformer encoder models; wherein, the input of the first transformer encoder model is the spectral features in each frame of the PPG sample signal, the spectral features in the ACC sample signal, and the spectral features in the IR sample signal, which are used to establish a mapping relationship between multi-channel features; wherein, the input of the second transformer encoder model is the spectral features after feature averaging of the output of the input of the first transformer encoder model, which is used to establish a temporal mapping relationship between multi-frame features.

[0023] According to the first aspect, or any implementation of the first aspect above, after obtaining the denoised PPG frequency domain signal, the method further includes: predicting the heart rate of the user wearing the wearable device based on the denoised PPG frequency domain signal.

[0024] Therefore, by denoising the PPG frequency domain signal based on the PPG spectrum denoising model, the spectral characteristics of the PPG frequency domain signal can be restored to the ideal state, and accurate measurement based on the PPG signal can be performed even in motion.

[0025] In a second aspect, the present application provides an electronic device. The electronic device includes: a memory and a processor, the memory and the processor being coupled; the memory storing program instructions, which, when executed by the processor, cause the electronic device to execute instructions of the method of the first aspect or any possible implementation of the first aspect.

[0026] Among them, in some possible implementations, the electronic device can be a wearable device, such as a smart watch, a smart bracelet, a smart ring, etc.

[0027] In some other possible implementations, the electronic device may be a device with powerful processing capabilities, such as a PC.

[0028] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0029] In a third aspect, the present application provides a computer-readable medium for storing a computer program, wherein the computer program includes instructions for executing the method in the first aspect or any possible implementation of the first aspect.

[0030] The third aspect and any implementation of the third aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the third aspect and any implementation of the third aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0031] In a fourth aspect, the present application provides a computer program comprising instructions for executing the method in the first aspect or any possible implementation of the first aspect.

[0032] The fourth aspect and any implementation of the fourth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fourth aspect and any implementation of the fourth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0033] In a fifth aspect, the present application provides a chip comprising a processing circuit and a transceiver pin. The transceiver pin and the processing circuit communicate with each other via an internal connection path, and the processing circuit executes the method of the first aspect or any possible implementation of the first aspect to control the receive pin to receive a signal and to control the transmit pin to send a signal.

[0034] The fifth aspect and any implementation of the fifth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fifth aspect and any implementation of the fifth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] FIG1A and FIG1B are schematic diagrams illustrating exemplary scenarios of using a wearable device to measure heart rate;

[0036] FIG2 is a schematic diagram showing an exemplary relationship between a noise-free PPG signal and heart rate;

[0037] FIG3A and FIG3B are schematic diagrams showing exemplary relationships between noisy PPG signals and heart rate;

[0038] FIG4A is a schematic diagram illustrating an exemplary relationship between the spectral characteristics of a PPG signal and the spectral characteristics of an ACC signal in a periodic motion state;

[0039] FIG4B is a schematic diagram illustrating the relationship between the spectral characteristics of a PPG signal and the spectral characteristics of an IR signal in a non-periodic motion state;

[0040] FIG5A and FIG5B are schematic diagrams illustrating spectral characteristics of a denoised PPG signal obtained by denoising the spectral characteristics of a noisy PPG signal using the denoising method for the PPG frequency domain signal provided in an embodiment of the present application;

[0041] FIG6 is a schematic diagram illustrating exemplary steps involved in implementing the method for denoising a PPG frequency domain signal provided in an embodiment of the present application;

[0042] FIG7A is a schematic diagram illustrating an exemplary PPG spectrum denoising model used in the PPG frequency domain signal denoising method provided in an embodiment of the present application based on a deep learning algorithm;

[0043] FIG7B is a schematic diagram illustrating another exemplary PPG spectrum denoising model used in the PPG frequency domain signal denoising method provided in an embodiment of the present application based on a deep learning algorithm;

[0044] FIG8 is a schematic flow chart illustrating a method for denoising a PPG frequency domain signal according to an embodiment of the present application;

[0045] FIG9 is a schematic diagram illustrating an exemplary software structure and hardware of a wearable device;

[0046] FIG10 is a schematic diagram showing the hardware structure of an exemplary wearable device. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0048] The term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0049] In the description and claims of the embodiments of this application, the terms "first" and "second" are used to distinguish different objects, rather than to describe a specific order of objects. For example, the terms "first target object" and "second target object" are used to distinguish different objects, rather than to describe a specific order of objects.

[0050] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0051] In the description of the embodiments of this application, unless otherwise specified, "multiple" means two or more. For example, "multiple processing units" means two or more processing units; "multiple systems" means two or more systems.

[0052] Heart rate is a key indicator of health. With the recent development and maturity of wearable devices such as smartwatches and smart bracelets, heart rate, blood oxygen concentration, and blood pressure algorithms, based on wearable sensor information, can accurately monitor a person's heart rate, blood oxygen concentration, and blood pressure in real time, thereby reflecting the user's physical health. This technology has a wide range of applications.

[0053] At present, in order to achieve accurate real-time monitoring of human heart rate, blood oxygen concentration, blood pressure, etc., wearable devices (taking smart watches as an example) usually set a PPG sensor on the side that contacts the user's skin, such as the back side as shown in Figure 1A. Based on the information collected by the PPG sensor, the heart rate algorithm, blood oxygen concentration algorithm, blood pressure algorithm, etc. are used to accurately monitor the human heart rate, blood oxygen concentration, blood pressure, etc. in real time.

[0054] The PPG sensor includes a light emitting diode (LED) and a receiver. For ease of explanation, let's take measuring heart rate as an example. Referring to FIG1B , for example, after a user wears the smartwatch including the PPG sensor shown in FIG1A on their arm and turns on the heart rate measurement function, in one possible implementation, in order to improve the user experience, a heart rate measurement picture and a text message "measuring" can be displayed on the user interface of the smartwatch, as shown in (1) in FIG1B .

[0055] For example, when the user interface of the smart watch is in the style shown in (1) in Figure 1B, the LED in the PPG sensor in the smart watch will continuously project light onto the skin, and the light will be absorbed by the blood through the skin tissue, while the receiver will receive the reflected light signal (hereinafter referred to as the PPG signal). Since the intensity of the PPG signal is related to the blood flow velocity, and the blood flow velocity is affected by the periodic heart rhythm. Therefore, by analyzing the spectral characteristics of the PPG signal in the frequency domain (which can be expressed as PPG frequency domain signal or PPG spectrum characteristics), the user's heart rate can be determined. Finally, the measured heart rate can be displayed on the user interface of the smart watch, as shown in (2) in Figure 1B.

[0056] It should be noted that, under ideal circumstances, PPG signals, like electrocardiogram (ECG) signals, can accurately detect the heart rate value of the human body at all times. However, due to the influence of noise from complex scenarios such as hardware, temperature, and motion, the real signal of PPG is distorted, which greatly restricts the accuracy of heart rate calculation. Since PPG frequency domain signals are particularly sensitive to motion, the biggest impact is motion artifacts. Motion artifacts will cause the peaks and valleys in the PPG time domain features to be missing or deformed, reducing the accuracy of the time domain counting method, and in the frequency domain they will appear as abnormal spectral peaks in the PPG spectrum features, making it impossible to accurately locate the main frequency of the PPG frequency domain signal at the current moment (corresponding to the true heart rate), resulting in inaccurate prediction results.

[0057] For ease of explanation, this embodiment expresses the PGG time domain features and PGG frequency domain features under ideal conditions as noise-free PPG time domain features and noise-free PPG frequency domain features; and expresses the PGG time domain features and PGG frequency domain features that are interfered by motion artifacts and have a large number of noise frequency peaks as noisy PPG time domain features and noisy PPG frequency domain features.

[0058] For example, for the noise-free PPG time domain feature, in one possible implementation, the waveform of the received signal strength of the PPG signal changing with time is shown in (1) in Figure 2, where the horizontal axis is time and the vertical axis is the received signal strength.

[0059] As can be seen from (1) in Figure 2, the noise-free PPG time domain features basically do not have any missing or deformed peaks or troughs. Therefore, after converting the PPG signal in the time domain shown in (1) in Figure 2 to the frequency domain, the noise-free PPG frequency domain features shown in (2) in Figure 2 can be obtained, where the horizontal axis is the frequency point and the vertical axis is the spectrum intensity.

[0060] As shown in Figure 2 (2), for example, for a noise-free PPG frequency domain feature, the dominant frequency is clear and there is no abnormal spectral peak interference. Therefore, the user's current heart rate can be directly determined based on the dominant frequency of the PPG frequency domain signal (the dotted line position shown in Figure 2 (2)).

[0061] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation on this embodiment.

[0062] For example, for the noisy PPG time domain features, in one possible implementation, the waveform of the received signal strength of the PPG signal changing with time is shown in (1) in FIG3A , or (1) in FIG3B , where the horizontal axis is time and the vertical axis is received signal strength.

[0063] It can be seen from (1) in Figure 3A and (1) in Figure 3B that the peaks and troughs of the noisy PPG time domain features are seriously missing or deformed. Therefore, after converting the PPG signals in the time domain shown in (1) in Figure 3A and (1) in Figure 3B from the time domain to the frequency domain, the noisy PPG frequency domain features shown in (2) in 3A and (2) in Figure 3B will be obtained, where the horizontal axis is the frequency point and the vertical axis is the spectrum intensity.

[0064] Referring to (2) in FIG. 3A and (2) in FIG. 3B , for example, for the noisy PPG frequency domain features, the main frequency is unclear and there is abnormal spectrum peak interference.

[0065] For example, in one possible implementation, for example, the PPG spectrum intensity corresponding to the user's actual heart rate is the spectrum intensity corresponding to the frequency points between 10 and 15 shown in (2) in FIG3A . However, due to abnormal spectrum peaks, such as interference from the spectrum intensity corresponding to the frequency point 5, when the maximum value method is adopted, that is, the spectrum intensity corresponding to the maximum frequency peak is selected as the basis for calculating the heart rate, the spectrum intensity corresponding to the frequency point 5 will be mistakenly used as the basis for calculating the heart rate, thereby causing the measured heart rate to be an erroneous heart rate.

[0066] Exemplarily, in another possible implementation, for example, the PPG spectrum intensity corresponding to the user's actual heart rate is shown in (2) in FIG. 3B .

[0067] As shown in (2) in FIG3A , the spectrum intensity corresponding to the frequency point between 10 and 15 is shown. However, due to the interference of noise, there is no spectrum peak at this frequency point. This will cause the abnormal spectrum peak, such as the one corresponding to the frequency point 5, to be mistakenly identified as the main frequency, thereby causing the measured heart rate to be an erroneous heart rate.

[0068] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation on this embodiment.

[0069] From the above description, it can be seen that the noise-free PPG frequency domain signal can ensure the accuracy of the heart rate measurement results. Therefore, how to remove the noise in the PPG frequency domain signal so that the spectral characteristics of the PPG frequency domain signal can meet the noise-free PPG spectral characteristics has become an urgent problem to be solved.

[0070] For example, in some possible implementations, for a user wearing a wearable device with an integrated PPG sensor, such as a smart watch, when performing periodic movements, such as periodically swinging the arm wearing the smart watch while running, the spectrum characteristics of the collected PPG signal in the frequency domain may be as shown in (1) in Figure 4A.

[0071] For example, if the frequency point and spectrum intensity corresponding to the user's current true heart rate are the frequency point and spectrum intensity corresponding to the dotted line shown in (1) in FIG4A, but due to the interference of noise during exercise, an abnormal spectrum peak exists at frequency point 5. Since the spectrum characteristics of the ACC signal collected by the acceleration transducer (ACC) and the spectrum characteristics of the noise signal basically show the same change trend during periodic exercise, that is, the spectrum characteristics of the ACC signal can reflect the spectrum characteristics of the noise signal. Therefore, a spectrum subtraction method can be used to subtract the spectrum characteristics of the ACC signal collected by the acceleration sensor in the same period and the same state, that is, in the process of collecting the PPG signal, from the noisy PPG frequency domain characteristics (noisy spectrum) obtained during the periodic exercise shown in (1) in FIG4A (the ACC spectrum characteristics shown in (2) in FIG4A, which can also be called the noise spectrum), so that the PPG frequency domain signal can be denoised, and then the frequency point and PPG spectrum intensity of the true heart rate can be accurately determined and calculated based on the denoised PPG frequency domain signal, thereby improving the accuracy of the heart rate measurement result.

[0072] It should be noted that the spectrum used in the spectrum subtraction method can be an amplitude spectrum or a power spectrum, and this application does not impose any limitation on this.

[0073] For example, in some other possible implementations, for a user wearing a wearable device with an integrated PPG sensor, such as a smart watch, when performing non-periodic movements, such as non-periodic swinging of the arm wearing the smart watch while walking, the spectrum characteristics of the collected PPG signal in the frequency domain may be as shown in (1) in Figure 4B.

[0074] For example, if the frequency point and spectrum intensity corresponding to the user's current true heart rate are the frequency point and spectrum intensity corresponding to the dotted line shown in (1) in Figure 4B, but due to the interference of noise during exercise, there are abnormal spectrum peaks at frequencies 0 to 10. Since the spectrum characteristics of the ACC signal cannot reflect the spectrum characteristics of the noise signal during non-periodic exercise. Therefore, the denoising of the PPG frequency domain signal during non-periodic exercise cannot be achieved with the ACC signal. However, the spectrum characteristics of the infrared sensor (IR) in the same period and the same state, that is, in the process of collecting PPG signals, the spectrum characteristics of the IR signal and the spectrum characteristics of the noise signal basically show the same change trend, that is, the spectrum characteristics of the IR signal can reflect the spectrum characteristics of the noise signal, as shown in (2) in Figure 4B. Therefore, the PPG frequency domain signal can be denoised by notching, that is, filtering, and then the frequency point and PPG spectrum intensity of the true heart rate can be accurately determined and calculated based on the denoised PPG frequency domain signal, thereby improving the accuracy of the heart rate measurement results.

[0075] In view of this, an embodiment of the present application provides a denoising method for PPG frequency domain signals. By using a transformer encoder model, ACC signals and IR signals are used as denoising reference signals, and a data-driven approach is adopted to learn and construct a PPG spectrum denoising model. In actual applications, the constructed PPG spectrum denoising model is used to denoise the PPG frequency domain signals through the collected ACC signals and IR signals, so that the heart rate can be accurately measured based on the PPG signal even in a state of exercise.

[0076] Based on the PPG frequency domain signal denoising method provided in the embodiments of the present application, different noisy PPG frequency domain signals can be denoised. For example, the noisy PPG spectrum feature shown in (1) of FIG. 5A can be processed into the denoised PPG spectrum feature shown in (2) of FIG. 5A . For another example, the noisy PPG spectrum feature shown in (1) of FIG. 5B can be processed into the denoised PPG spectrum feature shown in (2) of FIG. 5B .

[0077] In this way, based on the denoised PPG spectrum features, the user's true heart rate can be accurately determined.

[0078] The PPG frequency domain signal denoising method provided in the embodiments of the present application can be divided into two stages during implementation: the construction of a PPG spectrum denoising model and the application of the PPG spectrum denoising model. The construction of the PPG spectrum denoising model may include the data acquisition step, the spectrum feature gold standard construction step, and the spectrum reconstruction step shown in FIG6 ; the application of the PPG denoised spectrum may include the heart rate prediction step shown in FIG6 .

[0079] 6 , illustratively, the data collection process includes the collection of sensor signals required during the training process and the collection of heart rate gold standards for constructing supervisory signals during the training process.

[0080] Since the constructed PPG spectrum denoising model is applied to wearable devices to achieve heart rate prediction, the sensor signals required during the training process are specifically collected by the wearable devices.

[0081] Specifically, in the technical solution provided in the embodiments of the present application, the sensor signals collected by the wearable device include PPG signals collected by the PPG sensor, ACC signals collected by the ACC sensor, and IR signals collected by the IR sensor in different states (periodic motion, non-periodic motion, stillness, etc.).

[0082] Furthermore, it should be noted that in order to ensure that the constructed PPG spectrum denoising model can accurately remove noise from the PPG frequency domain signal, and thus enable the wearable device to accurately determine the user's heart rate based on the denoised PPG spectrum characteristics, professional equipment such as a medical-grade heart rate monitor can be used to collect the heart rate gold standard for the supervisory signal during the training process.

[0083] Understandably, in order to ensure the availability of the heart rate gold standard during the construction phase of the entire PPG spectrum denoising model, it is necessary to ensure that the heart rate is collected using a hospital-grade heart rate tester, and then the operation of obtaining the heart rate gold standard is carried out simultaneously with the operation of collecting sensor signals using wearable devices.

[0084] In addition, it should be noted that the so-called heart rate gold standard refers to the heart rate collected by a medical-grade heart rate tester, which is used to construct the parameter signal with absolute accuracy.

[0085] This concludes the introduction to the data collection process. For technical details not detailed in the data collection process, please refer to the existing sensor signal acquisition and the implementation plan for heart rate acquisition using a medical-grade heart rate meter. I will not go into details here.

[0086] Continuing to refer to FIG6 , illustratively, the operation performed in the spectral feature golden mark construction step is specifically to construct the PPG spectral feature using the heart rate golden mark collected in the data collection step, and then use the constructed spectral feature as the reference signal used in the PPG spectrum denoising model training stage (hereinafter referred to as the PPG spectrum golden mark).

[0087] It should be noted that in some implementations, the PPG spectrum gold label mentioned in this embodiment can also be understood as a heart rate label, that is, the real heart rate value used when constructing a PPG spectrum denoising model and evaluating the model performance through the mean squared error loss function.

[0088] For example, in some possible implementations, the PPG spectrum gold label may be constructed by filtering.

[0089] For example, in some other possible implementations, the PPG spectrum gold label may be constructed in a heuristic construction manner.

[0090] In order to facilitate the description of the two methods of constructing PPG spectrum gold labels, the final constructed PPG spectrum gold label in this embodiment includes the spectrum energy of 256 frequency points, that is, G∈R 256 For example, in the stage of constructing the model, G i Represents the spectrum energy of the i-th frequency point in the spectrum.

[0091] Based on the above premise, when constructing the PPG spectrum gold standard, we can first calculate the frequency f corresponding to each frequency point. p , and the frequency index hr corresponding to the frequency i .

[0092] After completing the above preparations, the PPG spectrum gold label can be constructed through filtering construction or heuristic construction.

[0093] Exemplarily, the method of constructing a PPG spectrum gold label by using a filtering construction method may include the following steps:

[0094] First, a frequency threshold σ is set according to business needs.

[0095] Then, the PPG signal in the time domain is filtered by three sets of bandpass filters.

[0096] For ease of description, these three groups of bandpass filters can be expressed as heart rate main frequency filter (also known as: first harmonic filter), second harmonic filter, and third harmonic filter. Among them, the frequency band range corresponding to the heart rate main frequency filter is {f p -σ,f p +σ}, the frequency band corresponding to the second harmonic filter is {2*f p -σ, 2*f p +σ}, the frequency band corresponding to the third harmonic filter is {3*f p -σ,3*f p +σ}.

[0097] Next, the PPG signal in the time domain is filtered using the three sets of filters mentioned above. The three filtered PPG signals are then summed. Finally, the summed PPG signal is converted from the time domain to the frequency domain based on the fast Fourier transform (FFT) algorithm. That is, based on the frequency-based feature extraction method in the FFT algorithm, the spectral characteristics of the PPG signal in the frequency domain can be obtained. The spectral characteristics of the PPG signal in the frequency domain obtained at this stage are the PPG spectrum gold standard mentioned above.

[0098] For example, for a PPG spectrum gold label constructed using a heuristic construction method, the following formula may be satisfied:

[0099] This concludes the introduction to the construction of the spectral feature golden label. For technical details not detailed in the construction of the spectral feature golden label, please refer to the existing descriptions of the filtering construction method and the heuristic construction method, which will not be repeated here.

[0100] Continuing with Figure 6, illustratively, in the spectrum reconstruction phase, a deep learning-based approach is used, with the constructed spectrum features serving as supervisory signals (as PPG spectrum gold standards), to train the PPG spectrum denoising model using a data-driven approach. In other words, the spectrum reconstruction phase involves training and constructing the PPG spectrum denoising model.

[0101] Regarding the specific implementation logic of the spectrum reconstruction link, in a possible implementation manner, it can be shown in FIG7A .

[0102] 7A , illustratively, in the spectrum reconstruction step, signal preprocessing and feature extraction are first performed on the PPG signal, ACC signal, and IR signal (which can be referred to as a timing signal) in the time domain collected by the PPG sensor, ACC sensor, and IR sensor in the wearable device.

[0103] Specifically, in the signal preprocessing stage, the three time series signals of PPG, ACC and IR are firstly subjected to windowing, slicing and framing processing in the same manner.

[0104] For example, in some possible implementations, the window size may be set to 8s, the window moving step may be set to 1s, and then based on this setting, framing is performed (one operation is one frame) to obtain a continuous frame signal.

[0105] Then, after completing the windowing, slicing and framing processing of the above three time series signals, the frequency domain characteristics of the PPG signal (subsequently expressed as PPG spectrum characteristics), the frequency domain characteristics of the ACC signal (subsequently expressed as ACC spectrum characteristics), and the frequency domain characteristics of the IR signal (subsequently expressed as IR spectrum characteristics) in each frame signal obtained by the slicing and framing processing can be extracted through the FFT algorithm, that is, the time domain to frequency domain conversion of the above three types of signals can be realized.

[0106] It should be understood that the ACC signal collected by the acceleration sensor includes three dimensions: x-axis, y-axis, and z-axis features. Therefore, the extracted ACC spectrum features include those in the x-axis, y-axis, and z-axis directions. For ease of distinction, this embodiment denotes the ACC spectrum features in the x-axis direction as ACCx spectrum, the ACC spectrum features in the y-axis direction as ACCy spectrum, and the ACC spectrum features in the z-axis direction as ACCz spectrum.

[0107] In addition, it should be understood that since the PPG frequency domain signal has not yet been denoised at this stage, the extracted PPG spectrum features actually contain noise. For ease of description, this embodiment represents it as a noisy PPG spectrum.

[0108] Correspondingly, for the IR spectrum feature, it is expressed as IR spectrum.

[0109] That is, the feature information extracted from each frame signal includes the frequency domain features of the five dimensions shown in FIG7A , namely, the noisy PPG spectrum, the ACCx spectrum, the ACCy spectrum, the ACCz spectrum, and the IR spectrum.

[0110] Continuing to refer to FIG7A , for example, after obtaining the noisy PPG spectrum, ACCx spectrum, ACCy spectrum, ACCz spectrum and IR spectrum corresponding to each frame signal, the frequency domain features of these five dimensions at the same moment can be input into the transformer encoder model, and the mapping relationship between the spectral features of the above five dimensions can be established with the help of the encoding part of the transformer encoder model.

[0111] For example, still taking the PPG spectrum gold standard with 256 frequency points as an example, the ACCx spectrum, ACCy spectrum, ACCz spectrum, and IR spectrum also reflect the spectral features of 256 frequency points. Based on this, the five-dimensional frequency domain features of the noisy PPG spectrum, ACCx spectrum, ACCy spectrum, ACCz spectrum, and IR spectrum extracted from each frame signal are input into the transformer encoder model. With the help of the encoding part of the transformer encoder model, the mapping relationship between the noisy PPG spectrum, ACCx spectrum, ACCy spectrum, ACCz spectrum, and IR spectrum is established. The result output by the transformer encoder model can be regarded as a 5×256-dimensional matrix, that is, the output feature dimension is 5×256.

[0112] Continuing with FIG7A , for example, to reduce computational complexity and mitigate the risk of overfitting, the output of the transformer encoder model can be feature averaged. In this embodiment, the 5×256 feature dimensions are specifically averaged to 256 feature dimensions.

[0113] Continuing to refer to FIG. 7A , illustratively, after the output features are averaged, the averaged spectral features with a feature dimension of 256 may be input into a multilayer perceptron for processing.

[0114] It should be noted that the multilayer perceptron used in this embodiment is a feedforward artificial neural network model. Through processing by the multilayer perceptron, multiple input data can be mapped to a single data set. Specifically, the spectral features of the noisy PPG spectrum, ACCx spectrum, ACCy spectrum, ACCz spectrum, and IR spectrum are mapped to the PPG spectrum features. The features output by the multilayer perceptron are the predicted PPG spectrum (hereinafter referred to as the PPG predicted spectrum).

[0115] In order to measure the difference between the model's PPG predicted spectrum and the true label, that is, the PPG spectrum gold standard, the PPG predicted spectrum corresponding to each frame and the PPG spectrum gold standard corresponding to each frame can be subjected to loss processing based on the mean square error loss (Mse loss) function to determine the prediction error of the current model.

[0116] Finally, through continuous iterative training, when the prediction error meets the preset convergence condition (the condition for stopping training, that is, not greater than the set threshold), the current model can be used as the PPG spectrum denoising model.

[0117] Thus, the construction of the PPG spectrum denoising model is completed, that is, the spectrum reconstruction is completed.

[0118] Regarding the specific implementation logic of the spectrum reconstruction link, in another possible implementation manner, it can be shown in FIG7B .

[0119] Referring to FIG. 7B , for example, the acquisition of the framed signal and the type of spectrum signal extracted from each frame of the framed signal in the spectrum reconstruction link are the same as those shown in FIG. 7A . For specific implementation details, please refer to the description of the embodiment shown in FIG. 7A , which will not be repeated here.

[0120] Continuing with FIG. 7B , for example, in this embodiment, in order to utilize the signal temporal characteristics of the five-dimensional spectral features of the noisy PPG spectrum, ACCx spectrum, ACCy spectrum, ACCz spectrum, and IR spectrum to better construct a PPG spectrum denoising model, the PPG spectrum denoising model can be constructed using two Transformer encoder models.

[0121] Specifically, one of the two Transformer encoder models (hereinafter referred to as Transformer encoder_1) is used to extract channel validity features by establishing a mapping relationship between multi-channel features, while the other Transformer encoder model (hereinafter referred to as Transformer encoder_2) is used to extract time dimension features by establishing a temporal mapping relationship between multi-frame features.

[0122] Based on this, referring to Figure 7B , for example, the noisy PPG spectrum, ACCx spectrum, ACCy spectrum, ACCz spectrum, and IR spectrum extracted from each frame signal can be input into the corresponding Transformer encoder_1, and then a mapping relationship between multi-channel features is established based on the encoding part of Transformer encoder_1. Finally, channel validity features are extracted based on this mapping relationship. In this way, a spectral feature with a feature dimension of 5×256 corresponding to each frame signal can be obtained.

[0123] Assuming the number of frames required for training is K, referring to Figure 7B, illustratively, after all Transformer encoder_1 processes, the output spectral features total K 5×256 dimensions. To reduce computational complexity and mitigate the risk of overfitting, the output of transformer encoder_1 can be feature averaged. In this embodiment, the K 5×256 feature dimensions are averaged to K×256 feature dimensions.

[0124] Continuing with Figure 7B, for example, after obtaining the spectral features of K×256 feature dimensions, these spectral features of K×256 feature dimensions are input into transformer encoder_2. A temporal mapping relationship between multi-frame features is then established based on the encoding portion of transformer encoder_2. Finally, the time dimension features are extracted based on this mapping relationship. Thus, a spectral feature corresponding to each frame signal can be obtained that reflects both channel characteristics and time characteristics (the feature dimension remains K×256).

[0125] Continuing with FIG7B , for example, to further reduce computational complexity and mitigate the risk of overfitting, feature averaging can be performed on the output of transformer encode_2. In this embodiment, the feature dimensions of K×256 are averaged to 256 feature dimensions.

[0126] Continuing to refer to FIG7B , for example, after the output feature averaging processing is performed on the output result of transformer encode_2, the averaged spectral features with a feature dimension of 256 (which reflect both the channel characteristics and the time characteristics) can be input into the multi-layer perceptron for processing, thereby obtaining the PPG predicted spectrum of the PPG signal of this short time series containing K frames.

[0127] Continuing to refer to FIG7B , illustratively, in order to measure the difference between the model's predicted PPG spectrum and the true label, i.e., the PPG spectrum gold standard, this embodiment also performs loss processing on the PPG predicted spectrum of the short-time series PPG signal and the corresponding PPG spectrum gold standard based on the mean square error loss (Mse loss) function, thereby determining the prediction error of the current model.

[0128] Finally, through continuous iterative training, when the prediction error meets the preset convergence condition (the condition for stopping training, that is, not greater than the set threshold), the current model can be used as the PPG spectrum denoising model.

[0129] Thus, the construction of the PPG spectrum denoising model is completed, that is, the spectrum reconstruction is completed.

[0130] This concludes the introduction to the spectrum reconstruction process. In specific implementation, the method shown in FIG. 7A or FIG. 7B can be selected according to business needs to construct a PPG spectrum denoising model. This application does not impose any restrictions on this.

[0131] Therefore, through the processing of data acquisition, spectrum feature gold standard construction and spectrum reconstruction, the construction of PPG spectrum denoising model can be realized.

[0132] Understandably, since the construction phase of the PPG spectrum denoising model requires large computing power and resource consumption, the construction of the PPG spectrum denoising model can be completed in electronic devices such as PCs.

[0133] Continuing with FIG6 , illustratively, after obtaining the PPG spectrum denoising model, the PPG spectrum denoising model can be transmitted to the wearable device. In this way, the user can predict the heart rate based on the wearable device.

[0134] Specifically, when using the PPG spectrum denoising model for heart rate prediction, the wearable device can perform signal preprocessing and feature extraction on the collected PPG signals, ACC signals and IR signals, and then input the extracted noisy PPG spectrum, ACCx spectrum, ACCy spectrum, ACCz spectrum and IR spectrum into the trained PPG spectrum denoising model to obtain the denoised PPG spectrum features, and then perform heart rate prediction based on the denoised PPG spectrum features.

[0135] Based on the description of the PPG spectrum denoising model construction stage and the PPG spectrum denoising model application stage shown in FIG6 , the implementation process of the PPG frequency domain signal denoising method provided in an embodiment of the present application can be shown in FIG8 .

[0136] 8 , the denoising method for PPG frequency domain signals provided in an embodiment of the present application may include:

[0137] S101, obtaining PPG signals, ACC signals, and IR signals collected by the wearable device.

[0138] Among them, the PPG signal, ACC signal and IR signal are PPG signals, ACC signals and IR signals in the time domain.

[0139] For example, the PPG signal in the time domain is collected by a PPG sensor that can be integrated into a wearable device. Regarding the location of the PPG sensor, for example, it can be located on the side that contacts the user's skin, such as the back of the smart watch shown in Figure 1A.

[0140] For example, the ACC signal in the time domain is collected by an ACC sensor that can be integrated into a wearable device.

[0141] For example, the IR signal in the time domain is collected by an IR sensor that can be integrated into a wearable device.

[0142] In addition, it should be noted that the PPG signal, ACC signal, and IR signal collected by the wearable device are collected by the wearable device in the same state and the same period.

[0143] In this way, by constraining the denoising reference signal selected when denoising the PPG frequency domain signal, that is, the ACC signal and the IR signal are collected in the same period as the PPG signal and when the wearable device is in the same state, the effectiveness of the denoising result of the PPG frequency domain signal is guaranteed, and the denoised PPG frequency domain signal is guaranteed to meet the spectrum characteristics under ideal conditions (noise-free), making the heart rate predicted based on the PPG spectrum characteristics more accurate.

[0144] S102 , performing denoising processing on the spectral features of the PPG signal according to the PPG spectrum denoising model, the spectral features in the ACC signal, and the spectral features in the IR signal to obtain a denoised PPG frequency domain signal.

[0145] Specifically, the PPG spectrum denoising model mentioned in this embodiment is the model obtained by training the transformer encoder model, which takes the spectral features in the PPG sample signal, the spectral features in the ACC sample signal, and the spectral features in the IR sample signal as input, and the PPG spectrum gold mark as the supervision signal.

[0146] Among them, the PPG sample signal, the ACC sample signal and the IR sample signal are, for example, the PPG signal, the ACC signal and the IR signal in the time domain collected by the PPG sensor, the ACC sensor and the IR sensor in the data collection link shown in Figure 6.

[0147] Accordingly, the spectral features in the PPG sample signal, the spectral features in the ACC sample signal, and the spectral features in the IR sample signal input into the transformer encoder model can be the noisy PPG spectrum, ACCx spectrum, ACCy spectrum, ACCz spectrum, and IR spectrum extracted from each frame signal in Figure 7A or Figure 7B.

[0148] In addition, it should be noted that in some possible implementations, the PPG spectrum denoising model can be obtained by training a transformer encoder model. The specific training method can be found in the description of the embodiment shown in FIG7A and will not be repeated here.

[0149] In addition, it should be noted that in some other possible implementations, the PPG spectrum denoising model is also obtained by training two transformer encoder models. Among them, the input of the first transformer encoder model is the spectral features in each frame of the PPG sample signal, the spectral features in the ACC sample signal, and the spectral features in the IR sample signal, which are used to establish a mapping relationship between multi-channel features; the input of the second transformer encoder model is the spectral features after feature averaging the output of the input of the first transformer encoder model, which is used to establish a temporal mapping relationship between multi-frame features.

[0150] It is understandable that the first transformer encoder model mentioned in this embodiment is, for example, transformer encoder_1 shown in FIG7B , and the second transformer encoder model is, for example, transformer encoder_2 shown in FIG7B .

[0151] Regarding the method of obtaining the PPG spectrum denoising model based on two transformer encoder model training, the specific training method can be found in the description of the embodiment shown in Figure 7B, which will not be repeated here.

[0152] In addition, the PPG spectrum gold standard mentioned in this embodiment is determined based on the heart rate gold standard collected by the heart rate monitor. The heart rate gold standard is the heart rate with absolute accuracy collected by the heart rate monitor when the wearable device collects PPG sample signals, ACC sample signals, and IR sample signals.

[0153] It should be noted that, in order to ensure the accuracy of the results, the heart rate meter used to collect the heart rate gold standard is, for example, a medical-grade heart rate meter.

[0154] For example, in some possible implementations, the PPG spectrum gold mark in this embodiment can be obtained by processing the heart rate gold mark using a filtering construction method. Regarding the implementation method of processing the heart rate gold mark using the filtering construction method to construct the PPG spectrum gold mark, please refer to the description of the filtering construction method in the spectral feature gold mark construction step in the embodiment shown in FIG6 , and will not be repeated here.

[0155] For example, in other possible implementations, the PPG spectrum golden label mentioned in this embodiment can be obtained by processing the heart rate golden label using a heuristic construction method. Regarding the implementation method of using the heuristic construction method to process the heart rate golden label and then construct the PPG spectrum golden label, please refer to the description of the heuristic construction method in the spectral feature golden label construction step in the embodiment shown in FIG6 , and will not be repeated here.

[0156] Furthermore, it should be noted that in this embodiment, the wearable device used to collect the PPG, ACC, and IR sample signals uses the same device type and resource configuration as the wearable device used to collect the PPG, ACC, and IR signals. This ensures that the sample data used to train the PPG spectrum denoising model originates from the same source as the data used in actual application scenarios, thereby further improving the accuracy of the denoising results.

[0157] Based on the above-mentioned limitations on the PPG sample signals, ACC sample signals, and IR sample signals used in the PPG spectrum denoising model training phase and the PPG signals, ACC signals, and IR signals used in the actual denoising phase, when denoising the spectral features of the PPG signal according to the PPG spectrum denoising model, the spectral features in the ACC signal, and the spectral features in the IR signal to obtain the denoised PPG frequency domain signal, the PPG signal, ACC signal, and IR signal acquired in the time domain can first be converted from the time domain to the frequency domain to obtain the PPG frequency domain signal, ACC frequency domain signal, and IR frequency domain signal in the frequency domain. Then, for each frame of the PPG frequency domain signal, the spectral features of the PPG frequency domain signal are extracted; for each frame of the ACC frequency domain signal, the spectral features of the ACC frequency domain signal on the x-axis, y-axis, and z-axis are extracted respectively; and for each frame of the IR frequency domain signal, the spectral features of the IR frequency domain signal are extracted. Finally, the spectral features of the PPG signal, the spectral features in the IR frequency domain signal, and the spectral features of the ACC frequency domain signal on the x-axis, y-axis, and z-axis are input into the PPG spectrum denoising model, and the output result of the PPG spectrum denoising model is used as the denoised PPG frequency domain signal.

[0158] In addition, it should be noted that in order to ensure that each frame signal for spectral feature extraction includes the PPG signal, ACC signal, and IR signal, when converting the PPG signal, ACC signal, and IR signal obtained in the time domain from the time domain to the frequency domain, the PPG signal, ACC signal, and IR signal obtained in the time domain can be windowed, sliced, and framed to obtain a continuous framed signal. Then, for each frame, the PPG signal, ACC signal, and IR signal in the time domain included in the framed signal is converted from the time domain to the frequency domain.

[0159] For example, in some possible implementations, the window size can be set to 8s, the window movement step can be set to 1s, and then based on the window setting, the PPG signal, ACC signal and IR signal acquired in the time domain are windowed, sliced ​​and framed.

[0160] Therefore, by performing windowing, slicing and framing processing on the PPG signal, ACC signal and IR signal collected by the wearable device, each frame signal includes the PPG signal, ACC signal and IR signal, thereby ensuring that the PPG frequency domain signal can be denoised based on the spectral characteristics of the ACC signal and the spectral characteristics of the IR signal in each frame, and further ensuring that the spectral characteristics of the denoised PPG frequency domain signal can be closer to the spectral characteristics of the PPG frequency domain signal under ideal conditions (noise-free).

[0161] It can be understood that, whether in the training stage of the PPG spectrum denoising model or in the actual denoising stage, the method of extracting spectral features from the PPG signal, ACC signal and IR signal in the time domain is the same.

[0162] Therefore, in the actual denoising stage, for each frame of PPG frequency domain signal, the extracted spectral features can be understood as the noisy PPG spectrum mentioned in the above embodiment; for each frame of ACC frequency domain signal, the spectral features extracted on the x-axis, y-axis and z-axis respectively can be understood as the ACCx spectrum, ACCy spectrum and ACCz spectrum mentioned in the above embodiment; for each frame of IR frequency domain signal, the extracted spectral features can be understood as the IR spectrum mentioned in the above embodiment.

[0163] Therefore, by using the transformer encoder model, taking the ACC signal and IR signal as denoising reference signals, a data-driven approach is adopted to learn and construct a PPG spectrum denoising model. Then, in practical applications, the constructed PPG spectrum denoising model is used to denoise the PPG frequency domain signal through the collected ACC signal and IR signal, thereby achieving denoising of the PPG frequency domain signal collected under periodic motion or non-periodic motion.

[0164] In addition, it should be noted that in some possible implementations, for example, when the wearable device has integrated the PPG spectrum denoising model mentioned in this embodiment, the above steps S101 and S102 can be implemented in the wearable device.

[0165] For example, in the scenario where the wearable device has integrated the PPG spectrum denoising model described in this embodiment, after the wearable device completes the denoising processing of the PPG frequency domain signal through the above steps S101 and S102 and realizes the heart rate prediction based on the spectrum characteristics of the denoised PPG frequency domain signal, the heart rate prediction result can be synchronized to an electronic device that establishes a communication connection with the wearable device, such as a heart rate health application on a mobile phone.

[0166] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation on this embodiment.

[0167] In addition, it should be noted that in other possible implementations, for example, when the PPG spectrum denoising model described in this embodiment is not integrated into the wearable device, the above steps S101 and S102 can be implemented by an electronic device that establishes a communication connection with the wearable device, such as a mobile phone.

[0168] For example, in a scenario where the wearable device does not integrate the PPG spectrum denoising model described in this embodiment, an electronic device that establishes a communication connection with the wearable device, such as a mobile phone, can synchronize the heart rate prediction result to the wearable device after completing the denoising processing of the PPG frequency domain signal through the above steps S101 and S102 and realizing the heart rate prediction based on the spectrum characteristics of the denoised PPG frequency domain signal.

[0169] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation on this embodiment.

[0170] Therefore, the PPG spectrum signal denoising method provided by the embodiments of the present application can quickly and accurately remove noise from the PPG spectrum characteristics, restoring the spectrum characteristics of the PPG frequency domain signal to an ideal state, thereby enabling accurate measurement based on the PPG signal even during exercise. In other words, even if the user wearing the wearable device is currently exercising, the accuracy of the heart rate prediction result can be guaranteed.

[0171] In addition, in order to better understand the technical solution provided in the embodiments of the present application, taking a wearable device as a smart watch as an example, based on the relationship between the software structure and hardware of the smart watch, the functional modules, hardware, and the interaction between the functional modules and hardware involved in implementing the denoising method for the PPG frequency domain signal provided in the embodiments of the present application are explained.

[0172] Before describing the software structure of a wearable device, the architectures that can be adopted by the software system of the wearable device are first described.

[0173] Specifically, in practical applications, the software system of wearable devices can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture.

[0174] In addition, it is understandable that the software systems used by the current mainstream wearable devices include but are not limited to Windows systems, Android systems and iOS systems. For the sake of convenience, the embodiment of the present application takes the layered architecture Android system as an example to illustrate the software structure of the wearable device.

[0175] In addition, it should be understood that the subsequent denoising method for PPG frequency domain signals provided in the embodiments of the present application is also applicable to other systems in specific implementations.

[0176] See Figure 9, which is a block diagram of the software structure and hardware structure of the wearable device according to an embodiment of the present application.

[0177] As shown in Figure 9, the layered architecture of wearable devices divides software into several layers, each with clear roles and divisions of labor. Layers communicate with each other through software interfaces. In some implementations, the Android system can be divided into five layers: from top to bottom, the application layer (Applications), the core layer (Application Framework, FWK), the runtime, and system libraries; and the underlying hardware abstraction layer (HAL) and Linux kernel layer.

[0178] The application layer may include a series of application packages. As shown in FIG9 , the application package may include applications such as camera, games, heart rate health, and settings. These applications are not listed here one by one and are not limited in this application.

[0179] Among them, the heart rate health application can be an application specially provided for enabling the heart rate measurement function.

[0180] It is understandable that in actual applications, the functions implemented by the heart rate health application can also be integrated into the sports health application that manages various sports information, or can also be integrated into the settings application. This application does not limit this.

[0181] The framework layer can provide application programming interfaces (APIs) and programming frameworks for applications in the application layer. In some implementations, these programming interfaces and programming frameworks can be described as functions. As shown in Figure 9, the framework layer can include functions such as content providers, window managers, view systems, and resource managers. These functions are not listed here and are not limited in this application.

[0182] It should be noted that the window manager located in the framework layer is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0183] Furthermore, it should be noted that the content provider located in the framework layer is used to store and retrieve data and make it accessible to applications. This data may include video, images, audio, dialed and received calls, browsing history and bookmarks, phone books, etc., which are not listed here and are not limited by this application.

[0184] In addition, it should be noted that the resource manager located in the framework layer is used to provide various resources for the application, such as localized strings, icons, pictures, layout files, video files, etc., which are not listed here one by one, and this application does not impose any restrictions on this.

[0185] In addition, it should be noted that the view system located in the framework layer includes visual controls, such as controls for displaying text and controls for displaying images. The view system can be used to build applications. A display interface can be composed of one or more views. For example, the display interface including a heart rate measurement image / icon shown in (1) and (2) in Figure 1B can include a view for displaying text and a view for displaying an image.

[0186] Continuing to refer to FIG9 , illustratively, the runtime, specifically the Android runtime (Android Runtime), may include a core library and a virtual machine, which are mainly responsible for scheduling and management of the Android system.

[0187] The core library consists of two parts: one containing the Java language's callable functions and the other containing the Android core library. The application layer and framework layer run in a virtual machine. The virtual machine executes the application and framework layer Java files as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0188] Continuing with FIG9 , illustratively, the system library may include multiple functional modules, such as a surface manager, a media library, a three-dimensional (3D) graphics processing library (e.g., OpenGL ES), a two-dimensional (2D) graphics engine (e.g., SGL), etc.

[0189] The surface manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. It supports a variety of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PnG. The 3D graphics processing library implements 3D graphics drawing, image rendering, compositing, and layer processing.

[0190] In addition, it is understandable that the 2D graphics engine mentioned above is a drawing engine for 2D drawing.

[0191] Continuing to refer to FIG9 , illustratively, the HAL layer is an interface layer located between the operating system kernel (kernel layer) and the hardware circuit. Its purpose is to isolate the FWK from the kernel so that Android does not become overly dependent on the kernel, thereby allowing the development of the FWK to be carried out without considering the driver.

[0192] Continuing to refer to FIG9 , illustratively, the HAL layer may include various interfaces, such as audio and video interfaces, GPS interfaces, call interfaces, WiFi interfaces, etc., which are not listed one by one here and are not limited in this application.

[0193] 9 , illustratively, the kernel layer in the Android system is the layer between hardware and software. The kernel layer may include various processes / threads, power management, various drivers, such as WiFi drivers, etc.

[0194] This concludes the introduction to the software structure of wearable devices. It should be understood that the layers in the software structure shown in FIG9 and the components contained in each layer do not constitute a specific limitation on the wearable device. In other embodiments of the present application, the wearable device may include more or fewer layers than shown, and each layer may include more or fewer components, and this application does not limit this.

[0195] Based on the software structure of the wearable device shown in Figure 9, when the user triggers a heart rate measurement operation through the heart rate health application / settings application installed in the application layer, the wearable device responds to the operation behavior, and the PPG sensor, ACC sensor, and IR sensor in the hardware part will synchronously collect PPG signals, ACC signals, and IR signals, and then hand them over to the processor. The processor will perform preprocessing and feature extraction on the sensor signals according to the program instructions stored in the internal memory, such as program instructions for preprocessing and feature extraction of the collected sensor signals, according to the set algorithm program, and then input the feature information of the extracted sensor signals into the PPG spectrum denoising model (program) stored in the memory for denoising, so that the denoised PPG frequency domain signal can be obtained. Finally, heart rate measurement can be achieved based on the denoised PPG frequency domain signal.

[0196] For the specific logic of the PPG frequency domain signal denoising method provided in the embodiment of the present application, please refer to Figures 6, 7A, 7B and 8, which will not be repeated here.

[0197] In order to better understand the technical solutions provided by the embodiments of the present application, still taking the wearable device as a smart watch as an example, refer to FIG10 , which specifically describes the specific structure of the smart watch and the devices involved in implementing the embodiments of the present application.

[0198] 10 , the wearable device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0199] Specifically, in the technical solution provided in the embodiments of the present application, in order to achieve denoising of the PPG frequency domain signal, the sensor module 180 needs to include a PPG sensor 180A, an ACC sensor 180B, and an IR sensor 180C. Among them, the PPG sensor can be used to collect PPG signals, the ACC sensor can be used to collect ACC signals, and the IR sensor can be used to collect IR signals.

[0200] Continuing with FIG10 , illustratively, since the PPG sensor 180A, the ACC sensor 180B, and the IR sensor 180C are all in communication with the processor 110, the PPG signals, ACC signals, and IR signals collected by the PPG sensor 180A, the ACC sensor 180B, and the IR sensor 180C are processed by the processor 110 according to the processing flow shown in the embodiment shown in FIG8 , thereby obtaining a denoised PPG frequency domain signal and achieving accurate heart rate measurement.

[0201] In addition, it should be noted that in actual applications, according to business needs and the scenarios in which the wearable device is applicable, the sensor 180 may also include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc., which are not listed one by one here, and this application does not impose any restrictions on this.

[0202] In addition, it should be noted that the processor (CPU) 110 may include one or more processing units, for example: the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc.

[0203] It is understandable that, in a specific implementation, different processing units may be independent devices or integrated into one or more processors.

[0204] In addition, it should be noted that the controller in the processor 110 can be the nerve center and command center of the wearable device 100. In actual applications, the controller can generate operation control signals based on instruction operation codes and timing signals to complete the control of instruction fetching and execution.

[0205] Furthermore, it should be noted that the processor 110 may also include a memory (internal memory) for storing instructions and data. In some implementations, the memory in the processor 110 is a cache memory. This memory can store recently used or recycled instructions or data read by the processor 110 from the internal memory 121. If the processor 110 needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves system efficiency.

[0206] Specifically, in the technical solution provided in the embodiment of the present application, the denoising processing of the PPG frequency domain signal can be implemented by AP110A in the processor 110.

[0207] In addition, it should be noted that the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0208] 10 , illustratively, the charging management module 140 is configured to receive charging input from a charger, which may be a wireless charger or a wired charger.

[0209] Continuing to refer to Figure 10, illustratively, the power management module 141 (i.e., the power management integrated circuit PMIC mentioned above) is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, battery health status (leakage, impedance), etc. In some other implementations, the power management module 141 can also be set in the processor 110. In other implementations, the power management module 141 and the charging management module 140 can also be set in the same device.

[0210] Continuing to refer to FIG. 10 , illustratively, the wireless communication function of the wearable device 100 can be implemented through antenna 1 , antenna 2 , mobile communication module 150 , wireless communication module 160 , a modem processor, and a baseband processor.

[0211] It should be noted that antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in wearable device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other implementations, the antennas can be used in conjunction with a tuning switch.

[0212] Continuing to refer to Figure 10, illustratively, the mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the wearable device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some implementations, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some implementations, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0213] Continuing to refer to Figure 10, illustratively, the wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., applied to the wearable device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0214] Specifically, in the technical solution provided in the embodiments of the present application, the PPG spectrum denoising model that triggers the wearable device 100 to implement denoising of the PPG frequency domain signal can be constructed by an electronic device with powerful processing capabilities, such as a PC, and then stored in a cloud server. In this way, the wearable device 100 can communicate with the cloud server via the mobile communication module 150 or the wireless communication module 160 to obtain the PPG spectrum denoising model.

[0215] For example, in some other possible implementations, the PPG spectrum denoising model constructed by an electronic device such as a PC can also be transmitted to the wearable device 100 through the USB interface 130.

[0216] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation on this embodiment.

[0217] 10 , illustratively, the display screen 194 is used to display images, videos, etc. In some implementations, the wearable device 100 may include 1 or N display screens 194 , where N is a positive integer greater than 1.

[0218] Continuing to refer to FIG. 10 , illustratively, the camera 193 is used to capture still images or videos.

[0219] Continuing with FIG10 , the external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the wearable device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage. For example, files such as music and videos can be stored on the external memory card.

[0220] Continuing to refer to Figure 10, illustratively, the internal memory 121 can be used to store computer executable program code, which includes instructions. In this way, the processor 110 executes various functional applications and data processing of the wearable device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the wearable device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0221] Specifically in the technical solution provided in the embodiment of the present application, the PPG spectrum denoising model can be pre-downloaded to the internal memory 121.

[0222] In addition, it should be noted that the wearable device 100 can implement audio functions such as music playback and recording through the audio module 170, such as the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0223] Continuing to refer to FIG10 , illustratively, the buttons 190 include a power button, a volume button, etc. The buttons 190 may be mechanical buttons or touch buttons.

[0224] 10 , for example, the motor 191 can generate a vibration prompt. The motor 191 can be used for an incoming call vibration prompt or for touch vibration feedback.

[0225] Continuing to refer to FIG. 10 , illustratively, the indicator 192 may be an indicator light, which may be used to indicate the charging status, power changes, messages, missed calls, notifications, and the like.

[0226] This concludes the introduction to the hardware structure of the wearable device 100. It should be understood that the wearable device 100 shown in FIG10 is merely an example. In a specific implementation, the wearable device 100 may have more or fewer components than shown in the figure, may combine two or more components, or may have a different component configuration. The various components shown in FIG10 may be implemented in hardware, including one or more signal processing and / or application-specific integrated circuits, software, or a combination of hardware and software.

[0227] In addition, it is understandable that electronic devices, such as wearable devices, include hardware and / or software modules that perform the corresponding functions in order to achieve the above functions. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of this application.

[0228] In addition, it should be noted that in actual application scenarios, the PPG frequency domain signal denoising methods provided in the above embodiments, implemented by electronic devices such as wearable devices, can also be performed by a chip system included in the electronic device, wherein the chip system may include a processor. The chip system can be coupled to a memory so that when the chip system is running, it calls a computer program stored in the memory to implement the steps performed by the above wearable device. The processor in the chip system can be an application processor or a processor other than an application processor.

[0229] In addition, an embodiment of the present application also provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on an electronic device, such as a wearable device, the electronic device executes the above-mentioned related method steps to implement the PPG frequency domain signal denoising method in the above-mentioned embodiment.

[0230] In addition, an embodiment of the present application further provides a computer program product. When the computer program product is run on an electronic device, such as a wearable device, the electronic device executes the above-mentioned related steps to implement the PPG frequency domain signal denoising method in the above-mentioned embodiment.

[0231] In addition, an embodiment of the present application further provides a chip (which may also be a component or module), which may include one or more processing circuits and one or more transceiver pins; wherein the transceiver pins and the processing circuit communicate with each other through an internal connection path, and the processing circuit executes the above-mentioned related method steps to implement the PPG frequency domain signal denoising method in the above-mentioned embodiment, so as to control the receiving pin to receive the signal, so as to control the transmitting pin to send the signal.

[0232] In addition, it can be seen from the above description that the electronic device, computer-readable storage medium, computer program product or chip provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0233] In addition, it should be noted that the embodiment of the present application only takes the scenario of heart rate measurement based on PPG frequency domain signals as an example. In actual applications, the measurement of blood oxygen concentration, blood pressure, etc. can also be achieved based on the same method, and the present application does not limit this.

[0234] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for denoising a PPG frequency domain signal, characterized in that: The method comprises: Obtain the photoelectric volume pulse wave (PPG) signal, acceleration (ACC) signal and infrared (IR) signal collected by the wearable device; According to the PPG spectrum denoising model, the spectrum features in the ACC signal, and the spectrum features in the IR signal, denoising the spectrum features of the PPG signal to obtain a denoised PPG frequency domain signal; Among them, the PPG spectrum denoising model takes the spectrum features in the PPG sample signal, the spectrum features in the ACC sample signal and the spectrum features in the IR sample signal as input, the PPG spectrum gold label as the supervision signal, and is trained by the transformer encoder model.

2. The method according to claim 1, characterized in that: The obtaining of the PPG signal, ACC signal and IR signal collected by the wearable device includes: The PPG signal, the ACC signal, and the IR signal collected by the wearable device in the same state and the same period are obtained.

3. The method according to claim 1 or 2, characterized in that: The PPG signal, the ACC signal and the IR signal are PPG signals, ACC signals and IR signals in the time domain; The method of performing denoising processing on the spectral features of the PPG signal according to the PPG spectrum denoising model, the spectral features in the ACC signal, and the spectral features in the IR signal to obtain a denoised PPG frequency domain signal includes: The PPG signal, the ACC signal, and the IR signal acquired in the time domain are converted from the time domain to the frequency domain to obtain a PPG frequency domain signal, an ACC frequency domain signal, and an IR frequency domain signal in the frequency domain; For each frame of the PPG frequency domain signal, extracting frequency spectrum features in the PPG frequency domain signal; For each frame of the ACC frequency domain signal, extracting the frequency spectrum features of the ACC frequency domain signal on the x-axis, y-axis and z-axis respectively; For each frame of the IR frequency domain signal, extracting a frequency spectrum feature in the IR frequency domain signal; Inputting the spectral features of the PPG signal, the spectral features in the IR frequency domain signal, and the spectral features of the ACC frequency domain signal on the x-axis, y-axis, and z-axis into the PPG spectrum denoising model; The output result of the PPG spectrum denoising model is used as the denoised PPG frequency domain signal.

4. The method according to claim 3, characterized in that The converting the acquired PPG signal, the ACC signal and the IR signal in the time domain into the frequency domain includes: Performing windowing, slicing and framing processing on the acquired PPG signal, the ACC signal and the IR signal in the time domain to obtain a continuous framed signal, wherein the framed signal includes the PPG signal, the ACC signal and the IR signal in the time domain; For each frame signal, the PPG signal, the ACC signal and the IR signal in the time domain included in the frame signal are converted from the time domain to the frequency domain.

5. The method according to claim 4, characterized in that The step of performing windowing, slicing and framing processing on the acquired PPG signal, the ACC signal and the IR signal in the time domain includes: With a windowing setting of a window size of 8s and a window moving step of 1s, the PPG signal, the ACC signal and the IR signal acquired in the time domain are subjected to windowing, slicing and framing processing.

6. The method according to any one of claims 1 to 5, characterized in that: The device type and resource configuration of the wearable device that collects the PPG sample signal, the ACC sample signal, and the IR sample signal are the same as those of the wearable device that collects the PPG signal, the ACC signal, and the IR signal.

7. The method according to any one of claims 1 to 5, characterized in that: The PPG spectrum gold standard is determined according to the heart rate gold standard collected by the heart rate tester; The heart rate gold standard is a heart rate with absolute accuracy collected by the heart rate tester when the wearable device collects the PPG sample signal, the ACC sample signal and the IR sample signal.

8. The method according to claim 7, characterized in that The PPG spectrum gold mark is obtained by processing the heart rate gold mark using a filtering construction method.

9. The method according to claim 7, characterized in that: The PPG spectrum gold label is obtained by processing the heart rate gold label in a heuristic construction manner.

10. The method according to any one of claims 1 to 9, characterized in that: The PPG spectrum denoising model is obtained based on training of a transformer encoder model.

11. The method according to any one of claims 1 to 9, characterized in that: The PPG spectrum denoising model is obtained based on training of two transformer encoder models; The input of the first transformer encoder model is the spectral features in each frame of the PPG sample signal, the spectral features in the ACC sample signal, and the spectral features in the IR sample signal, which are used to establish a mapping relationship between multi-channel features. The input of the second transformer encoder model is the spectral features after feature averaging the output of the input of the first transformer encoder model, which is used to establish a temporal mapping relationship between multi-frame features.

12. The method according to any one of claims 1 to 11, characterized in that: After obtaining the denoised PPG frequency domain signal, the method further includes: Based on the denoised PPG frequency domain signal, a heart rate of a user wearing the wearable device is predicted.

13. An electronic device, characterized in that: The electronic device includes: a memory and a processor, the memory and the processor are coupled; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the PPG frequency domain signal denoising method as described in any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that: It comprises a computer program, which, when executed on an electronic device, enables the electronic device to execute the PPG frequency domain signal denoising method as claimed in any one of claims 1 to 12.

Images