Physiological function monitoring method, system, terminal and medium based on multi-band laser

Abstract

The physiological function monitoring method, system, terminal and medium based on multi-band laser provided by the present invention specifically relate to the field of optical and physiological monitoring technology, and the scheme includes: based on a preset spatiotemporal mapping model, using a camera to collect a sequence of defocused speckle images of a target monitoring object under the irradiation of two laser lights within a preset time period; based on the defocused speckle image sequence, obtaining the vibration signal of the target monitoring object corresponding to the laser of each band, thereby obtaining the target physiological parameter monitoring value. The scheme combines multi-spectral and defocused speckle imaging technology, and utilizes the spatiotemporal coupling of the dual-band spot flow caused by the single-point motion of the target monitoring object to achieve non-contact and accurate recording of the motion of the target monitoring object, which is conducive to improving the accuracy of the target physiological parameter monitoring value calculated based on the vibration signal, while improving the flexibility of the recording, and effectively enhancing the robustness of the monitoring model.

Description

Technical Field

[0001] The present invention relates to the field of optical and physiological monitoring technology, and in particular to a physiological function monitoring method, system, terminal and medium based on multi-band laser. Background Art

[0002] Continuous cardiac function monitoring is essential for early detection of heart disease and healthy management of the cardiovascular system. In recent years, pulse wave imaging technology has made significant progress in both theory and practice, and has now entered the clinical trial stage. However, the monitoring method based on peripheral circulation is essentially unable to directly trace the heart activity, so it is impossible to accurately evaluate the heart itself or cardiovascular disease by video monitoring of skin perfusion.

[0003] At present, non-contact cardiac monitoring methods mainly sample and analyze the mechanical vibrations on the body surface generated by cardiac activity. Among them, the mainstream method uses the phase change produced by the electromagnetic wave echo of the radar signal to extract the cardiac vibration signal, but it is easily disturbed by the movement of non-target monitoring objects in the environment, resulting in inaccurate monitoring results. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a physiological function monitoring method, system, terminal and medium based on multi-band laser, aiming to solve the problem in the prior art that the vibration of physiological signals is easily disturbed by the movement of non-target monitoring objects in the environment, resulting in inaccurate monitoring results.

[0005] In order to achieve the above-mentioned object, the first aspect of the present invention provides a physiological function monitoring method based on multi-band laser, which is applied to a physiological function monitoring device based on multi-band laser, wherein the device comprises a camera and a laser lamp for emitting lasers of different bands, and the method comprises the following steps:

[0006] Based on a preset spatiotemporal mapping model, using the camera to collect a sequence of defocused speckle images of the target monitoring object under the irradiation of the laser light within a preset time period;

[0007] Based on the defocused speckle image sequence, a vibration signal of the target monitoring object corresponding to the laser in each wavelength band is obtained;

[0008] Based on all the vibration signals, a target physiological parameter monitoring value is obtained.

[0009] Optionally, the method of collecting a defocused speckle image sequence of a target monitoring object under the irradiation of the laser light within a preset time period by using the camera based on a preset spatiotemporal mapping model includes:

[0010] Based on a preset spatiotemporal mapping model, the laser light emitted by the laser lamp is used to irradiate different positions on the surface of the target monitoring object or at different positions within a preset distance range from the target monitoring object;

[0011] The camera is used to collect a sequence of defocused speckle images of the target monitoring object under the irradiation of the laser light within a preset time period.

[0012] Optionally, obtaining the vibration signal of the target monitoring object corresponding to the laser in each wavelength band based on the defocused speckle image sequence includes:

[0013] Performing signal separation on the defocused speckle image sequence according to the color channel corresponding to the laser band, to obtain the defocused speckle signal corresponding to the laser of each band;

[0014] The optical flow algorithm is used to extract the defocused speckle signal to obtain the vibration signal of the target monitoring object corresponding to the laser in each band.

[0015] Optionally, after obtaining the vibration signal of the target monitoring object corresponding to the laser of each wavelength band, the method further comprises:

[0016] Determining a peak value of each group of the vibration signals based on the amplitude of each group of the vibration signals;

[0017] Based on the peak value, calculating the average phase difference between the vibration signals corresponding to the lasers in different wavelength bands;

[0018] If the average phase difference does not meet the preset phase difference threshold, determine the laser after updating the wavelength band;

[0019] A sequence of defocused speckle images of the target monitoring object under the irradiation of the laser after the updated wavelength band is collected, and a vibration signal of the target monitoring object corresponding to the laser of each wavelength band is updated.

[0020] Optionally, after determining the peak value of each group of vibration signals, the method further comprises:

[0021] Calculating the time intervals between all adjacent peaks in each group of the vibration signals to obtain a time interval sequence corresponding to each group of the vibration signals;

[0022] If the time interval sequence does not meet the preset time interval threshold, a preset period checker is used to adjust the peak value sequence of at least one of the vibration signals to update the peak value.

[0023] Optionally, if the target monitoring object is a heart, after obtaining the target physiological parameter monitoring value, the following steps are included:

[0024] Based on the target physiological parameter monitoring value, calculating the heart rate variability index value of the target monitoring object after receiving a preset stimulus;

[0025] Based on the heart rate variability index value, a heart rate variability evaluation result is obtained.

[0026] The second aspect of the present invention provides a physiological function monitoring device based on multi-band laser, the device comprising a camera and a laser lamp for emitting lasers of different bands, and further comprising:

[0027] A defocused speckle image acquisition module, used to acquire a defocused speckle image sequence of a target monitoring object under the irradiation of the laser light within a preset time period using the camera based on a preset spatiotemporal mapping model;

[0028] A vibration signal extraction module, used for obtaining the vibration signal of the target monitoring object corresponding to the laser of each wavelength band based on the defocused speckle image sequence;

[0029] The monitoring result generating module is used to obtain the target physiological parameter monitoring value based on all the vibration signals.

[0030] The third aspect of the present invention provides a terminal, which includes a memory for storing executable instructions; and a processor for calling and running the executable instructions in the memory to execute any step of the above-mentioned multi-band laser-based physiological function monitoring method.

[0031] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores program instructions. When the program instructions are executed by a processor, any step of the above-mentioned multi-band laser-based physiological function monitoring method is implemented.

[0032] Compared with the prior art, the beneficial effects of this solution are as follows:

[0033] The present invention makes full use of the characteristics of defocused interferometric imaging, and uses lasers of different wavelengths to respectively locate at different positions of the target monitoring object. Therefore, there is a difference in the vibration information of the target monitoring object reflected under the irradiation of the two lasers. By using the camera to simultaneously monitor the reflected signals under the irradiation of lasers of different wavelengths, the spatial redundancy of the camera can be fully utilized to realize the activity monitoring of different positions of the target monitoring object, thereby realizing high-sensitivity monitoring of weak surface movements of the body; and by utilizing the motion differences of independent interference speckles generated by lasers of different wavelengths, multi-point simultaneous measurement of the vibration information of the target monitoring object is realized, that is, the combination of multi-spectral and defocused speckle imaging technologies is realized, and the spatiotemporal coupling of the dual-band spot flow caused by the single-point motion of the target monitoring object is utilized to realize non-contact and accurate recording of the motion of the target monitoring object, which is beneficial to improving the accuracy of the target physiological parameter monitoring value calculated based on the vibration signal, while improving the flexibility of the recording, and effectively enhancing the robustness of the monitoring model. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0035] Figure 1 It is a flow chart of the physiological function monitoring method based on multi-band laser of the present invention;

[0036] Figure 2 It is a schematic diagram of the rotational motion along the X-axis / Y-axis of the spatiotemporal mapping model between the heart beat and the speckle pixel motion of the present invention;

[0037] Figure 3 It is a schematic diagram of the translational movement along the Z axis of the spatiotemporal mapping model between the heart beat and the speckle pixel movement of the present invention;

[0038] Figure 4 It is a schematic diagram of the defocused speckle imaging and the corresponding optical flow field of the present invention;

[0039] Figure 5 It is a schematic diagram of the laser irradiation position of different wavelength bands of the present invention;

[0040] Figure 6 It is a schematic diagram of the phase difference between the heart vibration signals corresponding to the lasers of different wavelength bands of the present invention;

[0041] Figure 7 It is a schematic diagram of the performance comparison results of heart rate variability between different methods of the present invention;

[0042] Figure 8 It is a schematic diagram of a module of a physiological function monitoring device based on multi-band laser of the present invention;

[0043] Fig. 9 It is a schematic diagram of the terminal structure of the present invention. DETAILED DESCRIPTION

[0044] In the following description, specific details such as specific device structures, techniques, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details that hinder the description of the present invention.

[0045] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0046] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0047] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0048] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0049] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0051] In the prior art, the phase change generated by the electromagnetic wave echo of the radar signal is used to extract the cardiac vibration signal. However, the radar can usually only monitor the overall movement of the heart, and it is difficult to perform fine-grained monitoring of the local movement of the heart's chambers or valves. It is easily disturbed by the movement of non-target monitoring objects in the environment, resulting in poor robustness and inaccurate monitoring results. The present invention proposes a physiological function monitoring method based on multi-band lasers. On the one hand, it makes full use of the characteristics of defocused interferometric imaging and uses lasers of different bands to locate at different positions of the heart. Therefore, there are differences in the vibration information of the heart reflected under the irradiation of the two lasers. By simultaneously monitoring signals of different bands through the camera, the spatial redundancy of the camera can be fully utilized to realize the activity monitoring of different positions of the heart, thereby realizing high-sensitivity monitoring of weak movements on the chest surface; on the other hand, by utilizing the movement differences of independent interference speckles generated by lasers of different bands, multi-point simultaneous measurement of the vibration information of the heart's chambers and valve regions can be realized. That is, the present invention combines multi-spectral and defocused speckle imaging technologies, and utilizes the spatiotemporal coupling of the dual-band spot flow caused by the single-point motion of different chambers and valves of the heart to achieve non-contact and accurate recording of cardiac motion, and explores the phase information between the dual-band signals to provide a deeper and more accurate description of the functions of the heart and cardiovascular system, thereby improving the accuracy and real-time performance of non-contact continuous monitoring of cardiac function parameters.

[0052] The embodiment of the present invention provides a physiological function monitoring method based on multi-band laser, which is deployed on electronic devices such as computers and servers, and is applied to the scene of monitoring the physiological functions of target monitoring objects such as humans or animals, and is aimed at the situation of irradiating the target monitoring object with dual-band laser to collect defocused interference speckle images for analyzing target physiological parameters. Among them, the target physiological parameters include but are not limited to target physiological parameters related to heart beat or pulse beat, etc. The present invention takes the monitoring of physiological parameters related to heart beat as the target physiological parameters, and introduces the implementation steps of the method of the present invention in detail. All target physiological parameter monitoring functions implemented based on the concept of the method of the present invention are within the protection scope of the present invention.

[0053] The method of the present invention is applied to a physiological function monitoring device based on multi-band lasers, the device comprising a camera and a laser lamp for emitting lasers of different bands, each laser lamp being used to emit lasers of different bands to the surface of the target monitoring object or to different positions within a preset distance range from the target monitoring object. It should be stated that the preset distance range refers to a relatively small range of distance from the target monitoring object, within which information related to the target physiological parameters of the target detection object can also be normally monitored, which is described in detail below, and the camera is used to collect the defocused speckle image of the target monitoring object under the irradiation of lasers of different bands emitted by the laser lamp. The device also includes a spatiotemporal mapping model construction module, a defocused speckle image acquisition module, a vibration signal extraction module and a monitoring result generation module, and the functions of these modules will be described in detail in the following implementation process of the physiological function monitoring method based on multi-band lasers. In order to facilitate the description of the implementation process of the physiological function monitoring method based on multi-band lasers proposed by the present invention, this embodiment takes a basic physiological function monitoring device composed of a camera and two laser lamps for emitting lasers of different bands as an example, and elaborates on the implementation process of the physiological function monitoring method based on multi-band lasers.

[0054] Based on the basic physiological function monitoring device, the detailed process of the physiological function monitoring method based on multi-band laser of this embodiment is as follows: Figure 1 As shown, the details are as follows:

[0055] Step S100: Based on a preset spatiotemporal mapping model, the camera is used to collect a sequence of defocused speckle images of a target monitoring object under the irradiation of the laser light within a preset time period.

[0056] Specifically, since diffuse reflection occurs when the wavelength of light is smaller than the height of the undulations on the surface of the target monitoring object, the heart beat will be manifested on the chest surface in the form of microscopic vibrations with an amplitude of 0.2-0.5mm. Therefore, in principle, light in the visible light band can be used for this solution. In order to better extract the speckle patterns generated by light irradiation of different bands, this embodiment uses lasers of different bands as light sources. Each band of laser light contains only a single wavelength of light, which has good uniformity and is conducive to accurately separating the speckle patterns corresponding to each wavelength. In addition, according to the wave-particle duality of light, the wavelength of The light is modeled as:

[0057] (1),

[0058] in, is the amplitude of the light wave, i.e. the light intensity, is the wavelength of the light wave, For time, is the initial phase of the light wave. The incident direction of the light shown in formula (1) can be expressed as its first-order derivative, that is:

[0059] (2).

[0060] From formula (2), we can see that if the light intensity of the light wave With initial phase The wavelength of the incident light remains unchanged. When the height of the heart beat changes, the smaller the wavelength of the incident light, the greater the change in its incident direction, the greater the change in the direction of light reflection, and the easier it is for the reflected light to interfere with each other. Therefore, the different wavelengths of the incident light will only affect the location where interference occurs, that is, the distribution of the light spot. Since the conditions for light coherence include the same light frequency and different frequencies of light in different bands, the light in different bands interferes independently. Therefore, it is only necessary to construct an interference speckle model for one band to analyze the spatiotemporal mapping relationship between interference speckles and pixel motion in all bands.

[0061] After extensive research and experimental analysis, it was found that when the camera is in a defocused state, the interference speckle formed in the video after the laser irradiates and reflects on the surface of the human body not only does not become blurred, but instead appears as a granular shape with alternating light and dark and strong contrast. The slight movement of the chest cavity caused by the beating of the heart will change the reflection direction of the laser on its surface, resulting in changes in the spatial distribution of the speckle interference fringes in the reflected light field, and the intensity of the change in the speckle interference fringes will significantly increase with the increase in the defocus degree of the camera, where the defocus degree represents the ratio of the distance between the target monitoring object and the focal plane to the distance between the camera and the focal plane.

[0062] Therefore, this embodiment first constructs Figure 2 The spatiotemporal mapping model shown is specifically as follows: the camera and the target monitoring object are arranged on both sides of the focal plane of the camera. When the camera is out of focus, the target monitoring object moves to a distance from the focal plane. Taking the target monitoring object rotating along the Y axis as an example, the initial incident angle of the laser is , the imaging angle of the camera is , when the target monitoring object rotates a small angle along the Y axis The incident angle of the laser also changes by the same angle. The laser actually irradiates only a single pixel of the target monitoring object, but because the pixel is not on the focal plane, a speckle is generated on the camera imaging plane. The radial displacement of the speckle relative to the single pixel is The radial displacement is affected by two factors. First, the rotation of the target object itself will cause the same angle change in the orientation of the target object surface relative to the light source and the sensor. , obviously, ; Second, the change in the angle of illumination caused by the rotational motion will cause the reflected light to produce additional rotation In order to observe the same speckle after the target monitoring object is rotated, the constraint condition that the speckle before and after the rotation has the same initial phase difference (that is, the optical path difference before and after the rotation remains unchanged) should be satisfied. Therefore, according to this constraint condition, the change of the illumination angle is defined to satisfy the form shown in formula (3), that is:

[0063] (3),

[0064] Usually the rotation angle of the target monitoring object is very small, and the equivalent infinitesimal substitution principle can be obtained , then the radial displacement of the laser speckle on the camera imaging plane is The expression of is shown in formula (4), that is:

[0065] (4),

[0066] in, Indicates the distance between the target monitoring object and the focal plane, represents the distance between the camera lens and the focal plane, Indicates focal length.

[0067] Taking into account the restriction that the light irradiation angle is equal to the camera imaging angle, equation (4) can be simplified as:

[0068] (5),

[0069] Similarly, when the target monitoring object rotates along the X-axis, it will cause the pixel to move in the Y-direction of the camera imaging plane, and the expression of the motion displacement is the same as formula (4).

[0070] like Figure 3 As shown, when the camera is imaging vertically ( ), the translational motion of the target monitoring object dz It will not cause the movement of pixels on the camera imaging plane. When the camera imaging has an angle ( ), the light irradiated to the same position on the surface of the target monitoring object will have an angle of The rotation of the target monitoring object is proportional to the displacement dz of the target monitoring object. ).in, is the irradiation distance. From the above analysis, we can know that and equal, The expression of is the same as that of formula (3). Therefore, the displacement of the speckle on the camera sensor along the X-axis can be obtained , as shown in formula (6):

[0071] (6).

[0072] Since the camera is much more sensitive to out-of-plane rotation than other motions when it is highly out of focus, Much smaller than , the displacement of pixels in the X direction on the camera plane can be simplified as:

[0073] (7),

[0074] in, is a constant.

[0075] From the above analysis, it can be seen that the sensitivity of the pixels on the camera imaging plane to the motion of the target monitoring object in the X-axis direction and the Y-axis direction is positively correlated with the camera's defocus degree, where the defocus degree represents the distance between the target monitoring object and the focal plane. and the distance between the camera and the focal plane The ratio of Therefore, in this embodiment, the defocus degree of the camera is adjusted by the spatiotemporal mapping model construction module to achieve different sensitivity settings for the movement of the target monitoring object, so that the focal plane is located between the camera and the target monitoring object, and the defocus degree is set within the corresponding ratio range according to the sensitivity requirement of the movement of the target monitoring object, and the spatial position relationship between the camera and the target monitoring object is determined; based on the spatial position relationship, the vibration signal of the target monitoring object within a preset time period is collected, and the defocused speckle image within the time period is collected by the camera to determine the time sequence relationship between the vibration signal of the target monitoring object within the preset time period and the corresponding defocused speckle image movement; based on the spatial position relationship and the time sequence relationship, a spatiotemporal mapping model is constructed between the vibration signal of the target monitoring object within the time period and the defocused speckle image movement within the time period, that is, a spatiotemporal mapping model between the heart beat and the speckle pixel movement is constructed. The motion monitoring method of the target monitoring object in this embodiment is completely different from the motion monitoring in traditional video imaging. In traditional video monitoring, the farther the camera is from the target monitoring object, the lower the sensitivity of motion monitoring. Therefore, the imaging method of defocused speckle interference is particularly suitable for monitoring weak physiological movements at a long distance.

[0076] After the space-time mapping model is constructed, based on the preset space-time mapping model, the camera is used to collect a sequence of defocused speckle images of the target monitoring object under the irradiation of the laser light within a preset time period.

[0077] Specifically, a preliminary experiment was conducted on the imaging effect of the above-constructed spatiotemporal mapping model. In the experiment, the distance between the focal plane of the camera and the lens was controlled to adjust the defocus degree of the camera. Figure 4As shown in the figure, in the two experiments, the laser light emits lasers of the same wavelength, but the distance between the target monitoring object and the focal plane is different, which are called the long object distance experiment and the short object distance experiment respectively. By comparing the light interference results of the two experiments, it can be seen that both experiments have the same interference and the reverse interference. Compared with the short object distance experiment, the particle size of the speckle generated by the long object distance experiment is larger, and the movement of the optical flow field is more obvious. It can be seen that in the monitoring of laser interference phenomenon, as the distance between the target monitoring object and the focal plane increases (that is, the object distance is getting farther and farther), the particle size of the obtained speckle is also getting larger and the speckle movement is more sensitive (that is, the arrow representing the movement in the optical flow field is longer), which is consistent with the derivation conclusion related to the above-mentioned spatiotemporal mapping model between the heart beat and the speckle pixel movement. Therefore, according to the calculation accuracy and sensitivity requirements, the camera can be placed at a suitable distance from the subject to meet the requirements of the interference speckle to amplify the multiple of the heart vibration.

[0078] This embodiment takes into account that in actual operation, the staff will try their best to irradiate the position of the heart, but the laser irradiation position may deviate from the heart. In order to take into account both the flexibility and accuracy of the monitoring operation, this embodiment, based on the above-mentioned spatiotemporal mapping model, uses lasers of different bands to irradiate the surface of the target monitoring object or different positions within a preset distance range from the target monitoring object through a defocused speckle image acquisition module, and emits lasers of different bands, wherein the preset distance range represents a range in which the heart beat fluctuation can be normally monitored, and refers to a distance that deviates from the heart relatively small; uses a camera to acquire a sequence of defocused speckle images of the target monitoring object irradiated with lasers of different bands within a preset time period, wherein the defocused speckle image sequence includes defocused interference speckles of reflection signals formed under the irradiation of lasers of two bands, wherein the preset time period refers to a period of time corresponding to several heart beat cycles.

[0079] It is easy to understand that this embodiment takes a basic physiological function monitoring device composed of a camera and two laser lamps for emitting lasers of different wavelengths as an example to elaborate on the implementation process of the physiological function monitoring method based on multi-band lasers, but the basic physiological function monitoring device is not a limitation of the physiological function monitoring device based on multi-band lasers proposed by the present invention, and any device that implements the physiological function monitoring function based on the design principle of the basic physiological function monitoring device is within the protection scope of the present invention. For a physiological function monitoring device composed of multiple cameras and a laser lamp capable of emitting more than two wavelengths of lasers or multiple laser lamps, it is only necessary to select one of the cameras that meets the above-mentioned spatial position relationship and a two-band laser that can produce a defocused speckle image sequence when the lamp irradiates the target monitoring object.

[0080] Step S200: Based on the defocused speckle image sequence, a vibration signal of the target monitoring object corresponding to the laser light of each wavelength band is obtained.

[0081] Specifically, since each pixel in the defocused speckle image is composed of pixel values ​​of three channels of RGB, the color channels corresponding to the reflected signals generated under the irradiation of lasers of different bands are different, so the defocused speckles corresponding to the lasers of each band can be separated according to the color channels, and the vibration signal of the target monitoring object is extracted using the laser speckle of the optical mode, which will not be interfered by the magnetic field of the magnetic resonance system itself, so the vibration signal of the target monitoring object constructed is of high accuracy. Based on this, the present embodiment uses the vibration signal extraction module to perform signal separation on the defocused speckle image sequence according to the color channels corresponding to the laser bands to obtain the defocused speckle signals corresponding to the lasers of each band; then the optical flow algorithm is used to calculate the motion amplitude of the defocused speckle information to obtain the motion amplitude sequence; based on the motion amplitude sequence, the vibration signal of the target monitoring object corresponding to the lasers of each band is extracted. In the present embodiment, the vibration signal of the target monitoring object refers to the weak vibration of the chest cavity caused by the heart beat reflected by the laser, which is used to characterize the mechanical motion state of the heart beat.

[0082] The following is based on Figure 4 The corresponding conclusions and the operation steps of step S100 and step S200 are verified by experiments when the camera defocus degree is greater than 5. Specifically, the position of one laser light is fixed, and the other laser light is moved to Figure 5 For example, a fixed laser lamp emits a laser of one wavelength to illuminate the green pixel position of the upper part of the heart (left / right atrium), and another moving laser lamp illuminates the positions numbered 1-5 marked with red pixel points, wherein the pixel points numbered 1, 2, and 3 are on the left / right atrium, and the pixel points numbered 4 and 5 are within a range relatively close to the left / right atrium; or, a fixed laser lamp emits a laser of one wavelength to illuminate the red pixel position of the lower part of the heart (left / right ventricle), and another moving laser lamp illuminates the positions numbered 1-5 marked with green pixel points, wherein the pixel points numbered 1, 2, and 3 are on the left / right ventricle, and the pixel points numbered 4 and 5 are within a range relatively close to the left / right ventricle. Then, two laser lights were used to continuously emit lasers for 20 seconds and the lasers emitted by the two laser lights were different in wavelength, such as red light band (R band) and green light band (G band). The camera was used to collect the speckle images of the heart under the irradiation of the two lasers, and the heart beat signals under the irradiation of the two laser bands were extracted from the speckle images. Finally, the average phase difference between the aortic valve opening (AO) peaks of the two heart beat signals was calculated, as shown in Table 1 and Figure 6 As shown, Figure 6 In - They respectively represent the time delay between the AO peaks of two cardiac beat signals in different cardiac beat cycles.

[0083] Table 1:

[0084]

[0085] Depend on Figure 6 It can be seen that at different positions of the heart, the overall trend of the motion signal of the speckle (the low-frequency component of the signal) shows consistency, but there are obvious differences in the details of the motion signal (the high-frequency component of the signal). As shown in Table 1, when the laser is irradiated at different positions of the heart, even if it is slightly deviated from the position of the heart's atrioventricular chamber, the phase difference between the AO peaks generated by the two-band lasers is basically the same (error <10ms). Therefore, there is no need for precise positioning, and the two-band lasers can obtain the timing information between the heart's atrioventricular movements. It is worth stating that in actual operations, the staff will try to irradiate the position of the heart, and even if there is a deviation, it will not deviate too much from the position of the heart to ensure the effectiveness and accuracy of the average phase difference.

[0086] Step S300: obtaining a target physiological parameter monitoring value based on all the vibration signals.

[0087] Specifically, the vibration signal of the target monitoring object corresponding to each band of laser is first preprocessed by denoising and filtering using the monitoring result generation module to improve the signal quality. Then, the monitoring result generation module is used to extract the target physiological parameter monitoring values ​​such as heart rate and heart rate variability from the preprocessed vibration signal, and the difference of vibration signals corresponding to different bands is used to estimate the hemodynamic parameters of the heart such as blood pressure to obtain the target physiological parameter monitoring value.

[0088] In a preferred embodiment, after obtaining the vibration signal of the target monitoring object corresponding to the laser of each wavelength band in step S200, the following steps are included:

[0089] Step S400: determining the peak value of each group of vibration signals based on the amplitude of each group of vibration signals;

[0090] Step S500: Based on the peak value, calculating the average phase difference between the vibration signals corresponding to lasers of different wavelength bands;

[0091] Step S600: if the average phase difference does not meet the preset phase difference threshold, determine the laser after updating the wavelength band;

[0092] Step S700: collecting a sequence of defocused speckle images of the target monitoring object under the laser irradiation after the updated wavelength band, and updating the vibration signal of the target monitoring object corresponding to the laser of each wavelength band.

[0093] Specifically, the amplitude of the vibration signal is detected using a peak detection algorithm (such as a local maximum search algorithm, etc.) to determine all peak points of each vibration signal, each peak point representing the maximum amplitude in each vibration cycle; the phase difference between the peak points of the vibration signal of the target monitoring object corresponding to the lasers of different bands in each same cycle is calculated and averaged to obtain the average phase difference of the lasers of different bands, and the characteristics of the average phase difference are related to hemodynamic parameters such as blood pressure; a phase difference threshold is pre-set to ensure that the phases of the vibration signals collected under the irradiation of lasers of different bands can be clearly distinguished. If the average phase difference is less than the preset phase difference threshold, it indicates that there is a significant phase mismatch between the lasers of the corresponding bands, so the wavelength of at least one laser can be selected to be adjusted, and the laser parameters can be continuously monitored and adjusted for multiple times to achieve the desired phase matching effect, and the laser after the updated band is determined. Then, a sequence of defocused speckle images of the target monitoring object under the irradiation of the laser after the updated band is collected, and the vibration signal of the target monitoring object corresponding to the laser of each band is updated, so that the target physiological parameter monitoring value can be obtained later using the updated vibration signal.

[0094] This embodiment adjusts the average phase difference between the vibration signals corresponding to different bands based on a preset phase difference threshold, ensuring that the phases of the vibration signals collected under laser irradiation of the corresponding bands can be clearly distinguished, thereby achieving precise phase matching, which helps to improve the accuracy of subsequent target physiological parameter monitoring values.

[0095] Further, after determining the peak value of each group of vibration signals in step S400, the following steps are included:

[0096] For the vibration signals of the target monitoring object corresponding to any two lasers that generate defocused laser speckle interference, the time intervals between all adjacent peaks in each group of vibration signals are calculated to obtain a time interval sequence corresponding to each group of vibration signals, and the characteristics of the time interval sequence are related to physiological parameters such as heart rate and heart rate variability; a time interval threshold is pre-set to ensure that the peaks of the two groups of vibration signals match in each same period to ensure that the waveform periods of the two groups of vibration signals match. If the preset time interval threshold is not met between any two time interval sequences, it means that there is an obvious deviation in the waveform periods of the two groups of vibration signals. Then, a period checker based on inter-beat interval (IBI) is used to adjust the peak sequence of one or two groups of vibration signals, and the peak corresponding to each group of vibration signals is updated, so as to facilitate the subsequent calculation of the average phase difference between the vibration signals using the peak after peak check.

[0097] This embodiment uses a period checker based on the beat interval to perform peak value check to ensure that the peak values ​​of the two groups of vibration signals are compared within the same cardiac cycle, thereby ensuring the comparability between the corresponding peak values ​​of the two groups of vibration signals and the validity of the comparison results.

[0098] If the target monitoring object is the heart, then after obtaining the target physiological parameter monitoring value in step S300, the following steps are included:

[0099] Step S800: Calculating the heart rate variability index value of the target monitored object after receiving a preset stimulus based on the target physiological parameter monitoring value;

[0100] Step S900: obtaining a heart rate variability evaluation result based on the heart rate variability index value.

[0101] Specifically, the standard heart rate variability index measurement results of the target monitoring object before and after receiving the preset stimulation are obtained; based on the target physiological parameter monitoring value, the real heart rate variability index measurement result of the target monitoring object after receiving the stimulation is calculated; the difference between the real heart rate variability index measurement result and the standard heart rate variability index measurement result is calculated to obtain the heart rate variability evaluation result, which can provide strong support for the clinical application of heart rate variability.

[0102] In order to verify the effectiveness of the method of the present invention, experiments were conducted to verify that the specific device layout and important parameter settings are as follows: the camera and the two lasers are arranged in parallel to ensure that the illumination angle is equal to the imaging angle, and the angles between the three and the horizontal plane are all maintained at 90°. The distance between the camera plane and the bed surface is 1.2m, the distance between the camera focal plane and the camera lens is about 0.15m, and the distance between the camera focal plane and the chest surface is about 0.85m. According to the above definition of defocus, the camera's defocus degree can be obtained to be about 5.67. The acquisition frame rate of the camera is set to 250fps, and the laser bands emitted by the laser are 515nm and 650nm, respectively, and their powers are all ≤10mW.

[0103] Firstly, by recording the standard electrocardiogram (ECG), photoplethysmographic (PPG) and seismocardiography (SCG) signals of dozens of subjects, the ECG calculation results were used as the gold standard to analyze and compare the performance of contact pulse wave signals (PPG) and SCG in evaluating heart rate variability. During the experiment, ice water stimulation was used to induce changes in the heart rate of the subjects, and the standard deviation of the sequential sinus rhythm intervals corresponding to the ECG signal, PPG signal and the camera-based SCG signal of the present invention was calculated respectively ( ), mean square error of consecutive sinus rhythm intervals ( ), very low frequency band absolute power ( )、low frequency band absolute power( )、High frequency band absolute power( ) and the absolute power ratio of the low-frequency band to the high-frequency band. The results are as follows Figure 7 As shown in the figure, it can be seen that the various indicators of heart rate variability calculated using SCG can better follow the changes in the gold standard ECG results after ice grabbing. Compared with PPG signals, SCG has more potential in evaluating heart rate variability. This result shows that SCG is feasible in evaluating physiological indicators of cardiac activity.

[0104] In addition, by recording the ECG, PPG and dual-band cardiac shock signals (Dual-SCG) of dozens of subjects, and extracting features from them for estimating blood pressure in hemodynamic parameters, the performance of estimating blood pressure using dual-band phase information proposed in the present invention is compared with the existing mainstream method of estimating blood pressure by extracting pulse arrival time (PAT) from PPG-ECG. First, the SCG signal is extracted from the speckle video using an optical flow algorithm, and then the AO peak of the SCG, the R wave of the ECG, and the peak and valley of the PPG signal are detected. In order to ensure that the peaks between different signals correspond to the same period, that is, the beat-to-beat interval (IBI) should be the same, a period checker based on IBI is customized to update the detected peak. Since the frequency of blood pressure measurement is about 30 seconds / time, the signal is divided into several 28-second segments based on the time when blood pressure occurs. After extracting features, the correlation between waveform features and systolic blood pressure (SBP), diastolic blood pressure (DBP), and mean arterial blood pressure (MBP) is analyzed. The waveform features include average phase difference, peak pulse transmission time-1, peak pulse transmission time-2, valley pulse transmission time-1, valley pulse transmission time-2, phase difference 14 seconds before the signal, phase difference 14 seconds after the signal, and AO peak ratio of the red band / green band of the SCG signal. Among them, the mark -1 represents the peak or valley pulse transmission time feature before cycle correction, and the mark -2 represents the peak or valley pulse transmission time feature after cycle correction. The results of the correlation analysis between the monitoring signal features and blood pressure are shown in Table 2.

[0105] Table 2:

[0106]

[0107] From the results in Table 2, it can be seen that the correlation between the pulse transmission time characteristics and blood pressure after period verification has improved, which verifies the necessity of period verification. In addition, there is a good correlation between the dual-band phase difference and blood pressure, which is better than the correlation between pulse transmission time and blood pressure, and preliminarily demonstrates the feasibility of estimating blood pressure using the phase information between dual-band SCG. This method has great potential in evaluating cardiac activity. Compared with the existing pulse recording method, the method of the present invention can characterize cardiac movement more directly and accurately in a non-contact manner.

[0108] like Figure 8As shown, corresponding to the above-mentioned physiological function monitoring method based on multi-band laser, an embodiment of the present invention further provides a physiological function monitoring device based on multi-band laser, the device comprising a camera and a laser lamp for emitting lasers of different bands, and further comprising:

[0109] The defocused speckle image acquisition module 810 is used to acquire a defocused speckle image sequence of the target monitoring object under the irradiation of the laser light within a preset time period by using the camera based on a preset spatiotemporal mapping model;

[0110] A vibration signal extraction module 820 is used to obtain the vibration signal of the target monitoring object corresponding to the laser of each wavelength band based on the defocused speckle image sequence;

[0111] The monitoring result generating module 830 is used to obtain the target physiological parameter monitoring value based on all the vibration signals.

[0112] Specifically, in this embodiment, the specific functions of the above-mentioned physiological function monitoring device based on multi-band laser can also refer to the corresponding description in the above-mentioned physiological function monitoring method based on multi-band laser, and will not be repeated here.

[0113] Based on the above embodiment, the present invention further provides a terminal, whose principle block diagram can be as follows: Fig. 9 As shown. The terminal can be used to execute the physiological function monitoring method based on multi-band laser provided in the above embodiment, which will not be described here for the sake of brevity. The terminal includes: a processor, the processor is coupled to a memory, the memory is used to store computer programs or instructions, and the processor is used to execute the computer programs or instructions stored in the memory, so that the method in the above method embodiment is executed.

[0114] The present invention also provides a computer-readable storage medium on which computer instructions for implementing the method in the above method embodiment are stored.

[0115] For example, when the computer program is executed by a computer, the computer can implement the method in the above method embodiment.

[0116] An embodiment of the present application also provides a computer program product comprising instructions, which, when executed by a computer, enables the computer to implement the method in the above method embodiment.

[0117] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0118] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0119] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0120] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0121] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0122] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

Claims

1. A physiological function monitoring method based on multi-band laser, characterized in that: The invention is applied to a physiological function monitoring device based on multi-band laser, the device comprises a camera and a laser lamp for emitting lasers of different bands, and the method comprises the following steps: Based on a preset spatiotemporal mapping model, using the camera to collect a sequence of defocused speckle images of the target monitoring object under the irradiation of the laser light within a preset time period; Based on the defocused speckle image sequence, a vibration signal of the target monitoring object corresponding to the laser in each wavelength band is obtained; Based on all the vibration signals, obtaining a target physiological parameter monitoring value; After obtaining the vibration signal of the target monitoring object corresponding to the laser of each wavelength band, the method includes: Determining a peak value of each group of the vibration signals based on the amplitude of each group of the vibration signals; Based on the peak value, calculating the average phase difference between the vibration signals corresponding to the lasers in different wavelength bands; If the average phase difference does not meet the preset phase difference threshold, determine the laser after updating the wavelength band; A sequence of defocused speckle images of the target monitoring object under the irradiation of the laser after the updated wavelength band is collected, and a vibration signal of the target monitoring object corresponding to the laser of each wavelength band is updated.

2. The method for monitoring physiological functions based on multi-band laser according to claim 1, characterized in that: The method of collecting a defocused speckle image sequence of a target monitoring object under the irradiation of the laser light within a preset time period by using the camera based on the preset spatiotemporal mapping model includes: Based on a preset spatiotemporal mapping model, the laser light emitted by the laser lamp is used to irradiate different positions on the surface of the target monitoring object or at different positions within a preset distance range from the target monitoring object; The camera is used to collect a sequence of defocused speckle images of the target monitoring object under the irradiation of the laser light within a preset time period.

3. The method for monitoring physiological functions based on multi-band laser according to claim 1, characterized in that: The step of obtaining the vibration signal of the target monitoring object corresponding to the laser beam in each wavelength band based on the defocused speckle image sequence includes: Performing signal separation on the defocused speckle image sequence according to the color channel corresponding to the laser band, to obtain the defocused speckle signal corresponding to the laser of each band; The optical flow algorithm is used to extract the defocused speckle signal to obtain the vibration signal of the target monitoring object corresponding to the laser in each wavelength band.

4. The method for monitoring physiological functions based on multi-band laser according to claim 1, characterized in that: After determining the peak value of each group of vibration signals, the method further comprises: Calculating the time intervals between all adjacent peaks in each group of the vibration signals to obtain a time interval sequence corresponding to each group of the vibration signals; If the time interval sequence does not meet the preset time interval threshold, a preset period checker is used to adjust the peak value sequence of at least one of the vibration signals to update the peak value.

5. The method for monitoring physiological functions based on multi-band laser according to any one of claims 1 to 3, characterized in that: If the target monitoring object is the heart, after obtaining the target physiological parameter monitoring value, the following steps are included: Based on the target physiological parameter monitoring value, calculating the heart rate variability index value of the target monitoring object after receiving a preset stimulus; Based on the heart rate variability index value, a heart rate variability evaluation result is obtained.

6. A physiological function monitoring device based on multi-band laser, characterized in that: The device is used to implement the steps of the multi-band laser-based physiological function monitoring method according to any one of claims 1 to 5, and the device includes a camera and a laser lamp for emitting lasers of different bands, and also includes: A defocused speckle image acquisition module, used to acquire a defocused speckle image sequence of a target monitoring object under the irradiation of the laser light within a preset time period using the camera based on a preset spatiotemporal mapping model; A vibration signal extraction module, used for obtaining the vibration signal of the target monitoring object corresponding to the laser of each wavelength band based on the defocused speckle image sequence; The monitoring result generating module is used to obtain the target physiological parameter monitoring value based on all the vibration signals.

7. A terminal, characterized in that: include: A memory for storing executable instructions; A processor is used to call and run the executable instructions in the memory to execute the steps of the physiological function monitoring method based on multi-band laser as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and when the program instructions are executed by the processor, the physiological function monitoring method based on multi-band laser as described in any one of claims 1 to 5 is implemented.

Images