Method and device for detecting three-dimensional motion signal of heart, terminal equipment and storage medium

Abstract

The invention relates to the technical field of biomedical engineering. The invention discloses a method and a device for detecting a three-dimensional motion signal of a heart, terminal equipment and a storage medium. The accuracy of detecting the three-dimensional motion signal of the heart can be improved. The method comprises the following steps: when a plurality of sensors shoot the thoracic cavity surface of a user, acquiring a motion signal of a speckle in each sensor in the X-axis direction of an imaging plane, and obtaining an X-axis direction motion signal corresponding to each sensor; when all X-axis direction motion signals are obtained, Y-axis direction motion signals corresponding to each sensor are obtained; and calculating all the X-axis direction motion signals and all the Y-axis direction motion signals by adopting a preset algorithm to obtain a vertical motion signal of the thoracic cavity surface of the user, a rotary motion signal in the X-axis direction and a rotary motion signal in the Y-axis direction.

Description

Technical Field

[0001] This application relates to the field of biomedical engineering technology. More specifically, this application relates to a method, device, terminal device, and storage medium for detecting three-dimensional motion signals of the heart. Background Art

[0002] Traditional methods for detecting three-dimensional motion signals of the heart are usually based on radio frequency (such as radar, ultrasound) methods. The radio frequency-based method captures the overall motion of the heart by measuring the echo signal reflected when the radio frequency signal irradiates the surface of the user's chest cavity. During the propagation of the radio frequency signal, in the direction perpendicular to the transmitting antenna (i.e., the vertical direction), the energy distribution and reflection characteristics of the signal are the most significant. When the radio frequency signal irradiates the surface of the user's chest cavity, the motion of the heart in the vertical direction causes a significant change in the distance between the chest cavity surface and the radio frequency antenna, resulting in a large change in the characteristics of the reflected light (i.e., the echo signal). For the motion of the heart in the horizontal direction or other non-vertical directions, its impact on the propagation path of the radio frequency signal and the echo characteristics is relatively small. Because the horizontal motion does not significantly change the vertical distance between the chest cavity surface and the antenna, the propagation of the radio frequency signal in the horizontal direction is limitedly affected, and the change in the echo signal is not obvious. Therefore, the radio frequency-based method is difficult to accurately perceive and measure the motion in the horizontal direction or other non-vertical directions. Thus, the radio frequency-based method is only sensitive to the vertical motion of the surface of the user's chest cavity, resulting in the inability of this method to accurately restore the three-dimensional motion information of the heart. Summary of the Invention

[0003] The purpose of the embodiments of this application is to provide a method, device, terminal device, and storage medium for detecting three-dimensional motion signals of the heart, which can improve the accuracy of detecting three-dimensional motion signals of the heart. The embodiments of this application are mainly implemented through the following technical solutions: In the first aspect of the embodiments of this application, a method for detecting three-dimensional motion signals of the heart is provided, including: When multiple sensors capture the surface of the user's chest cavity, obtain the motion signals of the speckles in the X-axis direction in the imaging plane of each sensor, and obtain the X-axis direction motion signals corresponding to each sensor; While obtaining all the X-axis direction motion signals, obtain the Y-axis direction motion signals corresponding to each sensor; Use a preset algorithm to perform calculation processing on all the X-axis direction motion signals and all the Y-axis direction motion signals to obtain the vertical motion signal, the X-axis direction rotational motion signal, and the Y-axis direction rotational motion signal of the surface of the user's chest cavity.

[0004] According to an embodiment of the present application, the steps of calculating and processing all the X-axis direction motion signals and all the Y-axis direction motion signals by using a preset algorithm to obtain the vertical motion signal, the rotational motion signal in the X-axis direction, and the rotational motion signal in the Y-axis direction on the thoracic surface of the user include: Set a first coefficient, a second coefficient, a third coefficient, and a fourth coefficient; Obtain the distance between the lens plane and the imaging plane of the target sensor, and the distance between the focal plane and the lens plane of each sensor, where the target sensor is any one of the multiple sensors; Based on the first coefficient, the second coefficient, the third coefficient, the fourth coefficient, the distance between the lens plane and the imaging plane of the target sensor, the distance between the focal plane and the lens plane of each sensor, all the X-axis direction motion signals, and all the Y-axis direction motion signals, perform calculation and processing according to the preset algorithm to obtain the vertical motion signal, the rotational motion signal in the X-axis direction, and the rotational motion signal in the Y-axis direction on the thoracic surface of the user.

[0005] According to an embodiment of the present application, when the multiple sensors include a first sensor and a second sensor, based on the first coefficient, the second coefficient, the third coefficient, the fourth coefficient, the distance between the lens plane and the imaging plane of the target sensor, the distance between the focal plane and the lens plane of each sensor, all the X-axis direction motion signals, and all the Y-axis direction motion signals, the calculation formula for the steps of performing calculation and processing according to the preset algorithm to obtain the vertical motion signal, the rotational motion signal in the X-axis direction, and the rotational motion signal in the Y-axis direction on the thoracic surface of the user is: ; ; ; Wherein, is the vertical motion signal corresponding to the is time; is the distance between the lens plane and the imaging plane of the target sensor; is the third coefficient; is the distance between the focal plane and the lens plane of the first sensor; is the X-axis direction motion signal corresponding to the second sensor at the is the fourth coefficient; is the distance between the focal plane and the lens plane of the second sensor; is the X-axis direction motion signal corresponding to the moment of the first sensor; is the second coefficient; is the first coefficient; is the rotational motion signal in the Y-axis direction corresponding to the is the rotational motion signal in the X-axis direction corresponding to the moment of the first sensor; is the Y-axis direction motion signal corresponding to the is the moment of the second sensor;

[0006] According to an embodiment of the present application, when multiple sensors capture the chest surface of a user, the steps of obtaining the motion signal of the speckle in the X-axis direction of the imaging plane in each sensor and obtaining the X-axis direction motion signal corresponding to each sensor include: When the user makes a vertical motion, obtain the motion signal of the speckle in the X-axis direction of the imaging plane in each sensor, and obtain a first motion signal to be processed corresponding to each sensor; When the user makes a rotational motion, obtain the motion signal of the speckle in the X-axis direction of the imaging plane in each sensor, and obtain a second motion signal to be processed corresponding to each sensor; Sum the first motion signal to be processed and the second motion signal to be processed corresponding to each sensor to obtain the X-axis direction motion signal corresponding to each sensor.

[0007] According to an embodiment of the present application, when the user makes a vertical motion, the steps of obtaining the motion signal of the speckle in the X-axis direction of the imaging plane in each sensor and obtaining a first motion signal to be processed corresponding to each sensor include: When the user makes a vertical motion, obtain the distance between the focal plane of each sensor and the chest surface; Obtain the initial incident angle of the laser emitted by the laser onto the chest surface of the user; Based on the distance between the focal plane of each sensor and the chest surface, the initial incident angle, and a preset condition, calculate and obtain the first motion signal to be processed corresponding to each sensor.

[0008] According to an embodiment of the present application, when the user makes a rotational motion, the steps of obtaining the motion signal of the speckle in the X-axis direction of the imaging plane in each sensor and obtaining a second motion signal to be processed corresponding to each sensor include: When the user makes a rotational movement, obtain the reflection angle of the laser on each sensor; Based on the reflection angle on each sensor, the distance between the lens plane and the imaging plane of the target sensor, the distance between the focal plane of each sensor and the chest surface, and the distance between the focal plane of each sensor and the lens plane, calculate the second to-be-processed motion signal corresponding to each sensor.

[0009] According to an embodiment of the present application, when obtaining all the X-axis direction motion signals, the steps of obtaining the Y-axis direction motion signal corresponding to each sensor include: Obtain the angle of the user's rotation along the X-axis; Based on the distance between the lens plane and the imaging plane of the target sensor, the distance between the focal plane of each sensor and the lens plane, the distance between the focal plane of each sensor and the chest surface, the initial incident angle, and the initial imaging angle of each sensor, calculate the Y-axis direction motion signal corresponding to each sensor.

[0010] In a second aspect of the embodiments of the present application, there is provided a detection device for three-dimensional motion signals of the heart, including: An X-axis direction motion signal obtaining module, configured to obtain the motion signal of the speckle in the X-axis direction on the imaging plane of each sensor when multiple sensors capture the chest surface of the user, and obtain the X-axis direction motion signal corresponding to each sensor; A Y-axis direction motion signal obtaining module, configured to obtain the Y-axis direction motion signal corresponding to each sensor while obtaining all the X-axis direction motion signals; A calculation module, configured to perform calculation processing on all the X-axis direction motion signals and all the Y-axis direction motion signals by using a preset algorithm to obtain the vertical motion signal, the rotational motion signal in the X-axis direction, and the rotational motion signal in the Y-axis direction of the chest surface of the user.

[0011] In a third aspect of the embodiments of the present application, there is provided a terminal device, including: a processor and a memory, where the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the steps of the method for detecting three-dimensional motion signals of the heart provided in the first aspect of the embodiments of the present application.

[0012] In a fourth aspect of the embodiments of the present application, there is provided a computer-readable storage medium, where the computer-readable storage medium is used to store a computer program, and the computer program enables a computer to execute the steps of the method for detecting three-dimensional motion signals of the heart provided in the first aspect of the embodiments of the present application.

[0013] The beneficial effects of the embodiments of the present application include: An embodiment of the present application sets a preset algorithm, which can be used to effectively decouple the vertical motion signal and the rotational motion signal of the heart. Specifically, in the embodiment of the present application, when multiple sensors capture the surface of the user's chest cavity, the motion signals of the speckles in each sensor in the X-axis direction of the imaging plane are obtained to obtain the X-axis direction motion signals corresponding to each sensor; while obtaining all the X-axis direction motion signals, the Y-axis direction motion signals corresponding to each sensor are obtained; the preset algorithm is used to calculate and process all the X-axis direction motion signals and all the Y-axis direction motion signals to obtain the vertical motion signal of the surface of the user's chest cavity, the rotational motion signal in the X-axis direction, and the rotational motion signal in the Y-axis direction. Compared with the prior art, the embodiment of the present application can not only capture the vertical motion signal of the heart, but also has high sensitivity to the rotational motion signal in the X-axis direction and the rotational motion signal in the Y-axis direction, overcoming the limitation that the radio frequency-based method is only sensitive to the vertical motion signal, thereby improving the accuracy of detecting the three-dimensional motion signal of the heart. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0015] Figure 1 It is a flowchart of the method for detecting the three-dimensional motion signal of the heart of the present application in some embodiments; Figure 2 It is a reference diagram of the relationship between the laser, the first sensor, the second sensor and the user when the user makes a vertical motion in the present application; Figure 3 It is a reference diagram of the relationship between the laser, the first sensor, the second sensor and the user when the user makes a rotational motion in the present application; Figure 4 It is a reference diagram of the reconstructed motion signal corresponding to a 5mm motion amplitude; Figure 5 It is a reference diagram of the reconstructed motion signal corresponding to a 10mm motion amplitude; Figure 6 It is a reference diagram of the reconstructed motion signal corresponding to a 15mm motion amplitude; Figure 7 It is a reference diagram of the spectrum analysis of the motion signal restored by the method for detecting the three-dimensional motion signal of the heart when the motion frequency of the motion simulator is 1 Hz; Figure 8Reference diagram for spectral analysis of the motion signal recovered by the detection method of the three-dimensional motion signal of the heart when the motion frequency of the motion simulator is 1.5 Hz; Figure 9 Reference diagram for spectral analysis of the motion signal recovered by the detection method of the three-dimensional motion signal of the heart when the motion frequency of the motion simulator is 2 Hz; Figure 10 Reference diagram for spectral analysis of the motion signal recovered by the detection method of the three-dimensional motion signal of the heart when the motion frequency of the motion simulator is 3 Hz; Figure 11 Reference diagram for reconstructing the vertical motion signal, the reconstructed vertical motion average signal and the reference vertical motion average signal; Figure 12 Reference diagram for the original speckle motion signal in the X direction, the original speckle motion average signal in the X direction and the reference vertical motion average signal corresponding to the first sensor; Figure 13 Reference diagram for the original speckle motion signal in the X direction, the original speckle motion average signal in the X direction and the reference vertical motion average signal corresponding to the second sensor; Figure 14 Reference diagram for reconstructing the rotational motion signal in the Y-axis direction, the reconstructed rotational motion average signal in the Y-axis direction and the reference rotational motion average signal in the Y-axis direction; Figure 15 Reference diagram for the original speckle motion signal in the X direction, the original speckle motion average signal in the X direction and the reference rotational motion average signal in the Y-axis direction corresponding to the first sensor; Figure 16 Reference diagram for the original speckle motion signal in the X direction, the original speckle motion average signal in the X direction and the reference rotational motion average signal in the Y-axis direction corresponding to the second sensor; Figure 17 Reference diagram for reconstructing the rotational motion signal in the X-axis direction, the reconstructed rotational motion average signal in the X-axis direction and the reference rotational motion average signal in the X-axis direction; Figure 18 Reference diagram for the original speckle motion signal in the Y direction, the original speckle motion average signal in the Y direction and the reference rotational motion average signal in the X-axis direction corresponding to the first sensor; Figure 19 Reference diagram for the original speckle motion signal in the Y direction, the original speckle motion average signal in the Y direction and the reference rotational motion average signal in the X-axis direction corresponding to the second sensor; Figure 20 Principle block diagram of the detection device for the three-dimensional motion signal of the heart of the present application in some embodiments; Figure 21The block diagram of the principle of the terminal device of the present application in some embodiments. Detailed implementation manners

[0016] To make the above objects, features, and advantages of the present application more obvious and understandable, the following will describe the detailed implementation manners of the present application in conjunction with the accompanying drawings. Many specific details are set forth in the following description to fully understand the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the connotation of the present application. Therefore, the present application is not limited by the specific embodiments disclosed below.

[0017] It should be noted that the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In the description of the present application, "a plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0018] The term "exemplary" or "for example" and the like are used to represent examples, illustrations, or descriptions. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of the terms "exemplary" or "for example" and the like is intended to present the relevant concepts in a specific manner.

[0019] The term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily limit to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0020] Unless otherwise defined, all technical and scientific terms used in the specification of the present application have the same meaning as commonly understood by those skilled in the art belonging to the technical field of the present application. The terms used in the specification of the present application are only for the purpose of describing specific implementation manners and are not intended to limit the present application. The term "and / or" used in the specification of the present application includes any and all combinations of one or more of the related listed items.

[0021] The following further describes the detailed implementation manners of the present application in conjunction with the accompanying drawings.

[0022] Refer to Figure 1 As shown, it is a flowchart of a method for detecting three-dimensional motion signals of a heart provided in the first aspect of the embodiments of the present application. In Figure 1Among them, the method for detecting the three-dimensional motion signal of the heart includes: S1. When multiple sensors capture the surface of the user's chest cavity, obtain the motion signal of the speckle in the X-axis direction on the imaging plane in each sensor, and obtain the X-axis direction motion signal corresponding to each sensor.

[0023] In the embodiments of the present application, the multiple sensors include a first sensor and a second sensor. In other implementation manners, the number of the sensors can be set by those skilled in the art according to actual needs, that is, the multiple sensors are not limited to the foregoing first sensor and second sensor.

[0024] S2. While obtaining all the X-axis direction motion signals, obtain the Y-axis direction motion signal corresponding to each sensor.

[0025] S3. Use a preset algorithm to perform calculation processing on all the X-axis direction motion signals and all the Y-axis direction motion signals to obtain the vertical motion signal of the user's chest cavity surface, the rotational motion signal in the X-axis direction, and the rotational motion signal in the Y-axis direction.

[0026] Further, step S3 includes: setting a first coefficient, a second coefficient, a third coefficient, and a fourth coefficient; obtaining the distance between the lens plane and the imaging plane of the target sensor and the distance between the focal plane and the lens plane of each sensor, where the target sensor is any one of the multiple sensors; based on the first coefficient, the second coefficient, the third coefficient, the fourth coefficient, the distance between the lens plane and the imaging plane of the target sensor, the distance between the focal plane and the lens plane of each sensor, all the X-axis direction motion signals, and all the Y-axis direction motion signals, perform calculation processing according to the preset algorithm to obtain the vertical motion signal of the user's chest cavity surface, the rotational motion signal in the X-axis direction, and the rotational motion signal in the Y-axis direction.

[0027] It should be understood that the focal plane refers to the two-dimensional plane where light rays in the sensor converge to form a clear image after passing through a lens or a mirror. In the case of focusing, the focal plane should be on the surface of the imaging object.

[0028] Further, in the case where the multiple sensors include a first sensor and a second sensor, based on the first coefficient, the second coefficient, the third coefficient, the fourth coefficient, the distance between the lens plane and the imaging plane of the target sensor, the distance between the focal plane and the lens plane of each sensor, all the X-axis direction motion signals, and all the Y-axis direction motion signals, the calculation formula for the step of performing calculation processing according to the preset algorithm to obtain the vertical motion signal of the user's chest cavity surface, the rotational motion signal in the X-axis direction, and the rotational motion signal in the Y-axis direction is: ; ; ; Wherein, is the vertical motion signal corresponding to the moment; is the moment; is the distance between the lens plane and the imaging plane of the target sensor; is the third coefficient; is the distance between the focal plane and the lens plane of the first sensor; is the X-axis direction motion signal corresponding to the moment of the second sensor at ; is the fourth coefficient; is the distance between the focal plane and the lens plane of the second sensor; is the X-axis direction motion signal corresponding to the moment of the first sensor at ; is the second coefficient; is the first coefficient; is the rotational motion signal in the Y-axis direction corresponding to the moment; is the rotational motion signal in the X-axis direction corresponding to the moment; is the Y-axis direction motion signal corresponding to the moment of the first sensor at ; is the Y-axis direction motion signal corresponding to the moment of the second sensor at ;

[0029] Through the above embodiments, the embodiments of the present application set a preset algorithm, and the preset algorithm can be used to effectively decouple the vertical motion signal and the rotational motion signal of the heart. Compared with the prior art, the embodiments of the present application can not only capture the vertical motion signal of the heart, but also have high sensitivity to the rotational motion signal in the X-axis direction and the rotational motion signal in the Y-axis direction, overcoming the limitation that the radio frequency-based method is only sensitive to the vertical motion signal, thereby improving the accuracy of the three-dimensional motion signal detection of the heart.

[0030] In some embodiments, step S1 includes: when the user makes a vertical movement, acquiring the movement signals of the speckles in each sensor in the X-axis direction of the imaging plane, and obtaining a first movement signal to be processed corresponding to each sensor; when the user makes a rotational movement, acquiring the movement signals of the speckles in each sensor in the X-axis direction of the imaging plane, and obtaining a second movement signal to be processed corresponding to each sensor; performing a summation process on the first movement signal to be processed and the second movement signal corresponding to each sensor to obtain a movement signal in the X-axis direction corresponding to each sensor.

[0031] When the user makes a vertical movement, the relationship reference diagram among the laser, the first sensor, the second sensor and the user can be referred to Figure 2 as shown.

[0032] Further, when the user makes a vertical movement, the step of acquiring the movement signals of the speckles in each sensor in the X-axis direction of the imaging plane and obtaining a first movement signal to be processed corresponding to each sensor includes: when the user makes a vertical movement, acquiring the distance between the focal plane of each sensor and the thoracic surface; acquiring the initial incident angle of the laser emitted by the laser onto the thoracic surface of the user; calculating and obtaining a first movement signal to be processed corresponding to each sensor based on the distance between the focal plane of each sensor and the thoracic surface, the initial incident angle and a preset condition.

[0033] The preset condition includes a first preset condition and a second preset condition.

[0034] In the embodiments of the present application, the calculation formula for the first movement signal to be processed corresponding to the first sensor is: ; ; where is the first movement signal to be processed corresponding to the first sensor at time; is the distance between the lens plane and the imaging plane of the target sensor; is the vertical movement distance of the user at time; is the initial incident angle; is the distance between the focal plane of the first sensor and the thoracic surface; is the first preset condition; is the distance between the focal plane and the lens plane of the first sensor; is the distance between the laser emitting the laser and the thoracic surface of the user; is the initial imaging angle of the first sensor.

[0035] The target sensor may be the first sensor or the second sensor. The distance between the lens plane and the imaging plane of the first sensor is equal to the distance between the lens plane and the imaging plane of the second sensor. In other embodiments, these two distances may also be unequal, and specifically can be set by those skilled in the art according to actual needs.

[0036] The calculation formula for the first motion signal to be processed corresponding to the second sensor is: ; ; wherein, is the first motion signal to be processed corresponding to the second sensor at the moment; is the distance between the focal plane of the second sensor and the thoracic surface; is the second preset condition; is the distance between the focal plane and the lens plane of the second sensor; is the initial imaging angle of the second sensor.

[0037] It should also be understood that all the first motion signals to be processed are obtained when the multiple sensors are in a defocus state.

[0038] The magnitude of the change in the incident angle of the laser is proportional to the distance of the user's vertical movement, satisfying ; wherein, as a whole is the change in the incident angle of the laser.

[0039] When the user makes a vertical movement, the imaging angles of the multiple sensors remain unchanged.

[0040] When the user makes a rotational movement, the relationship reference diagram between the laser, the first sensor, the second sensor and the user can refer to Figure 3 as shown.

[0041] Further, when the user makes a rotational movement, the step of obtaining the movement signal of the speckle in the X-axis direction of the imaging plane in each sensor and obtaining the second movement signal to be processed corresponding to each sensor includes: when the user makes a rotational movement, obtaining the reflection angle of the laser on each sensor; calculating and obtaining the second movement signal to be processed corresponding to each sensor based on the reflection angle on each sensor, the distance between the lens plane of the target sensor and the imaging plane, the distance between the focal plane of each sensor and the thoracic surface, and the distance between the focal plane of each sensor and the lens plane.

[0042] In the embodiment of the present application, the calculation formula for the second movement signal to be processed corresponding to the first sensor is: ; wherein, is the second movement signal to be processed corresponding to the first sensor; as a whole is the reflection angle on the first sensor; is the user at the angle of rotation along the Y-axis at the moment.

[0043] When the user rotates by an angle along the Y-axis, the initial incident angle will also change by the same angle, that is , is the deviation between the angle of rotation of the user along the Y-axis at the moment and the initial incident angle.

[0044] The calculation formula for the second movement signal to be processed corresponding to the second sensor is: ; wherein, is the second movement signal to be processed corresponding to the second sensor; as a whole is the reflection angle on the second sensor.

[0045] The change in the reflection angle is caused by two parts of changes. One is caused by the rotational movement of the user itself, and the magnitude of the reflection angle is the same as the rotational angle of the user; the other is caused by the change in the illumination angle. In order to observe the same speckle pattern after the user rotates, it is necessary to ensure that the initial phase difference of the speckle patterns before and after rotation is the same (that is, the optical path difference remains unchanged). According to this constraint, the angle change caused by the change in the illumination angle needs to satisfy and .

[0046] It should be understood that when the user rotates along the Y axis, it will cause the speckle pixels to move in the X-axis direction of the imaging plane among the multiple sensors; when the user rotates along the X axis, it will cause the speckle pixels to move in the Y-axis direction of the imaging plane among the multiple sensors.

[0047] In the embodiments of the present application, the calculation formula for the X-axis direction motion signal corresponding to the first sensor is: ; where is the X-axis direction motion signal corresponding to the first sensor at moment, that is, the motion signal in the X-axis direction corresponding to the speckle on the imaging plane of the first sensor at moment.

[0048] In the calculation formula of , is the first coefficient, is the third coefficient, , .

[0049] The calculation formula for the X-axis direction motion signal corresponding to the second sensor is: ; where is the X-axis direction motion signal corresponding to the second sensor at moment, that is, the motion signal in the X-axis direction corresponding to the speckle on the imaging plane of the second sensor at moment.

[0050] In the calculation formula of , is the second coefficient, is the fourth coefficient, , .

[0051] It should also be noted that when the user makes vertical and rotational movements, the multiple sensors all capture the surface of the user's chest cavity.

[0052] In some embodiments, step S2 includes: Obtaining the angle of rotation of the user along the X axis; Calculating the Y-axis direction motion signal corresponding to each sensor based on the distance between the lens plane and the imaging plane of the target sensor, the distance between the focal plane and the lens plane of each sensor, the distance between the focal plane of each sensor and the surface of the chest cavity, the initial incident angle, and the initial imaging angle of each sensor.

[0053] In the embodiment of the present application, the calculation formula for the Y-axis direction motion signal corresponding to the first sensor is: ; where is the Y-axis direction motion signal corresponding to the first sensor at time, that is, the motion signal in the Y-axis direction corresponding to the speckle on the imaging plane of the first sensor at time; is the angle of rotation of the user along the X-axis at time.

[0054] The calculation formula for the Y-axis direction motion signal corresponding to the second sensor is: ; where is the Y-axis direction motion signal corresponding to the second sensor at time, that is, the motion signal in the Y-axis direction corresponding to the speckle on the imaging plane of the second sensor at time.

[0055] To verify the mapping relationship between the motion signals (i.e., the vertical motion signal, the rotational motion signal in the X-axis direction, and the rotational motion signal in the Y-axis direction) restored by the method and the true motion signals, the embodiment of the present application further uses a motion simulator that can precisely control the motion amplitude and frequency to evaluate the performance of the method in extracting depth motion (i.e., vertical motion). Specifically, in the embodiment of the present application, two square wave pulse signals are provided to the motion simulator through a signal generator, where one square wave pulse is used to control the motion direction (i.e., set the motion frequency), and the other square wave pulse is used to adjust the motion amplitude. Through the above control method, the motion simulator performs precise motion according to the set parameters to verify the accuracy and applicability of the method under different motion conditions.

[0056] During the implementation process of the present application, to verify the effectiveness of the method, when the motion amplitudes of the motion simulator are controlled to be 5 mm, 10 mm, and 15 mm respectively, a 120 s video signal is processed, and the corresponding motion signals are restored using the method. Specifically, as shown in Figure 4 , it is the reconstructed motion signal corresponding to a motion amplitude of 5 mm, Figure 4 the average peak-to-peak value of the reconstructed motion signal in Figure 5 is 36.878; as shown in Figure 5 , it is the reconstructed motion signal corresponding to a motion amplitude of 10 mm, Figure 6 the average peak-to-peak value of the reconstructed motion signal inFigure 5 The average peak-to-peak value of the reconstructed motion signal in Figure 5 is 115.2757. It should be understood that the average peak-to-peak value refers to Figure 4 , Figure 5 or Figure 6 the value obtained by subtracting the value corresponding to the blue line from the value corresponding to the yellow line in Figure 6 and then dividing by the number of peaks.

[0057] After adjusting the motion frequency of the motion simulator to 1 Hz, perform spectral analysis on the motion signal recovered by the method. As shown in Figure 7 , the main frequency of the obtained signal is consistent with the true motion frequency; after adjusting the motion frequency of the motion simulator to 1.5 Hz, perform spectral analysis on the motion signal recovered by the method. As shown in Figure 8 , the main frequency of the obtained signal is consistent with the true motion frequency; after adjusting the motion frequency of the motion simulator to 2 Hz, perform spectral analysis on the motion signal recovered by the method. As shown in Figure 9 , the main frequency of the obtained signal is consistent with the true motion frequency; after adjusting the motion frequency of the motion simulator to 3 Hz, perform spectral analysis on the motion signal recovered by the method. As shown in Figure 10 , the main frequency of the obtained signal is consistent with the true motion frequency. The above experimental results show that the method of the present application can effectively recover the amplitude and frequency information of the motion signal, and preliminarily verify its feasibility and accuracy in non-contact cardiac motion detection.

[0058] In some embodiments, after the step of S3, the method for detecting the three-dimensional motion signal of the heart further includes: performing normalization processing on the vertical motion signal, the rotational motion signal in the X-axis direction, and the rotational motion signal in the Y-axis direction to obtain the normalized vertical motion signal, the normalized rotational motion signal in the X-axis direction, and the normalized rotational motion signal in the Y-axis direction; performing band-pass filtering processing on the normalized vertical motion signal, the normalized rotational motion signal in the X-axis direction, and the normalized rotational motion signal in the Y-axis direction to obtain the filtered vertical motion signal, the filtered rotational motion signal in the X-axis direction, and the filtered rotational motion signal in the Y-axis direction.

[0059] The filtering range of the band-pass filtering processing is 0.5 - 50 Hz. In other embodiments, those skilled in the art can set the specific value of the filtering range according to actual needs, and this is not further limited herein.

[0060] The implementation of the above steps can remove the interference of noise in the signal and improve the signal quality.

[0061] In some embodiments, after the steps of performing band-pass filtering on the standardized vertical motion signal, the standardized rotational motion signal in the X-axis direction, and the standardized rotational motion signal in the Y-axis direction to obtain the filtered vertical motion signal, the filtered rotational motion signal in the X-axis direction, and the filtered rotational motion signal in the Y-axis direction, the method for detecting the three-dimensional motion signal of the heart further includes: using the R peak in the electrocardiogram signal as the time reference, detecting the AO peak (the AO peak refers to the reflected wave peak when the aortic valve closes) in the filtered vertical motion signal, the Gi peak in the filtered rotational motion signal in the X-axis direction, and the Gj peak in the filtered rotational motion signal in the Y-axis direction, and calculating the interval between adjacent heartbeats (i.e., the IBI value, where IBI refers to the time interval between two consecutive heartbeats). Based on the AO peak, the Gi peak, the Gj peak, and the IBI value, perform periodic segmentation on the filtered vertical motion signal, the filtered rotational motion signal in the X-axis direction, and the filtered rotational motion signal in the Y-axis direction, where the starting point of the i-th cycle in the filtered vertical motion signal is defined as AOi+1 - 0.3×IBIi, and the ending point is defined as AOi+1 + 0.7×IBIi+1, AOi is the AO peak of the i-th cycle, and IBIi is the IBI value of the i-th cycle. Finally, perform interpolation normalization on the n signal cycles in each video record to a unified length and calculate their average waveforms to improve the stability and comparability of the signals.

[0062] The Gi peak is the first main peak in the rotational motion signal in the X-axis direction, and the Gi peak occurs approximately simultaneously with the mitral valve closure; the Gj peak is the main maximum peak in the rotational motion signal in the Y-axis direction, and the Gj peak is related to the aortic valve opening event.

[0063] Reference Figures 11 - 19 is an example of the signal average waveform, the original speckle motion average signal, and the reference signal average signal reconstructed by the method described in the embodiments of the present application for a user. In Figures 11 - 19 , for the SCG signal (i.e., the vertical motion signal) and the GCGy signal (i.e., the rotational motion signal in the Y-axis direction), the reconstructed waveform is closer to the average waveform of the reference signal in multiple detailed features, preliminarily verifying the effectiveness of the method proposed in the embodiments of the present application in separating the SCG signal and the GCGy signal from the original speckle motion signal. In addition, for the GCGx signal (i.e., the rotational motion signal in the X-axis direction), from the perspective of theoretical analysis, the rotational motion in this direction can be completely reproduced through the motion characteristics of the speckle along the Y-axis. Therefore, according to the method principle, the reconstructed signal should be highly consistent with the original speckle motion signal. The experimental results further verify this theoretical expectation. Figures 11 - 19The comparison of the visualization results shows that the waveform characteristics of the reconstructed signal in the GCGx signal direction are consistent with the waveform trend of the original speckle motion signal, further demonstrating the reliability of the method proposed in the embodiments of the present application in separating different motion components.

[0064] In the embodiments of the present application, the Pearson correlation coefficient between the average waveform of the reconstructed signal, the average signal of the original speckle motion, and the average signal of the reference signal is calculated to quantify the accuracy of the method in three-dimensional cardiac motion reconstruction. Specifically, according to the band independence of laser interference, the embodiments of the present application use the R band and the G band to verify the repeatability of the experiment at two adjacent positions respectively, and calculate the correlation index based on the data of 10 users (about 20,000 cardiac cycles). The experimental results are shown in Table 1.

[0065] Table 1

[0066] According to the data in Table 1, it can be seen that the method has achieved high reconstruction accuracy for both the SCG signal and the GCGy signal. Among them, the reconstruction effect of the SCG signal is particularly significant, that is, the correlation between the reconstructed signal (i.e., the reconstructed vertical motion signal in Table 1) and the original speckle X-axis motion signal (i.e., the original speckle motion signals in the X direction corresponding to the first sensor and the second sensor in Table 1) has the most obvious improvement, and the Pearson correlation coefficient with the reference signal reaches a relatively high level (R band: 0.6269, G band: 0.5889), proving that the present application can effectively extract and separate different motion components of the heart, especially showing superiority in the extraction of the vertical motion component (motion depth). For the reconstruction result in the GCGx direction, as mentioned above, compared with the original speckle Y-axis motion signal (i.e., the original speckle motion waveforms in the Y direction corresponding to the first sensor and the second sensor in Table 1), the Pearson correlation coefficient of the reconstructed signal with the reference signal is similar, which meets the theoretical expectation and further supports the effectiveness of the present application in decoupling different motion components. It is particularly worth noting that consistent trends are shown at the positions of both the R and G bands, which further improves the credibility of the experimental results and proves the robustness and applicability of the present application.

[0067] In view of the fact that the small time deviation between the average signals may lead to a decrease in the Pearson correlation coefficient, thus affecting the accuracy of the correlation analysis, the present application further introduces a waveform similarity index to more comprehensively quantify the effectiveness of the reconstructed signal, thereby improving the reliability of the experimental results. The definition of the similarity index is as follows: ; where is the similarity index between the reference signal and the reconstructed signal, or the similarity index between the reference signal and the original speckle motion signal; is the reconstructed signal or the original speckle motion signal; is the reference signal; is the product of the maximum height of the reference signal and its length; is the dynamic time warping distance between the reference signal and the reconstructed signal, or the dynamic time warping distance between the reference signal and the original speckle motion signal.

[0068] According to the band independence of laser interference, the present application uses the R band and the G band to verify the repeatability of experiments at two adjacent positions respectively, and calculates the similarity index based on the data of 10 users (about 20,000 cardiac cycles). The experimental results are shown in Table 2.

[0069] Table 2

[0070] The data analysis in Table 2 shows that the reconstruction effect of the SCG signal is the most significant. The similarity indices of the reconstructed signal (i.e., the reconstructed vertical motion signal in Table 2) reach 93.288% and 92.809% under the R band and the G band respectively, which are significantly higher than those of the original speckle motion signal, further verifying the superiority of the present application in extracting the vertical motion component (motion depth). For the GCGy signal, the similarity indices of the reconstructed signal (i.e., the reconstructed rotational motion signal in the Y-axis direction in Table 2) are also higher than those of the original signal, reaching 88.380% (R band) and 89.557% (G band) respectively, indicating that the motion reconstruction ability of the present application in this direction is also reliable. At the same time, the similarity indices of the reconstructed signal in the GCGx direction (i.e., the reconstructed rotational motion signal in the X-axis direction in Table 2) and the reference signal reach 92.372% and 90.170% under the R band and the G band respectively, and are very similar to the performance of the original speckle Y-axis motion signal (i.e., the original speckle motion waveforms in the Y direction corresponding to the first sensor and the second sensor in Table 2), which is consistent with the theoretical expectation, further verifying the effectiveness of the present application in separating different motion components.

[0071] In summary, the experimental results fully prove the accurate reconstruction ability of the present application for different motion components, and show consistency under the R band and the G band, further improving the stability and applicability of the method.

[0072] Reference Figure 20 As shown, it is the principle block diagram of a detection device for three-dimensional motion signals of the heart provided in the second aspect of the embodiment of the present application. In Figure 20 it, the detection device 100 for three-dimensional motion signals of the heart includes: The X-axis direction motion signal acquisition module 101 is configured to obtain the motion signals of the speckles in each sensor in the X-axis direction on the imaging plane when multiple sensors photograph the chest surface of the user, and obtain the X-axis direction motion signals corresponding to each sensor. The Y-axis direction motion signal acquisition module 102 is configured to obtain the Y-axis direction motion signals corresponding to each sensor while obtaining all the X-axis direction motion signals. The calculation module 103 is configured to perform calculation processing on all the X-axis direction motion signals and all the Y-axis direction motion signals by using a preset algorithm to obtain the vertical motion signal, the rotational motion signal in the X-axis direction, and the rotational motion signal in the Y-axis direction of the chest surface of the user.

[0073] In the third aspect of the embodiments of the present application, a terminal device is provided. The principle block diagram of the terminal device may be as shown in Figure 21 the figure. The terminal device includes a processor, a memory, a network interface, a display screen, and a temperature sensor connected through a system bus. Among them, the processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for detecting three-dimensional motion signals of the heart is implemented. The display screen may be a liquid crystal display screen or an electronic ink display screen, and the temperature sensor is pre-set inside the terminal device to detect the operating temperature of the internal device.

[0074] Those skilled in the art can understand that Figure 21 the principle block diagram shown in is only a block diagram of some structures related to the solution of the present invention, and does not constitute a limitation on the terminal device to which the solution of the present invention is applied. The specific terminal device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0075] In some embodiments, the embodiments of the present application provide a terminal device. The terminal device includes a processor and a memory. The memory is used to store a computer program. The processor is used to call and run the computer program stored in the memory and execute the steps of the method for detecting three-dimensional motion signals of the heart provided in the first aspect of the embodiments of the present application. In the fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided. The computer-readable storage medium is used to store a computer program. The computer program enables a computer to execute the steps of the method for detecting three-dimensional motion signals of the heart provided in the first aspect of the embodiments of the present application.

[0076] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided by the present invention can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0077] Without changing the basic principle of the present application, the technical features of the above embodiments can be combined. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0078] The above embodiments only represent several implementation manners of the present application, and their descriptions are relatively specific and detailed, but they should not be construed as a limitation on the scope of the patent application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the patent protection scope of the present application should be subject to the appended claims.

Claims

1. A method for detecting three-dimensional motion signals of the heart, characterized in that, Including: When multiple sensors capture the surface of the user's chest cavity, obtain the motion signals of the speckles in each sensor in the X-axis direction on the imaging plane, and obtain the X-axis direction motion signals corresponding to each sensor; While obtaining all the X-axis direction motion signals, obtain the Y-axis direction motion signals corresponding to each sensor; Use a preset algorithm to perform calculation processing on all the X-axis direction motion signals and all the Y-axis direction motion signals to obtain the vertical motion signal, the rotational motion signal in the X-axis direction, and the rotational motion signal in the Y-axis direction of the surface of the user's chest cavity.

2. The detection method of the three-dimensional motion signal of the heart according to claim 1, wherein, The step of using a preset algorithm to perform calculation processing on all the X-axis direction motion signals and all the Y-axis direction motion signals to obtain the vertical motion signal, the rotational motion signal in the X-axis direction, and the rotational motion signal in the Y-axis direction of the surface of the user's chest cavity includes: Set the first coefficient, the second coefficient, the third coefficient, and the fourth coefficient; Obtain the distance between the lens plane and the imaging plane of the target sensor and the distance between the focal plane and the lens plane of each sensor, where the target sensor is any one of the multiple sensors; Based on the first coefficient, the second coefficient, the third coefficient, the fourth coefficient, the distance between the lens plane and the imaging plane of the target sensor, the distance between the focal plane and the lens plane of each sensor, all the X-axis direction motion signals, and all the Y-axis direction motion signals, perform calculation processing according to the preset algorithm to obtain the vertical motion signal, the rotational motion signal in the X-axis direction, and the rotational motion signal in the Y-axis direction of the surface of the user's chest cavity.

3. The method for detecting three-dimensional motion signals of the heart according to claim 2, wherein When the multiple sensors include a first sensor and a second sensor, based on the first coefficient, the second coefficient, the third coefficient, the fourth coefficient, the distance between the lens plane and the imaging plane of the target sensor, the distance between the focal plane and the lens plane of each sensor, all the X-axis direction motion signals, and all the Y-axis direction motion signals, the calculation formula for the step of performing calculation processing according to the preset algorithm to obtain the vertical motion signal, the rotational motion signal in the X-axis direction, and the rotational motion signal in the Y-axis direction of the surface of the user's chest cavity is: ； ； ； Wherein, is the vertical motion signal corresponding to the moment; is the moment; is the distance between the lens plane and the imaging plane of the target sensor; is the third coefficient; is the distance between the focal plane and the lens plane of the first sensor; is the X-axis direction motion signal of the second sensor at the moment; is the fourth coefficient; is the distance between the focal plane and the lens plane of the second sensor; is the X-axis direction motion signal of the first sensor at the moment; is the second coefficient; is the first coefficient; is the rotational motion signal in the Y-axis direction corresponding to the moment; is the rotational motion signal in the X-axis direction corresponding to the moment; is the Y-axis direction motion signal of the first sensor at the moment; is the Y-axis direction motion signal of the second sensor at the moment.

4. The detection method of the three-dimensional motion signal of the heart according to claim 2, characterized in that When multiple sensors capture the surface of the user's chest cavity, the step of obtaining the motion signals of the speckles in each sensor in the X-axis direction on the imaging plane and obtaining the X-axis direction motion signals corresponding to each sensor includes: When the user makes a vertical motion, obtain the motion signals of the speckles in each sensor in the X-axis direction on the imaging plane, and obtain the first motion signal to be processed corresponding to each sensor; When the user makes a rotational motion, obtain the motion signals of the speckles in each sensor in the X-axis direction on the imaging plane, and obtain the second motion signal to be processed corresponding to each sensor; Perform a summation process on the first motion signal to be processed and the second motion signal to be processed corresponding to each sensor to obtain the X-axis direction motion signals corresponding to each sensor.

5. The detection method of the three-dimensional motion signal of the heart according to claim 4, characterized in that, When the user makes a vertical movement, the steps of obtaining the movement signal of the speckle in the X-axis direction of the imaging plane in each sensor and obtaining the first movement signal to be processed corresponding to each sensor include: When the user makes a vertical movement, obtain the distance between the focal plane of each sensor and the thoracic surface; Obtain the initial incident angle of the laser emitted by the laser irradiating the thoracic surface of the user; Based on the distance between the focal plane of each sensor and the thoracic surface, the initial incident angle, and a preset condition, calculate and obtain the first movement signal to be processed corresponding to each sensor.

6. The detection method of the three-dimensional motion signal of the heart according to claim 5, characterized in that, When the user makes a rotational movement, the steps of obtaining the movement signal of the speckle in the X-axis direction of the imaging plane in each sensor and obtaining the second movement signal to be processed corresponding to each sensor include: When the user makes a rotational movement, obtain the reflection angle of the laser on each sensor; Based on the reflection angle on each sensor, the distance between the lens plane and the imaging plane of the target sensor, the distance between the focal plane of each sensor and the thoracic surface, and the distance between the focal plane of each sensor and the lens plane, calculate and obtain the second movement signal to be processed corresponding to each sensor.

7. The method for detecting the three-dimensional motion signal of the heart according to claim 5, wherein, When obtaining all the movement signals in the X-axis direction, the steps of obtaining the movement signal in the Y-axis direction corresponding to each sensor include: Obtain the angle of the user's rotation along the X-axis; Based on the distance between the lens plane and the imaging plane of the target sensor, the distance between the focal plane of each sensor and the lens plane, the distance between the focal plane of each sensor and the thoracic surface, the initial incident angle, and the initial imaging angle of each sensor, calculate the movement signal in the Y-axis direction corresponding to each sensor.

8. A detection device for three-dimensional motion signals of the heart, characterized in that, Include: An X-axis direction movement signal acquisition module, configured to, when multiple sensors capture the thoracic surface of the user, obtain the movement signal of the speckle in the X-axis direction of the imaging plane in each sensor, and obtain the movement signal in the X-axis direction corresponding to each sensor; A Y-axis direction movement signal acquisition module, configured to, when obtaining all the movement signals in the X-axis direction, obtain the movement signal in the Y-axis direction corresponding to each sensor; A calculation module, configured to perform calculation processing on all the movement signals in the X-axis direction and all the movement signals in the Y-axis direction by using a preset algorithm to obtain the vertical movement signal, the rotational movement signal in the X-axis direction, and the rotational movement signal in the Y-axis direction of the thoracic surface of the user.

9. A terminal device, characterized in that, Include: A processor and a memory, where the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the steps of the method for detecting the three-dimensional movement signal of the heart according to any one of claims 1 to 7 above.

10. A computer-readable storage medium, characterized in that, For storing a computer program, the computer program causes a computer to execute the steps of the method for detecting the three-dimensional movement signal of the heart according to any one of claims 1 to 7 above.

Images