Adaptive acupuncture manipulation monitoring method

Abstract

The present invention relates to the technical field of acupuncture manipulation monitoring, and proposes an adaptive acupuncture manipulation monitoring method, comprising the following steps: determining a hand posture based on an adaptive direction domain adaptation framework to capture acupuncture movements; and determining the frequency of acupuncture based on the hand posture through a sliding window algorithm.

Description

Technical Field

[0001] The present invention generally relates to the technical field of acupuncture manipulation monitoring, and in particular to an adaptive acupuncture manipulation monitoring method. Background Art

[0002] The existing technology still has the following problems in the process of monitoring acupuncture techniques:

[0003] Since existing techniques are usually based on traditional manual features, it is difficult to accurately estimate hand movements: and

[0004] There are domain differences between acupuncture technique data and data in general scenarios. Directly applying the hand posture estimation model trained in natural scenes to the acupuncture technique estimation task will inevitably cause performance loss. Summary of the invention

[0005] In order to at least partially solve the above problems in the prior art, the present invention proposes an adaptive acupuncture manipulation monitoring method, which comprises the following steps:

[0006] Determining hand poses based on an adaptive directional domain adaptation framework to capture acupuncture motion; and

[0007] The frequency of acupuncture is determined based on the hand gesture by a sliding window algorithm.

[0008] In one embodiment of the present invention, determining the hand posture based on the adaptive direction domain adaptation framework includes:

[0009] Using PeCLR (Pose Equivariant Contrastive Learning) as the benchmark framework, 3D hand poses are estimated from RGB images to generate the first hand pose dataset.

[0010] In one embodiment of the present invention, it is provided that determining the hand posture based on the adaptive direction domain adaptation framework further includes:

[0011] According to the hand orientation similarity, the first hand posture dataset is matched with the FreiHAND dataset to generate a second hand posture dataset.

[0012] In one embodiment of the present invention, it is provided that the direction in which the palm faces is represented by a ray from the wrist key point to the finger key point, and the hand orientation similarity is calculated, which is expressed as the following formula:

[0013]

[0014]

[0015] Among them, PS represents the predicted hand pose from the source domain, Represents the key point coordinates of the wrist, Represents the key point coordinates of the first joint of each finger, P t represents the predicted hand pose from the target domain, r i Represents radial lines.

[0016] In one embodiment of the present invention, it is provided that the second hand posture data set includes hand displacement data and hand movement time data.

[0017] In one embodiment of the present invention, determining the frequency of acupuncture based on the hand gesture by a sliding window algorithm includes:

[0018] filtering outliers in the second hand posture dataset;

[0019] estimating frequency peaks and troughs of hand displacement and hand movement time distribution according to the hand displacement data and hand movement time data in the second hand gesture data set; and

[0020] The frequency of acupuncture is determined based on the frequency peaks and troughs of the hand displacement and hand movement time distribution.

[0021] In one embodiment of the present invention, it is provided that interference noise in the second hand gesture data set is filtered by a Savitzky-Golay filter.

[0022] In one embodiment of the present invention, it is provided that in the sliding window algorithm, the frequency peaks and troughs of the hand displacement and the hand movement time distribution are estimated by the argrelextrema function.

[0023] The present invention further provides a monitor, characterized in that it comprises:

[0024] A controller configured to perform the following actions:

[0025] Determining hand poses based on an adaptive directional domain adaptation framework to capture acupuncture motion; and

[0026] determining the frequency of acupuncture based on the hand gesture by a sliding window algorithm; and a display configured to display the movement and frequency of acupuncture.

[0027] It should be pointed out that the present invention does not involve a method for diagnosing and treating a disease, but only provides a method for monitoring acupuncture techniques, which is an intelligent detection method. In other words, the present invention is neither intended to determine a patient's disease, nor to provide a parameter or indicator for diagnosing a disease, nor is it a method for initial screening of a disease. On the contrary, the information provided by the scheme of the present invention cannot be used for the diagnosis and treatment of a disease, but the corresponding diagnosis and treatment should be provided to the user by the hospital / doctor.

[0028] The present invention has at least the following beneficial effects: the present invention proposes an adaptive acupuncture manipulation monitoring method, which can determine the hand posture based on an adaptive directional domain adaptation framework to capture the movement of acupuncture; and determine the frequency of acupuncture based on the hand posture through a sliding window algorithm. A unified framework is used to estimate the amplitude and frequency of acupuncture in real time. The present invention can use a large amount of unlabeled hand acupuncture video data to improve the performance of amplitude estimation. Compared with the prior art, the present invention has a significant improvement in the accuracy of acupuncture movement and frequency monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] To further illustrate the advantages and features of various embodiments of the present invention, a more detailed description of various embodiments of the present invention will be presented with reference to the accompanying drawings. It will be understood that these drawings only depict typical embodiments of the present invention and are therefore not to be considered as limiting the scope thereof. In the accompanying drawings, for clarity, identical or corresponding parts will be represented by identical or similar reference numerals.

[0030] Figure 1 A schematic flow chart of an adaptive acupuncture manipulation monitoring method in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0031] It should be noted that the components in the various figures may be shown exaggeratedly for the sake of illustration and are not necessarily true to scale. In the various figures, identical or functionally identical components are provided with the same reference numerals.

[0032] In the present invention, unless otherwise specified, "arranged on...", "arranged above..." and "arranged above..." do not exclude the existence of an intermediate between the two. In addition, "arranged on or above..." merely indicates the relative positional relationship between two components, and in certain cases, such as after reversing the product direction, it can also be converted into "arranged below or below...", and vice versa.

[0033] In the present invention, each embodiment is only intended to illustrate the aspects of the present invention and should not be construed as limiting.

[0034] In the present invention, unless otherwise specified, the quantifiers "a", "an" and "an" do not exclude the presence of a plurality of elements.

[0035] It should also be noted that in the embodiments of the present invention, for the sake of clarity and simplicity, only a portion of the parts or components may be shown, but those of ordinary skill in the art will understand that under the teachings of the present invention, the required parts or components may be added according to the needs of the specific scenario. In addition, unless otherwise specified, the features in different embodiments of the present invention may be combined with each other. For example, a feature in the second embodiment may be used to replace a corresponding or functionally identical or similar feature in the first embodiment, and the resulting embodiment also falls within the disclosure scope or recorded scope of the present application.

[0036] It should also be noted that within the scope of the present invention, the terms "same", "equal", "equal to" and the like do not mean that the values ​​of the two are absolutely equal, but allow a certain reasonable error, that is, the terms also cover "substantially the same", "substantially equal", "substantially equal to". By analogy, in the present invention, the terms "perpendicular to", "parallel to" and the like indicating directions also cover the meanings of "substantially perpendicular to" and "substantially parallel to".

[0037] In addition, the numbering of the steps of the methods of the present invention does not limit the execution order of the method steps. Unless otherwise specified, the method steps can be executed in different orders.

[0038] The present invention will be further described below in conjunction with specific embodiments with reference to the accompanying drawings.

[0039] Figure 1 FIG. 2 is a flow chart showing an adaptive acupuncture manipulation monitoring method according to an embodiment of the present invention. Figure 1 As shown, the method may include the following steps:

[0040] Step 101, determining a hand posture based on an adaptive direction domain adaptation framework to capture acupuncture movements; and

[0041] Step 102: Determine the frequency of acupuncture based on the hand gesture by using a sliding window algorithm.

[0042] The technical solution of the present invention is further described below in conjunction with specific implementation methods.

[0043] The inventors found during the research that the monitoring of acupuncture techniques can be performed based on capturing the movement of acupuncture and obtaining the frequency of acupuncture.

[0044] Since the movement of the thumb and the needle during acupuncture are synchronized, the movement of the needle can be captured by monitoring the changes in hand posture.

[0045] Firstly, the present invention proposes an Adaptive Orientation Domain Adaptation (AODA) framework to obtain a first hand posture dataset, during which PeCLR can be used as a reference framework to estimate 3D hand posture from a single RGB image.

[0046] The basic idea of ​​PeCLR is to use a large amount of unlabeled data to pre-train the backbone through equivalent contrast learning, because the task of hand pose estimation essentially requires equivalence under affine transformations (such as rotation and translation). PeCLR provides a strong baseline and backbone for hand pose estimation, while there is still a domain shift between the FreiHAND dataset and the first hand pose dataset.

[0047] Since it is difficult to directly estimate a hand gesture with high accuracy, and there is currently a high-accuracy hand gesture dataset such as the FreiHAND dataset, and there is a domain transfer between the FreiHAND dataset and the first hand gesture dataset, the first hand gesture dataset can be matched with the FreiHAND dataset according to the hand orientation similarity to generate a second hand gesture dataset with high accuracy.

[0048] Specifically, according to the hand orientation similarity, the first hand posture dataset is matched with the FreiHAND dataset to generate a second hand posture dataset with high accuracy.

[0049] The direction the palm faces is represented by the ray from the wrist key point to the finger key point, and the hand orientation similarity is calculated, which is expressed as the following formula:

[0050]

[0051]

[0052] Among them, p S represents the predicted hand pose from the source domain, Represents the key point coordinates of the wrist, Represents the key point coordinates of the first joint of each finger, P t represents the predicted hand pose from the target domain, r i Represents radial lines.

[0053] By calculating the cosine similarity in the above directions, a function for calculating similarity can be obtained to select samples that are more suitable for the method of the present invention in the feature space.

[0054] The operation of searching for extreme values ​​can be performed on an array of a fixed window size by a sliding window algorithm. In the present invention, the fixed window can start from the first element in the array and continuously move backward until the last signal point. The frequency of acupuncture can be estimated using these extreme values.

[0055] Determining the frequency of acupuncture based on the hand gesture by a sliding window algorithm may include:

[0056] filtering outliers in the second hand posture dataset;

[0057] estimating frequency peaks and troughs of hand displacement and hand movement time distribution according to the hand displacement data and hand movement time data in the second hand gesture data set; and

[0058] The frequency of acupuncture is determined based on the frequency peaks and troughs of the hand displacement and hand movement time distribution.

[0059] In the process of estimating the frequency of acupuncture by sliding window algorithm, Savitzky-Golay filter can be used as a filtering algorithm for signal smoothing to suppress the influence of noise interference. This is one of the best noise data smoothing methods based on local least squares polynomial approximation.

[0060] In addition, the frequency peaks and troughs of the hand displacement and hand movement time distribution can be estimated by using the argrelextrema function, which is a function that calculates the local maximum or minimum value of the distribution based on a sliding window algorithm.

[0061] The present invention also provides a monitor, comprising:

[0062] A controller configured to perform the following actions:

[0063] Determining hand poses based on an adaptive directional domain adaptation framework to capture acupuncture motion; and

[0064] determining the frequency of acupuncture based on the hand gesture by a sliding window algorithm; and a display configured to display the movement and frequency of acupuncture.

[0065] Compared with PeCLR, the error of posture estimation is improved by 10% and the accuracy of frequency estimation can reach 98.6%.

[0066] Although various embodiments of the present invention are described above, it should be understood that they are presented as examples only and not as limitations. It is obvious to those skilled in the relevant art that various combinations, modifications and changes can be made thereto without departing from the spirit and scope of the present invention. Therefore, the breadth and scope of the present invention disclosed herein should not be limited by the exemplary embodiments disclosed above, but should only be defined according to the attached claims and their equivalents.

Claims

1. An adaptive acupuncture technique monitoring method, characterized in that: The following steps are involved: Determine hand poses based on an adaptive directional domain adaptation framework to capture acupuncture motions; as well as determining the frequency of acupuncture based on the hand gesture by a sliding window algorithm; The hand posture determination based on the adaptive direction domain adaptation framework includes: Using PeCLR as a reference framework, estimating 3D hand poses from RGB images to generate a first hand pose dataset; and matching the first hand posture dataset with the FreiHAND dataset according to the hand orientation similarity to generate a second hand posture dataset; Determining the frequency of acupuncture based on the hand gesture by a sliding window algorithm includes: filtering outliers in the second hand posture dataset; estimating frequency peaks and troughs of hand displacement and hand movement time distribution according to the hand displacement data and hand movement time data in the second hand gesture data set; and The frequency of acupuncture is determined based on the frequency peaks and troughs of the hand displacement and hand movement time distribution.

2. The adaptive acupuncture manipulation monitoring method according to claim 1, characterized in that: The direction the palm faces is represented by the ray from the wrist key point to the finger key point, and the hand orientation similarity is calculated, which is expressed as the following formula: Among them, P S represents the predicted hand pose from the source domain, Represents the key point coordinates of the wrist, Represents the key point coordinates of the first joint of each finger, P t represents the predicted hand pose from the target domain, r i Represents radial lines.

3. The adaptive acupuncture technique monitoring method according to claim 2, characterized in that: The second hand posture data set includes hand displacement data and hand movement time data.

4. The adaptive acupuncture manipulation monitoring method according to claim 3, characterized in that: The interference noise in the second hand gesture dataset is filtered by a Savitzky-Golay filter.

5. The adaptive acupuncture manipulation monitoring method according to claim 4, characterized in that: In the sliding window algorithm, the frequency peaks and troughs of the hand displacement and hand movement time distribution are estimated by the argrelextrema function.

6. A system for adaptive acupuncture manipulation monitoring method according to any one of claims 1-5, characterized in that: include: A controller configured to perform the following actions: Determine hand poses based on an adaptive directional domain adaptation framework to capture acupuncture motions; as well as determining the frequency of acupuncture based on the hand gesture by a sliding window algorithm; as well as A display configured to display acupuncture motion and frequency.

Images