Acupuncture efficacy detection method and system for facial paralysis patients based on image processing

Optimizing the differential characteristic values ​​of optical flow in patients with facial paralysis through optical flow algorithms and facial key point detection algorithms, solving the problem of inaccurate evaluation caused by facial nerve damage, and improving the accuracy of acupuncture efficacy evaluation.

CN118710644BActive Publication Date: 2024-11-15BAOJI MATERNAL & CHILD HEALTH HOSPITAL
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411188875.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-11-15
Estimated Expiration
2044-08-28

AI Technical Summary

Technical Problem

The prior art mutant expression movements caused by facial nerve damage in the acupuncture effect evaluation of facial paralysis patients lead to inaccurate differences in optical flow characteristics, affecting the accuracy of the evaluation.

Method used

The optical flow algorithm and facial key point detection algorithm are used to obtain the optical flow difference characteristic values ​​of the optical flow vector and feature images in facial videos. By constructing feature sequences and pre-sequences, the optimization factor is calculated, and the optical flow difference characteristic values ​​are optimized to improve the Assess accuracy.

Benefits of technology

By optimizing the differential characteristic value of optical flow, the accuracy of acupuncture efficacy evaluation is improved, the impact of mutation data is reduced, and more accurate results of efficacy evaluation are provided.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118710644B_ABST
    Figure CN118710644B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of image processing technology, and in particular to a method and system for detecting the efficacy of acupuncture for patients with facial paralysis based on image processing. The method comprises: obtaining an optical flow difference characteristic value corresponding to a feature image in a facial video, a feature sequence, and a preamble subsequence corresponding to the feature image in the feature sequence, and obtaining an optimization factor corresponding to the feature image according to the preamble subsequence corresponding to the feature image in the feature sequence and the optical flow difference characteristic value corresponding to each feature image in the preamble subsequence; optimizing the optical flow difference characteristic value corresponding to the feature image in the feature sequence using the optimization factor corresponding to the feature image, and obtaining an optimized optical flow difference characteristic value corresponding to the feature image in the feature sequence; obtaining an efficacy evaluation result of a patient with facial paralysis after acupuncture treatment according to the optimized optical flow difference characteristic value. The present invention can improve the accuracy of obtaining the efficacy evaluation result of a patient with facial paralysis after acupuncture treatment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and system for detecting the therapeutic effect of acupuncture on patients with facial paralysis based on image processing. Background Art

[0002] At present, acupuncture is widely used in the treatment of facial paralysis, that is, acupuncture treatment is one of the most effective methods for treating facial paralysis. Since evaluating the efficacy of facial paralysis patients after acupuncture treatment is more helpful for the formulation of subsequent treatment plans, the efficacy evaluation is currently carried out after acupuncture treatment for facial paralysis patients. In the prior art, the process of evaluating the acupuncture efficacy of facial paralysis patients is as follows: a camera is used to shoot the face of the patient after acupuncture treatment, and relevant medical staff will also give specific expression instructions during the shooting, so as to obtain a facial video of the facial paralysis patient after acupuncture treatment, and the facial video is a time-series image sequence, and then the time-series image sequence is analyzed based on the optical flow algorithm, so as to extract the optical flow difference features between the healthy side and the affected side of the facial paralysis patient, and then a comprehensive evaluation of the acupuncture efficacy is performed directly based on the extracted optical flow difference features.

[0003] However, the facial nerves of patients with facial paralysis may be damaged, and the damage to the facial nerves may cause random or sudden facial expressions on the face of patients with facial paralysis, such as sudden facial spasms. The existence of this situation will lead to inaccurate evaluation of the efficacy of facial paralysis patients after acupuncture treatment based directly on the extracted optical flow difference features. The optical flow difference features at this time refer to the optical flow difference features between the healthy side and the affected side of facial paralysis patients obtained by existing methods. Therefore, how to optimize the obtained optical flow difference features to improve the accuracy of evaluating the efficacy of facial paralysis patients after acupuncture treatment is an urgent problem to be solved. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a method and system for detecting the therapeutic effect of acupuncture on patients with facial paralysis based on image processing. The technical solutions adopted are as follows:

[0005] In a first aspect, an embodiment of the present invention provides a method for detecting the therapeutic effect of acupuncture on patients with facial paralysis based on image processing, comprising the following steps:

[0006] Acquire a facial video set, wherein the facial video set is composed of facial videos recorded by the facial paralysis patient after each acupuncture treatment;

[0007] According to the optical flow algorithm and the facial key point detection algorithm, the optical flow vector corresponding to the facial key point on the feature image in the facial video is obtained; according to the optical flow vector, the optical flow difference feature value corresponding to the feature image in the facial video is obtained, and the time sequence constructed by all the feature images in all the facial videos in the facial video set is recorded as the feature sequence;

[0008] Obtaining a preamble subsequence corresponding to a feature image in the feature sequence, and obtaining an optimization factor corresponding to the feature image in the feature sequence according to the preamble subsequence corresponding to the feature image in the feature sequence and an optical flow difference feature value corresponding to each feature image in the preamble subsequence;

[0009] Optimizing the optical flow difference characteristic values ​​corresponding to the feature images in the feature sequence using the optimization factors corresponding to the feature images in the feature sequence to obtain the optimized optical flow difference characteristic values ​​corresponding to the feature images in the feature sequence;

[0010] According to the optimized optical flow difference characteristic value, the efficacy evaluation result of the facial paralysis patient after acupuncture treatment is obtained.

[0011] In a second aspect, an embodiment of the present invention provides an acupuncture efficacy detection system for facial paralysis patients based on image processing, comprising a memory and a processor, wherein the processor executes a computer program stored in the memory to implement the above-mentioned acupuncture efficacy detection method for facial paralysis patients based on image processing.

[0012] Beneficial effects: The present invention first obtains a facial video set, obtains the optical flow vector corresponding to the facial key points on the feature image in the facial video according to the optical flow algorithm and the facial key point detection algorithm, obtains the optical flow difference eigenvalue corresponding to the feature image in the facial video according to the optical flow vector, and obtains the feature sequence; then obtains the preamble subsequence corresponding to the feature image in the feature sequence, and obtains the optimization factor corresponding to the feature image in the feature sequence according to the preamble subsequence corresponding to the feature image in the feature sequence and the optical flow difference eigenvalue corresponding to each feature image in the preamble subsequence; then optimizes the optical flow difference eigenvalue corresponding to the feature image in the feature sequence using the optimization factor corresponding to the feature image in the feature sequence, and obtains the optimized optical flow difference eigenvalue corresponding to the feature image in the feature sequence; finally, obtains the efficacy evaluation result of the facial paralysis patient after acupuncture treatment according to the optimized optical flow difference eigenvalue. The present invention can improve the accuracy of the subsequent evaluation of the efficacy of the facial paralysis patient after acupuncture treatment based on the optimized optical flow difference eigenvalue, that is, based on the optimized optical flow difference eigenvalue, it can improve the accuracy of the efficacy evaluation result of the facial paralysis patient after acupuncture treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0014] Figure 1 The present invention is a flow chart of a method for detecting the therapeutic effect of acupuncture on patients with facial paralysis based on image processing. DETAILED DESCRIPTION

[0015] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the embodiments of the present invention.

[0016] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0017] This embodiment provides a method for detecting the therapeutic effect of acupuncture on patients with facial paralysis based on image processing, which is described in detail as follows:

[0018] like Figure 1 As shown, the method for detecting the therapeutic effect of acupuncture on patients with facial paralysis based on image processing comprises the following steps:

[0019] Step S001, obtaining a facial video set, wherein the facial video set is composed of facial videos recorded by the facial paralysis patient after each acupuncture treatment.

[0020] The main purpose of this embodiment is to optimize the optical flow difference characteristics obtained between the healthy side and the affected side of the facial paralysis patient, and then based on the optimized optical flow difference characteristics, improve the accuracy of subsequent evaluation of the therapeutic effect of the facial paralysis patient after acupuncture treatment.

[0021] For the convenience of analysis, this embodiment will be analyzed by taking any facial paralysis patient as an example, that is, the facial paralysis patients appearing in this embodiment are all the same facial paralysis patients, and the facial paralysis patients in this embodiment have undergone acupuncture treatments for many times.

[0022] In order to subsequently realize accurate evaluation of the therapeutic effect of acupuncture treatment on patients with facial paralysis, the present embodiment needs to use a camera to shoot and record the patients with facial paralysis after each acupuncture treatment, and record the recorded video as the facial video of the patients with facial paralysis after each acupuncture treatment, and each frame image in each group of facial videos obtained is recorded as a facial image, and the facial image is a grayscale image.

[0023] In addition, during the process of filming and recording the facial paralysis patient, the therapist will give facial expression instructions to the facial paralysis patient, and the type and order of the facial expression instructions given by the therapist to the facial paralysis patient are the same during each recording.

[0024] In the present embodiment, the length of each recording time needs to be determined by the treating physician. For example, if at a certain moment the treating physician indicates that the patient with facial paralysis has completed all given facial expressions and no longer needs to be recorded, then the recording will be ended immediately. In addition, in the present embodiment, the resolution of the camera involved in the recording is at least 12 resolution, and during the entire recording process, not only is it required to record the entire face of the patient with facial paralysis, but it is also required that the recording environment has sufficient lighting, and the patient participating in the recording will only actively make facial expressions after the treating physician gives action instructions.

[0025] In addition, it should be noted that before filming and recording facial paralysis patients, the purpose of filming, how to use these videos, and how to protect these data will be clearly explained to the patients to ensure that they understand and agree to all terms of use. It is also required that during the video storage and processing process, all information that may identify the patient's personal identity will be removed, that is, anonymous identifiers will be used to replace names and other personal identification information, and strong encryption measures will be used to protect the stored and transmitted videos to ensure that only authorized personnel can access these videos, that is, to ensure that only professionals involved in treatment or research can access these videos, and set up strict access control and audit tracking for all personnel accessing these videos to protect the privacy of patients.

[0026] Therefore, this embodiment obtains the facial videos recorded by the facial paralysis patient after each acupuncture treatment through the above process, and records the set of all recorded facial videos as the facial video set, that is, the number of facial videos in the facial video set is consistent with the number of times the facial paralysis patient has been treated from the start of acupuncture treatment to the current moment.

[0027] Step S002, according to the optical flow algorithm and the facial key point detection algorithm, obtain the optical flow vector corresponding to the facial key point on the feature image in the facial video; according to the optical flow vector, obtain the optical flow difference feature value corresponding to the feature image in the facial video, and record the time sequence constructed by all the feature images in all the facial videos in the facial video set as a feature sequence; obtain the preamble subsequence corresponding to the feature image in the feature sequence, and obtain the optimization factor corresponding to the feature image in the feature sequence according to the preamble subsequence corresponding to the feature image in the feature sequence and the optical flow difference feature value corresponding to each feature image in the preamble subsequence.

[0028] At present, the optical flow algorithm is usually used to analyze the acquired facial video, and the optical flow difference eigenvalues ​​corresponding to each feature image in the facial video are extracted, and then the comprehensive evaluation of the acupuncture efficacy is directly performed based on the extracted optical flow difference eigenvalues; however, the facial nerves of patients with facial paralysis may be damaged, and the damage to the facial nerves may cause facial expressions to appear randomly or without warning on the face of patients with facial paralysis, such as sudden facial spasms, and the existence of this situation will cause the time series formed by all the extracted optical flow difference eigenvalues ​​to have some more prominent or larger amplitude optical flow difference eigenvalues, and the prominent or larger amplitude optical flow difference eigenvalues ​​are also mutation data , and these mutation data are the important reasons for the inaccurate evaluation of the efficacy of facial paralysis patients after acupuncture treatment. That is, these more prominent or larger-amplitude optical flow difference eigenvalues ​​will lead to inaccurate evaluation in the subsequent evaluation of the efficacy of facial paralysis patients after acupuncture treatment. In order to improve the accuracy of efficacy evaluation, this embodiment will subsequently optimize the extracted optical flow difference eigenvalues ​​by obtaining the optimization factors corresponding to the optical flow difference eigenvalues. Subsequently, based on the optimized optical flow difference eigenvalues, the efficacy of facial paralysis patients after acupuncture treatment is accurately evaluated. That is, this embodiment can improve the accuracy of evaluating the efficacy of facial paralysis patients after acupuncture treatment based on the optimized optical flow difference eigenvalues.

[0029] Based on the above analysis, it can be known that the optimization factor is crucial to the accuracy of evaluating the therapeutic effect. In this embodiment, the parameters related to the optimization factor are the optical flow vector and the optical flow difference eigenvalue. Therefore, before obtaining the optimization factor, this embodiment needs to first obtain the optical flow vector according to the optical flow algorithm and the facial key point detection algorithm, and obtain the optical flow difference eigenvalue based on the optical flow vector. In order to facilitate understanding, this embodiment will be described by taking the acquisition process of the optical flow vector corresponding to the facial key point on the feature image in any facial video A in the facial video set as an example, specifically:

[0030] First, a facial key point detection algorithm is used to detect facial key points on the first frame of facial image in facial video A, and facial key points on the first frame of facial image in facial video A and symmetrical facial key point pairs on the first frame of facial image are obtained; in addition, this embodiment uses an AAM algorithm (active appearance model) to detect facial key points, and the AAM algorithm is an algorithm for facial key point detection.

[0031] In specific applications, the implementer needs to determine the area of ​​facial key points that need to be detected when performing facial key point detection on the first frame of facial image in facial video A based on actual conditions. For example, the facial key points that need to be detected in this example include but are not limited to facial key points in the left corner of the mouth area, facial key points in the right corner of the mouth area, facial key points in the left pupil area, facial key points in the right pupil area, facial key points in the left corner of the eye area, facial key points in the right corner of the eye area, facial key points in the left end of the eye area, facial key points in the right end of the eye area, etc.

[0032] Then, the optical flow algorithm is used to track the facial key points on the first frame of the facial image in the facial video A, and the facial key points on each feature image in the facial video A, the optical flow vectors corresponding to each facial key point on each feature image, and the symmetrical facial key point pairs on each feature image are obtained. The feature image in the facial video A refers to all facial images except the first frame of the facial image in the facial video A, that is, the optical flow vectors corresponding to each facial key point on all facial images except the first frame of the facial image in the facial video A can be obtained through the above process.

[0033] It is necessary to give an example to illustrate that, for the optical flow vector corresponding to the nth facial key point on the kth frame facial image in the facial video A, where k is not 1: the optical flow vector corresponding to the nth facial key point on the kth frame facial image is composed of the speed parameter of the nth facial key point on the k-1th frame facial image moving to the nth facial key point on the kth frame facial image, where the speed parameter includes the movement speed in the horizontal direction and the movement speed in the vertical direction, and the nth facial key point on the kth frame facial image and the nth facial key point on the k-1th frame facial image here belong to the same facial key point at different times, for example, both are facial key points of the left pupil area at different times; in addition, the k-1th frame facial image also belongs to the facial video A.

[0034] And the description of the symmetrical facial key point pairs on the image in this embodiment includes: if after facial key point detection, there is a facial key point in the left eye pupil area and the right eye pupil area on a frame of facial image respectively, then the combination of the facial key points belonging to the left eye pupil area and the facial key points belonging to the right eye pupil area on the frame of facial image is a symmetrical facial key point pair on the frame of facial image.

[0035] Therefore, through the above process, each facial key point on each feature image in each facial video in the facial video set, the optical flow vector corresponding to each facial key point on each feature image, and the symmetrical facial key point pair on each feature image can be obtained. In addition, the facial key point detection algorithm for facial images is a well-known technology, so it will not be described in detail in this embodiment; the process of tracking facial key points using an optical flow algorithm to obtain each facial key point on each feature image in the facial video, the optical flow vector corresponding to each facial key point on each feature image, and the symmetrical facial key point pair on each feature image is also a well-known technology, so it will not be described in detail in this embodiment.

[0036] Next, this embodiment will obtain the optical flow difference feature value corresponding to the feature image according to the optical flow vector corresponding to the facial key point on the feature image, and for ease of understanding, the process of obtaining the optical flow difference feature value corresponding to any feature image a in the facial video A will be described as an example, specifically:

[0037] The optical flow difference value of each symmetrical facial key point pair on the feature image a is obtained, and the average of the optical flow difference values ​​of all symmetrical facial key point pairs on the feature image a is recorded as the optical flow difference feature value corresponding to the feature image a; and the method for obtaining the optical flow difference value of the symmetrical facial key point pair on the feature image a is as follows: for any symmetrical facial key point pair a0 on the feature image a, the two facial key points in the symmetrical facial key point pair a0 are respectively recorded as the first key point and the second key point, and the Euclidean distance between the optical flow vector corresponding to the first key point and the optical flow vector corresponding to the second key point is recorded as the optical flow difference value of the symmetrical facial key point pair a0.

[0038] Therefore, this embodiment can obtain the optical flow difference characteristic values ​​corresponding to other feature images through the process of obtaining the optical flow difference characteristic values ​​corresponding to the above-mentioned feature image a, and the optical flow difference characteristic values ​​corresponding to the feature image refer to the optical flow difference characteristics between the healthy side and the affected side on the corresponding feature image.

[0039] After obtaining the optical flow difference feature value, this embodiment also needs to obtain a feature sequence, and the feature sequence refers to a time sequence constructed by all feature images in all facial videos in the facial video set, that is, the feature images in the feature sequence are arranged in chronological order according to the shooting time.

[0040] After obtaining the feature sequence, this embodiment will obtain the preamble subsequence corresponding to each feature image in the feature sequence based on the position of each feature image in the feature sequence, and the preamble subsequence is crucial for the subsequent determination of the optimization factor corresponding to each feature image. Then the process of obtaining the preamble subsequence is as follows: for any feature image B in the feature sequence, in the feature sequence, the sequence formed by all feature images located on the left side of the feature image B is recorded as the preamble subsequence corresponding to the feature image B, that is, the preamble subsequence corresponding to the feature image B is relative to the feature image B, and the feature images in the preamble subsequence are historical images of the feature image B.

[0041] Therefore, this embodiment can obtain the preamble subsequence corresponding to each feature image in the feature sequence through the above method. After obtaining the preamble subsequence, this embodiment will obtain the optimization factor corresponding to the feature image in the feature sequence according to the preamble subsequence corresponding to the feature image in the feature sequence and the optical flow difference feature value corresponding to each feature image in the preamble subsequence. The specific acquisition process of the optimization factor corresponding to the feature image in the feature sequence is:

[0042] First, the neighbor distance sequence corresponding to each facial key point on each feature image in the feature sequence is obtained, and the neighbor distance sequence is more important for the subsequent determination of the optimization factor. In addition, for the sake of ease of understanding, this embodiment will be described by taking the acquisition process of the neighbor distance sequence corresponding to any facial key point w0 on the feature image B as an example, that is, the acquisition process of the neighbor distance sequence corresponding to the facial key point w0 is:

[0043] On the feature image B, obtain the preset number of facial key points closest to the facial key point w0, and record them as the neighboring key points of the facial key point w0, and then record the set constructed by all the neighboring key points of the facial key point w0 as the neighboring key point set Cw of the facial key point w0. Then obtain the neighbor distance of each neighboring key point in the set Cw; and for any neighboring key point v in the neighboring key point set of the facial key point w0, the neighboring distance of the neighboring key point v refers to the Euclidean distance between the neighboring key point v and the facial key point w0. Then, all the neighbor key points in the set Cw are sorted in order of neighbor distances from small to large, and the sorted sequence is recorded as the neighbor key point sequence Dw corresponding to the facial key point w0. Then, the sequence formed by the neighbor distances of all the neighbor key points in the neighbor key point sequence Dw is obtained, and recorded as the neighbor distance sequence corresponding to the facial key point w0, that is, the tth neighbor distance in the neighbor distance sequence corresponding to the facial key point w0 is the neighbor distance of the tth neighbor key point in the sequence Dw.

[0044] In a specific application, the implementer needs to set the value of the preset number according to the actual situation. For example, in this embodiment, the value of the preset number is set to 5.

[0045] After obtaining the nearest neighbor distance sequence, this embodiment will determine the similarity between each feature image in the feature sequence and each image in its corresponding preceding subsequence based on the optical flow vector, preceding subsequence, and nearest neighbor distance sequence obtained above, and the similarity is an important indicator for the subsequent determination of the optimization factor; in addition, for ease of understanding, this embodiment will be described by taking the acquisition process of the similarity between the feature image B in the feature sequence and each image in its corresponding preceding subsequence as an example, that is, the acquisition process of the similarity between the feature image B and each image in its corresponding preceding subsequence is:

[0046] First, the preamble subsequence corresponding to the feature image B is recorded as the preamble subsequence C, and all the feature images in the preamble subsequence C are recorded as images to be analyzed; then, according to the optical flow vectors corresponding to the facial key points on the feature image B, the optical flow vectors corresponding to the facial key points on each image to be analyzed in the preamble subsequence C, the neighbor distance sequence corresponding to the facial key points on the feature image B, and the neighbor distance sequence corresponding to the facial key points on each image to be analyzed, the similarity between the feature image B and each image to be analyzed in the preamble subsequence C is obtained.

[0047] After obtaining the similarity between the feature image B and each image to be analyzed, the cumulative sum of the similarities between the feature image B and each image to be analyzed in the preceding subsequence C is calculated and recorded as the comprehensive similarity, and the comprehensive similarity is mainly used for the normalization process of the similarity between the feature image B and the image to be analyzed; therefore, for any image D to be analyzed in the preceding subsequence C, the ratio of the similarity between the feature image B and the image D to be analyzed to the comprehensive similarity is obtained, and the ratio of the similarity between the feature image B and the image D to be analyzed to the comprehensive similarity is recorded as the weight factor corresponding to the image D to be analyzed, and then the product of the optical flow difference eigenvalue corresponding to the image D to be analyzed and the weight factor corresponding to the image D to be analyzed is obtained, and recorded as the contribution representation value of the image D to be analyzed.

[0048] After obtaining the contribution representation value, the present embodiment calculates the cumulative sum of the contribution representation values ​​of all the images to be analyzed in the preceding subsequence C, and uses the cumulative sum of the contribution representation values ​​of all the images to be analyzed in the preceding subsequence C as the optimization factor corresponding to the feature image B; and the specific expression for obtaining the optimization factor corresponding to the feature image B is:

[0049] in, is the optimization factor corresponding to the feature image B, is the total number of images to be analyzed in the preceding subsequence C corresponding to the feature image B, is the weight factor corresponding to the mth image to be analyzed in the preceding subsequence C, is the optical flow difference feature value corresponding to the mth image to be analyzed in the previous subsequence C, is the contribution representation value of the image D to be analyzed.

[0050] And when the weight factor corresponding to the mth image to be analyzed is larger, it indicates that the optical flow difference eigenvalue corresponding to the mth image to be analyzed contributes more to the optimization factor corresponding to the feature image B. That is, when and The larger the value, the larger the optimization factor corresponding to the feature image B, and vice versa. The larger the value of the weight factor corresponding to the m-th image to be analyzed, the more similar the expression features on the feature image B are to the expression features on the m-th image to be analyzed.

[0051] In addition, this embodiment also needs to specifically analyze and describe the process of obtaining the similarity between the feature image B and each image to be analyzed in the previous subsequence C. In order to facilitate understanding, this embodiment will be described by taking the process of obtaining the similarity between the feature image B and the image to be analyzed D as an example. That is, the specific process of obtaining the similarity between the feature image B and the image to be analyzed D is:

[0052] First, all the same facial key point groups corresponding to the feature image B and the image to be analyzed D are obtained, and the specific method for obtaining the same facial key point group is: for the facial key point w0 on the feature image B, if the facial key point w1 on the image to be analyzed D and the facial key point w0 belong to the same facial key point, then the combination of the facial key point w0 and the facial key point w1 is recorded as the same facial key point group corresponding to the feature image B and the image to be analyzed D, that is, the two facial key points in the same facial key point group come from images of different frames and belong to the same facial key point.

[0053] For example: if there is only one facial key point in the left pupil area on all feature images, and the facial key point w0 on feature image B refers to the facial key point in the left pupil area on feature image B, and the facial key point w1 on the image D to be analyzed refers to the facial key point in the left pupil area on image D to be analyzed, then the combination of facial key point w0 and facial key point w1 is an identical facial key point group corresponding to feature image B and image D to be analyzed.

[0054] Therefore, through the above process, all the same facial key point groups corresponding to the feature image B and the image to be analyzed D can be obtained, and the set formed by all the same facial key point groups corresponding to the feature image B and the image to be analyzed D is recorded as the set to be analyzed; next, each of the same facial key point groups in the set to be analyzed is analyzed, and the comprehensive index value corresponding to each of the same facial key point groups is obtained, and the comprehensive index value is an important parameter for obtaining the similarity between the feature image B and the image to be analyzed D; in addition, for ease of understanding, this embodiment will be described below by taking the process of obtaining the comprehensive index value corresponding to the qth same facial key point group in the set to be analyzed as an example, specifically:

[0055] First, the two facial key points in the qth group of identical facial key points are respectively recorded as the first feature point and the second feature point in the qth group of identical facial key points, and then the nearest neighbor distance sequence corresponding to the first feature point is subtracted from the nearest neighbor distance sequence corresponding to the second feature point, and the new sequence obtained by subtraction is recorded as the difference sequence corresponding to the qth group of identical facial key points, all parameters in the difference sequence are recorded as initial differences, and then the absolute value of each initial difference in the difference sequence is recorded as the target difference corresponding to the corresponding initial difference, and then each difference in the difference sequence is calculated. The cumulative sum of the target differences corresponding to the initial differences is calculated, and the cumulative sum of the target differences corresponding to each initial difference in the difference sequence is recorded as the first index value corresponding to the qth group of identical facial key points; then the Euclidean distance between the optical flow vector corresponding to the first feature point and the optical flow vector corresponding to the second feature point is calculated, and recorded as the second index value corresponding to the qth group of identical facial key points; then the normalized value of the product of the first index value and the corresponding second index value corresponding to the qth group of identical facial key points is obtained, and recorded as the comprehensive index value corresponding to the qth group of identical facial key points.

[0056] Therefore, through the above process, the comprehensive index value corresponding to each identical facial key point group in the set to be analyzed can be obtained, and the set constructed by the comprehensive index values ​​corresponding to all identical facial key point groups in the set to be analyzed is recorded as the comprehensive index value set corresponding to the feature image B and the image D to be analyzed, and the mean of the comprehensive index value set corresponding to the feature image B and the image D to be analyzed is obtained, and then the mean of the comprehensive index value set is recorded as the feature difference value between the feature image B and the image D to be analyzed, and then the difference between the preset first constant and the feature difference value is obtained, and recorded as the degree of similarity between the feature image B and the image D to be analyzed.

[0057] In a specific application, the implementer needs to set the preset first constant according to the actual situation. For example, in this embodiment, the preset first constant is set to 1.

[0058] In addition, in this embodiment, the specific expression for obtaining the similarity between the feature image B and the image to be analyzed D is:

[0059] in, is the similarity between the feature image B and the image to be analyzed D, is the total number of identical facial key point groups in the set to be analyzed, Norm() is the normalization function, is the first index value corresponding to the qth identical facial key point group in the set to be analyzed, is the second index value corresponding to the qth identical facial key point group in the set to be analyzed, It is the comprehensive index value corresponding to the qth same facial key point group in the set to be analyzed.

[0060] in addition, , ,in, is the number of neighbor distances in the neighbor distance sequence corresponding to the first feature point in the qth same facial key point group, and It is also the number of neighbor distances in the neighbor distance sequence corresponding to the second feature point in the qth same facial key point group, is the zth nearest neighbor distance in the nearest neighbor distance sequence corresponding to the first feature point in the qth same facial key point group, is the zth nearest neighbor distance in the nearest neighbor distance sequence corresponding to the second feature point in the qth same facial key point group, is the nearest neighbor distance sequence The zth nearest neighbor distance and the nearest neighbor distance sequence in The absolute value of the difference between the zth nearest neighbor distances in , and the nearest neighbor distance sequence is the neighbor distance sequence corresponding to the first feature point in the qth group of the same facial key points, the neighbor distance sequence is the neighbor distance sequence corresponding to the second feature point in the qth same facial key point group, is the optical flow vector corresponding to the first feature point, is the optical flow vector corresponding to the second feature point, is the Euclidean distance between the optical flow vector corresponding to the first feature point and the optical flow vector corresponding to the second feature point in the qth same facial key point group.

[0061] And when The smaller the value of The larger the value of The larger the value of , the more similar the expression features on the feature image B are to the expression features on the image to be analyzed D, and the more similar it is, the larger the weight factor of the image to be analyzed D is. The larger the weight factor of the image to be analyzed D is, the more important the image to be analyzed D is or the greater the contribution of the optical flow difference feature value corresponding to the image to be analyzed D to the optimization factor corresponding to the feature image B is. On the contrary, when The larger the value of The smaller the value of The smaller the value of , the less similar the expression features on the feature image B are to the expression features on the image to be analyzed D, and the less similar they are, the smaller the weight factor of the image to be analyzed D is. The smaller the weight factor of the image to be analyzed D is, the less important the image to be analyzed D is or the smaller the contribution of the optical flow difference eigenvalue corresponding to the image to be analyzed D to the optimization factor of the feature image B is.

[0062] Therefore, this embodiment can obtain the optimization factor corresponding to each feature image in the feature sequence through the above process; in addition, for the feature image without a preceding subsequence, this embodiment uses 0 as the optimization factor of the corresponding feature image.

[0063] Step S003, using the optimization factor corresponding to the feature image in the feature sequence, optimize the optical flow difference eigenvalue corresponding to the feature image in the feature sequence to obtain the optimized optical flow difference eigenvalue corresponding to the feature image in the feature sequence; based on the optimized optical flow difference eigenvalue, obtain the efficacy evaluation result of the facial paralysis patient after acupuncture treatment.

[0064] In step S002 of the present embodiment, the optimization factor corresponding to the feature image has been obtained. Next, based on the obtained optimization factor, the present embodiment optimizes the optical flow difference characteristic value corresponding to the feature image to obtain the optimized optical flow difference characteristic value or the optimized optical flow difference characteristic value. The specific process of obtaining the optimized optical flow difference characteristic value is as follows: for the feature image B in the feature sequence, the value obtained by adding the optical flow difference characteristic value corresponding to the feature image B and the optimization factor corresponding to the feature image B is recorded as the optimized optical flow difference characteristic value corresponding to the feature image B.

[0065] In this embodiment, based on the optimization factor corresponding to each feature image, the optical flow difference eigenvalue corresponding to the corresponding feature image is optimized, which can eliminate some mutation data in the optical flow difference eigenvalue sequence as much as possible, thereby improving the accuracy of evaluating the therapeutic effect of facial paralysis patients after acupuncture treatment; in addition, mutation data refers to some more prominent or larger-amplitude optical flow difference eigenvalues ​​in the optical flow difference eigenvalue sequence, and the more prominent or larger-amplitude optical flow difference eigenvalues ​​are caused by the sudden appearance of facial expressions on the face of facial paralysis patients randomly or without warning, and the optical flow difference eigenvalue sequence at this time refers to the time series sequence formed by the optical flow difference eigenvalues ​​corresponding to all the feature images in this embodiment.

[0066] Therefore, through the above process, the optimized optical flow difference feature value corresponding to each feature image in the feature sequence can be obtained, and then the optimized sequence is obtained in this embodiment, and the optimized sequence is a time series sequence constructed by the optimized optical flow difference features corresponding to all feature images in the feature sequence.

[0067] After obtaining the optimized sequence, this embodiment uses the least squares method to perform linear fitting on all optimized optical flow difference features in the optimized sequence to obtain a fitting straight line and obtain the slope of the fitting straight line; and the process of linear fitting using the least squares method is a well-known technology, so it will not be described in detail.

[0068] Because under normal circumstances, as the number of acupuncture treatments increases, the optical flow difference characteristic value between the healthy side and the affected side of the facial paralysis patient should become smaller and smaller, that is, the smaller the optical flow difference characteristic value, the lower the degree of facial paralysis of the patient, so usually the slope of the fitting straight line is negative; and when the slope of the fitting straight line is negative, the steeper the fitting straight line, that is, the smaller the slope of the fitting straight line, the better the therapeutic effect of the facial paralysis patient after acupuncture treatment, so when the slope of the fitting straight line is negative, this embodiment performs a negative adjustment on the slope of the fitting straight line. Normalization is performed, and the normalized slope is recorded as the characteristic slope value, and the difference between the preset first constant 1 and the characteristic slope value is recorded as the efficacy evaluation value of the facial paralysis patient after acupuncture treatment; and if the slope of the fitting line is a negative number, then the flatter the fitting line, the larger the slope of the fitting line, and the steeper the fitting line, the smaller the slope of the fitting line; and if the slope of the fitting line is a negative number, and the flatter the slope of the fitting line, the closer the normalized slope value is to 1, and the steeper the slope of the fitting line, the closer the normalized slope value is to 0.

[0069] Therefore, based on the above analysis, it can be seen that the larger the efficacy evaluation value is, the better the treatment effect of the treatment process from the beginning of acupuncture treatment to the end of the last acupuncture treatment for patients with facial paralysis; conversely, the smaller the efficacy evaluation value is, the worse the treatment effect of the treatment process from the beginning of acupuncture treatment to the end of the last acupuncture treatment for patients with facial paralysis, and the last acupuncture treatment refers to the acupuncture treatment closest to the current moment; in addition, the slope of the fitting straight line may be a positive number. When the slope is a positive number, it indicates that the treating physician may have made a mistake in the treatment plan when performing acupuncture treatment on patients with facial paralysis, and timely inspection or adjustment is required.

[0070] The present embodiment provides a system for detecting the efficacy of acupuncture and moxibustion on patients with facial paralysis based on image processing, comprising a memory and a processor, wherein the processor executes a computer program stored in the memory to implement the above-mentioned method for detecting the efficacy of acupuncture and moxibustion on patients with facial paralysis based on image processing.

[0071] In summary, the present embodiment first obtains a facial video set, obtains the optical flow vector corresponding to the facial key points on the feature image in the facial video according to the optical flow algorithm and the facial key point detection algorithm, obtains the optical flow difference feature value corresponding to the feature image in the facial video according to the optical flow vector, and obtains the feature sequence; then obtains the preamble subsequence corresponding to the feature image in the feature sequence, and obtains the optimization factor corresponding to the feature image in the feature sequence according to the preamble subsequence corresponding to the feature image in the feature sequence and the optical flow difference feature value corresponding to each feature image in the preamble subsequence; then optimizes the optical flow difference feature value corresponding to the feature image in the feature sequence using the optimization factor corresponding to the feature image in the feature sequence, and obtains the optimized optical flow difference feature value of the feature image in the feature sequence; finally, according to the optimized optical flow difference feature value, obtains the efficacy evaluation result of the facial paralysis patient after acupuncture treatment. Based on the optimized optical flow difference feature value, the present embodiment can improve the accuracy of the subsequent evaluation of the efficacy of the facial paralysis patient after acupuncture treatment, that is, based on the optimized optical flow difference feature value, it can improve the accuracy of the efficacy evaluation result of the facial paralysis patient after acupuncture treatment.

[0072] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for detecting the efficacy of acupuncture for patients with facial paralysis based on image processing, characterized in that: The method comprises the following steps: Acquire a facial video set, wherein the facial video set is composed of facial videos recorded by the facial paralysis patient after each acupuncture treatment; According to the optical flow algorithm and the facial key point detection algorithm, the optical flow vector corresponding to the facial key point on the feature image in the facial video is obtained; according to the optical flow vector, the optical flow difference feature value corresponding to the feature image in the facial video is obtained, and the time sequence constructed by all the feature images in all the facial videos in the facial video set is recorded as the feature sequence; Obtaining a preamble subsequence corresponding to a feature image in the feature sequence, and obtaining an optimization factor corresponding to the feature image in the feature sequence according to the preamble subsequence corresponding to the feature image in the feature sequence and an optical flow difference feature value corresponding to each feature image in the preamble subsequence; Optimizing the optical flow difference characteristic values ​​corresponding to the feature images in the feature sequence using the optimization factors corresponding to the feature images in the feature sequence to obtain the optimized optical flow difference characteristic values ​​corresponding to the feature images in the feature sequence; According to the optimized optical flow difference characteristic value, the efficacy evaluation result of the facial paralysis patient after acupuncture treatment is obtained; The method for obtaining the optimization factor corresponding to the feature image in the feature sequence includes: Obtaining a neighbor distance sequence corresponding to each facial key point on each feature image in the feature sequence; For any feature image B in the feature sequence: The preamble subsequence corresponding to the feature image B is recorded as the preamble subsequence C, and all images in the preamble subsequence C are recorded as images to be analyzed; according to the optical flow vectors corresponding to the facial key points on the feature image B, the optical flow vectors corresponding to the facial key points on each image to be analyzed in the preamble subsequence C, the neighbor distance sequence corresponding to the facial key points on the feature image B, and the neighbor distance sequence corresponding to the facial key points on each image to be analyzed, the similarity between the feature image B and each image to be analyzed is obtained; The cumulative sum of the similarities between the feature image B and each image to be analyzed in the preceding subsequence C is recorded as the comprehensive similarity; For any image D to be analyzed in the preceding subsequence C, the ratio of the similarity between the feature image B and the image D to be analyzed to the comprehensive similarity is recorded as the weight factor corresponding to the image D to be analyzed, and the product of the optical flow difference feature value corresponding to the image D to be analyzed and the weight factor corresponding to the image D to be analyzed is recorded as the contribution representation value of the image D to be analyzed; The cumulative sum of the contribution representation values ​​of all the images to be analyzed in the preceding subsequence C is recorded as the optimization factor corresponding to the feature image B; The method for obtaining the nearest neighbor distance sequence corresponding to each facial key point on each feature image in the feature sequence comprises: For any facial key point w0 on the feature image B, on the feature image B, a set constructed by a preset number of facial key points closest to the facial key point w0 is recorded as a neighbor key point set of the facial key point w0, all facial key points in the neighbor key point set are recorded as neighbor key points, and the Euclidean distance between each neighbor key point in the neighbor key point set and the facial key point w0 is recorded as the neighbor distance of each neighbor key point in the neighbor key point set, all neighbor key points in the neighbor key point set are sorted in ascending order of neighbor distance to obtain a neighbor key point sequence corresponding to the facial key point w0, and a sequence formed by the neighbor distances of all neighbor key points in the neighbor key point sequence is recorded as the neighbor distance sequence corresponding to the facial key point w0; The method for obtaining the similarity between the characteristic image B and each of the images to be analyzed includes: For the feature image B and the image D to be analyzed in the preceding subsequence C: Acquire all the same facial key point groups corresponding to the feature image B and the image to be analyzed D, and record the set formed by all the same facial key point groups corresponding to the feature image B and the image to be analyzed D as the set to be analyzed; For the qth identical facial key point group in the set to be analyzed, two facial key points in the qth identical facial key point group are respectively recorded as a first feature point and a second feature point, a new sequence obtained by subtracting a nearest neighbor distance sequence corresponding to the first feature point from a nearest neighbor distance sequence corresponding to the second feature point is recorded as a difference sequence, a cumulative sum of absolute values ​​of parameters in the difference sequence is recorded as a first index value corresponding to the qth identical facial key point group, a Euclidean distance between an optical flow vector corresponding to the first feature point and an optical flow vector corresponding to the second feature point is recorded as a second index value corresponding to the qth identical facial key point group, and a normalized value of the product of the first index value and the second index value is recorded as a comprehensive index value corresponding to the qth identical facial key point group; The average of the comprehensive index values ​​corresponding to all the same facial key point groups in the set to be analyzed is recorded as the characteristic difference value between the feature image B and the image D to be analyzed, and the value obtained by subtracting the preset first constant from the characteristic difference value is recorded as the similarity between the feature image B and the image D to be analyzed; The method for obtaining the same facial key point group corresponding to the feature image B and the image to be analyzed D includes: for the facial key point w0 on the feature image B, if the facial key point w1 on the image to be analyzed D and the facial key point w0 are the same facial key point, then the combination formed by the facial key point w0 and the facial key point w1 is recorded as the same facial key point group corresponding to the feature image B and the image to be analyzed D.

2. The method for detecting the therapeutic effect of acupuncture on facial paralysis patients based on image processing as claimed in claim 1, characterized in that: The method for obtaining the optical flow vector corresponding to the facial key point on the feature image in the facial video includes: Recording each image in each facial video in the facial video set as a facial image; For any facial video A in the facial video set, the facial key point detection algorithm is used to perform facial key point detection on the first frame facial image in the facial video A to obtain facial key points on the first frame facial image and symmetrical facial key point pairs on the first frame facial image; the optical flow algorithm is used to track the facial key points on the first frame facial image to obtain facial key points on each feature image in the facial video A, optical flow vectors corresponding to each facial key point on each feature image, and symmetrical facial key point pairs on each feature image, wherein the feature images in the facial video A refer to other facial images except the first frame facial image in the facial video A.

3. The method for detecting the therapeutic effect of acupuncture on facial paralysis patients based on image processing as claimed in claim 2, characterized in that: The method for obtaining the optical flow difference feature value corresponding to the feature image in the facial video includes: For any symmetrical facial key point pair a0 on any feature image a in the facial video A, two facial key points in the symmetrical facial key point pair a0 are respectively recorded as a first key point and a second key point, and a Euclidean distance between an optical flow vector corresponding to the first key point and an optical flow vector corresponding to the second key point is recorded as an optical flow difference value of the symmetrical facial key point pair a0; For the feature image a, the average of the optical flow difference values ​​of all symmetrical facial key point pairs on the feature image a is recorded as the optical flow difference feature value corresponding to the feature image a.

4. The method for detecting the therapeutic effect of acupuncture on facial paralysis patients based on image processing as claimed in claim 1, characterized in that: The method for obtaining a preceding subsequence corresponding to a feature image in the feature sequence comprises: For any feature image B in the feature sequence, in the feature sequence, a sequence formed by all feature images located on the left side of the feature image B is recorded as a preceding subsequence corresponding to the feature image B.

5. A method for detecting the therapeutic effect of acupuncture on facial paralysis patients based on image processing as claimed in claim 1, characterized in that: The method for obtaining the optimized optical flow difference feature value corresponding to the feature image in the feature sequence includes: For any feature image B in the feature sequence, the value obtained by adding the optical flow difference feature value corresponding to the feature image B and the optimization factor corresponding to the feature image B is recorded as the optimized optical flow difference feature value corresponding to the feature image B.

6. The method for detecting the therapeutic effect of acupuncture on facial paralysis patients based on image processing as claimed in claim 1, characterized in that: The method for obtaining the efficacy evaluation results of the facial paralysis patient after acupuncture treatment includes: The time series sequence constructed by the optimized optical flow difference features corresponding to all the feature images in the feature sequence is recorded as the optimized sequence, and the optimized sequence is linearly fitted to obtain a fitting straight line. The normalized value of the slope of the fitting straight line is recorded as the characteristic slope value, and the difference between the preset first constant and the characteristic slope value is recorded as the efficacy evaluation value of the facial paralysis patient after acupuncture treatment.

7. A facial paralysis patient acupuncture efficacy detection system based on image processing, comprising a memory and a processor, characterized in that: The processor executes the computer program stored in the memory to implement the method for detecting the efficacy of acupuncture for patients with facial paralysis based on image processing as described in any one of claims 1 to 6.

Citation Information

Cited By

  • Facial paralysis rehabilitation evaluation and treatment method and system based on multi-modal data fusion

    CN119700146A