A method for monitoring abnormal posture of patients during rehabilitation training

Abstract

The present invention relates to the field of data processing technology, and specifically to a method for monitoring abnormal posture of patients during rehabilitation training, comprising: monitoring abnormal posture monitoring results through a deep learning neural network, judging whether the trained deep learning neural network can accurately monitor posture, and removing specific joints in a posture image, comparing the changes in the monitoring results of the deep learning neural network before and after removal, obtaining the interference degree of the joints on the monitored posture, and further combining the interference degree of each joint with its adjacent joints to obtain the monitoring weight of the joints when the deep learning neural network detects posture; finally, collecting the posture image of the patient, and monitoring the patient's posture according to the monitoring weight of the joints in the posture image of the patient. The present invention analyzes the influence of each joint point on the monitored posture, thereby assigning a monitoring weight to each joint point and monitoring the patient's posture, thereby improving the accuracy of posture monitoring.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method for monitoring abnormal body posture of a patient during rehabilitation training. Background Art

[0002] The present application is a method for monitoring abnormal posture of patients during rehabilitation training, which is a method for monitoring whether the posture of people of all ages (including the elderly and children) is abnormal; when traditionally monitoring whether the patient's posture is abnormal through a deep learning neural network, the monitoring weight of each joint point is the same during the training of the deep learning neural network, but in actual posture abnormalities, specific posture abnormalities will only cause changes in related joint points, that is, when monitoring various posture abnormalities, the monitoring weights of each joint point are different, which may cause the traditional deep learning neural network to be unable to accurately monitor various posture abnormalities. Summary of the invention

[0003] The present invention provides a method for monitoring abnormal posture of patients during rehabilitation training to solve the existing problem that the traditional deep learning neural network F-CNN cannot accurately monitor various abnormal postures.

[0004] A method for monitoring abnormal posture of a patient during rehabilitation training of the present invention adopts the following technical solution:

[0005] The following steps are involved:

[0006] Acquire all posture images, and use a deep learning neural network to monitor posture information in all posture images, the posture information including but not limited to: whether the spine is abnormal and whether the cervical spine is abnormal; obtain the accuracy of the deep learning neural network in evaluating various posture information; and determine whether the deep learning neural network can accurately evaluate various posture information based on the accuracy of the deep learning neural network in evaluating various posture information;

[0007] Classify all posture images according to the monitoring results of the deep learning neural network to obtain a set of posture images; after removing the joints in the posture images, use the deep learning neural network to monitor the posture images after the joints are removed to obtain the monitoring results and accuracy; according to the changes in the monitoring results, obtain the influence of the joints on the evaluation of posture information; according to the influence of the joints on the evaluation of posture information, combined with the changes in the accuracy, obtain the interference degree of the joints in evaluating the posture information;

[0008] Acquire the adjacent joint points of each joint point, classify the adjacent joint points according to the interference degree of the posture information evaluated by the adjacent joint points, and acquire the interference points and auxiliary points of the joint points; evaluate the interference degree of the posture information according to the interference points and auxiliary points of the joint points, and acquire the influence of the joint points on the posture information when evaluating the posture information; acquire the corrected interference degree of the posture information evaluated by the joint points in combination with the interference degree of the posture information evaluated by the joint points according to the influence of the adjacent joint points when evaluating the posture information; acquire the monitoring weight of the posture information evaluated by the joint points according to the corrected interference degree;

[0009] Collect patient posture images, evaluate the monitoring weight of posture information based on joint points, assign monitoring weights to each joint point in the collected patient posture images, and monitor the patient's posture.

[0010] Preferably, the method of obtaining the accuracy of the deep learning neural network in evaluating various body posture information includes:

[0011] For any posture information, the deep learning neural network F-CNN is used to monitor the posture information in all posture images, and the monitoring result of the deep learning neural network F-CNN on the posture information in all posture images is obtained; if the monitoring result of the deep learning neural network F-CNN on the posture information in the posture image is the same as the posture information in the posture image corresponding to the posture image, then the monitoring result of the deep learning neural network F-CNN on the posture information in the posture image is correct; otherwise, the monitoring result is wrong;

[0012] The ratio of the number of correct posture images of the deep learning neural network's monitoring results for the posture information to the number of posture images corresponding to the posture information monitored by the deep learning neural network F-CNN is used as the accuracy of the posture information evaluated by the deep learning neural network F-CNN.

[0013] Preferably, the specific method of determining whether the deep learning neural network can accurately evaluate various body posture information includes:

[0014] For any body information, preset a correctness threshold , if the accuracy of the posture information evaluated by the deep learning neural network F-CNN is greater than or equal to , then the deep learning neural network F-CNN can accurately evaluate the body information. If the accuracy of the deep learning neural network F-CNN evaluation is less than , then the deep learning neural network F-CNN cannot accurately evaluate the body posture information.

[0015] Preferably, the obtaining of a plurality of posture image sets includes the following specific methods:

[0016] For any posture information, according to the monitoring result of the posture information evaluated by the deep learning neural network F-CNN, all posture images are divided into: a set of posture images that can be correctly monitored by the deep learning neural network F-CNN, a set of posture images that the deep learning neural network F-CNN recognizes abnormal posture information as normal posture information, and a set of posture images that the deep learning neural network F-CNN recognizes normal posture information as abnormal posture information, which are respectively denoted as Class collection, Class collections and A collection of classes.

[0017] Preferably, the acquisition of the influence of the joint points on evaluating the posture information includes the following specific methods:

[0018] For any joint point when evaluating any posture information, Class Collections and The joint points in all posture images in the class set are removed, and the joint points are removed. Class collection and removal of the joint point Class collections are denoted as Class collection, Class collection;

[0019] Monitoring using deep learning neural network F-CNN Class Collections and The body information in all posture images in the class set is obtained by deep learning neural network F-CNN monitoring Class Collections and The accuracy of the body information in all posture images in the class set is calculated, and the deep learning neural network F-CNN monitoring Class Collections and The set of correct posture images consisting of the monitoring results of the posture information in all posture images in the class set is recorded as Class collection, deep learning neural network F-CNN monitoring Class Collections and The set of posture images with incorrect monitoring results of the posture information in all posture images in the class set is recorded as Class collection;

[0020] Get Class Collections and The similarity between sets of classes, and Class Collections and The similarity between class sets is calculated as follows:

[0021]

[0022] In the formula, express Class Collections and Similarity between sets of classes; express Class collection; express Class collection; express Class Collections and Similarity between sets of classes; express Class collection; express Class collection; Represents a union operation; represents the intersection operation;

[0023] according to Class Collections and similarities and differences between sets of classes, and Class Collections and The similarity and difference between the class sets are used to obtain the influence of the joint points in the under-learning to evaluate the posture information. The specific calculation formula is:

[0024]

[0025] In the formula, Indicates the influence degree of the joint point in under-learning in evaluating the posture information; express Class Collections and Similarity between sets of classes; express Class Collections and Similarity between sets of classes; express The number of pose images in the class set; express The number of pose images in the class set; express The number of pose images in the class set; express The number of pose images in the class collection.

[0026] Preferably, the obtaining of the interference degree of the joint point assessment body posture information includes the following specific methods:

[0027] For any joint point when evaluating any posture information, the difference between the accuracy of the posture information evaluated by the deep learning neural network F-CNN of all posture images before removing the joint point and the accuracy of the posture information evaluated by the deep learning neural network F-CNN after removing the joint point is taken as the accuracy change; the product of the influence degree of the joint point in evaluating the posture information in the under-learning and the accuracy change is taken as the interference degree of the joint point in evaluating the posture information in the under-learning.

[0028] Preferably, the obtaining of the interference points and auxiliary points of the joint points includes the following specific methods:

[0029] Any joint point when evaluating any body posture information is recorded as a target joint point, and all joint points adjacent to the target joint point are obtained and recorded as adjacent joint points of the target joint point; the interference degree of the body posture information evaluated by all joint points is used as a metric distance, and all joint points are clustered into three clusters using a k-means clustering algorithm; the mean interference degree of all relevant nodes in each cluster is obtained, and the adjacent joint points of the target joint point in the cluster with the largest mean interference degree are recorded as interference points of the target joint point, and the adjacent joint points of the target joint point in the cluster with the smallest mean interference degree are recorded as auxiliary points of the target joint point.

[0030] Preferably, the joint point is affected by the adjacent joint points when obtaining and evaluating the posture information, and the specific method includes:

[0031] Preset a transfer coefficient The product of the sum of the interference degrees of all interference points and all auxiliary points in the adjacent joints of the target joint in under-learning and the transfer coefficient is used as the influence of the target joint on the adjacent joints when evaluating the posture information in under-learning.

[0032] Preferably, the obtaining of the interference degree of the joint point evaluation body posture information after correction includes the following specific methods:

[0033] For any joint point when evaluating any posture information, the sum of the influence of the joint point on the posture information when evaluating the posture information in the under-learning and the interference degree of the joint point in evaluating the posture information in the under-learning is used as the corrected interference degree of the joint point in evaluating the posture information in the under-learning;

[0034] Obtain the interference degree after evaluating all posture information corrections of all relevant nodes in the learning.

[0035] Preferably, the specific method of obtaining the monitoring weights of the joint points for evaluating the posture information includes:

[0036] For any joint point when evaluating any body posture information, a pre-set under-learning monitoring weight ; Let the over-learning monitoring weight equal ; According to the interference degree of the corrected posture information evaluated by the joint points in under-learning and over-learning, combined with the under-learning and over-learning monitoring weights, the monitoring weight of the joint points evaluating the posture information is obtained, and the specific calculation formula is:

[0037]

[0038] In the formula, represents the monitoring weight of the joint point in evaluating the body posture information; represents the under-learning monitoring weight; Indicates the interference degree of the corrected posture information evaluated by the joint point in under-learning; Indicates the interference degree after the correction of the posture information evaluated by the joint points in learning; represents the linear normalization function.

[0039] The beneficial effects of the technical solution of the present invention are as follows: in actual posture abnormalities, specific posture abnormalities will only cause changes in related joints, that is, the monitoring weights of various joints are different when monitoring various posture abnormalities. In order to assign monitoring weights to each joint when evaluating various posture information, the characteristic joints are removed, and the changes in monitoring results after the joints are removed are compared to obtain the interference degree of the joints in evaluating posture information; and because the deep learning neural network monitors posture according to the relationship between multiple joints, when assigning weights to each joint, it is necessary to consider not only the interference of the joint itself in evaluating posture, but also the interference of the joints around the joints in evaluating posture, so as to assign monitoring weights to the joints; finally, the patient's posture image is collected, and according to the monitoring weights of the joints in evaluating posture information, monitoring weights are assigned to each joint in the collected patient posture image to improve the accuracy of monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 A flowchart of the steps of a method for monitoring abnormal posture of a patient during rehabilitation training according to the present invention;

[0042] Figure 2 This is an example of a posture image. DETAILED DESCRIPTION

[0043] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, features and effects of a method for monitoring abnormal posture of patients in rehabilitation training proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0044] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0045] The specific scheme of the method for monitoring abnormal posture of patients during rehabilitation training provided by the present invention is described in detail below with reference to the accompanying drawings.

[0046] See also Figure 1 , which shows a flowchart of a method for monitoring abnormal posture of a patient during rehabilitation training provided by an embodiment of the present invention, the method comprising the following steps:

[0047] Step S001: Acquire all posture images, and use a deep learning neural network to monitor the posture information in all posture images; obtain the accuracy of the deep learning neural network in evaluating various posture information; based on the accuracy of the deep learning neural network in evaluating various posture information, determine whether the deep learning neural network can accurately evaluate various posture information.

[0048] It should be noted that this embodiment is a method for monitoring abnormal posture of patients during rehabilitation training, specifically, by inputting the patient's posture image into the deep learning neural network F-CNN, so as to realize accurate and rapid monitoring of the patient's posture. However, the traditional deep learning neural network F-CNN monitors posture by extracting the patient's joints in the image, and the deep learning neural network F-CNN identifies abnormal posture according to each joint. However, in actual posture abnormalities, specific posture abnormalities will only cause changes in related joints. For example, the joints of the legs are not helpful for identifying spinal abnormalities. That is, when monitoring various posture abnormalities, the monitoring weights of each joint are different, which may cause the traditional deep learning neural network F-CNN to fail to accurately monitor various posture abnormalities. Therefore, this embodiment first obtains the accuracy of the monitoring results of the patient by acquiring the traditional deep learning neural network F-CNN, and determines whether to optimize the monitoring weights of each joint for the deep learning neural network F-CNN according to the accuracy.

[0049] Specifically, in one embodiment of the present invention, all body images are retrieved from a hospital diagnosis history database; the body images are images that have been professionally diagnosed by doctors, and each body image may contain some body information, including but not limited to: whether the spine is abnormal and whether the cervical spine is abnormal, etc.;

[0050] The human skeleton in the body image is extracted by a human posture estimation algorithm to obtain a posture image of the body image. There are several joint points in the posture image, such as Figure 2 As shown, Figure 2 is an example of a posture image; since the human posture estimation algorithm is a well-known prior art, it will not be described in detail in this embodiment;

[0051] For any posture information, the deep learning neural network F-CNN is used to monitor the posture information in all posture images to obtain the monitoring results of the deep learning neural network F-CNN on the posture information in all posture images; if the monitoring result of the deep learning neural network F-CNN on the posture information in the posture image is the same as the posture information in the posture image corresponding to the posture image, then the monitoring result of the deep learning neural network F-CNN on the posture information in the posture image is correct; otherwise, the monitoring result is wrong; since the deep learning neural network F-CNN is a well-known prior art, it will not be described in detail in this embodiment.

[0052] Furthermore, the accuracy of the deep learning neural network F-CNN in evaluating the posture information is obtained, and the method for obtaining the accuracy is: the ratio of the number of correct posture images of the monitoring results of the deep learning neural network F-CNN for the posture information to the number of posture images of the posture information monitored by the deep learning neural network F-CNN is used as the accuracy of the deep learning neural network F-CNN in evaluating the posture information, and the calculation for obtaining the accuracy is the same in this application, so it will not be repeated here.

[0053] It should be noted that the higher the accuracy of the deep learning neural network F-CNN in monitoring a certain body information, the more accurately the deep learning neural network F-CNN can monitor the body information, and the more the deep learning neural network F-CNN can monitor the body information; on the contrary, the lower the accuracy of the deep learning neural network F-CNN in monitoring a certain body information, the less accurately the deep learning neural network F-CNN can monitor the body information, that is, when detecting the body information, it is more necessary to optimize its monitoring weight so that the deep learning neural network F-CNN can accurately monitor the body information. Therefore, the accuracy of the deep learning neural network F-CNN in monitoring a certain body information can be used to evaluate the deep learning neural network F-CNN.

[0054] For any body information, preset a correctness threshold , The specific value of can be set according to the actual situation. This embodiment does not make a hard requirement. If the accuracy of the posture information evaluated by the deep learning neural network F-CNN is greater than or equal to , then the deep learning neural network F-CNN can accurately evaluate the body information. If the accuracy of the deep learning neural network F-CNN evaluation is less than , then the deep learning neural network F-CNN cannot accurately evaluate the body posture information.

[0055] Step S002: classify all posture images according to the monitoring results of the deep learning neural network to obtain a set of posture images; after removing the joint points in the posture images, use the deep learning neural network to monitor the posture images after the joint points are removed to obtain the monitoring results and accuracy; according to the changes in the monitoring results, obtain the influence of the joint points in evaluating the posture information; according to the influence of the joint points in evaluating the posture information, combined with the changes in the accuracy, obtain the interference degree of the joint points in evaluating the posture information.

[0056] It should be noted that the traditional deep learning neural network F-CNN may not be able to accurately evaluate certain posture information, that is, the monitoring weights of each joint point in the posture image are different, which leads to the inability to accurately monitor the posture. Therefore, different monitoring weights need to be assigned to different joint points. In actual posture abnormalities, specific posture abnormalities will only cause changes in related joint points, that is, when monitoring various posture abnormalities, the monitoring weights of each joint point are different. Therefore, the deep learning neural network F-CNN can be reused after removing specific joint points for re-evaluation. According to the changes in the re-evaluation results, the influence of each joint point can be obtained. The greater the change in the evaluation result, the greater the influence of the joint point on the evaluation of specific posture information. This provides a theoretical basis for the subsequent acquisition of the monitoring weight of each joint point.

[0057] Preferably, in a specific embodiment of the present invention, for any posture information, according to the monitoring result of the posture information evaluated by the deep learning neural network F-CNN, all posture images are divided into: a posture image set that can be correctly monitored by the deep learning neural network F-CNN, a posture image set that the deep learning neural network F-CNN recognizes abnormal posture information as normal posture information, and a posture image set that the deep learning neural network F-CNN recognizes normal posture information as abnormal posture information, which are respectively denoted as Class collection, Class collections and A collection of classes.

[0058] It needs to be explained that, The posture images in the class set are posture images in which the deep learning neural network F-CNN recognizes abnormal posture information as normal posture information, which is a specific manifestation of the lack of learning of the deep learning neural network F-CNN during the monitoring process; The posture images in the class set are posture images in which the deep learning neural network F-CNN identifies normal posture information as abnormal posture information, which is a specific manifestation of over-learning of the deep learning neural network F-CNN during the monitoring process; therefore, it is necessary to analyze under-learning and over-learning respectively. The specific analysis process and logic of under-learning and over-learning are the same, so this embodiment takes the analysis of under-learning as an example for description.

[0059] For any joint point when evaluating any posture information, Class Collections and The joint points in all posture images in the class set are removed, and the joint points are removed. Class collection and removal of the joint point Class collections are denoted as Class collection, A collection of classes.

[0060] Monitoring using deep learning neural network F-CNN Class Collections and The body information in all posture images in the class set is obtained by deep learning neural network F-CNN monitoring Class Collections and The accuracy of the body information in all posture images in the class set is calculated, and the deep learning neural network F-CNN monitoring Class Collections and The set of posture images with correct posture information monitoring results in all posture images in the class set is recorded as Class collection, deep learning neural network F-CNN monitoring Class Collections and The set of posture images with incorrect monitoring results of the posture information in all posture images in the class set is recorded as Class collection;

[0061] Further, obtain Class Collections and The similarity between sets of classes, and Class Collections and The similarity between class sets is calculated as follows:

[0062]

[0063] In the formula, express Class Collections and Similarity between sets of classes; express Class collection; express Class collection; express Class Collections and Similarity between sets of classes; express Class collection; express Class collection; Represents a union operation; Represents the intersection operation.

[0064] It should be noted that and It represents the similarity between the monitoring result of the body information evaluated by the deep learning neural network F-CNN after removing the joint point and the monitoring result of the body information evaluated by the deep learning neural network F-CNN without removing the joint point. The more similar the monitoring results before and after removing the joint point are, the smaller the influence of the joint point on the monitoring of the body information is.

[0065] It should be further explained that the greater the difference in the monitoring results before and after removing the joint point, the greater the influence of the joint point on monitoring the posture information. Combined with the similarity of the monitoring results before and after removing the joint point, the degree of influence of the joint point on the evaluation of the posture information can be obtained.

[0066] Specifically, according to Class Collections and similarities and differences between sets of classes, and Class Collections and The similarity and difference between the class sets are used to obtain the influence of the joint points in the under-learning to evaluate the posture information. The specific calculation formula is:

[0067]

[0068] In the formula, Indicates the influence degree of the joint point in under-learning in evaluating the posture information; express Class Collections and Similarity between sets of classes; express Class Collections and Similarity between sets of classes; express The number of pose images in the class set; express The number of pose images in the class set; express The number of pose images in the class set; express The number of pose images in the class set, Represents a linear normalization function, and the normalization range is the evaluation of all posture information of all relevant nodes in the under-learning .

[0069] It should be noted that and express Class Collections and The difference between the class sets, Class Collections and The greater the value, the greater the difference between the monitoring results before and after the joint point is removed, that is, the greater the influence of the joint point on the monitoring of the body posture information. and The larger the value of , the smaller the influence of the joint point on monitoring the posture information. Therefore, the influence degree of the joint point on evaluating the posture information can be obtained.

[0070] It should be further explained that the influence of joints on the evaluation of posture information has both positive and negative effects, that is, some joints can help improve the accuracy of evaluating specific posture features, while some joints will interfere with the evaluation of specific posture information, thereby reducing the accuracy of evaluating the posture information. Therefore, after obtaining the degree of influence of the joints in evaluating the posture information, it is necessary to further obtain the degree of interference in the evaluation of the posture information by the nodes based on the changes in the accuracy of the monitoring results before and after removing the joints.

[0071] Furthermore, for any joint point when evaluating any posture information, the difference between the accuracy of the posture information evaluated by the deep learning neural network F-CNN before removing the joint point and the accuracy of the posture information evaluated by the deep learning neural network F-CNN after removing the joint point is taken as the accuracy change; the product of the influence degree of the joint point in evaluating the posture information in the under-learning and the accuracy change is taken as the interference degree of the joint point in evaluating the posture information in the under-learning, and the specific calculation formula is:

[0072]

[0073] In the formula, Indicates the interference degree of the body posture information evaluated by the joint point in under-learning; Indicates the degree of influence of the joint points mentioned in under-learning; It indicates the accuracy of the posture information evaluated by the deep learning neural network F-CNN before removing the joint point in under-learning; It represents the accuracy of the posture information evaluated by the deep learning neural network F-CNN after removing the joint point in under-learning.

[0074] Similarly, the interference degree of the joint point evaluation body information in the learning process is obtained. Since the specific process and logic of obtaining the interference degree of the joint point evaluation body information in the learning process are the same as the interference degree of the joint point evaluation body information in the unlearned process, the interference degree of the joint point evaluation body information in step S002 is obtained. Replace the collection-like operations with Therefore, this embodiment will not be described in detail.

[0075] It should be noted that when the interference degree of the joint point in evaluating the posture information is a negative number, it means that the joint point can assist in evaluating the posture information, and when the interference degree of the joint point in evaluating the posture information is a positive number, it means that the joint point will interfere with the evaluation of the posture information.

[0076] At this point, the interference degree of all posture information evaluated by all relevant nodes in under-learning and over-learning is obtained.

[0077] Step S003: Obtain the adjacent joint points of each joint point, classify the adjacent joint points according to the interference degree of the posture information evaluated by the adjacent joint points, and obtain the interference points and auxiliary points of the joint points; evaluate the interference degree of the posture information according to the interference points and auxiliary points of the joint points, and obtain the influence of the adjacent joint points on the joint points when evaluating the posture information; according to the influence of the adjacent joint points on the joint points when evaluating the posture information, combined with the interference degree of the joint point in evaluating the posture information, obtain the corrected interference degree of the joint point evaluation posture information; according to the corrected interference degree, obtain the monitoring weight of the joint point evaluation posture information.

[0078] It should be noted that since the deep learning neural network monitors body posture based on the relationship between multiple joints, when assigning weights to each joint, it is necessary to consider not only the interference of the joint itself on the evaluation of body posture, but also the interference of the joints around the joint on the evaluation of body posture; when assigning joint monitoring weights, it is necessary to combine the degree of interference of the joint assessment posture information adjacent to the joint.

[0079] Preferably, in a specific embodiment of the present invention, any joint point when evaluating any body posture information is recorded as a target joint point, and all joint points adjacent to the target joint point are obtained and recorded as adjacent joint points of the target joint point, for example: Figure 2 The adjacent joints of the middle joint "1" are joint "0", joint "2", joint "5", joint "8" and joint "11";

[0080] Furthermore, the interference degree of the body information evaluated for all joint points is used as a metric distance, and the k-means clustering algorithm is used to cluster all joint points into three clusters. Since the k-means clustering algorithm is a well-known prior art, it will not be described in detail in this embodiment. The mean interference degree of all relevant nodes in each cluster is obtained, and the adjacent joint points of the target joint point in the cluster with the largest mean interference degree are recorded as interference points, and the adjacent joint points of the target joint point in the cluster with the smallest mean interference degree are recorded as auxiliary points.

[0081] It should be noted that when evaluating whether a certain body shape information is abnormal, not all relevant nodes have an impact on this body shape information; in order to avoid the influence of joint points that have no influence on the evaluation, this implementation screens the joint points by clustering. When the interference degree of the joint point in evaluating the body shape information is closer to 0, the joint point will less affect the evaluation of the body shape information.

[0082] Preset a transfer coefficient , The specific value of can be set according to the actual situation. This embodiment does not make a hard requirement. The sum of the interference degrees of all the interference points and all the auxiliary points in the adjacent joints of the target joint in the under-learning and the product of the transfer coefficient are used as the influence of the target joint on the adjacent joints when evaluating the posture information. The specific calculation formula is:

[0083]

[0084] In the formula, Indicates that the joint point is affected by the adjacent joint points when evaluating the posture information in under-learning; represents the preset transfer coefficient; represents the number of interference points of the joint point when evaluating the posture information in under-learning; represents the number of auxiliary points of the joint point when evaluating the posture information in under-learning; Indicates the first joint point in the under-learning interference points to evaluate the interference degree of the body posture information; Indicates the first joint point in the under-learning The interference degree of the posture information is evaluated by using auxiliary points.

[0085] Preferably, in a specific embodiment of the present invention, for any joint point when evaluating any posture information, the sum of the influence of the joint point on the posture information when evaluating the posture information in the under-learning and the interference degree of the joint point in evaluating the posture information in the under-learning is used as the interference degree of the joint point in evaluating the posture information after correction, and the specific calculation formula is:

[0086]

[0087] In the formula, Indicates the interference degree of the corrected posture information evaluated by the joint point in under-learning; Indicates that the joint point is affected by the adjacent joint points when evaluating the posture information in under-learning; It indicates the interference degree of the body posture information evaluated by the joint points in under-learning.

[0088] It should be noted that when evaluating posture information, the greater the influence of the joint point on the adjacent joint point and the greater the interference degree with the posture information evaluated by the joint point, the greater the actual interference degree of the posture information evaluated by the joint point.

[0089] At this point, the interference degree of all the relevant nodes in the under-learning after evaluating all the posture information is corrected is obtained. Similarly, the interference degree of all the relevant nodes in the over-learning after evaluating all the posture information is corrected is obtained. Since the process and logic of obtaining the interference degree of all the relevant nodes in the over-learning after evaluating all the posture information is corrected are the same as those of obtaining the interference degree of all the relevant nodes in the under-learning after evaluating all the posture information is corrected, they will not be repeated in this embodiment.

[0090] It should be noted that under-learning is due to the fact that the monitoring weight assigned by the deep learning neural network is too small, so a large monitoring weight needs to be assigned, while over-learning is due to the fact that the monitoring weight assigned by the deep learning neural network is too large, so a small monitoring weight needs to be assigned, which can be used to assign monitoring weights to each joint point.

[0091] Preferably, in a specific embodiment of the present invention, for any joint point when evaluating any posture information, a pre-set under-learning monitoring weight in the range of (0.5, 1) is , The specific value of can be set according to the actual situation. This embodiment does not make a rigid requirement. Take the example to describe; let the over-learning monitoring weight equal to ; According to the interference degree of the corrected posture information evaluated by the joint points in under-learning and over-learning, combined with the under-learning and over-learning monitoring weights, the monitoring weight of the joint points evaluating the posture information is obtained, and the specific calculation formula is:

[0092]

[0093] In the formula, represents the monitoring weight of the joint point in evaluating the body posture information; represents the under-learning monitoring weight; Indicates the interference degree of the corrected posture information evaluated by the joint point in under-learning; Indicates the interference degree after the correction of the posture information evaluated by the joint points in learning; represents a linear normalization function, whose normalization range is the evaluation of the body information by all relevant nodes .

[0094] At this point, the monitoring weights of all relevant nodes evaluating all body posture information are obtained.

[0095] Step S004: Collect the patient's posture image, and evaluate the monitoring weight of the posture information according to the joint points, assign a monitoring weight to each joint point in the collected patient posture image, and monitor the patient's posture.

[0096] It should be noted that after obtaining the monitoring weights of all relevant nodes for evaluating all posture information through step S003, the patient's posture image can be collected, and the monitoring weights of the posture information can be evaluated based on the joint points, and monitoring weights can be assigned to each joint point in the collected patient posture image to monitor the patient's posture.

[0097] Specifically, the patient is asked to stand in front of a solid color background and pose in a specific posture, and a shadowless lamp is used as a light source to collect the patient's body image. The specific posture includes but is not limited to: standing posture, sideways posture, twisting waist posture, etc.; the human skeleton and the patient's posture image in the patient's body image are extracted through a human posture estimation algorithm; since the human posture estimation algorithm is a well-known prior art, it will not be described in detail in this embodiment.

[0098] The patient's posture image and the monitoring weights of various posture information evaluated at each joint in the patient's posture image are input into the deep learning neural network F-CNN to accurately monitor the patient's posture.

[0099] At this point, this embodiment is completed.

[0100] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for monitoring abnormal posture of patients during rehabilitation training, characterized in that: The method comprises the following steps: Acquire all posture images, and use a deep learning neural network to monitor posture information in all posture images, the posture information including: whether the spine is abnormal and whether the cervical spine is abnormal; obtain the accuracy of the deep learning neural network in evaluating various posture information; and determine whether the deep learning neural network can accurately evaluate various posture information based on the accuracy of the deep learning neural network in evaluating various posture information; Classify all posture images according to the monitoring results of the deep learning neural network to obtain a set of posture images; after removing the joints in the posture images, use the deep learning neural network to monitor the posture images after the joints are removed to obtain the monitoring results and accuracy; according to the changes in the monitoring results, obtain the influence of the joints on the evaluation of posture information; according to the influence of the joints on the evaluation of posture information, combined with the changes in the accuracy, obtain the interference degree of the joints in evaluating the posture information; Acquire the adjacent joint points of each joint point, classify the adjacent joint points according to the interference degree of the posture information evaluated by the adjacent joint points, and acquire the interference points and auxiliary points of the joint points; evaluate the interference degree of the posture information according to the interference points and auxiliary points of the joint points, and acquire the influence of the joint points on the posture information when evaluating the posture information; acquire the corrected interference degree of the posture information evaluated by the joint points in combination with the interference degree of the posture information evaluated by the joint points according to the influence of the adjacent joint points when evaluating the posture information; acquire the monitoring weight of the posture information evaluated by the joint points according to the corrected interference degree; Collect patient posture images, and evaluate the monitoring weight of posture information according to joint points, assign monitoring weights to each joint point in the collected patient posture images, and monitor the patient's posture; The specific method of obtaining the interference points and auxiliary points of the joint points includes: Any joint point when evaluating any body posture information is recorded as a target joint point, and all joint points adjacent to the target joint point are obtained and recorded as adjacent joint points of the target joint point; the interference degree of the body posture information evaluated by all joint points is used as a metric distance, and all joint points are clustered into three clusters using a k-means clustering algorithm; the mean interference degree of all relevant nodes in each cluster is obtained, and the adjacent joint points of the target joint point in the cluster with the largest mean interference degree are recorded as interference points of the target joint point, and the adjacent joint points of the target joint point in the cluster with the smallest mean interference degree are recorded as auxiliary points of the target joint point.

2. A method for monitoring abnormal posture of a patient during rehabilitation training according to claim 1, characterized in that: The specific method of obtaining the accuracy of the deep learning neural network in evaluating various body posture information includes: For any posture information, the deep learning neural network F-CNN is used to monitor the posture information in all posture images, and the monitoring result of the deep learning neural network F-CNN on the posture information in all posture images is obtained; if the monitoring result of the deep learning neural network F-CNN on the posture information in the posture image is the same as the posture information in the posture image corresponding to the posture image, then the monitoring result of the deep learning neural network F-CNN on the posture information in the posture image is correct; otherwise, the monitoring result is wrong; The ratio of the number of correct posture images of the deep learning neural network's monitoring results for the posture information to the number of posture images corresponding to the posture information monitored by the deep learning neural network F-CNN is used as the accuracy of the posture information evaluated by the deep learning neural network F-CNN.

3. A method for monitoring abnormal posture of a patient during rehabilitation training according to claim 1, characterized in that: The specific method of judging whether the deep learning neural network can accurately evaluate various body posture information includes: For any body information, preset a correctness threshold , if the accuracy of the posture information evaluated by the deep learning neural network F-CNN is greater than or equal to , then the deep learning neural network F-CNN can accurately evaluate the body information. If the accuracy of the deep learning neural network F-CNN evaluation is less than , then the deep learning neural network F-CNN cannot accurately evaluate the body posture information.

4. A method for monitoring abnormal posture of a patient during rehabilitation training according to claim 1, characterized in that: The specific method of obtaining a plurality of posture image sets includes: For any posture information, according to the monitoring result of the posture information evaluated by the deep learning neural network F-CNN, all posture images are divided into: a set of posture images that can be correctly monitored by the deep learning neural network F-CNN, a set of posture images that the deep learning neural network F-CNN recognizes abnormal posture information as normal posture information, and a set of posture images that the deep learning neural network F-CNN recognizes normal posture information as abnormal posture information, which are respectively denoted as Class collection, Class collections and A collection of classes.

5. A method for monitoring abnormal posture of a patient during rehabilitation training according to claim 4, characterized in that: The specific method of obtaining the influence degree of the joint point evaluation body posture information includes: For any joint point when evaluating any posture information, Class Collections and The joint points in all posture images in the class set are removed, and the joint points are removed. Class collection and removal of the joint point Class collections are denoted as Class collection, Class collection; Monitoring using deep learning neural network F-CNN Class Collections and The body information in all posture images in the class set is obtained by deep learning neural network F-CNN monitoring Class Collections and The accuracy of the body information in all posture images in the class set is calculated, and the deep learning neural network F-CNN monitoring Class Collections and The set of correct posture images consisting of the monitoring results of the posture information in all posture images in the class set is recorded as Class collection, deep learning neural network F-CNN monitoring Class Collections and The set of posture images with incorrect monitoring results of the posture information in all posture images in the class set is recorded as Class collection; Get Class Collections and The similarity between sets of classes, and Class Collections and The similarity between class sets is calculated as follows: In the formula, express Class Collections and Similarity between sets of classes; express Class collection; express Class collection; express Class Collections and Similarity between sets of classes; express Class collection; express Class collection; Represents a union operation; represents the intersection operation; according to Class Collections and similarities and differences between sets of classes, and Class Collections and The similarity and difference between the class sets are used to obtain the influence of the joint points in the under-learning to evaluate the posture information. The specific calculation formula is: In the formula, Indicates the influence degree of the joint point in under-learning in evaluating the posture information; express Class Collections and Similarity between sets of classes; express Class Collections and Similarity between sets of classes; express The number of pose images in the class set; express The number of pose images in the class set; express The number of pose images in the class set; express The number of pose images in the class collection.

6. A method for monitoring abnormal posture of a patient during rehabilitation training according to claim 5, characterized in that: The method of obtaining the interference degree of the joint point evaluation body posture information includes: For any joint point when evaluating any posture information, the difference between the accuracy of the posture information evaluated by the deep learning neural network F-CNN of all posture images before removing the joint point and the accuracy of the posture information evaluated by the deep learning neural network F-CNN after removing the joint point is taken as the accuracy change; the product of the influence degree of the joint point in evaluating the posture information in the under-learning and the accuracy change is taken as the interference degree of the joint point in evaluating the posture information in the under-learning.

7. A method for monitoring abnormal posture of a patient during rehabilitation training according to claim 1, characterized in that: The joint point is affected by the adjacent joint points when obtaining the evaluation posture information, and the specific method includes: Preset a transfer coefficient The product of the sum of the interference degrees of all interference points and all auxiliary points in the adjacent joints of the target joint in under-learning and the transfer coefficient is used as the influence of the target joint on the adjacent joints when evaluating the posture information in under-learning.

8. A method for monitoring abnormal posture of a patient during rehabilitation training according to claim 1, characterized in that: The method of obtaining the interference degree of the joint point evaluation body posture information after correction includes: For any joint point when evaluating any posture information, the sum of the influence of the joint point on the posture information when evaluating the posture information in the under-learning and the interference degree of the joint point in evaluating the posture information in the under-learning is used as the corrected interference degree of the joint point in evaluating the posture information in the under-learning; Obtain the interference degree after evaluating all posture information corrections of all relevant nodes in the learning.

9. A method for monitoring abnormal posture of a patient during rehabilitation training according to claim 1, characterized in that: The specific method of obtaining the monitoring weight of the joint point for evaluating the posture information includes: For any joint point when evaluating any body posture information, a pre-set under-learning monitoring weight ; Let the over-learning monitoring weight equal ; According to the interference degree of the corrected posture information evaluated by the joint points in under-learning and over-learning, combined with the under-learning and over-learning monitoring weights, the monitoring weight of the joint points evaluating the posture information is obtained, and the specific calculation formula is: In the formula, represents the monitoring weight of the joint point in evaluating the body posture information; represents the under-learning monitoring weight; Indicates the interference degree of the corrected posture information evaluated by the joint point in under-learning; Indicates the interference degree after the correction of the posture information evaluated by the joint points in learning; represents the linear normalization function.

Images