Rehabilitation action recognition method and device based on image recognition

Abstract

The invention relates to the technical field of action recognition, and discloses a rehabilitation action recognition method and device based on image recognition, and the method comprises the steps: presetting a lateral bending angle range and a buckling angle range for a finger through the combination of a sensor and an image processing technology; the angle offset, the lateral bending angle range and the buckling angle range are calculated based on the real-time angle value of each finger, so that a standard evaluation value is calculated, and the normalization of rehabilitation actions is determined. According to the comprehensive method, the precision and reliability of rehabilitation action monitoring are improved, and the rehabilitation effect and experience of the patient are remarkably improved. By calculating the standard evaluation value, the action standardization of each finger can be quantified, and doctors and patients can be helped to better understand the rehabilitation progress. Finally, according to the method, the workload of medical staff is relieved, the working efficiency is improved, the risk of postoperative complications can be remarkably reduced, the patient is promoted to recover health more quickly, and the method has wide application prospects and actual value.

Description

Technical Field

[0001] The present invention relates to the technical field of motion recognition, and in particular to a method and device for recognizing rehabilitation motion based on image recognition. Background Art

[0002] Coronary Angiography is a common medical procedure used to diagnose and treat cardiovascular disease. The procedure involves inserting a catheter into the patient's blood vessels and guiding it to the coronary arteries of the heart, injecting a contrast agent to display detailed images of the blood vessels under X-rays, thereby helping doctors assess the condition of the blood vessels and perform necessary treatment. After surgery, in order to prevent bleeding, a tourniquet is usually used on the wrist to compress and stop bleeding. However, wearing a tourniquet for a long time may cause poor blood circulation in the hands, which in turn may cause numbness, swelling and other discomfort symptoms in the fingers. To avoid these problems, patients need to perform finger activities regularly to promote blood circulation and restore hand function.

[0003] Most hospitals use on-site supervision by doctors or nurses to guide patients in hand and finger movements. Medical staff are required to supervise on-site after each operation, which increases the manpower burden of the hospital. Due to the limited time of medical staff, it is impossible to monitor the rehabilitation progress of each patient in real time, which may cause some patients to fail to receive effective rehabilitation training in time. The judgment of medical staff may be subjective to a certain extent, and it is impossible to accurately quantify whether the patient's rehabilitation movements are standardized, which affects the rehabilitation effect. The lack of systematic data recording and analysis methods makes it difficult to comprehensively track and evaluate the patient's rehabilitation process. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a method for recognizing rehabilitation movements based on image recognition, comprising: Step 1, installing a sensor on the patient's finger to monitor the lateral bending and / or flexion angle of the finger in real time; Step 2, obtaining an image sequence of the patient's finger movements through a camera; Step 3, preprocessing the image sequence; Step 4, segmenting the finger area in the preprocessed image sequence to obtain a finger image sequence; Step 5, before the rehabilitation exercise begins, the finger image sequence is analyzed to determine the initial position of the finger, and the initial position is used as the standard position; Step 6, after the rehabilitation exercise begins, the finger is lateral bent and / or flexed from the standard position and then returns to the reset position; a sensor is used to collect a time series angle value of the finger being lateral bent and / or flexed from the standard position and then returning to the reset position; Step 7, presetting a scoliosis angle range for finger scoliosis and a flexion angle range for finger flexion, for determining the standardization of the rehabilitation exercise; the scoliosis angle range includes a maximum scoliosis angle and a minimum scoliosis angle, and the flexion angle range includes a maximum flexion angle and a minimum flexion angle; Step 8, collecting the real-time angle value of each finger at the current moment through the sensor, and calculating the angle offset according to the real-time angle value and the standard position; Step 9, calculate the standard evaluation value according to the angle offset, scoliosis angle range and flexion angle range, and determine the standardization of the rehabilitation movement according to the standard evaluation value.

[0005] Furthermore, when the midline of each finger is in the same straight line as the midline of the palm, the midline position of the current finger is taken as the standard position, and the angle value of the standard position is 0°.

[0006] Further, a flexion angle range is defined for the thumb, and the flexion angle range is [10°, 20°].

[0007] Further, a lateral bending angle range is defined for the index finger, middle finger, ring finger, and little finger, and the lateral bending angle range is [20°, 60°].

[0008] Furthermore, the calculation formula of the normative evaluation value is: ; Where F represents the standard evaluation value of the five finger rehabilitation movements, represents the angle offset of the i-th finger, Represents the angle value of the reset position in the time series angle value of the i-th finger, represents the first adjustment factor of the angle offset of the i-th finger, The second adjustment factor representing the angle value of the reset position in the time series angle value of the i-th finger.

[0009] Furthermore, the time series angle values ​​include standard position angle values, lateral bending and / or flexion angle values ​​and reset angle values ​​during the finger rehabilitation movement. The reset angle value range is [3°, 8°], and the reset angle value range includes a maximum reset angle value and a minimum reset angle value.

[0010] Further, ; In the formula, represents the maximum lateral bending and / or flexion angle, represents the angle value of the i-th finger at the current time t, Represents the minimum lateral bending and / or flexion angle.

[0011] Further, ; In the formula, Represents the minimum angle value of the reset position, Represents the maximum angle value of the reset position.

[0012] A rehabilitation action recognition device based on image recognition includes the following modules: Angle monitoring module: used to install sensors on the patient's fingers to monitor the angle values ​​of lateral bending and / or flexion of the fingers in real time; Image sequence acquisition module: used to acquire the image sequence of the patient's finger movement through a camera; Preprocessing module: connected with the image sequence acquisition module, used for preprocessing the image sequence; Segmentation module: connected to the preprocessing module, used to segment the finger area in the preprocessed image sequence to obtain a finger image sequence; Standard position determination module: connected to the segmentation module, used to analyze the finger image sequence to determine the initial position of the finger before the rehabilitation exercise begins, and use the initial position as the standard position; Angle value acquisition module: connected to the standard position determination module, used for collecting the time series angle value of the finger returning to the reset position after the finger is bent and / or flexed from the standard position after the rehabilitation exercise begins; Standardization determination module: connected to the angle value acquisition module, used to preset a scoliosis angle range for finger scoliosis and a flexion angle range for finger flexion, so as to determine the standardization of rehabilitation exercise; the scoliosis angle range includes a maximum scoliosis angle and a minimum scoliosis angle, and the flexion angle range includes a maximum flexion angle and a minimum flexion angle; Angle offset calculation module: connected to the standardization determination module, used to collect the real-time angle value of each finger at the current moment through the sensor, and calculate the angle offset according to the real-time angle value and the standard position; Normative evaluation module: connected with the angle offset calculation module and the normative determination module, used to calculate the normative evaluation value according to the angle offset, scoliosis angle range and flexion angle range, and determine the normativeness of the rehabilitation movement according to the normative evaluation value.

[0013] The embodiments of the present invention have the following technical effects: The present invention combines sensors with image processing technology to preset a lateral bending angle range and a flexion angle range for the fingers, and calculates the angle offset, lateral bending angle range and flexion angle range based on the real-time angle value of each finger, thereby calculating the standard evaluation value to determine the standardization of rehabilitation movements. This comprehensive method not only improves the accuracy and reliability of rehabilitation movement monitoring, but also significantly improves the rehabilitation effect and experience of patients. By calculating the standard evaluation value, the standardization of the movements of each finger can be quantified to help doctors and patients better understand the progress of rehabilitation. Ultimately, this method not only reduces the workload of medical staff and improves work efficiency, but also significantly reduces the risk of postoperative complications and promotes patients to recover faster. It has broad application prospects and practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 is a flow chart of a rehabilitation action recognition method based on image recognition provided by an embodiment of the present invention; Figure 2 It is a structural diagram of a rehabilitation movement recognition device based on image recognition provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0017] Figure 1 is a flow chart of a method for recognizing rehabilitation movements based on image recognition provided by an embodiment of the present invention. Figure 1 , including: Step 1: Install a sensor on the patient's finger to monitor the lateral bending and / or flexion angle of the finger in real time.

[0018] Lateral bending refers to the bending of the index finger, middle finger, ring finger and little finger towards the center of the palm. Flexion refers to the bending of the index finger and thumb towards the center of the palm.

[0019] Step 2: Acquire an image sequence of the patient's finger movements through a camera.

[0020] Fix the camera in a stable position so that it covers the patient's hand area and ensures that the lighting is sufficient and even without interfering shadows. Turn on the camera and start recording a video, or capture a sequence of images frame by frame. 25-30 frames per second can be captured to ensure smooth movements and capture of details.

[0021] Step 3: preprocess the image sequence.

[0022] Preprocessing includes: Denoising: Use Gaussian filter or other denoising algorithms to remove noise from the image and improve image quality; Enhance contrast: Use histogram equalization or adaptive histogram equalization (CLAHE) to enhance the contrast of the image and make the finger outline clearer.

[0023] Step 4: segment the finger area in the preprocessed image sequence to obtain a finger image sequence.

[0024] The Canny edge detection algorithm is used to extract the finger contour for subsequent segmentation. The finger area is separated from the background using the binarization method to generate a black and white image, where white represents the finger area and black represents the background. Morphological operations such as dilation and erosion are used to remove small noise points and smooth the finger contour. Contour detection is used to find all connected areas and filter out the finger area based on features such as area and shape.

[0025] Step 5: Before the rehabilitation exercise begins, the finger image sequence is analyzed to determine the initial position of the finger, and the initial position is used as the standard position.

[0026] When the midline of each finger is in the same straight line with the midline of the palm, the midline position of the current finger is taken as the standard position, and the angle value of the standard position is 0°.

[0027] Step 6, after the rehabilitation exercise begins, the finger is lateral bent and / or flexed from the standard position and then returns to the reset position; a sensor is used to collect a time series angle value of the finger being lateral bent and / or flexed from the standard position and then returning to the reset position; The time series angle values ​​include the standard position angle value, lateral bending and / or flexion angle value and reset angle value of each finger during the rehabilitation movement. The reset angle value range is [3°, 8°], and the reset angle value range includes the maximum reset angle value and the minimum reset angle value.

[0028] Step 7, presetting a scoliosis angle range for finger scoliosis and a flexion angle range for finger flexion, for determining the standardization of the rehabilitation exercise; the scoliosis angle range includes a maximum scoliosis angle and a minimum scoliosis angle, and the flexion angle range includes a maximum flexion angle and a minimum flexion angle; The flexion angle range was defined for the thumb, and the flexion angle range was [10°, 20°].

[0029] The lateral bending angle range is defined for the index finger, middle finger, ring finger, and little finger, and the lateral bending angle range is [20°, 60°].

[0030] Step 8, collecting the real-time angle value of each finger at the current moment through the sensor, and calculating the angle offset according to the real-time angle value and the minimum value of the scoliosis angle range and / or the flexion angle range.

[0031] Step 9, calculate the standard evaluation value according to the angle offset, scoliosis angle range, and flexion angle range, and determine the standardization of the rehabilitation movement according to the standard evaluation value.

[0032] ; Where F represents the standard evaluation value of the five finger rehabilitation movements, represents the angle offset of the i-th finger, Represents the angle value of the reset position in the time series angle value of the i-th finger, represents the first adjustment factor of the angle offset of the i-th finger, Represents a first adjustment factor of the angle value of the reset position in the time series angle value of the i-th finger.

[0033] Further: ; In the formula, represents the maximum lateral bending and / or flexion angle, represents the angle value of the i-th finger at the current time t, Represents the minimum lateral bending and / or flexion angle.

[0034] ; In the formula, Represents the minimum angle value of the reset position, Represents the maximum angle value of the reset position.

[0035] In order to determine the standardization of rehabilitation movements, a threshold is preset for the standard evaluation value. Considering that the patient is in the recovery period, the threshold is set to 0.5, and rehabilitation movements with standard evaluation values ​​greater than 0.5 are defined as standard movements. Rehabilitation movements with standard evaluation values ​​less than or equal to 0.5 are defined as non-standard movements. At this time, doctors need to intervene to supervise and urge patients to perform rehabilitation movements.

[0036] like Figure 2 As shown, this embodiment also discloses a rehabilitation action recognition device based on image recognition, including the following modules: Angle monitoring module: used to install sensors on the patient's fingers to monitor the angle values ​​of lateral bending and / or flexion of the fingers in real time; Image sequence acquisition module: used to acquire the image sequence of the patient's finger movement through a camera; Preprocessing module: connected with the image sequence acquisition module, used for preprocessing the image sequence; Segmentation module: connected to the preprocessing module, used to segment the finger area in the preprocessed image sequence to obtain a finger image sequence; Standard position determination module: connected to the segmentation module, used to analyze the finger image sequence to determine the initial position of the finger before the rehabilitation exercise begins, and use the initial position as the standard position; Angle value acquisition module: connected to the standard position determination module, used for collecting the time series angle value of the finger returning to the reset position after the finger is bent and / or flexed from the standard position after the rehabilitation exercise begins; Standardization determination module: connected to the angle value acquisition module, used to preset a scoliosis angle range for finger scoliosis and a flexion angle range for finger flexion, so as to determine the standardization of rehabilitation exercise; the scoliosis angle range includes a maximum scoliosis angle and a minimum scoliosis angle, and the flexion angle range includes a maximum flexion angle and a minimum flexion angle; Angle offset calculation module: connected to the standardization determination module, used to collect the real-time angle value of each finger at the current moment through the sensor, and calculate the angle offset according to the real-time angle value and the standard position; Normative evaluation module: connected with the angle offset calculation module and the normative determination module, used to calculate the normative evaluation value according to the angle offset, scoliosis angle range and flexion angle range, and determine the normativeness of the rehabilitation movement according to the normative evaluation value.

Claims

1. A method for recognizing rehabilitation movements based on image recognition, characterized in that: include: Step 1, installing a sensor on the patient's finger to monitor the lateral bending and / or flexion angle of the finger in real time; Step 2, obtaining an image sequence of the patient's finger movements through a camera; Step 3, preprocessing the image sequence; Step 4, segmenting the finger area in the preprocessed image sequence to obtain a finger image sequence; Step 5, before the rehabilitation exercise begins, the finger image sequence is analyzed to determine the initial position of the finger, and the initial position is used as the standard position; Step 6, after the rehabilitation exercise begins, the finger is lateral bent and / or flexed from the standard position and then returns to the reset position; a sensor is used to collect a time series angle value of the finger being lateral bent and / or flexed from the standard position and then returning to the reset position; Step 7, presetting a scoliosis angle range for finger scoliosis and a flexion angle range for finger flexion, for determining the standardization of the rehabilitation exercise; the scoliosis angle range includes a maximum scoliosis angle and a minimum scoliosis angle, and the flexion angle range includes a maximum flexion angle and a minimum flexion angle; Step 8, collecting the real-time angle value of each finger at the current moment through the sensor, and calculating the angle offset according to the real-time angle value and the standard position; Step 9, calculate the standard evaluation value according to the angle offset, scoliosis angle range and flexion angle range, and determine the standardization of the rehabilitation movement according to the standard evaluation value.

2. The method for recognizing rehabilitation movements based on image recognition according to claim 1, characterized in that: When the midline of each finger is in the same straight line with the midline of the palm, the midline position of the current finger is taken as the standard position, and the angle value of the standard position is 0°.

3. The method for recognizing rehabilitation movements based on image recognition according to claim 1, characterized in that: The flexion angle range was defined for the thumb, and the flexion angle range was [10°, 20°].

4. The method for recognizing rehabilitation movements based on image recognition according to claim 1, characterized in that: The lateral bending angle range is defined for the index finger, middle finger, ring finger, and little finger, and the lateral bending angle range is [20°, 60°].

5. The method for recognizing rehabilitation movements based on image recognition according to claim 1, characterized in that: The formula for calculating the normative assessment value is: ； Where F represents the standard evaluation value of the five finger rehabilitation movements, represents the angle offset of the i-th finger, Represents the angle value of the reset position in the time series angle value of the i-th finger, represents the first adjustment factor of the angle offset of the i-th finger, The second adjustment factor representing the angle value of the reset position in the time series angle value of the i-th finger.

6. The method for recognizing rehabilitation movements based on image recognition according to claim 1, characterized in that: The time series angle values ​​include the standard position angle values, lateral bending and / or flexion angle values ​​and reset angle values ​​of the fingers during the rehabilitation movement. The reset angle value range is [3°, 8°], and the reset angle value range includes the maximum reset angle value and the minimum reset angle value.

7. The method for recognizing rehabilitation movements based on image recognition according to claim 5, characterized in that: ； In the formula, represents the maximum lateral bending and / or flexion angle, represents the angle value of the i-th finger at the current time t, Represents the minimum lateral bending and / or flexion angle.

8. The method for recognizing rehabilitation movements based on image recognition according to claim 5, characterized in that: ； In the formula, Represents the minimum angle value of the reset position, Represents the maximum angle value of the reset position.

9. A rehabilitation action recognition device based on image recognition, used to execute a rehabilitation action recognition method based on image recognition as described in any one of claims 1 to 8, characterized in that: Includes the following modules: Angle monitoring module: used to install sensors on the patient's fingers to monitor the angle values ​​of lateral bending and / or flexion of the fingers in real time; Image sequence acquisition module: used to acquire the image sequence of the patient's finger movement through a camera; Preprocessing module: connected with the image sequence acquisition module, used for preprocessing the image sequence; Segmentation module: connected to the preprocessing module, used to segment the finger area in the preprocessed image sequence to obtain a finger image sequence; Standard position determination module: connected to the segmentation module, used to analyze the finger image sequence to determine the initial position of the finger before the rehabilitation exercise begins, and use the initial position as the standard position; Angle value acquisition module: connected to the standard position determination module, used for collecting the time series angle value of the finger returning to the reset position after the finger is bent and / or flexed from the standard position after the rehabilitation exercise begins; Standardization determination module: connected to the angle value acquisition module, used to preset a scoliosis angle range for finger scoliosis and a flexion angle range for finger flexion, so as to determine the standardization of rehabilitation exercise; the scoliosis angle range includes a maximum scoliosis angle and a minimum scoliosis angle, and the flexion angle range includes a maximum flexion angle and a minimum flexion angle; Angle offset calculation module: connected to the standardization determination module, used to collect the real-time angle value of each finger at the current moment through the sensor, and calculate the angle offset according to the real-time angle value and the standard position; Normative evaluation module: connected with the angle offset calculation module and the normative determination module, used to calculate the normative evaluation value according to the angle offset, scoliosis angle range and flexion angle range, and determine the normativeness of the rehabilitation movement according to the normative evaluation value.

Images