Finger movement ability rehabilitation training system and method based on image instruction

Abstract

The invention is suitable for the technical field of image recognition, and provides a finger exercise ability rehabilitation training system and method based on an image instruction, and the method comprises the steps: obtaining a real-time image corresponding to a mirror image palm; obtaining a pre-stored palm contour, and aligning the pre-stored palm contour with a palm region in the real-time image to obtain a reference image; calculating bending data corresponding to each finger to be trained according to a position distribution relationship between a palm region and a palm contour in the reference image; and according to the bending data, controlling the pneumatic structure glove corresponding to the to-be-trained finger to execute a bending action. According to the scheme, the bending data of each finger to be trained are accurately obtained by calculating the position distribution relation between the palm region and the palm contour in the reference image. The technical step ensures accurate mapping between the image instruction and the actual hand action.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image recognition, and particularly relates to a finger movement ability rehabilitation training system and method based on image instructions. BACKGROUND

[0002] With the development of modern medical technology, finger movement ability rehabilitation training plays an important role in treating hand dysfunction and helping patients recover movement ability. Finger movement ability rehabilitation training is usually used to treat hand dysfunction caused by conditions such as stroke, brain injury, spinal cord injury, etc. Such disorders can cause patients to lose hand flexibility and coordination, thereby affecting their daily life and work ability. Therefore, how to help patients recover finger flexibility and movement ability through effective training methods has become an important issue in the field of rehabilitation.

[0003] At present, finger movement rehabilitation training mainly adopts physical therapy, hand training equipment and intelligent auxiliary equipment, among which intelligent and automated rehabilitation training systems have gradually become the mainstream. Traditional finger rehabilitation training methods mainly rely on the guidance and supervision of physical therapists, which not only has limited treatment effect, but also is subject to human factors. In addition, existing finger rehabilitation training equipment mostly uses mechanical devices for auxiliary training, lacks real-time feedback and guidance of patient movement state, and the training process is difficult to accurately adjust and customize.

[0004] In order to overcome the shortcomings of traditional training methods and equipment, in recent years, rehabilitation training methods based on visual perception and control have gradually attracted the attention of researchers. Through real-time image processing technology, the movement information of the patient's hand is obtained, and the working state of the training equipment is adjusted according to the image data feedback, which not only provides more intuitive training feedback, but also better adapts to the individual needs of patients, achieving more accurate rehabilitation effect.

[0005] However, the existing finger rehabilitation training system based on image instructions still has certain limitations. For example, in terms of image processing and mapping accuracy of hand movements, how to ensure that the training system can accurately recognize the movements of the patient's hand and provide real-time feedback is still a problem to be solved. SUMMARY

[0006] Therefore, the embodiments of the present application provide a finger movement ability rehabilitation training system and method based on image instructions to solve the technical problem of mapping accuracy of image processing and hand movements in traditional methods.

[0007] A first aspect of the embodiments of the present application provides a finger movement ability rehabilitation training system based on image instructions, which comprises a control module, a pneumatic structure glove and a vision module. The vision module is used to acquire real-time images; the pneumatic structure glove is used to perform bending actions based on bending data. The control module is used to acquire real-time images corresponding to the mirrored palm; the mirrored palm refers to a palm with finger movement capabilities, used to guide the fingers to be trained to perform the same movements. The control module is used to acquire a pre-stored palm outline and align the pre-stored palm outline with the palm area in the real-time image to obtain a reference image; the pre-stored palm outline is the palm outline acquired when the palm is naturally open. The control module is used to calculate the bending data corresponding to each finger to be trained based on the positional distribution relationship between the palm region and the palm outline in the reference image; wherein, the fingers of the mirrored palm and the fingers to be trained in the palm to be trained correspond one-to-one. The control module is used to control the pneumatic structure glove corresponding to the finger to be trained to perform a bending action based on the bending data; wherein, each finger to be trained wears its own corresponding pneumatic structure glove, and the bending action is used to drive the finger to be trained to perform a rehabilitation action through the pneumatic structure glove.

[0008] A second aspect of this invention provides a method for rehabilitation training of finger motor skills based on image commands. This method is applied to a system for rehabilitation training of finger motor skills based on image commands. The method includes: S1: Obtain the real-time image corresponding to the mirrored palm; the mirrored palm refers to a palm with finger movement capabilities, used to guide the finger to be trained to perform the same action; S2: Obtain the pre-stored palm outline and align the pre-stored palm outline with the palm area in the real-time image to obtain a reference image; the pre-stored palm outline is the palm outline collected when the palm is in a naturally open state. S3: Based on the positional distribution relationship between the palm region and the palm outline in the reference image, calculate the bending data corresponding to each finger to be trained; wherein, the fingers of the mirrored palm and the fingers to be trained in the palm to be trained correspond one-to-one. S4: Based on the bending data, control the pneumatic structure glove corresponding to the finger to be trained to perform a bending action; wherein, each finger to be trained wears its own corresponding pneumatic structure glove, and the bending action is used to drive the finger to be trained to perform a rehabilitation action through the pneumatic structure glove.

[0009] Further, S2 includes: S21: Obtain the pre-stored palm outline; S22: Extract the palm region from the real-time image; the palm region includes the palm and wrist; S23: Extract the current wrist width value in the palm area; S24: Extract the standard wrist width value from the pre-stored palm contour; S25: Calculate the ratio between the current wrist width value and the standard wrist width value; S26: Enlarge or reduce the pre-stored palm outline according to the ratio to obtain a reference palm outline; S27: Merge the reference palm contour into the real-time image to obtain a reference image; wherein the line segment corresponding to the wrist width value in the reference palm contour is coincident with the line segment corresponding to the current wrist width value.

[0010] Further, S23 includes: S231: Extract the edge contour of the palm region; S232: The lower half of the edge contour is cut off in the palm width direction; the lower half contour includes the wrist; S233: Generate multiple parallel straight lines at fixed intervals in the palm width direction, and extract the first line segment that intersects the multiple parallel straight lines with the lower half contour; S234: Use the length of the minimum first line segment as the current wrist width value.

[0011] Further, S3 includes: S31: Extract the multiple finger movement areas of the palm contour in the reference image respectively; the multiple finger movement areas include the thumb movement area, index thumb movement area, middle finger movement area, ring finger movement area and little finger movement area, and the finger movement area refers to the area traversed by the finger during movement; S32: Calculate the first percentage value of the finger corresponding to the finger activity area in the finger activity area; the first percentage value is used to represent the percentage of the finger distribution in the longitudinal direction; S33: Obtain multiple preset percentage values ​​and the bending data corresponding to each preset percentage value; S34: Calculate the difference between the first percentage value and multiple preset percentage values ​​respectively; S35: Extract the target preset percentage value corresponding to the minimum difference; S36: Use the bending data corresponding to the target preset proportion value as the bending data for each finger to be trained.

[0012] Further, S32 includes: S321: Construct a first coordinate system with the length of the finger's active area as the vertical axis and the width of the finger's active area as the horizontal axis; S322: Obtain the pixel value range corresponding to the finger, and extract multiple current finger pixels in the finger activity area based on the pixel value range; S323: Based on the first coordinate system, extract the maximum ordinate among multiple current finger pixels; S324: Based on the first coordinate system, extract the maximum ordinate of the finger activity area; S325: Divide the maximum ordinate of the plurality of current finger pixels by the maximum ordinate of the finger activity area to obtain the first proportion value.

[0013] Furthermore, prior to S33, the following is also included: S51: Acquire video data of the sample palm and sample bending data at multiple sampling times; the video data includes video data corresponding to the entire process of each sample finger changing from a naturally open state to a maximum bent state; S52: Extract video frames corresponding to multiple sampling times from the video data; S53: In the video frame, count the multiple pixel positions that each sample finger passes through throughout the entire process, and use the multiple pixel positions as the first image region; S54: Construct a second coordinate system with the length direction of the first image region as the vertical axis and the width direction of the first image region as the horizontal axis; S55: Obtain the pixel value range corresponding to the finger, and extract multiple sample finger pixels in the first image area based on the pixel value range; S56: Based on the second coordinate system, extract the maximum ordinate among multiple sample finger pixels; S57: Based on the second coordinate system, extract the maximum ordinate in the first image region; S58: Divide the maximum ordinate of the plurality of sample finger pixels by the maximum ordinate of the first image region to obtain the preset proportion value; S59: Use the sample curvature data corresponding to the sampling time of the video frame as the curvature data corresponding to the preset proportion value.

[0014] A second aspect of the present invention provides a finger movement ability rehabilitation training device based on image commands, comprising: The first acquisition unit is used to acquire a real-time image corresponding to the mirrored palm; the mirrored palm refers to a palm with finger movement ability, which is used to guide the finger to be trained to perform the same action. The second acquisition unit is used to acquire a pre-stored palm outline and align the pre-stored palm outline with the palm area in the real-time image to obtain a reference image; the pre-stored palm outline is the palm outline acquired when the palm is naturally open. The calculation unit is used to calculate the bending data corresponding to each finger to be trained based on the positional distribution relationship between the palm region and the palm outline in the reference image; wherein, the fingers of the mirrored palm and the fingers to be trained in the palm to be trained correspond one-to-one. The control unit is used to control the pneumatic structure glove corresponding to the finger to be trained to perform a bending action based on the bending data; wherein each finger to be trained wears its own corresponding pneumatic structure glove, and the bending action is used to drive the finger to be trained to perform a rehabilitation action through the pneumatic structure glove.

[0015] A third aspect of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in the image instruction-based finger motor ability rehabilitation training method described in the first aspect above.

[0016] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the image instruction-based finger motor ability rehabilitation training method described in the first aspect.

[0017] The beneficial effects of this invention compared to existing technologies are as follows: By acquiring real-time images of a mirrored palm and aligning them with pre-stored palm contours, the movement state of the patient's hand can be accurately analyzed. By calculating the positional distribution relationship between the palm region and the palm contour in the reference image, the bending data of each finger to be trained is precisely obtained. This technical step ensures a precise mapping between image commands and actual hand movements, enabling the system to make personalized training adjustments based on each patient's specific hand condition, thereby maximizing the effectiveness of rehabilitation training. Using the image information of the mirrored palm, the system can track changes in the patient's hand movements in real time and convert these changes into corresponding pneumatic control signals to drive a pneumatic structure glove to perform precise bending movements. Each finger to be trained wears a specialized pneumatic structure glove, through which it performs bending movements, directly participating in the finger's rehabilitation training. This training method not only reduces reliance on external physical therapists but also allows for real-time adjustments based on the patient's rehabilitation needs without external intervention, achieving a personalized and automated training process. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A schematic flowchart of a finger motor ability rehabilitation training method based on image commands provided by the present invention is shown. Figure 2 The diagram shows a schematic of a finger movement ability rehabilitation training device based on image commands according to an embodiment of the present invention; Figure 3 A schematic diagram of a terminal device provided in an embodiment of the present invention is shown. Detailed Implementation

[0020] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0021] This invention provides a finger movement ability rehabilitation training system and method based on image instructions to solve the technical problem of the accuracy of image processing and hand movement mapping in traditional methods.

[0022] First, the present invention provides a finger motor ability rehabilitation training system based on image instructions, the finger motor ability rehabilitation training system based on image instructions includes a control module, a pneumatic structure glove and a vision module; The vision module is used to acquire real-time images; the pneumatic structure glove is used to perform bending actions based on bending data. The control module is used to acquire real-time images corresponding to the mirrored palm; the mirrored palm refers to a palm with finger movement capabilities, used to guide the fingers to be trained to perform the same movements. The control module is used to acquire a pre-stored palm outline and align the pre-stored palm outline with the palm area in the real-time image to obtain a reference image; the pre-stored palm outline is the palm outline acquired when the palm is naturally open. The control module is used to calculate the bending data corresponding to each finger to be trained based on the positional distribution relationship between the palm region and the palm outline in the reference image; wherein, the fingers of the mirrored palm and the fingers to be trained in the palm to be trained correspond one-to-one. The control module is used to control the pneumatic structure glove corresponding to the finger to be trained to perform a bending action based on the bending data; wherein, each finger to be trained wears its own corresponding pneumatic structure glove, and the bending action is used to drive the finger to be trained to perform a rehabilitation action through the pneumatic structure glove.

[0023] Secondly, this invention provides a method for finger motor function rehabilitation training based on image commands. Please see below. Figure 1 , Figure 1 A schematic flowchart of a finger motor ability rehabilitation training method based on image commands provided by the present invention is shown. Figure 1 As shown, this image-based finger motor function rehabilitation training method may include the following steps: S1: Obtain the real-time image corresponding to the mirrored palm; the mirrored palm refers to a palm with finger movement capabilities, used to guide the finger to be trained to perform the same action; Because some patients with finger dysfunction only have dysfunction in one hand, while the other hand functions normally, the rehabilitation training system provides a mirror training mode. This mode identifies the movements of the hand with normal motor function (the mirror hand) and generates corresponding control commands, which in turn control the pneumatic glove to make the other hand with dysfunction perform the same movements.

[0024] First, the user's mirrored hand makes corresponding movements under the vision module. The vision module captures real-time images and then recognizes the movements of the mirrored hand in the real-time images.

[0025] S2: Obtain the pre-stored palm outline and align the pre-stored palm outline with the palm area in the real-time image to obtain a reference image; the pre-stored palm outline is the palm outline collected when the palm is in a naturally open state. In this step, the system uses a pre-captured and stored palm contour. This contour is data captured and recognized when the palm is naturally open.

[0026] The real-time acquired mirrored hand image and the pre-stored hand contour need to be aligned, that is, matched together using image processing algorithms. The purpose of this process is to ensure that the hand region in the current image is consistent with the pre-stored template, providing a standardized reference for subsequent analysis. The image obtained through the alignment operation is the "reference image." The role of this image is to provide an accurate benchmark to help with data calculations in subsequent steps.

[0027] Specifically, S2 includes S21 to S27: S21: Obtain the pre-stored palm outline; Extract a standardized hand contour from a database or storage. This pre-stored hand contour is the outline of the hand when it is naturally open, serving as a standard reference for hand shape. This contour will serve as the basis for subsequent image processing and alignment.

[0028] S22: Extract the palm region from the real-time image; the palm region includes the palm and wrist; Since the background under the mirrored palm is a known solid color background, it can be removed from the real-time image based on the pixel values ​​corresponding to the solid color background to obtain the palm area.

[0029] S23: Extract the current wrist width value in the palm area; Since the wrist is the shortest part of the lower half of the palm, wrist width is one of the important features of the palm contour. Therefore, after extracting the palm region, it is necessary to further extract the width of the wrist within the palm region.

[0030] Specifically, S23 includes S231 to S234: S231: Extract the edge contour of the palm region; By extracting edges, the system can clearly identify the contour of the palm area, thus providing accurate shape information for subsequent steps.

[0031] S232: The lower half of the edge contour is cut off in the palm width direction; the lower half contour includes the wrist; The wrist area is located at the bottom of the palm. The lower half of the outline is cropped to extract the part where the wrist is located, because the width of the wrist is usually a small part of the width of the palm, located below the edge outline.

[0032] S233: Generate multiple parallel straight lines at fixed intervals in the palm width direction, and extract the first line segment that intersects the multiple parallel straight lines with the lower half contour; To measure wrist width, the system needs to generate multiple parallel straight lines along the width of the palm. These parallel straight lines will intersect the lower half of the contour (including the wrist), generating multiple intersection points. These intersection points and line segments will help determine the width of the wrist.

[0033] Along the width of the palm, the system generates multiple parallel straight lines at regular intervals. These lines intersect the lower half of the palm's contour, forming multiple intersection points. The distance between each intersection point is a "first line segment." The lengths of these line segments are used to analyze the width of the wrist.

[0034] S234: Use the length of the minimum first line segment as the current wrist width value.

[0035] The width of the wrist is a feature of the hand's outline. The actual width of the wrist can be determined by calculating the length of the first line segment formed by the intersection of all parallel lines with the lower half of the outline. Typically, the width of the wrist appears at the narrowest point of the lower half of the outline; therefore, the length of the smallest first line segment usually represents the width of the wrist.

[0036] The system will select the shortest line segment from all generated first line segments as the width of the wrist. This is because the width of the wrist is usually located in a narrower area of ​​the lower half of the contour, and the width corresponding to the shortest first line segment is the actual width of the current wrist.

[0037] In the embodiments corresponding to S231 to S234, the edge contour of the palm region is extracted and analyzed step by step, and the width of the wrist is measured using the intersection points of parallel lines and the contours. This process involves precise image segmentation, edge detection, and geometric calculations. By selecting the minimum intersection line segment length, the system can accurately identify the current wrist width, providing crucial data for subsequent palm contour alignment and image processing.

[0038] S24: Extract the standard wrist width value from the pre-stored palm contour; In this step, the system extracts a standard wrist width value from the pre-stored palm contour. This standard value is based on the normal wrist width when the palm is naturally open, serving as a reference data point for the palm contour.

[0039] The processing logic of S24 is similar to that of S23, and will not be specified here.

[0040] S25: Calculate the ratio between the current wrist width value and the standard wrist width value; By calculating the ratio between the current wrist width and the standard wrist width, the system can determine the relative size difference between the palm in the real-time image and a pre-stored standard palm. This ratio is crucial for image resizing. The ratio reflects the degree of scaling of the palm in the image, ensuring that the pre-stored contour fits the palm area in the real-time image.

[0041] S26: Enlarge or reduce the pre-stored palm outline according to the ratio to obtain a reference palm outline; Since it is impossible to guarantee the consistency of each shooting distance, and cameras have the characteristic of objects appearing larger when closer and smaller when farther away, registration based on proportions is necessary.

[0042] Based on the ratio calculated in step S25, the system performs a corresponding scaling operation on the pre-stored palm outline. If the wrist width in the real-time image is greater than the standard width, the pre-stored palm outline will be enlarged; if the wrist width in the real-time image is less than the standard width, the pre-stored palm outline will be shrunk. In this way, the scaled palm outline is aligned in size with the palm area in the real-time image, ensuring the accuracy of subsequent alignment operations.

[0043] S27: Merge the reference palm contour into the real-time image to obtain a reference image; wherein the line segment corresponding to the wrist width value in the reference palm contour is coincident with the line segment corresponding to the current wrist width value.

[0044] This is the final step in the entire process, merging the scaled and adjusted reference hand contour into the live image to form a reference image. This reference image includes both the hand region from the live image and the adjusted standard hand contour, ensuring spatial consistency between the two.

[0045] In this step, it's crucial to ensure that the line segment corresponding to the wrist width in the reference hand contour coincides with the line segment corresponding to the wrist width in the real-time image. This ensures precise alignment between the reference hand contour and the hand region in the real-time image, guaranteeing accurate image analysis and finger motion mapping in the subsequent process.

[0046] In the embodiments corresponding to S21 to S27, precise image processing and size adjustment techniques are used to align the standard palm outline with the palm region in the real-time image, ensuring that the rehabilitation training system can accurately map movements based on the real-time image. By comparing and proportionally calculating the wrist width, the size of the palm outline is adjusted, thereby achieving precise alignment with the palm region in the real-time image. This step provides a stable and consistent reference framework for subsequent finger movement calculations and pneumatic glove control.

[0047] S3: Based on the positional distribution relationship between the palm region and the palm outline in the reference image, calculate the bending data corresponding to each finger to be trained; wherein, the fingers of the mirrored palm and the fingers to be trained in the palm to be trained correspond one-to-one. In this step, the system analyzes the overlap between the individual fingers of the hand in the reference image and the pre-stored hand contour. By comparing the position of the individual fingers in the hand contour and the real-time image, the system can calculate the bending angle of each finger.

[0048] Specifically, S3 includes S31 to S36: S31: Extract the multiple finger movement areas of the palm contour in the reference image respectively; the multiple finger movement areas include the thumb movement area, index thumb movement area, middle finger movement area, ring finger movement area and little finger movement area, and the finger movement area refers to the area traversed by the finger during movement; The movement areas of different fingers in the palm outline are known marked areas. These multiple finger movement areas include, but are not limited to, the movement areas of the thumb, index finger, middle finger, ring finger, and little finger. These movement areas refer to the areas that the fingers pass through during movement (i.e., the locations where the skin and muscles of the fingers change when the fingers are bent).

[0049] S32: Calculate the first percentage value of the finger corresponding to the finger activity area in the finger activity area; the first percentage value is used to represent the percentage of the finger distribution in the longitudinal direction; Calculate the proportion of each finger's active area within its own region, especially the proportion in the longitudinal direction, to reflect the distribution of each finger. The first proportion value represents the percentage of the finger's active area in the longitudinal direction, and this proportion is usually used to characterize the degree of bending of the finger within its active area.

[0050] It is understandable that, since the fingers may have a certain lateral deviation during each movement, and the combination between the palm area and the palm outline also has a certain lateral deviation, the lateral distribution has a large degree of uncertainty. Therefore, this embodiment only focuses on the proportion in the vertical direction.

[0051] Specifically, S32 includes S321 to S325: S321: Construct a first coordinate system with the length of the finger's active area as the vertical axis and the width of the finger's active area as the horizontal axis; Select the finger's active area on the image and determine the orientation of the coordinate system based on the shape of the area. The length direction corresponds to the vertical direction of the finger, and the width direction corresponds to the horizontal direction. In this way, the pixels of the entire finger's active area can be located and measured according to this coordinate system.

[0052] S322: Obtain the pixel value range corresponding to the finger, and extract multiple current finger pixels in the finger activity area based on the pixel value range; The pixel value range corresponding to the finger is pre-stored known data. Based on this range, all pixels within the finger's active area are extracted. These pixels represent the actual shape of the finger and form the basis for subsequent calculations of proportions.

[0053] S323: Based on the first coordinate system, extract the maximum ordinate among multiple current finger pixels; Among all the extracted current finger pixels, select the point with the largest vertical (length) coordinate value. This point usually represents the "top" or "top" of the finger. This largest vertical coordinate value is crucial for subsequent proportion calculations.

[0054] S324: Based on the first coordinate system, extract the maximum ordinate of the finger activity area; The purpose of this step is to find the maximum vertical coordinate value of the entire finger movement area (length direction), which is usually the top boundary of the finger movement area or the highest point of the finger movement area.

[0055] S325: Divide the maximum ordinate of the plurality of current finger pixels by the maximum ordinate of the finger activity area to obtain the first proportion value.

[0056] By dividing the maximum ordinate of the finger within the finger's active area by the maximum ordinate of the entire active area, we can obtain the proportion of that finger in the longitudinal direction of the active area. This proportion reflects the degree of finger distribution within the active area, particularly the finger's "longitudinal position."

[0057] In this step, the ratio between the maximum vertical coordinate of the finger within the finger's active area and the maximum vertical coordinate of the entire finger's active area is calculated. This ratio is the first percentage value, representing the proportion of the finger's vertical position within the finger's active area. For example, if the finger's maximum vertical coordinate is close to the maximum vertical coordinate of the entire active area, the percentage value will be larger; if the finger is positioned lower, the percentage value will be smaller.

[0058] It is worth noting that since the length of a finger gradually decreases in the longitudinal direction when it is bent, the proportion of the finger in the longitudinal direction can represent a kind of bending data. Therefore, in this embodiment, the ordinate (maximum ordinate) corresponding to the finger apex is obtained in the longitudinal direction and divided by the maximum ordinate of the finger's active area to obtain a first proportion value. The first proportion value is used to represent the distribution position of the finger apex in the finger's active area, and then the corresponding bending data is matched according to the distribution position.

[0059] In the embodiments corresponding to S321 to S325, through these steps, the system can accurately calculate the longitudinal proportion of the finger within the finger's active area. This proportion reflects the distribution of the finger within the active area, especially its longitudinal distribution. The first proportion provides important quantitative data for the subsequent calculation of bending data, helping the system to more accurately estimate the degree of finger bending.

[0060] S33: Obtain multiple preset percentage values ​​and the bending data corresponding to each preset percentage value; The purpose of this step is to prepare a set of preset reference data, including multiple percentage values ​​and the corresponding finger bending data for each percentage value. The preset percentage values ​​represent different finger bending states, while the corresponding bending data indicates the specific degree of finger bending in these states.

[0061] The system predefines a set of preset percentage values ​​and assigns corresponding finger bending data (such as angle or degree of bending) to each percentage value. Each preset percentage value corresponds to the bending state of a finger to aid in subsequent calculations.

[0062] As an optional embodiment of this application, before S33, S51 to S59 (calculating the preset proportion value and its corresponding bending data) are also included: S51: Acquire video data of the sample palm and sample bending data at multiple sampling times; the video data includes video data corresponding to the entire process of each sample finger changing from a naturally open state to a maximum bent state; First, video data of the sampled hand needs to be collected, including the entire process of the fingers moving from a naturally open state to their maximum flexed state. Simultaneously, flexion data at each sampling moment needs to be acquired, reflecting the degree of finger flexion at that specific time.

[0063] S52: Extract video frames corresponding to multiple sampling times from the video data; Specific video frames at multiple sampling moments were extracted from the video data. These video frames represent instantaneous images of the finger in different bending positions.

[0064] S53: In the video frame, count the multiple pixel positions that each sample finger passes through throughout the entire process, and use the multiple pixel positions as the first image region; In each video frame, the pixel positions traversed by the finger during its movement are counted. These pixel positions reflect the finger's movement trajectory and can be used to determine the finger's distribution within the active area. All extracted pixel positions are integrated to form a "first image region," which includes all positions traversed by the finger as it moves from open to bent.

[0065] S54: Construct a second coordinate system with the length direction of the first image region as the vertical axis and the width direction of the first image region as the horizontal axis; A new coordinate system is established in the first image region to facilitate subsequent calculations. The vertical axis of the second coordinate system represents the length of the finger's active area, and the horizontal axis represents its width.

[0066] S55: Obtain the pixel value range corresponding to the finger, and extract multiple sample finger pixels in the first image area based on the pixel value range; Obtain the pre-stored pixel value range, and extract the pixels representing the fingers within the first image region based on this range. These points represent the actual shape of the fingers and are the basis for subsequent calculations of the proportion values.

[0067] S56: Based on the second coordinate system, extract the maximum ordinate among multiple sample finger pixels; Among all the extracted finger pixels, select the point with the largest coordinate value in the vertical (length direction), which usually represents the "top" or "top" position of the finger.

[0068] S57: Based on the second coordinate system, extract the maximum ordinate in the first image region; Find the position of the largest vertical dimension in the entire finger movement area (i.e., the first image area), which is the top boundary of the area.

[0069] S58: Divide the maximum ordinate of the plurality of sample finger pixels by the maximum ordinate of the first image region to obtain the preset proportion value; The vertical proportion of the finger within the active area is obtained by dividing the maximum vertical coordinate of the finger by the maximum vertical coordinate of the entire finger's active area.

[0070] S59: Use the sample curvature data corresponding to the sampling time of the video frame as the curvature data corresponding to the preset proportion value.

[0071] The bending data at each sampling moment is associated with its corresponding preset proportion value, forming a mapping relationship between the proportion value and the bending data. Each sampling moment of a video frame has a corresponding bending data, which is obtained through angle measurement or sensor readings. These bending data are matched with the corresponding proportion values ​​to form a reference dataset for subsequent analysis and training.

[0072] In the embodiments corresponding to S51 to S59, a relationship between the preset proportion value and the bending data was established using video data and corresponding bending data. By extracting the pixel position of the finger during movement, calculating the longitudinal proportion value, and associating it with the bending data, a complete reference dataset was constructed. This data can be used for subsequent finger bending prediction, training, and control.

[0073] S34: Calculate the difference between the first percentage value and multiple preset percentage values ​​respectively; The system compares a first proportion value for each finger with a preset proportion value and calculates the difference between the two. These differences help the system find the bending data that best matches real-time finger activity.

[0074] S35: Extract the target preset percentage value corresponding to the minimum difference; From multiple preset percentage values, select the percentage value that differs least from the first percentage value. This step is to find the best matching percentage value to ensure accurate mapping of the curvature data.

[0075] S36: Use the bending data corresponding to the target preset proportion value as the bending data for each finger to be trained.

[0076] Once the target preset percentage value is determined, the system can find the corresponding bending data. This bending data can include parameters such as bending angle and joint angle, which can be used for further training or control. Through this data, finger movement can be effectively simulated and controlled.

[0077] In the embodiments corresponding to S31 to S36, the finger activity areas are extracted from the reference image, and the proportion of each finger's activity area is calculated. Combined with preset bending data, the specific bending data of each finger is deduced. This process ensures accurate mapping of finger bending, providing precise data support for subsequent motion training and simulation.

[0078] S4: Based on the bending data, control the pneumatic structure glove corresponding to the finger to be trained to perform a bending action; wherein, each finger to be trained wears its own corresponding pneumatic structure glove, and the bending action is used to drive the finger to be trained to perform a rehabilitation action through the pneumatic structure glove.

[0079] The hand to be trained wears a pneumatically structured glove, with each finger having its own pneumatic structure. This glove consists of a pneumatic device that controls airflow to allow the fingers to flex or extend. The pneumatic glove adjusts the movement of each finger based on flexion data provided by the system, thereby aiding in finger rehabilitation training. Once the flexion data is calculated, the pneumatic glove performs consistent movements mirroring those of the hand.

[0080] It should be noted that the numbers S1 to S4 and the specific steps do not constitute a restriction on the order of the method steps. The execution order of the steps can be adjusted based on the actual situation, and no restrictions are imposed here.

[0081] In the embodiments corresponding to S1 to S4, by acquiring real-time images of a mirrored palm and aligning them with pre-stored palm contours, the movement state of the patient's palm can be accurately analyzed. By calculating the positional distribution relationship between the palm region and the palm contour in the reference image, the bending data of each finger to be trained is precisely obtained. This technical step ensures a precise mapping between image commands and actual hand movements, enabling the system to make personalized training adjustments based on each patient's specific hand condition, thereby maximizing the effectiveness of rehabilitation training. Using the image information of the mirrored palm, the system can track changes in the patient's hand movements in real time and convert these changes into corresponding pneumatic control signals to drive a pneumatic structure glove to perform precise bending movements. Each finger to be trained wears a specialized pneumatic structure glove, through which it performs bending movements, directly participating in the finger's rehabilitation training. This training method not only reduces reliance on external physical therapists but also allows for real-time adjustments based on the patient's rehabilitation needs without external intervention, achieving a personalized and automated training process.

[0082] like Figure 2 This invention provides a finger movement ability rehabilitation training device based on image commands. Please refer to [link / reference]. Figure 2 , Figure 2 This diagram illustrates a finger movement rehabilitation training device based on image commands provided by the present invention, as shown below. Figure 2 The image-based finger motor ability rehabilitation training device shown includes: The first acquisition unit 21 is used to acquire a real-time image corresponding to the mirrored palm; the mirrored palm refers to a palm with finger movement ability, which is used to guide the finger to be trained to make the same movement. The second acquisition unit 22 is used to acquire a pre-stored palm outline and align the pre-stored palm outline with the palm area in the real-time image to obtain a reference image; the pre-stored palm outline is the palm outline acquired when the palm is naturally open. The calculation unit 23 is used to calculate the bending data corresponding to each finger to be trained based on the positional distribution relationship between the palm region and the palm outline in the reference image; wherein, the fingers of the mirrored palm and the fingers to be trained in the palm to be trained correspond one-to-one. The control unit 24 is used to control the pneumatic structure glove corresponding to the finger to be trained to perform a bending action according to the bending data; wherein each finger to be trained wears its own corresponding pneumatic structure glove, and the bending action is used to drive the finger to be trained to perform a rehabilitation action through the pneumatic structure glove.

[0083] This invention provides a finger movement rehabilitation training device based on image commands. By acquiring real-time images of a mirrored palm and aligning them with pre-stored palm contours, it can accurately analyze the patient's hand movement state. Through calculation of the positional distribution relationship between the palm region and the palm contour in the reference image, the bending data of each finger to be trained is precisely obtained. This technical step ensures a precise mapping between image commands and actual hand movements, enabling the system to make personalized training adjustments based on each patient's specific hand condition, thereby maximizing the effectiveness of rehabilitation training. Using the image information of the mirrored palm, the system can track changes in the patient's hand movements in real time and convert these changes into corresponding pneumatic control signals to drive a pneumatically structured glove to perform precise bending movements. Each finger to be trained wears a specialized pneumatically structured glove, which performs bending movements through the glove, directly participating in finger rehabilitation training. This training method not only reduces reliance on external physical therapists but also allows for real-time adjustments based on the patient's rehabilitation needs without external intervention, achieving a personalized and automated training process.

[0084] Figure 3 This is a schematic diagram of a terminal device provided in an embodiment of the present invention. Figure 3 As shown, a terminal device 3 in this embodiment includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as a finger motor ability rehabilitation training program based on image instructions. When the processor 30 executes the computer program 32, it implements the steps in the various embodiments of the finger motor ability rehabilitation training method based on image instructions described above, for example... Figure 1 S1 to S4 are shown. Alternatively, when the processor 30 executes the computer program 32, it implements the functions of each unit in the above-described device embodiments, for example... Figure 2 The function of the unit shown.

[0085] For example, the computer program 32 can be divided into one or more units, which are stored in the memory 31 and executed by the processor 30 to complete the present invention. The one or more units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 32 in the terminal device 3. For example, the specific functions of each unit of the computer program 32 can be divided as follows: The first acquisition unit is used to acquire a real-time image corresponding to the mirrored palm; the mirrored palm refers to a palm with finger movement ability, which is used to guide the finger to be trained to perform the same action. The second acquisition unit is used to acquire a pre-stored palm outline and align the pre-stored palm outline with the palm area in the real-time image to obtain a reference image; the pre-stored palm outline is the palm outline acquired when the palm is naturally open. The calculation unit is used to calculate the bending data corresponding to each finger to be trained based on the positional distribution relationship between the palm region and the palm outline in the reference image; wherein, the fingers of the mirrored palm and the fingers to be trained in the palm to be trained correspond one-to-one. The control unit is used to control the pneumatic structure glove corresponding to the finger to be trained to perform a bending action based on the bending data; wherein each finger to be trained wears its own corresponding pneumatic structure glove, and the bending action is used to drive the finger to be trained to perform a rehabilitation action through the pneumatic structure glove.

[0086] The terminal device includes, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will understand that... Figure 3 This is merely an example of a terminal device 3 and does not constitute a limitation on a terminal device 3. It may include more or fewer components than shown, or combine certain components, or different components. For example, the terminal device may also include input / output devices, network access devices, buses, etc.

[0087] The processor 30 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0088] The memory 31 can be an internal storage unit of the terminal device 3, such as a hard disk or memory of the terminal device 3. The memory 31 can also be an external storage device of the terminal device 3, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal device 3. Furthermore, the memory 31 can include both internal and external storage units of the terminal device 3. The memory 31 is used to store the computer program and other programs and data required by the roaming control device. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0089] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0090] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0091] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0092] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0093] This invention provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0094] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0095] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0096] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0097] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0098] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units.

[0099] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0100] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0101] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [the described condition or event] is detected," or "in response to detection of [the described condition or event]."

[0102] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0103] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0104] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A finger motor ability rehabilitation training system based on image commands, characterized in that, The image-instruction-based finger movement rehabilitation training system includes a control module, a pneumatic structure glove, and a vision module. The vision module is used to acquire real-time images; the pneumatic structure glove is used to perform bending actions based on bending data. The control module is used to acquire real-time images corresponding to the mirrored palm; the mirrored palm refers to a palm with finger movement capabilities, used to guide the fingers to be trained to perform the same movements. The control module is used to acquire a pre-stored palm outline and align the pre-stored palm outline with the palm area in the real-time image to obtain a reference image; the pre-stored palm outline is the palm outline acquired when the palm is naturally open. The control module is used to calculate the bending data corresponding to each finger to be trained based on the positional distribution relationship between the palm region and the palm outline in the reference image; wherein, the fingers of the mirrored palm and the fingers to be trained in the palm to be trained correspond one-to-one. The control module is used to control the pneumatic structure glove corresponding to the finger to be trained to perform a bending action based on the bending data; wherein, each finger to be trained wears its own corresponding pneumatic structure glove, and the bending action is used to drive the finger to be trained to perform a rehabilitation action through the pneumatic structure glove.

2. A method for rehabilitation training of finger motor skills based on image commands, characterized in that, The image-instruction-based finger motor ability rehabilitation training method is applied to an image-instruction-based finger motor ability rehabilitation training system. The image-instruction-based finger motor ability rehabilitation training method includes: S1: Obtain the real-time image corresponding to the mirrored palm; the mirrored palm refers to a palm with finger movement capabilities, used to guide the finger to be trained to perform the same action; S2: Obtain a pre-stored palm outline and align the pre-stored palm outline with the palm area in the real-time image to obtain a reference image; the pre-stored palm outline is the palm outline collected when the palm is in a naturally open state. S3: Based on the positional distribution relationship between the palm region and the palm outline in the reference image, calculate the bending data corresponding to each finger to be trained; wherein, the fingers of the mirrored palm and the fingers to be trained in the palm to be trained correspond one-to-one. S4: Based on the bending data, control the pneumatic structure glove corresponding to the finger to be trained to perform a bending action; wherein, each finger to be trained wears its own corresponding pneumatic structure glove, and the bending action is used to drive the finger to be trained to perform a rehabilitation action through the pneumatic structure glove.

3. The image-instruction-based finger motor ability rehabilitation training method as described in claim 2, characterized in that, S2 includes: S21: Obtain the pre-stored palm outline; S22: Extract the palm region from the real-time image; the palm region includes the palm and wrist; S23: Extract the current wrist width value in the palm area; S24: Extract the standard wrist width value from the pre-stored palm contour; S25: Calculate the ratio between the current wrist width value and the standard wrist width value; S26: Enlarge or reduce the pre-stored palm outline according to the ratio to obtain a reference palm outline; S27: Merge the reference palm contour into the real-time image to obtain a reference image; wherein the line segment corresponding to the wrist width value in the reference palm contour is coincident with the line segment corresponding to the current wrist width value.

4. The image-instruction-based finger motor ability rehabilitation training method as described in claim 3, characterized in that, S23 includes: S231: Extract the edge contour of the palm region; S232: The lower half of the edge contour is cut off in the palm width direction; the lower half contour includes the wrist; S233: Generate multiple parallel straight lines at fixed intervals in the palm width direction, and extract the first line segment that intersects the multiple parallel straight lines with the lower half contour; S234: Use the length of the minimum first line segment as the current wrist width value.

5. The image-instruction-based finger motor ability rehabilitation training method as described in claim 2, characterized in that, S3 includes: S31: Extract the multiple finger movement areas of the palm contour in the reference image respectively; the multiple finger movement areas include the thumb movement area, index thumb movement area, middle finger movement area, ring finger movement area and little finger movement area, and the finger movement area refers to the area traversed by the finger during movement; S32: Calculate the first percentage value of the finger corresponding to the finger activity area in the finger activity area; the first percentage value is used to represent the percentage of the finger distribution in the longitudinal direction; S33: Obtain multiple preset percentage values ​​and the bending data corresponding to each preset percentage value; S34: Calculate the difference between the first percentage value and multiple preset percentage values ​​respectively; S35: Extract the target preset percentage value corresponding to the minimum difference; S36: Use the bending data corresponding to the target preset proportion value as the bending data for each finger to be trained.

6. The image-instruction-based finger motor ability rehabilitation training method as described in claim 5, characterized in that, S32 includes: S321: Construct a first coordinate system with the length of the finger's active area as the vertical axis and the width of the finger's active area as the horizontal axis; S322: Obtain the pixel value range corresponding to the finger, and extract multiple current finger pixels in the finger activity area based on the pixel value range; S323: Based on the first coordinate system, extract the maximum ordinate among multiple current finger pixels; S324: Based on the first coordinate system, extract the maximum ordinate of the finger activity area; S325: Divide the maximum ordinate of the plurality of current finger pixels by the maximum ordinate of the finger activity area to obtain the first proportion value.

7. The image-instruction-based finger motor ability rehabilitation training method as described in claim 5, characterized in that, Before S33, it also includes: S51: Acquire video data of the sample palm and sample bending data at multiple sampling times; the video data includes video data corresponding to the entire process of each sample finger changing from a naturally open state to a maximum bent state; S52: Extract video frames corresponding to multiple sampling times from the video data; S53: In the video frame, count the multiple pixel positions that each sample finger passes through throughout the entire process, and use the multiple pixel positions as the first image region; S54: Construct a second coordinate system with the length direction of the first image region as the vertical axis and the width direction of the first image region as the horizontal axis; S55: Obtain the pixel value range corresponding to the finger, and extract multiple sample finger pixels in the first image area based on the pixel value range; S56: Based on the second coordinate system, extract the maximum ordinate among multiple sample finger pixels; S57: Based on the second coordinate system, extract the maximum ordinate in the first image region; S58: Divide the maximum ordinate of the plurality of sample finger pixels by the maximum ordinate of the first image region to obtain the preset proportion value; S59: Use the sample curvature data corresponding to the sampling time of the video frame as the curvature data corresponding to the preset proportion value.

8. A finger motor ability rehabilitation training device based on image commands, characterized in that, The image-instruction-based finger movement rehabilitation training device includes: The first acquisition unit is used to acquire a real-time image corresponding to the mirrored palm; the mirrored palm refers to a palm with finger movement ability, which is used to guide the finger to be trained to perform the same action. The second acquisition unit is used to acquire a pre-stored palm outline and align the pre-stored palm outline with the palm area in the real-time image to obtain a reference image; the pre-stored palm outline is the palm outline acquired when the palm is naturally open. The calculation unit is used to calculate the bending data corresponding to each finger to be trained based on the positional distribution relationship between the palm region and the palm outline in the reference image; wherein, the fingers of the mirrored palm and the fingers to be trained in the palm to be trained correspond one-to-one. The control unit is used to control the pneumatic structure glove corresponding to the finger to be trained to perform a bending action based on the bending data; wherein each finger to be trained wears its own corresponding pneumatic structure glove, and the bending action is used to drive the finger to be trained to perform a rehabilitation action through the pneumatic structure glove.

9. A terminal device, characterized in that, The terminal device includes: a memory, a processor, and an image-instruction-based finger motor ability rehabilitation training program stored in the memory and executable on the processor, the image-instruction-based finger motor ability rehabilitation training program being configured to implement the steps of the image-instruction-based finger motor ability rehabilitation training method as described in any one of claims 2 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps in the image instruction-based finger motor ability rehabilitation training method as described in any one of claims 2 to 7.

Images