Evaluation Method, Device, System and Storage Medium for Upper Limb Motor Ability Rehabilitation

Abstract

The present invention discloses an evaluation method, device, system and storage medium for upper limb motor ability rehabilitation. The method includes: after the user to be evaluated completes a painting test game and a grip strength game using a rehabilitation ball, obtaining an initial painting image and a current painting image corresponding to the painting test game, as well as pressure information and game completion information corresponding to the grip strength game; determining the painting completion quality evaluation of the user to be evaluated based on the initial painting image and the current painting image, and determining the user muscle strength level of the user to be evaluated based on the pressure information and the game completion information, where the painting completion quality evaluation includes mean square error, peak signal-to-noise ratio and covariance; determining the upper limb motor ability rehabilitation evaluation information of the user to be evaluated based on the mean square error, peak signal-to-noise ratio, covariance and user muscle strength level, and the upper limb motor ability rehabilitation evaluation information includes muscle strength level and wrist joint damage condition. The present invention realizes the evaluation of comprehensive upper limb motor ability rehabilitation.

Description

Technical Field

[0001] The present invention relates to the technical field of sports rehabilitation assessment, and particularly to an assessment method, device, system and storage medium for upper limb motor ability rehabilitation. Background Art

[0002] Human daily life cannot do without the participation of upper limb functions. The decline in hand mobility will directly affect the patient's ability to perform daily activities and social participation ability. Normal upper limb functions mainly include keeping the forearm in different positions, grasping and releasing objects of different shapes, transferring objects from one place to another, operating various tools, and two-handed operations. Therefore, the presence of upper limb dysfunction will seriously affect the patient's ability to perform daily activities, and upper limb function assessment runs through the entire rehabilitation treatment process. However, the current assessment of upper limb motor ability rehabilitation relies on professional medical staff, and the process is boring. At the same time, there is a lack of assessment means and standards for children. Summary of the Invention

[0003] The present invention provides an assessment method, device, system and storage medium for upper limb motor ability rehabilitation to solve the problem that the current assessment of upper limb motor ability rehabilitation cannot be accurate and to determine assessment means and standards at the same time.

[0004] According to one aspect of the present invention, an assessment method for upper limb motor ability rehabilitation is provided. The assessment method for upper limb motor ability rehabilitation is applied to an assessment system for upper limb motor ability rehabilitation. The assessment system for upper limb motor ability rehabilitation includes a rehabilitation ball. The assessment method for upper limb motor ability rehabilitation includes:

[0005] After a user to be assessed completes a painting test game and a grip strength game using the rehabilitation ball, obtain an initial painting image and a current painting image corresponding to the painting test game, as well as pressure information and game completion information corresponding to the grip strength game;

[0006] Determine the painting completion quality assessment of the user to be assessed according to the initial painting image and the current painting image, and determine the user muscle strength level of the user to be assessed according to the pressure information and the game completion information. The painting completion quality assessment includes mean square error, peak signal-to-noise ratio and covariance;

[0007] Determine the upper limb motor ability rehabilitation assessment information of the user to be assessed according to the mean square error, the peak signal-to-noise ratio, the covariance and the user muscle strength level. The upper limb motor ability rehabilitation assessment information includes muscle strength level and wrist joint damage condition.

[0008] Optionally, determining the painting completion quality assessment of the user to be assessed according to the initial painting image and the current painting image includes:

[0009] Determine the mean square error of the user to be evaluated based on the initial painting image and the current painting image, specifically:

[0010]

[0011] where MSE is the mean square error of the user to be evaluated; m and n are the height and width of the initial painting image and the current painting image respectively; I[i, j] is the pixel value of the initial painting image at position (i, j); I′[i, j] is the pixel value of the current painting image at the same position (i, j);

[0012] Or,

[0013] where I[i, j] is the pixel value of the small area in the current painting image at position (i, j); K(i, j) is the pixel value of the small area in the initial painting image at the same position; M and N are the width and height of the small area respectively.

[0014] Optionally, the smaller the mean square error, the smaller the determined difference between the initial painting image and the current painting image;

[0015] The larger the mean square error, the larger the determined difference between the initial painting image and the current painting image.

[0016] Optionally, determining the painting completion quality assessment of the user to be evaluated based on the initial painting image and the current painting image includes:

[0017] Determine the peak signal-to-noise ratio of the user to be evaluated based on the initial painting image and the current painting image, specifically:

[0018]

[0019] where PSNR is the peak signal-to-noise ratio of the user to be evaluated; Peak is the maximum value representing the pixel intensity of the image;

[0020] Determine the covariance of the user to be evaluated based on the initial painting image and the current painting image, specifically:

[0021]

[0022] where SSIM(x, y) is the covariance of the user to be evaluated; x and y represent the pixel values of the initial painting image and the current painting image respectively; μ x and μ y are the means of the initial painting image and the current painting image; and is the variance between the initial painting image and the current painting image; σ xy is the covariance between the initial painting image and the current painting image; C1 and C2 are constants.

[0023] Optionally, obtaining the pressure information and game completion information corresponding to the grip strength game includes:

[0024] Detecting the pressure of each finger of the user to be evaluated through the pressure sensor on the rehabilitation ball, and determining the pressure information corresponding to the grip strength game according to the pressure of each finger of the user to be evaluated;

[0025] Obtaining the control steering time and the completion target grip strength reaction time when the user to be evaluated completes the grip strength game using the rehabilitation ball, and determining the game completion information corresponding to the grip strength game according to the control steering time and the completion target grip strength reaction time.

[0026] Optionally, the evaluation method for upper limb motor ability rehabilitation further includes:

[0027] Collecting the maximum activity angles of the user to be evaluated in palmar flexion, dorsiflexion, radial deviation, and ulnar deviation, and storing the maximum activity angles of palmar flexion, dorsiflexion, radial deviation, and ulnar deviation in the upper limb motor ability rehabilitation evaluation information of the user to be evaluated.

[0028] According to another aspect of the present invention, there is provided an evaluation device for upper limb motor ability rehabilitation. The evaluation device for upper limb motor ability rehabilitation is applied to an evaluation system for upper limb motor ability rehabilitation. The evaluation system for upper limb motor ability rehabilitation includes a rehabilitation ball. The evaluation device for upper limb motor ability rehabilitation includes:

[0029] An information acquisition module, configured to obtain the initial painting image and the current painting image corresponding to the painting test game, and the pressure information and game completion information corresponding to the grip strength game after the user to be evaluated completes the painting test game and the grip strength game using the rehabilitation ball;

[0030] An evaluation information determination module, configured to determine the painting completion quality evaluation of the user to be evaluated according to the initial painting image and the current painting image, and determine the user muscle strength level of the user to be evaluated according to the pressure information and the game completion information. The painting completion quality evaluation includes mean square error, peak signal-to-noise ratio, and covariance;

[0031] A rehabilitation evaluation determination module, configured to determine the upper limb motor ability rehabilitation evaluation information of the user to be evaluated according to the mean square error, the peak signal-to-noise ratio, the covariance, and the user muscle strength level. The upper limb motor ability rehabilitation evaluation information includes muscle strength level and wrist joint impairment condition.

[0032] According to another aspect of the present invention, there is provided an evaluation system for upper limb motor ability rehabilitation, the evaluation system for upper limb motor ability rehabilitation including a rehabilitation ball and a computer terminal;

[0033] The computer terminal includes:

[0034] At least one processor; and,

[0035] A memory communicatively connected to the at least one processor; wherein,

[0036] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the evaluation method for upper limb motor ability rehabilitation according to any embodiment of the present invention.

[0037] Optionally, the rehabilitation ball includes an upper hemispherical body, a lower hemispherical body disposed opposite to the upper hemispherical body, and a pressure sensor embedded in a card slot inside the upper hemispherical body.

[0038] According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for enabling a processor to implement the evaluation method for upper limb motor ability rehabilitation according to any embodiment of the present invention when executed.

[0039] The technical solution of the embodiment of the present invention, after a user to be evaluated completes a painting test game and a grip strength game using the rehabilitation ball, obtains an initial painting image and a current painting image corresponding to the painting test game, and pressure information and game completion information corresponding to the grip strength game; determines a painting completion quality evaluation of the user to be evaluated according to the initial painting image and the current painting image, and determines a user muscle strength level of the user to be evaluated according to the pressure information and the game completion information, the painting completion quality evaluation including mean square error, peak signal-to-noise ratio, and covariance; determines upper limb motor ability rehabilitation evaluation information of the user to be evaluated according to the mean square error, the peak signal-to-noise ratio, the covariance, and the user muscle strength level, the upper limb motor ability rehabilitation evaluation information including muscle strength level and wrist joint damage condition. The present invention solves the problem that the current evaluation of upper limb motor ability rehabilitation cannot be accurate and at the same time determines the evaluation means and criteria, realizes a comprehensive and accurate evaluation of upper limb motor ability rehabilitation, increases the fun of interaction, also helps to improve the user's participation and interest, and at the same time can provide a more effective and personalized rehabilitation experience.

[0040] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become readily understood from the following description. Description of the Drawings

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0042] Figure 1 is a flowchart of an evaluation method for upper limb motor ability rehabilitation provided according to an embodiment of the present invention;

[0043] Figures 2 to 5 is a schematic diagram of a game interface of a painting test game provided according to an embodiment of the present invention;

[0044] Figures 6 to 7 is a schematic diagram of a game interface of a grip strength game provided according to an embodiment of the present invention;

[0045] Figure 8 is a schematic structural diagram of an evaluation device for upper limb motor ability rehabilitation provided according to an embodiment of the present invention;

[0046] Figure 9 is a schematic structural diagram of an evaluation system for upper limb motor ability rehabilitation that implements the evaluation method for upper limb motor ability rehabilitation provided by the embodiments of the present invention;

[0047] Figure 10 is a three-dimensional schematic diagram of a rehabilitation ball provided according to an embodiment of the present invention;

[0048] Figure 11 is a detailed three-dimensional schematic diagram of a rehabilitation ball provided according to an embodiment of the present invention;

[0049] Figure 12 is an application scenario diagram of the evaluation method for upper limb motor ability rehabilitation provided by the embodiments of the present invention. Detailed Embodiments

[0050] In order to enable those skilled in the art to better understand the solutions of the present invention, the following clearly and completely describes the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, rather than all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0051] It should be noted that the terms "first", "second", etc. in the description, claims and above-mentioned drawings of the present invention are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that such data used can be interchanged under appropriate circumstances, so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0052] Figure 1 FIG. 1 is a flowchart of an evaluation method for upper limb motor ability rehabilitation according to Embodiment 1 of the present invention. This embodiment is applicable to the situation of rehabilitation and evaluation for those in need of upper limb motor ability rehabilitation. The evaluation method for upper limb motor ability rehabilitation can be executed by an evaluation device for upper limb motor ability rehabilitation. The evaluation device for upper limb motor ability rehabilitation can be implemented in the form of hardware and / or software, and the evaluation device for upper limb motor ability rehabilitation can be configured in an evaluation system for upper limb motor ability rehabilitation. The evaluation method for upper limb motor ability rehabilitation is applied to an evaluation system for upper limb motor ability rehabilitation. The evaluation system for upper limb motor ability rehabilitation includes a rehabilitation ball, as Figure 1 shown, and the evaluation method for upper limb motor ability rehabilitation includes:

[0053] S110. After the user to be evaluated completes the painting test game and the grip strength game using the rehabilitation ball, obtain the initial painting image and the current painting image corresponding to the painting test game, as well as the pressure information and game completion information corresponding to the grip strength game.

[0054] Among them, the painting test game is used to test the hand movements of the user to be evaluated, especially the fine hand movements. For example, when drawing edges and details, care should be taken to avoid smearing the color outside the border. Through the painting test game, the user to be evaluated can precisely control the position and strength of their hands, thereby improving the coordination ability between the hand and the eye.

[0055] The user to be evaluated can be an adult or a child in need of evaluating upper limb motor ability rehabilitation. Especially for children with cerebral palsy, the painting test game and the grip strength game provided in this embodiment are more helpful for the training and evaluation of their upper limb motor ability.

[0056] The game interface of the painting test game is as Figures 2 to 5 shown, Figure 2 showing the start interface of the painting test game, Figure 3Shows the gameplay introduction interface of the painting test game. Figure 4 Shows selectable painting graphics, such as a square, a car shape, or a cat shape, etc. Figure 5 Shows the painting image area of the painting test game. Eventually, the current painting image is Figure 5 displayed on the shown interface, and at the same time Figure 5 the initial painting image is also simultaneously displayed on the shown interface. It is known that the current painting image is the real-time image that can be obtained from the Figure 5 shown interface after the user to be evaluated completes the painting test game using the rehabilitation ball.

[0057] The core objective of the painting test game is: the user to be evaluated selects a suitable graphic according to their own ability and interest for coloring, so as to demonstrate their hand-eye coordination ability and fine motor ability.

[0058] Continue to refer to Figure 4 as shown, the painting test game provides three different difficulty levels of graphics for the user to be evaluated to choose from, and the user to be evaluated can color according to the reference diagram beside the drawing board.

[0059] Continue to refer to Figure 5 as shown, the painting test game provides five basic colors for the user to be evaluated to select. If there are mistakes during the coloring process, the eraser tool can be used for correction. In addition, while coloring, encouraging sounds can be provided simultaneously to simulate the scenario of assisting the user to be evaluated in the rehabilitation assessment in real life. This user-friendly design aims to relieve the tension of the user to be evaluated and provide a positive participation environment.

[0060] The game interface of the grip strength game is as Figures 6 to 7 shown. Figure 6 Shows the gameplay introduction interface of the grip strength game. Figure 7 Shows the operation interface of the grip strength game. The core objective of the grip strength game is: make the game character face the direction of the mushroom; the user to be evaluated needs to carefully control the strength in each round of the game to ensure that the mushroom can be picked smoothly.

[0061] On the above basis, the grip strength game detects the pressure of each finger of the user to be evaluated through the pressure sensor on the rehabilitation ball, and determines the corresponding pressure information of the grip strength game according to the pressure of each finger of the user to be evaluated; and, obtains the control steering time and the completion target grip strength reaction time when the user to be evaluated completes the grip strength game using the rehabilitation ball, and determines the corresponding game completion information of the grip strength game according to the control steering time and the completion target grip strength reaction time, and then obtains the corresponding pressure information and game completion information of the grip strength game.

[0062] S120. Determine the painting completion quality assessment of the user to be evaluated based on the initial painting image and the current painting image, and determine the user muscle strength level of the user to be evaluated based on the pressure information and the game completion information. The painting completion quality assessment includes mean square error, peak signal-to-noise ratio, and covariance.

[0063] Once effective coloring is completed in the painting test game, the user to be evaluated will enter the settlement interface, that is, obtain the current painting image. The degree to which the user to be evaluated paints outside the border during painting is reflected by the initial painting image and the current painting image. The degree to which the user to be evaluated paints outside the border during painting is directly related to hand-eye coordination ability and fine motor ability. It is not linear, but can be analyzed by statistical methods, such as mean square error (MSE).

[0064] The mean square error (MSE) is an index that measures the degree of difference between the predicted value and the true value of the model. In this embodiment, the mean square error of the user to be evaluated is determined based on the initial painting image and the current painting image. Specifically:

[0065]

[0066] where MSE is the mean square error of the user to be evaluated; m and n are the height and width of the initial painting image and the current painting image respectively; I[i, j] is the pixel value of the initial painting image at the position (i, j); I′[i, j] is the pixel value of the current painting image at the same position (i, j).

[0067] By double summation, each pixel in the initial painting image and the current painting image is traversed, so as to calculate (I[i, j] - I′[i, j]) 2 That is, the square of the difference at each pixel position. Finally, the sum of the whole is divided by the total number of pixels mn to obtain the mean square error.

[0068] It can be known that the smaller the mean square error, the smaller the difference between the initial painting image and the current painting image is determined; the larger the mean square error, the larger the difference between the initial painting image and the current painting image is determined.

[0069] In another embodiment, a distinct border is set in each painting, which is the edge pixels of the image. The internal differences between the initial painting image and the current painting image are generally small, and the main content (i.e., the non-border area) is relatively consistent. Therefore, calculating the MSE of the border can more quickly and accurately evaluate the similarity between the two images. Specifically, through the border area, the pixel differences between the initial painting image and the current painting image are calculated, and during the evaluation process, the initial painting image and the current painting image are each evenly divided into four corresponding small regions, thereby obtaining four sub-images of equal size. For each small region, it is compared with the corresponding small region in the reference image, and the MSE is calculated.

[0070]

[0071] Where I[i, j] is the pixel value of the small region in the current painting image at position (i, j); K(i, j) is the pixel value of the small region in the initial painting image at the same position; M and N are the width and height of the small region respectively.

[0072] Calculations are performed for each small region to obtain the mean square error (MSE) of the four small regions themselves and the total mean square error.

[0073] PSNR is an image quality evaluation metric based on MSE. It represents the ratio of the maximum possible power of the image signal to the power of the destructive noise that affects its representation accuracy. The unit of PSNR is usually decibels (dB), and a higher value indicates better image quality. Determine the peak signal-to-noise ratio of the user to be evaluated based on the initial painting image and the current painting image, specifically:

[0074]

[0075] Where PSNR is the peak signal-to-noise ratio of the user to be evaluated; Peak is the maximum value representing the pixel intensity of the image. If each sampling point is represented by 8 bits, then Peak = 255.

[0076] SSIM is a metric for measuring the visual similarity between the initial painting image and the current painting image. It comprehensively evaluates the image quality from three key dimensions: luminance similarity, contrast similarity, and structural similarity. Among them, luminance similarity is estimated by calculating the mean value, contrast similarity is measured by the standard deviation, and the measure of structural similarity degree depends on the calculation of covariance.

[0077] Determine the covariance of the user to be evaluated based on the initial painting image and the current painting image, specifically:

[0078]

[0079] Among them, SSIM(x, y) is the covariance of the user to be evaluated; x and y represent the pixel values of the initial painting image and the current painting image respectively; μ x and μ y are the means of the initial painting image and the current painting image; and are the variances of the initial painting image and the current painting image; σ xy is the covariance of the initial painting image and the current painting image; C1 and C2 are constants used to avoid a zero denominator.

[0080] It can be understood that since PSNR mainly focuses on pixel-level errors while SSIM focuses more on the structure and visual perception of images. Therefore, using both of them together can provide a more comprehensive image quality assessment.

[0081] In addition, it should be noted that the maximum activity angles of the user to be evaluated in palmar flexion, dorsiflexion, radial deviation, and ulnar deviation are collected through the WHEELTEC N100 inertial measurement unit in the rehabilitation ball, and the maximum activity angles of palmar flexion, dorsiflexion, radial deviation, and ulnar deviation are saved in the upper limb motor ability rehabilitation assessment information of the user to be evaluated, so as to obtain more accurate information on the degree of wrist joint damage. The collection and analysis of these data are of great significance for clinicians to develop personalized rehabilitation plans.

[0082] Before the grip strength game starts, voice guidance can be played to remind the user to be evaluated to hold the small ball with all their strength and conduct a maximum grip strength test. This is done to evaluate and quantify the patient's muscle control ability and exercise the patient's hand muscles to the greatest extent.

[0083] The maximum grip strength test is specifically as follows: in each of the ten finger grooves of the rehabilitation ball, a pressure sensor is designed. The pressure sensor responds very quickly to pressure changes. Therefore, when the user to be evaluated holds the small ball with all their strength, each pressure sensor can react to the corresponding pressure F in real time. After the user to be evaluated completes the maximum grip strength test, the mean value of the numerical values of the ten pressure sensors is calculated as the maximum grip strength of the user to be evaluated. This maximum grip strength will be used as a target for each subsequent round of the game.

[0084] Continue to refer to Figure 6 and Figure 7As shown, after the test of the maximum grip strength is completed, the grip strength game officially starts. The user to be evaluated needs to rotate the wrist to make the game character face the direction of the mushroom, which is the key to achieving the goal. The user to be evaluated needs to continuously adjust the force of holding the ball until the required grip strength is reached. When the hand is released, the game character will move in the direction it is facing. If the user to be evaluated uses insufficient force, the mushroom cannot be reached and no score can be obtained. On the contrary, if the user to be evaluated uses excessive force, it may cause the game character to cross over the mushroom and also no score can be obtained. The game will gradually increase the rotation angle over time, and correspondingly, the grip strength required by the user to be evaluated will also increase.

[0085] Furthermore, after the grip strength game ends, a comprehensive analysis is carried out based on the maximum rotation angle and the maximum force that can be held by the user to be evaluated in each round of the game. The most important evaluation parameter in the comprehensive analysis is time. The completion time T of each round of the game will be recorded. t , and this time will be used as an important basis in the subsequent analysis report. In addition, the completion time T of each round of the game is divided into: the time T d for the user to be evaluated to control the game character to turn by the wrist and the reaction time T s for the user to be evaluated to hold the small ball tightly to achieve the target grip strength. T d can represent the hand-eye coordination and the quick response ability of the user to be evaluated in controlling the game. As the number of games increases, the user to be evaluated will gradually shorten T d through game training, which indicates the continuous improvement of the wrist rotation ability of the user to be evaluated. T s reflects the reaction speed and grip strength control ability of the patient and can also be improved through game training. T = T d + T s , T t = nT = T1 + T2 + … + T n .

[0086] Therefore, in the grip strength game, the time parameter is crucial for evaluating the wrist and muscle abilities of the patient, optimizing the game design, and enhancing the user experience. By deeply analyzing these time parameters and the wrist rotation transmitted by the IMU, a comprehensive analysis is carried out to give a grade of the muscle strength of the user to be evaluated.

[0087] S130. Determine the rehabilitation assessment information of the upper limb motor ability of the user to be evaluated according to the mean square error, peak signal-to-noise ratio, covariance, and the user muscle strength grade. The rehabilitation assessment information of the upper limb motor ability includes the muscle strength level and the wrist joint damage condition.

[0088] The mean squared error (MSE) reflects the degree to which the user to be evaluated paints outside the border during painting. The degree to which the user to be evaluated paints outside the border during painting is directly related to the hand-eye coordination ability and fine motor ability. After calculating the mean squared error (MSE), the peak signal-to-noise ratio (PSNR) and the structural similarity index (SSIM) are further used to evaluate the similarity and quality difference between the initial painting image and the current painting image. Through the analysis of PSNR and SSIM, the wrist control ability of the user is reflected from the side, that is, the damaged condition of the wrist joint of the user to be evaluated is obtained.

[0089] The muscle strength level of the user not only reflects the current muscle strength level of the user to be evaluated, but also reveals the degree of progress of the user to be evaluated in controlling muscle strength, that is, the muscle strength level of the user to be evaluated is obtained.

[0090] In the technical solution of the embodiment of the present invention, after the user to be evaluated completes the painting test game and the grip strength game using the rehabilitation ball, the initial painting image and the current painting image corresponding to the painting test game, as well as the pressure information and the game completion information corresponding to the grip strength game are obtained; the painting completion quality evaluation of the user to be evaluated is determined according to the initial painting image and the current painting image, and the muscle strength level of the user to be evaluated is determined according to the pressure information and the game completion information. The painting completion quality evaluation includes the mean squared error, the peak signal-to-noise ratio and the covariance; the upper limb motor ability rehabilitation evaluation information of the user to be evaluated is determined according to the mean squared error, the peak signal-to-noise ratio, the covariance and the muscle strength level. The upper limb motor ability rehabilitation evaluation information includes the muscle strength level and the damaged condition of the wrist joint. The present invention solves the problem that the current evaluation of the upper limb motor ability rehabilitation cannot be accurate, and at the same time determines the evaluation means and standards, realizes the comprehensive and accurate evaluation of the upper limb motor ability rehabilitation, increases the fun of interaction, and also helps to improve the user's participation and interest. At the same time, it can provide a more effective and personalized rehabilitation experience.

[0091] Figure 8 It is a schematic structural diagram of an evaluation device for upper limb motor ability rehabilitation provided by an embodiment of the present invention. The evaluation device for upper limb motor ability rehabilitation is applied to an evaluation system for upper limb motor ability rehabilitation. The evaluation system for upper limb motor ability rehabilitation includes a rehabilitation ball, as Figure 8 shown, the evaluation device for upper limb motor ability rehabilitation includes:

[0092] An information acquisition module 210, configured to execute the operation of obtaining the initial painting image and the current painting image corresponding to the painting test game, as well as the pressure information and the game completion information corresponding to the grip strength game after the user to be evaluated completes the painting test game and the grip strength game using the rehabilitation ball;

[0093] An evaluation information determination module 220, configured to perform a determination of the painting completion quality evaluation of the user to be evaluated based on the initial painting image and the current painting image, and a determination of the user muscle strength level of the user to be evaluated based on the pressure information and the game completion information. The painting completion quality evaluation includes mean square error, peak signal-to-noise ratio, and covariance;

[0094] A rehabilitation evaluation determination module 230, configured to perform a determination of the upper limb motor ability rehabilitation evaluation information of the user to be evaluated based on the mean square error, peak signal-to-noise ratio, covariance, and the user muscle strength level. The upper limb motor ability rehabilitation evaluation information includes muscle strength level and wrist joint impairment condition.

[0095] Optionally, determining the painting completion quality evaluation of the user to be evaluated based on the initial painting image and the current painting image is specifically used for:

[0096] Determining the mean square error of the user to be evaluated based on the initial painting image and the current painting image, specifically as follows:

[0097]

[0098] where MSE is the mean square error of the user to be evaluated; m and n are the height and width of the initial painting image and the current painting image respectively; I[i, j] is the pixel value of the initial painting image at the position (i, j); I′[i, j] is the pixel value of the current painting image at the same position (i, j);

[0099] Or,

[0100] where I[i, j] is the pixel value of the small area in the current painting image at the position (i, j); K(i, j) is the pixel value of the small area in the initial painting image at the same position; M and N are the width and height of the small area respectively.

[0101] Optionally, the smaller the mean square error, the smaller the difference between the initial painting image and the current painting image is determined;

[0102] The larger the mean square error, the larger the difference between the initial painting image and the current painting image is determined.

[0103] Optionally, determining the painting completion quality evaluation of the user to be evaluated based on the initial painting image and the current painting image is specifically used for:

[0104] Determining the peak signal-to-noise ratio of the user to be evaluated based on the initial painting image and the current painting image, specifically as follows:

[0105]

[0106] Wherein, PSNR is the peak signal-to-noise ratio of the user to be evaluated; Peak represents the maximum value of the pixel intensity of the image;

[0107] Determine the covariance of the user to be evaluated according to the initial painting image and the current painting image, specifically:

[0108]

[0109] Wherein, SSIM(x, y) is the covariance of the user to be evaluated; x and y respectively represent the pixel values of the initial painting image and the current painting image; μ x and μ y are the means of the initial painting image and the current painting image; and are the variances of the initial painting image and the current painting image; σ xy is the covariance of the initial painting image and the current painting image; C1 and C2 are constants.

[0110] Optionally, obtain the pressure information and game completion information corresponding to the grip strength game, specifically for:

[0111] Detect the pressure of each finger of the user to be evaluated through the pressure sensor on the rehabilitation ball, and determine the pressure information corresponding to the grip strength game according to the pressure of each finger of the user to be evaluated;

[0112] Obtain the control steering time and the completion target grip strength reaction time when the user to be evaluated uses the rehabilitation ball to complete the grip strength game, and determine the game completion information corresponding to the grip strength game according to the control steering time and the completion target grip strength reaction time.

[0113] Optionally, the evaluation device for upper limb motor ability rehabilitation further includes:

[0114] An activity angle collection module, configured to collect the maximum activity angles of the user to be evaluated in palmar flexion, dorsiflexion, radial deviation, and ulnar deviation, and save the maximum activity angles of palmar flexion, dorsiflexion, radial deviation, and ulnar deviation in the upper limb motor ability rehabilitation evaluation information of the user to be evaluated.

[0115] The evaluation device for upper limb motor ability rehabilitation provided by the embodiments of the present invention can execute the evaluation method for upper limb motor ability rehabilitation provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the evaluation method for upper limb motor ability rehabilitation.

[0116] Figure 9FIG. 0 shows a schematic structural diagram of an upper limb motor ability rehabilitation evaluation system 310 that can be used to implement embodiments of the present invention. The upper limb motor ability rehabilitation evaluation system includes a rehabilitation ball and a computer terminal. The computer terminal is intended to represent various forms of digital computers, such as, for example, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The computer terminal can also represent various forms of mobile devices, such as, for example, personal digital processors, cellular telephones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0117] As Figure 9 shown, the computer terminal 310 includes at least one processor 311, and a memory communicatively connected to the at least one processor 311, such as read-only memory (ROM 312), random access memory (RAM 313), etc. The memory stores a computer program executable by the at least one processor. The processor 311 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM 312) or the computer program loaded from the storage unit 318 into the random access memory (RAM 313). In the RAM 313, various programs and data required for the operation of the computer terminal 310 can also be stored. The processor 311, ROM 312, and RAM 313 are connected to each other via a bus 314. The I / O (input / output) interface 315 is also connected to the bus 314.

[0118] Multiple components in the computer terminal 310 are connected to the I / O interface 315, including: an input unit 316, such as a keyboard, a mouse, etc.; an output unit 317, such as various types of displays, speakers, etc.; a storage unit 318, such as a magnetic disk, an optical disk, etc.; and a communication unit 319, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 319 allows the computer terminal 310 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0119] The processor 311 can be various general and / or special processing components with processing and computing capabilities. Some examples of the processor 311 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 311 executes the various methods and processes described above, such as the upper limb motor ability rehabilitation evaluation method.

[0120] In some embodiments, the evaluation method for upper limb motor ability rehabilitation can be implemented as a computer program, which is tangibly embodied in a computer-readable storage medium, such as storage unit 318. In some embodiments, part or all of the computer program can be loaded and / or installed onto computer terminal 310 via ROM 312 and / or communication unit 319. When the computer program is loaded into RAM 313 and executed by processor 311, one or more steps of the evaluation method for upper limb motor ability rehabilitation described above can be performed. Alternatively, in other embodiments, processor 311 can be configured to execute the evaluation method for upper limb motor ability rehabilitation by any other suitable means (e.g., by means of firmware).

[0121] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs, which can be executed and / or interpreted on a programmable system including at least one programmable processor, the programmable processor can be a dedicated or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0122] The computer program for implementing the method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a dedicated computer, or other programmable data processing devices, such that when the computer programs are executed by the processor, the functions / operations specified in the flowchart and / or block diagram are implemented. The computer programs can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0123] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0124] To provide for interaction with a user, the systems and techniques described herein can be implemented on a computer terminal having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer terminal. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0125] The systems and techniques described herein can be implemented in a computing system that includes backend components (such as, for example, a data server), or a computing system that includes middleware components (such as, for example, an application server), or a computing system that includes frontend components (such as, for example, a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (such as, for example, a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0126] A computing system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The client-server relationship is created by computer programs running on respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0127] The rehabilitation ball included in the upper limb motor ability rehabilitation evaluation system can be a 3D printed spherical interaction device. Figure 10 For the three-dimensional schematic diagram of the rehabilitation ball provided by the embodiment of the present invention, see Figure 10 As shown, the rehabilitation ball includes an upper hemisphere, a lower hemisphere disposed opposite to the upper hemisphere, and a pressure sensor embedded in the internal card slot of the upper hemisphere. The pressure sensor can use an SF15 thin film flexible pressure sensor. Grooves designed specifically for children are reserved on the surfaces of the upper and lower layers of the rehabilitation ball, which can meet the special needs of children when grasping or rotating the device, increasing the practicality of the device. The internal structure of the rehabilitation ball is precise, containing multiple key components to ensure the accuracy and efficiency of data processing.

[0128] The interior of the rehabilitation ball includes an ESP32 microcontroller, a Wheeltec N100 IMU inertial measurement unit (i.e., an IMU gyroscope), and a TP4056 charging module. These components work together to ensure that the device can achieve precise motion control and data recording. See Figure 11 As shown, when the ESP32 microcontroller is configured in the rehabilitation ball, the ESP32 microcontroller is fixed by an ESP32 upper bracket and an ESP32 lower bracket. The ESP32 microcontroller is used to integrate the three-axis acceleration data from the IMU gyroscope and the data from the flexible thin film pressure sensor, connect the IMU gyroscope to the main controller through the Usart interface, and combine the high-precision detection ability of the flexible thin film pressure sensor to accurately capture the fluctuations of various values during the game operation. This enables children to feel more realistic physical feedback during the game.

[0129] The ESP32 microcontroller is the core of the entire rehabilitation ball. It is a high-performance, low-power microcontroller that depends on a built-in battery during normal operation. The ESP32 microcontroller receives data from the IMU gyroscope and the flexible thin film pressure sensor, and also uses Bluetooth technology to achieve rapid information transmission. Once the data collection is completed, the ESP32 microcontroller quickly transmits the data to a remote computer or other intelligent devices through the Bluetooth module, improving the flexibility and practicality of the system.

[0130] Thin film flexible pressure sensors are particularly suitable for integration into rehabilitation balls. They can provide accurate and reliable data both when accurately measuring tiny pressure changes and when simulating pressure conditions in real environments.

[0131] In order to improve the convenience of children's operation, the rehabilitation ball is equipped with a Bluetooth module, which can easily transmit data to the game program developed by Unity. In this way, children can freely choose to control the ball with one hand or two hands. Whether it is single-handed palm control or two-handed collaboration, the game experience can be kept consistent and stable.

[0132] The outer shell of the rehabilitation ball is made with 3D printing technology. After printing, when children place their fingers in the grooves on the surface of the device, they can easily hold the device and accurately fix it in the right position. These finger grooves are not only for support, but also embedded with high-precision pressure sensors that can sensitively sense changes in children's strength, thereby effectively measuring their strength level. At the same time, in order to further enhance the convenience of the device for users, a C-type charging interface and a switching switch groove are reserved on the surface of the rehabilitation ball. Considering various situations that may be encountered in daily use, it provides convenience for charging the device.

[0133] There are some card slots inside the upper and lower layers of the rehabilitation ball, which are inserted into and fixed in a special test board to minimize the mechanical damage to the chip and sensor components in the rehabilitation ball. Whether it is daily use or working environment under extreme conditions, the stability and reliability of the rehabilitation ball can be ensured, and it is avoided as much as possible from being affected by external factors, so that the rehabilitation ball can stably and reliably play its due performance.

[0134] The upper shell of the rehabilitation ball is equipped with five finger grooves to reserve fixed positions for internal components. The thin film flexible pressure sensor uses resistive sensing technology to measure the relationship between a specific pressure value and a change in resistance. Similarly, the lower shell of the rehabilitation ball is also equipped with five finger grooves, see Figure 11 As shown in the figure, the internal card slot of the lower layer of the rehabilitation ball contains the charging module, battery and ESP32 lower bracket. The battery of the rehabilitation ball is a high-efficiency lithium battery with a capacity of 2500mAh and a voltage standard of 3.7V to meet the power requirements of the entire device. The battery is charged through the charging module, which is not only responsible for charging the battery, but also for storing the charged electricity, ensuring the durability and stability of the rehabilitation ball.

[0135] For example, see Figure 12As shown, during the game process, the computer terminal continuously monitors the visual and auditory feedback generated by the eyes and ears of the user to be evaluated. This data is then analyzed by the computer to better understand the child's physical condition. By carefully processing the data generated during the game and combining the results of gesture tracking, the computer terminal can generate a detailed evaluation report. This report is an important basis for the personalized rehabilitation plan, which details the child's activity performance and health status. After receiving this report, doctors and parents can quickly understand the child's performance in the game, thereby timely adjusting or formulating a new rehabilitation plan. This interactive evaluation method not only enhances the enthusiasm of children during the rehabilitation process but also provides valuable reference information for doctors, helping to further optimize the rehabilitation plan.

[0136] It should be understood that various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. No limitations are imposed herein.

[0137] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub - combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. An evaluation method for upper limb motor ability rehabilitation, characterized in that, The evaluation method for upper limb motor ability rehabilitation is applied to an evaluation system for upper limb motor ability rehabilitation. The evaluation system for upper limb motor ability rehabilitation includes a rehabilitation ball. The evaluation method for upper limb motor ability rehabilitation includes: After the user to be evaluated completes a painting test game and a grip strength game using the rehabilitation ball, obtain the initial painting image and the current painting image corresponding to the painting test game, as well as the pressure information and game completion information corresponding to the grip strength game; wherein, the painting test game is used to evaluate the fine hand movements of the user to be evaluated, and the grip strength game is used to evaluate and quantify the muscle control ability of the user to be evaluated; Determine the painting completion quality evaluation of the user to be evaluated based on the initial painting image and the current painting image, and determine the user muscle strength level of the user to be evaluated based on the pressure information and the game completion information. The painting completion quality evaluation includes mean square error, peak signal-to-noise ratio, and covariance; wherein, the mean square error is used to reflect the degree to which the user to be evaluated exceeds the border during painting, and the user muscle strength level is used to obtain the muscle strength level of the user to be evaluated; Determine the upper limb motor ability rehabilitation evaluation information of the user to be evaluated based on the mean square error, the peak signal-to-noise ratio, the covariance, and the user muscle strength level. The upper limb motor ability rehabilitation evaluation information includes muscle strength level and wrist joint damage condition.

2. The evaluation method for upper limb motor ability rehabilitation according to claim 1, wherein Determining the painting completion quality evaluation of the user to be evaluated based on the initial painting image and the current painting image includes: Determine the mean square error of the user to be evaluated based on the initial painting image and the current painting image. Specifically: where MSE is the mean square error of the user to be evaluated; m and n are the height and width of the initial painting image and the current painting image respectively; I[i, j] is the pixel value of the initial painting image at the position (i, j); I ′ [i, j] is the pixel value of the current painting image at the same position (i, j); Or, where I[i,j] is the pixel value of the small area in the current painting image at position (i,j); K(i,j) is the pixel value of the small area in the initial painting image at the same position; M and N are the width and height of the small area respectively.

3. The evaluation method for upper limb motor ability rehabilitation according to claim 2, characterized in that, The smaller the mean square error, the smaller the difference between the initial painting image and the current painting image is determined; The larger the mean square error, the larger the difference between the initial painting image and the current painting image is determined.

4. The evaluation method for upper limb motor ability rehabilitation according to claim 1, characterized in that, Determining the painting completion quality evaluation of the user to be evaluated based on the initial painting image and the current painting image includes: Determine the peak signal-to-noise ratio of the user to be evaluated based on the initial painting image and the current painting image. Specifically: where PSNR is the peak signal-to-noise ratio of the user to be evaluated; Peak is the maximum value representing the pixel intensity of the image; Determine the covariance of the user to be evaluated based on the initial painting image and the current painting image. Specifically: Among them, SSIM(x, y) is the covariance of the user to be evaluated; x and y respectively represent the pixel values of the initial painting image and the current painting image; μ x and μ y are the means of the initial painting image and the current painting image; and are the variances of the initial painting image and the current painting image; σ xy is the covariance of the initial painting image and the current painting image; C1 and C2 are constants.

5. The evaluation method for upper limb motor ability rehabilitation according to claim 1, characterized in that, Obtaining the pressure information and game completion information corresponding to the grip strength game includes: Detect the pressure of each finger of the user to be evaluated through the pressure sensor on the rehabilitation ball, and determine the pressure information corresponding to the grip strength game based on the pressure of each finger of the user to be evaluated; Obtain the control steering time and the completion target grip force response time when the user to be evaluated completes the grip force game using the rehabilitation ball, and determine the game completion information corresponding to the grip force game according to the control steering time and the completion target grip force response time.

6. The evaluation method for upper limb motor ability rehabilitation according to claim 1, wherein The evaluation method for upper limb motor ability rehabilitation further includes: Collect the maximum movement angles of the user to be evaluated in palmar flexion, dorsiflexion, radial deviation, and ulnar deviation, and save the maximum movement angles of palmar flexion, dorsiflexion, radial deviation, and ulnar deviation in the upper limb motor ability rehabilitation evaluation information of the user to be evaluated.

7. An evaluation device for upper limb motor ability rehabilitation, characterized in that, The evaluation device for upper limb motor ability rehabilitation is applied to an upper limb motor ability rehabilitation evaluation system, the upper limb motor ability rehabilitation evaluation system includes a rehabilitation ball, and the evaluation device for upper limb motor ability rehabilitation includes: An information acquisition module, configured to execute to obtain the initial drawing image and the current drawing image corresponding to the drawing test game, and the pressure information and game completion information corresponding to the grip force game after the user to be evaluated completes the drawing test game and the grip force game using the rehabilitation ball; wherein, the drawing test game is used to evaluate the fine hand movements of the user to be evaluated, and the grip force game is used to evaluate and quantify the muscle control ability of the user to be evaluated. An evaluation information determination module, configured to execute to determine the drawing completion quality evaluation of the user to be evaluated according to the initial drawing image and the current drawing image, and determine the user muscle strength level of the user to be evaluated according to the pressure information and the game completion information, the drawing completion quality evaluation includes mean square error, peak signal-to-noise ratio, and covariance; wherein, the mean square error is used to reflect the degree to which the user to be evaluated protrudes beyond the border when drawing, and the user muscle strength level is used to obtain the muscle strength level of the user to be evaluated. A rehabilitation evaluation determination module, configured to execute to determine the upper limb motor ability rehabilitation evaluation information of the user to be evaluated according to the mean square error, the peak signal-to-noise ratio, the covariance, and the user muscle strength level, the upper limb motor ability rehabilitation evaluation information includes muscle strength level and wrist joint damage condition.

8. An evaluation system for upper limb motor ability rehabilitation, characterized in that, The upper limb motor ability rehabilitation evaluation system includes a rehabilitation ball and a computer terminal; The computer terminal includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the upper limb motor ability rehabilitation evaluation method according to any one of claims 1-6.

9. The evaluation system for upper limb motor ability rehabilitation according to claim 8, characterized in that, The rehabilitation ball includes an upper hemispherical body, a lower hemispherical body disposed opposite to the upper hemispherical body, and a pressure sensor embedded in a card slot inside the upper hemispherical body.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a processor to execute the upper limb motor ability rehabilitation evaluation method according to any one of claims 1-6 when executed.

Images