Evaluation method, device and system for upper limb exercise ability rehabilitation and storage medium

Abstract

The invention discloses an assessment method, device and system for upper limb exercise ability rehabilitation and a storage medium. The method comprises the following steps: after a to-be-evaluated user completes a drawing test game and a grip game by using a rehabilitation ball, obtaining an initial drawing image and a current drawing image corresponding to the drawing test game, and pressure information and game completion information corresponding to the grip game; according to the initial drawing image and the current drawing image, drawing completion quality evaluation of the to-be-evaluated user is determined, according to the pressure information and the game completion information, the user muscle strength level of the to-be-evaluated user is determined, and drawing completion quality evaluation comprises mean square error, peak signal-to-noise ratio and covariance; according to the mean square error, the peak signal-to-noise ratio, the covariance and the user muscle strength level, upper limb exercise ability rehabilitation evaluation information of the user to be evaluated is determined, wherein the upper limb exercise ability rehabilitation evaluation information comprises the muscle strength level and the wrist joint damage condition. According to the invention, comprehensive upper limb exercise ability rehabilitation evaluation is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of sports rehabilitation assessment, and in particular to an assessment method, device, system and storage medium for upper limb motor ability rehabilitation. Background Art

[0002] Human daily life cannot be separated from the participation of upper limb functions. Decreased hand mobility will directly affect the patient's ability to carry out daily activities and social participation. Normal upper limb functions mainly include keeping the forearm in different positions, grasping and releasing objects of different shapes, transferring objects from one place to another, operating various tools, and operating with both hands. Therefore, residual upper limb dysfunction will seriously affect the patient's ability to carry out daily activities. Upper limb function assessment runs through the entire rehabilitation treatment process, but the current assessment of upper limb motor rehabilitation relies on professional medical staff, and the process is boring. At the same time, there is a lack of assessment methods and standards for children. Summary of the invention

[0003] The present invention provides an upper limb motor ability rehabilitation assessment method, device, system and storage medium to solve the current problem that the upper limb motor ability rehabilitation assessment cannot be accurate and the assessment means and standards cannot be determined at the same time.

[0004] According to one aspect of the present invention, a method for evaluating upper limb motor ability rehabilitation is provided. The method for evaluating upper limb motor ability rehabilitation is applied to an evaluation system for upper limb motor ability rehabilitation. The evaluation system for upper limb motor ability rehabilitation includes a rehabilitation ball. The method for evaluating upper limb motor ability rehabilitation includes:

[0005] After the user to be evaluated completes the drawing test game and the grip strength game using the rehabilitation ball, obtaining an initial drawing image and a current drawing image corresponding to the drawing test game, and pressure information and game completion information corresponding to the grip strength game;

[0006] Determine a painting completion quality assessment of the user to be assessed according to the initial painting image and the current painting image, and determine a user muscle strength level of the user to be assessed according to the pressure information and the game completion information, wherein the painting completion quality assessment includes a mean square error, a peak signal-to-noise ratio, and a covariance;

[0007] The upper limb motor ability rehabilitation assessment information of the user to be assessed is determined according to the mean square error, the peak signal-to-noise ratio, the covariance and the user's muscle strength level, and the upper limb motor ability rehabilitation assessment information includes muscle strength level and wrist joint damage.

[0008] Optionally, determining the quality assessment of the painting completion of the to-be-assessed user according to the initial painting image and the current painting image includes:

[0009] Determining the mean square error of the user to be evaluated according to the initial painting image and the current painting image is specifically as follows:

[0010]

[0011] Wherein, MSE is the mean square error of the user to be evaluated; m and n are the height and width of the initial painting image and the current painting image, respectively; I[i, j] is the pixel value of the initial painting image at position (i, j); I′[i, j] is the pixel value of the current painting image at the same position (i, j);

[0012] or,

[0013] Among them, I[i, j] is the pixel value of the small area at position (i, j) in the current painting image; K(i, j) is the pixel value of the small area at the same position in the initial painting image; M and N are the width and height of the small area respectively.

[0014] Optionally, the smaller the mean square error is, the smaller the difference between the initial painting image and the current painting image is determined to be;

[0015] The larger the mean square error is, the greater the difference between the initial painting image and the current painting image is determined to be.

[0016] Optionally, determining the quality assessment of the painting completion of the to-be-assessed user according to the initial painting image and the current painting image includes:

[0017] Determining the peak signal-to-noise ratio of the user to be evaluated according to the initial painting image and the current painting image is specifically as follows:

[0018]

[0019] Wherein, PSNR is the peak signal-to-noise ratio of the user to be evaluated; Peak is the maximum value of the image pixel intensity;

[0020] Determining the covariance of the user to be evaluated according to the initial painting image and the current painting image is specifically as follows:

[0021]

[0022] Wherein, SSIM(x, y) is the covariance of the user to be evaluated; x and y represent the pixel values ​​of the initial painting image and the current painting image respectively; μ x and μ y is the mean of the initial painting image and the current painting image; and is the variance of the initial painting image and the current painting image; σ xy is the covariance between the initial painting image and the current painting image; C1 and C2 are constants.

[0023] Optionally, obtaining pressure information and game completion information corresponding to the grip game includes:

[0024] Detecting the pressure of each finger of the user to be evaluated through the pressure sensor on the rehabilitation ball, and determining the pressure information corresponding to the grip game according to the pressure of each finger of the user to be evaluated;

[0025] The control turning time and the target grip reaction time of the user to be evaluated when completing the grip game using the rehabilitation ball are obtained, and the game completion information corresponding to the grip game is determined according to the control turning time and the target grip reaction time.

[0026] Optionally, the upper limb motor ability rehabilitation assessment method further includes:

[0027] The maximum activity angles of the user to be evaluated in palmar flexion, dorsiflexion, radial deviation and ulnar deviation are collected, and the maximum activity angles of the palmar flexion, dorsiflexion, radial deviation and ulnar deviation are stored in the upper limb motor ability rehabilitation assessment information of the user to be evaluated.

[0028] According to another aspect of the present invention, an upper limb motor ability rehabilitation assessment device is provided, the upper limb motor ability rehabilitation assessment device is applied to an upper limb motor ability rehabilitation assessment system, the upper limb motor ability rehabilitation assessment system includes a rehabilitation ball, and the upper limb motor ability rehabilitation assessment device includes:

[0029] an information acquisition module, configured to acquire an initial drawing image and a current drawing image corresponding to the drawing test game, and pressure information and game completion information corresponding to the grip strength game after the user to be evaluated completes the drawing test game and the grip strength game using the rehabilitation ball;

[0030] An evaluation information determination module, configured to determine a painting completion quality evaluation of the user to be evaluated according to the initial painting image and the current painting image, and to determine a user muscle strength level of the user to be evaluated according to the pressure information and the game completion information, wherein the painting completion quality evaluation includes a mean square error, a peak signal-to-noise ratio, and a covariance;

[0031] The rehabilitation assessment determination module is used to determine the upper limb motor ability rehabilitation assessment information of the user to be assessed based on the mean square error, the peak signal-to-noise ratio, the covariance and the user's muscle strength level, wherein the upper limb motor ability rehabilitation assessment information includes muscle strength level and wrist joint damage.

[0032] According to another aspect of the present invention, there is provided an upper limb motor ability rehabilitation evaluation system, the upper limb motor ability rehabilitation evaluation system comprising a rehabilitation ball and a computer terminal;

[0033] The computer terminal comprises:

[0034] at least one processor; and,

[0035] a memory communicatively connected to the at least one processor; wherein,

[0036] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the assessment method for upper limb motor ability rehabilitation described in any embodiment of the present invention.

[0037] Optionally, the rehabilitation ball includes an upper hemisphere, a lower hemisphere arranged opposite to the upper hemisphere, and a pressure sensor embedded in a slot inside the upper hemisphere.

[0038] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the upper limb motor ability rehabilitation assessment method described in any embodiment of the present invention when executed.

[0039] The technical solution of the embodiment of the present invention is to obtain the initial drawing image and the current drawing image corresponding to the drawing test game, as well as the pressure information and the game completion information corresponding to the grip game after the user to be evaluated completes the drawing test game and the grip game using the rehabilitation ball; determine the drawing completion quality evaluation of the user to be evaluated according to the initial drawing image and the current drawing image, and determine the user muscle strength level of the user to be evaluated according to the pressure information and the game completion information, wherein the drawing completion quality evaluation includes mean square error, peak signal-to-noise ratio and covariance; determine the upper limb motor ability rehabilitation evaluation information of the user to be evaluated according to the mean square error, the peak signal-to-noise ratio, the covariance and the user muscle strength level, wherein the upper limb motor ability rehabilitation evaluation information includes muscle strength level and wrist joint damage. The present invention solves the problem that the current evaluation of upper limb motor ability rehabilitation cannot be accurate, and the evaluation means and standards cannot be determined at the same time, realizes a comprehensive and accurate evaluation of upper limb motor ability rehabilitation, increases the fun of interaction, and also helps to improve the user's participation and interest, and can provide a more effective and personalized rehabilitation experience.

[0040] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 is a flow chart of an upper limb motor ability rehabilitation evaluation method provided according to an embodiment of the present invention;

[0043] Figures 2 to 5 is a schematic diagram of a game interface of a drawing test game provided according to an embodiment of the present invention;

[0044] Figure 6 to Figure 7 is a schematic diagram of a game interface of a grip strength game provided according to an embodiment of the present invention;

[0045] Figure 8 is a schematic structural diagram of an upper limb motor ability rehabilitation assessment device provided according to an embodiment of the present invention;

[0046] Fig. 9 It is a structural schematic diagram of an upper limb motor ability rehabilitation evaluation system that implements the upper limb motor ability rehabilitation evaluation method provided by an embodiment of the present invention;

[0047] Fig.10 It is a three-dimensional schematic diagram of a rehabilitation ball provided by an embodiment of the present invention;

[0048] Fig.11 is a detailed three-dimensional schematic diagram of a rehabilitation ball provided by an embodiment of the present invention;

[0049] Fig.12 It is an application scenario diagram of the method for evaluating upper limb motor ability rehabilitation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0051] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0052] Figure 1 A flowchart of an upper limb motor ability rehabilitation evaluation method is provided for the first embodiment of the present invention. This embodiment can be applied to situations where upper limb motor ability rehabilitation is required for rehabilitation and evaluation. The upper limb motor ability rehabilitation evaluation method can be performed by an upper limb motor ability rehabilitation evaluation device. The upper limb motor ability rehabilitation evaluation device can be implemented in the form of hardware and / or software. The upper limb motor ability rehabilitation evaluation device can be configured in an upper limb motor ability rehabilitation evaluation system. The upper limb motor ability rehabilitation evaluation method is applied to an upper limb motor ability rehabilitation evaluation system. The upper limb motor ability rehabilitation evaluation system includes a rehabilitation ball, such as Figure 1 As shown, the assessment method for upper limb motor rehabilitation includes:

[0053] S110. After the user to be evaluated completes the drawing test game and the grip strength game using the rehabilitation ball, an initial drawing image and a current drawing image corresponding to the drawing test game, and pressure information and game completion information corresponding to the grip strength game are obtained.

[0054] Among them, the painting test game is used to test the hand movements of the users to be evaluated, especially the fine hand movements. For example, when drawing edges and details, one must be careful to avoid applying colors outside the borders. Through the painting test game, the users to be evaluated can accurately control their hand position and strength, thereby improving the coordination between their hands and eyes.

[0055] The user to be evaluated may be an adult or child who needs to be evaluated for upper limb motor ability rehabilitation, especially for children with cerebral palsy. The drawing test game and grip game provided in this embodiment are more helpful for the training and evaluation of their upper limb motor ability.

[0056] The game interface of the painting test game is as follows Figures 2 to 5 As shown, Figure 2 The start interface of the painting test game is shown. Figure 3The gameplay introduction interface of the painting test game is shown. Figure 4 Shows selectable drawing shapes, such as a square, a car shape, or a cat shape, etc. Figure 5 The drawing image area of ​​the drawing test game is shown. Finally, the current drawing image is in Figure 5 The interface shown is displayed, and Figure 5 The interface also displays an initial drawing image. It can be seen that the current drawing image is the image that can be viewed after the user to be evaluated completes the drawing test game using the rehabilitation ball. Figure 5 The real-time image obtained on the interface shown.

[0057] The core goal of the drawing test game is that the user to be evaluated chooses appropriate graphics to color according to his or her abilities and interests to demonstrate his or her hand-eye coordination and fine motor skills.

[0058] Continue to see Figure 4 As shown, the drawing test game provides three graphics of different difficulty levels for the user to be evaluated to choose from, and the user to be evaluated can color them according to the reference picture next to the drawing board.

[0059] Continue to see Figure 5 As shown in the figure, the painting test game provides five basic colors for the user to be evaluated to choose from. If there is a mistake in the coloring process, the eraser tool can be used to correct it. In addition, while coloring, encouraging voices can be provided at the same time, simulating the scene of assisting the user to be evaluated in rehabilitation assessment in real life. This humanized design aims to reduce the tension of the user to be evaluated and provide an environment for active participation.

[0060] The game interface of the grip game is as follows Figure 6 to Figure 7 As shown, Figure 6 The gameplay introduction interface of the grip strength game is shown. Figure 7 The operation interface of the grip game is shown. The core goal of the grip game is to make the game character face the direction of the mushrooms. The user to be evaluated needs to carefully control the strength in each round of the game to ensure that the mushrooms can be picked smoothly.

[0061] On the basis of the above, the grip strength game detects the pressure of each finger of the user to be evaluated through the pressure sensor on the rehabilitation ball, and determines the pressure information corresponding to the grip strength game according to the pressure of each finger of the user to be evaluated; and obtains the control turning time and the target grip reaction time when the user to be evaluated uses the rehabilitation ball to complete the grip strength game, and determines the game completion information corresponding to the grip strength game according to the control turning time and the target grip reaction time, thereby obtaining the pressure information and game completion information corresponding to the grip strength game.

[0062] S120, determining a drawing completion quality assessment of the user to be assessed based on the initial drawing image and the current drawing image, and determining a user muscle strength level of the user to be assessed based on the pressure information and the game completion information, wherein the drawing completion quality assessment includes mean square error, peak signal-to-noise ratio, and covariance.

[0063] Once the effective coloring is completed in the painting test game, the user to be evaluated will enter the settlement interface and obtain the current painting image. The initial painting image and the current painting image reflect the degree to which the user to be evaluated has colored the border when painting. The degree to which the user to be evaluated has colored the border when painting is directly related to the hand-eye coordination ability and fine motor ability. It is not linear, but can be analyzed through statistical methods, such as mean square error (MSE).

[0064] The mean square error (MSE) is an indicator that measures the degree of difference between the model prediction value and the true value. In this embodiment, the mean square error of the user to be evaluated is determined based on the initial painting image and the current painting image, specifically:

[0065]

[0066] Where MSE is the mean square error of the user to be evaluated; m and n are the height and width of the initial painting image and the current painting image, respectively; I[i, j] is the pixel value of the initial painting image at position (i, j); I′[i, j] is the pixel value of the current painting image at the same position (i, j).

[0067] By double summing, each pixel in the initial painting image and the current painting image is traversed to calculate (I[i, j]-I′[i, j]) 2 That is, the square of the difference at each pixel position. Finally, the entire summation result is divided by the total number of pixels mn to obtain the mean squared error.

[0068] It can be known that the smaller the mean square error is, the smaller the difference between the initial painting image and the current painting image is; the larger the mean square error is, the larger the difference between the initial painting image and the current painting image is.

[0069] In another embodiment, a clear border is set in each painting, which is the edge pixel of the image. The internal difference between the initial painting image and the current painting image is generally small, and the main content (i.e., the non-border area) is relatively consistent, so calculating the MSE of the border can evaluate the similarity of the two images more quickly and accurately. Specifically, the pixel difference between the initial painting image and the current painting image is calculated through the border area, and the initial painting image and the current painting image are each evenly divided into four corresponding small areas during the evaluation process, thereby obtaining four sub-images of equal size. For each small area, it is compared with the corresponding small area in the reference image, and the MSE is calculated.

[0070]

[0071] Among them, I[i, j] is the pixel value of the small area at position (i, j) in the current painting image; K(i, j) is the pixel value of the small area at the same position in the initial painting image; M and N are the width and height of the small area respectively.

[0072] For each small area, the mean square error (MSE) of the four small areas and the total mean square error are calculated.

[0073] PSNR is an image quality evaluation indicator based on MSE. It represents the ratio of the maximum possible power of the image signal to the destructive noise power that affects its representation accuracy. The unit of PSNR is usually decibel (dB). The higher the value, the better the image quality. The peak signal-to-noise ratio of the user to be evaluated is determined based on the initial painting image and the current painting image, specifically:

[0074]

[0075] Wherein, PSNR is the peak signal-to-noise ratio of the user to be evaluated; Peak is the maximum value representing the pixel intensity of the image. If each sampling point is represented by 8 bits, then Peak = 255.

[0076] SSIM is an indicator that measures the visual similarity between the initial painting image and the current painting image. It comprehensively evaluates the image quality from three key dimensions: brightness similarity, contrast similarity, and structural similarity. Among them, brightness similarity is estimated by calculating the mean, contrast similarity is measured by standard deviation, and the measurement of structural similarity depends on the calculation of covariance.

[0077] The covariance of the user to be evaluated is determined based on the initial painting image and the current painting image, specifically:

[0078]

[0079] Where SSIM(x, y) is the covariance of the user to be evaluated; x and y represent the pixel values ​​of the initial painting image and the current painting image respectively; μ x and μ y is the mean of the initial painting image and the current painting image; and is the variance between the initial painting image and the current painting image; σ xy is the covariance of the initial drawing image and the current drawing image; C1 and C2 are constants used to avoid the denominator being zero.

[0080] Understandably, because PSNR focuses on pixel-level errors, while SSIM focuses more on the structure and visual perception of the image, using the two together can provide a more comprehensive image quality assessment.

[0081] It should also be noted that the WHEELTEC N100 inertial measurement unit in the rehabilitation ball collects the maximum activity angles of the user to be evaluated in palmar flexion, dorsiflexion, radial deviation and ulnar deviation, and the maximum activity angles of palmar flexion, dorsiflexion, radial deviation and ulnar deviation are saved in the upper limb motor ability rehabilitation assessment information of the user to be evaluated, thereby obtaining more accurate information on the degree of wrist joint damage. The collection and analysis of these data are of great significance for clinicians to formulate personalized rehabilitation plans.

[0082] Before the grip strength game begins, voice guidance can be played to remind the user to be evaluated to hold the ball with all his strength and conduct a maximum grip strength test. This is done to evaluate and quantify the patient's muscle control ability and maximize the exercise of the patient's hand muscles.

[0083] The maximum grip strength test is as follows: Each of the ten finger grooves of the rehabilitation ball is designed with a pressure sensor. The pressure sensor responds very quickly to pressure changes. Therefore, when the user to be evaluated holds the ball with all his strength, each pressure sensor can respond to the corresponding pressure F in real time. After the user to be evaluated completes the maximum grip strength test, the average value of the ten pressure sensors is calculated as the maximum grip strength of the user to be evaluated. This maximum grip strength will be used as a target for each subsequent round of the game.

[0084] Continue to see Figure 6 and Figure 7As shown in the figure, after completing the test of the maximum grip strength, the grip strength game is officially started. The user to be evaluated needs to turn the wrist to make the game character face the direction of the mushroom, which is the key to achieving the goal. The user to be evaluated needs to constantly adjust the strength of the grip until the required grip strength is reached. When the hand is released, the game character will move in the direction it is facing. If the user to be evaluated does not use enough force, it will not be able to touch the mushroom and will not get points. On the contrary, if the user to be evaluated uses too much force, it may cause the game character to pass over the mushroom and also not get points. The game will gradually increase the rotation angle over time, and accordingly, the grip strength required by the user to be evaluated will also increase.

[0085] Furthermore, after the grip game is over, a comprehensive analysis is conducted based on the maximum rotation angle and the maximum grip force of the user to be evaluated in each round of the game, where the most important evaluation parameter for the comprehensive analysis is time. The completion time T of each round of the game is recorded. t This time will be used as an important basis in the subsequent analysis report. In addition, the completion time T of each round of the game is divided into: the time T of the user to be evaluated controlling the game character's turn by wrist d and the reaction time T of the user to be evaluated to hold the ball tightly and achieve the target grip force s . T d It can represent the hand-eye coordination and quick response ability of the user to be evaluated to game control. As the number of games increases, the user to be evaluated will gradually shorten T through game training. d , which indicates that the wrist rotation ability of the user being evaluated is continuously improving. s It reflects the patient's reaction speed and grip control ability, which can also be improved through game training. d +T s ,T t =nT=T1+T2+…+T n .

[0086] Therefore, in the grip strength game, time parameters are crucial for evaluating the patient's wrist and muscle capabilities, optimizing game design, and improving user experience. Through in-depth analysis of these time parameters and the wrist rotation transmitted by the IMU, a comprehensive analysis is conducted to give a level of muscle strength of the user to be evaluated.

[0087] S130, determining upper limb motor ability rehabilitation assessment information of the user to be assessed according to the mean square error, peak signal-to-noise ratio, covariance and user muscle strength level, the upper limb motor ability rehabilitation assessment information including muscle strength level and wrist joint damage.

[0088] The mean square error (MSE) reflects the degree to which the user to be evaluated draws out the border when painting. The degree to which the user to be evaluated draws out the border when painting is directly related to the hand-eye coordination ability and fine motor ability. After calculating the mean square error (MSE), the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) are further used to evaluate the similarity and quality difference between the initial painting image and the current painting image. Through PSNR and SSIM analysis, the user's wrist control ability is reflected from the side, that is, the wrist joint damage of the user to be evaluated is obtained.

[0089] The user's muscle strength level not only reflects the current muscle strength level of the user to be evaluated, but also reveals the degree of progress of the user to be evaluated in controlling muscle strength, that is, the muscle strength level of the user to be evaluated is obtained.

[0090] The technical solution of the embodiment of the present invention is to obtain the initial drawing image and the current drawing image corresponding to the drawing test game, as well as the pressure information and the game completion information corresponding to the grip game after the user to be evaluated completes the drawing test game and the grip game using the rehabilitation ball; determine the drawing completion quality evaluation of the user to be evaluated according to the initial drawing image and the current drawing image, and determine the user muscle strength level of the user to be evaluated according to the pressure information and the game completion information, and the drawing completion quality evaluation includes the mean square error, the peak signal-to-noise ratio and the covariance; determine the upper limb motor ability rehabilitation evaluation information of the user to be evaluated according to the mean square error, the peak signal-to-noise ratio, the covariance and the user muscle strength level, and the upper limb motor ability rehabilitation evaluation information includes the muscle strength level and the wrist joint damage. The present invention solves the problem that the current evaluation of upper limb motor ability rehabilitation cannot be accurate, and the evaluation means and standards cannot be determined at the same time, and realizes a comprehensive and accurate evaluation of upper limb motor ability rehabilitation, increases the fun of interaction, and also helps to improve the user's participation and interest, and can provide a more effective and personalized rehabilitation experience.

[0091] Figure 8 The schematic diagram of the structure of an upper limb motor ability rehabilitation evaluation device provided by an embodiment of the present invention. The upper limb motor ability rehabilitation evaluation device is applied to an upper limb motor ability rehabilitation evaluation system, and the upper limb motor ability rehabilitation evaluation system includes a rehabilitation ball, such as Figure 8 As shown, the upper limb motor ability rehabilitation assessment device comprises:

[0092] The information acquisition module 210 is used to acquire the initial drawing image and the current drawing image corresponding to the drawing test game, and the pressure information and the game completion information corresponding to the grip strength game after the user to be evaluated completes the drawing test game and the grip strength game using the rehabilitation ball;

[0093] An evaluation information determination module 220 is used to determine the quality of the painting completion of the user to be evaluated according to the initial painting image and the current painting image, and to determine the user muscle strength level of the user to be evaluated according to the pressure information and the game completion information, wherein the painting completion quality evaluation includes mean square error, peak signal-to-noise ratio and covariance;

[0094] The rehabilitation assessment determination module 230 is used to determine the upper limb motor ability rehabilitation assessment information of the user to be assessed based on the mean square error, peak signal-to-noise ratio, covariance and user muscle strength level. The upper limb motor ability rehabilitation assessment information includes muscle strength level and wrist joint damage.

[0095] Optionally, determining the quality assessment of the painting completion of the user to be assessed according to the initial painting image and the current painting image is specifically used for:

[0096] The mean square error of the user to be evaluated is determined based on the initial painting image and the current painting image, specifically:

[0097]

[0098] Where MSE is the mean square error of the user to be evaluated; m and n are the height and width of the initial painting image and the current painting image, respectively; I[i, j] is the pixel value of the initial painting image at position (i, j); I′[i, j] is the pixel value of the current painting image at the same position (i, j);

[0099] or,

[0100] Among them, I[i, j] is the pixel value of the small area at position (i, j) in the current painting image; K(i, j) is the pixel value of the small area at the same position in the initial painting image; M and N are the width and height of the small area respectively.

[0101] Optionally, the smaller the mean square error is, the smaller the difference between the initial painting image and the current painting image is determined to be;

[0102] The larger the mean square error is, the greater the difference between the initial painting image and the current painting image is determined to be.

[0103] Optionally, determining the quality assessment of the painting completion of the user to be assessed according to the initial painting image and the current painting image is specifically used for:

[0104] The peak signal-to-noise ratio of the user to be evaluated is determined according to the initial painting image and the current painting image, specifically:

[0105]

[0106] Wherein, PSNR is the peak signal-to-noise ratio of the user to be evaluated; Peak is the maximum value of the image pixel intensity;

[0107] The covariance of the user to be evaluated is determined based on the initial painting image and the current painting image, specifically:

[0108]

[0109] Where SSIM(x, y) is the covariance of the user to be evaluated; x and y represent the pixel values ​​of the initial painting image and the current painting image respectively; μ x and μ y is the mean of the initial painting image and the current painting image; and is the variance between the initial painting image and the current painting image; σ xy is the covariance of the initial painting image and the current painting image; C1 and C2 are constants.

[0110] Optionally, obtain pressure information and game completion information corresponding to the grip game, specifically for:

[0111] The pressure of each finger of the user to be evaluated is detected by the pressure sensor on the rehabilitation ball, and the pressure information corresponding to the grip game is determined according to the pressure of each finger of the user to be evaluated;

[0112] The control turning time and the target grip reaction time of the user to be evaluated when completing the grip game using the rehabilitation ball are obtained, and the game completion information corresponding to the grip game is determined according to the control turning time and the target grip reaction time.

[0113] Optionally, the assessment device for upper limb motor rehabilitation may also include:

[0114] The activity angle collection module is used to collect the maximum activity angles of the user to be evaluated in palmar flexion, dorsiflexion, radial deviation and ulnar deviation, and save the maximum activity angles of palmar flexion, dorsiflexion, radial deviation and ulnar deviation in the upper limb motor ability rehabilitation assessment information of the user to be evaluated.

[0115] The upper limb motor ability rehabilitation assessment device provided in the embodiment of the present invention can execute the upper limb motor ability rehabilitation assessment method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the upper limb motor ability rehabilitation assessment method.

[0116] Fig. 9The structural schematic diagram of the evaluation system 310 for upper limb motor ability rehabilitation that can be used to implement an embodiment of the present invention is shown. The evaluation system for upper limb motor ability rehabilitation includes a rehabilitation ball and a computer terminal, and the computer terminal is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The computer terminal can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0117] like Fig. 9 As shown, the computer terminal 310 includes at least one processor 311, and a memory connected to the at least one processor 311, such as a read-only memory (ROM 312), a random access memory (RAM 313), etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 311 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM 312) or the computer program loaded from the storage unit 318 to the random access memory (RAM 313). In RAM 313, various programs and data required for the operation of the computer terminal 310 can also be stored. The processor 311, ROM 312 and RAM 313 are connected to each other via a bus 314. An I / O (input / output) interface 315 is also connected to the bus 314.

[0118] A number of components in the computer terminal 310 are connected to the I / O interface 315, including: an input unit 316, such as a keyboard, a mouse, etc.; an output unit 317, such as various types of displays, speakers, etc.; a storage unit 318, such as a disk, an optical disk, etc.; and a communication unit 319, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 319 allows the computer terminal 310 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0119] The processor 311 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 311 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 311 executes the various methods and processes described above, such as the assessment method for upper limb motor ability rehabilitation.

[0120] In some embodiments, the assessment method for upper limb motor ability rehabilitation may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 318. In some embodiments, part or all of the computer program may be loaded and / or installed on the computer terminal 310 via the ROM 312 and / or the communication unit 319. When the computer program is loaded into the RAM 313 and executed by the processor 311, one or more steps of the assessment method for upper limb motor ability rehabilitation described above may be performed. Alternatively, in other embodiments, the processor 311 may be configured to perform the assessment method for upper limb motor ability rehabilitation in any other appropriate manner (e.g., by means of firmware).

[0121] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0122] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0123] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0124] To provide interaction with a user, the systems and techniques described herein may be implemented on a computer terminal having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer terminal. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0125] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0126] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0127] The rehabilitation ball included in the upper limb motor ability rehabilitation assessment system can be a 3D printed spherical interactive device. Fig.10 A three-dimensional schematic diagram of a rehabilitation ball provided by an embodiment of the present invention is shown in FIG. Fig.10 As shown, the rehabilitation ball includes an upper hemisphere, a lower hemisphere arranged opposite to the upper hemisphere, and a pressure sensor embedded in a card slot inside the upper hemisphere. The pressure sensor can adopt an SF15 thin film flexible pressure sensor. Grooves specially designed for children are reserved on the upper and lower surfaces of the rehabilitation ball. These grooves can meet the special needs of children when grasping or rotating the device, and increase the practicality of the device. The internal structure of the rehabilitation ball is precise, and it contains multiple key components to ensure the accuracy and efficiency of data processing.

[0128] The rehabilitation ball contains an ESP32 microcontroller, a Wheeltec N100 IMU inertial measurement unit (i.e., IMU gyroscope), and a TP4056 charging module. These components work together to ensure that the device can achieve precise motion control and data recording. Fig.11 As shown, the rehabilitation ball is equipped with an ESP32 microcontroller, which is fixed by the ESP32 upper bracket and the ESP32 lower bracket. The ESP32 microcontroller is used to integrate the three-axis acceleration data from the IMU gyroscope and the data from the flexible thin film pressure sensor. The IMU gyroscope is connected to the main controller through the Usart interface, and combined with the high-precision detection capability of the thin film flexible pressure sensor, it can accurately capture the fluctuations of various values ​​when the game is running, which allows children to feel more realistic physical feedback when playing the game.

[0129] The ESP32 microcontroller is the core of the entire rehabilitation ball. It is a high-performance, low-power microcontroller that relies on a built-in battery for normal operation. The ESP32 microcontroller receives data from the IMU gyroscope and the pressure-flexible film force sensor, and also uses Bluetooth technology to quickly transmit information. Once the data is collected, the ESP32 microcontroller quickly transmits it to a remote computer or other smart device through the Bluetooth module, which improves the flexibility and practicality of the system.

[0130] Thin film flexible pressure sensors are particularly suitable for integration into rehabilitation balls. They can provide accurate and reliable data both when accurately measuring tiny pressure changes and when simulating pressure conditions in real environments.

[0131] In order to improve the convenience of children's operation, the rehabilitation ball is equipped with a Bluetooth module, which can easily transmit data to the game program developed by Unity. In this way, children can freely choose to control the ball with one hand or two hands. Whether it is single-handed palm control or two-handed collaboration, the game experience can be kept consistent and stable.

[0132] The outer shell of the rehabilitation ball is made with 3D printing technology. After printing, when children place their fingers in the grooves on the surface of the device, they can easily hold the device and accurately fix it in the right position. These finger grooves are not only for support, but also embedded with high-precision pressure sensors that can sensitively sense changes in children's strength, thereby effectively measuring their strength level. At the same time, in order to further enhance the convenience of the device for users, a C-type charging interface and a switching switch groove are reserved on the surface of the rehabilitation ball. Considering various situations that may be encountered in daily use, it provides convenience for charging the device.

[0133] There are some card slots inside the upper and lower layers of the rehabilitation ball, which are inserted into and fixed in a special test board to minimize the mechanical damage to the chip and sensor components in the rehabilitation ball. Whether it is daily use or working environment under extreme conditions, the stability and reliability of the rehabilitation ball can be ensured, and it is avoided as much as possible from being affected by external factors, so that the rehabilitation ball can stably and reliably play its due performance.

[0134] The upper shell of the rehabilitation ball is equipped with five finger grooves to reserve fixed positions for internal components. The thin film flexible pressure sensor uses resistive sensing technology to measure the relationship between a specific pressure value and a change in resistance. Similarly, the lower shell of the rehabilitation ball is also equipped with five finger grooves, see Fig.11 As shown in the figure, the internal card slot of the lower layer of the rehabilitation ball contains the charging module, battery and ESP32 lower bracket. The battery of the rehabilitation ball is a high-efficiency lithium battery with a capacity of 2500mAh and a voltage standard of 3.7V to meet the power requirements of the entire device. The battery is charged through the charging module, which is not only responsible for charging the battery, but also for storing the charged electricity, ensuring the durability and stability of the rehabilitation ball.

[0135] For example, see Fig.12As shown in the figure, during the game, the computer terminal will continuously monitor the visual and auditory feedback generated by the eyes and ears of the user to be evaluated, and the data will then be analyzed by the computer to better understand the child's physical condition. By carefully processing the data generated during the game and combining it with the results of gesture tracking, the computer terminal can generate a detailed evaluation report. This report is an important basis for personalized rehabilitation programs, and it records the child's activity performance and health status in detail. After receiving the report, doctors and parents can quickly understand the child's performance in the game, so as to adjust or formulate a new rehabilitation plan in time. This interactive evaluation method not only enhances the children's enthusiasm in the rehabilitation process, but also provides doctors with valuable reference information, which helps to further optimize the rehabilitation plan.

[0136] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0137] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for evaluating upper limb motor rehabilitation, characterized in that: The upper limb motor ability rehabilitation evaluation method is applied to an upper limb motor ability rehabilitation evaluation system, the upper limb motor ability rehabilitation evaluation system includes a rehabilitation ball, and the upper limb motor ability rehabilitation evaluation method includes: After the user to be evaluated completes the drawing test game and the grip strength game using the rehabilitation ball, obtaining an initial drawing image and a current drawing image corresponding to the drawing test game, and pressure information and game completion information corresponding to the grip strength game; Determine a painting completion quality assessment of the user to be assessed according to the initial painting image and the current painting image, and determine a user muscle strength level of the user to be assessed according to the pressure information and the game completion information, wherein the painting completion quality assessment includes a mean square error, a peak signal-to-noise ratio, and a covariance; The upper limb motor ability rehabilitation assessment information of the user to be assessed is determined according to the mean square error, the peak signal-to-noise ratio, the covariance and the user's muscle strength level, wherein the upper limb motor ability rehabilitation assessment information includes muscle strength level and wrist joint damage.

2. The method for evaluating upper limb motor ability rehabilitation according to claim 1, characterized in that: Determining the quality assessment of the painting completion of the to-be-assessed user according to the initial painting image and the current painting image includes: Determining the mean square error of the user to be evaluated according to the initial painting image and the current painting image is specifically as follows: Wherein, MSE is the mean square error of the user to be evaluated; m and n are the height and width of the initial painting image and the current painting image, respectively; I[i, j] is the pixel value of the initial painting image at position (i, j); I ′ [i, j] is the pixel value of the current drawing image at the same position (i, j); or, Among them, I[i,j] is the pixel value of the small area at position (i,j) in the current painting image; K(i,j) is the pixel value of the small area at the same position in the initial painting image; M and N are the width and height of the small area respectively.

3. The method for evaluating upper limb motor ability rehabilitation according to claim 2, characterized in that: The smaller the mean square error is, the smaller the difference between the initial painting image and the current painting image is determined to be; The larger the mean square error is, the greater the difference between the initial painting image and the current painting image is determined to be.

4. The method for evaluating upper limb motor ability rehabilitation according to claim 1, characterized in that: Determining the quality assessment of the painting completion of the to-be-assessed user according to the initial painting image and the current painting image includes: Determining the peak signal-to-noise ratio of the user to be evaluated according to the initial painting image and the current painting image is specifically as follows: Wherein, PSNR is the peak signal-to-noise ratio of the user to be evaluated; Peak is the maximum value of the image pixel intensity; Determining the covariance of the user to be evaluated according to the initial painting image and the current painting image is specifically as follows: Wherein, SSIM(x, y) is the covariance of the user to be evaluated; x and y represent the pixel values ​​of the initial painting image and the current painting image respectively; μ x and μ y is the mean of the initial painting image and the current painting image; and is the variance of the initial painting image and the current painting image; σ xy is the covariance between the initial painting image and the current painting image; C1 and C2 are constants.

5. The method for evaluating upper limb motor ability rehabilitation according to claim 1, characterized in that: Obtaining pressure information and game completion information corresponding to the grip game, including: Detecting the pressure of each finger of the user to be evaluated through the pressure sensor on the rehabilitation ball, and determining the pressure information corresponding to the grip game according to the pressure of each finger of the user to be evaluated; The control turning time and the target grip reaction time of the user to be evaluated when completing the grip game using the rehabilitation ball are obtained, and the game completion information corresponding to the grip game is determined according to the control turning time and the target grip reaction time.

6. The method for evaluating upper limb motor rehabilitation according to claim 1, characterized in that: The assessment method for upper limb motor ability rehabilitation also includes: The maximum activity angles of the user to be evaluated in palmar flexion, dorsiflexion, radial deviation and ulnar deviation are collected, and the maximum activity angles of the palmar flexion, dorsiflexion, radial deviation and ulnar deviation are stored in the upper limb motor ability rehabilitation assessment information of the user to be evaluated.

7. An upper limb motor ability rehabilitation assessment device, characterized in that: The upper limb motor ability rehabilitation evaluation device is applied to an upper limb motor ability rehabilitation evaluation system, the upper limb motor ability rehabilitation evaluation system includes a rehabilitation ball, and the upper limb motor ability rehabilitation evaluation device includes: an information acquisition module, configured to acquire an initial drawing image and a current drawing image corresponding to the drawing test game, and pressure information and game completion information corresponding to the grip strength game after the user to be evaluated completes the drawing test game and the grip strength game using the rehabilitation ball; An evaluation information determination module, configured to determine a painting completion quality evaluation of the user to be evaluated according to the initial painting image and the current painting image, and to determine a user muscle strength level of the user to be evaluated according to the pressure information and the game completion information, wherein the painting completion quality evaluation includes a mean square error, a peak signal-to-noise ratio, and a covariance; The rehabilitation assessment determination module is used to determine the upper limb motor ability rehabilitation assessment information of the user to be assessed based on the mean square error, the peak signal-to-noise ratio, the covariance and the user's muscle strength level, wherein the upper limb motor ability rehabilitation assessment information includes muscle strength level and wrist joint damage.

8. An upper limb motor ability rehabilitation assessment system, characterized in that: The upper limb motor ability rehabilitation assessment system comprises a rehabilitation ball and a computer terminal; The computer terminal comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the upper limb motor ability rehabilitation assessment method described in any one of claims 1-6.

9. The upper limb motor ability rehabilitation evaluation system according to claim 8, characterized in that: The rehabilitation ball comprises an upper hemisphere, a lower hemisphere arranged opposite to the upper hemisphere, and a pressure sensor embedded in a card slot inside the upper hemisphere.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the upper limb motor ability rehabilitation assessment method according to any one of claims 1 to 6 when executed.

Images