Lower limb hip and knee rehabilitation equipment operation optimization method based on machine vision

Abstract

The invention discloses a lower limb hip and knee rehabilitation equipment operation optimization method based on machine vision. The method comprises the following steps: firstly, acquiring multi-angle video data when a rehabilitation person uses equipment, extracting a key frame, positioning a target object, analyzing movement characteristics and dividing a movement period; motion features are detected, segmented and extracted through an HOG algorithm, multi-scale fusion is carried out through a Transform model, and fused motion feature data are generated; on the basis of the rehabilitation plan, standard motion features in an ideal state are simulated through a GAN generation model, and multi-scale fusion is carried out to generate standard fusion feature data; calculating the similarity between the fused motion feature data and the standard fused feature data, evaluating the operation state of rehabilitation equipment, and generating a rehabilitation motion index; and optimizing equipment module parameters based on the rehabilitation exercise index. According to the invention, the continuity, the standard and the corresponding rehabilitation state of the operation of the rehabilitation equipment can be accurately analyzed, the rehabilitation effect can be effectively evaluated, and the dynamic optimization and the real-time adjustment of the equipment parameters can be realized.

Description

Technical Field

[0001] The present invention relates to the field of rehabilitation equipment operation analysis, and more specifically, to a lower limb hip and knee rehabilitation equipment operation optimization method based on machine vision. Background Art

[0002] For the elderly, the treatment of lower limb hip and knee diseases is particularly important, especially for lower limb sports rehabilitation, which is the key to whether people can move freely. At the same time, trauma, disability and other problems can also lead to motor dysfunction, which further increases people's demand for rehabilitation training. Traditional lower limb hip and knee rehabilitation methods generally use manual techniques to achieve patient rehabilitation exercises. This method is inefficient and has poor effects. For general rehabilitation equipment, the existing technology lacks effective monitoring and effectiveness evaluation of rehabilitation exercise status, and lacks refined equipment operation effect analysis for the rehabilitation exercise process, which makes it difficult for the equipment to adapt to different rehabilitation personnel and difficult to meet the target rehabilitation plan. Summary of the invention

[0003] The present invention overcomes the defects of the prior art and proposes a method for optimizing the operation of lower limb hip and knee rehabilitation equipment based on machine vision.

[0004] The first aspect of the present invention provides a method for optimizing the operation of a lower limb hip and knee rehabilitation device based on machine vision, comprising: Through machine vision technology, multi-angle video data of rehabilitation personnel using lower limb hip and knee rehabilitation equipment is obtained; Based on each angle, key frames are extracted from the video data to form a rehabilitation process image dataset. Each angle corresponds to an image dataset. The target object of the rehabilitation equipment is located in the image dataset, and the movement characteristics of the target object are analyzed. Combined with the rehabilitation plan, the use process time period is divided into multiple movement cycles; In one motion cycle, the image data sets at multiple angles are subjected to motion target detection, target segmentation and feature extraction based on the HOG algorithm to form motion feature data for each image data set. Taking one angle as a scale, the motion feature data is subjected to multi-scale fusion through the hierarchical multi-scale Transformer model to generate fused motion feature data. Based on the rehabilitation plan and exercise cycle, the multi-angle motion parameters are set when the exercise target is in an ideal state. The GAN generation model is used to generate simulation features of the exercise target in combination with the motion parameters to obtain multi-angle standard motion features. The hierarchical multi-scale Transformer model is used to perform multi-scale fusion of the standard motion features to generate standard fusion feature data. The fused motion feature data and the standard fused feature data are taken as two samples, and the similarity between the samples is calculated based on the Mahalanobis distance, and the running status of the rehabilitation equipment is evaluated based on the similarity to generate a rehabilitation motion index; The corresponding rehabilitation movement index is calculated for each exercise cycle, and the rehabilitation movement index is compared with the preset range to optimize the settings of various modules of the corresponding rehabilitation equipment.

[0005] In this solution, the multi-angle video data of the rehabilitation personnel when using the lower limb hip and knee rehabilitation equipment is obtained through machine vision technology, specifically: Through machine vision technology, set up multi-angle video acquisition devices and apply them to the preset range of motion of rehabilitation personnel; Based on the multi-angle video acquisition device, corresponding multi-angle video data is obtained during the use period of a lower limb hip and knee rehabilitation device.

[0006] In this solution, based on each angle, key frames are extracted from the video data to form a rehabilitation process image data set. Each angle corresponds to an image data set. The target object of the rehabilitation equipment is located in the image data set, and the movement characteristics of the target object are analyzed. Combined with the rehabilitation plan, the use process time period is divided into multiple motion cycles, specifically: Based on each angle, key frames are extracted from the video data to form a rehabilitation process image dataset; Perform image denoising, enhancement and standardization preprocessing on image datasets; Based on the rehabilitation equipment, target objects are set for multiple equipment joints, and key image features of the target objects are stored; An image data set at an angle is selected, and a recognition module is used to perform multi-frame image recognition on the image data set. The recognition process is to detect and locate the target object by combining key image features, and to analyze the movement features of the target object in the multi-frame images to generate movement data of the target object. Analyze the mobile data, analyze the continuity of the rehabilitation personnel's movements in combination with the rehabilitation plan, and divide the usage process time period into multiple movement cycles.

[0007] In this solution, within a motion cycle, the image data sets at multiple angles are subjected to motion target detection, target segmentation and feature extraction based on the HOG algorithm to form motion feature data for each image data set. With one angle as a scale, the motion feature data is subjected to multi-scale fusion through a hierarchical multi-scale Transformer model to generate fused motion feature data, specifically: Grayscale the image data set, calculate the gradient size and direction of local pixels in each frame based on the HOG algorithm, extract multiple object contour features, perform target detection and segmentation from multiple object contour features, and construct the target feature vector for each frame; Perform feature change analysis on the target feature vectors of two adjacent frames of images to generate feature difference information, and integrate the target feature vector of each frame of image with the feature difference information to form motion feature data; For each image data set, generate corresponding motion feature data; Through the hierarchical multi-scale Transformer model, multiple Transformer sub-modules are constructed based on each angle as a scale, and multiple motion feature data are respectively input into the Transformer sub-module for feature conversion to form multi-scale feature data; Through the attention mechanism, multi-scale feature data are fused and represented to generate fused motion feature data.

[0008] In this solution, each motion cycle includes corresponding fused motion feature data.

[0009] In this scheme, based on the rehabilitation plan and exercise cycle, the multi-angle motion parameters are set when the motion target is in an ideal state, and the GAN generation model is used to generate simulation features of the motion target in combination with the motion parameters to obtain multi-angle standard motion features. The standard motion features are multi-scale fused through the hierarchical multi-scale Transformer model to generate standard fusion feature data, specifically: Based on the rehabilitation plan and exercise cycle division, set multi-angle exercise parameters under ideal exercise goals; According to the motion parameters, multiple angle image frames of the moving target in the initial state and the moving state are obtained from the system database, and the image frames at each angle are marked as the initial image set; Extract features from the initial image set based on the HOG algorithm to form initial motion feature data; Construct a generative model based on GAN, import the initial motion feature data and motion parameters into the generative model for training and learning, and cyclically generate simulated motion feature data. During each training process, analyze the motion state of the object based on the simulated motion feature data and the initial motion feature data, and use the consistency between the object's motion state and the motion parameters as the basis for judging the accuracy of the generated data. Repeated training is performed until the generative model reaches the preset number of training times. Generate standard motion features through the trained generative model; Generate multi-angle standard motion features based on the multi-angle initial image set; The standard motion features are fused at multiple scales through a hierarchical multi-scale Transformer model to generate standard fused feature data.

[0010] In this solution, the fusion motion feature data and the standard fusion feature data are used as two samples, and the similarity between the samples is calculated based on the Mahalanobis distance, and the operation status of the rehabilitation equipment is evaluated based on the similarity to generate a rehabilitation motion index, which is specifically: The fused motion feature data and the standard fused feature data are taken as two samples, and the distance value between the samples is calculated based on the Mahalanobis distance; The similarity between samples is analyzed by distance value, and the operation status of rehabilitation equipment is evaluated by similarity to generate rehabilitation movement index; The rehabilitation movement index is proportional to the similarity.

[0011] In this solution, the corresponding rehabilitation exercise index is calculated for each exercise cycle, and the rehabilitation exercise index is compared with the preset range to optimize the settings of each module of the corresponding rehabilitation equipment, specifically: Calculate and analyze each exercise cycle to obtain the corresponding rehabilitation exercise index; Determine whether the rehabilitation exercise index of each exercise cycle meets the preset range. If not, mark the corresponding exercise cycle as an optimized exercise cycle; Mark the equipment modules operated by the rehabilitation equipment during the optimized exercise cycle, optimize the parameters of the equipment modules, and generate an optimization plan.

[0012] The second aspect of the present invention further provides a lower limb hip and knee rehabilitation equipment operation optimization system based on machine vision, the system comprising: a memory, a processor, the memory comprising a lower limb hip and knee rehabilitation equipment operation optimization program based on machine vision, the lower limb hip and knee rehabilitation equipment operation optimization program based on machine vision when executed by the processor implements the following steps: Through machine vision technology, multi-angle video data of rehabilitation personnel using lower limb hip and knee rehabilitation equipment is obtained; Based on each angle, key frames are extracted from the video data to form a rehabilitation process image dataset. Each angle corresponds to an image dataset. The target object of the rehabilitation equipment is located in the image dataset, and the movement characteristics of the target object are analyzed. Combined with the rehabilitation plan, the use process time period is divided into multiple movement cycles; In one motion cycle, the image data sets at multiple angles are subjected to motion target detection, target segmentation and feature extraction based on the HOG algorithm to form motion feature data for each image data set. Taking one angle as a scale, the motion feature data is subjected to multi-scale fusion through the hierarchical multi-scale Transformer model to generate fused motion feature data. Based on the rehabilitation plan and exercise cycle, the multi-angle motion parameters are set when the exercise target is in an ideal state. The GAN generation model is used to generate simulation features of the exercise target in combination with the motion parameters to obtain multi-angle standard motion features. The hierarchical multi-scale Transformer model is used to perform multi-scale fusion of the standard motion features to generate standard fusion feature data. The fused motion feature data and the standard fused feature data are taken as two samples, and the similarity between the samples is calculated based on the Mahalanobis distance, and the running status of the rehabilitation equipment is evaluated based on the similarity to generate a rehabilitation motion index; The corresponding rehabilitation movement index is calculated for each exercise cycle, and the rehabilitation movement index is compared with the preset range to optimize the settings of various modules of the corresponding rehabilitation equipment.

[0013] The third aspect of the present invention also provides a computer-readable storage medium, which includes a lower limb hip and knee rehabilitation equipment operation optimization program based on machine vision. When the lower limb hip and knee rehabilitation equipment operation optimization program based on machine vision is executed by a processor, the steps of the lower limb hip and knee rehabilitation equipment operation optimization method based on machine vision as described in any one of the above items are implemented.

[0014] The present invention discloses a method for optimizing the operation of lower limb hip and knee rehabilitation equipment based on machine vision. First, obtain multi-angle video data when rehabilitation personnel use the equipment, extract key frames and locate target objects, analyze movement features and divide movement cycles; use HOG algorithm to detect, segment and extract movement features, perform multi-scale fusion through Transformer model, and generate fused movement feature data; based on the rehabilitation plan, use GAN to generate a model to simulate standard movement features under ideal conditions, and perform multi-scale fusion to generate standard fused feature data; calculate the similarity between fused movement feature data and standard fused feature data, evaluate the operation status of rehabilitation equipment and generate a rehabilitation movement index; optimize equipment module parameters based on the rehabilitation movement index. Through the present invention, it is possible to accurately analyze the continuity, standardization and corresponding rehabilitation status of the operation of rehabilitation equipment, effectively evaluate the rehabilitation effect, and realize dynamic optimization and real-time adjustment of equipment parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A flow chart of a method for optimizing the operation of lower limb hip and knee rehabilitation equipment based on machine vision according to the present invention is shown; Figure 2 A block diagram of a lower limb hip and knee rehabilitation equipment operation optimization system based on machine vision of the present invention is shown. DETAILED DESCRIPTION

[0016] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0017] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0018] Figure 1 A flow chart of a method for optimizing the operation of lower limb hip and knee rehabilitation equipment based on machine vision according to the present invention is shown.

[0019] like Figure 1 As shown, the first aspect of the present invention provides a method for optimizing the operation of a lower limb hip and knee rehabilitation device based on machine vision, comprising: S102, using machine vision technology to obtain multi-angle video data of rehabilitation personnel when using lower limb hip and knee rehabilitation equipment; S104, extracting key frames from the video data based on each angle to form a rehabilitation process image data set, where each angle corresponds to an image data set, locating the target object of the rehabilitation device in the image data set, analyzing the movement characteristics of the target object, and dividing the use process time period into multiple motion cycles in combination with the rehabilitation plan; S106, performing moving target detection, target segmentation and feature extraction based on the HOG algorithm on the image data sets at multiple angles within a motion cycle, forming motion feature data for each image data set, taking one angle as a scale, performing multi-scale fusion on the motion feature data through a hierarchical multi-scale Transformer model, and generating fused motion feature data; S108, based on the rehabilitation plan and the exercise cycle, set the multi-angle motion parameters of the exercise target in an ideal state, generate the simulation features of the exercise target by using the GAN generation model, combine the motion parameters, obtain the multi-angle standard motion features, perform multi-scale fusion on the standard motion features by using the hierarchical multi-scale Transformer model, and generate standard fusion feature data; S110, taking the fused motion feature data and the standard fused feature data as two samples, calculating the similarity between the samples based on the Mahalanobis distance, and evaluating the operating status of the rehabilitation device based on the similarity to generate a rehabilitation motion index; S112, calculating a corresponding rehabilitation exercise index for each exercise cycle, and optimizing settings for each module of the corresponding rehabilitation equipment by comparing the rehabilitation exercise index with a preset range.

[0020] According to an embodiment of the present invention, the multi-angle video data of the rehabilitation personnel when using the lower limb hip and knee rehabilitation equipment is obtained by using machine vision technology, specifically: Through machine vision technology, set up multi-angle video acquisition devices and apply them to the preset range of motion of rehabilitation personnel; Based on the multi-angle video acquisition device, corresponding multi-angle video data is obtained during the use period of a lower limb hip and knee rehabilitation device.

[0021] It should be noted that the video acquisition device includes multiple devices to acquire video data from multiple angles. The multi-angle video data includes multiple video data corresponding to multiple angles.

[0022] According to an embodiment of the present invention, based on each angle, key frames are extracted from the video data to form a rehabilitation process image data set, each angle corresponds to an image data set, the image data set is used to locate the target object of the rehabilitation device, and the movement characteristics of the target object are analyzed. Combined with the rehabilitation plan, the use process time period is divided into multiple motion cycles, specifically: Based on each angle, key frames are extracted from the video data to form a rehabilitation process image dataset; Perform image denoising, enhancement and standardization preprocessing on image datasets; Based on the rehabilitation equipment, target objects are set for multiple equipment joints, and key image features of the target objects are stored; An image data set at an angle is selected, and a recognition module is used to perform multi-frame image recognition on the image data set. The recognition process is to detect and locate the target object by combining key image features, and to analyze the movement features of the target object in the multi-frame images to generate movement data of the target object. Analyze the mobile data, analyze the continuity of the rehabilitation personnel's movements in combination with the rehabilitation plan, and divide the usage process time period into multiple movement cycles.

[0023] It should be noted that the image data set includes image frames in the entire time period. The recognition module can perform target detection and recognition through the YOLO recognition model. The movement data includes information such as the movement amplitude, movement speed, and movement angle of the target object at the image level. Each motion cycle corresponds to a continuous action time period, and different motion cycles correspond to the operation time periods of different rehabilitation equipment modules. Dividing multiple motion cycles helps to conduct multi-dimensional motion state analysis of the rehabilitation equipment. Rehabilitation equipment refers to lower limb hip and knee rehabilitation equipment, which generally includes multiple joint modules, and different joint modules correspond to independent control parameters, such as power assistance parameters, electrical stimulation parameters, and auxiliary function parameters, which are the same as setting the rehabilitation operation state of the rehabilitation equipment. The target object is multiple equipment joint parts.

[0024] According to an embodiment of the present invention, within one motion cycle, the image data sets at multiple angles are subjected to motion target detection, target segmentation and feature extraction based on the HOG algorithm to form motion feature data for each image data set, and one angle is used as a scale to perform multi-scale fusion of the motion feature data through a hierarchical multi-scale Transformer model to generate fused motion feature data, specifically: Grayscale the image data set, calculate the gradient size and direction of local pixels in each frame based on the HOG algorithm, extract multiple object contour features, perform target detection and segmentation from multiple object contour features, and construct the target feature vector for each frame; Perform feature change analysis on the target feature vectors of two adjacent frames of images to generate feature difference information, and integrate the target feature vector of each frame of image with the feature difference information to form motion feature data; For each image data set, generate corresponding motion feature data; Through the hierarchical multi-scale Transformer model, multiple Transformer sub-modules are constructed based on each angle as a scale, and multiple motion feature data are respectively input into the Transformer sub-module for feature conversion to form multi-scale feature data; Through the attention mechanism, multi-scale feature data are fused and represented to generate fused motion feature data.

[0025] It should be noted that the motion target is generally the overall target of the rehabilitation personnel when wearing or using the rehabilitation equipment, and is used to analyze the motion state of the rehabilitation equipment and the rehabilitation personnel during the combined use of the rehabilitation equipment and the rehabilitation personnel. Feature change analysis can be performed based on vector differences.

[0026] According to an embodiment of the present invention, each motion cycle includes corresponding fused motion feature data.

[0027] It should be noted that the fused motion feature data can effectively describe the characteristic information of rehabilitation movement within a movement cycle, and the fused feature is based on multi-angle fusion analysis, which can more comprehensively extract the motion state and motion conditions of the rehabilitation movement process. Furthermore, it can subsequently compare the rehabilitation conditions and screen out the rehabilitation equipment modules to be optimized, thereby realizing the operation optimization and personalized settings of the equipment to meet the needs of different rehabilitation plans and rehabilitation populations.

[0028] According to an embodiment of the present invention, based on the rehabilitation plan and the exercise cycle, the multi-angle motion parameters are set when the motion target is in an ideal state, the GAN generation model is used to generate simulation features of the motion target in combination with the motion parameters, and the multi-angle standard motion features are obtained. The standard motion features are multi-scale fused through the hierarchical multi-scale Transformer model to generate standard fusion feature data, specifically: Based on the rehabilitation plan and exercise cycle division, set multi-angle exercise parameters under ideal exercise goals; According to the motion parameters, multiple angle image frames of the moving target in the initial state and the moving state are obtained from the system database, and the image frames at each angle are marked as the initial image set; Extract features from the initial image set based on the HOG algorithm to form initial motion feature data; Construct a generative model based on GAN, import the initial motion feature data and motion parameters into the generative model for training and learning, and cyclically generate simulated motion feature data. During each training process, analyze the motion state of the object based on the simulated motion feature data and the initial motion feature data, and use the consistency between the object's motion state and the motion parameters as the basis for judging the accuracy of the generated data. Repeated training is performed until the generative model reaches the preset number of training times. Generate standard motion features through the trained generative model; Generate multi-angle standard motion features based on the multi-angle initial image set; The standard motion features are fused at multiple scales through a hierarchical multi-scale Transformer model to generate standard fused feature data.

[0029] It should be noted that the initial image set includes multiple images, each of which corresponds to an angle. The initial image set is a set of multiple simple frames of the target object in a preset motion state, which is obtained from the system database. Subsequently, the GAN generation model is used to simulate the features of the entire motion process of the initial image set, thereby obtaining multi-angle simulated feature data. The simulated feature data is simulated data of coherent movements, which is used for real-time feature comparison, so as to evaluate the rehabilitation and motion conditions during the operation of the equipment. The multi-scale fusion process of the standard motion features through the hierarchical multi-scale Transformer model is consistent with the analysis and fusion process of the fused motion feature data.

[0030] According to an embodiment of the present invention, the fusion motion feature data and the standard fusion feature data are used as two samples, and the similarity between the samples is calculated based on the Mahalanobis distance, and the running state of the rehabilitation device is evaluated based on the similarity to generate the rehabilitation motion index, specifically: The fused motion feature data and the standard fused feature data are taken as two samples, and the distance value between the samples is calculated based on the Mahalanobis distance; The similarity between samples is analyzed by distance value, and the operation status of rehabilitation equipment is evaluated by similarity to generate rehabilitation movement index; The rehabilitation movement index is proportional to the similarity.

[0031] According to an embodiment of the present invention, the corresponding rehabilitation exercise index is calculated for each exercise cycle, and the rehabilitation exercise index is compared with a preset range to optimize the settings of various modules of the corresponding rehabilitation equipment, specifically: Calculate and analyze each exercise cycle to obtain the corresponding rehabilitation exercise index; Determine whether the rehabilitation exercise index of each exercise cycle meets the preset range. If not, mark the corresponding exercise cycle as an optimized exercise cycle; Mark the equipment modules operated by the rehabilitation equipment during the optimized exercise cycle, optimize the parameters of the equipment modules, and generate an optimization plan.

[0032] It should be noted that the optimization scheme includes angle setting, power setting, auxiliary function setting, electrical stimulation parameter setting, etc. of each module of the rehabilitation equipment. The equipment module is the joint module.

[0033] It is worth mentioning here that when rehabilitation personnel use rehabilitation equipment, the movement state of their lower limbs and the operation state of the equipment are a series of complex processes. Traditional technology makes it difficult to effectively analyze the rehabilitation movement process and conduct multi-dimensional movement evaluation, which makes it difficult to achieve precise equipment parameter optimization and equipment regulation, causing the rehabilitation equipment to remain at the popular parameter settings, and it is difficult to further improve the operating efficiency and use effect of the rehabilitation equipment.

[0034] In the present invention, the fusion of multi-scale feature analysis helps to accurately analyze the continuity, standardization and rehabilitation status of movements in each overall movement cycle or a movement mode, and the fusion analysis of the equipment movement conditions from multiple angles of the rehabilitation movement process is performed, and the corresponding rehabilitation movement index is generated to realize the effectiveness evaluation of the rehabilitation movement; further, the rehabilitation movement index of each movement cycle is compared and analyzed, and the parameters of the corresponding equipment modules are optimized and feedback adjusted.

[0035] According to an embodiment of the present invention, the method of taking the fused motion feature data and the standard fused feature data as two samples and calculating the similarity between the samples based on the Mahalanobis distance further includes: Select two consecutive motion cycles; Taking the fused motion feature data corresponding to the two motion cycles as the first sample data; The standard fusion feature data corresponding to the two motion cycles are used as the second sample data; Using the DBSCAN clustering algorithm, density clustering analysis is performed on the first sample data and the second sample data respectively, and two clustering results are formed; The first clustering result includes N1 feature clustering groups; The second clustering result includes N2 feature clustering groups; If N1 is equal to N2, it is determined that the first sample data is similar to the second sample data; The N1 feature clustering groups and the N2 feature clustering groups are sorted based on the amount of data in the clustering groups, and two feature clustering groups are selected in sequence from the N1 feature clustering groups and the N2 feature clustering groups to calculate the similarity, and the average similarity is calculated based on the number N1; The similarity calculation of the feature clustering groups is specifically to perform feature vectorization on the two feature clustering groups and calculate the similarity of the two groups of data based on the standard Euclidean distance evaluation; The average similarity is used to evaluate the operating status of the rehabilitation equipment and generate a rehabilitation exercise index based on two exercise cycles.

[0036] It should be noted that, for cases where the rehabilitation cycle is long and the motion features are more complex, clustering can be used to analyze the similarity between the fused motion feature data and the standard fused feature data. In the present invention, by integrating and analyzing the fused motion feature data of two consecutive motion cycles, motion features of a sufficient amount of data can be integrated and used as sample data for clustering analysis. The clustering process is actually a process of classifying the motions. If the classification results (number of classifications) are consistent, it can be considered that the two feature data are highly similar, and the similarity is calculated accordingly. The Mahalanobis distance calculation is applicable to the case where there are fewer motion features in each motion cycle, and the calculation efficiency is high.

[0037] Figure 2 A block diagram of a lower limb hip and knee rehabilitation equipment operation optimization system based on machine vision of the present invention is shown.

[0038] The second aspect of the present invention further provides a lower limb hip and knee rehabilitation equipment operation optimization system 2 based on machine vision, the system comprising: a memory 21, a processor 22, the memory 21 comprising a lower limb hip and knee rehabilitation equipment operation optimization program based on machine vision, the lower limb hip and knee rehabilitation equipment operation optimization program based on machine vision when executed by the processor 22 implements the following steps: Through machine vision technology, multi-angle video data of rehabilitation personnel using lower limb hip and knee rehabilitation equipment is obtained; Based on each angle, key frames are extracted from the video data to form a rehabilitation process image dataset. Each angle corresponds to an image dataset. The target object of the rehabilitation equipment is located in the image dataset, and the movement characteristics of the target object are analyzed. Combined with the rehabilitation plan, the use process time period is divided into multiple movement cycles; In one motion cycle, the image data sets at multiple angles are subjected to motion target detection, target segmentation and feature extraction based on the HOG algorithm to form motion feature data for each image data set. Taking one angle as a scale, the motion feature data is subjected to multi-scale fusion through the hierarchical multi-scale Transformer model to generate fused motion feature data. Based on the rehabilitation plan and exercise cycle, the multi-angle motion parameters are set when the exercise target is in an ideal state. The GAN generation model is used to generate simulation features of the exercise target in combination with the motion parameters to obtain multi-angle standard motion features. The hierarchical multi-scale Transformer model is used to perform multi-scale fusion of the standard motion features to generate standard fusion feature data. The fused motion feature data and the standard fused feature data are taken as two samples, and the similarity between the samples is calculated based on the Mahalanobis distance, and the running status of the rehabilitation equipment is evaluated based on the similarity to generate a rehabilitation motion index; The corresponding rehabilitation movement index is calculated for each exercise cycle, and the rehabilitation movement index is compared with the preset range to optimize the settings of various modules of the corresponding rehabilitation equipment.

[0039] According to an embodiment of the present invention, the multi-angle video data of the rehabilitation personnel when using the lower limb hip and knee rehabilitation equipment is obtained by using machine vision technology, specifically: Through machine vision technology, set up multi-angle video acquisition devices and apply them to the preset range of motion of rehabilitation personnel; Based on the multi-angle video acquisition device, corresponding multi-angle video data is obtained during the use period of a lower limb hip and knee rehabilitation device.

[0040] It should be noted that the video acquisition device includes multiple devices to acquire video data from multiple angles. The multi-angle video data includes multiple video data corresponding to multiple angles.

[0041] According to an embodiment of the present invention, based on each angle, key frames are extracted from the video data to form a rehabilitation process image data set, each angle corresponds to an image data set, the image data set is used to locate the target object of the rehabilitation device, and the movement characteristics of the target object are analyzed. Combined with the rehabilitation plan, the use process time period is divided into multiple motion cycles, specifically: Based on each angle, key frames are extracted from the video data to form a rehabilitation process image dataset; Perform image denoising, enhancement and standardization preprocessing on image datasets; Based on the rehabilitation equipment, target objects are set for multiple equipment joints, and key image features of the target objects are stored; An image data set at an angle is selected, and a recognition module is used to perform multi-frame image recognition on the image data set. The recognition process is to detect and locate the target object by combining key image features, and to analyze the movement features of the target object in the multi-frame images to generate movement data of the target object. Analyze the mobile data, analyze the continuity of the rehabilitation personnel's movements in combination with the rehabilitation plan, and divide the usage process time period into multiple movement cycles.

[0042] It should be noted that the image data set includes image frames in the entire time period. The recognition module can perform target detection and recognition through the YOLO recognition model. The movement data includes information such as the movement amplitude, movement speed, and movement angle of the target object at the image level. Each motion cycle corresponds to a continuous action time period, and different motion cycles correspond to the operation time periods of different rehabilitation equipment modules. Dividing multiple motion cycles helps to conduct multi-dimensional motion state analysis of the rehabilitation equipment. Rehabilitation equipment refers to lower limb hip and knee rehabilitation equipment, which generally includes multiple joint modules, and different joint modules correspond to independent control parameters, such as power assistance parameters, electrical stimulation parameters, and auxiliary function parameters, which are the same as setting the rehabilitation operation state of the rehabilitation equipment. The target object is multiple equipment joint parts.

[0043] According to an embodiment of the present invention, within one motion cycle, the image data sets at multiple angles are subjected to motion target detection, target segmentation and feature extraction based on the HOG algorithm to form motion feature data for each image data set, and one angle is used as a scale to perform multi-scale fusion of the motion feature data through a hierarchical multi-scale Transformer model to generate fused motion feature data, specifically: Grayscale the image data set, calculate the gradient size and direction of local pixels in each frame based on the HOG algorithm, extract multiple object contour features, perform target detection and segmentation from multiple object contour features, and construct the target feature vector for each frame; Perform feature change analysis on the target feature vectors of two adjacent frames of images to generate feature difference information, and integrate the target feature vector of each frame of image with the feature difference information to form motion feature data; For each image data set, generate corresponding motion feature data; Through the hierarchical multi-scale Transformer model, multiple Transformer sub-modules are constructed based on each angle as a scale, and multiple motion feature data are respectively input into the Transformer sub-module for feature conversion to form multi-scale feature data; Through the attention mechanism, multi-scale feature data are fused and represented to generate fused motion feature data.

[0044] It should be noted that the motion target is generally the overall target of the rehabilitation personnel when wearing or using the rehabilitation equipment, and is used to analyze the motion state of the rehabilitation equipment and the rehabilitation personnel during the combined use of the rehabilitation equipment and the rehabilitation personnel. Feature change analysis can be performed based on vector differences.

[0045] According to an embodiment of the present invention, each motion cycle includes corresponding fused motion feature data.

[0046] It should be noted that the fused motion feature data can effectively describe the characteristic information of rehabilitation movement within a movement cycle, and the fused feature is based on multi-angle fusion analysis, which can more comprehensively extract the motion state and motion conditions of the rehabilitation movement process. Furthermore, it can subsequently compare the rehabilitation conditions and screen out the rehabilitation equipment modules to be optimized, thereby realizing the operation optimization and personalized settings of the equipment to meet the needs of different rehabilitation plans and rehabilitation populations.

[0047] According to an embodiment of the present invention, based on the rehabilitation plan and the exercise cycle, the multi-angle motion parameters are set when the motion target is in an ideal state, the GAN generation model is used to generate simulation features of the motion target in combination with the motion parameters, and the multi-angle standard motion features are obtained. The standard motion features are multi-scale fused through the hierarchical multi-scale Transformer model to generate standard fusion feature data, specifically: Based on the rehabilitation plan and exercise cycle division, set multi-angle exercise parameters under ideal exercise goals; According to the motion parameters, multiple angle image frames of the moving target in the initial state and the moving state are obtained from the system database, and the image frames at each angle are marked as the initial image set; Extract features from the initial image set based on the HOG algorithm to form initial motion feature data; Construct a generative model based on GAN, import the initial motion feature data and motion parameters into the generative model for training and learning, and cyclically generate simulated motion feature data. During each training process, analyze the motion state of the object based on the simulated motion feature data and the initial motion feature data, and use the consistency between the object's motion state and the motion parameters as the basis for judging the accuracy of the generated data. Repeated training is performed until the generative model reaches the preset number of training times. Generate standard motion features through the trained generative model; Generate multi-angle standard motion features based on the multi-angle initial image set; The standard motion features are fused at multiple scales through a hierarchical multi-scale Transformer model to generate standard fused feature data.

[0048] It should be noted that the initial image set includes multiple images, each of which corresponds to an angle. The initial image set is a set of multiple simple frames of the target object in a preset motion state, which is obtained from the system database. Subsequently, the GAN generation model is used to simulate the features of the entire motion process of the initial image set, thereby obtaining multi-angle simulated feature data. The simulated feature data is simulated data of coherent movements, which is used for real-time feature comparison, so as to evaluate the rehabilitation and motion conditions during the operation of the equipment. The multi-scale fusion process of the standard motion features through the hierarchical multi-scale Transformer model is consistent with the analysis and fusion process of the fused motion feature data.

[0049] According to an embodiment of the present invention, the fusion motion feature data and the standard fusion feature data are used as two samples, and the similarity between the samples is calculated based on the Mahalanobis distance, and the running state of the rehabilitation device is evaluated based on the similarity to generate the rehabilitation motion index, specifically: The fused motion feature data and the standard fused feature data are taken as two samples, and the distance value between the samples is calculated based on the Mahalanobis distance; The similarity between samples is analyzed by distance value, and the operation status of rehabilitation equipment is evaluated by similarity to generate rehabilitation movement index; The rehabilitation movement index is proportional to the similarity.

[0050] According to an embodiment of the present invention, the corresponding rehabilitation exercise index is calculated for each exercise cycle, and the rehabilitation exercise index is compared with a preset range to optimize the settings of various modules of the corresponding rehabilitation equipment, specifically: Calculate and analyze each exercise cycle to obtain the corresponding rehabilitation exercise index; Determine whether the rehabilitation exercise index of each exercise cycle meets the preset range. If not, mark the corresponding exercise cycle as an optimized exercise cycle; Mark the equipment modules operated by the rehabilitation equipment during the optimized exercise cycle, optimize the parameters of the equipment modules, and generate an optimization plan.

[0051] It should be noted that the optimization scheme includes angle setting, power setting, auxiliary function setting, electrical stimulation parameter setting, etc. of each module of the rehabilitation equipment. The equipment module is the joint module.

[0052] It is worth mentioning here that when rehabilitation personnel use rehabilitation equipment, the movement state of their lower limbs and the operation state of the equipment are a series of complex processes. Traditional technology makes it difficult to effectively analyze the rehabilitation movement process and conduct multi-dimensional movement evaluation, which makes it difficult to achieve precise equipment parameter optimization and equipment regulation, causing the rehabilitation equipment to remain at the popular parameter settings, and it is difficult to further improve the operating efficiency and use effect of the rehabilitation equipment.

[0053] In the present invention, the fusion of multi-scale feature analysis helps to accurately analyze the continuity, standardization and rehabilitation status of movements in each overall movement cycle or a movement mode, and the fusion analysis of the equipment movement conditions from multiple angles of the rehabilitation movement process is performed, and the corresponding rehabilitation movement index is generated to realize the effectiveness evaluation of the rehabilitation movement; further, the rehabilitation movement index of each movement cycle is compared and analyzed, and the parameters of the corresponding equipment modules are optimized and feedback adjusted.

[0054] The third aspect of the present invention also provides a computer-readable storage medium, which includes a lower limb hip and knee rehabilitation equipment operation optimization program based on machine vision. When the lower limb hip and knee rehabilitation equipment operation optimization program based on machine vision is executed by a processor, the steps of the lower limb hip and knee rehabilitation equipment operation optimization method based on machine vision as described in any one of the above items are implemented.

[0055] The present invention discloses a method for optimizing the operation of lower limb hip and knee rehabilitation equipment based on machine vision. First, obtain multi-angle video data when rehabilitation personnel use the equipment, extract key frames and locate target objects, analyze movement features and divide movement cycles; use HOG algorithm to detect, segment and extract movement features, perform multi-scale fusion through Transformer model, and generate fused movement feature data; based on the rehabilitation plan, use GAN to generate a model to simulate standard movement features under ideal conditions, and perform multi-scale fusion to generate standard fused feature data; calculate the similarity between fused movement feature data and standard fused feature data, evaluate the operation status of rehabilitation equipment and generate a rehabilitation movement index; optimize equipment module parameters based on the rehabilitation movement index. Through the present invention, it is possible to accurately analyze the continuity, standardization and corresponding rehabilitation status of the operation of rehabilitation equipment, effectively evaluate the rehabilitation effect, and realize dynamic optimization and real-time adjustment of equipment parameters.

[0056] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0057] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0058] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0059] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0060] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0061] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for optimizing the operation of lower limb hip and knee rehabilitation equipment based on machine vision, characterized in that: include: Through machine vision technology, multi-angle video data of rehabilitation personnel using lower limb hip and knee rehabilitation equipment is obtained; Based on each angle, key frames are extracted from the video data to form a rehabilitation process image dataset. Each angle corresponds to an image dataset. The target object of the rehabilitation equipment is located in the image dataset, and the movement characteristics of the target object are analyzed. Combined with the rehabilitation plan, the use process time period is divided into multiple movement cycles; In one motion cycle, the image data sets at multiple angles are subjected to motion target detection, target segmentation and feature extraction based on the HOG algorithm to form motion feature data for each image data set. Taking one angle as a scale, the motion feature data is subjected to multi-scale fusion through the hierarchical multi-scale Transformer model to generate fused motion feature data. Based on the rehabilitation plan and exercise cycle, the multi-angle motion parameters are set when the exercise target is in an ideal state. The GAN generation model is used to generate simulation features of the exercise target in combination with the motion parameters to obtain multi-angle standard motion features. The hierarchical multi-scale Transformer model is used to perform multi-scale fusion of the standard motion features to generate standard fusion feature data. The fused motion feature data and the standard fused feature data are taken as two samples, and the similarity between the samples is calculated based on the Mahalanobis distance, and the running status of the rehabilitation equipment is evaluated based on the similarity to generate a rehabilitation motion index; The corresponding rehabilitation movement index is calculated for each exercise cycle, and the rehabilitation movement index is compared with the preset range to optimize the settings of various modules of the corresponding rehabilitation equipment.

2. The method for optimizing the operation of lower limb hip and knee rehabilitation equipment based on machine vision according to claim 1, characterized in that: The multi-angle video data of the rehabilitation personnel when using the lower limb hip and knee rehabilitation equipment is obtained by machine vision technology, specifically: Through machine vision technology, set up multi-angle video acquisition devices and apply them to the preset range of motion of rehabilitation personnel; Based on the multi-angle video acquisition device, corresponding multi-angle video data is obtained during the use period of a lower limb hip and knee rehabilitation device.

3. The method for optimizing the operation of lower limb hip and knee rehabilitation equipment based on machine vision according to claim 1, characterized in that: Based on each angle, the video data is keyframe extracted to form a rehabilitation process image data set. Each angle corresponds to an image data set. The target object of the rehabilitation device is located in the image data set, and the movement characteristics of the target object are analyzed. Combined with the rehabilitation plan, the use process time period is divided into multiple movement cycles, specifically: Based on each angle, key frames are extracted from the video data to form a rehabilitation process image dataset; Perform image denoising, enhancement and standardization preprocessing on image datasets; Based on the rehabilitation equipment, target objects are set for multiple equipment joints, and key image features of the target objects are stored; An image data set at an angle is selected, and a recognition module is used to perform multi-frame image recognition on the image data set. The recognition process is to detect and locate the target object by combining key image features, and to analyze the movement features of the target object in the multi-frame images to generate movement data of the target object. Analyze the mobile data, analyze the continuity of the rehabilitation personnel's movements in combination with the rehabilitation plan, and divide the usage process time period into multiple movement cycles.

4. The method for optimizing the operation of lower limb hip and knee rehabilitation equipment based on machine vision according to claim 1, characterized in that: In one motion cycle, the image data sets at multiple angles are subjected to motion target detection, target segmentation and feature extraction based on the HOG algorithm to form motion feature data for each image data set. With one angle as a scale, the motion feature data are subjected to multi-scale fusion through a hierarchical multi-scale Transformer model to generate fused motion feature data, specifically: Grayscale the image data set, calculate the gradient size and direction of local pixels in each frame based on the HOG algorithm, extract multiple object contour features, perform target detection and segmentation from multiple object contour features, and construct the target feature vector for each frame; Perform feature change analysis on the target feature vectors of two adjacent frames of images to generate feature difference information, and integrate the target feature vector of each frame of image with the feature difference information to form motion feature data; For each image data set, generate corresponding motion feature data; Through the hierarchical multi-scale Transformer model, multiple Transformer sub-modules are constructed based on each angle as a scale, and multiple motion feature data are respectively input into the Transformer sub-module for feature conversion to form multi-scale feature data; Through the attention mechanism, multi-scale feature data are fused and represented to generate fused motion feature data.

5. The method for optimizing the operation of lower limb hip and knee rehabilitation equipment based on machine vision according to claim 1, characterized in that: Each motion cycle includes corresponding fused motion feature data.

6. The method for optimizing the operation of lower limb hip and knee rehabilitation equipment based on machine vision according to claim 1, characterized in that: Based on the rehabilitation plan and exercise cycle, the multi-angle motion parameters are set when the motion target is in an ideal state, and the GAN generation model is used to generate simulation features of the motion target in combination with the motion parameters to obtain multi-angle standard motion features. The hierarchical multi-scale Transformer model is used to perform multi-scale fusion of the standard motion features to generate standard fusion feature data, specifically: Based on the rehabilitation plan and exercise cycle division, set multi-angle exercise parameters under ideal exercise goals; According to the motion parameters, multiple angle image frames of the moving target in the initial state and the moving state are obtained from the system database, and the image frames at each angle are marked as the initial image set; Extract features from the initial image set based on the HOG algorithm to form initial motion feature data; Construct a generative model based on GAN, import the initial motion feature data and motion parameters into the generative model for training and learning, and cyclically generate simulated motion feature data. During each training process, analyze the motion state of the object based on the simulated motion feature data and the initial motion feature data, and use the consistency between the object's motion state and the motion parameters as the basis for judging the accuracy of the generated data. Repeated training is performed until the generative model reaches the preset number of training times. Generate standard motion features through the trained generative model; Generate multi-angle standard motion features based on the multi-angle initial image set; The standard motion features are fused at multiple scales through a hierarchical multi-scale Transformer model to generate standard fused feature data.

7. The method for optimizing the operation of lower limb hip and knee rehabilitation equipment based on machine vision according to claim 1, characterized in that: The fusion motion feature data and the standard fusion feature data are used as two samples, and the similarity between the samples is calculated based on the Mahalanobis distance, and the operation status of the rehabilitation equipment is evaluated based on the similarity to generate a rehabilitation motion index, which is specifically: The fused motion feature data and the standard fused feature data are taken as two samples, and the distance value between the samples is calculated based on the Mahalanobis distance; The similarity between samples is analyzed by distance value, and the operation status of rehabilitation equipment is evaluated by similarity to generate rehabilitation movement index; The rehabilitation movement index is proportional to the similarity.

8. The method for optimizing the operation of lower limb hip and knee rehabilitation equipment based on machine vision according to claim 1, characterized in that: The corresponding rehabilitation exercise index is calculated for each exercise cycle, and the rehabilitation exercise index is compared with the preset range to optimize the settings of various modules of the corresponding rehabilitation equipment, specifically: Calculate and analyze each exercise cycle to obtain the corresponding rehabilitation exercise index; Determine whether the rehabilitation exercise index of each exercise cycle meets the preset range. If not, mark the corresponding exercise cycle as an optimized exercise cycle; Mark the equipment modules operated by the rehabilitation equipment during the optimized exercise cycle, optimize the parameters of the equipment modules, and generate an optimization plan.

9. A lower limb hip and knee rehabilitation equipment operation optimization system based on machine vision, characterized in that: The system includes: a memory and a processor, wherein the memory includes a lower limb hip and knee rehabilitation device operation optimization program based on machine vision, and the lower limb hip and knee rehabilitation device operation optimization program based on machine vision is executed by the processor to implement the following steps: Through machine vision technology, multi-angle video data of rehabilitation personnel using lower limb hip and knee rehabilitation equipment is obtained; Based on each angle, key frames are extracted from the video data to form a rehabilitation process image dataset. Each angle corresponds to an image dataset. The target object of the rehabilitation equipment is located in the image dataset, and the movement characteristics of the target object are analyzed. Combined with the rehabilitation plan, the use process time period is divided into multiple movement cycles; In one motion cycle, the image data sets at multiple angles are subjected to motion target detection, target segmentation and feature extraction based on the HOG algorithm to form motion feature data for each image data set. Taking one angle as a scale, the motion feature data is subjected to multi-scale fusion through the hierarchical multi-scale Transformer model to generate fused motion feature data. Based on the rehabilitation plan and exercise cycle, the multi-angle motion parameters are set when the exercise target is in an ideal state. The GAN generation model is used to generate simulation features of the exercise target in combination with the motion parameters to obtain multi-angle standard motion features. The hierarchical multi-scale Transformer model is used to perform multi-scale fusion of the standard motion features to generate standard fusion feature data. The fused motion feature data and the standard fused feature data are taken as two samples, and the similarity between the samples is calculated based on the Mahalanobis distance, and the running status of the rehabilitation equipment is evaluated based on the similarity to generate a rehabilitation motion index; The corresponding rehabilitation movement index is calculated for each exercise cycle, and the rehabilitation movement index is compared with the preset range to optimize the settings of various modules of the corresponding rehabilitation equipment.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a lower limb hip and knee rehabilitation equipment operation optimization program based on machine vision. When the lower limb hip and knee rehabilitation equipment operation optimization program based on machine vision is executed by a processor, the steps of the lower limb hip and knee rehabilitation equipment operation optimization method based on machine vision as described in any one of claims 1 to 8 are implemented.

Images