Joint range of motion measurement method and device, storage medium and computer equipment

Abstract

The application discloses a kind of joint range of motion measurement method, device, storage medium and computer equipment.Method comprising: obtaining the joint label of joint to be measured and multiple human body part labels;Real-time acquisition of muscle electrical signal parameters is carried out by electrical signal collector arranged at a predetermined position, and the muscle electrical signal parameters are input into the pre-trained range of motion determination model to obtain the judgment value of whether the joint reaches the maximum range of motion;When the judgment value is that the joint reaches the maximum range of motion, the image information of the target person is collected;Image information, joint label and human body part label are input into the pre-trained convolutional neural network model to obtain the joint position in image information and the human body part position of human body part;Based on joint position and human body part position, the range of motion of joint is determined, and the range of motion is confirmed as the maximum range of motion of joint.The above method can improve the accuracy of determining the range of motion of joint.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and in particular to a joint activity measurement method and device, a storage medium and computer equipment. BACKGROUND

[0002] In the insurance industry, the disability compensation is an important part of the car injury claim. In order to determine the amount of disability compensation, the car injury level must be first identified, and the car injury level identification is directly related to the compensation amount of the car. An important part of the disability identification is the disability identification of the loss of function of the six major joints of the human body.

[0003] Currently, the passive activity of each joint is measured by an angle measuring device during the disability identification of the loss of function of the six major joints of the human body. However, in the actual measurement process, the measurement accuracy of the joint activity is seriously low due to the operation errors of the operators or the intentional disguising and hiding of the measured personnel. SUMMARY

[0004] Therefore, the present application provides a joint activity measurement method and device, a storage medium and computer equipment, which mainly aims to solve the technical problem of low measurement accuracy of joint activity.

[0005] According to a first aspect of the present application, a joint activity measurement method is provided, which comprises:

[0006] obtaining a joint label of a joint to be measured of a target person and a plurality of human body part labels corresponding to the joint label, wherein each human body part label corresponds to a human body part;

[0007] real-time acquisition of muscle electrical signal parameters by an electrical signal collector arranged at a preset position corresponding to the joint, and inputting the muscle electrical signal parameters into a pre-trained activity determination model to obtain a judgment value of whether the joint reaches the maximum activity;

[0008] when the judgment value is that the joint reaches the maximum activity, collecting image information of the target person, wherein the image information includes an image of the joint and an image of a human body part corresponding to each human body part label;

[0009] inputting the image information, the joint label and the human body part label into a pre-trained convolutional neural network model to obtain a joint position of the joint in the image information and a human body part position of the human body part;

[0010] Based on the joint position and the body part position, an activity angle of the joint is determined, and the activity angle is determined as the maximum activity degree of the joint.

[0011] According to a second aspect of the present application, there is provided a joint activity degree measuring device, which comprises:

[0012] A label obtaining module is configured to obtain joint labels of joints to be measured of a target person and a plurality of body part labels corresponding to the joint labels, wherein each body part label corresponds to a body part.

[0013] A judgment executing module is configured to obtain muscle electrical signal parameters in real time through an electrical signal collector arranged at a preset position corresponding to the joint, and input the muscle electrical signal parameters into a pre-trained activity degree determining model to obtain a judgment value of whether the joint reaches the maximum activity degree.

[0014] An image obtaining module is configured to collect image information of the target person when the judgment value is that the joint reaches the maximum activity degree, wherein the image information comprises an image of the joint and an image of a body part corresponding to each body part label.

[0015] An image recognizing module is configured to input the image information, the joint labels and the body part labels into a pre-trained convolutional neural network model to obtain a joint position of the joint in the image information and a body part position of the body part.

[0016] A result determining module is configured to determine an activity angle of the joint based on the joint position and the body part position, and determine the activity angle as the maximum activity degree of the joint.

[0017] According to a third aspect of the present application, there is provided a storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the joint activity degree measuring method.

[0018] According to a fourth aspect of the present application, there is provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the joint activity degree measuring method when executing the program.

[0019] The application provides a joint activity measurement method, device, storage medium and computer equipment, which comprises the following steps: firstly, obtaining a joint label of a joint to be tested of a testee and a human body part label of a human body part, so that the program knows a target joint to be measured and a plurality of human body parts which jointly form a measurement angle; then, collecting a muscle electrical signal parameter of a muscle at a position based on an electrical signal collector arranged at the position of the target joint, and determining whether the joint reaches a maximum activity based on an activity determination model; after that, when the target joint reaches the maximum activity, collecting image information of the testee containing the target joint and the human body part, and determining position information of the target joint and the human body part based on a convolutional neural network model; finally, determining an angle of the measurement angle formed by the target joint and the human body part based on the position information of the target joint and the human body part, that is, the maximum activity of the target joint. The method provided by the application can accurately determine whether the joint of the testee reaches the maximum activity, avoids the misjudgment of whether the joint reaches the maximum activity due to the disguise of the testee, and determines the positions of the target joint and the human body part based on the convolutional neural network model, so that the activity value of the joint can be more accurately obtained.

[0020] The above description is only a summary of the technical scheme of the application, in order to more clearly understand the technical means of the application, and to implement the content of the specification, and in order to make the above and other purposes, features and advantages of the application more obvious and easy to understand, the following specific embodiments of the application are described. BRIEF DESCRIPTION OF DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the application, and form a part of the application. The schematic embodiments of the application and the description thereof are used to explain the application, and do not constitute an improper limitation on the application. In the drawings:

[0022] Figure 1 A flowchart of a joint activity measurement method provided by an embodiment of the application is shown;

[0023] Figure 2 A schematic diagram of confirming a target joint and a human body part in image information provided by an embodiment of the application is shown;

[0024] Figure 3 A schematic diagram of a key point heat map provided by an embodiment of the application is shown;

[0025] Figure 4 A structural schematic diagram of a joint activity measurement device provided by an embodiment of the application is shown;

[0026] Figure 5 A structural schematic diagram of another joint activity measurement device provided by an embodiment of the application is shown. DETAILED DESCRIPTION

[0027] The application will be described in detail below with reference to the drawings and embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0028] At present, the disability compensation is an important part of the injury compensation paid by major insurance companies for the injured in motor vehicle insurance. In order to determine the amount of disability compensation, the injury grade of the injured in motor vehicle insurance must be identified first, and the identification of the injury grade of the injured in motor vehicle insurance is directly related to the compensation amount for the injured in motor vehicle insurance. An important part of the identification of the injury grade is the identification of the disability of the loss of function of the six major joints of the human body. However, in the existing identification of the disability of the loss of function of the six major joints of the human body, an angle measuring device is often used to measure the passive range of motion of each joint. However, in the actual measurement process, the measurement accuracy of the range of motion of the joint is seriously low due to the operation errors of the operator or the intentional disguise or concealment of the measured person.

[0029] In view of the above problems, in one embodiment, as shown in Figure 1 a joint range of motion measurement method is provided, which is taken as an example of application to a computer device and includes the following steps:

[0030] 101. Obtain the joint label of the joint to be measured of the target person and the plurality of human body part labels corresponding to the joint label.

[0031] Each of the human body part labels corresponds to a human body part. Further, the joint label is used to mark the joint of the target person that needs to be measured for the range of motion, and the joint is determined as the target joint. Further, the target joint can be the left elbow, the right elbow, the left wrist, the right wrist, the left shoulder, the right shoulder, the left knee, the right knee, the left ankle, and the right ankle. The human body part label is used to mark other human body parts that need to be associated with the target joint for the measurement of the range of motion of the target joint, and the human body part can be the nose, the left ear, the right ear, the left elbow, the right elbow, the left wrist, the right wrist, the left shoulder, the right shoulder, the left knee, the right knee, the left ankle, and the right ankle. The target joint can be used as the common endpoint of the angle, each human body part and the target joint mark a line to form an angle, and the target joint and the plurality of human body parts together form an angle, and the range of motion of the target joint is determined by the angle of the angle. Further, each target joint can also correspond to a plurality of human body parts that are preset, so that when the joint label of a target joint is determined, the human body part label corresponding to the joint label is also automatically generated.

[0032] As an example, when measuring the range of motion of a target person's right elbow, the joint label of the right elbow, the body part label of the right wrist, and the body part label of the right shoulder, input by the staff, can be obtained. The lines formed by the right elbow and the right wrist, and the lines formed by the right elbow and the right shoulder, with the right elbow as the common endpoint, can then be determined to form an angle representing the range of motion of the right elbow, which can then be used for subsequent right elbow range of motion testing. It should be noted that the method of measuring joint range of motion based on the angle formed by the body part and the target joint can be determined based on the actual situation and is applicable to this embodiment.

[0033] At the same time, corresponding human body part labels can be pre-set for each joint label. When a joint label is obtained, a human body part label corresponding to that joint label can be automatically generated to improve measurement efficiency.

[0034] 102. Acquire muscle electrical signal parameters in real time by setting an electrical signal acquisition device at a preset position corresponding to the joint, and input the muscle electrical signal parameters into a pre-trained range of motion determination model to obtain a judgment value on whether the joint has reached its maximum range of motion.

[0035] The judgment value can be whether the joint has reached its maximum range of motion or not. Furthermore, the preset position can be a pre-defined location for measuring the electromyographic (EMG) signals of the target joint, used to set up the EMG acquisition device. Once the target joint for which range of motion measurement is determined, the EMG acquisition device can be set at the preset position corresponding to the target joint to receive the EMG signals from the muscles at that position and obtain the EMG signal parameters. Furthermore, the range of motion determination model can be a neural network model, trained based on the EMG signal parameters at the preset position corresponding to the joint as features, with the maximum and minimum range of motion of the joint as labels. This model receives the EMG signal parameters corresponding to the target joint and outputs a judgment value indicating whether the target joint has reached its maximum range of motion. Furthermore, a range of motion determination model can be trained for each joint, or a model applicable to each joint can be trained; the specific form can be determined based on the actual situation.

[0036] Specifically, after identifying the target joint for which range of motion measurement is required, an electromyography (EMG) sensor can be positioned at a preset location corresponding to the target joint, allowing the target person to perform relevant movements, such as progressively bending the target joint. At this time, the EMG parameters collected in real-time by the sensor positioned at the target joint are received and input into the range of motion determination model to determine whether the target joint has reached its maximum range of motion.

[0037] 103、in the judgment value is the joint reaches the maximum range of motion, the image information of the target person is collected.

[0038] The image information includes images of the joint and images of the body parts corresponding to each of the joint labels. Further, the image information can be a frame of image of the target person acquired by an image acquisition device.

[0039] Specifically, the range of motion determination model receives muscle electrical signal parameters corresponding to the target joint in real time, and determines in real time whether the target joint reaches the maximum range of motion. When the target person moves the target joint to the maximum range of motion, the range of motion determination model outputs a judgment value that the target joint reaches the maximum range of motion based on the muscle electrical signal parameters received at this time.

[0040] Further, when the judgment value is that the joint reaches the maximum range of motion, the image of the target person can be collected based on an image acquisition device such as a camera. The image includes the target joint and the body part corresponding to the target joint. Specifically, the image acquisition device can be set in the area corresponding to the target joint of the target person to be measured. The focal plane of the image acquisition device is parallel to the plane formed by the target joint and the body part corresponding to the target joint, and the image including the target joint and the body part corresponding to the target joint is completely collected for subsequent angle determination work to obtain the image of the target joint and the body part. It should be noted that the image acquisition device can acquire multiple frames of image information within a short time when the target joint is in the maximum activity position, so as to prevent the image information from being abnormal and causing the convolutional neural network model to be unable to recognize.

[0041] 104、the image information, the joint label and the body part label are input into a pre-trained convolutional neural network model to obtain the joint position of the joint in the image information and the body part position of the body part.

[0042] The convolutional neural network model can be pre-trained and can determine the positions of the target joints and body parts in the image based on the input image information.

[0043] Specifically, the joint label and the body part label can be input into the convolutional neural network model to determine the joints and body parts that need to be recognized by the convolutional neural network model. At the same time, the image information when the target joint reaches the maximum range of motion is input into the convolutional neural network model, so that the convolutional neural network model can determine the positions of the target joint and the body part in the image information. As an example, as shown in Figure 2As shown, if the target joint is the right elbow, and the human body parts are the right shoulder and the right wrist, the image information is input into the convolutional neural network model together with the joint label of the right elbow, the human body part label of the right shoulder, and the human body part label of the right wrist, and the convolutional neural network model outputs an image with the joint position 210 and the human body part position 220, so as to obtain the joint position 210 of the joint and the human body part position 220 of the human body part in the image information.

[0044] 105. Based on the joint position and the human body part position, the activity angle of the joint is determined, and the maximum activity degree of the joint is determined as the activity angle.

[0045] Specifically, the position of the target joint can be taken as a common end point, and the activity angle of the target joint can be obtained based on the line formed by each human body part and the common end point, and then the maximum activity degree of the joint can be calculated.

[0046] The activity degree measurement method of the joint provided in the embodiment first acquires the joint label of the joint to be tested of the tested person and the human body part label of the human body part, so that the program knows the target joint to be measured for the activity degree and the multiple human body parts that jointly form a measurement angle with the target joint. Then, the muscle electrical signal parameter of the muscle at the corresponding position of the target joint is collected based on the electrical signal collector arranged at the position, and whether the joint reaches the maximum activity degree is determined based on the activity degree determination model. After that, when the target joint reaches the maximum activity degree, the image information of the target joint and the human body part of the tested person is collected, and the position information of the target joint and the human body part is determined based on the convolutional neural network model. Finally, the angle of the measurement angle formed by the target joint and the human body part, i.e., the maximum activity degree of the target joint, is determined based on the position information of the target joint and the human body part. The method provided in the application can accurately determine whether the joint of the tested person reaches the maximum activity degree, avoid the misjudgment of whether the joint reaches the maximum activity position caused by the tested person's disguise, and determine the positions of the target joint and the human body part based on the convolutional neural network model, so that the activity degree value of the joint can be more accurately obtained.

[0047] In one embodiment, the implementation method of step 104 can be: first, determining the joint and the human body part to be recognized based on the joint label and the human body part label. Specifically, inputting the joint label and the human body part label into the convolutional neural network model can make the convolutional neural network model determine the target joint and the human body part to be recognized. Then, the image information is input into the convolutional neural network model based on the function of outputting the heat map of the convolutional neural network model, and the joint key point heat map of the joint and the part key point heat map of the human body part are output respectively. Specifically, as shown in FIG. 4, the image information is input into the convolutional neural network model, and the joint key point heat map of the target joint and the part key point heat map of the human body part are output respectively. Figure 3As shown, each key point heatmap 300 (Heatmap) in the joint key point heatmap and the part key point heatmap contains a plurality of point positions 310, each of which has a probability value or weight value indicating the probability of the point position being the position of the identified object. The key point heatmap is obtained based on the image information input into the convolutional neural network model, and the size and scale of the key point heatmap are the same as those of the image information. Therefore, the positions of the target joints and body parts in the image information can correspond to the positions in the key point heatmap, and the joint positions and body part positions in the image information can be obtained based on the positions of the target joints and body parts in the key point heatmap. Finally, the point position with the highest probability value in the joint key point heatmap is determined as the joint position, and the point position with the highest probability value in the part key point heatmap is determined as the body part position. Specifically, if the key point heatmap 300 is a joint key point heatmap, the point position with the highest probability value is determined as the joint position of the target joint in the image information; if the key point heatmap 300 is a part key point heatmap, the point position with the highest probability value is determined as the body part position of the body part in the image information.

[0048] In the embodiments of the present application, based on the key point heatmap output by the convolutional neural network model, the point position with the highest probability value is located on the key point heatmap as the point position of the joint or body part to be identified in the image, and the positions of the joint and body part in the image are further determined, so that the relative positions between the joint and body part can be obtained, and the positioning speed of the joint and body part is improved.

[0049] In one embodiment, the implementation method of determining the point position with the highest probability value in the joint key point heatmap as the joint position and the point position with the highest probability value in the part key point heatmap as the body part position includes: first, obtaining the probability value of the point position with the highest probability value in the joint key point heatmap and the probability value of the point position with the highest probability value in the part key point heatmap. Specifically, the point position with the highest probability value is located in the joint key point heatmap, and the highest probability value is obtained; the point position with the highest probability value is located in the part key point heatmap, and the highest probability value is obtained. Then, it is judged whether the probability value of the point position with the highest probability value in the joint key point heatmap and the probability value of the point position with the highest probability value in the part key point heatmap are greater than or equal to a preset probability threshold. Specifically, the probability value of the point position with the highest probability value in the joint key point heatmap and the probability value of the point position with the highest probability value in the part key point heatmap are compared with the preset probability threshold. The probability threshold can be obtained based on tests or experiments in advance, and is used to determine when the probability value is greater than or equal to a certain value, the identification of the joint or body part by the convolutional neural network model is accurate. The value of the probability threshold can be determined based on actual conditions.

[0050] Further, if the probability value of the point with the highest probability value in the joint key point heat map and the probability value of the point with the highest probability value in the part key point heat map are both greater than or equal to the probability threshold value, the point with the highest probability value in the joint key point heat map is determined as the joint position, and the point with the highest probability value in the part key point heat map is determined as the human body part position. As an example, in the case of a probability threshold value of 0.7, if the probability value of the point with the highest probability value in the joint key point heat map is 0.9, and the probability values of the two points with the highest probability value in the two part key point heat maps are 0.7 and 0.8 respectively, the point with the highest probability value in the joint key point heat map is determined as the joint position, and the point with the highest probability value in the part key point heat map is determined as the human body part position. Correspondingly, if the probability value of the point with the highest probability value in the joint key point heat map is 0.9, and the probability values of the two points with the highest probability value in the two part key point heat maps are 0.7 and 0.5 respectively, the image information is discarded and the identification is performed based on other obtained image information that the target joint is at the maximum activity position, such as making the target person continue to perform the joint activity action, and then collecting image information again when the target joint reaches the maximum activity position again.

[0051] In the embodiments of the present application, whether the identification and positioning results of the target joint and the human body part are effective can be judged based on the preset probability threshold value, so as to prevent the joint activity measurement from failing due to identification errors.

[0052] In an embodiment, step 104 further comprises: first, establishing a reference coordinate system for the image information. Specifically, a two-dimensional reference coordinate system can be established with a certain endpoint or any point of the single-frame image as the coordinate origin. Since the size and scale of the joint keypoint heat map and the part keypoint heat map are the same as those of the image information, the point in the joint keypoint heat map and the part keypoint heat map that is the same as the point of the single-frame image that is taken as the coordinate origin is taken as the origin of the reference coordinate system, so that the points with the same coordinates in the image information, the joint keypoint heat map and the part keypoint heat map are in the same position relative to the image information, the joint keypoint heat map and the part keypoint heat map. Then, the joint coordinate value of the point with the highest probability value in the joint keypoint heat map in the reference coordinate system is obtained, and the joint position is determined based on the joint coordinate value. Specifically, the coordinate value of the point with the highest probability value in the reference coordinate system corresponding to the joint keypoint heat map is determined, and the coordinate value is determined as the joint position. Further, the part coordinate value of the point with the highest probability value in the part keypoint heat map in the reference coordinate system is obtained, and the part position is determined based on the part coordinate value. Specifically, the coordinate value of the point with the highest probability value in the reference coordinate system corresponding to the part keypoint heat map is determined, and the coordinate value is determined as the part position. In addition, the probability value of the point and the coordinate mapping can also be stored, thereby avoiding the problem of high storage space occupation caused by storing the joint keypoint heat map. In the embodiments of the present application, the joint position and the part position can be accurately obtained based on the coordinate value, thereby providing a calculation basis for subsequent angle calculation work.

[0053] In an embodiment, the implementation of inputting the image information into the convolutional neural network model and respectively outputting the joint keypoint heat map of the joint and the part keypoint heat map of the body part can be: first, performing convolution operation on the image information based on a plurality of preset size convolution kernels to obtain a basic image corresponding to each convolution kernel. Wherein, the size of each convolution kernel is different, and performing convolution operation on image information based on convolution kernels of different sizes can obtain convolution images of different resolutions. The larger the convolution kernel, the larger the receptive field, the more picture information is seen, and the better the global features obtained. And the smaller the convolution kernel, the higher resolution convolution image can be obtained. Specifically, a plurality of convolution kernels of different sizes are used in the feature extraction stage of the convolutional neural network model to perform convolution operation on the image information to obtain basic images of multiple resolutions. Then, the basic image with the largest feature size among the plurality of basic images is determined as the reference basic image. Specifically, the basic image with the largest resolution can be determined as the reference basic image. As an example, if the feature size of the basic image with the largest resolution is 5x5, the basic image can be determined as the reference basic image. Then, the basic images other than the reference basic image are padded to make the feature size of each basic image equal to the feature size of the reference basic image, and all the basic images are spliced to obtain a multi-dimensional image. Specifically, if the feature size of the reference basic image is 5x5, and the feature sizes of the other basic images are 4x4, 3x3 and 2x2 respectively, the other basic images can be padded to have a feature size of 5x5. Further, all the basic images can be spliced to obtain a multi-dimensional image. Finally, the multi-dimensional image is classified and recognized to obtain the joint keypoint heat map and the part keypoint heat map. Specifically, the multi-dimensional image can be classified and recognized in the convolutional neural network model based on the target joint and body part confirmed by the joint label and body part label to obtain the joint keypoint heat map and the part keypoint heat map.

[0054] In the embodiment of the present application, different resolutions of images are obtained by performing convolution operation on image information based on convolution kernels of different sizes. The convolution results with high resolution pay more attention to detailed features, and the convolution results with low resolution pay attention to global features. At the same time, interaction operations are introduced between different branches to splice the outputs of multiple convolution branches, so as to improve the performance of the model, improve the recognition accuracy of the convolutional neural network model, and further improve the measurement accuracy of the maximum activity of the joint.

[0055] In one embodiment, the implementation manner of acquiring the muscle electrical signal parameter in real time by the electrical signal collector arranged at the preset position corresponding to the joint in step 102 can be: first, the electrical signal waveform of the muscle at the preset position is collected based on the electrical signal collector. Specifically, the electrical signal collector can be arranged at the preset position corresponding to the joint to be measured. Wherein, the muscle electrical signal parameter used for training the activity degree determination model of the joint to be measured can be determined, and the muscle generating the muscle electrical signal parameter is determined, and then the position of the muscle is determined as the preset position. Further, the electrical signal waveform of the muscle at the position can be collected by the electrical signal collector arranged at the position. Correspondingly, the corresponding preset position can be preset for each joint. Then, the electrical signal waveform is denoised based on the filter, and the electrical signal parameter of the denoised electrical signal waveform is extracted. Specifically, the electrical signal waveform can be filtered and denoised by using the Butterworth filter, and the electrical signal parameter of the characteristic electrical signal waveform is extracted by the sliding window. Specifically, the electrical signal parameter can be the time domain feature of the electrical signal waveform, such as the average absolute value, the root mean square, the variance, the waveform length, the Willision amplitude, the zero crossing frequency, the logarithmic detection value, the autoregressive model coefficient, etc., the frequency domain feature of the electrical signal waveform, such as the median frequency, the mean frequency, etc., and the time-frequency feature of the electrical signal waveform. At least one of the above features can be used as a training feature of the activity degree determination model. In actual activity degree determination, the same type of feature as the training feature can be collected and input into the model to obtain the judgment value of whether the target joint reaches the maximum activity degree. Finally, the electrical signal parameter is determined as the muscle electrical signal parameter. In the embodiments of the present application, the electrical signal waveform can be collected based on the electrical signal collector arranged at the preset position, and the waveform is denoised to obtain the muscle electrical signal parameter, so as to improve the judgment accuracy of whether the joint reaches the maximum activity position.

[0056] In an embodiment, before step 102, the method further comprises: receiving an input value of whether the target person inputs that the joint reaches the maximum range of motion. Specifically, an input value of whether the target joint reaches the maximum range of motion input by the target person can be received, for example, when the target person thinks that the target joint reaches the maximum range of motion, the confirmation information that the target joint has reached the maximum range of motion can be input in the program. Further, before step 103, the method further comprises: when the input value is that the maximum range of motion is reached, if the judgment value is that the joint does not reach the maximum range of motion, a prompt information is sent. Specifically, if the input value of whether the target joint reaches the maximum range of motion input by the target person is that the maximum range of motion is reached, but the judgment value output by the activity degree determination model is that the joint does not reach the maximum range of motion, a prompt information is sent to prompt the measurement operator that the target person is suspected of faking the joint range of motion limited, so that the staff can make further processing. In the embodiment of the application, the misjudgment of whether the joint reaches the maximum range of motion due to the testee faking can be avoided, and the joint range of motion value can be more accurately obtained.

[0057] The joint range of motion measurement method provided in the embodiment can accurately determine whether the joint of the testee reaches the maximum range of motion, and can also send a prompt information when the testee fakes the joint range of motion limited, to prevent the joint range of motion measurement from being inaccurate due to the testee faking. Further, the position of the target joint and the human body part is accurately determined based on the convolutional neural network model, and the joint range of motion value can be more accurately obtained.

[0058] Further, as a specific implementation of the method shown in Figure 1 The embodiment provides a joint range of motion measurement device, as shown in Figure 4 The device comprises a label acquisition module 41, a judgment execution module 42, an image acquisition module 43, an image recognition module 44, and a result determination module 45.

[0059] The label acquisition module 41 can be used to acquire a joint label of a joint to be measured of a target person and a plurality of human body part labels corresponding to the joint label, wherein each human body part label corresponds to a human body part.

[0060] The judgment execution module 42 can be used to acquire a muscle electrical signal parameter in real time through an electrical signal collector arranged at a preset position corresponding to the joint, and input the muscle electrical signal parameter into a pre-trained activity degree determination model to obtain a judgment value of whether the joint reaches the maximum range of motion.

[0061] The image acquisition module 43 is configured to acquire image information of the target person when the judgment value is that the joint reaches the maximum range of motion, and the image information includes an image of the joint and an image of each human body part corresponding to the human body part label.

[0062] The image recognition module 44 is configured to input the image information, the joint label and the human body part label into a pre-trained convolutional neural network model to obtain a joint position of the joint in the image information and a human body part position of the human body part.

[0063] The result determination module 45 is configured to determine a range of motion of the joint based on the joint position and the human body part position, and determine the range of motion as the maximum range of motion of the joint.

[0064] In a specific application scenario, the image recognition module 44 is specifically configured to determine the joint and the human body part to be recognized based on the joint label and the human body part label, input the image information into the convolutional neural network model to respectively output a joint key point heat map of the joint and a part key point heat map of the human body part, and determine a point with the highest probability value in the joint key point heat map as the joint position and a point with the highest probability value in the part key point heat map as the human body part position.

[0065] In a specific application scenario, the image recognition module 44 is specifically configured to acquire a probability value of the point with the highest probability value in the joint key point heat map and a probability value of the point with the highest probability value in the part key point heat map, determine whether the probability value of the point with the highest probability value in the joint key point heat map and the probability value of the point with the highest probability value in the part key point heat map are greater than or equal to a preset probability threshold, and if the probability value of the point with the highest probability value in the joint key point heat map and the probability value of the point with the highest probability value in the part key point heat map are greater than or equal to the probability threshold, determine the point with the highest probability value in the joint key point heat map as the joint position and the point with the highest probability value in the part key point heat map as the human body part position.

[0066] In a specific application scenario, the image recognition module 44 is specifically configured to establish a reference coordinate system for the image information, acquire a joint coordinate value of the point with the highest probability value in the joint key point heat map in the reference coordinate system and determine the joint position based on the joint coordinate value, and acquire a part coordinate value of the point with the highest probability value in the part key point heat map in the reference coordinate system and determine the part position based on the part coordinate value.

[0067] In a specific application scenario, the image recognition module 44 can be specifically used for performing convolution operation on the image information based on a plurality of preset size convolution kernels to obtain a plurality of basic images corresponding to each of the convolution kernels, wherein the size of each of the convolution kernels is different; determining a basic image with the largest feature size in the plurality of basic images as a reference basic image; performing padding processing on the basic images other than the reference basic image, so that the feature size of each of the basic images is equal to the feature size of the reference basic image, and performing splicing processing on all the basic images to obtain a multi-dimensional image; and performing classification recognition on the multi-dimensional image to obtain the joint key point heat map and the part key point heat map.

[0068] In a specific application scenario, the judgment execution module 42 can be specifically used for collecting an electrical signal waveform of a muscle at the preset position based on the electrical signal collector; performing denoising processing on the electrical signal waveform based on a filter, and extracting an electrical signal parameter of the electrical signal waveform after denoising processing; and determining the electrical signal parameter as a muscle electrical signal parameter.

[0069] In a specific application scenario, as shown in Figure 5 The device further includes an information input module 52, which can be specifically used for receiving an input value of whether the joint reaches the maximum range of motion input by the target person.

[0070] In a specific application scenario, as shown in Figure 5 The device further includes an information input module 52, which can be specifically used for receiving an input value of whether the joint reaches the maximum range of motion input by the target person.

[0071] It should be noted that other corresponding descriptions of the functions of the joint range of motion measuring device provided in this embodiment can be referred to the corresponding descriptions in Figure 1 , which will not be described here.

[0072] Based on the above method as shown in Figure 1 , accordingly, the present embodiment also provides a storage medium having a computer program stored thereon, which is executed by a processor to implement the joint range of motion measuring method as shown in Figure 1 .

[0073] Based on such understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.), including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various implementation scenarios of the present application.

[0074] Based on the above method as shown in Figure 1 , and Figure 4 and Figure 5 The joint range of motion measuring device embodiment shown in the above method, in order to achieve the above purpose, the embodiment also provides a joint range of motion measuring entity device, which can be a personal computer, a server, a smart phone, a tablet computer, a smart watch, or other network devices, etc. The entity device comprises a storage medium and a processor; the storage medium is used for storing a computer program; the processor is used for executing the computer program to realize the above method as shown in Figure 1 .

[0075] Optionally, the entity device can also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface can include a display screen (Display), an input unit such as a keyboard (Keyboard), etc. The optional user interface can also include a USB interface, a card reader interface, etc. The network interface can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), etc.

[0076] Those skilled in the art can understand that the structure of the joint range of motion measuring entity device provided by the embodiment does not constitute a limitation on the entity device, and can include more or fewer components, or combine certain components, or different component arrangements.

[0077] The storage medium can also include an operating system, a network communication module. The operating system is a program for managing the hardware and the to-be-identified software resources of the above entity device, supporting the running of the information processing program and other to-be-identified software and / or programs. The network communication module is used for realizing the communication between the components in the storage medium, and the communication with other hardware and software in the information processing entity device.

[0078] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be realized by means of software and necessary general hardware platforms, or by hardware. Through the application of the technical solutions of the present application, first, the joint label of the joint to be measured of the target person and the plurality of human body part labels corresponding to the joint label are acquired, wherein each human body part label corresponds to a human body part; then, the muscle electrical signal parameters are acquired in real time by the electrical signal collector arranged at the preset position corresponding to the joint, and the muscle electrical signal parameters are input into the pre-trained activity degree determination model to obtain the judgment value of whether the joint reaches the maximum activity degree; then, when the judgment value is that the joint reaches the maximum activity degree, the image information of the target person is collected, wherein the image information includes the image of the joint and the image of the human body part corresponding to each human body part label; then, the image information, the joint label and the human body part label are input into the pre-trained convolutional neural network model to obtain the joint position of the joint in the image information and the human body part position of the human body part; finally, based on the joint position and the human body part position, the activity angle of the joint is determined, and the activity angle is determined as the maximum activity degree of the joint. Compared with the prior art, it can accurately judge whether the joint of the test person reaches the maximum activity degree, avoid the false judgment of whether the joint reaches the maximum activity position caused by the test person disguising, and determine the positions of the target joint and the human body part based on the convolutional neural network model, so that the activity degree value of the joint can be more accurately obtained.

[0079] Those skilled in the art can understand that the drawings are only schematic diagrams of a preferred implementation scenario, and the modules or flows in the drawings are not necessarily necessary for implementing the present application. Those skilled in the art can understand that the modules in the device in the implementation scenario can be distributed in the device in the implementation scenario according to the description of the implementation scenario, or can be changed to be located in one or more devices different from the implementation scenario. The modules of the above implementation scenario can be combined into one module, or can be further split into a plurality of sub-modules.

[0080] The above application number is only for description, not representing the advantages and disadvantages of the implementation scenario. The above disclosure is only a few specific implementation scenarios of the present application, but the present application is not limited thereto, and any changes that can be thought of by those skilled in the art should fall within the protection scope of the present application.

Claims

1. A method for measuring the range of motion of a joint, characterized in that, The method includes: Obtain joint labels of the joints to be measured of the target person and multiple body part labels corresponding to the joint labels, wherein each body part label corresponds to a body part, the joint serves as the common endpoint of an angle, and each body part and the joint form a line of angle, so that the joint and multiple body parts together form an angle, so as to determine the degree of motion of the joint by the angle. The muscle electrical signal parameters are acquired in real time by an electrical signal acquisition device set at a preset position corresponding to the joint, and the muscle electrical signal parameters are input into a pre-trained range of motion determination model to obtain a judgment value on whether the joint has reached the maximum range of motion. When the judgment value is that the joint has reached its maximum range of motion, image information of the target person is acquired, wherein the image information is a single frame image, and the image information includes an image of the joint and an image of the human body part corresponding to each human body part label; The image information, the joint labels, and the human body part labels are input into a pre-trained convolutional neural network model to obtain the joint positions of the joints and the human body part positions in the image information. Based on the joint position and the human body part position, the joint's range of motion is determined, and the range of motion is defined as the joint's maximum range of motion.

2. The method according to claim 1, characterized in that, The step of inputting the image information, the joint labels, and the human body part labels into a pre-trained convolutional neural network model to obtain the joint positions and human body part positions in the image information includes: The joint and the human body part to be identified are determined based on the joint label and the human body part label; The image information is input into the convolutional neural network model, which outputs the joint key point heatmap of the joint and the part key point heatmap of the human body. The point with the highest probability value in the joint key point heatmap is determined as the joint position, and the point with the highest probability value in the body part key point heatmap is determined as the body part position.

3. The method according to claim 2, characterized in that, The step of determining the point with the highest probability value in the joint key point heatmap as the joint position, and determining the point with the highest probability value in the body part key point heatmap as the body part position, includes: Obtain the probability value of the point with the highest probability value in the heat map of the joint key points and the probability value of the point with the highest probability value in the heat map of the part key points; Determine whether the probability value of the point with the highest probability value in the joint key point heatmap and the probability value of the point with the highest probability value in the part key point heatmap are greater than or equal to a preset probability threshold. If the probability value of the point with the highest probability value in the joint key point heatmap and the probability value of the point with the highest probability value in the body part key point heatmap are both greater than or equal to the probability threshold, then the point with the highest probability value in the joint key point heatmap is determined as the joint position, and the point with the highest probability value in the body part key point heatmap is determined as the body part position.

4. The method according to claim 2, characterized in that, The step of determining the point with the highest probability value in the joint key point heatmap as the joint position, and determining the point with the highest probability value in the body part key point heatmap as the body part position, further includes: Establish a reference coordinate system for the image information; Obtain the joint coordinate value of the point with the highest probability value in the joint key point heatmap in the reference coordinate system, and determine the joint position based on the joint coordinate value; The coordinates of the point with the highest probability value in the heat map of the key points of the part are obtained in the reference coordinate system, and the location of the part is determined based on the coordinates of the part.

5. The method according to any one of claims 2-4, characterized in that, The step of inputting the image information into the convolutional neural network model and outputting joint key point heatmaps of the joints and part key point heatmaps of the human body parts includes: The image information is convolved with multiple convolution kernels of preset sizes to obtain a base image corresponding to each convolution kernel, wherein the size of each convolution kernel is different; The base image with the largest feature size among the multiple base images is determined as the reference base image; The base images other than the reference base image are filled to make the feature size of each base image equal to the feature size of the reference base image, and all the base images are stitched together to obtain a multidimensional image; The multidimensional image is classified and identified to obtain the joint key point heatmap and the part key point heatmap.

6. The method according to claim 5, characterized in that, The step of acquiring muscle electrical signal parameters in real time through an electrical signal acquisition device positioned at a preset location corresponding to the joint includes: The electrical signal waveform of the muscle at the preset position is acquired based on the electrical signal acquisition device; The electrical signal waveform is denoised using a filter, and the electrical signal parameters of the denoised electrical signal waveform are extracted. The electrical signal parameters were determined to be muscle electrical signal parameters.

7. The method according to claim 1, characterized in that, Before acquiring muscle electrical signal parameters in real time through an electrical signal acquisition device set at a preset position corresponding to the joint, and inputting the muscle electrical signal parameters into a pre-trained range of motion determination model to obtain a judgment value on whether the joint has reached its maximum range of motion, the method further includes: Receive the input value from the target person indicating whether the joint has reached its maximum range of motion; Before acquiring image information of the target person when the judgment value is that the joint has reached its maximum range of motion, the method further includes: If the input value is that the joint has reached its maximum range of motion, and the judgment value is that the joint has not reached its maximum range of motion, a prompt message will be issued.

8. A joint range of motion measuring device, characterized in that, The device includes: The tag acquisition module is used to acquire joint tags of the joints to be measured of the target person and multiple human body part tags corresponding to the joint tags. Each human body part tag corresponds to a human body part. The joint serves as the common endpoint of an angle. Each human body part and the joint form a line of angle, so that the joint and multiple human body parts together form an angle, so that the degree of motion of the joint can be determined by the angle of the angle. The judgment execution module is used to acquire muscle electrical signal parameters in real time through an electrical signal acquisition device set at a preset position corresponding to the joint, and input the muscle electrical signal parameters into a pre-trained range of motion determination model to obtain a judgment value on whether the joint has reached the maximum range of motion. The image acquisition module is used to acquire image information of the target person when the judgment value is that the joint has reached its maximum range of motion. The image information is a frame image and includes an image of the joint and an image of the human body part corresponding to each human body part label. An image recognition module is used to input the image information, the joint labels, and the human body part labels into a pre-trained convolutional neural network model to obtain the joint position of the joint and the human body part position in the image information. The result determination module is used to determine the joint's range of motion based on the joint position and the human body part position, and to confirm the range of motion as the joint's maximum range of motion.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images