Sitting forward bend assessment method, system, electronic device, and storage medium

Abstract

The present application relates to the field of computer vision, and provide a kind of sitting forward bending evaluation method, system, electronic equipment and storage medium, wherein the method comprises: obtaining the video stream to be measured;Each frame image in the video stream to be measured is detected by human skeleton point and tester, and the skeleton point position information and tester position information of each frame image are obtained;Based on the skeleton point position information and tester position information of each frame image, the action attribute classification is carried out to each frame image, and the action attribute of each frame image is obtained;Based on the action attribute of each frame image, sitting forward bending evaluation is carried out.The sitting forward bending evaluation method, system, electronic equipment and storage medium provided by the present application can simply and effectively learn the action relationship between skeleton points compared with the action attribute classification method based on image, can reduce the amount of calculation and improve the running efficiency;At the same time, the device is simple, and convenient to use.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method, system, electronic device, and storage medium for assessing sit-and-reach posture. Background Technology

[0002] The sit-and-reach test, a physical fitness test item for primary, secondary, and tertiary schools, aims to measure the range of motion of the trunk, waist, hip, and other joints in a static state. It mainly reflects the extensibility and elasticity of the joints, ligaments, and muscles in these areas, as well as the development level of physical flexibility.

[0003] Traditional sit-and-reach test methods require manual viewing and recording of scores on the test instrument's scale, resulting in low efficiency. Solutions based on ultrasonic and capacitive sensors can automatically acquire test scores, but violations and cheating still require manual evaluation and cannot be directly adapted to traditional test instruments. Some computer vision-based solutions focus only on score measurement, while others are inconvenient to use. Summary of the Invention

[0004] This invention provides a method, system, electronic device, and storage medium for assessing sit-and-reach posture, in order to address the shortcomings of existing sit-and-reach testing methods that suffer from low efficiency and inconvenience due to manual assessment.

[0005] This invention provides a method for assessing sit-and-reach flexibility, comprising:

[0006] Obtain the video stream to be tested;

[0007] Human skeleton points and testing instrument detection are performed on each frame of the video stream under test to obtain the skeleton point position information and testing instrument position information of each frame.

[0008] Based on the skeletal point position information and the tester position information of each frame image, the motion attributes of each frame image are classified to obtain the motion attributes of each frame image.

[0009] Based on the motion attributes of each frame of the image, a sit-and-reach assessment is performed.

[0010] According to the sit-and-reach assessment method provided by the present invention, the step of classifying the motion attributes of each frame of images based on the skeletal point position information and the tester position information to obtain the motion attributes of each frame of images includes:

[0011] The skeletal point position information and the tester position information of any frame image are normalized respectively to obtain the normalized skeletal point position information and the normalized tester position information.

[0012] Based on the normalized position information of the skeleton points and the normalized position information of the tester, the motion attributes of any frame image are classified to obtain the motion attributes of any frame image.

[0013] According to the sit-and-reach assessment method provided by the present invention, the normalization processing of the skeletal point position information and the test instrument position information of any frame image to obtain normalized skeletal point position information and normalized test instrument position information includes:

[0014] Based on the hip and knee position information in the skeletal point position information of any frame image, determine the distance between the hip and knee, as well as the position information of the hip center point;

[0015] Based on the distance and the hip center point location information, the skeletal point location information and the tester location information of any frame image are normalized to obtain the normalized skeletal point location information and the normalized tester location information.

[0016] According to the sit-and-reach assessment method provided by the present invention, the step of performing sit-and-reach assessment based on the motion attributes of each frame image includes:

[0017] Based on the motion attributes of each frame image, determine the pedal frame image from each frame image;

[0018] Based on the difference between the knee and ankle position information in the skeletal point position information of any subsequent frame image and the knee and ankle position information in the skeletal point position information of the foot pedal frame image, a seated forward bend foot pedal cheating evaluation is performed on any frame image.

[0019] According to the sit-and-reach assessment method provided by the present invention, the step of performing a sit-and-reach pedal cheating assessment on any frame image based on the difference between the knee and ankle position information in the skeletal point position information of any subsequent frame image and the knee and ankle position information in the skeletal point position information of the pedal frame image includes:

[0020] The difference between the knee and ankle position information is normalized to obtain normalized difference information;

[0021] Based on the normalized difference information, a sit-and-reach pedal cheating evaluation is performed on any frame of the image.

[0022] According to the sit-and-reach assessment method provided by the present invention, the step of performing sit-and-reach assessment based on the motion attributes of each frame image includes:

[0023] Based on the motion attributes of each frame image, as well as the skeletal point position information and the tester position information of each frame image, the state of each frame image is determined.

[0024] Based on the state of each frame of the image, the sit-and-reach state is switched and the performance is measured.

[0025] According to the sit-and-reach assessment method provided by the present invention, the step of switching sit-and-reach states and determining the score based on the state of each frame of the image includes:

[0026] Select any frame from the frames that are in a stopped state;

[0027] Perform tester detection and image segmentation on any frame image to obtain the tester block and vernier block in any frame image;

[0028] The scale point clusters are obtained by detecting the scale points on the tester block.

[0029] Based on the vernier plot and the scale point cluster, the sit-and-reach test score is measured.

[0030] The present invention also provides a sit-and-reach assessment device, comprising:

[0031] The acquisition unit is used to acquire the video stream to be tested;

[0032] The detection unit is used to detect human skeleton points and the testing instrument in each frame of the video stream under test, and obtain the skeleton point position information and the testing instrument position information of each frame.

[0033] The classification unit is used to classify the action attributes of each frame image based on the skeletal point position information and the tester position information of each frame image, so as to obtain the action attributes of each frame image.

[0034] The evaluation unit is used to evaluate the sit-and-reach posture based on the motion attributes of each frame of image.

[0035] The present invention also provides a sit-and-reach assessment system, comprising: a testing instrument, an image acquisition device, and a processor;

[0036] The testing device is used to perform sit-and-reach tests;

[0037] The image acquisition device is used to acquire the video stream to be tested in the sit-and-reach test area and transmit the video stream to be tested to the processor.

[0038] The processor is configured to perform human skeleton point and testing instrument detection on each frame of the video stream under test, obtain skeleton point position information and testing instrument position information of each frame, classify the motion attributes of each frame based on the skeleton point position information and testing instrument position information of each frame, obtain the motion attributes of each frame, and perform sit-and-reach assessment based on the motion attributes of each frame.

[0039] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the sit-and-reach assessment methods described above.

[0040] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the sit-and-reach assessment method as described above.

[0041] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the sit-and-reach assessment method as described above.

[0042] The sit-and-reach assessment method, system, electronic device, and storage medium provided by this invention classify the motion attributes of each frame of images based on the skeletal point position information and the tester position information, and then perform sit-and-reach assessment based on the motion attributes of each frame of images. Compared with image-based motion attribute classification methods, this method can simply and effectively learn the motion relationships between skeletal points, reducing computational load and improving operational efficiency; at the same time, the equipment is simple and easy to use. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0044] Figure 1 This is one of the flowcharts of the sit-and-reach assessment method provided by the present invention;

[0045] Figure 2 This is a schematic diagram of a sit-and-reach measurement instrument;

[0046] Figure 3 A schematic diagram of human skeletal points;

[0047] Figure 4 This is the second flowchart of the sit-and-reach assessment method provided by the present invention;

[0048] Figure 5 This is the third flowchart of the sit-and-reach assessment method provided by the present invention;

[0049] Figure 6 This is the fourth flowchart of the sit-and-reach assessment method provided by the present invention;

[0050] Figure 7 This is a schematic diagram of the foot pedal for the person under test provided by the present invention;

[0051] Figure 8 This is a schematic diagram showing the foot of the person being tested leaving the baffle, as provided by the present invention.

[0052] Figure 9 This is the fifth flowchart of the sit-and-reach assessment method provided by the present invention;

[0053] Figure 10 This is a schematic diagram of the seated forward flexion state switching provided by the present invention;

[0054] Figure 11 This is a flowchart illustrating the performance measurement method provided by the present invention;

[0055] Figure 12 This is a schematic diagram of the seated forward flexion assessment device provided by the present invention;

[0056] Figure 13 This is a schematic diagram of the seated forward flexion assessment system provided by the present invention;

[0057] Figure 14 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0059] Existing sit-and-reach testing solutions typically include traditional sit-and-reach testing instruments plus human intervention, sit-and-reach testing instruments with integrated sensors plus human intervention, and sit-and-reach assessment systems based on AI vision.

[0060] (1) Based on the traditional sit-and-reach test instrument + manual method

[0061] Traditional sit-and-reach testing instruments typically consist of only a base, a ruler, and a vernier, making them simple devices. The judge instructs the test-taker to prepare and push the vernier; during this process, the judge checks for any violations. After the test, the judge manually checks and records the final score, which is prone to human error and unfairness. Current technology primarily focuses on improving the instrument, such as adding straps and curved clips to help flatten the knee after extension and prevent leg bending; the instrument is also foldable for greater convenience.

[0062] This approach is labor-intensive, requiring additional time to manually read and record the values ​​on the calipers, which is time-consuming and prone to misreading. Secondly, the judgment of violations is highly subjective, and the standards are biased and inconsistent.

[0063] (2) A solution based on an integrated sensor-based sit-and-reach testing instrument plus manual intervention.

[0064] Semi-intelligent sit-and-reach testing instruments integrating sensors include distance sensors in addition to traditional testing instruments. The judge notifies the test taker to prepare and push the cursor. During the cursor pushing process, the judge checks for any violations. After the cursor is pushed, the distance sensor is used to obtain and record the test score. Furthermore, some technical solutions install multiple pressure sensors in the instrument to determine whether the test taker has pushed off the board or straightened their legs, further avoiding human error and unfairness.

[0065] While solutions incorporating distance sensors can automatically acquire test results, manual evaluation is still required for violations and cheating actions such as leg bending and foot pedaling. Solutions that add pressure sensors to the distance sensor base will firstly increase the weight of the equipment; secondly, the criteria for judging leg bending will be inconsistent for test subjects of different heights when the pressure sensor for judging leg bending is not adjustable, and even when the pressure sensor for judging leg bending is adjustable, the added step of manual adjustment is still possible, which could lead to improper adjustment.

[0066] (3) Sit-and-reach assessment scheme based on artificial intelligence (AI) vision

[0067] AI visual evaluation systems typically use RGB-D depth cameras or RGB color cameras to capture video and analyze the video of the test subject's movements. For each frame of the captured video, image processing is performed to determine whether the test subject has pushed off the board or straightened their legs during the preparation phase, whether they violated any rules during the cursor-pushing process, and to determine the end frame of the cursor-pushing and output the test score.

[0068] The solution may only focus on performance measurement and cannot handle violations that occur during the test; or it may require the deployment of additional cameras to detect violations, meaning that violations and performance measurement cannot reuse a single camera; or it may require camera calibration to obtain camera intrinsic parameters, adding extra processing steps and making it inconvenient to use.

[0069] Based on the above considerations, in order to improve the efficiency and convenience of sit-and-reach assessment, the inventive concept of this invention is as follows: For each frame of the video stream to be tested, human skeletal points and testing instruments are detected separately, and based on the detected skeletal point position information and testing instrument position information, the motion attributes of each frame of the image are classified to obtain the motion attributes of each frame of the image. Based on the motion attributes of each frame of the video stream to be tested, sit-and-reach assessment is performed.

[0070] Based on the above-mentioned inventive concept, the present invention provides a method, system, electronic device and storage medium for sit-and-reach assessment, which are applied to sit-and-reach assessment scenarios in artificial intelligence technology to improve the efficiency and convenience of sit-and-reach assessment.

[0071] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings. Figure 1 This is one of the flowcharts illustrating the sit-and-reach assessment method provided by the present invention. The execution entity for each step in this method can be a processor within the sit-and-reach assessment system. This processor can be integrated into an electronic device, which can be a terminal device (such as a smartphone, personal computer, etc.) or a server (such as a local server, cloud server, or server cluster, etc.). Figure 1 As shown, the method may include the following steps:

[0072] Step 110: Obtain the video stream to be tested.

[0073] Specifically, the video stream to be tested is the video stream for which a sit-and-reach assessment needs to be performed, which can be obtained by real-time shooting and recording via a camera.

[0074] In this embodiment of the invention, the sit-and-reach assessment process can utilize a traditional sit-and-reach measurement instrument, eliminating the need for integrated sensors and offering strong adaptability. The diagram shows a sit-and-reach measurement instrument; the testing area may include, for example... Figure 2 The system shown is a traditional sit-and-reach measurement device and an image acquisition device. After the person being tested enters the designated testing area, the system will prompt the person to complete the preparation actions and start the test, acquiring and processing the video stream in real time.

[0075] The person being tested can face the image acquisition device from the side and perform the sit-and-reach test by pushing the vernier in the evaluation area. Depending on the orientation of the testing device, the person being tested can face right or left. This embodiment of the invention does not make a specific limitation on this.

[0076] Step 120: Perform human skeleton point and test instrument detection on each frame of the video stream to be tested to obtain the skeleton point position information and test instrument position information of each frame.

[0077] Specifically, once the person to be tested enters the testing area, they can be identified using a combination of human detection and skeletal point tracking. The video stream to be tested can include multiple frames of images. Human detection can be performed every 25 frames, and at other times, human detection is not performed. Instead, skeletal point tracking is used to identify the person to be tested, avoiding interference from people during the evaluation process that could affect the identification of the person to be tested.

[0078] After identifying the person to be tested, human skeleton points and testing equipment can be detected for each frame of the video stream to be tested. This yields the positional information of the human skeleton points and the positional information of the testing equipment contained in each frame, i.e., the positional information of the skeleton points and the positional information of the testing equipment in each frame.

[0079] Here, for the detection of skeletal point position information in any frame of an image, human detection can be performed on the frame to locate the human bounding box, and then image cropping can be performed based on the human bounding box. Human skeletal point detection can then be performed on the cropped human image to obtain the position information of key human skeletal points in the frame. Alternatively, image features can be extracted from the frame, and the extracted image features can be applied simultaneously for human detection and human skeletal point detection, thereby simultaneously obtaining the human bounding box and the position information of key skeletal points of the person in the frame. This embodiment of the invention does not specifically limit this approach. Here, key human skeletal points may include hand skeletal points and foot skeletal points, etc.

[0080] For example, the position information of the skeleton points in each frame of the image can be obtained through a skeleton point tracking algorithm. Figure 3 A schematic diagram of human skeletal points, such as Figure 3 As shown, the input to the skeleton point tracking algorithm can be the previous frame image with skeleton points and the current frame image. The output can be the coordinates of 30 key skeleton points in the current frame image, as well as information on whether the 30 key skeleton points are visible.

[0081] The tester position information can specifically include the position information of the tester, the base, and the vernier. For any frame of an image, the tester position information can be obtained by performing tester detection on that frame of the image to locate the bounding box of the tester, the base, and the vernier in that frame of the image, and then the tester position information can be determined by the coordinate points of the bounding box.

[0082] Step 130: Based on the skeletal point position information and the tester position information of each frame image, classify the action attributes of each frame image to obtain the action attributes of each frame image.

[0083] Specifically, action attribute classification refers to outputting the current action attributes of the test subject in any given frame of image. The action attributes represent the current state of the test subject, which may include the orientation during the assessment (facing left or right), whether the legs are straight, whether the feet are on the board, and whether the hands touch the cursor (single hand, two hands, no touch), etc.

[0084] In related technologies, image-based action classification models are commonly used, with images as the input to the model. However, image-based classification networks have a large number of parameters and high computational cost, while image-based convolutional networks have a high computational cost.

[0085] To further improve the efficiency of action attribute classification and reduce computational load, this embodiment classifies action attributes for each frame of image based on the skeletal point position information and the testing instrument position information. Human actions can be constructed using the skeletal point position information, and instrument position information can be constructed using the testing instrument position information. The positional relationship between the person and the instrument can be constructed using both skeletal point position information and testing instrument position information. Compared to image-based methods, this approach can learn the action relationships between skeletal points simply and effectively, and it also has higher operational efficiency.

[0086] In some embodiments, for the sit-and-reach assessment scenario, the specific skeletal point location information may include the location information of 26 skeletal points, including the top of the head, nose, left ear, right ear, chin, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left palm, right palm, left middle finger, right middle finger, left hip, right hip, left knee, right knee, left ankle, right ankle, left eye, right eye, left thumb, and right thumb. This information can provide information about human movement. Among these, the hip, knee, and ankle are the most important skeletal points, while other skeletal points participate in the data augmentation part to increase the generalization of the model.

[0087] The tester's position information may include the upper left and lower right coordinates of the tester's base frame, providing the base plate's position information to determine whether a foot pedal is present, and the upper left and lower right coordinates of the vernier frame, providing the vernier's position information to determine if a hand touches the vernier.

[0088] For the classification of action attributes of any frame image, the skeletal point position information and the tester position information of the frame image can be input into a pre-constructed classification network. The classification network performs action attribute classification and outputs the action attributes of the frame image. The classification network here can be a network containing fully connected layers, such as a multilayer perceptron (MLP), VGGNet, ResNet, etc., and the embodiments of the present invention do not specifically limit it.

[0089] Step 140: Based on the motion attributes of each frame image, perform a sit-and-reach assessment.

[0090] Specifically, the sit-and-reach assessment can be conducted by tracking and evaluating each step of the assessment process until the assessment is completed and a score is obtained. Since the motion attributes of each frame of the image can characterize the current state of the test subject, the sit-and-reach assessment can be performed based on the motion attributes of each frame of the image.

[0091] For example, after the person to be tested enters the assessment area, they can be reminded to prepare for the sit-and-reach test, and the system can be checked to ensure they are in the ready position, with their legs straight and feet on the footrest. Once in the ready position, the person can be reminded to push the cursor, and any violations can be detected, such as whether their legs are bent, their feet are off the footrest, or they are pushing the cursor with only one hand. Furthermore, if any violations are detected, a voice warning can be issued, and in serious cases, the assessment can be stopped immediately. After the cursor pushing is completed, the test results can be measured to complete the sit-and-reach assessment.

[0092] The method provided in this invention classifies the motion attributes of each frame of image based on the skeletal point position information and the tester position information, and then performs a sit-and-reach assessment based on the motion attributes of each frame. Compared with image-based motion attribute classification methods, this method can simply and effectively learn the motion relationships between skeletal points, reducing computational load and improving operational efficiency; at the same time, the equipment is simple and easy to use.

[0093] Based on the above embodiments, Figure 4 This is the second flowchart of the sit-and-reach assessment method provided by the present invention, as shown below. Figure 4 As shown, based on the skeletal point position information and the tester position information of each frame image, the motion attributes of each frame image are classified to obtain the motion attributes of each frame image. Specifically, step 130 includes:

[0094] Step 131: Normalize the bone point position information and the tester position information of any frame image to obtain normalized bone point position information and normalized tester position information.

[0095] Step 132: Based on the normalized position information of the skeleton points and the normalized position information of the tester, classify the action attributes of the frame image to obtain the action attributes of the frame image.

[0096] Specifically, considering that the skeletal point position information and the tester position information will change with the distance from the image acquisition device, and that the skeletal point position information and the tester position information are used as inputs for action attribute classification, in order to further improve the accuracy of action attribute classification of the classification network, before classifying the action attributes of the frame image, the skeletal point position information and the tester position information of the frame image can be normalized respectively to obtain normalized skeletal point position information and normalized tester position information.

[0097] For the normalization of location information, this can be achieved by first selecting a reference distance and a reference point, and then transforming the location information to the same dimension based on the reference distance and the reference point. Here, the reference distance can be a distance value that varies with the shooting distance, such as the thigh length or arm length of the person being tested. The reference point can be the center point of the hip of the person being tested, or the center point of the base of the testing instrument, etc., and this embodiment of the invention does not specifically limit this.

[0098] It should be noted that the reference distance and reference point selected for normalizing the skeletal point position information are the same as those selected for normalizing the tester position information.

[0099] Subsequently, after obtaining the normalized position information of the skeletal points and the normalized position information of the testing instrument, the motion attributes of the frame image can be directly classified based on the normalized position information of the skeletal points and the normalized position information of the testing instrument to obtain the motion attributes of the frame image; alternatively, the normalized position information of the skeletal points and the normalized position information of the testing instrument can be stitched together and the coordinates straightened, and the motion attributes of the frame image can be classified based on the stitched and coordinate straightened information to obtain the motion attributes of the frame image.

[0100] The method provided in this invention normalizes the skeletal point location information and the tester location information respectively. The normalized skeletal point location information and the normalized tester location information obtained after normalization are unified under the same dimension, thereby improving the accuracy of action attribute classification of the classification network.

[0101] Based on any of the above embodiments Figure 5 This is the third flowchart of the sit-and-reach assessment method provided by the present invention, as shown below. Figure 5 As shown, the skeletal point position information and the tester position information of any frame image are normalized to obtain the normalized skeletal point position information and the normalized tester position information. Specifically, step 131 includes:

[0102] Step 131-1: Based on the hip and knee position information in the skeletal point position information of any frame image, determine the distance between the hip and knee, as well as the position information of the hip center point;

[0103] Step 131-2: Based on the distance and hip center point location information, normalize the bone point location information and the tester location information of the frame image to obtain normalized bone point location information and normalized tester location information.

[0104] Specifically, in order to further improve the accuracy of action attribute classification, for the normalization processing of skeletal point position information and tester position information, the reference distance can be selected as the thigh pixel length that changes with the shooting distance, that is, the distance between the hip bone point and the knee bone point, and the reference point can be selected as the center point of the hip at the center of the human body.

[0105] The distance between the hip and knee, i.e., the distance between the hip bone point and the knee bone point, can be obtained by averaging the distance between the center point of the hip and the center point of the knee. The hip center point location information can be obtained by averaging the coordinates of the left and right hip bone points in the bone point location information, and the knee center point location information can be obtained by averaging the coordinates of the left and right knee bone points in the bone point location information.

[0106] After obtaining the distance between the hip and knee, and the position information of the hip center point, the distance between the hip and knee can be used as the reference distance, and the hip center point as the reference point. The position information of the skeletal points and the position information of the tester in this frame image are normalized respectively to obtain the normalized position information of the skeletal points and the normalized position information of the tester.

[0107] In some embodiments, taking the location information of any bone point as an example, the location information of the bone point can be normalized using the following formula:

[0108]

[0109] In the formula, joints refers to the position information of the skeletal points. norm The normalized positional information of the skeleton points; factor refers to the scaling factor; hip. center Information on the center point location of the hip, leg len The length of the thigh in pixels is the distance between the hip and the knee.

[0110]

[0111] In the formula, hip left This refers to the location information of the bones in the left hip. right This is the location information of the right hip bone points.

[0112]

[0113] In the formula, knee left Information on the location of the bone points on the left kneecap, knee right For the location information of the right kneecap, knee center This is the location information of the center point of the knee.

[0114] leg len =Euclidean(hip) center,knee center )

[0115] In the formula, Euclidean(hip) center ,knee center The distance between the center point of the hip and the center point of the knee is represented by .

[0116] Understandably, the same method can be used to normalize the tester's position information to obtain normalized position information. This position information can include the upper left and lower right coordinates of the base frame and the upper left and lower right coordinates of the cursor frame.

[0117] Based on this, taking the position information of 26 bone points and the position information of 4 testing instruments as an example, the 26*2 coordinate point sequence ((x0,y0),(x1,y1),…) composed of the normalized position information of the 26 bone points and the 4*2 coordinate point sequence composed of the normalized position information of the 4 testing instruments are straightened into a 60*1 vector, which can be represented as (x0,y0,x1,y1,x2,y2,…).

[0118] For action attribute classification, this 60-dimensional vector can be input into a multilayer perceptron network.

[0119] Prior to this, a multilayer perceptron network can be constructed: 1) input a 60-dimensional vector, 2) balancing model performance and computational cost, with intermediate hidden layers of 128, 256, and 128 dimensions respectively, and 3) output a 6-dimensional vector.

[0120] The first three values ​​of the output vector of the multilayer perceptron network are subjected to sigmoid operation to obtain normalized values ​​from 0 to 1, as shown in the following formula. Each value corresponds to a threshold. If the value is greater than the threshold, the output is 1, and if it is less than or equal to the threshold, the output is 0. The last three values ​​of the output vector are taken as a vector, and the index of the largest value in the vector is taken.

[0121]

[0122] The four outputs obtained by the multilayer perceptron network represent: left and right (1 for right, 0 for left), leg straight (1 for yes, 0 for no), foot pedal (1 for yes, 0 for no), and cursor touch (0 for no cursor touch, 1 for single-handed cursor touch, 2 for double-handed cursor touch).

[0123] Based on any of the above embodiments Figure 6 This is the fourth flowchart of the sit-and-reach assessment method provided by the present invention. Based on the motion attributes of each frame image, sit-and-reach assessment is performed, specifically step 140 includes:

[0124] Step 141: Based on the motion attributes of each frame image, determine the pedal board frame image from each frame image;

[0125] Step 142: Based on the difference between the knee and ankle position information in the skeletal point position information of any frame image after the pedal frame image and the knee and ankle position information in the skeletal point position information of the pedal frame image, perform a sit-and-reach pedal cheating evaluation on the frame image.

[0126] Specifically, considering that in the sit-and-reach scenario, the test subject may engage in cheating behavior by leaving the foot off the board while pushing the cursor, in order to further automatically evaluate the cheating behavior of leaving the foot off the board, the sit-and-reach foot pedal cheating evaluation can be performed based on the motion attributes of each frame image.

[0127] Figure 7 This is a schematic diagram of the footrest provided by the present invention for the person under test. Figure 8 This is a schematic diagram of a test subject's foot leaving the barrier, provided by the present invention. To evaluate whether the foot has left the barrier, the foot pedal frame image can first be determined from each frame image based on the motion attributes of each frame image. The foot pedal frame image is the image frame where the foot pedal output is 1 in the motion attributes.

[0128] Once the pedal board frame image is determined, pedal board cheating evaluation can be performed on any frame image after the pedal board frame image.

[0129] Since the cheating behavior of the foot pedal comes from leg movement, we can start with the position information of the knee and ankle in the skeletal point position information. Specifically, we can select four skeletal points: left knee, right knee, left ankle, and right ankle. In order to highlight the information of the foot pedal and the moment when the foot is not on the pedal, this embodiment uses the difference between the knee and ankle position information in the frame image and the knee and ankle position information in the foot pedal frame image as input to evaluate the seated forward flexion foot pedal cheating behavior of the frame image.

[0130] Understandably, the greater the difference in knee and ankle positional information, the greater the likelihood of pedalboard cheating; conversely, the smaller the difference in knee and ankle positional information, the less likely pedalboard cheating is to occur.

[0131] For detecting pedalboard cheating, a classification network can be used. The positional differences between the knee and ankle are input into a pre-trained classification network, which then performs pedalboard cheating detection and outputs the results.

[0132] For example, a classification network can be a multilayer perceptron network: 1) input is an 8-dimensional vector, 2) balancing model performance and computational cost, the intermediate hidden layers are 64-dimensional and 128-dimensional respectively, and 3) the output layer is a 2-dimensional vector. The index of the largest value in the multilayer perceptron network's output vector indicates whether cheating exists: 0 represents no cheating, and 1 represents cheating.

[0133] The method provided in this embodiment of the invention uses the difference between the knee and ankle position information in the skeletal point position information of the frame image and the knee and ankle position information in the skeletal point position information of the pedal frame image to perform a sit-and-reach pedal cheating evaluation on the frame image, thereby enabling the sit-and-reach pedal cheating evaluation.

[0134] Based on any of the above embodiments, based on the difference between the knee and ankle position information in the skeletal point position information of any subsequent frame image and the knee and ankle position information in the skeletal point position information of the foot pedal frame image, a sit-and-reach foot pedal cheating evaluation is performed on any frame image. Specifically, step 142 includes:

[0135] Step 142-1: Normalize the difference between the position information of the knee and the ankle to obtain normalized difference information;

[0136] Step 142-2: Based on the normalized difference information, perform a sit-and-reach pedal cheating evaluation on the frame image.

[0137] Specifically, to further improve the accuracy of the seated forward bend foot pedal cheating assessment, after obtaining the difference between the knee and ankle position information, it can be normalized to obtain normalized difference information. This normalization can be based on a reference distance, which can be the thigh length of the person being tested, i.e., the Euclidean distance between the center point of the hip and the center point of the knee.

[0138] After obtaining the normalized difference information, the image frame can be used to evaluate seated forward bend foot pedal cheating based on the normalized difference information.

[0139] In some embodiments, taking four skeletal points—left knee, right knee, left ankle, and right ankle—as examples, the process of evaluating seated forward flexion footboard cheating on this frame of image can be represented as follows:

[0140] 1) Take the left and right knee and left and right ankle bone points of the frame image and the left and right knee and left and right ankle bone points of the footboard frame image, and then subtract the x and y coordinates of the corresponding bone points to obtain 4 sets of coordinate differences:

[0141]

[0142] In the formula, The coordinate difference between any bone point in the left and right knees and the left and right ankles; These are the coordinates of the skeletal points in this frame of the image. Let i be the coordinates of the skeletal points in the pedal frame image, i∈(knee) left ,knee right ,ankle left ,ankle right )

[0143] 2) Normalize the coordinate differences of the four bone points by selecting the thigh length:

[0144]

[0145] (x diff ,y diff ) norm This refers to the difference in coordinates of the normalized skeletal points, i.e., the normalized difference information; (x diff ,y diff This refers to the difference in the original skeletal point coordinates, that is, the difference between the positional information of the knee and ankle; leg len The thigh length is the Euclidean distance between the center point of the hip and the center point of the knee.

[0146] leg len =Euclidean(hip) center ,knee center )

[0147] hip center The center point of the hip is obtained by averaging the coordinates of the left and right hip points:

[0148]

[0149] knee center The center point of the knee is obtained by averaging the coordinates of the left and right knees:

[0150]

[0151] 3) Straighten the coordinate difference of skeletal points

[0152] Normalized bone point coordinate difference sequence Straighten into an 8*1 vector

[0153] 4) Cheating classification network

[0154] The vector after straightening the coordinate difference of the skeleton points is input into the cheating classification network. The index of the largest value in the output vector represents whether cheating exists: 0 represents no cheating and 1 represents cheating.

[0155] The method provided in this embodiment of the invention normalizes the difference between the position information of the knee and the ankle to obtain normalized difference information. Based on the normalized difference information, the image frame is used to perform a sit-and-reach pedal cheating assessment, which can further improve the accuracy of pedal cheating assessment.

[0156] Based on any of the above embodiments Figure 9 This is the fifth flowchart of the sit-and-reach assessment method provided by the present invention, as shown below. Figure 9 As shown, based on the motion attributes of each frame image, the sit-and-reach assessment is performed, specifically step 140 includes:

[0157] Step 143: Based on the motion attributes of each frame image, as well as the skeletal point position information and the tester position information of each frame image, determine the state of each frame image.

[0158] Step 144: Based on the state of each frame image, switch the seated forward bend state and determine the score.

[0159] Specifically, for the sit-and-reach assessment scenario, this embodiment of the invention sets five states to drive the assessment process. The state of each frame image can include one of the following five states: start state, ready state, cursor touch state, moving state (by default, the cursor touch state directly switches to the moving state), and stopped moving state. In addition, a pre-processing procedure is included to activate the state machine for testing.

[0160] The state of each frame can be determined by a state machine based on the motion attributes of each frame, as well as the skeletal point position information and the tester position information of each frame.

[0161] State machine preprocessing: Human body detection and skeletal point tracking obtain the skeletal point coordinates of the person to be tested, and instrument detection obtains the instrument, base, and cursor detection frame, thereby activating the state machine; then, the attribute classification model is used to obtain the action attributes; next, the skeletal point coordinates, instrument coordinates, and action attributes are sent to the sit-and-reach state machine for state judgment and switching.

[0162] Start State: After the state machine is successfully activated, it enters the start state. In the start state, it continuously checks whether the subject's posture is standard. If the posture is met, it switches to the ready state. The standard conditions are: 1) The subject faces the instrument and sits on the mat (ankle to the left of the hip and the instrument to the left of the subject; otherwise, ankle to the right of the hip and the instrument to the right of the subject; shoulder, hip, and ankle are approximately right angles; shoulder and hip are approximately perpendicular to the ground, with the perpendicular line to the ground less than a threshold); 2) Both legs are extended forward (in practice, the leg angle formed by the hip, knee, and ankle is greater than 140°); 3) Both feet are placed on the tester's baffle (ankle, heel, and toes define the foot frame, with the distance from the instrument base frame less than 100 pixels); if any of the three conditions are not met, it remains in the start state.

[0163] Preparation State: First, during the preparation state, the system determines whether the legs are bent and whether the feet are on the board based on the action attribute results. If the legs are not bent, the feet are on the board, and the board count exceeds 10 frames, the subject's preparation action is considered satisfactory, and the cursor can be pushed. Then, the system cyclically checks whether the conditions for switching to the cursor-touching state are met: 1) The cursor is at the starting position, and the speed is 0 (calculate the average x-coordinate of the cursor frame closest to the subject in the current frame and the previous 9 frames, calculate the difference, and then take the average to obtain a relative speed; it is not determined whether the cursor frame is at the starting position); 2) The left and right fingertips touch the cursor (the positional relationship between the cursor frame in the current frame and the key points of the two fingertips, with a distance set to 15 pixels). If the above conditions are met for 3 consecutive frames, the system switches to the cursor-touching state.

[0164] Touching the cursor: Initially, the cursor is in the default state for 20 frames before switching to the moving state. Then, it continuously checks whether the conditions for switching to the stopped moving state are met: 1) The cursor speed is detected to be 0, and the cursor movement ends (similar to determining that the cursor speed at the starting point is 0, calculating the coordinate differences of the first 8 cursor frames stored in the current frame, and then taking the average to obtain a relative speed); 2) The finger leaves the cursor (the positional relationship between the left and right fingertip key points and the cursor frame in the current frame, with the distance set to 20).

[0165] Stopped Movement State: In the stopped movement state, a frame suitable for measurement results is found, i.e., the subject's hand leaves the instrument frame. If the hand does not leave the instrument area for 10 consecutive frames, an alert is played; if the hand does not leave the instrument area for 100 consecutive frames, an abnormality is displayed (state: 301), and the result measurement is performed; otherwise, when the hand leaves the instrument area, the result measurement is performed in a frame that does not affect the result measurement.

[0166] Figure 10 This is a schematic diagram of the seated forward flexion state switching provided by the present invention, as shown below. Figure 10 As shown, the entire evaluation involves determining the state of each frame of the video under test, thereby advancing the testing process. The state machine transitions are as follows:

[0167] After the assessment begins, it checks whether the person being tested is in the seated area and whether valid human skeletal points have been obtained, thus determining whether the state machine has been successfully entered. If the state machine is successfully entered, it enters the start state, and the person's preparatory actions are compliant. The state machine then switches to the ready state; otherwise, it remains in the start state, continuously checking for the conditions for the ready state. In the ready state, it loops to check if the condition for touching the cursor is met. If the condition is met, it switches to the cursor-touching state and, by default, directly switches to the start-movement state. In the movement state, it loops to check if the condition for stopping movement is met. If the condition is met, it switches to the stop-movement state. During this process, violation detection is performed simultaneously; if a violation occurs, the assessment ends. In the stop-movement state, the assessment results are measured and output, and the assessment ends.

[0168] Detection of cheating violations:

[0169] During the period from the cursor touch state to the stop state, the results of motion attributes are used to detect violations such as leg bending and pushing the cursor with one hand. A violation is determined when more than 3 consecutive frames of violation actions are detected.

[0170] During the period from touching the cursor to stopping, the foot pedal is used to detect cheating by lifting the foot off the base baffle to improve the score. The model classification results are obtained every ten frames, and the test subject is determined by voting to determine whether the test subject has cheated by lifting the foot off the baffle.

[0171] Based on any of the above embodiments, the sitting-forward flexion state switching and performance measurement are performed based on the state of each frame image, including:

[0172] Select any frame from the frames that are in a stopped state;

[0173] Perform instrument detection and image segmentation on any frame of image to obtain the instrument patch and vernier patch in any frame of image;

[0174] The scale points of the tester block are detected to obtain a cluster of scale points.

[0175] The sit-and-reach test score was determined based on vernier plots and scale point clusters.

[0176] Specifically, considering that in the existing technology, the score is usually calculated for each frame and the largest score value is taken at the end, which is inefficient, in order to save computing resources and improve the efficiency of score measurement, the embodiments of the present invention first select any frame image from the frames that are in the stopped moving state, and perform score measurement based on the frame image, that is, the score measurement is only performed once after the cursor stops moving, thereby improving the efficiency of score measurement.

[0177] Figure 11 This is a flowchart illustrating the performance measurement method provided by the present invention, as shown below. Figure 11As shown, the performance measurement scheme relies on three models: instrument detection, scale point detection, and vernier segmentation.

[0178] 1) Obtain the measuring instrument frame and vernier frame using instrument detection; 2) Then, detect the scale points on the instrument image and segment the vernier image; 3) The detected scale points (31 in total) may be offset, have more or fewer points due to vernier occlusion. Use RANSAC and scale point addition / removal to obtain the corresponding 31 scale points; 4) Finally, use the vernier mask and the fitted curve of the scale points to calculate the position of the vernier on the scale. Combine the complete scale point position to calculate the final score. The score accuracy can be accurate to the decimal point.

[0179] The method provided in this invention does not require the camera to be directly focused on the scale of the instrument, but rather on the entire process of the person and the instrument. The detection and segmentation method effectively utilizes the various components of the traditional instrument, without adding any additional markers to the instrument, thus preventing the error caused by marker detection. When determining the final score position, the vernier segmentation provides more accurate boundary information than conventional manual detection, and the score measurement is only performed once when the vernier stops moving, eliminating the need to calculate the score for each frame and take the maximum score value in existing solutions.

[0180] Based on any of the above embodiments, a method for assessing sit-and-reach flexibility is provided, including:

[0181] S1 uses a single RGB camera to acquire the video stream to be tested.

[0182] S2, perform human skeleton point and tester detection on each frame of the video stream to be tested, and obtain the skeleton point position information and tester position information of each frame.

[0183] S3, for any frame in each frame image, based on the hip and knee position information in the bone point position information of that frame image, determine the distance between the hip and knee, as well as the hip center point position information;

[0184] Based on the distance and hip center point location information, the bone point location information and the tester location information of this frame image are normalized respectively to obtain the bone point normalized location information and the tester normalized location information.

[0185] Based on the normalized position information of the skeleton points and the normalized position information of the tester, the action attributes of the frame image are classified to obtain the action attributes of the frame image.

[0186] S4. For any frame image after the pedal frame image, based on the difference between the knee and ankle position information in the skeletal point position information of that frame image and the knee and ankle position information in the skeletal point position information of the pedal frame image, perform a seated forward bend pedal cheating evaluation on that frame image.

[0187] S5 determines the state of each frame image based on the motion attributes of each frame image, as well as the skeletal point position information and the tester position information of each frame image; based on the state of each frame image, it performs the sitting forward flexion state switching and performance measurement.

[0188] S6. For performance measurement, select any frame image from the frames of images in the stopped movement state; perform instrument detection and image segmentation on the frame image to obtain the instrument block and vernier block in the frame image; perform scale point detection on the instrument block to obtain the scale point cluster; and perform sit-and-reach performance measurement based on the vernier block and the scale point cluster.

[0189] In this embodiment of the invention, a classification scheme based on skeletal points and instrument positions is proposed for violation detection, which is more efficient and robust to the viewpoint. At the same time, the detection of foot pedal cheating during the process of pushing the vernier is added.

[0190] A new counting scheme is proposed for performance measurement, which allows cameras to be deployed at relatively far distances, thus reusing violation detection cameras, and the scores can be accurate to decimals. Combined with the system process, it only needs to be performed once.

[0191] Compared to sensor-based solutions, the device in this invention is simpler and can conveniently handle violation detection during the process while measuring performance through vision, thus assisting in judging the standard of the sit-and-reach movement.

[0192] Compared with current visual solutions for performance measurement and violation detection, the embodiments of the present invention only use a single camera, do not require the detection of precise graphic code information for performance measurement, and only perform performance measurement once; in addition, violation and cheating detection based on skeletal point models is more efficient than image-based detection and more robust than point-to-point rule detection.

[0193] Compared with current vision solutions, using skeletal point tracking algorithms can improve the robustness of main target tracking in complex scenes with interfering targets.

[0194] The sit-and-reach assessment device provided by the present invention is described below. The sit-and-reach assessment device described below can be referred to in correspondence with the sit-and-reach assessment method described above.

[0195] Based on the above embodiments, Figure 12 This is a schematic diagram of the seated forward flexion assessment device provided by the present invention, as shown below. Figure 12 As shown, the sit-and-reach assessment device includes an acquisition unit 1210, a detection unit 1220, a classification unit 1230, and an assessment unit 1240, wherein:

[0196] Acquisition unit 1210 is used to acquire the video stream to be tested;

[0197] The detection unit 1220 is used to detect human skeleton points and the testing instrument in each frame of the video stream under test, and obtain the skeleton point position information and the testing instrument position information of each frame.

[0198] The classification unit 1230 is used to classify the action attributes of each frame image based on the skeletal point position information and the tester position information of each frame image, so as to obtain the action attributes of each frame image.

[0199] Evaluation unit 1240 is used to evaluate sit-and-reach based on the motion attributes of each frame image.

[0200] The sit-and-reach assessment device provided in this invention classifies the motion attributes of each frame of image based on the skeletal point position information and the tester position information, and then performs sit-and-reach assessment based on the motion attributes of each frame of image. Compared with image-based motion attribute classification methods, it can simply and effectively learn the motion relationships between skeletal points, reduce computational load and improve operating efficiency; at the same time, the device is simple and easy to use.

[0201] Based on the above embodiments, the classification unit is specifically used for:

[0202] The skeletal point position information and the tester position information of any frame image are normalized respectively to obtain the normalized skeletal point position information and the normalized tester position information.

[0203] Based on the normalized position information of the skeleton points and the normalized position information of the tester, the motion attributes of any frame image are classified to obtain the motion attributes of any frame image.

[0204] Based on the above embodiments, the classification unit is further specifically used for:

[0205] Based on the hip and knee position information in the skeletal point position information of any frame image, determine the distance between the hip and knee, as well as the position information of the hip center point;

[0206] Based on the distance and the hip center point location information, the skeletal point location information and the tester location information of any frame image are normalized to obtain the normalized skeletal point location information and the normalized tester location information.

[0207] Based on the above embodiments, the evaluation unit is specifically used for:

[0208] Based on the motion attributes of each frame image, determine the pedal frame image from each frame image;

[0209] Based on the difference between the knee and ankle position information in the skeletal point position information of any subsequent frame image and the knee and ankle position information in the skeletal point position information of the foot pedal frame image, a seated forward bend foot pedal cheating evaluation is performed on any frame image.

[0210] Based on the above embodiments, the evaluation unit is also specifically used for:

[0211] The difference between the knee and ankle position information is normalized to obtain normalized difference information;

[0212] Based on the normalized difference information, a sit-and-reach pedal cheating evaluation is performed on any frame of the image.

[0213] Based on the above embodiments, the evaluation unit is also specifically used for:

[0214] Based on the motion attributes of each frame image, as well as the skeletal point position information and the tester position information of each frame image, the state of each frame image is determined.

[0215] Based on the state of each frame of the image, the sit-and-reach state is switched and the performance is measured.

[0216] Based on the above embodiments, the evaluation unit is also specifically used for:

[0217] Select any frame from the frames that are in a stopped state;

[0218] Perform tester detection and image segmentation on any frame image to obtain the tester block and vernier block in any frame image;

[0219] The scale point clusters are obtained by detecting the scale points on the tester block.

[0220] Based on the vernier plot and the scale point cluster, the sit-and-reach test score is measured.

[0221] Based on any of the above embodiments Figure 13 This is a schematic diagram of the seated forward flexion assessment system provided by the present invention. As shown in the figure, the seated forward flexion assessment system includes a tester 1310, an image acquisition device 1320, and a processor 1330.

[0222] The testing instrument 1310 is used to perform sit-and-reach test;

[0223] The image acquisition device 1320 is used to acquire the video stream to be tested in the sit-and-reach test area and transmit the video stream to be tested to the processor.

[0224] The processor 1330 is used to perform human skeleton point and testing instrument detection on each frame of the video stream under test, obtain skeleton point position information and testing instrument position information of each frame, classify the motion attributes of each frame based on the skeleton point position information and testing instrument position information of each frame, obtain the motion attributes of each frame, and perform sit-and-reach assessment based on the motion attributes of each frame.

[0225] The system provided in this invention classifies the motion attributes of each frame of image based on the skeletal point position information and the tester position information, and then performs a sit-and-reach assessment based on the motion attributes of each frame. Compared with image-based motion attribute classification methods, this system can learn the motion relationships between skeletal points simply and effectively, reducing computational load and improving operational efficiency; at the same time, the equipment is simple and easy to use.

[0226] Figure 14 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 14 As shown, the electronic device may include: a processor 1410, a communications interface 1420, a memory 1430, and a communication bus 1440, wherein the processor 1410, the communications interface 1420, and the memory 1430 communicate with each other via the communication bus 1440. The processor 1410 can call logical instructions in the memory 1430 to execute a sit-and-reach assessment method, which includes:

[0227] Obtain the video stream to be tested;

[0228] Human skeleton points and testing instrument detection are performed on each frame of the video stream under test to obtain the skeleton point position information and testing instrument position information of each frame.

[0229] Based on the skeletal point position information and the tester position information of each frame image, the motion attributes of each frame image are classified to obtain the motion attributes of each frame image.

[0230] Based on the motion attributes of each frame of the image, a sit-and-reach assessment is performed.

[0231] Furthermore, the logical instructions in the aforementioned memory 1430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0232] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to perform the sit-and-reach assessment method provided by the above methods, the method comprising:

[0233] Obtain the video stream to be tested;

[0234] Human skeleton points and testing instrument detection are performed on each frame of the video stream under test to obtain the skeleton point position information and testing instrument position information of each frame.

[0235] Based on the skeletal point position information and the tester position information of each frame image, the motion attributes of each frame image are classified to obtain the motion attributes of each frame image.

[0236] Based on the motion attributes of each frame of the image, a sit-and-reach assessment is performed.

[0237] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the sit-and-reach assessment method provided by the methods described above, the method comprising:

[0238] Obtain the video stream to be tested;

[0239] Human skeleton points and testing instrument detection are performed on each frame of the video stream under test to obtain the skeleton point position information and testing instrument position information of each frame.

[0240] Based on the skeletal point position information and the tester position information of each frame image, the motion attributes of each frame image are classified to obtain the motion attributes of each frame image.

[0241] Based on the motion attributes of each frame of the image, a sit-and-reach assessment is performed.

[0242] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0243] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0244] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for assessing sit-and-reach flexibility, characterized in that, include: Obtain the video stream to be tested; Human skeleton points and testing instrument detection are performed on each frame of the video stream under test to obtain the skeleton point position information and testing instrument position information of each frame. For any frame of image, the skeletal point position information and the tester position information are selected, and a reference distance and a reference point are selected. Then, based on the reference distance and the reference point, the position information is transformed to the same dimension and normalized respectively to obtain the normalized position information of the skeletal points and the normalized position information of the tester. The reference distance and reference point selected for the normalization of the skeletal point position information and the tester position information are the same. The coordinate point sequence composed of the normalized position information of the skeletal points and the coordinate point sequence composed of the normalized position information of the testing instrument are combined into a vector; the vector is input into a multilayer perceptron network to obtain the action attributes of any frame image; and the sit-and-reach assessment is performed based on the action attributes of each frame image. The sit-and-reach assessment includes: Based on the motion attributes of each frame image, a foot pedal frame image is determined from each frame image; based on the difference between the knee and ankle position information in the skeletal point position information of any frame image after the foot pedal frame image and the knee and ankle position information in the skeletal point position information of the foot pedal frame image, a seated forward bend foot pedal cheating evaluation is performed on any frame image.

2. The method for assessing sit-and-reach flexibility according to claim 1, characterized in that, The obtained normalized position information of the bone points and the normalized position information of the testing instrument include: Based on the hip and knee position information in the skeletal point position information of any frame image, determine the distance between the hip and knee, as well as the position information of the hip center point; Based on the distance and the hip center point location information, the skeletal point location information and the tester location information of any frame image are normalized to obtain the normalized skeletal point location information and the normalized tester location information.

3. The method for assessing sit-and-reach flexibility according to claim 1, characterized in that, The method of evaluating seated forward flexion pedal cheating based on the difference between the knee and ankle position information in the skeletal point position information of any subsequent frame image and the knee and ankle position information in the skeletal point position information of the pedal frame image includes: The difference between the knee and ankle position information is normalized to obtain normalized difference information; Based on the normalized difference information, a sit-and-reach pedal cheating evaluation is performed on any frame of the image.

4. The method for assessing sit-and-reach flexibility according to any one of claims 1-3, characterized in that, The sit-and-reach assessment based on the motion attributes of each frame of image includes: Based on the motion attributes of each frame image, as well as the skeletal point position information and the tester position information of each frame image, the state of each frame image is determined. Based on the state of each frame of the image, the sit-and-reach state is switched and the performance is measured.

5. The method for assessing sit-and-reach flexibility according to claim 4, characterized in that, The process of switching between sit-and-reach states and determining performance based on the state of each frame of the image includes: Select any frame from the frames that are in a stopped state; Perform tester detection and image segmentation on any frame image to obtain the tester block and vernier block in any frame image; The scale point clusters are obtained by detecting the scale points on the tester block. Based on the vernier plot and the scale point cluster, the sit-and-reach test score is measured.

6. A sit-and-reach assessment system, characterized in that, include: Tester, image acquisition device and processor; The testing device is used to perform sit-and-reach tests; The image acquisition device is used to acquire the video stream to be tested in the sit-and-reach test area and transmit the video stream to be tested to the processor. The processor is configured to perform human skeleton point and testing instrument detection on each frame of the video stream under test, and obtain the skeleton point position information and testing instrument position information of each frame. For any frame of image, the skeletal point position information and the tester position information are selected, and a reference distance and a reference point are selected. Then, based on the reference distance and the reference point, the position information is transformed to the same dimension and normalized respectively to obtain the normalized position information of the skeletal points and the normalized position information of the tester. The reference distance and reference point selected for the normalization of the skeletal point position information and the tester position information are the same. The coordinate point sequence composed of the normalized position information of the skeletal points and the coordinate point sequence composed of the normalized position information of the testing instrument are combined into a vector; the vector is input into the multilayer perceptron network to obtain the motion attributes of any frame image, and the sit-and-reach assessment is performed based on the motion attributes of each frame image. The sit-and-reach assessment includes: Based on the motion attributes of each frame image, a foot pedal frame image is determined from each frame image; based on the difference between the knee and ankle position information in the skeletal point position information of any frame image after the foot pedal frame image and the knee and ankle position information in the skeletal point position information of the foot pedal frame image, a seated forward bend foot pedal cheating evaluation is performed on any frame image.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the sit-and-reach assessment method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the sit-and-reach assessment method as described in any one of claims 1 to 5.

Images