Real-time detecting and scoring method for sit-and-reach and related products

Abstract

The invention discloses a sit-and-reach real-time detection and scoring method and a related product thereof. The method comprises the following steps: acquiring an image sequence when sit-and-reach detection starts; performing key point identification and positioning on each tester included in each image frame in the image sequence by using a pre-trained attitude estimation model; performing effective detection area identification calibration on each image frame in the image sequence to obtain at least one effective detection area; extracting a hand movement change area from each effective detection area by adopting a background difference technology; performing noise removal processing on the hand movement change area, performing calibration line and scale line identification calibration processing on each effective detection area in real time, performing real-time correction processing on a real-time calibration result according to an initial calibration line coordinate and an initial scale line coordinate, and obtaining a finger touch position according to the real-time correction result; and automatically calculating a final score according to a preset score calculation rule and the finger touch position. The test efficiency is effectively improved.

Description

Technical Field

[0001] The present disclosure generally relates to the field of image processing technology. More specifically, the present disclosure relates to a real-time detection and scoring method for sitting forward flexion and related products. Background Art

[0002] The sit-and-reach test is a common flexibility test item, which is widely used in school physical education classes, fitness clubs and professional athletes' training. However, traditional testing methods mainly rely on manual observation and recording, which is not only time-consuming and laborious, but also easily affected by subjective factors, resulting in inaccurate and inconsistent scoring results. In addition, in some large-scale competitions, due to the large number of participants, manual testing methods are even more difficult to meet the needs of fast and efficient evaluation.

[0003] Therefore, there is an urgent need to develop a solution that can automatically detect the sit-and-reach movement and provide objective scoring. Summary of the invention

[0004] In order to at least solve one or more of the technical problems mentioned above, the present disclosure proposes a real-time detection and scoring scheme for sit-and-forth flexion in multiple aspects.

[0005] In a first aspect, the present disclosure provides a real-time detection and scoring method for sitting forward flexion, the method comprising: at the beginning of sitting forward flexion detection, obtaining an image sequence, each image frame in the image sequence containing at least one tester performing a forward flexion action; using a pre-trained posture estimation model to identify and locate key points of each tester contained in each image frame in the image sequence, and obtaining key point data corresponding to each tester; evaluating the action standardization of each tester according to the key point data, and obtaining an evaluation result; identifying and calibrating effective detection areas for each image frame in the image sequence, and obtaining at least one effective detection area, each effective detection area is obtained by identifying and calibrating a marker block pre-set in each forward flexion test panel, and the forward flexion test panel is also pre-set with scale lines and calibration lines; for each effective detection area, the following processing is performed: using background difference technology to extract from each effective detection area the key points corresponding to each tester The method comprises the following steps: determining the hand movement change area corresponding to the hand movement change area; removing noise from the hand movement change area to obtain an accurate hand movement change area; performing calibration line and scale line recognition and calibration processing on each effective detection area in real time to obtain current calibration line coordinates and current scale line coordinates; acquiring a stored standard image frame, which is obtained by performing calibration line and scale line recognition and calibration processing on the collected initial image frame before the sitting forward flexion detection begins, and the standard image frame includes calibrated initial calibration line coordinates and initial scale line coordinates; performing real-time correction processing on the current calibration line coordinates and current scale line coordinates according to the initial calibration line coordinates and initial scale line coordinates to obtain accurate scale line coordinates; calculating the finger touch position of each tester during the forward flexion action according to the accurate hand movement change area and the accurate scale line coordinates; and automatically calculating the final score corresponding to each tester according to the preset score calculation rules and finger touch position.

[0006] In a second aspect, the present disclosure provides a real-time detection and scoring device for sitting forward flexion, the device comprising: an image sequence acquisition module, used to acquire an image sequence at the beginning of sitting forward flexion detection, each image frame in the image sequence containing at least one tester performing a forward flexion action; an action specification evaluation module, used to use a pre-trained posture estimation model to identify and locate key points of each tester contained in each image frame in the image sequence, and obtain key point data corresponding to each tester; evaluate the action specification of each tester according to the key point data to obtain an evaluation result; an effective detection area identification module, used to identify and calibrate the effective detection area of ​​each image frame in the image sequence, and obtain at least one effective detection area, each effective detection area is obtained by identification and calibration based on a marker block pre-set in each forward flexion test panel, and the forward flexion test panel is also pre-set with scale lines and calibration lines; a hand movement area extraction module, used to perform the following processing on each effective detection area: extract the key points corresponding to each tester from each effective detection area using background difference technology The hand movement change area corresponding to the tester is processed by removing noise from the hand movement change area to obtain an accurate hand movement change area; a scale line real-time calibration module is used to perform calibration and identification and calibration processing of calibration lines and scale lines on each effective detection area in real time to obtain the current calibration line coordinates and the current scale line coordinates; a standard image frame acquisition module is used to acquire the stored standard image frame, which is obtained by performing calibration and identification and calibration processing on the collected initial image frame before the sitting forward flexion detection begins, and the standard image frame includes the calibrated initial calibration line coordinates and initial scale line coordinates; a scale line real-time correction module is used to perform real-time correction processing on the current calibration line coordinates and the current scale line coordinates according to the initial calibration line coordinates and the initial scale line coordinates to obtain accurate scale line coordinates; the finger touch position of each tester during the forward flexion action is calculated according to the accurate hand movement change area and the accurate scale line coordinates; a final score calculation module is used to automatically calculate the final score corresponding to each tester according to the preset calculation score rules and finger touch position.

[0007] In a third aspect, the present disclosure provides an electronic device comprising: a processor; and a memory storing computer program instructions of a real-time detection and scoring method for sit-and-forth flexion implemented by a computer, wherein when the computer program instructions are executed by the processor, the electronic device executes the method described in the first aspect.

[0008] In a fourth aspect, the present disclosure provides a computer-readable storage medium comprising computer program instructions for a real-time detection and scoring method for sit-and-forth flexion implemented by a computer, wherein when the computer program instructions are executed by a processor, the method described in the first aspect is implemented.

[0009] Through the real-time sit-and-forward flexion detection and scoring method and related products provided above, the disclosed embodiment obtains an image sequence at the beginning of the sit-and-forward flexion detection; then, a pre-trained posture estimation model is used to identify and locate the key points of each tester contained in each image frame in the image sequence to evaluate the movement standardization of each tester; each image frame in the image sequence is identified and calibrated with an effective detection area to obtain at least one effective detection area; for each effective detection area, the following processing is performed: a background difference technique is used to extract a hand movement change area corresponding to each tester from each effective detection area; noise is removed from the hand movement change area to obtain an accurate hand movement change area. area; for each effective detection area, the calibration line and scale line recognition and calibration processing are performed in real time to obtain the current calibration line coordinates and the current scale line coordinates; the stored standard image frame is obtained by performing the calibration line and scale line recognition and calibration processing on the collected initial image frame before the sitting forward flexion detection begins, and the standard image frame contains the calibrated initial calibration line coordinates and initial scale line coordinates; according to the initial calibration line coordinates and the initial scale line coordinates, the current calibration line coordinates and the current scale line coordinates are corrected in real time to obtain accurate scale line coordinates; according to the precise hand movement change area and the accurate scale line coordinates, the finger touch position of each tester during the forward flexion action is calculated. According to the pre-set calculation score rules and finger touch position, the final score corresponding to each tester is automatically calculated. The disclosed embodiment uses background difference technology and scale line real-time calibration algorithm to quickly calculate the score, realizes the calibration of errors and instant feedback of test results, and improves the efficiency and accuracy of the test. The entire testing process is completed automatically by computer without human intervention, which reduces the difficulty of operation and labor intensity, and also reduces the impact of human factors on test results. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] By reading the detailed description below with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0011] Figure 1 An exemplary flow chart of a method 100 for real-time detection of sitting forward flexion according to some embodiments of the present disclosure is shown;

[0012] Figure 2 An exemplary flow chart showing the specific implementation of step S103 in some other embodiments of the present disclosure;

[0013] Figure 3 An exemplary flow chart showing step S104 of some other embodiments of the present disclosure;

[0014] Figure 4 An exemplary flow chart showing step S107 of some other embodiments of the present disclosure;

[0015] Figure 5 A schematic block diagram showing the structure of a real-time detection and scoring device 500 for sitting forward flexion according to some embodiments of the present disclosure is shown;

[0016] Figure 6 A schematic diagram showing a forward flexion test panel according to some embodiments of the present disclosure is shown;

[0017] Figure 7 A schematic diagram showing the positional relationship between fingers and a forward flexion test panel according to some embodiments of the present disclosure;

[0018] Figure 8 A schematic block diagram of an electronic device 800 according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0020] It should be understood that the terms "include" and "comprising" used in the specification and claims of the present disclosure indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0021] It should also be understood that the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. As used in this disclosure and claims, the singular forms of "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this disclosure and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.

[0022] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0023] The following is combined with Figure 1-8 The real-time detection and scoring method of sit-and-forward flexion in the disclosed embodiment is described in detail.

[0024] Figure 1 An exemplary flow chart of the sit-and-forth real-time detection and scoring 100 of some embodiments of the present disclosure is shown.

[0025] like Figure 1 As shown, in step S101, when the sit-and-reach test starts, an image sequence is acquired, and each image frame in the image sequence contains at least one tester performing a sit-and-reach action.

[0026] In the above steps, a high-definition camera is installed at an appropriate position of the sports field to continuously capture the image sequence of the tester when performing the forward bending movement. The high-definition camera can cover the entire test area to ensure that the tester's movements are fully captured. Each image frame in the image sequence collected by the disclosed embodiment can include one or more testers, that is, multiple testers can be synchronously detected at the same time.

[0027] In step S102, a pre-trained posture estimation model is used to identify and locate the key points of each tester contained in each image frame in the image sequence to obtain key point data corresponding to each tester; the action standardization of each tester is evaluated based on the key point data to obtain an evaluation result.

[0028] In the above steps, the posture estimation model can be a convolutional neural network model, and each image frame is automatically identified by the convolutional neural network model to obtain key point data corresponding to each tester. The image sequence obtained from step S101 is input into the posture estimation model to obtain key point data corresponding to each tester. Among them, the key point data can include the horizontal coordinates, vertical coordinates, and visibility of 21 skeletal key points on the human body of all testers contained in each image frame, where the coordinates are the coordinates in the pixel coordinate system, and the visibility is whether the key point is directly visible under the camera viewing angle.

[0029] The standardization of the movements of each tester is evaluated based on the key point data, which may include sequentially obtaining the horizontal coordinates, vertical coordinates, and visibility of the skeleton key points of all testers in each image frame, evaluating the movements of the testers according to a preset movement evaluation algorithm, and obtaining an evaluation result, which is used to evaluate whether there is any irregular behavior in the sit-and-forward flexion movement of the tester. The irregular behavior refers to the movement of the tester bending the legs, bending the arms, impacting, using one hand, etc. when the tester is sitting and bending.

[0030] In some embodiments, the skeleton key point information can be obtained using an anchor-free and single-stage target detection model, which can be designed using the Yolov8-pose architecture. The posture estimation model designed using the Yolov8-pose architecture is advanced and efficient, and is suitable for real-time detection. In order to improve the performance of the model, a public image dataset containing human bodies can be used to train the posture estimation model using labeled human skeleton point data. The reasoning efficiency of the posture estimation model can also be accelerated through pruning, distillation, quantization and other technologies.

[0031] In step S103, effective detection area identification and calibration are performed on each image frame in the image sequence to obtain at least one effective detection area. Each effective detection area is obtained by identification and calibration based on a marking block pre-set in each forward flexion test panel. Scale lines and calibration lines are also pre-set in the forward flexion test panel.

[0032] In the above steps, the effective detection area identification and calibration for each image frame in the image sequence may include: using an adaptive image enhancement algorithm to perform enhancement and optimization processing on each image frame in the image sequence to obtain each optimized image frame; obtaining the coordinate position of the marker block on each optimized image frame, and using perspective transformation to extract the effective detection area of ​​the forward flexion test panel from each optimized image frame.

[0033] The image acquired by the camera is optimized using an adaptive image enhancement algorithm, which can automatically adjust the brightness, contrast and other parameters of the image by counting the overall brightness and contrast of the image to ensure that the marked block is clearly visible and convenient for subsequent processing.

[0034] A red marking block is pre-set on the forward flexion test panel, and the red marking block is used as a reference point to determine the effective detection area according to the reference point. The forward flexion test panel is also pre-set with scale lines and calibration lines, such as Figure 6 As shown. The red marking block is set as a square mark with a side length of 5 cm and a pure red color. The red marking blocks are symmetrically set at the four corners of the forward flexion panel.

[0035] In some embodiments, the marking block may also be set to a green marking block, or a marking block of another color.

[0036] In step S104, for each effective detection area, the following processing is performed: the hand movement change area corresponding to each tester is extracted from each effective detection area using background difference technology; the hand movement change area is subjected to noise removal processing to obtain an accurate hand movement change area.

[0037] In step S105, the calibration line and scale line identification and calibration processing is performed in real time on each effective detection area to obtain the current calibration line coordinates and the current scale line coordinates.

[0038] In step S106, a stored standard image frame is obtained. The standard image frame is obtained by performing calibration line and scale line recognition and calibration processing on the collected initial image frame before the sitting forward flexion detection begins. The standard image frame includes calibrated initial calibration line coordinates and initial scale line coordinates.

[0039] In step S107, the current calibration line coordinates and the left side of the current scale line are corrected in real time according to the initial calibration line coordinates and the initial scale line coordinates to obtain accurate scale line coordinates; the finger touch position of each tester during the forward flexion movement is calculated according to the precise hand movement change area and the accurate scale line coordinates.

[0040] In step S108, the final score corresponding to each tester is automatically calculated according to the preset score calculation rules and the finger touch position.

[0041] In the above steps, if Figure 7 As shown, substitute the horizontal coordinate x of the finger touch position, i.e. the finger tip point P, into the following function:

[0042]

[0043] Among them, X i Indicates the horizontal coordinate position of the midpoint of the i+1th scale line;

[0044] The final score automatically calculated for each test taker includes:

[0045]

[0046] Where T is the scale and T represents the actual spacing between adjacent scale lines.

[0047] In the above steps S105 and S106, the calibration line and scale line identification and calibration processing may include: graying and binarization preprocessing the image frame to be calibrated, the image frame to be calibrated is the initial image frame or each image frame; using the Canny edge detection algorithm to perform edge detection on the image frame to be calibrated to obtain edge detection results; using the Hough transform straight line detection algorithm to detect the edge detection results to obtain straight line segments corresponding to the scale lines and calibration lines; screening the straight line segments according to the structural characteristics of the calibration lines and scale lines to obtain coordinates corresponding to the calibration lines and coordinates corresponding to the scale lines. When the image frame to be calibrated is the initial image frame, the coordinates corresponding to the calibration lines are the initial calibration line coordinates, and the coordinates corresponding to the scale lines are the initial scale line coordinates. When the image frame to be calibrated is each image frame in the image sequence, the coordinates corresponding to the calibration lines are the current calibration line coordinates, and the coordinates corresponding to the scale lines are the current scale line coordinates.

[0048] The real-time detection and scoring method for sitting forward flexion of the disclosed embodiment detects at least one effective detection area from the image sequence, uses background difference technology to extract the hand movement change area corresponding to each tester from the effective detection area, and removes noise from the hand movement change area, effectively improving the accuracy of hand detection; then, according to the initial calibration line coordinates and the initial scale line coordinates, the current calibration line coordinates and the current scale line coordinates are corrected in real time to obtain accurate scale line coordinates, and the finger touch position is determined using the accurate scale line coordinates and the hand movement change area, which can effectively detect the real-time finger touch position during the forward flexion action, further improving the accuracy of the test results. Finally, automatic scoring is performed based on the detection results, and the entire process does not require manual intervention, which reduces the difficulty and labor intensity of the test operation, and also reduces the impact of human factors on the test results.

[0049] Figure 2 FIG. 1 shows an exemplary flow chart of step S103 of some other embodiments of the present disclosure. It can be understood that Figure 2 The real-time detection and scoring method for sitting forward flexion shown in the figure is a specific implementation of the aforementioned step S103. Figure 1 The related features described can similarly apply here.

[0050] In step S201, the average gray value is calculated for each image frame to obtain the average brightness of the image.

[0051] In step S202, histogram statistics are performed on each image frame to obtain a histogram corresponding to each image frame.

[0052] In step S203, adaptive gamma correction is performed according to the average brightness of the image to obtain an adjusted average brightness of the image.

[0053] In the above steps, assuming that the average brightness of the image is B, adaptive gamma correction is performed according to the average brightness of the image to obtain the adjusted average brightness of the image. The following formula can be used to perform adaptive gamma correction on the average brightness of the image B. The mathematical expression of adaptive gamma correction is:

[0054]

[0055] Among them, I in (x, y) represents each image frame in the image sequence as the current image frame; I out (x, y) represents each image frame after gamma correction; γ(B) represents the gamma value dynamically calculated based on the average brightness B of the image. γ(B) can be expressed by the following mathematical expression:

[0056]

[0057] In step S204, contrast-limited adaptive histogram equalization is performed on the histogram corresponding to each image frame to obtain an adjusted image contrast.

[0058] In the above steps, Contrast Limited Adaptive Histogram Equalization (CLAHE) is an adaptive image enhancement technology that is mainly used to improve the contrast of an image while avoiding overexposure or underexposure. This technology performs histogram equalization on local areas of an image, making the pixel distribution in each local area more uniform, thereby improving the visual effect of the overall image.

[0059] In practical applications, CLAHE is usually used to deal with the delay spread problem caused by multipath fading, which can lead to interference between code elements during high-speed data transmission. CLAHE adaptive equalization technology is an effective image enhancement and load balancing strategy that improves system performance and reliability by adaptively adjusting image brightness and contrast and reasonably distributing system load.

[0060] In step S205, each image frame is adjusted according to the adjusted image average brightness and the adjusted image contrast to obtain each optimized image frame.

[0061] In step 206, the coordinate position of the marker block on each optimized image frame is obtained, and the effective detection area of ​​the forward flexion test panel is extracted from each optimized image frame using perspective transformation.

[0062] The disclosed embodiment performs image enhancement optimization processing on the captured image sequence by using the above-mentioned adaptive image enhancement algorithm, so that the marked blocks in the image sequence can be made clearer and more visible, thereby improving the detection accuracy of the effective detection area.

[0063] Figure 3 FIG. 1 shows an exemplary flow chart of step S104 of some other embodiments of the present disclosure. It can be understood that Figure 3 The real-time detection and scoring method for sitting forward flexion shown in the figure is a specific implementation of the aforementioned step S104. Figure 1 The related features described can similarly apply here.

[0064] like Figure 3 As shown, in step S301, for each pixel position in each effective detection area, the absolute value of the grayscale value difference between each image frame and the standard image frame is calculated to obtain the absolute value of the frame difference corresponding to each pixel position.

[0065] In step S302, a threshold determination process is performed on the absolute value of the frame difference corresponding to each pixel position to obtain a difference map corresponding to each effective detection area.

[0066] In the above steps, the absolute values ​​of the frame differences at all pixel positions are thresholded, and the pixels exceeding the preset threshold are regarded as the changed area, and the pixels below the preset threshold are regarded as the background unchanged area, and the difference map is obtained after binarization.

[0067] In step S303, noise removal processing is performed on the difference image corresponding to each effective detection area to obtain a denoised hand movement area.

[0068] In the above steps, noise removal processing is performed on the difference map corresponding to each effective detection area to obtain the denoised hand movement area, including: obtaining the pixel value of each pixel in the difference map corresponding to each image frame; determining the target pixel whose pixel value is the first value; obtaining the three channel values ​​corresponding to the RGB color representation model of the target pixel from the standard image frame and each image frame respectively; calculating the frame difference corresponding to each color channel respectively; performing weighted summation on the frame differences to obtain the summation result; screening the target pixel points according to a preset threshold value and the summation result to obtain false points; setting the pixel value of the false point to the second value, and not processing the pixel point of the second value; in the screened difference map, for the remaining target pixel points, obtaining the three channel values ​​corresponding to the HSV color representation model of the target pixel point from the standard image frame; screening the target pixel points according to the preset three channel thresholds corresponding to the HSV color representation model and the three channel values ​​corresponding to the HSV color representation model of the target pixel point to obtain false points; setting the pixel value of the false point to the second value, and not processing the pixel point of the second value.

[0069] The pixel value is 0 or 255, the first value can be 255, and the second value can be 0. For the summation result, a preset threshold is set for screening, and target pixels above the preset threshold are regarded as false points, and the pixel values ​​of the pixels regarded as false points in the differential image are set to 0, and target pixels below the preset threshold are regarded as approximate points and are not processed. For the remaining target pixels in the processed differential image, for the target pixel with a pixel value of 255, the HSV spatial data of the target pixel in the initial image frame is extracted, and the HSV spatial data of the target pixel is screened according to the preset HSV three-channel threshold. Target pixels that exceed the preset HSV three-channel threshold range are regarded as false points, and the pixel values ​​of the target pixel in the differential image are set to 0. Target pixels below the preset HSV three-channel threshold range are regarded as approximate points and are not processed.

[0070] Assume that the RGB value of pixel A in the initial image frame is (R orig ,G orig ,B orig ), the RGB value of the same pixel A in each image frame is (R curr ,G curr ,B curr ), the processed pixel value can be calculated by the following formula:

[0071] P new (x,y)=(b diff (x,y)×1.25)+(g diff (x,y)×0.5)+(r diff (x,y)×1.25)

[0072] Among them, b diff (x,y),g diff (x,y),r diff (x, y) are the frame differences of the blue channel, green channel, and red channel, respectively, and the calculation formulas are:

[0073] b diff (x,y)=|B curr (x,y)-B orig (x,y)

[0074] g diff (x,y)=|G curr (x,y)-G orig (x,y)

[0075] r diff (x,y)=|R curr (x,y)-R orig (x,y)

[0076] In step S304, the denoised hand movement region is optimized through morphological transformation to obtain an accurate hand movement region.

[0077] The disclosed embodiment can effectively remove interference factors that appear in the effective detection area by detecting false points from the effective detection area. These interference factors, such as stains on the forward flexion test panel or shadows cast by other objects on the forward flexion test panel, affect the test results. By removing these false points from the test results, the accuracy of the hand movement change area detection results can be improved.

[0078] Figure 4 FIG. 1 shows an exemplary flow chart of step S107 of some other embodiments of the present disclosure. It can be understood that Figure 4 The real-time detection and scoring method for sitting forward flexion shown in the figure is a specific implementation of the aforementioned step S107. Figure 1 The related features described can similarly apply here.

[0079] like Figure 4 As shown, in step S401, two calibration lines on the forward flexion test panel are identified from each effective detection area.

[0080] In step S402, current calibration line coordinates of two calibration lines are obtained in each effective detection area.

[0081] In step S403, the acquired current calibration line coordinates are matched with the initial calibration line coordinates, and a geometric transformation matrix between the two is calculated.

[0082] In step S404, the current scale line coordinates are corrected according to the geometric transformation matrix and the initial scale line coordinates to obtain accurate scale line coordinates.

[0083] The disclosed embodiment can perform correction processing after real-time calibration of the current scale line coordinates in the effective detection area, so as to overcome the motion deformation of the image detection results caused by the test subject during the real-time detection of sitting forward flexion, thereby effectively improving the accuracy of the test results.

[0084] Based on the same inventive concept, some embodiments of the present disclosure describe a real-time detection and scoring device for sitting forward flexion. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method. For the specific definition of the real-time detection and scoring device for sitting forward flexion, please refer to the definition of the real-time detection and scoring method for sitting forward flexion above, which will not be repeated here.

[0085] Figure 5 FIG. 5 is a schematic block diagram showing the structure of a real-time detection and scoring device 500 for sitting forward flexion according to some embodiments of the present disclosure. Figure 5 As shown, the real-time detection and scoring device 500 for sitting forward flexion includes:

[0086] The image sequence acquisition module 501 is used to acquire an image sequence when the sit-and-reach test starts, wherein each image frame in the image sequence contains at least one test subject performing a sit-and-reach action.

[0087] The action standard evaluation module 502 is used to use a pre-trained posture estimation model to identify and locate the key points of each tester contained in each image frame in the image sequence, and obtain the key point data corresponding to each tester; evaluate the action standard of each tester based on the key point data to obtain an evaluation result.

[0088] The effective detection area identification module 503 is used to perform effective detection area identification and calibration on each image frame in the image sequence to obtain at least one effective detection area. Each effective detection area is obtained by identification and calibration based on a marking block pre-set in each forward flexion test panel. The forward flexion test panel is also pre-set with scale lines and calibration lines.

[0089] The hand movement area extraction module 504 is used to perform the following processing on each effective detection area: using background difference technology to extract the hand movement change area corresponding to each tester from each effective detection area; removing noise from the hand movement change area to obtain an accurate hand movement change area.

[0090] The scale line real-time calibration module 505 is used to perform real-time identification and calibration processing of calibration lines and scale lines on each effective detection area to obtain the current calibration line coordinates and the current scale line coordinates.

[0091] The standard image frame acquisition module 506 is used to acquire the stored standard image frame. The standard image frame is obtained by performing calibration line and scale line recognition and calibration processing on the collected initial image frame before the sitting forward flexion detection begins. The standard image frame includes the calibrated initial calibration line coordinates and initial scale line coordinates.

[0092] The scale line real-time correction module 507 is used to perform real-time correction processing on the current calibration line coordinates and the current scale line coordinates according to the initial calibration line coordinates and the initial scale line coordinates to obtain accurate scale line coordinates; and calculate the finger touch position of each tester during the forward flexion movement according to the precise hand movement change area and the accurate scale line coordinates.

[0093] The final score calculation module 508 is used to automatically calculate the final score corresponding to each tester according to the preset score calculation rules and the finger touch position.

[0094] In some embodiments, the effective detection area identification module 503 includes:

[0095] An image enhancement processing unit, used to perform enhancement and optimization processing on each image frame in the image sequence using an adaptive image enhancement algorithm to obtain each optimized image frame;

[0096] The effective area extraction unit is used to obtain the coordinate position of the marker block on each optimized image frame, and extract the effective detection area of ​​the forward bending test panel from each optimized image frame using perspective transformation.

[0097] In some embodiments, the image enhancement processing unit is further used to calculate the average grayscale value for each image frame to obtain the average brightness of the image; perform histogram statistics on each image frame to obtain a histogram corresponding to each image frame; perform adaptive gamma correction based on the image average brightness to obtain an adjusted image average brightness; perform contrast-limited adaptive histogram equalization on the histogram corresponding to each image frame to obtain an adjusted image contrast; for each image frame, adjust the image frame based on the adjusted image average brightness and the adjusted image contrast to obtain each optimized image frame.

[0098] In some embodiments, the device also includes a calibration line and scale line recognition and calibration module, which is used to perform grayscale and binarization preprocessing on the image frame to be calibrated, where the image frame to be calibrated is the initial image frame or each image frame; using the Canny edge detection algorithm to perform edge detection on the image frame to be calibrated to obtain an edge detection result; using the Hough transform straight line detection algorithm to detect the edge detection result to obtain straight line segments corresponding to the scale lines and calibration lines one by one; screening the straight line segments according to the structural characteristics of the calibration lines and scale lines to obtain coordinates corresponding to the scale lines and coordinates corresponding to the calibration lines.

[0099] In some embodiments, the hand movement area extraction module 504 is also used to: calculate the absolute value of the grayscale value difference between each image frame and the standard image frame for each pixel position in each effective detection area, and obtain the absolute value of the frame difference corresponding to each pixel position; perform threshold judgment processing on the absolute value of the frame difference corresponding to each pixel position, and obtain a differential map corresponding to each effective detection area.

[0100] In some embodiments, the hand movement area extraction module 504 is also used to: perform noise removal processing on the differential image corresponding to each effective detection area to obtain a denoised hand movement area; optimize the denoised hand movement area through morphological transformation to obtain an accurate hand movement area.

[0101] The hand movement area extraction module 504 is further used to: obtain the pixel value of each pixel point from the differential map corresponding to each effective detection area; filter out at least one target pixel point whose pixel value is the first value from the differential map corresponding to each effective detection area; for each target pixel point in the at least one target pixel point, obtain three color channel values ​​corresponding to the RGB color representation model of each target pixel point from the standard image frame and each image frame respectively; calculate the frame difference corresponding to each color channel respectively; perform weighted summation on the frame difference to obtain the summation result; filter out at least one target pixel point according to a preset threshold value and the summation result to obtain at least one target pixel point. first false points; setting the pixel value of each first false point to the second value, and not processing the target pixel points of the second value; for the remaining target pixel points among the at least one target pixel points, obtaining three channel values ​​corresponding to the HSV color representation model corresponding to the remaining target pixel points from the standard image frame; screening the remaining target pixel points according to the pre-set three channel thresholds corresponding to the HSV color representation model and the three channel values ​​corresponding to the HSV color representation model corresponding to the remaining target pixel points to obtain at least one second false point; setting the pixel value of each second false point to the second value, and not processing the target pixel points of the second value.

[0102] In some embodiments, the scale line real-time correction module 507 is also used to identify two calibration lines on the forward flexion test panel from each image frame; obtain the current calibration line coordinates of the two calibration lines in each image frame; match the obtained current calibration line coordinates with the initial calibration line coordinates, and calculate the geometric transformation matrix between the two; correct the current scale line coordinates according to the geometric transformation matrix and the initial scale line coordinates to obtain accurate scale line coordinates.

[0103] In some embodiments, the final score calculation module 508 is further used to substitute the horizontal coordinate x of the finger touch position into the following function:

[0104]

[0105] Among them, X i Indicates the horizontal coordinate position of the midpoint of the i+1th scale line;

[0106] The final score automatically calculated for each test taker includes:

[0107]

[0108] Where T represents the actual spacing between adjacent tick marks.

[0109] The real-time detection and scoring method for sitting forward flexion of the disclosed embodiment detects at least one effective detection area from the image sequence, uses background difference technology to extract the hand movement change area corresponding to each tester from the effective detection area, and removes noise from the hand movement change area, effectively improving the accuracy of hand detection; then, according to the initial calibration line coordinates and the initial scale line coordinates, the current calibration line coordinates and the current scale line coordinates are corrected in real time to obtain accurate scale line coordinates, and the finger touch position is determined using the accurate scale line coordinates and the hand movement change area, which can effectively detect the real-time finger touch position during the forward flexion action, further improving the accuracy of the test results. Finally, automatic scoring is performed based on the detection results, and the entire process does not require manual intervention, which reduces the difficulty and labor intensity of the test operation, and also reduces the impact of human factors on the test results.

[0110] After introducing the above embodiments, next, refer to Figure 8 The products related to the real-time detection and scoring method for sit-and-forth flexion according to the exemplary embodiment of the present disclosure are described.

[0111] Figure 8 FIG. 8 is a schematic block diagram of an electronic device 800 according to an embodiment of the present disclosure. Figure 8 As shown, the electronic device 800 may include a processor 801 and a memory 802. The memory 802 stores computer program instructions for executing the sit-and-reach real-time detection and scoring method. When the computer program instructions are executed by the processor 801, the electronic device 800 executes the method according to the above combined method. Figures 1 to 4For example, in some embodiments, the electronic device 800 can obtain an image sequence at the beginning of the sit-and-forward flexion test; then, use a pre-trained posture estimation model to identify and locate the key points of each tester contained in each image frame in the image sequence to obtain key point data corresponding to each tester; evaluate the action standardization of each tester according to the key point data to obtain an evaluation result; then, identify and calibrate the effective detection area for each image frame in the image sequence to obtain at least one effective detection area; for each effective detection area, perform the following processing: use background difference technology to extract the hand movement change area corresponding to each tester from each effective detection area; remove the hand movement change area; De-noise processing is performed to obtain an accurate hand movement change area; the calibration lines and scale lines of each effective detection area are identified and calibrated in real time to obtain the current calibration line coordinates and the current scale line coordinates; the stored standard image frame is acquired, and the standard image frame contains the calibrated initial calibration line coordinates and the initial scale line coordinates; according to the initial calibration line coordinates and the initial scale line coordinates, the current calibration line coordinates and the current scale line coordinates are corrected in real time to obtain accurate scale line coordinates; according to the accurate hand movement change area and the accurate scale line coordinates, the finger touch position of each tester during the forward flexion movement is calculated; according to the pre-set calculation score rules and finger touch position, the final score corresponding to each tester is automatically calculated, etc. Based on this, the electronic device 800 can estimate the posture of the human body through the deep learning model, and can accurately identify the key points of the human body, such as the fingertips and toes, which is more accurate than the manual observation and recording methods in the relevant technology, and reduces human errors; then, the background difference technology and the scale line real-time calibration algorithm are used to perform the forward flexion action test on the collected image, and the score is quickly calculated according to the test results, which effectively improves the test efficiency and accuracy of the forward flexion test. The entire detection process is automatically completed by the computer without human intervention, which reduces the difficulty of operation and labor intensity, and also reduces the impact of human factors on the test results.

[0112] Those skilled in the art will appreciate that the embodiments of the present disclosure may be implemented as a system, method, or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," "unit," or "system." In addition, in some embodiments, the present disclosure may also be implemented in the form of a computer program product in one or more computer-readable media, which contains a computer-readable program code.

[0113] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive examples) of computer-readable storage media may include, for example: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.

[0114] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0115] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0116] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or combinations thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0117] The following will describe the implementation of the present disclosure with reference to the flowchart of the method of the present disclosure embodiment and the block diagram of the device (or system). It should be understood that each square frame of the flowchart and / or block diagram and the combination of each square frame in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, thereby producing a machine, and these computer program instructions are executed by a computer or other programmable data processing device to produce a device that implements the function / operation specified in the square frame in the flowchart and / or block diagram.

[0118] These computer program instructions may also be stored in a computer-readable medium that enables a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable medium produce a product that includes an instruction device that implements the functions / operations specified in the blocks in the flowchart and / or block diagram.

[0119] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby enabling the instructions executed on the computer or other programmable device to provide a process for implementing the functions / operations specified in the blocks in the flowchart and / or block diagram.

[0120] According to the embodiments of the present disclosure, a method for generating an article implemented by a computer, an electronic device and a computer-readable storage medium are proposed. In addition, any number of elements in the drawings is used for illustration rather than limitation, and any naming is only used for distinction and does not have any limiting meaning.

[0121] It should be noted that although several devices or sub-devices of the device are mentioned in the above detailed description, such division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more devices described above can be embodied in one device. Conversely, the features and functions of one device described above can be further divided into multiple devices to be embodied.

[0122] Although multiple embodiments of the present disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art may think of many changes, modifications, and alternatives without departing from the thought and spirit of the present disclosure. It should be understood that in the process of practicing the present disclosure, various alternatives to the embodiments of the present disclosure described herein may be adopted. The attached claims are intended to define the scope of protection of the present disclosure, and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A real-time detection and scoring method for sit-and-forth flexion, characterized in that: The method includes: At the beginning of the sit-and-reach test, an image sequence is acquired, wherein each image frame in the image sequence contains at least one test subject performing a sit-and-reach action; Using a pre-trained posture estimation model, key point recognition and positioning are performed on each tester included in each image frame in the image sequence to obtain key point data corresponding to each tester; and the action standardization of each tester is evaluated according to the key point data to obtain an evaluation result; Performing effective detection area identification and calibration on each image frame in the image sequence to obtain at least one effective detection area, wherein each effective detection area is obtained by identification and calibration based on a marker block pre-set in each forward flexion test panel, wherein the forward flexion test panel is also pre-set with scale lines and calibration lines; For each of the effective detection areas, the following processing is performed: A background difference technique is used to extract a hand movement change region corresponding to each tester from each effective detection region; noise is removed from the hand movement change region to obtain an accurate hand movement change region; Performing calibration line and scale line identification and calibration processing on each of the effective detection areas in real time to obtain current calibration line coordinates and current scale line coordinates; Acquire a stored standard image frame, wherein the standard image frame is obtained by performing a calibration line and scale line identification and calibration process on the collected initial image frame before the sitting forward flexion detection begins, and the standard image frame includes calibrated initial calibration line coordinates and initial scale line coordinates; According to the initial calibration line coordinates and the initial scale line coordinates, the current calibration line coordinates and the current scale line coordinates are corrected in real time to obtain accurate scale line coordinates; according to the precise hand movement change area and the accurate scale line coordinates, the finger touch position of each tester during the forward flexion movement is calculated; The final score corresponding to each tester is automatically calculated according to the preset score calculation rules and the finger touch position.

2. The method according to claim 1, characterized in that: Performing effective detection area identification and calibration on each image frame in the image sequence to obtain at least one effective detection area includes: Performing enhancement and optimization processing on each image frame in the image sequence using an adaptive image enhancement algorithm to obtain each optimized image frame; The coordinate position of the marking block on each optimized image frame is obtained, and the effective detection area of ​​the forward flexion test panel is extracted from each optimized image frame using perspective transformation.

3. The method according to claim 2, characterized in that Using an adaptive image enhancement algorithm to perform enhancement and optimization processing on each image frame in the image sequence to obtain each optimized image frame, including: Calculate the average gray value for each image frame to obtain the average brightness of the image; Performing histogram statistics on each image frame to obtain a histogram corresponding to each image frame; Performing adaptive gamma correction according to the average brightness of the image to obtain an adjusted average brightness of the image; performing contrast-limited adaptive histogram equalization on a histogram corresponding to each image frame to obtain an adjusted image contrast; Each image frame is adjusted according to the adjusted image average brightness and the adjusted image contrast to obtain each optimized image frame.

4. The method according to claim 1, characterized in that The calibration line and scale line identification and calibration process comprises: Performing grayscale and binarization preprocessing on the image frame to be calibrated, wherein the image frame to be calibrated is the initial image frame or each image frame; Using the Canny edge detection algorithm to perform edge detection on the image frame to be calibrated to obtain an edge detection result; The edge detection result is detected by using a Hough transform straight line detection algorithm to obtain straight line segments corresponding to the scale lines and the calibration lines one by one; The straight line segments are screened according to the structural characteristics of the calibration lines and the scale lines to obtain coordinates corresponding to the calibration lines and coordinates corresponding to the scale lines.

5. The method according to claim 1, characterized in that The background difference technology is used to extract the hand movement change area corresponding to each tester from each effective detection area, including: For each pixel position in each effective detection area, the absolute value of the grayscale value difference between each image frame and the standard image frame is calculated to obtain the absolute value of the frame difference corresponding to each pixel position; A threshold determination process is performed on the absolute value of the frame difference corresponding to each pixel position to obtain a differential map corresponding to each effective detection area.

6. The method according to claim 5, characterized in that The hand movement change region is subjected to noise removal processing to obtain an accurate hand movement change region, including: De-noising the difference image corresponding to each valid detection area to obtain the denoised hand movement area; The denoised hand movement area is optimized through morphological transformation to obtain the accurate hand movement area.

7. The method according to claim 6, characterized in that De-noising is performed on the difference image corresponding to each valid detection area to obtain the denoised hand movement area, including: Obtaining the pixel value of each pixel from the difference map corresponding to each effective detection area; Filtering at least one target pixel point whose pixel value is a first value from the difference map corresponding to each effective detection area; For each target pixel of the at least one target pixel, respectively obtaining three color channel values ​​corresponding to the RGB color representation model of each target pixel from the standard image frame and each image frame; Calculate the frame difference corresponding to each color channel separately; Performing weighted summation on the frame differences to obtain a summation result; Screening the at least one target pixel point according to a preset threshold and the summation result to obtain at least one first false point; The pixel value of each of the first false points is set to a second value, and the target pixel point of the second value is not processed; For the remaining target pixel points among the at least one target pixel point, acquiring three channel values ​​corresponding to the HSV color representation model corresponding to the remaining target pixel points from the standard image frame; The remaining target pixels are screened according to the three channel thresholds corresponding to the HSV color representation model and the three channel values ​​corresponding to the HSV color representation model corresponding to the remaining target pixels to obtain at least one second false point; The pixel value of each of the second false points is set to a second value, and the target pixel point of the second value is not processed.

8. The method according to claim 1, characterized in that According to the initial calibration line coordinates and the initial scale line coordinates, the current calibration line coordinates and the current scale line coordinates are corrected in real time to obtain accurate scale line coordinates, including: Identify two calibration lines on the forward flexion test panel from each image frame; Obtain the current calibration line coordinates of the two calibration lines from each image frame; Matching the acquired current calibration line coordinates with the initial calibration line coordinates, and calculating the geometric transformation matrix between the two; The current scale line coordinates are corrected according to the geometric transformation matrix and the initial scale line coordinates to obtain accurate scale line coordinates.

9. The method according to claim 1, characterized in that: According to the preset scoring rules and the finger touch position, the final score corresponding to each tester is automatically calculated, including: Substitute the abscissa x of the finger touch position into the following function: Among them, X i Indicates the horizontal coordinate position of the midpoint of the i+1th scale line; The final score automatically calculated for each test taker includes: Where T represents the actual spacing between adjacent tick marks.

10. An electronic device, characterized in that: The equipment includes: processor; and, A memory storing computer program instructions for a real-time detection and scoring method for sit-and-forth flexion implemented by a computer, wherein when the computer program instructions are executed by the processor, the electronic device executes the method according to any one of claims 1-9.

Images