Body skeleton key point detection method and system for moving person

Abstract

The invention relates to the technical field of image processing based on deep learning, in particular to a body skeleton key point detection method and system for a moving figure, and the method comprises the steps: determining a fuzzy region of each body part in a current frame image in a video in which a first user and a second user perform target movement; determining a weight value of a blurring kernel of a blurring region in the current frame image; recovering the definition of the blurred region in the current frame image according to the weight value of the blurred kernel of the blurred region in the current frame image to obtain a clear video frame of the current frame image; and taking the video of the second user as a standard video, and performing normative judgment on the action of performing the target motion by the first user according to the spatial distance difference between the body skeleton key points in the clear video frame of the first user and the clear video frame of the second user. The method improves the reliability of the detection result of the key points of the human skeleton.

Description

Technical Field

[0001] This invention relates to the field of deep learning-based image processing technology, specifically to a method and system for detecting key points of the body skeleton of a moving person. Background Technology

[0002] Freestyle rope skipping combines athleticism with artistic beauty, demanding a high degree of fluidity and aesthetic appeal in its movements. However, its diverse and complex movement system often leads to the neglect of proper form. Improper form significantly increases the risk of sports injuries, easily causing various health problems such as ankle ligament sprains, lumbar facet joint disorders, shoulder strain, and wrist tenosynovitis. Therefore, standardized testing of freestyle rope skipping movements is of significant practical importance.

[0003] In some scenarios, current mainstream methods for detecting the proper form of jump rope movements rely on video footage of professional athletes captured by cameras. This data is then used to extract the coordinates of key joints such as the wrists, ankles, and shoulders through a human pose estimation model. Finally, a spatiotemporal graph convolutional network or long short-term memory network is used to analyze the movement trajectory, ultimately determining the correct form. However, because the object being detected is in dynamic motion, single frames of the video are prone to motion blur, making it difficult to accurately identify key points of the human skeleton. This directly affects the accuracy of subsequent joint coordinate extraction and movement trajectory analysis, resulting in low reliability of the detection results for key points of the human skeleton. Summary of the Invention

[0004] To address the technical problem of low reliability in the detection results of key points of the human skeleton, the present invention aims to provide a method and system for detecting key points of the human skeleton for athletes.

[0005] To solve the above technical problems, the specific technical solution adopted is as follows: In a first aspect, embodiments of the present invention provide a method for detecting key points of the body skeleton of a moving person, comprising: determining blurred regions of each body part in a current frame image of a video of a first user and a second user performing a target movement; determining the weight value of the blur kernel of the blurred region in the current frame image based on the average gradient of each edge line in the blurred region, the slope between adjacent pixels on the edge line, and the gray value of corresponding pixels at any two different positions on the edge line; restoring the clarity of the blurred region in the current frame image based on the weight value of the blur kernel of the blurred region in the current frame image to obtain a clear video frame of the current frame image; and using the video of the second user as a standard video, performing a normative evaluation of the action of the first user performing the target movement based on the spatial distance difference between the key points of the body skeleton in the clear video frame of the first user and the clear video frame of the second user.

[0006] Optionally, determining the blurred regions of each body part in the current frame image of the video of the first and second users performing target motion includes: calculating the Laplacian variance of each video frame to obtain the overall Laplacian variance of each video frame; sorting the overall Laplacian variances of each video frame in ascending order to obtain a Laplacian variance sequence; subtracting adjacent overall Laplacian variances in the Laplacian variance sequence to obtain a difference sequence; selecting the position of the largest difference in the difference sequence, which divides the Laplacian variance sequence into two parts, and selecting the video frame corresponding to the left data sequence as the blurred video frame image; segmenting different body parts in the current frame image of the blurred video frame image to obtain multiple local blocks; calculating the Laplacian variance of each local block and the Euclidean distance between any two local blocks; and determining the blurred regions of each body part in the current frame image based on the Laplacian variance and Euclidean distance between any two local blocks of different body parts in the current frame image of the video.

[0007] Optionally, determining the blurred regions of each body part in the current frame image based on the Laplacian variance and Euclidean distance of any two local blocks of different body parts in the current frame image includes: calculating the first difference between the Laplacian variances of any two local blocks and the first sum between the Laplacian variances of any two local blocks; determining the blurred similarity between any two local blocks based on the first difference, the first sum, and the Euclidean distance; and clustering and merging any two local blocks based on the blurred similarity between them to obtain the blurred regions.

[0008] Optionally, determining the weight value of the blur kernel in the current frame image based on the average gradient of each edge line in the blurry region, the slope between adjacent pixels on the edge line, and the grayscale values ​​of corresponding pixels at any two different positions on the edge line includes: selecting a first target edge line of the blurry region based on the average gradient of all edge lines in the blurry region, wherein the average gradient of the first target edge line is greater than the average gradient of other edge lines in the blurry region; determining the second target edge line as a blur edge line of the first target edge line based on the slope between adjacent pixels of a second target edge line with the same length as the first target edge line in the blurry region and the slope between adjacent pixels of the first target edge line. The algorithm considers the following: Based on the grayscale values ​​of corresponding pixels at any two different positions on the first and second target edge lines, it determines the degree of consistency in the grayscale value changes of corresponding pixels on the first and second target edge lines; based on the probability and the degree of consistency, it determines the authenticity of the second target edge line as a blurred edge line of the first target edge line; based on the authenticity, it determines a set of blurred edge lines of the first target edge line, which includes at least one blurred edge line of the first target edge line; based on the blur amount between the blurred edge lines in the set of blurred edge lines and the authenticity of the blurred edge lines, it determines the weight value of the blur kernel of the blurred region in the current frame image.

[0009] Optionally, selecting the first target edge line of the fuzzy region based on the average gradient of all edge lines in the fuzzy region includes: sorting the average gradients of all edge lines in the fuzzy region in ascending order to obtain a gradient sequence; subtracting adjacent average gradients in the gradient sequence to obtain a gradient difference sequence; determining the position of the maximum value in the gradient difference sequence; dividing the gradient sequence into two parts at the position of the maximum value, and taking the edge line corresponding to the gradient sequence on the right side of the position as the first target edge line of the fuzzy region.

[0010] Optionally, determining the probability that the second target edge line is a blurred edge line of the first target edge line based on the slope between adjacent pixels of the second target edge line with the same length as the first target edge line within the blurred region and the slope between adjacent pixels of the first target edge line includes: calculating the absolute value of a second difference between the slope between adjacent pixels of the second target edge line and the slope between corresponding adjacent pixels of the first target edge line; and determining the probability that the second target edge line is a blurred edge line of the first target edge line based on the absolute value of the second difference between the slopes of each adjacent pixel.

[0011] Optionally, determining the consistency of the grayscale value changes of corresponding pixels at any two different positions on the first target edge line and the second target edge line, based on the grayscale values ​​of corresponding pixels at any two different positions on the first target edge line and the second target edge line, includes: determining the first change in grayscale value between corresponding pixels at any two different positions on the first target edge line and the second target edge line; determining the absolute value of the third difference in the first change in grayscale value between corresponding pixels at any two different positions; and determining the consistency of the grayscale value changes based on the absolute value of the third difference in the first change in grayscale value between corresponding pixels at any two different positions.

[0012] Optionally, determining the weight value of the blur kernel of the blurred region in the current frame image based on the blur amount between the blurred edge lines in the blurred edge line set and the realism corresponding to the blurred edge lines includes: determining the gradient difference between the average gradient of the first target edge line and the average gradient of any blurred edge line in the blurred edge line set as the blur amount; determining the motion direction of the first target edge line based on the displacement change of the body part where the first target edge line is located in the current frame image in adjacent frame images; and determining the weight value of the blur kernel of the blurred region in the current frame image in the current motion direction based on the blur amount between the average gradient of the first target edge line in the current motion direction and any blurred edge line in the blurred edge line set, as well as the realism corresponding to the blurred edge lines.

[0013] Optionally, restoring the clarity of the blurred regions in the current frame image based on the weight values ​​of the blur kernels of the blurred regions in the current frame image to obtain a clear video frame of the current frame image includes: restoring the clarity of the blurred regions in the current frame image using a non-blind deblurring algorithm based on the positional distribution of the blur kernels of each blurred region in the current frame image and their corresponding weight values ​​to obtain a clear video frame of the current frame image.

[0014] In a second aspect, embodiments of the present invention provide a system for detecting key points of the body skeleton of a moving person, comprising: a processor and a memory; wherein the memory is used to store a computer program that can run on the processor; the processor is used to execute the program stored in the memory to implement the steps of the method for detecting key points of the body skeleton of a moving person as mentioned in the first aspect.

[0015] This invention offers the following advantages: The embodiments of this invention clearly identify blurred areas of various body parts in the current frame of a video, specifically focusing on key areas prone to blurring under dynamic motion. This provides precise targeting for subsequent sharpening processes, avoiding the problem of skeletal key point identification deviation caused by blurring at the source. Then, by combining the average gradient of edge lines, the slope of adjacent pixels, and multi-dimensional features of pixel grayscale values ​​at different positions, a blur kernel is calculated to comprehensively characterize the blur characteristics. This makes the clarity restoration of blurred areas more consistent with the actual scene, significantly improving the quality of clear video frames and providing a high-quality image foundation for key skeletal point coordinate extraction. By restoring the clarity of blurred areas through the blur kernel, the impact of motion blur on single-frame images is effectively reduced, ensuring accurate identification of human skeletal key points, improving the accuracy of joint coordinate extraction and motion trajectory analysis, and significantly enhancing the reliability of human skeletal key point detection results. Finally, using the clear video of the second user as a standard, the spatial distance difference between the skeletal key points in the first user's video and the standard video is calculated for evaluation. This aligns with the core requirements of freestyle jump rope for smoothness, aesthetics, and standardization of movements, effectively reducing the risk of sports injuries caused by non-standard movements and strengthening the detection of standardized movements. Attached Figure Description

[0016] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating a method for detecting key points of the body skeleton of a moving person, as provided in one embodiment of the present invention; Figure 2 This is a schematic diagram of a key point detection system for the body skeleton of a moving person, provided in one embodiment of the present invention. Detailed Implementation

[0018] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a method and system for detecting key points of the body skeleton of a moving person according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0020] The main objective of this invention is to identify key points of the body skeleton in each frame of a jump rope during the detection of fancy rope skipping movements. This requires uploading jump rope videos recorded by both the user and a standard athlete to the system. However, since the person is in motion during jump rope, motion blur is easily generated. Blurred images make it difficult to identify key points of the body skeleton. Therefore, it is necessary to deblur the blurred frames in the video to accurately identify the key points of the jumper's body skeleton.

[0021] The scenario addressed in this invention is as follows: A user uploads one or more videos of standard freestyle rope skipping demonstrated by professional athletes. The system performs human posture recognition on each frame of the video and extracts the coordinates of key points of the body skeleton, recording them as standard template points. The user uploads their own corresponding freestyle rope skipping video, and the system extracts the user's key points in the same way, comparing them frame-by-frame with the template. It then provides an objective, reproducible evaluation and scoring of the user's rope skipping technique, along with visual feedback.

[0022] The following describes in detail, with reference to the accompanying drawings, a specific scheme for a method of detecting key points of the body skeleton of a moving person provided by the present invention.

[0023] Example 1: Please see Figure 1 The flowchart illustrates a method for detecting key points of the body skeleton of a moving person according to an embodiment of the present invention, including: Step S101: Determine the blurred areas of each body part in the current frame image of the video of the first user and the second user performing the target motion.

[0024] Specifically, in this embodiment of the invention, the video of the second user can be a standard video of a professional athlete. The target sport can be fancy rope skipping, etc. The first user is the user who performs the standard evaluation of the fancy rope skipping movements to be performed. This embodiment of the invention acquires a standard fancy rope skipping video demonstrated by a professional athlete and a rope skipping video recorded by the user. The human posture is identified frame by frame in each of the two videos. If the uploaded video is too dark, red and blue channel gain + histogram equalization and high ISO noise reduction are automatically enabled to process each frame of the video image to obtain a well-lit video image. Each frame of the video image is converted into a grayscale image and smoothed and denoised. The obtained image is used for subsequent analysis.

[0025] Furthermore, during rope skipping, the person is in motion, resulting in blurred frames in the video due to body movement. For images with clear motion, the system can automatically detect key points of the human skeleton; however, it is difficult to accurately identify these key points in blurred frames. Therefore, it is necessary to locate strong edges (the first target edge) in the blurred image. Based on the characteristics that the selected strong edge has the same length as its blurred edge (same number of pixels), the slope values ​​calculated for two adjacent pixels at corresponding positions are the same, and the gray value change of any two corresponding pixels on the edge is the same as the gray value change between two other corresponding pixels, all blurred edges corresponding to the strong edge can be found in all directions. Averaging the blur amount of the blurred edges in all directions yields the blur amount of the blurred region in all directions. By setting the weight of the blur kernel in each direction according to the magnitude of the blur amount, the blur kernel of the blurred region can be obtained. A non-blind deblurring algorithm can then be used to restore the clarity of the blurred frame image.

[0026] Furthermore, the core basis for blur assessment of single-frame images in a video is the richness of edge and texture details within each frame. Blurry images, due to smoothed edges and loss of detail, can cause the detection of key points on the body skeleton to fail. Therefore, all frames in the video need to be screened to distinguish between clear and blurry frames. Clear frames can be directly used for key point detection on the body skeleton; while blurry frames need to be deblurred first to restore their image quality before key point recognition, thus improving the accuracy of subsequent analysis for determining the correctness of jump rope movements. Therefore, all images in the video need to be screened to obtain all blurry frame images. Therefore, as an optional embodiment of the present invention, determining the blurred regions of each body part in the current frame image of the video in which the first user and the second user are performing target motion includes: calculating the Laplacian variance of each video frame in the video to obtain the overall Laplacian variance of each video frame; sorting the overall Laplacian variances of each video frame in ascending order to obtain a Laplacian variance sequence; subtracting adjacent overall Laplacian variances in the Laplacian variance sequence to obtain a difference sequence; selecting the position of the largest difference in the difference sequence, which divides the Laplacian variance sequence into two parts, and selecting the video frame corresponding to the left data sequence as the blurred video frame image; segmenting different body parts in the current frame image of the blurred video frame image to obtain multiple local blocks; calculating the Laplacian variance of each local block and the Euclidean distance between any two local blocks; and determining the blurred regions of each body part in the current frame image based on the Laplacian variance and Euclidean distance between any two local blocks of different body parts in the current frame image of the video.

[0027] Specifically, this embodiment of the invention uses an existing Laplacian operator to calculate the overall Laplacian variance of each video frame. A larger overall Laplacian variance indicates a more pronounced gradient change in the video frame image, meaning a clearer image; conversely, a smaller overall Laplacian variance indicates a smoother gradient change in the image, meaning a blurrier image. Taking a complete video as an example, the Laplacian variance is calculated for each frame of the video. The calculated Laplacian variances are arranged in ascending order to obtain a Laplacian variance sequence. Then, the difference between any two adjacent overall Laplacian variance values ​​in the sequence is calculated (i.e., the latter overall Laplacian variance minus the former). All differences are iterated through until the position of the maximum difference is located. This position divides the entire Laplacian variance sequence into two parts. The data sequence on the left is selected and marked. The video frame corresponding to each data point (overall Laplacian variance) is the blurred video frame image. Subsequent deblurring processing can then be performed on these blurred video frame images.

[0028] Furthermore, the movements in fancy rope skipping are far more diverse and complex than those in regular rope skipping. When evaluating the standardization of fancy rope skipping movements, the overall movement can be broken down into key parts such as the torso, arms, legs, and feet for independent analysis due to the different movement directions of each part. However, for each segmented part, the different movement speeds in different areas within each frame may lead to varying degrees of blurring in different areas of a particular part during high-speed shooting (there may also be situations where some areas are blurry while others are clear, i.e., localized blurring occurs in certain areas. For example, during rope skipping, the forearm and hand move rapidly, while other parts of the body move relatively less; therefore, during image capture, the forearm and hand may be blurry while other parts of the body are clear). Therefore, this embodiment of the invention inputs a large number of fancy rope skipping images labeled with different movements of various body parts into a neural network for model training. The trained neural network model can identify and label the regions containing various body parts in newly input blurred frames of fancy rope skipping images. For the input blurred frames, not all regions are blurred; therefore, further analysis of the regions containing each body part is required.

[0029] Furthermore, this embodiment of the invention takes the arm region of each segmented video frame as an example for analysis. Taking any frame as an example, the arm region image of that frame is divided into 7×7 local blocks. The Laplacian variance of each local block is calculated to obtain a sharpness value map corresponding to the size of the original arm region image. The larger the Laplacian variance of the local block, the sharper the region. Based on the sharpness of the region and the distance between regions, the blurred regions are clustered to obtain the blurred regions of the arm region image. Therefore, as an optional embodiment of the invention, determining the blurred regions of each body part in the current frame image based on the Laplacian variance and Euclidean distance of any two local blocks of different body parts in the current frame image includes: calculating the first difference between the Laplacian variances of any two local blocks and the first sum between the Laplacian variances of any two local blocks; determining the blurriness similarity between any two local blocks based on the first difference, the first sum, and the Euclidean distance; and clustering and merging any two local blocks based on the blurriness similarity to obtain the blurred regions.

[0030] Specifically, the embodiments of the present invention take the first... The local block and the first Taking a local block as an example, the following formula is used to calculate the first local block. The local block and the first Fuzzy similarity between local blocks: In the above formula, Indicates the first The local block and the first Fuzzy similarity between local blocks. Indicates the first The Laplace variance of a local block. Indicates the first The Laplace variance of a local block. Indicates the first The local block and the first The normalized value of the Euclidean distance between local blocks. Wherein, Indicates the first The local block and the first The similarity of the Laplace variances between local blocks; the smaller the value, the more similar the Laplace variances of the two regions. This indicates the sharpness of two local blocks; the larger the value, the sharper local blocks i and j are.

[0031] Furthermore, the blur similarity (A) between all local blocks is calculated. The larger the A value, the closer the variances of the two calculated local blocks are, and the more blurred the two local blocks are on the current frame image, and the closer their blur levels are. The calculated two local blocks can be clustered into one class using the K-means clustering algorithm. That is, n blurred regions on the arm region image of the current frame are obtained.

[0032] Step S102: Determine the weight value of the blur kernel in the current frame image based on the average gradient of each edge line in the blurred region, the slope between adjacent pixels on the edge line, and the gray value of corresponding pixels at any two different positions on the edge line.

[0033] Specifically, for each of the identified blurred regions, the Sobel operator is used to detect the edge lines in the blurred regions, and the average gradient of all edge lines is calculated. Further, as an optional embodiment of the present invention, determining the weight value of the blur kernel of the blurred region in the current frame image based on the average gradient of each edge line in the blurred region, the slope between adjacent pixels on the edge line, and the grayscale values ​​of corresponding pixels at any two different positions on the edge line includes: selecting a first target edge line of the blurred region based on the average gradient of all edge lines in the blurred region, wherein the average gradient of the first target edge line is greater than the average gradient of other edge lines in the blurred region; determining the second target edge line as the first target edge line based on the slope between adjacent pixels of a second target edge line with the same length as the first target edge line in the blurred region and the slope between adjacent pixels of the first target edge line. The probability of blurring edge lines is determined; based on the grayscale values ​​of corresponding pixels at any two different positions on the first and second target edge lines, the consistency of the grayscale value changes of corresponding pixels on the first and second target edge lines is determined; based on the probability and consistency, the authenticity of the second target edge line as a blurring edge line of the first target edge line is determined; based on the authenticity, a set of blurring edge lines of the first target edge line is determined, the set of blurring edge lines including at least one blurring edge line of the first target edge line is determined; based on the blur amount between the blurring edge lines in the set of blurring edge lines and the authenticity of the blurring edge lines, the weight value of the blur kernel of the blurring region in the current frame image is determined.

[0034] Specifically, the first target edge line can be a strong edge line in the fuzzy region. Further, as an optional embodiment of the present invention, selecting the first target edge line of the fuzzy region based on the average gradient of all edge lines in the fuzzy region includes: sorting the average gradients of all edge lines in the fuzzy region in ascending order to obtain a gradient sequence; subtracting adjacent average gradients in the gradient sequence to obtain a gradient difference sequence; determining the position of the maximum value in the gradient difference sequence; dividing the gradient sequence into two parts at the position of the maximum value, and taking the edge line corresponding to the gradient sequence on the right side of the position as the first target edge line of the fuzzy region.

[0035] Specifically, in this embodiment of the invention, the average gradient values ​​of all edge lines are arranged in ascending order to obtain a new data sequence. The difference between two adjacent data (i.e., the next data minus the previous data) is calculated, and all differences are traversed to locate the position with the largest difference. At the position with the largest difference, the gradient sequence is divided into two parts, and the right data (the side with the larger average gradient) is marked. The corresponding edge line is the strong edge line in the fuzzy area, i.e., the first target edge line.

[0036] Furthermore, in the image of the arm region, the motion direction of all edge lines should be consistent. Find any clear edge line in the image of this arm region (e.g., the upper arm moves slowly, allowing for a clear edge to be captured during movement). By tracking the displacement of this clear edge line in the arm region image between adjacent image frames, the motion direction of all strong edge lines in the blurred region can be calculated (the motion direction calculated within a blurred region applies to all strong edge lines within that region). Performing the above operation on other segmented areas yields the strong edge lines and their motion directions for all body parts within any blurred region of that body part.

[0037] Furthermore, the blurring produced by skipping rope is motion blur. Even if the direction of motion in the blurred region has been determined, strong edge lines may still be blurred in all directions. Moreover, the blurred edge and its blurred edge should be parallel and of equal length. This embodiment of the invention takes a strong edge line in a single blurred region as an example. This edge line may be blurred in all directions. The analysis is performed on this strong edge line and all edge lines within its blurred region. As an optional embodiment of the invention, determining the probability that the second target edge line is a blurred edge line of the first target edge line, based on the slope between adjacent pixels of a second target edge line with the same length as the first target edge line within the blurred region and the slope between adjacent pixels of the first target edge line, includes: calculating the absolute value of a second difference between the slope between adjacent pixels of the second target edge line and the slope between corresponding adjacent pixels of the first target edge line; and determining the probability that the second target edge line is a blurred edge line of the first target edge line based on the absolute value of the second difference between the slopes of each adjacent pixel.

[0038] Specifically, in this embodiment of the invention, the length of the selected strong edge (first target edge line) is denoted as... Its length is equal to the number of pixels on the edge of the first target. A slope can be calculated between any two adjacent pixels on this strong edge. The slope calculated on this strong edge is then arranged according to { , Sort them. This represents the slope between the first pair of adjacent pixels on a strong edge line. This represents the slope between the second pair of adjacent pixels on the strong edge line. Indicates the first on the strong edge line The slope between two adjacent pixels; filtering out all pixels within the blurred area that have a length similar to the selected strong edge line. Equal second target edge lines can yield the same number of slopes. , This represents the slope between the first pair of adjacent pixels on the edge line of the second target. This represents the slope between the second pair of adjacent pixels on the edge line of the second target. Indicates the second target edge line. The slope between two adjacent pixels is used to calculate the probability that the second target edge line is a blurred edge line of the first target edge line: In the above formula, Indicates the first within the fuzzy region The possibility that the edge line of the second target is a blurred edge line of the edge line of the first target. The first pixel segmentation is performed on adjacent pixels along the edge of a first target within any blurred region. The slope of each line segment. Let the second target edge line be any second target edge line within the same fuzzy region that has the same length as the first target edge line, and let the second target edge line be the first target edge line corresponding to the first target edge line. The slope of each line segment. u is the number of pixels on the edge line of the first target. This is a constant between (0, 0.1) to prevent the denominator from being 0; for example, a value of 0.001. The larger the value, the more likely the c-th second target edge line (the edge line with the same length as the selected first target edge line) within the blurred area is to be a blurred edge line of the selected strong edge.

[0039] Furthermore, the grayscale value change of all corresponding pixels on the selected strong edge and its blurred edge line should be consistent. In this embodiment of the invention, the grayscale value change of any corresponding pixel position on the first target edge line and the second target edge line is denoted as... In one optional embodiment of the present invention, determining the degree of consistency of the grayscale value changes of corresponding pixels at any two different positions on the first target edge line and the second target edge line, based on the grayscale values ​​of corresponding pixels at any two different positions on the first target edge line and the second target edge line, includes: a first change in grayscale value between corresponding pixels at any two different positions on the first target edge line and the second target edge line; determining the absolute value of a third difference in the first change in grayscale value between corresponding pixels at any two different positions; and determining the degree of consistency of the grayscale value changes based on the absolute value of the third difference in the first change in grayscale value between corresponding pixels at any two different positions.

[0040] Specifically, the embodiments of the present invention employ the following formula to calculate the first fuzzy region. The degree of consistency between the grayscale value changes of all corresponding pixels on the edge line of the second target and the corresponding edge line of the first target: In the above formula, Indicates the first within the fuzzy region The degree of consistency between the grayscale value changes of all corresponding pixels on the edge line of the second target and the edge line of the corresponding first target. and This is represented as any two different positions (e and s) on the edge of the first target and the first target within the same fuzzy region. The amount of grayscale value change between corresponding pixels on the edge line of the second target. Let x be a constant between (0, 0.1) to prevent the denominator from being 0, such as taking a value of 0.001. The quantity.

[0041] Furthermore, in this embodiment of the invention, the following formula is used to calculate the first fuzzy region. The authenticity of the blurred edge line where the edge of the second target is the edge of the first target: In the above formula, The calculated fuzzy area is determined by the authenticity of the c-th second target edge line belonging to the fuzzy edge line corresponding to the selected first target edge line. Indicates the first within the fuzzy region The degree of consistency between the grayscale value changes of all corresponding pixels on the edge line of the second target and the edge line of the corresponding first target. Indicates the first within the fuzzy region The probability that the c-th second target edge line is a blurred edge line of the first target edge line. The greater the probability that the c-th second target edge line is a blurred edge line of the first target edge line, and the higher the consistency of the gray value changes of all corresponding pixels on the second target edge line and the first target edge line, the stronger the authenticity of the c-th second target edge line belonging to the blurred edge line corresponding to the selected first target edge line.

[0042] Furthermore, within a certain fuzzy region, all second target edge lines with the same length as the selected first target edge line can have their fuzzy edge line realism values ​​calculated. Arranging all realism values ​​in ascending order yields a realism sequence. Then, subtracting any two adjacent realism values ​​(i.e., subtracting the previous value from the next) is performed, iterating through all differences to locate the position of the largest difference. This position divides the entire realism sequence into two parts; the right-hand data sequence represents the set of fuzzy edge lines with high realism among all fuzzy edge sets within the fuzzy region. Analyzing these high-realism fuzzy edge lines reveals a fuzziness quantity (H) between any fuzzy edge line and the first target edge line (the gradient difference between the first target edge line and any fuzzy edge line is the fuzziness quantity). The selected first target edge line may have multiple real fuzzy edge lines in each direction. A fuzziness quantity can be calculated for any fuzzy edge line in each direction. Averaging all fuzziness quantities in a certain direction yields the fuzziness quantity in that direction; this process is repeated for fuzziness quantities in other directions.

[0043] Furthermore, within the same blurred region, the blurring amount of all edge lines is the same. Therefore, the calculated blurring amount of strong edges in each direction is the blurring amount of the blurred region in each direction. The weight of the blur kernel of the blurred region is set according to the magnitude of the blurring amount in each direction, with higher weights assigned to directions with larger blurring amounts and lower weights assigned to directions with smaller blurring amounts. Therefore, as an optional embodiment of the present invention, determining the weight value of the blur kernel of the blurred region in the current frame image based on the blurring amount between the blurred edge lines in the blurred edge line set and the realism corresponding to the blurred edge lines includes: determining the gradient difference between the average gradient of the first target edge line and the average gradient of any blurred edge line in the blurred edge line set as the blurring amount; determining the motion direction of the first target edge line based on the displacement change of the body part where the first target edge line is located in the current frame image in adjacent frame images; and determining the weight value of the blur kernel of the blurred region in the current frame image in the current motion direction based on the average gradient of the first target edge line in the current motion direction, the blurring amount between the first target edge line and any blurred edge line in the blurred edge line set, and the realism corresponding to the blurred edge lines.

[0044] Specifically, in this embodiment of the invention, the weight value of the blur kernel of the blurred region in the current frame image in the current motion direction is calculated using the following formula: In the above formula, This represents the weight value of the fuzzy kernel in the m-th direction of motion for the fuzzy region. Let be the blurring amount between the a-th blurry edge line with high realism among all blurry edge line sets in the m-th direction and the strong edge lines in that blurry region. Assume there are V blurry edge lines in the selected set of high realism blurry edge lines. Indicates the calculated fuzzy region of the first... The authenticity of the second target edge line as it belongs to the fuzzy edge line corresponding to the selected first target edge line. This represents the normalization function.

[0045] Step S103: Recover the clarity of the blurred region in the current frame image based on the weight value of the blur kernel in the blurred region of the current frame image to obtain the clear video frame of the current frame image.

[0046] Specifically, in this embodiment of the invention, based on the positional distribution of the blur kernels of each blurry region in the current frame image and their corresponding weight values, a non-blind deblurring algorithm is used to determine the clarity of the blurry regions in the current frame image, thereby obtaining a clear video frame of the current frame image.

[0047] Specifically, this embodiment of the invention first clarifies the distribution characteristics and weight assignments of the blur kernel positions corresponding to each blurred region in the current frame image. Then, relying on the advantages of non-blind deblurring algorithms, it performs targeted processing based on the characteristics of different blurred regions and their blur kernel parameters. The algorithm precisely optimizes and enhances the clarity of the blurred regions, effectively counteracting the occlusion of image details by the blur effect, and restoring the original texture, contours, and other key information of the image to the greatest extent possible. Finally, it generates a high-quality, clear video frame corresponding to the current frame image, providing a clear image foundation for subsequent video processing or playback.

[0048] Thus, by performing the above operations on all blurred images, clear images of all blurred video frames can be obtained. Arranging these clear images and the clear images in the original video frames in a time sequence can yield a clear video of the original fancy rope skipping.

[0049] Step S104: Using the second user's video as the standard video, and based on the spatial distance difference between the key points of the body skeleton in the clear video frames of the first user and the clear video frames of the second user, the action of the first user performing the target movement is evaluated in a standardized manner.

[0050] Specifically, in this embodiment of the invention, OpenPose (BODY_25) is used to obtain all the key points of the jumper's body skeleton in each video frame after the first and second users' clear video frames have been deblurred. After deblurring each video frame, the hand area moving during the jump rope process can be clearly observed. There are many key points in the hands, and the hand movements in fancy rope skipping are very rich. Therefore, the position of the arm key points plays an important role in the evaluation of the standardization of fancy rope skipping movements. In this embodiment of the invention, 21 fine-grained key points of the hands can be obtained through MediaPipe Hands. After obtaining the key points of the hands and the key points of the body skeleton, the positions of the key points in all video frames of the user and the athlete are compared, so that the user's fancy rope skipping movements can be judged according to standards (the second user's movements are considered standard movements). If the spatial distance difference between key points (using the human body normalized coordinates to compare the distance difference between each key point identified by the user and the athlete and the normalized coordinates) is greater than a threshold, it indicates that the user's fancy rope skipping movements in that frame are not standard. The threshold can be determined based on historical experience and the accuracy requirements of different scenarios. In this embodiment of the invention, the value is 2cm.

[0051] This invention explicitly identifies blurred regions of various body parts in the current frame of a video, specifically focusing on key areas prone to blurring under dynamic motion. This provides precise targeting for subsequent sharpening processes, avoiding the problem of skeletal key point identification deviation caused by blurring at the source. Then, a blur kernel is calculated by combining the average gradient of edge lines, the slope of adjacent pixels, and the grayscale values ​​of pixels at different positions—multi-dimensional features—to comprehensively characterize blur characteristics. This makes the clarity restoration of blurred regions more consistent with the actual scene, significantly improving the quality of clear video frames and providing a high-quality image foundation for key skeletal point coordinate extraction. By restoring the clarity of blurred regions through the blur kernel, the impact of motion blur on single-frame images is effectively reduced, ensuring accurate identification of human skeletal key points, improving the accuracy of joint coordinate extraction and motion trajectory analysis, and significantly enhancing the reliability of human skeletal key point detection results. Finally, using the clear video of a second user as a standard, the spatial distance difference between the skeletal key points in the first user's video and the standard video is calculated for evaluation. This aligns with the core requirements of freestyle jump rope for smoothness, aesthetics, and standardization of movements, effectively reducing the risk of sports injuries caused by non-standard movements and strengthening the detection of standardized movements.

[0052] Furthermore, this embodiment of the invention can filter out blurred frame images in the video by calculating the Laplacian variance of each frame. Multiple blurred regions can be obtained by clustering the segmented local blocks. Based on the characteristic that strong edges have high gradients within the region, all strong edges in the blurred region can be filtered out. By calculating the slope between adjacent pixel values ​​of all edge lines of the same length as the strong edge line within the same blurred region and the consistency of the grayscale value changes of all corresponding pixels of the two edge lines, all blurred edges with high realism can be obtained. Averaging the blur amount of the blurred edges in all directions yields the blur amount of the blurred region in all directions. By setting the weight of the blur kernel in each direction according to the magnitude of the blur amount, the blur kernel of the blurred region can be obtained. Then, a non-blind deblurring algorithm can be used to obtain a clear image of the region.

[0053] Example 2: Corresponding to the above-described method for detecting key points of a sports skeleton, based on the same technical concept, this invention also provides a system for detecting key points of a sports skeleton. This system is used to execute the above-described method for detecting key points of a sports skeleton. Figure 2 This is a schematic diagram of another body skeleton key point detection system for moving figures provided in one embodiment of the present invention, as shown below. Figure 2As shown. A body skeleton key point detection system for moving figures can vary considerably due to differences in configuration or performance. It may include one or more processors 201 and memory 202. The memory 202 stores computer programs that can run on the processor 201. The processor 201 executes the programs stored in the memory 202 to achieve the above... Figure 1 The various steps in the method embodiment are described. The memory 202 can be temporary or persistent storage. The application stored in the memory 202 may include one or more modules (not shown in the figures), each module including a series of computer-executable instructions for a body skeleton keypoint detection system for moving human figures.

[0054] Furthermore, the processor 201 can be configured to communicate with the memory 202 and execute a series of computer-executable instructions in the memory 202 on the body skeleton keypoint detection system for moving figures. The body skeleton keypoint detection system for moving figures may also include one or more power supplies 203, one or more wired or wireless network interfaces 204, one or more input / output interfaces 205, and one or more keyboards 206.

[0055] Specifically, in this embodiment, the body skeleton key point detection system for moving figures includes a processor, a communication interface, a memory, and a communication bus; wherein, the processor, communication interface, and memory communicate with each other via the bus; the memory is used to store computer programs; and the processor is used to execute the programs stored in the memory to achieve the above... Figure 1 The various steps in the method embodiments are the same as those in the above method embodiments, and have the same beneficial effects. To avoid repetition, the embodiments of the present invention will not be described again here.

[0056] It should be noted that the body skeleton key point detection system for athletes provided in this embodiment of the invention and the body skeleton key point detection method for athletes provided in this embodiment of the invention are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned body skeleton key point detection method for athletes, and has the same or similar beneficial effects. Repeated parts will not be described again.

[0057] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0058] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0059] This invention also proposes a computer-readable storage medium storing one or more programs, which, when executed by a motion-oriented human skeleton keypoint detection system including multiple applications, cause the motion-oriented human skeleton keypoint detection system to perform... Figure 1 The methods disclosed in the embodiments shown achieve the functions and beneficial effects of the methods in the preceding method embodiments, and will not be repeated here.

[0060] The computer-readable storage media include read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for detecting key points of the body skeleton of a moving person, characterized in that, include: Identify the blurred regions of each body part in the current frame image of the video of the first and second users performing target motion; The weight value of the blur kernel in the current frame image is determined based on the average gradient of each edge line in the blurred region, the slope between adjacent pixels on the edge line, and the gray value of corresponding pixels at any two different positions on the edge line. The clarity of the blurred region in the current frame image is recovered based on the weight value of the blur kernel of the blurred region in the current frame image, and a clear video frame of the current frame image is obtained; Using the video of the second user as the standard video, the actions of the first user performing the target movement are evaluated in a standardized manner based on the spatial distance difference between the key points of the body skeleton in the clear video frames of the first user and the clear video frames of the second user.

2. The method for detecting key points of the body skeleton of a moving person according to claim 1, characterized in that, The blurred regions of each body part in the current frame image of the video in which the first user and the second user are performing target motion include: The Laplacian variance is calculated for each video frame in the video to obtain the overall Laplacian variance for each video frame; The overall Laplacian variance of each video frame is sorted in ascending order to obtain the Laplacian variance sequence. The difference sequence is obtained by subtracting adjacent global Laplace variances in the Laplace variance sequence. Select the position where the maximum difference is located in the difference sequence. The position divides the Laplacian variance sequence into two parts. Select the video frame corresponding to the data sequence on the left as the blurred video frame image. The blurred video frame image is segmented into different body parts in the current frame image to obtain multiple local blocks; Calculate the Laplace variance of each local block and the Euclidean distance between any two local blocks; Based on the Laplacian variance and Euclidean distance of any two local blocks of different body parts in the current frame image of the video, determine the blurred regions of each body part in the current frame image.

3. The method for detecting key points of the body skeleton of a moving person according to claim 2, characterized in that, The step of determining the blurred regions of each body part in the current frame image based on the Laplacian variance and Euclidean distance of any two local blocks of different body parts in the current frame image of the video includes: Calculate the first difference between the Laplace variances of any two local blocks, and the first sum between the Laplace variances of any two local blocks; The fuzzy similarity between any two local blocks is determined based on the first difference, the first sum, and the Euclidean distance. The fuzzy region is obtained by clustering and merging the two local blocks based on the fuzzy similarity between them.

4. The method for detecting key points of the body skeleton of a moving person according to claim 1, characterized in that, The step of determining the weight value of the blur kernel in the current frame image based on the average gradient of each edge line in the blurred region, the slope between adjacent pixels on the edge line, and the gray value of corresponding pixels at any two different positions on the edge line includes: The first target edge line of the fuzzy region is selected based on the average gradient of all edge lines in the fuzzy region, and the average gradient of the first target edge line is greater than the average gradient of other edge lines in the fuzzy region. Based on the slope between adjacent pixels of the second target edge line with the same length as the first target edge line within the blurred region and the slope between adjacent pixels of the first target edge line, the probability that the second target edge line is a blurred edge line of the first target edge line is determined. Based on the gray values ​​of corresponding pixels at any two different positions on the first target edge line and the second target edge line, determine the degree of consistency of the gray value changes of corresponding pixels on the first target edge line and the second target edge line; Based on the probability and the degree of consistency, the authenticity of the second target edge line as the fuzzy edge line of the first target edge line is determined. Based on the realism, a set of fuzzy edge lines for the first target edge line is determined, wherein the set of fuzzy edge lines includes at least one fuzzy edge line of the first target edge line. The weight value of the blur kernel of the blurred region in the current frame image is determined based on the amount of blur between the blurred edge lines in the set of blurred edge lines and the realism of the blurred edge lines.

5. The method for detecting key points of the body skeleton of a moving person according to claim 4, characterized in that, The step of selecting the first target edge line of the blurred region based on the average gradient of all edge lines in the blurred region includes: Sort the average gradients of all edge lines in the blurred region in ascending order to obtain the gradient sequence; Subtracting the average gradients of adjacent gradients in the gradient sequence yields a gradient difference sequence; Determine the location of the maximum value in the gradient difference sequence; The location of the maximum value divides the gradient sequence into two parts, and the edge line corresponding to the gradient sequence on the right side of the location is taken as the first target edge line of the blurred region.

6. The method for detecting key points of the body skeleton of a moving person according to claim 4, characterized in that, The possibility of determining that the second target edge line is a blurred edge line of the first target edge line based on the slope between adjacent pixels of the second target edge line with the same length as the first target edge line within the blurred region and the slope between adjacent pixels of the first target edge line includes: Calculate the absolute value of a second difference between the slope between adjacent pixels of the second target edge line and the slope between corresponding adjacent pixels of the first target edge line; The probability that the second target edge line is a blurred edge line of the first target edge line is determined based on the absolute value of the second difference between the slopes of each of the adjacent pixels.

7. The method for detecting key points of the body skeleton of a moving person according to claim 4, characterized in that, The step of determining the consistency of the grayscale value changes of corresponding pixels on the first target edge line and the second target edge line based on the grayscale values ​​of corresponding pixels at any two different positions on the first target edge line and the second target edge line includes: The first change in grayscale value between corresponding pixels at any two different positions on the first target edge line and the second target edge line; Determine the absolute value of the third difference of the first change in the grayscale values ​​of corresponding pixels at any two different positions; The degree of consistency of the gray value change is determined based on the absolute value of the third difference of the first change between the gray values ​​of corresponding pixels at any two different positions.

8. The method for detecting key points of the body skeleton of a moving person according to claim 4, characterized in that, The step of determining the weight value of the blur kernel of the blurred region in the current frame image based on the blur amount between the blurred edge lines in the set of blurred edge lines and the realism corresponding to the blurred edge lines includes: The gradient difference between the average gradient of the first target edge line and the average gradient of any blurred edge line in the set of blurred edge lines is defined as the fuzziness quantity. The direction of motion of the first target edge line is determined based on the displacement change of the body part where the first target edge line is located in the current frame image in adjacent frame images; Based on the average gradient of the first target edge line in the current motion direction and the blur amount between any blurry edge line in the set of blurry edge lines, as well as the realism corresponding to the blurry edge line, the weight value of the blur kernel of the blurry region in the current frame image in the current motion direction is determined.

9. The method for detecting key points of the body skeleton of a moving person according to claim 1, characterized in that, The step of restoring the sharpness of the blurred region in the current frame image based on the weight value of the blur kernel of the blurred region in the current frame image to obtain the sharp video frame of the current frame image includes: Based on the position distribution of the blur kernels of each blurry region in the current frame image and the corresponding weight values, a non-blind deblurring algorithm is used to restore the clarity of the blurry regions in the current frame image, thereby obtaining a clear video frame of the current frame image.

10. A system for detecting key points of the human skeleton in motion, characterized in that, include: Processor and memory; wherein the memory is used to store computer programs that can run on the processor; A processor is used to execute a program stored in memory to implement the steps of the method for detecting key points of the body skeleton of a moving human as described in any one of claims 1-9.

Images