Method and system for recognizing spiking action of volleyball player

Abstract

This application provides a method and system for recognizing volleyball player spiking actions. The method involves acquiring multiple consecutive frames of depth and visible light images of the athlete's upper limbs. Interference is assessed in each frame of the visible light image. When motion blur and light reflection interference exist, the visible light image of the current interfering frame is decomposed into a high-frequency detail layer and a low-frequency main body layer. Interference regions in the high-frequency detail layer are then filtered out based on the depth edge information of the depth image in the same frame, while edge-preserving filtering is applied to the low-frequency main body layer. The high-frequency detail layer and the low-frequency main body layer, after interference removal, are reconstructed into a corrected image, resulting in multiple frames of corrected images. The three-dimensional motion trajectory and angle change features of the elbow, shoulder, and wrist joints are extracted from each frame of corrected images, and the method outputs whether it is a standard spiking action and the action confidence level. Based on this scheme, adaptive removal of motion blur and reflection interference can be achieved.

Description

Technical Field

[0001] This application relates to the field of motion recognition technology, and more specifically, to a method and system for recognizing the spiking motion of a volleyball player. Background Technology

[0002] A volleyball player's spike is a continuous aerial hitting motion. The player first takes a running start and jumps, extending their body backward into a reverse arch shape in the air; then they tuck in their abdomen and rotate their body, swinging their arm forward and upward; finally, they strike the middle and back of the volleyball with their palm, causing the ball to drop rapidly over the net. This motion involves a noticeable lifting of the shoulder joint, an explosive extension of the elbow joint from flexion to extension, and a downward pressure of the wrist joint.

[0003] In existing volleyball spiking motion recognition methods, when an athlete's arm swings rapidly, the arm undergoes significant displacement within the image plane during camera exposure, forming a continuous trailing stripe along the direction of movement—motion blur. Within the blurred area, the originally sharp edges of the limbs are uniformly stretched, causing the image edge gradient to decrease from steep peaks to gentle, broad peaks. Traditional 2D pose estimation networks rely on edge features with high gradient responses to locate joints. In the blurred area, a single real edge may be misdetected as multiple false edges spaced several pixels apart. This causes the network to randomly select peak positions from discontinuous gradient responses, ultimately resulting in systematic localization drift of the 2D pixel coordinates of the elbow and wrist joints, with the drift direction highly correlated with the arm's movement direction. Therefore, how to adaptively remove motion blur and reflection interference to improve the robustness of spiking motion recognition to complex environments and rapid movements has become a challenge for the industry. Summary of the Invention

[0004] This application provides a method and system for recognizing volleyball player spiking actions, which can adaptively remove motion blur and reflection interference, thereby improving the robustness of spiking action recognition to complex environments and fast movements.

[0005] Firstly, this application provides a method for recognizing the spiking motion of a volleyball player, including: Acquire continuous multi-frame depth and visible light images of the athlete's upper limbs; Interference is judged for each frame of visible light image. When there is motion blur and light reflection interference caused by fast movement in the visible light image, the visible light image of the current interfering frame is obtained. The visible light image of the current interference frame is decomposed into a high-frequency detail layer and a low-frequency main body layer. Then, based on the depth edge information of the depth image in the same frame, the interference region in the high-frequency detail layer is filtered out, while the low-frequency main body layer is subjected to edge-preserving filtering. The high-frequency detail layer after filtering out interference and the low-frequency main layer after filtering are reconstructed into a corrected image, and then multiple corrected images are obtained. The three-dimensional motion trajectory and angle change features of the elbow, shoulder and wrist joints are extracted from each corrected image, and then the output is whether it is a standard spiking action and the action confidence.

[0006] In some embodiments, interference determination for each frame of visible light image specifically includes: For each frame of visible light image, calculate the grayscale gradient amplitude distribution of the current frame of visible light image, and count the proportion of pixels whose gradient amplitude exceeds a preset threshold. When the proportion of pixels is higher than the first threshold, it is determined that there is motion blur interference. Simultaneously detect continuous bright areas in the visible light image of the current frame whose brightness values ​​exceed the second threshold. If the area ratio of the bright areas exceeds the third threshold, it is determined that there is light reflection interference. If any type of interference is determined to exist, the current frame is marked as an interference frame, and the type of interference is recorded. This process is repeated to determine the interference in each frame of the visible light image and obtain multiple interference frames.

[0007] In some embodiments, decomposing the visible light image of the current interfering frame into a high-frequency detail layer and a low-frequency main body layer specifically includes: The visible light image of the current interference frame is converted to grayscale to obtain a single-channel brightness image; The single-channel brightness image is smoothed to obtain the low-frequency main layer; The high-frequency detail layer is obtained by subtracting the original brightness image of the visible light image of the current interference frame from the low-frequency main layer pixel by pixel.

[0008] In some embodiments, filtering out interference regions in the high-frequency detail layer based on depth edge information of the same frame depth image specifically includes: Edge detection is performed on depth images in the same frame to extract depth edge information, where depth edges correspond to object contours or joint boundaries; The high-frequency detail layer and depth edge information are spatially aligned, and isolated high-response regions far from the depth edge are marked in the high-frequency detail layer as candidate interference regions. The pixel values ​​within the candidate interference area are set to zero, while the values ​​of the remaining pixels remain unchanged, thereby selectively filtering out motion blur stripes and reflective highlights, and outputting a high-frequency detail layer after interference removal.

[0009] In some embodiments, edge-preserving filtering of the low-frequency main body layer specifically includes: Using the same frame depth image as a guide image, guided filtering is performed on the low-frequency main layer; In guided filtering, the normalization parameter at the depth edge is set to a value lower than that in the flat region to enhance edge preservation strength and output the low-frequency main body layer after filtering.

[0010] In some embodiments, reconstructing the high-frequency detail layer after filtering out interference and the low-frequency main layer after filtering into a corrected image, thereby obtaining multiple frames of corrected images, specifically includes: The high-frequency detail layer after filtering out interference and the low-frequency main layer after filtering are added pixel by pixel to obtain the reconstructed single-channel brightness image; The reconstructed luminance image and the chromaticity components of the original visible light image are merged to restore a color-corrected image; For multiple consecutive interfering frames, repeat the above reconstruction process and organize them into corrected images according to the frame number. For non-interfering frames, directly retain the original visible light image as the corrected image to obtain multiple corrected images.

[0011] In some embodiments, the three-dimensional motion trajectory and angle change features of the elbow, shoulder, and wrist joints are extracted from each frame of the corrected image, and then output whether it is a standard spiking action and the action confidence score. Specifically, this includes: Each frame of calibrated image is registered with the corresponding depth image to obtain a sequence of color-depth image pairs. This sequence is then input into a pre-trained 2D pose estimation network to obtain the 2D pixel coordinates of the elbow, shoulder, and wrist joints in each frame. Based on the depth value of the corresponding pixel position in the depth image, the two-dimensional coordinates are mapped to three-dimensional spatial coordinates, and the three-dimensional motion trajectory of every three consecutive joints is obtained. Based on the spatial positions of the elbow, shoulder, and wrist joints in every three frames, the elbow flexion angle and the angle between the shoulder-wrist line and the horizontal plane are calculated to obtain the angular change characteristics of every three consecutive joints. All three-dimensional motion trajectories and angle change features are input into a temporal classification network, and the output is whether it is a standard spike action and its confidence level.

[0012] Secondly, this application provides a volleyball player spiking action recognition system, comprising: The acquisition module is used to acquire continuous multi-frame depth images and visible light images of the athlete's upper limbs; The processing module is used to determine interference in each frame of visible light image. When there is motion blur and light reflection interference caused by fast movement in the visible light image, the visible light image of the current interference frame is obtained. The processing module is also used to decompose the visible light image of the current interference frame into a high-frequency detail layer and a low-frequency main body layer, and then filter out the interference region in the high-frequency detail layer based on the depth edge information of the depth image in the same frame, while performing edge-preserving filtering on the low-frequency main body layer. The execution module is used to reconstruct the high-frequency detail layer after filtering out interference and the low-frequency main layer after filtering into a corrected image, thereby obtaining multiple frames of corrected images. The three-dimensional motion trajectory and angle change features of the elbow, shoulder and wrist joints are extracted from each frame of corrected images, and then output whether it is a standard spiking action and the action confidence.

[0013] Thirdly, this application provides a computer device, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for calling and running the computer program from the memory, so that the computer device performs the above-described volleyball player spike action recognition method.

[0014] Fourthly, this application provides a computer-readable storage medium storing instructions or code that, when executed on a computer, cause the computer to implement the aforementioned volleyball player spike action recognition method.

[0015] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: This application provides a method and system for recognizing volleyball player spiking actions, which involves acquiring multiple consecutive frames of depth images and visible light images of the athlete's upper limbs; judging interference in each frame of visible light images; when motion blur and light reflection interference caused by rapid movement exist in the visible light image, acquiring the visible light image of the current interference frame; decomposing the visible light image of the current interference frame into a high-frequency detail layer and a low-frequency main body layer; filtering out interference regions in the high-frequency detail layer based on the depth edge information of the depth image in the same frame, while simultaneously performing edge-preserving filtering on the low-frequency main body layer; reconstructing the high-frequency detail layer and the filtered low-frequency main body layer into a corrected image, thereby obtaining multiple frames of corrected images; extracting the three-dimensional motion trajectory and angle change features of the elbow, shoulder, and wrist joints from each frame of corrected images; and then outputting whether it is a standard spiking action and the action confidence level.

[0016] Therefore, in this application, the three-dimensional motion trajectory and angle change features of the elbow, shoulder, and wrist joints are extracted from each frame of the corrected image to output whether it is a standard spiking action and the action confidence. First, interference judgment is performed on each frame of visible light image to obtain the result of whether interference exists and the type of interference. This avoids the waste of computational resources and texture distortion of normal images caused by indiscriminate processing, making the entire correction process selectively triggered and adaptive. For motion blur frames where the edge gradient decreases as a whole due to rapid arm swing, high-frequency detail layer filtering is selectively activated. For frames where reflective spots appear on the skin or jersey surface due to stadium lighting, isolated high-response areas are specifically identified and filtered out. This avoids the simple processing method of treating all images the same, realizing a differentiated strategy of interference correction and interference-free preservation, and improving the spatial adaptability and temporal selectivity of the correction operation. Then, by determining multiple frames of corrected images, each A complete image sequence in which motion blur stripes and reflected light spots are selectively removed while limb contour edges are enhanced and preserved is generated. The trailing stripes originally caused by rapid movement are removed, and the bright spots originally caused by specular reflection no longer exist. At the same time, the contour edges between the arm and torso remain clear and sharp during the filtering process due to the guidance of depth information. This effectively eliminates joint positioning deviations caused by motion blur and false high response interference caused by reflected light spots. This allows the subsequent 2D pose estimation network to stably and accurately detect the pixel positions of the elbow, shoulder, and wrist joints in each frame. As a result, the extraction of 3D motion trajectory and the calculation of angle change features are no longer affected by image quality fluctuations, improving the stability and discrimination accuracy of spike action recognition under complex lighting and rapid movement conditions. In summary, the above scheme can achieve adaptive removal of motion blur and reflection interference, thereby improving the robustness of spike action recognition to complex environments and rapid movements. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is an exemplary flowchart of a volleyball player spiking action recognition method according to some embodiments of this application; Figure 2 This is a schematic flowchart illustrating the process of determining multiple frames of corrected images according to some embodiments of this application; Figure 3 This is a schematic diagram of the structure of a volleyball player spiking action recognition system according to some embodiments of this application; Figure 4 This is a schematic diagram of the structure of a computer device that implements a method for recognizing volleyball player spike actions, according to some embodiments of this application. Detailed Implementation

[0019] To better understand the technical solution of this application, the technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] refer to Figure 1 The figure is an exemplary flowchart of a volleyball player spiking action recognition method according to some embodiments of this application. The volleyball player spiking action recognition method mainly includes the following steps: In step 101, multiple consecutive depth images and visible light images of the athlete's upper limbs are acquired.

[0021] It should be noted that, in this application, the depth image is a two-dimensional image used to record the actual distance value from each pixel in the scene to the depth camera; the visible light image is a two-dimensional image used to record the color and texture information of the object surface in the scene.

[0022] In practice, a depth camera and a visible light camera are fixedly installed on one side of the volleyball court. The optical axes of the two cameras are kept parallel and their fields of view are consistent, ensuring that the cameras can completely cover the area where the athlete's upper limbs are located during the jump and spike motion. The depth camera actively projects near-infrared coded light towards the athlete and receives the reflected light signals. By calculating the time difference between the light emission and return or the degree of deformation of the coded pattern, a depth image is generated. The value of each pixel in the depth image represents the distance from that point to the camera, with the distance value in millimeters; the closer the distance, the smaller the value, and the farther the distance, the larger the value. Simultaneously, the visible light camera uses ambient lighting or supplementary lighting to collect natural light reflected from the athlete's upper limbs through a charge-coupled device or complementary metal-oxide-semiconductor sensor, thereby generating a visible light image. Each pixel in a visible light image contains brightness values ​​for three color channels: red, green, and blue. These three values ​​collectively determine the color of the pixel. Two cameras are connected via a hardware synchronization signal line and controlled by the same external trigger signal, ensuring that each frame of depth image and each frame of visible light image are captured simultaneously. Both the depth camera and the visible light camera continuously acquire images at a frame rate of 30 frames per second, resulting in a sequence of depth images and a sequence of visible light images arranged chronologically. For each frame, the acquisition system simultaneously records the corresponding timestamp for pairing the two types of images at the same time. To ensure the acquired images are clear and usable, before formal acquisition, an athlete performs a complete spike motion to observe whether there is motion blur caused by excessive speed. If the motion blur is severe, the exposure time of the visible light camera is appropriately increased or the supplementary lighting brightness is increased. Recording continues during the acquisition process until the athlete completes the spike motion. Each acquired depth image and its timestamp-aligned visible light image are considered as an image pair, thus obtaining multiple consecutive frames of depth and visible light images.

[0023] In step 102, interference is judged for each frame of visible light image. When there is motion blur and light reflection interference caused by rapid movement in the visible light image, the visible light image of the current interference frame is obtained.

[0024] In some embodiments, interference determination of each frame of visible light image can be achieved by the following steps: For each frame of visible light image, calculate the grayscale gradient amplitude distribution of the current frame of visible light image, and count the proportion of pixels whose gradient amplitude exceeds a preset threshold. When the proportion of pixels is higher than the first threshold, it is determined that there is motion blur interference. Simultaneously detect continuous bright areas in the visible light image of the current frame whose brightness values ​​exceed the second threshold. If the area ratio of the bright areas exceeds the third threshold, it is determined that there is light reflection interference. If any type of interference is determined to exist, the current frame is marked as an interference frame, and the type of interference is recorded. This process is repeated to determine the interference in each frame of the visible light image and obtain multiple interference frames.

[0025] It should be noted that in this application, in the judgment of motion blur interference, when an object moves rapidly relative to the camera, the brightness change at the edge of the object will change from steep to gentle, which is reflected in the image as a decrease in gradient magnitude.

[0026] In practice, the visible light image of the current frame is first converted from the red-green-blue color space to the grayscale space. The conversion method is to multiply the red channel value of each pixel by 0.299, the green channel value by 0.587, and the blue channel value by 0.114, and then sum them to obtain the grayscale value of that pixel. The grayscale image can be obtained in this way. Then, for each pixel in the grayscale image, the grayscale difference between its adjacent pixels in the horizontal direction is calculated as the horizontal gradient, and the grayscale difference between its adjacent pixels in the vertical direction is calculated as the vertical gradient. The squares of the horizontal gradient and the vertical gradient of the pixel are summed and the square root is taken to obtain the gradient magnitude of the pixel. Next, the number of pixels in the entire grayscale image whose gradient magnitude is greater than a preset threshold is counted and divided by the total number of pixels in the image to obtain the proportion of pixels whose gradient magnitude exceeds the preset threshold. The preset threshold is a pre-set empirical value, ranging from 30 to 50 grayscale gradient units. This value is obtained by collecting multiple sets of arm motion images at different speeds and manually calibrating blurred and non-blurred areas. When the calculated pixel ratio is higher than the first threshold, it is determined that there is motion blur interference in the current frame. The first threshold ranges from 70% to 85%, with a default of 80%. It can be adjusted through historical experience and is not limited here. This value reflects the lower limit of the proportion of fast-moving areas in the scene to the total area of ​​the image.

[0027] It should be noted that in this application, in the judgment of light reflection interference, the light reflection area is usually manifested as a bright patch with a brightness much higher than the surrounding skin or jersey, and these patches have spatial continuity.

[0028] In practice, each pixel in the grayscale image of the current frame is traversed, and pixels with grayscale values ​​greater than a second threshold are marked as candidate bright spots. The second threshold ranges from 200 to 230, and the grayscale value ranges from 0 to 255; the larger the value, the brighter the image. This threshold is used to distinguish between normal skin reflections and abnormal specular reflections. Then, a four-connected or eight-connected region growing algorithm is used to connect adjacent candidate bright spots into multiple consecutive bright regions. Specifically, starting from any unvisited candidate bright spot, its four adjacent points in the four directions (up, down, left, and right) are checked to see if they are candidate bright spots. If so, they are included in the same region and the process continues until no further expansion is possible, thus obtaining multiple consecutive bright regions. For each consecutive bright area, the number of pixels it contains is counted and summed to obtain the total number of pixels in all bright areas. Dividing this by the total number of pixels in the image gives the area ratio of the bright areas. When the area ratio of the bright areas is higher than the third threshold, it is determined that there is light reflection interference in the current frame. The third threshold ranges from 5% to 15%, with a default value of 10%. It can be adjusted based on historical experience and is not limited here. This value reflects the upper limit of the area occupied by occasional reflected light spots on the skin and jersey surface under normal indoor volleyball court lighting conditions.

[0029] Finally, in the specific implementation, for each frame of visible light image, the motion blur judgment and light reflection judgment are performed sequentially. If either judgment result indicates the presence of interference, the frame is marked as an interfering frame, and the interference type is recorded. Interference types include motion blur, light reflection, and both. After performing the above judgment on all frames in chronological order, a set of interfering frames is obtained. Each interfering frame contains its original image data, frame number, and interference type label. When it is necessary to obtain a visible light image with motion blur and light reflection interference caused by rapid movement, the frame that meets the interference type requirements is directly extracted from this set of interfering frames to obtain the visible light image of the current interfering frame.

[0030] In step 103, the visible light image of the current interference frame is decomposed into a high-frequency detail layer and a low-frequency main body layer. Then, the interference region in the high-frequency detail layer is filtered out based on the depth edge information of the depth image in the same frame, while the low-frequency main body layer is subjected to edge-preserving filtering.

[0031] In some embodiments, decomposing the visible light image of the current interfering frame into a high-frequency detail layer and a low-frequency main body layer can be achieved by the following steps: The visible light image of the current interference frame is converted to grayscale to obtain a single-channel brightness image; The single-channel brightness image is smoothed to obtain the low-frequency main layer; The high-frequency detail layer is obtained by subtracting the original brightness image of the visible light image of the current interference frame from the low-frequency main layer pixel by pixel.

[0032] It should be noted that in this application, the single-channel brightness image is a grayscale image used to represent the brightness of each pixel without containing color information; the low-frequency main layer is an image used to represent areas in the image where the brightness changes gently; and the high-frequency detail layer is an image used to represent areas in the image where the brightness changes dramatically.

[0033] In practice, firstly, the visible light image of the current interference frame is converted to grayscale to obtain a single-channel brightness image. This can be achieved as follows: The visible light image of the current interference frame is acquired. Each pixel in this image contains values ​​for three color channels: red, green, and blue. The value of each channel ranges from zero to 255. To combine these three values ​​into a single value representing brightness, a weighted average method is used. Specifically, the red channel value is multiplied by 0.299, the green channel value by 0.587, and the blue channel value by 0.114. Then, the three products are added together to obtain the final brightness value. The result is the brightness value of that pixel. The same calculation is performed on each pixel in the visible light image of the current interference frame to generate a new image. Each pixel in this image has only one brightness value, ranging from zero to 255. A larger value indicates a brighter pixel. This new image is used as a single-channel brightness image. Then, the single-channel brightness image is smoothed to obtain the low-frequency main layer. This can be achieved by using a guided filter to smooth the single-channel brightness image. Guided filtering requires a guide image, which is directly used as the guide image itself. During filtering, a fixed-size square window slides pixel by pixel across the image. The default window side length is 1% of the shorter side of the image. At each window position, there is a linear relationship between the brightness value of all pixels within the window and the corresponding pixel's brightness value in the guide image. The linear coefficient is calculated by minimizing the difference between the actual brightness value within the window and the linear model estimate. The average of the linear estimates calculated for all windows covering the same pixel is taken to obtain the smoothed brightness value of that pixel. Smoothing can keep areas with large brightness differences on both sides of the edge clear, while areas with gentle brightness changes, such as the inner arm, remain clear. On the side or front of the torso, the brightness values ​​of adjacent pixels are averaged, and the messy, fine woven textures are weakened, leaving only large areas of limb outline information. The smoothed image is used as the low-frequency main layer. Finally, the original brightness image of the visible light image of the current interference frame and the low-frequency main layer are subtracted pixel by pixel to obtain the high-frequency detail layer. This can be achieved in the following way: the original brightness image is clearly the original version of the single-channel brightness image. For each pixel position in the image, the brightness value of the pixel position in the original brightness image is subtracted from the brightness value of the pixel position in the low-frequency main layer, and the difference is used as the output value of the pixel.It should be noted that the subtraction result may be negative. When the brightness value of a point in the original brightness image is lower than the brightness value of the corresponding point in the low-frequency main layer, the difference is negative, indicating that the position belongs to a descending edge from bright to dark. In order to preserve the direction information of the edge and the integrity of subsequent reconstruction, the absolute value is not taken or truncated to zero, but negative values ​​are allowed. The negative values ​​are preserved as is in the high-frequency detail layer. After performing the above subtraction operation on all pixel positions in sequence, a new image with the same size as the original image is obtained, in which the value of each pixel can be zero, positive or negative. This new image is used as the high-frequency detail layer.

[0034] In some embodiments, filtering out interference regions in the high-frequency detail layer based on depth edge information of the same frame depth image can be achieved using the following steps: Edge detection is performed on depth images in the same frame to extract depth edge information, where depth edges correspond to object contours or joint boundaries; The high-frequency detail layer and depth edge information are spatially aligned, and isolated high-response regions far from the depth edge are marked in the high-frequency detail layer as candidate interference regions. The pixel values ​​within the candidate interference area are set to zero, while the values ​​of the remaining pixels remain unchanged, thereby selectively filtering out motion blur stripes and reflective highlights, and outputting a high-frequency detail layer after interference removal.

[0035] It should be noted that in this application, the depth edge information is used to identify the set of locations in the depth image where the distance value changes drastically, and each location corresponds to the outline of the athlete's limbs and the boundaries of joints; the candidate interference region is used to represent isolated pixel blocks in the high-frequency detail layer that are far from the real physical edges and have drastic brightness changes, and each pixel block corresponds to motion blur stripes and light reflection spots; the high-frequency detail layer after filtering out interference is used to represent the high-frequency detail image after removing motion blur stripes and light reflection spots, in which only detail information related to the real physical edges is retained.

[0036] In specific implementation, firstly, edge detection is performed on the depth image of the same frame to extract depth edge information. The depth edges correspond to object contours or joint boundaries, which can be achieved as follows: A depth image acquired at the same time as the visible light image of the current interfering frame is obtained. The value of each pixel in this depth image represents the distance from that point to the camera, with the distance value in millimeters. To extract the edge positions in the depth image, the Sobel operator can be used for edge detection. Specifically, for each pixel in the depth image, the horizontal gradient is calculated by subtracting the depth value of the left-hand neighboring pixel from the depth value of the right-hand neighboring pixel. The vertical gradient is also calculated by subtracting the depth value of the top-hand neighboring pixel from the depth value of the bottom-hand neighboring pixel. The squares of the horizontal and vertical gradients are added together, and the square root of the sum is taken to obtain the gradient magnitude of the pixel. When the gradient magnitude of a pixel exceeds a preset edge threshold, the pixel is determined to be an edge. Edge points, with edge threshold values ​​ranging from 20 mm to 50 mm, are preset based on the noise level of the depth camera and the distance difference between the athlete's arm and body, combined with historical experience. After performing the above calculations on all pixels, a binary edge image is obtained, where edge points are set to 1 and non-edge points are set to 0. Each continuous edge line in this binary edge image corresponds to the boundary between the athlete's arm and torso, the boundary between the wrist and the ball, and the boundary between the arm and the background. This binary edge image is used as depth edge information. Then, the high-frequency detail layer and depth... Spatial alignment of edge information is performed, and isolated high-response regions far from the depth edge are marked in the high-frequency detail layer as candidate interference regions. This can be achieved as follows: Since the depth image and the visible light image are acquired by two different cameras, the resolution of the two cameras may differ and their installation positions may not completely overlap. Therefore, it is necessary to spatially register the depth edge information image with the high-frequency detail layer image. The specific registration method is to obtain the coordinate mapping relationship between the two cameras in advance through a calibration board. That is, for each pixel in the visible light image, the corresponding pixel position in the depth image can be calculated. Using this mapping relationship, each edge point in the depth edge information image is transformed to the pixel coordinate system of the high frequency detail layer, thus obtaining an aligned edge mask image. After spatial alignment, the high frequency detail layer is traversed pixel by pixel. Each pixel in the high frequency detail layer has a value, which can be positive, negative, or zero. The larger the absolute value of the value, the more drastic the brightness change of that point. For each pixel in the high frequency detail layer, it is determined whether the absolute value of the pixel is greater than the high response threshold. The high response threshold is 10 brightness units by default. This threshold is used to distinguish between normal detail textures and abnormal strong interference.If the absolute value of a pixel is greater than the high response threshold, further checks are performed to determine if there are aligned depth edge points in the pixel's neighborhood. The check area is a circular region with a radius of 5 pixels centered on the pixel. If no depth edge points exist within this 5-pixel radius, the pixel is marked as a member of the candidate interference region. All pixels that meet the above two conditions are grouped together, and adjacent candidate pixels are connected to form blocks. Each block is a candidate interference region. Each candidate interference region appears as a thin stripe-like light spot or a circular highlight patch, independent of the athlete's limb contour. This allows us to obtain all candidate interference regions and their location information. Finally, the pixel values ​​within the candidate interference regions are set to zero, while the values ​​of the remaining pixels remain unchanged. This achieves selective filtering of motion blur stripes and reflective highlight patches. The high-frequency detail layer after filtering out interference can be implemented as follows: For each candidate interference region, locate the positions of all pixels covered by the region, and modify the corresponding values ​​of each pixel in the working image to zero. For pixels outside the candidate interference region, keep their values... The original values ​​remain unchanged, regardless of whether they are positive, negative, or zero. The technical principle is that pixels within the candidate interference region originally correspond to motion blur stripes and reflective highlights. These signals are false information from non-physical real edges; setting them to zero means that the brightness change contribution from these locations is completely removed during subsequent reconstruction. Meanwhile, the real edges retained next to the motion blur stripes, being near depth edges, have pixels that were never marked as candidate interference regions, so their values ​​remain unchanged, continuing to provide necessary edge information for reconstruction. For example, in a high-frequency detail layer image, if there is a stripe composed of high-response pixels near the real outline of the arm, and the stripe is spatially close to the depth edge point, then these are retained. Conversely, if an isolated bright spot appears in the background far outside the arm outline, and no pixel in the bright spot has a depth edge point in its neighborhood, then the entire bright spot is set to zero. After zeroing all candidate interference regions, only those high-frequency details close to the real physical edges are retained in the working image; motion blur stripes and reflective highlights are completely removed. The processed working image is used as the high-frequency detail layer after filtering out interference.

[0037] In some embodiments, edge-preserving filtering of the low-frequency main body layer can be achieved by the following steps: Using the same frame depth image as a guide image, guided filtering is performed on the low-frequency main layer; In guided filtering, the normalization parameter at the depth edge is set to a value lower than that in the flat region to enhance edge preservation strength and output the low-frequency main body layer after filtering.

[0038] It should be noted that in this application, the guide image is a reference image used to guide how to preserve edges and smooth flat areas during the filtering process; the normalization parameter is a value used to control the smoothness of the image; and the filtered low-frequency main body layer is used to represent a low-frequency image with smoother brightness distribution and clearer limb contours after deep edge enhancement processing.

[0039] In specific implementation, firstly, using the same-frame depth image as a guide image, the guided filtering of the low-frequency main body layer can be implemented in the following way: Obtain the same-frame depth image corresponding to the current interference frame, where the value of each pixel in the depth image represents the distance from that point to the camera. Simultaneously, obtain the low-frequency main body layer obtained in the previous steps; this low-frequency main body layer is an image with smooth brightness changes extracted from the visible light image. Here, the depth image is used as the guide image, and the low-frequency main body layer is used as the image to be filtered. A square window with a fixed side length is set, typically 2% of the shorter side of the image. The window is slid across the image pixel by pixel. At each window position, the linear coefficient that minimizes the difference between the linear model estimate and the true value of the low-frequency main body layer is calculated. The linear coefficient can be obtained by mapping the information of all pixels within the window using a pre-stored table. The pixel information specifically includes the mean and variance of the pixel values ​​of the guide image, the mean of the pixel values ​​of the image to be filtered, and the covariance between the guide image and the image to be filtered. Since the depth image is used as the guide image, the pixel values ​​at the depth edges change drastically, and their variance is large. This causes the linear coefficient to change abruptly on both sides of the edge, thus causing the filtered output image to also change at the same position, achieving the edge preservation effect. After completing the calculation of all windows, for each image... For each pixel, the average of the linear estimates calculated from all windows covering that pixel is used as the final output brightness value. The image after guided filtering is used as the output of this guided filtering. Then, in the guided filtering, the normalization parameter at the depth edge is set to a value lower than that in the flat area to enhance the edge preservation strength. The low-frequency main body layer after output filtering can be implemented in the following way: In the guided filtering calculation process, the normalization parameter is an important parameter for stable calculation. It is added to the denominator of the variance term to prevent division by zero error when the pixel value changes too little within the window. It also controls the smoothing strength. The conventional use of the normalization parameter is to use the same fixed value for the entire image. In order to improve the preservation ability at the depth edge, an adaptive normalization parameter is used here. Specifically, it is determined whether each pixel is located on the depth edge based on the gradient information of the depth image. The method for determining depth edges is the same as described above: calculate the gradient magnitude of each pixel in the depth image, and determine a pixel as an edge when the gradient magnitude is greater than an edge threshold; for pixels determined to be on depth edges, set the normalization parameter of the window containing that pixel to a first value, which is 10. -6This is a very small value, so that the filtering process produces almost no smoothing, thus completely preserving the original brightness transitions at depth edges; for pixels determined to be flat areas, i.e., not on depth edges, the normalization parameter of the window containing them is set to the second value, which is 10. -4 The value is relatively large, which makes the filtering process produce a significant smoothing effect on flat areas, removing residual fine textures and noise. For example, when the center pixel of the window falls on the outline of the athlete's arm, the depth value at that position jumps by about 200 mm from the arm area to the background area. With an edge threshold of 30 mm, it is determined to be a depth edge. Using a smaller first value, the brightness difference on both sides of the arm outline is completely preserved. When the center pixel of the window falls on the inside of the athlete's arm, the depth value changes gradually and is determined to be a flat area. Using a larger second value, the small reflective spots remaining on the skin of the inside of the arm are smoothed out. Through the above adaptive adjustment, in the filtered low-frequency main body layer, the outline boundaries of the main objects such as the arm, torso, and ball remain sharp, while the brightness distribution inside each object is more uniform and smooth. The image after the above adaptive guided filtering is used as the filtered low-frequency main body layer.

[0040] In step 104, the high-frequency detail layer after filtering out interference and the low-frequency main body layer after filtering are reconstructed into a corrected image, thereby obtaining multiple frames of corrected images. The three-dimensional motion trajectory and angle change features of the elbow, shoulder and wrist joints are extracted from each frame of corrected images, and then output whether it is a standard spiking action and the action confidence.

[0041] In some embodiments, the high-frequency detail layer after interference removal and the low-frequency main layer after filtering are reconstructed into a corrected image, thereby obtaining multiple frames of corrected images, for reference. Figure 2 The figure described above is a flowchart illustrating the process of determining multiple frames of corrected images in some embodiments of this application. In this embodiment, determining multiple frames of corrected images can be achieved using the following steps: In step 1041, the high-frequency detail layer after filtering out interference and the low-frequency main layer after filtering are added pixel by pixel to obtain the reconstructed single-channel brightness image; In step 1042, the chromaticity components of the reconstructed luminance image and the original visible light image are merged to restore the color-corrected image; In step 1043, for multiple consecutive interfering frames, the above reconstruction process is repeated, and the frames are organized into corrected images according to their frame numbers. For non-interfering frames, the original visible light image is directly retained as the corrected image, resulting in multiple corrected images.

[0042] It should be noted that, in this application, the reconstructed single-channel brightness image is used to represent a grayscale image in which each pixel contains only one brightness value after interference filtering and edge enhancement processing; the color-corrected image is used to represent a color image in which each pixel contains red, green and blue color channel information after removing motion blur and light reflection interference; and the multi-frame corrected image is used to represent an image sequence arranged in chronological order after processing all interfering and non-interfering frames.

[0043] In practice, firstly, the high-frequency detail layer after interference removal and the low-frequency main body layer after filtering are added pixel by pixel to obtain the reconstructed single-channel brightness image. This can be achieved as follows: Firstly, the image obtained after interference removal from the high-frequency detail layer is acquired. The value of each pixel in this image can be positive, negative, or zero, where non-zero values ​​represent brightness changes related to the actual physical edges. Simultaneously, the image obtained after adaptive guided filtering of the low-frequency main body layer is acquired. The value of each pixel in this image is a non-negative brightness value, representing the brightness distribution of the overall structure, such as the limb contour and torso. For each pixel in the image, the value of that pixel in the high-frequency detail layer after filtering out interference is added to the value of that pixel in the low-frequency main layer after filtering out interference. The sum is used as the output brightness value of that pixel. Since the positions in the high-frequency detail layer that originally corresponded to motion blur stripes and reflected light spots have been set to zero, only the value of the low-frequency main layer is retained after adding all positions, which is equivalent to removing the interference. At the real edge positions, the high-frequency detail layer contributes positive or negative adjustment, so that the brightness difference on both sides of the edge is restored to its original state. After performing the above addition operation on all pixel positions in sequence, an image with the same scale as the original image can be obtained. Given identical grayscale images where each pixel contains a luminance value that theoretically may exceed the range of 0 to 255, a linear stretching of the entire image is required. This involves mapping the minimum value to 0 and the maximum value to 255, ensuring that the luminance values ​​of all pixels fall within the standard range. This stretched grayscale image is then used as the reconstructed single-channel luminance image. The reconstructed luminance image is then merged with the chromaticity components of the original visible light image to restore the color-corrected image. This can be achieved by: acquiring the original visible light image of the current interference frame, where each pixel contains values ​​for the red, green, and blue color channels; and extracting the chromaticity components from the original image by converting it from a red-green-blue color space to another color space containing one luminance channel and two chromaticity channels (where the chromaticity channels represent color information only and are unaffected by luminance changes). Specifically, for each pixel, the values ​​of the two chromaticity components are calculated using a linear combination of the red, green, and blue values. The first chromaticity component roughly corresponds to the difference between blue and luminance, and the second roughly corresponds to the difference between red and luminance. After the conversion is complete, the luminance channel of the original image is completely replaced with the reconstructed single-channel luminance image, while the two chrominance channels retain their original values. Then, the replaced image is converted back from this color space to the red-green-blue color space. The reverse conversion is performed by using the reconstructed luminance values ​​and the two original chrominance values ​​to calculate the new red, green, and blue values ​​through a linear combination.The technical principle behind this approach is that motion blur and light reflection interference primarily affect luminance information, with a smaller impact on color information. By replacing only the luminance channel while preserving the original chroma channel, the interference can be removed while preserving the most accurate color information, ensuring the accuracy of the athlete's jersey color, skin tone, and ball color. The color image obtained after the reverse conversion is used as the color correction image. Finally, for multiple consecutive interfering frames, the above reconstruction process is repeated, organizing the images according to frame number. For non-interfering frames, the original visible light image is directly retained as the correction image. The resulting multi-frame correction images can be achieved by: obtaining a list of frame numbers for all marked interfering frames and a list of frame numbers for all identified non-interfering frames; and for each frame in the list of interfering frames, sequentially performing the above steps. The operation involves first reconstructing a single-channel luminance image, then merging it with the original chrominance components to recover the color image. Each processed color-corrected image is stored in the corresponding position of an output sequence according to its original frame number. For each frame in the non-interference frame list, since there is no motion blur or light reflection interference, no correction processing is required; the original visible light image is directly used as the correction image for that frame, and stored in the corresponding position of the output sequence according to its original frame number. All processed images are arranged in ascending order of frame number to obtain a complete image sequence. Each frame in this sequence is a clear, interference-free color image, and the sequence length is exactly the same as the number of frames originally acquired, with continuous frame numbers. This complete image sequence organized according to frame numbers is used as a multi-frame correction image.

[0044] In some embodiments, extracting the three-dimensional motion trajectory and angle change features of the elbow, shoulder, and wrist joints from each frame of corrected images, and then outputting whether it is a standard spiking action and the action confidence, can be achieved by the following steps: Each frame of calibrated image is registered with the corresponding depth image to obtain a sequence of color-depth image pairs. This sequence is then input into a pre-trained 2D pose estimation network to obtain the 2D pixel coordinates of the elbow, shoulder, and wrist joints in each frame. Based on the depth value of the corresponding pixel position in the depth image, the two-dimensional coordinates are mapped to three-dimensional spatial coordinates, and the three-dimensional motion trajectory of every three consecutive joints is obtained. Based on the spatial positions of the elbow, shoulder, and wrist joints in every three frames, the elbow flexion angle and the angle between the shoulder-wrist line and the horizontal plane are calculated to obtain the angular change characteristics of every three consecutive joints. All three-dimensional motion trajectories and angle change features are input into a temporal classification network, and the output is whether it is a standard spike action and its confidence level.

[0045] It should be noted that in this application, the color-depth image pair sequence is a dual-channel data set arranged in chronological order after pairing each frame of color-corrected image with its corresponding depth image; the two-dimensional pixel coordinates are two values ​​used to represent the specific positions of the elbow, shoulder, and wrist joints in the image plane, representing the horizontal and vertical positions respectively; the three-dimensional motion trajectory is a path formed by connecting the spatial position points of each joint at different times in chronological order, including coordinate changes in the horizontal, vertical, and depth dimensions; the angle change feature is a sequence of values ​​representing the change of the degree of elbow flexion and the overall arm raising angle over time; the standard spiking action is a classification label used to indicate whether the athlete's completed action conforms to the spiking posture and motion pattern defined in the technical specifications; and the confidence score is used to indicate the degree of confidence of the classification network in determining that the current action is a standard spiking action.

[0046] In specific implementation, firstly, the corrected images of each frame are registered with their corresponding depth images to obtain a color-depth image pair sequence. This sequence is then input into a pre-trained 2D pose estimation network to obtain the 2D pixel coordinates of the elbow, shoulder, and wrist joints in each frame. This can be achieved as follows: Since the resolutions of the color camera and the depth camera may differ and their installation positions may vary, spatial registration is required for each pair of corrected and depth images. The registration process has already been described above and will not be repeated here. After registration, each frame of the registered color corrected image and depth image is grouped into an image pair, and arranged in ascending order of frame number to form a continuous image pair sequence. This sequence is the color-depth image pair sequence. The input is fed into a pre-trained 2D pose estimation network. This network employs a convolutional neural network structure from deep learning and was trained using a large number of volleyball match images labeled with human joint positions. This enabled the network to learn the ability to locate various human joints from color images. For each frame of the input color-corrected image, the network outputs the pixel coordinates of the elbow, shoulder, and wrist joints. Each coordinate contains two values: the first value represents the joint's position in the horizontal direction of the image, and the second value represents its position in the vertical direction. This process is repeated for each frame in the sequence to obtain the 2D pixel coordinates of the three joints in each frame. Then, based on the depth values ​​of the corresponding pixel positions in the depth image... To map two-dimensional coordinates to three-dimensional spatial coordinates, the three-dimensional motion trajectory of every three consecutive joints can be achieved as follows: For the two-dimensional pixel coordinates of the shoulder, elbow, and wrist joints already acquired in each frame, find the pixel points of these coordinate positions in the corresponding registered depth image, read the depth value of the pixel point (representing the actual distance from the joint to the camera in millimeters), multiply the horizontal position value of the joint's two-dimensional pixel coordinates by the depth value, and then divide by the horizontal focal length of the camera to obtain the horizontal coordinate value of the joint in three-dimensional space. Multiply the vertical position value of the joint's two-dimensional pixel coordinates by the depth value, and then divide by the vertical focal length of the camera to obtain the vertical coordinate value of the joint in three-dimensional space. The depth value itself is used as the depth direction coordinate value of the joint in three-dimensional space. Each joint obtains three values ​​in each frame, representing the horizontal position, vertical position and depth position respectively. Together, they are called the three-dimensional space coordinates of the joint in that frame. For the same joint, the three-dimensional space coordinates in different frames are connected in ascending order of frame number to obtain a point sequence in three-dimensional space. This point sequence describes the position change of the joint over time, which is the three-dimensional motion trajectory of the joint. The above steps are repeated for the shoulder joint, elbow joint and wrist joint to extract their respective three-dimensional motion trajectories, resulting in a total of three trajectories. Each trajectory contains the spatial position points of the joint in multiple consecutive frames.Furthermore, based on the spatial positions of the elbow, shoulder, and wrist joints in every three frames, the elbow flexion angle and the angle between the shoulder-wrist line and the horizontal plane are calculated. The angular change characteristics of every three consecutive joints can be obtained as follows: For each frame, using the three-dimensional spatial coordinates of the shoulder, elbow, and wrist joints in that frame, two angle values ​​are calculated. The first angle is the elbow flexion angle, calculated by taking the direction vectors from the elbow joint towards the shoulder joint and from the elbow joint towards the wrist joint, and then calculating the angle between these two direction vectors. That is, first calculate the component differences of the straight line from the elbow joint to the shoulder joint in three directions, and the component differences of the straight line from the elbow joint to the wrist joint in three directions. Multiply the corresponding components of the two direction vectors, sum them, and divide by the product of the lengths of the two direction vectors to obtain the cosine value of the angle. Then, use the inverse cosine to obtain the angle value in degrees. When the arm is straight, this angle is close to 180 degrees; when the arm is bent, this angle becomes smaller. The second angle is the angle between the line connecting the shoulder joint and the wrist joint and the horizontal plane. The calculation method is to first calculate the three-dimensional coordinates of the wrist joint and then subtract the three-dimensional coordinates of the shoulder joint. The three-dimensional coordinates are used to obtain a three-dimensional vector pointing from the shoulder joint to the wrist joint. Then, the angle between this vector and the horizontal plane is calculated. The horizontal plane is the plane with a vertical coordinate of zero. The angle is calculated by dividing the vertical component of the vector by its horizontal length to obtain the tangent value. Then, the arctangent is used to obtain the angle value in degrees. This angle reflects the overall range of arm elevation. When spiking, the arm needs to be raised high, so this angle is usually larger. The above two angle values ​​are calculated for each frame and arranged according to the frame number to obtain two angles. The two sequences that change over time are collectively referred to as the angle change features. Finally, all the three-dimensional motion trajectories and angle change features are input into the temporal classification network. The output of whether it is a standard spiking action and its confidence level can be achieved in the following way: the three three-dimensional motion trajectories of the shoulder joint, elbow joint and wrist joint, as well as the elbow joint bending angle sequence and the angle sequence between the shoulder and wrist line and the horizontal plane are merged into a complete feature vector sequence. Each time point in this sequence contains nine values ​​of the three directional coordinates of the three joints, plus two angle values, for a total of eleven values.The feature vector sequence is input into a pre-trained temporal classification network. This network, employing a recurrent neural network or temporal convolutional network structure, is used to process data with temporal order. During training, the network learns a large number of positive and negative samples of standard and non-standard spiking actions, mastering the typical patterns of standard spiking actions in terms of three-dimensional trajectory and angle changes. The patterns of standard spiking actions include: the shoulder joint moving upwards then forwards; the elbow joint bending first then quickly extending; the wrist joint suddenly decelerating near the point of impact; the elbow joint bending angle rapidly decreasing and then increasing again from a large value; and the angle between the shoulder-wrist line and the horizontal plane reaching its maximum at the take-off point. After layer-by-layer calculation of the input feature sequence, the network outputs two values. The first value is the binary classification result, determining whether the current action belongs to a standard spiking action or not; by default, 1 represents a standard spiking action and 0 represents a non-standard spiking action. The second value is the confidence score, ranging from 0 to 1. It represents the network's confidence level in its judgment. For example, an output of 0.92 means the network is 92% confident that the current action is a standard spike. These two values ​​output by the network are used as the final result of the entire recognition method: whether it is a standard spike and its confidence score.

[0047] In another aspect, in some embodiments, this application provides a volleyball player spiking action recognition system, with reference to... Figure 3 The figure is a schematic diagram of the structure of a volleyball player spiking action recognition system according to some embodiments of this application. The volleyball player spiking action recognition system includes: a data acquisition module 201, a processing module 202, and an execution module 203, which are described below: The acquisition module 201 in this application is mainly used to acquire continuous multi-frame depth images and visible light images of the athlete's upper limbs; Processing module 202, in this application, is used to determine interference in each frame of visible light image. When there is motion blur and light reflection interference caused by rapid movement in the visible light image, the visible light image of the current interference frame is obtained. It should be noted that the processing module 202 is also used to decompose the visible light image of the current interference frame into a high-frequency detail layer and a low-frequency main body layer, and then filter out the interference area in the high-frequency detail layer based on the depth edge information of the depth image of the same frame, while performing edge-preserving filtering on the low-frequency main body layer. Execution module 203, in this application, is mainly used for In step 104, the high-frequency detail layer after filtering out interference and the low-frequency main body layer after filtering are reconstructed into a corrected image, thereby obtaining multiple frames of corrected images. The three-dimensional motion trajectory and angle change features of the elbow, shoulder and wrist joints are extracted from each frame of corrected images, and then output whether it is a standard spiking action and the action confidence.

[0048] The foregoing has detailed examples of the volleyball player spiking action recognition method and system provided in the embodiments of this application. It is understood that the corresponding apparatus, in order to achieve the above functions, includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0049] In some embodiments, this application also provides a computer device, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for calling and running the computer program from the memory, so that the computer device performs the above-described volleyball player spike action recognition method.

[0050] In some embodiments, reference Figure 4 The dashed lines in the figure indicate that the unit or module is optional. This figure is a structural schematic diagram of a computer device for implementing a volleyball player spiking action recognition method according to an embodiment of this application. The volleyball player spiking action recognition method described in the above embodiments can... Figure 4 The computer device shown is used to implement this, and the computer device includes at least one processor 301, a memory 302 and at least one communication unit 305. The computer device may be a terminal device, a server or a chip.

[0051] Processor 301 can be a general-purpose processor or a special-purpose processor. For example, processor 301 can be a central processing unit (CPU), which can be used to control computer devices, execute software programs, and process data from software programs. The computer device may also include a communication unit 305 for inputting (receiving) and outputting (transmitting) signals.

[0052] For example, the computer device can be a chip, and the communication unit 305 can be the input and / or output circuit of the chip, or the communication unit 305 can be the communication interface of the chip, which can be a component of a terminal device, network device or other device.

[0053] For example, the computer device may be a terminal device or a server, and the communication unit 305 may be a transceiver of the terminal device or the server, or the communication unit 305 may be a transceiver circuit of the terminal device or the server.

[0054] The computer device may include one or more memories 302 storing a program 304. The program 304 can be executed by a processor 301 to generate instructions 303, causing the processor 301 to execute the method described in the above method embodiments according to the instructions 303. Optionally, the memory 302 may also store data (such as a target audit model). Optionally, the processor 301 may also read data stored in the memory 302, which may be stored at the same storage address as the program 304, or it may be stored at a different storage address than the program 304.

[0055] The processor 301 and memory 302 can be configured separately or integrated together, for example, integrated on the system on chip (SOC) of the terminal device.

[0056] It should be understood that each step of the above method embodiment can be completed by hardware logic circuits or software instructions in the processor 301. The processor 301 can be a CPU, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0057] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0058] For example, in some embodiments, this application also provides a computer-readable storage medium storing instructions or code that, when executed on a computer, cause the computer to implement the above-described volleyball player spike action recognition method.

[0059] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0060] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for recognizing the spiking motion of a volleyball player, characterized in that, Includes the following steps: Acquire continuous multi-frame depth and visible light images of the athlete's upper limbs; Interference is judged for each frame of visible light image. When there is motion blur and light reflection interference caused by fast movement in the visible light image, the visible light image of the current interfering frame is obtained. The visible light image of the current interference frame is decomposed into a high-frequency detail layer and a low-frequency main body layer. Then, based on the depth edge information of the depth image in the same frame, the interference region in the high-frequency detail layer is filtered out, while the low-frequency main body layer is subjected to edge-preserving filtering. The high-frequency detail layer after filtering out interference and the low-frequency main layer after filtering are reconstructed into a corrected image, and then multiple corrected images are obtained. The three-dimensional motion trajectory and angle change features of the elbow, shoulder and wrist joints are extracted from each corrected image, and then the output is whether it is a standard spiking action and the action confidence.

2. The method as described in claim 1, characterized in that, The interference assessment for each frame of visible light image specifically includes: For each frame of visible light image, calculate the grayscale gradient amplitude distribution of the current frame of visible light image, and count the proportion of pixels whose gradient amplitude exceeds a preset threshold. When the proportion of pixels is higher than the first threshold, it is determined that there is motion blur interference. Simultaneously detect continuous bright areas in the visible light image of the current frame whose brightness values ​​exceed the second threshold. If the area ratio of the bright areas exceeds the third threshold, it is determined that there is light reflection interference. If any type of interference is determined to exist, the current frame is marked as an interference frame, and the type of interference is recorded. This process is repeated to determine the interference in each frame of the visible light image and obtain multiple interference frames.

3. The method as described in claim 1, characterized in that, Decomposing the visible light image of the current interfering frame into a high-frequency detail layer and a low-frequency main body layer specifically includes: The visible light image of the current interference frame is converted to grayscale to obtain a single-channel brightness image; The single-channel brightness image is smoothed to obtain the low-frequency main layer; The high-frequency detail layer is obtained by subtracting the original brightness image of the visible light image of the current interference frame from the low-frequency main layer pixel by pixel.

4. The method as described in claim 1, characterized in that, Filtering out interference regions in the high-frequency detail layer based on depth edge information from the same frame depth image specifically includes: Edge detection is performed on depth images in the same frame to extract depth edge information, where depth edges correspond to object contours or joint boundaries; The high-frequency detail layer and depth edge information are spatially aligned, and isolated high-response regions far from the depth edge are marked in the high-frequency detail layer as candidate interference regions. The pixel values ​​within the candidate interference area are set to zero, while the values ​​of the remaining pixels remain unchanged, thereby selectively filtering out motion blur stripes and reflective highlights, and outputting a high-frequency detail layer after interference removal.

5. The method as described in claim 1, characterized in that, The edge-preserving filtering of the low-frequency main layer specifically includes: Using the same frame depth image as a guide image, guided filtering is performed on the low-frequency main layer; In guided filtering, the normalization parameter at the depth edge is set to a value lower than that in the flat region to enhance edge preservation strength and output the low-frequency main body layer after filtering.

6. The method as described in claim 1, characterized in that, The high-frequency detail layer after filtering out interference and the low-frequency main layer after filtering are reconstructed into a corrected image, thus obtaining a multi-frame corrected image, specifically including: The high-frequency detail layer after filtering out interference and the low-frequency main layer after filtering are added pixel by pixel to obtain the reconstructed single-channel brightness image; The reconstructed luminance image and the chromaticity components of the original visible light image are merged to restore a color-corrected image; For multiple consecutive interfering frames, repeat the above reconstruction process and organize them into corrected images according to the frame number. For non-interfering frames, directly retain the original visible light image as the corrected image to obtain multiple corrected images.

7. The method as described in claim 1, characterized in that, The three-dimensional motion trajectory and angle change features of the elbow, shoulder, and wrist joints are extracted from each frame of the calibrated image, and then output whether it is a standard spiking action and the action confidence score. Specifically, this includes: Each frame of calibrated image is registered with the corresponding depth image to obtain a sequence of color-depth image pairs. This sequence is then input into a pre-trained 2D pose estimation network to obtain the 2D pixel coordinates of the elbow, shoulder, and wrist joints in each frame. Based on the depth value of the corresponding pixel position in the depth image, the two-dimensional coordinates are mapped to three-dimensional spatial coordinates, and the three-dimensional motion trajectory of every three consecutive joints is obtained. Based on the spatial positions of the elbow, shoulder, and wrist joints in every three frames, the elbow flexion angle and the angle between the shoulder-wrist line and the horizontal plane are calculated to obtain the angular change characteristics of every three consecutive joints. All three-dimensional motion trajectories and angle change features are input into a temporal classification network, and the output is whether it is a standard spike action and its confidence level.

8. A volleyball player spiking action recognition system, characterized in that, include: The acquisition module is used to acquire continuous multi-frame depth images and visible light images of the athlete's upper limbs; The processing module is used to determine interference in each frame of visible light image. When there is motion blur and light reflection interference caused by fast movement in the visible light image, the visible light image of the current interference frame is obtained. The processing module is also used to decompose the visible light image of the current interference frame into a high-frequency detail layer and a low-frequency main body layer, and then filter out the interference region in the high-frequency detail layer based on the depth edge information of the depth image in the same frame, while performing edge-preserving filtering on the low-frequency main body layer. The execution module is used to reconstruct the high-frequency detail layer after filtering out interference and the low-frequency main layer after filtering into a corrected image, thereby obtaining multiple frames of corrected images. The three-dimensional motion trajectory and angle change features of the elbow, shoulder and wrist joints are extracted from each frame of corrected images, and then output whether it is a standard spiking action and the action confidence.

9. A computer device, characterized in that, The computer device includes a memory and a processor. The memory is used to store computer programs, and the processor is used to call and run the computer programs from the memory, causing the computer device to perform the volleyball player spiking action recognition method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions or code that, when executed on a computer, cause the computer to implement the volleyball player spike action recognition method as described in any one of claims 1 to 7.

Images