Method and device for evaluating throwing motion and computer storage medium

Abstract

The invention discloses a method and equipment for evaluating throwing motion and a computer storage medium. The method comprises the following steps: acquiring a first video and a second video of throwing a target object by a target person; detecting feet of the target person in each frame of image of the first video to obtain foot positions of the target person in each frame of image; detecting a target object in each frame of image of the second video to obtain a throwing position of the target object in each frame of image, and determining a landing position of the target object according to the throwing position; determining whether the target person has an illegal behavior or not according to the foot position and a preset throwing circle; and when the target person does not have the illegal behavior, determining the throwing distance of the target object by using perspective transformation based on the landing position and the preset calibration area, and obtaining the throwing score of the target person. According to the scheme disclosed by the invention, the evaluation process is simplified, so that the evaluation efficiency can be improved.

Description

Technical Field

[0001] The present disclosure generally relates to the field of video processing technology. More specifically, the present disclosure relates to a method, an electronic device, and a computer-readable storage medium for evaluating a throwing motion. Background Art

[0002] In the current field of throwing sports evaluation, the traditional manual evaluation method is gradually being replaced by intelligent technology, thanks to the development of science and technology, especially the widespread application of software and hardware facilities. The evaluation methods are showing a trend of diversification, mainly divided into evaluation methods based on sensors and evaluation methods based on gravity sensing.

[0003] The evaluation method based on sensors requires the configuration of a ball sensing unit, a venue information unit, and a server processing center. The sensing unit captures the landing signal and transmits it to the venue information unit, which then sends the coordinates of the ball's landing point to the server processing center to calculate the throwing distance. The evaluation method that relies on gravity sensing requires the site to lay a gravity sensing layer to detect the concave points and then determine the throwing distance. It can be seen that these two solutions have cumbersome equipment construction, inconvenient maintenance, low evaluation efficiency, and are difficult to promote widely.

[0004] In view of this, there is an urgent need to provide a solution for evaluating throwing sports, so as to simplify the evaluation process of throwing sports and improve the efficiency and accuracy of evaluation results. Summary of the invention

[0005] In order to at least solve one or more of the technical problems mentioned above, the present disclosure proposes a solution for evaluating a throwing motion in multiple aspects.

[0006] In a first aspect, the present disclosure provides a method for evaluating a throwing sport, comprising: obtaining a video to be evaluated of a target person throwing a target object, wherein the throwing sport includes a preparation stage and a movement stage, and the video to be evaluated includes a first video of the preparation stage and a second video of the movement stage; detecting the feet of the target person in each frame image of the first video to obtain the foot position of the target person in each frame image; detecting the target object in each frame image of the second video to obtain the throwing position of the target object in each frame image, and determining the landing position of the target object based on the throwing position; determining whether the target person has violated the rules based on the foot position and a preset throwing circle; and in response to the target person not violating the rules, determining the throwing distance of the target object based on the landing position and a preset calibration area and using perspective transformation to obtain the throwing score of the target person.

[0007] In some embodiments, the feet of the target person in each frame image of the first video are detected to obtain the foot position of the target person in each frame image, including: using a human body detection model to perform human body detection on each frame image of the first video to obtain the human body detection result of each frame image, and determining the target person in each frame image according to the human body detection result; using a key point detection model to perform key point detection on the target person in each frame image to obtain the key point detection result of the target person in each frame image; intercepting the local image of the foot of the target person in each frame image according to the key point detection result; and using a segmentation model to perform foot segmentation on the local image of the foot to obtain the foot position of the target person in each frame image.

[0008] In some embodiments, detecting the target object in each frame image of the second video to obtain the throwing position of the target object in each frame image includes: determining the target frame image in which the target object appears for the first time in the second video; cropping each frame image from the target frame image to the last frame image in the second video according to a preset cropping rule to obtain an image to be detected containing the target object in each frame image, and detecting the image to be detected containing the target object in each frame image to obtain the throwing position of the target object in each frame image, wherein the throwing position is represented by a bounding box.

[0009] In some embodiments, cropping each frame image from the target frame image to the last frame image in the second video according to a preset cropping rule to obtain an image to be detected containing a target object in each frame image, and detecting the image to be detected containing a target object in each frame image to obtain the throwing position of the target object in each frame image includes: detecting the target object in the target frame image to obtain the throwing position of the target object in the target frame image, and taking the center point of the throwing position as the cropping center; based on the cropping center, cropping the next frame image of the target frame image according to a preset cropping size to obtain Obtain the image to be detected containing the target object in the subsequent frame image; detect the target object in the image to be detected in the subsequent frame image to obtain the throwing position of the target object in the subsequent frame image, and use the center point of the throwing position of the target object in the subsequent frame image as a new cropping center, and use the subsequent frame image as a new target frame image; and continue to execute the step of cropping the subsequent frame image of the target frame image according to a preset cropping size based on the cropping center to obtain the image to be detected containing the target object in the subsequent frame image, until the throwing position of the target object in the last frame image is obtained.

[0010] In some embodiments, detecting the target object in the target frame image to obtain the throwing position of the target object in the target frame image includes: cropping the target frame image based on the center point of the target frame image to obtain a cut image on the throwing direction side as the image to be detected of the target frame image containing the target object; and detecting the target object in the image to be detected of the target frame image to obtain the throwing position of the target object in the target frame image.

[0011] In some embodiments, determining the landing position of the target object according to the throwing position includes: determining the vertical speed of the target object in each frame image according to the throwing position; determining the landing condition and landing frame of the target object according to the vertical speed of the target object in each frame image; and determining the throwing position of the target object in the landing frame as the landing position.

[0012] In some embodiments, determining the landing condition and landing frame of the target object according to the vertical speed of the target object in each frame image includes: comparing the vertical speed of the target object in each frame image, when the vertical speed of the Nth frame image is opposite to the vertical speed of the N-1th frame image, determining that the landing condition of the target object is ground bounce, and determining the N-1th frame image as the landing frame, wherein N is a positive integer greater than 1; or, when the vertical speeds of consecutive M frame images starting from the Nth frame image are all less than a preset threshold, determining that the landing condition of the target object is ground roll, and determining the Nth frame image as the landing frame, wherein M is a positive integer greater than 2.

[0013] In some embodiments, determining the throwing distance of the target object based on the landing position and the preset calibration area and using perspective transformation includes: determining whether the landing position is located in the preset calibration area, wherein the preset calibration area is used to identify the effective throwing area of ​​the target object and includes multiple preset calibration blocks; when the landing position is not located in the preset calibration area, determining that the target object is out of bounds and the target person has no throwing score; when the landing position is located in the preset calibration area, determining the preset calibration block where the target object is located according to the landing position; determining the mapping matrix of the preset calibration block according to the image position information and actual position information of the preset calibration block, wherein the image position information identifies the position of the preset calibration block in the image, and the actual position information identifies the position of the preset calibration block in the throwing venue; using the mapping matrix to perform perspective transformation on the center point of the landing position to obtain the relative distance of the target object in the preset calibration block; and determining the throwing distance of the target object based on the relative distance and the preset throwing circle.

[0014] In a second aspect, the present disclosure provides an electronic device comprising: a processor; and a memory storing program instructions for evaluating a throwing motion, wherein when the program instructions are executed by the processor, the method described in the first aspect and its multiple embodiments are implemented.

[0015] In a third aspect, the present disclosure provides a computer-readable storage medium having stored thereon program instructions for evaluating a throwing motion, wherein when the program instructions are executed by a processor, the method described in the first aspect and its multiple embodiments are implemented.

[0016] Through the scheme for evaluating the throwing motion provided above, the first video of the preparation phase and the second video of the motion phase can be detected and analyzed frame by frame to obtain the foot position of the target person and the landing position of the target object. Furthermore, the preset throwing circle of the venue can be combined to determine whether the target person has violated the rules, and the preset calibration area of ​​the venue can be combined, and the perspective transformation can be used to determine the throwing distance of the target object to obtain the throwing score of the target person. The scheme disclosed in this invention does not require cumbersome equipment construction and maintenance, and the evaluation process is simplified, thereby improving the evaluation efficiency. In addition, the scheme disclosed in this invention can use perspective transformation to accurately calculate the throwing distance of the target object, which can reduce the impact of image distortion on the accuracy of the evaluation results, and help improve the accuracy and reliability of the evaluation results.

[0017] Furthermore, in some embodiments, when detecting the feet of the target person in each frame image, the human body detection model can be used to perform human body detection on each frame image of the first video to obtain the human body detection results of each frame image, and the target person in each frame image is determined according to the human body detection results. Therefore, in the subsequent key point detection, foot partial image interception and foot segmentation, only the target person can be focused on, without paying attention to and processing other persons appearing in the first video. Based on this, the computational complexity and computational cost can be effectively reduced, which helps to improve the evaluation efficiency.

[0018] Furthermore, in some embodiments, pixel-level segmentation of the target person's feet can be achieved through key point detection, foot local image capture, and foot segmentation. The pixel-level foot segmentation result can more accurately capture the target person's line-stepping behavior, thereby ensuring the fairness of the game.

[0019] During actual throwing, the target object is very small in the image. In order to improve the recall rate of the target object, a larger model and shallower network features are usually required, which will cause a certain degree of resource waste. Based on this, in some embodiments, when detecting the target object in each frame image of the second video, the target frame image in which the target object appears for the first time in the second video can be determined first. Then, each frame image from the target frame image to the last frame image in the second video can be cropped according to the preset cropping rules to obtain the image to be detected containing the target object in each frame image. Then, the image to be detected containing the target object in each frame image can be detected to obtain the throwing position of the target object in each frame image. Thus, while ensuring the recall rate of the target object, a larger model and shallower network features are no longer required, thereby reducing resource waste. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] By reading the detailed description below with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0021] Figure 1 is an exemplary flow chart of a method for evaluating a throwing motion according to some embodiments of the present disclosure;

[0022] Figure 2 It is shown Figure 1 An exemplary flow chart of a step in the method;

[0023] Figure 3 It is shown Figure 1 an exemplary flow chart of another step in the method;

[0024] Figure 4 is an exemplary flow chart showing a method for determining a landing position of a target object according to a throwing position according to other embodiments of the present disclosure;

[0025] Figure 5 is a block diagram showing an exemplary structure of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0027] It should be understood that when the terms "first", "second", "third" and "fourth" are used in the claims, the specification and the drawings of the present application, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the specification and claims of the present disclosure indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0028] It should also be understood that the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. As used in this disclosure and claims, the singular forms of "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this disclosure and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.

[0029] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0030] The specific implementation of the present disclosure is described in detail below with reference to the accompanying drawings.

[0031] Overview of this Disclosure

[0032] In the process of obtaining the scheme disclosed herein, as mentioned in the above background technology description, the applicant found that the efficiency and accuracy of the existing throwing sports evaluation methods in the industry are not ideal. Specifically, the evaluation methods that rely on sensors and gravity sensing are cumbersome to set up and inconvenient to maintain, resulting in low evaluation efficiency. The evaluation method based on sensors requires the configuration of a spherical sensing unit, a venue information unit and a server processing center, while the evaluation method that relies on gravity sensing requires the laying of a gravity sensing layer on the venue. The equipment for these two solutions is cumbersome to set up and inconvenient to maintain, and the evaluation efficiency is low, making it difficult to promote them widely.

[0033] In this regard, the applicant has found through research that the throwing distance can be obtained by detecting and analyzing the video of the target person throwing the target object to be evaluated, thereby avoiding cumbersome equipment construction and maintenance, thereby improving the evaluation efficiency. Based on this, the applicant has further found that after obtaining the landing position of the target object, the perspective transformation can be used to overcome the influence of image distortion on the accuracy of the evaluation results, thereby accurately calculating the throwing distance of the target object and improving the accuracy and reliability of the evaluation results.

[0034] For ease of understanding, before introducing the solution of this disclosure in detail, the technical terms involved in this disclosure are first introduced as follows:

[0035] Object detection: Object detection is an important task in the field of computer vision, which aims to identify and locate specific objects in images or video frames.

[0036] Keypoint detection: Keypoint detection is a technology in the field of computer vision that is used to identify and locate specific significant feature points in an image or video frame, which are usually features such as the hands and feet of the human body. Keypoints are stable and distinguishable points in an image that can be re-identified and matched even after the image has been transformed, such as rotation, scaling, or partial occlusion.

[0037] Semantic segmentation: Semantic segmentation is an important task in the field of computer vision, which aims to assign each pixel in an image to a specific category label, thereby distinguishing different objects and scenes in the image. In the solution disclosed in this disclosure, the foot is segmented.

[0038] Object tracking: Object tracking is a key technology in the field of computer vision, which involves locating and tracking one or more target objects in a video sequence. The object tracking algorithm needs to maintain the recognition of the target between consecutive video frames. In the disclosed solution, object tracking can associate video-level frame relationships within a video and record trajectories.

[0039] Face recognition: Face recognition is a biometric technology that authenticates and identifies a person based on their facial features, meaning that given a facial image, the true identity of the corresponding athlete can be identified.

[0040] Next, combine Figure 1 The method 100 for evaluating a throwing motion according to some embodiments of the present disclosure is exemplarily introduced. It is understood that the method 100 may be executed by any appropriate device having data processing capabilities, such as but not limited to a terminal device and a server.

[0041] like Figure 1As shown, at step S101, method 100 can obtain a video to be evaluated of a target person throwing a target object. The throwing movement disclosed herein can include a preparation phase and a movement phase, and the video to be evaluated includes a first video of the preparation phase and a second video of the movement phase.

[0042] The target person of the present disclosure is an athlete who is preparing to perform a throwing sport. The target object is the throwing object of the target person in the throwing sport, and can be any appropriate sphere that can be thrown, for example, it can include but is not limited to a shot put and a solid ball. In the throwing venue, a preset throwing circle and a preset calibration area for determining the throwing results can be pre-drawn. The preset calibration area is used to identify the effective throwing area of ​​the target object, and can include multiple preset calibration blocks. The preset calibration block has a corresponding preset calibration line, and the preset calibration line can be used to calibrate the relative distance between the preset calibration block and the inner edge of the preset throwing circle.

[0043] In the preparation stage of the throwing movement, the target person should be located in a preset throwing circle, holding the target object in hand, and the body is in a ready position. After the start throwing instruction (or the start throwing command or the start throwing voice, etc.) sounds, the target person can throw the target object. Under the action of the thrust, the target object separates from the hand and begins to fly in the air, thus entering the movement stage until it finally lands. The start throwing instruction, the start throwing command or the start throwing voice here can be outputted audibly by the execution device of method 100.

[0044] After the target object lands, the throwing score of the target person can be determined according to the landing position of the target object. Specifically, when the landing position of the target object is not located in the preset calibration area, it can be determined that the target object is out of bounds and the target person has no throwing score. When the landing position of the target object is located in a preset calibration block (in order to distinguish, it can be called a landing calibration block) in the preset calibration area, the relative distance of the target object in the landing calibration block can be determined based on the landing position and the predetermined calibration line of the landing calibration block (in order to distinguish, it can be called a first relative distance). Then, the absolute distance of the target object (or it can also be called a throwing distance) can be determined based on the first relative distance and the relative distance from the predetermined calibration line of the landing calibration block to the inner edge of the preset throwing circle (in order to distinguish, it can be called a second relative distance). The throwing distance is equal to the sum of the first relative distance and the second relative distance.

[0045] In the solution disclosed herein, at least one camera device may be pre-installed at the throwing venue to collect the video to be evaluated. Furthermore, high poles may be pre-installed at the throwing venue to install at least one camera device to a specified height. The execution device of method 100 may be connected to the at least one camera device in communication to obtain the video to be evaluated from the at least one camera device, and the throwing motion may be evaluated by detecting and analyzing the video to be evaluated.

[0046] In one implementation scenario, a camera device can be pre-installed at the throwing venue, and the preparation phase and the motion phase can be distinguished by a video data processing algorithm, so as to divide the video to be evaluated into a first video and a second video. In order to reduce the difficulty of video data processing and improve the efficiency of throwing motion evaluation, in another implementation scenario, two cameras can be pre-installed at the throwing venue (for distinction, they can be called the first camera device and the second camera device), and the first camera device is used to obtain the first video of the preparation phase, and the second device is used to obtain the second video of the motion phase.

[0047] It is understandable that those skilled in the art can select the position, shape and size of the preset throwing circle and multiple preset calibration blocks according to actual needs, and can select the installation position, installation height and shooting orientation of the camera equipment according to actual needs. In actual operation, before using the camera equipment to collect the video to be evaluated, the installation position, installation height and shooting orientation of the camera equipment can be debugged or calibrated to minimize the image distortion of the preset throwing circle and multiple preset calibration blocks in the image captured by the camera equipment. Thereafter, when using the camera equipment to collect the video to be evaluated of each target person, the installation position, installation height and shooting orientation of the camera equipment should be kept unchanged to ensure that the acquisition conditions of the video to be evaluated are consistent, reduce the influence of external factors on the accuracy of the evaluation results, and help improve the fairness of the competition.

[0048] In one embodiment, before obtaining a video to be evaluated of a target person throwing a target object, a reference image containing only a preset throwing circle and a plurality of preset calibration blocks can be obtained. Then, any appropriate image detection or image segmentation algorithm can be used to determine the image position information of the preset throwing circle and the plurality of preset calibration blocks in the reference image. Since the mounting position, mounting height and shooting orientation of the camera device remain unchanged, the image position information of the preset throwing circle and the plurality of preset calibration blocks in each frame of the video image to be evaluated will also remain unchanged, so that the image position information determined here can be applied to subsequent links such as determining the target person, determining whether line-stepping behavior occurs, and determining the throwing results.

[0049] Next, at step S102, method 100 can detect the feet of the target person in each frame of the first video to obtain the foot position of the target person in each frame. Further, method 100 can determine whether the target person has violated the rules, such as stepping on the line or crossing the line, based on the relative position of the foot position and the preset throwing circle. The stepping on the line behavior disclosed herein refers to the target person's feet stepping on the preset throwing circle, and the crossing the line behavior refers to the target person's feet crossing the preset throwing circle and appearing outside the preset throwing circle, both of which are behaviors that do not comply with the throwing rules. In order to facilitate understanding of how to detect the feet of the target person in each frame of the first video to obtain the foot position of the target person in each frame, it will be combined later. Figure 2 The step S102 is described in detail.

[0050] Next, at step S103, the method 100 can detect the target object in each frame image of the second video to obtain the position of the target object in each frame image (for distinction, it can also be called the throwing position), and determine the landing position of the target object according to the throwing position. Specifically, according to the landing situation of the target object, the landing frame and landing position of the target object can be determined from the throwing position of the target object in each frame image.

[0051] In practical applications, the landing condition of the target object may include at least ground bounce and ground roll. The landing frame is an image in the second video, in which the target object first touches the ground. In each frame image before the landing frame, the target object has not yet touched the ground and is still in the air. The landing position is the throwing position of the target object in the landing frame. In order to facilitate understanding of how to detect the target object in each frame image of the second video to obtain the throwing position of the target object in each frame image, we will combine Figure 3 The step S103 is described in detail. Figure 4 A detailed description is given of how to determine the landing position of the target object based on the throwing position.

[0052] Next, at step S104, the method 100 can determine whether the target person has violated the rules based on the foot position and the preset throwing circle. Further, the method 100 can also determine whether to execute step S105 or step S106 based on the determination result of step S104. Specifically, when it is determined that the target person has violated the rules, the method 100 can execute step S105; when it is determined that the target person has not violated the rules, the method 100 can execute step S106.

[0053] At step S105, the method 100 may determine that the evaluation result of the target person is not in compliance with the throwing rules and has no throwing results.

[0054] At step S106 , the method 100 may determine the throwing distance of the target object based on the landing position and the preset calibration area and by using perspective transformation to obtain the throwing score of the target person.

[0055] In the scheme disclosed in the present invention, the perspective transformation is the transformation between the actual position information and the image position information of the preset calibration block, and the mapping matrix between the actual position information and the image position information can be used to represent the perspective transformation. Here, the image position information can identify the position, shape and size of the preset calibration block in the image, and the actual position information can identify the position, shape and size of the preset calibration block in the throwing field. The throwing distance of the target object refers to the distance from the landing position of the target object in the throwing field to the inner edge of the preset throwing circle. The throwing results disclosed in the present invention can be the throwing distance, or it can be the score obtained by mapping the throwing distance. It can be understood that those skilled in the art can determine the mapping relationship between the throwing distance and the score according to actual needs, for example, it can be determined that the present disclosure does not make specific limitations on this.

[0056] In the scheme of the present disclosure, after obtaining the throwing score of the target person, method 100 can also associate the throwing score with the real identity of the target person. In order to obtain the real identity of the target person, in the scheme of the present disclosure, face recognition technology can be used to authenticate and identify the target person in the aforementioned first video based on the facial feature information of the target person to obtain the real identity of the target person. It should be noted that the face recognition technology here can be any existing or future face recognition technology, as long as it can identify the real identity of the target person, it can be used in the scheme of the present disclosure, and the present disclosure does not make specific limitations on this.

[0057] In the solution disclosed herein, after associating the throwing result with the real identity of the target person, the method 100 can also combine the throwing result and the real identity of the target person into an evaluation result, and output the evaluation result externally. Additionally or alternatively, the method 100 can also output the evaluation result externally in a visual or audible manner.

[0058] Combination of the above Figure 1A method 100 for evaluating a throwing sport is described. The method 100 can obtain the foot position of the target person and the landing position of the target object by performing frame-by-frame detection and analysis on a first video in a preparation phase and a second video in a motion phase. Furthermore, it can be combined with a preset throwing circle of a throwing venue to determine whether the target person has committed a violation, and combined with a preset calibration area of ​​the throwing venue, and using perspective transformation to determine the throwing distance of the target object, so as to obtain the throwing score of the target person. The use of the scheme disclosed in the present invention does not require cumbersome equipment construction and maintenance, and the evaluation process is simplified, thereby improving evaluation efficiency. In addition, the scheme disclosed in the present invention can use perspective transformation to accurately calculate the throwing distance of the target object, which can reduce the impact of image distortion on the accuracy of the evaluation results, and help improve the accuracy and reliability of the evaluation results.

[0059] Figure 2 Shows Figure 1 An exemplary flow chart of step S102 in the method. It is understood that the following Figure 2 The description is a specific implementation of the aforementioned step S102. Figure 1 The features described in can be applied analogously here.

[0060] like Figure 2 As shown, at step S1021, method 100 can use a human body detection model to perform human body detection on each frame image of the first video to obtain the human body detection results of each frame image, and determine the target person in each frame image according to the human body detection results. The human body detection model disclosed herein can adopt the yolov8 target detection model, and the human body detection result can at least include a bounding box and a confidence level. Thus, the target person in each frame image can be determined based on whether the confidence level is greater than a preset threshold and whether the bounding box is within a preset throwing circle. In one embodiment, a person whose confidence level is greater than a preset threshold and whose bounding box is within a preset throwing circle can be determined as a target person. It is understandable that those skilled in the art can select a specific value or value range of the preset threshold according to actual needs, and the disclosed embodiment is not limited in this respect.

[0061] In the scheme disclosed herein, after the target person in each frame image is determined, in order to achieve continuous attention to the throwing behavior of the target person, the target person in each frame image should also be tracked. In one embodiment, the ByteTrack target tracking algorithm can be used to achieve the associated tracking of the target person.

[0062] At step S1022, method 100 can use the key point detection model to perform key point detection on the target person in each frame image to obtain the key point detection result of the target person in each frame image. The key point detection model disclosed herein can adopt the hrnet top-down key point detection model, and the key point detection result includes the foot key points of the target person. Further, at step S1023, method 100 can intercept the foot partial image of the target person in each frame image according to the foot key points in the key point detection result. Here, the foot partial image is the minimum rectangular image containing the foot of the target person. The size of the minimum rectangular image can be determined by the position of the foot key points.

[0063] At step S1024, method 100 can use the segmentation model to segment the foot of the local image of the foot to obtain the foot position of the target person in each frame image. The segmentation model disclosed herein can use the PPHumanSeg segmentation model, which can perform pixel-level segmentation on the foot along the contour or edge of the foot to obtain the pixel-level foot position. Based on the pixel-level foot position, it can be determined whether any edge position of the foot has stepped on the line, so that the target person's stepping on the line behavior can be captured more accurately, thereby ensuring the fairness of the game.

[0064] Figure 3 Shows Figure 1 An exemplary flow chart of step S103 in the method. It is understood that the following Figure 3 The description is a specific implementation of the aforementioned step S103. Figure 1 The features described in can be applied analogously here.

[0065] like Figure 3 As shown, at step S1031, the method 100 may determine the target frame image in which the target object appears for the first time in the second video. The first appearance here refers to the first time that the target object appears alone in the image after being separated from the hand.

[0066] At step S1032, the method 100 can crop each frame image from the target frame image to the last frame image in the second video according to the preset cropping rule to obtain an image to be detected containing the target object in each frame image. The preset cropping rule disclosed herein consists of a cropping center and a preset cropping size. Based on the cropping center, the image is cropped according to the preset cropping size, and a sub-image of the preset cropping size of the image can be obtained, and the image center of the sub-image is the cropping center.

[0067] Based on the description of the context, it can be known that the position of the target object in each frame image is different, so the cutting center of each frame image should also be different, so that the image to be detected containing the target object in each frame image can be obtained. Thus, the cutting center of each frame image can be dynamically determined. In one embodiment, the throwing position of the target object in the previous frame image can be used as the cutting center of the current frame image. Similarly, the throwing position of the target object in the current frame image can be used as the cutting center of the next frame image. By analogy, the cutting center of each frame image can be determined. It can be understood that the preset cutting size should be larger than the size of the target object in the image so that the image to be detected can contain the target object. In actual operation, those skilled in the art can select the specific value of the preset cutting size according to actual needs or the actual size of the target object, and this disclosure does not specifically limit it. Preferably, the preset cutting size can be 640*640.

[0068] At step S1033, method 100 can detect the image to be detected containing the target object in each frame image to obtain the throwing position of the target object in each frame image. Here, the image to be detected containing the target object is detected to obtain the throwing position of the target object in the image to be detected. Then, based on the relative position relationship between the image to be detected and the original image, the throwing position of the target object in the original image can be obtained. The throwing position of the target object disclosed in the present invention can be represented by its enclosing frame, and the relative position relationship can be determined by the aforementioned cutting center and preset cutting size.

[0069] In actual operation, step S1032 and step S1033 can be performed alternately until the throwing position of the target object in the last frame image is obtained. Specifically, the target object in the target frame image can be detected first to obtain the throwing position of the target object in the target frame image, and the center point of the throwing position is used as the cutting center. Then, based on the cutting center, the next frame image of the target frame image can be cut according to the preset cutting size to obtain the image to be detected containing the target object in the next frame image. Then, the target object in the image to be detected of the next frame image can be detected to obtain the throwing position of the target object in the next frame image, and the center point of the throwing position of the target object in the next frame image can be used as the new cutting center, and the next frame image can be used as the new target frame image. Thereafter, the aforementioned cutting center can be continued to be used to cut the next frame image of the target frame image according to the preset cutting size to obtain the image to be detected containing the target object in the next frame image, until the throwing position of the target object in the last frame image is obtained.

[0070] In the scheme disclosed herein, when detecting the target object in the target frame image where the target object appears for the first time to obtain the throwing position of the target object in the target frame image (for distinction, it can be called the first throwing position), the target frame image can be cropped based on the center point of the target frame image, and the cut image on the throwing direction side can be obtained as the image to be detected of the target frame image containing the target object. Then, the target object in the image to be detected can be detected, and the first throwing position of the target object can be obtained. In one example, assuming that the size of the target frame image is 1920*1080, its center point is 960*960, and the left side of the target frame image is the throwing direction side, then the image of 960*1080 on the left side of the target frame image can be cropped as the image to be detected.

[0071] In actual operation, when the horizontal speed (or also referred to as the initial horizontal speed) or the vertical speed (or also referred to as the initial vertical speed) of the target object when it first appears is not greater than the preset speed, it can be determined that the throwing distance of the target object is insufficient and the target person has no throwing achievement. In this case, it is not necessary to detect the target object in other frame images after the target frame image in which the target object first appears. Here, the initial horizontal speed of the target object can be determined by the horizontal displacement of the target object between two consecutive frame images, and the initial vertical speed of the target object can be determined by the vertical displacement of the target object between two consecutive frame images.

[0072] It should be noted that, similar to the aforementioned detection and associated tracking of the target person in each frame image of the first video, the present disclosure can use the yolov8 target detection model to detect the target object in each frame image of the second video, and use the ByteTrack target tracking algorithm to achieve associated tracking of the target object. In addition, the present disclosure can also use the time series model positioning to track and locate the target object in each frame image of the second video.

[0073] Additionally or optionally, the illegal behavior disclosed herein may also include two-handed throwing, and a timing model may be used to analyze whether the target person is throwing with one hand. When the target person is throwing with one hand, it may be determined that the throwing action of the target person meets the throwing rules. On the contrary, when the target person is not throwing with one hand (e.g., throwing with two hands), it may be determined that the throwing action of the target person does not meet the throwing rules, is an illegal behavior, and the target person has no throwing results.

[0074] After obtaining the throwing position of the target object in each frame image, the method 100 can determine the landing position of the target object according to the throwing position. Figure 4, which shows an exemplary flow chart of a method 400 for determining the landing position of a target object according to a throwing position provided by an embodiment of the present disclosure. It can be understood that the method 400 is a specific implementation of the aforementioned step S103, so the aforementioned method 400 is combined with the method described in the embodiment of the present disclosure. Figure 1 and Figure 3 The features described can analogously apply here.

[0075] like Figure 4 As shown, at step S401, method 400 may determine the vertical speed of the target object in each frame of image according to the throwing position of the target object in each frame of image.

[0076] In one embodiment, for any two consecutive frames of images in each frame image (for distinction, they may be referred to as the first image and the second image), the vertical displacement of the target object in the second image relative to the target object in the first image may be used as the vertical velocity of the target object in the second image. Further, the vertical displacement of the target object in the second image relative to the target object in the first image may be determined in the following manner: respectively determining the center point of the throwing position of the target object in the second image and the first image (for distinction, they may be referred to as the second center point and the first center point); determining the vertical displacement of the target object based on the vertical coordinates of the second center point and the vertical coordinates of the first center point.

[0077] Next, at step S402, the method 400 may determine the landing condition and landing frame of the target object according to the vertical speed of the target object in each frame image. In actual operation, the method 400 may determine the landing condition and landing frame of the target object according to the direction and / or magnitude of the vertical speed of the target object in each frame image. Specifically, the vertical speed of the target object in each frame image may be compared. When the vertical speed of the Nth frame image is opposite to the vertical speed of the N-1th frame image, it may be determined that the landing condition of the target object is a ground bounce, and the N-1th frame image is determined to be a ground bounce. In addition, when the vertical speeds of the consecutive M frames of images starting from the Nth frame image are all less than a preset threshold, it may be determined that the landing condition of the target object is a ground roll, and the Nth frame image is determined to be a ground frame. Here, N is a positive integer greater than 1, and M is a positive integer greater than 2.

[0078] Furthermore, at step S403, method 400 may determine the throwing position of the target object in the landing frame as the landing position. Furthermore, the throwing distance of the target object may be determined based on the landing position and the preset calibration area and by using perspective transformation.

[0079] In one embodiment, when determining the throwing distance of the target object based on the landing position and the preset calibration area and using perspective transformation, it can be determined whether the landing position is located in the preset calibration area. When the landing position is not located in the preset calibration area, it can be determined that the target object is out of bounds and the target person has no throwing score. When the landing position is located in the preset calibration area, the preset calibration block where the target object is located can be determined according to the landing position. Then, the mapping matrix of the preset calibration block can be determined according to the image position information and the actual position information of the preset calibration block. Then, the center point of the landing position can be perspectively transformed using the mapping matrix to obtain a new center point. Then, the relative distance of the target object in the preset calibration block can be determined according to the new center point and the preset calibration line in the preset calibration block. Finally, the throwing distance of the target object can be determined based on the relative distance and the relative distance from the preset calibration line to the inner edge of the preset throwing circle.

[0080] Next, combine Figure 5 An electronic device 500 provided in the embodiment of the present disclosure is exemplarily introduced. Figure 5 As shown, the electronic device 500 of the embodiment of the present disclosure may include a processor 501 , a memory 502 , and a communication bus 503 .

[0081] In the process of a specific embodiment, the processor 501 may be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing image processing device (DSPD), a programmable logic image processing device (PLD), a field programmable gate array (FPGA), a CPU, a controller, a microcontroller, and a microprocessor. It is understandable that for different devices, the electronic device used to implement the function of the processor may also be other, which is not specifically limited in this embodiment.

[0082] In the embodiment of the present disclosure, the communication bus 503 is used to realize the connection and communication between the processor 501 and the memory 502; the memory 502 stores program instructions for evaluating the throwing motion; when the processor 501 executes the program instructions stored in the memory 502, the present disclosure is realized in combination with the attached Figures 1 to 4 A method for evaluating a throwing motion is described.

[0083] Combination of the above Figure 5The device for evaluating the throwing motion that can be used to perform the present disclosure is described. It should be understood that the device structure or architecture here is only exemplary, and the implementation and implementation entity of the present application are not limited thereto, but can be changed without departing from the spirit of the present application.

[0084] It is understandable that the description of each embodiment in this disclosure emphasizes the differences between the embodiments, and the same or corresponding parts can be referenced to each other. For the purpose of brevity, this disclosure will not repeat them one by one.

[0085] According to the above description in combination with the accompanying drawings, those skilled in the art can also understand that the embodiments of the present application can also be implemented by a software program. Therefore, the present application also provides a computer-readable storage medium. The computer-readable storage medium can be used to implement the present disclosure in combination with the accompanying drawings. Figures 1 to 4 A method for evaluating a throwing motion is described.

[0086] It should be noted that although the operations of the method of the present application are described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flow chart can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0087] Although multiple embodiments of the present disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art may think of many changes, modifications, and alternatives without departing from the thought and spirit of the present disclosure. It should be understood that in the process of practicing the present disclosure, various alternatives to the embodiments of the present disclosure described herein may be adopted. The attached claims are intended to define the scope of protection of the present disclosure, and therefore cover equivalents or alternatives within the scope of these claims.

[0088] The collection and acquisition of various data in this application complies with relevant laws and regulations and is authorized by the data provider. Any organization or individual that needs to obtain external data must obtain authorization in accordance with the law and ensure data security. It is not allowed to illegally collect, use, process, or transmit unauthorized or unprotected data, or to illegally buy, sell, provide, or disclose unauthorized or unprotected data.

Claims

1. A method for evaluating a throwing motion, comprising: Acquire a video to be evaluated of a target person throwing a target object, wherein the throwing motion includes a preparation phase and a movement phase, and the video to be evaluated includes a first video of the preparation phase and a second video of the movement phase; Detecting the feet of the target person in each frame image of the first video to obtain the foot position of the target person in each frame image; Detecting the target object in each frame image of the second video to obtain a throwing position of the target object in each frame image, and determining a landing position of the target object according to the throwing position; Determining whether the target person has committed a violation based on the foot position and the preset throwing circle; and In response to the target person not having any illegal behavior, the throwing distance of the target object is determined based on the landing position and the preset calibration area and by using perspective transformation to obtain the throwing score of the target person.

2. The method according to claim 1, wherein: Detecting the feet of the target person in each frame image of the first video to obtain the foot position of the target person in each frame image includes: Performing human body detection on each frame image of the first video using a human body detection model to obtain a human body detection result for each frame image, and determining a target person in each frame image according to the human body detection result; Performing key point detection on the target person in each frame image using a key point detection model to obtain a key point detection result of the target person in each frame image; A partial image of the foot of the target person in each frame image is captured according to the key point detection result; and the partial image of the foot is segmented by the segmentation model to obtain the foot position of the target person in each frame image.

3. The method according to claim 1, wherein: Detecting the target object in each frame image of the second video to obtain the throwing position of the target object in each frame image includes: Determine a target frame image in which the target object appears for the first time in the second video; The frame images from the target frame image to the last frame image in the second video are cropped according to preset cropping rules to obtain an image to be detected containing a target object in each frame image, and the image to be detected containing a target object in each frame image is detected to obtain a throwing position of the target object in each frame image, wherein the throwing position is represented by a bounding box.

4. The method according to claim 3, wherein: Cropping each frame image from the target frame image to the end frame image in the second video according to a preset cropping rule to obtain an image to be detected containing a target object in each frame image, and detecting the image to be detected containing a target object in each frame image to obtain a throwing position of the target object in each frame image includes: Detecting the target object in the target frame image to obtain the throwing position of the target object in the target frame image, and taking the center point of the throwing position as the cutting center; Based on the cropping center, cropping the subsequent frame image of the target frame image according to a preset cropping size to obtain an image to be detected containing the target object in the subsequent frame image; Detecting the target object in the image to be detected of the subsequent frame image to obtain the throwing position of the target object in the subsequent frame image, and taking the center point of the throwing position of the target object in the subsequent frame image as a new cropping center, and taking the subsequent frame image as a new target frame image; and Continue to execute the step of cropping the subsequent frame image of the target frame image based on the cropping center according to a preset cropping size to obtain an image to be detected containing the target object in the subsequent frame image, until the throwing position of the target object in the last frame image is obtained.

5. The method according to claim 4, wherein: Detecting the target object in the target frame image to obtain the throwing position of the target object in the target frame image includes: Cutting the target frame image based on the center point of the target frame image to obtain a cut image on the throwing direction side as the image to be detected of the target frame image containing the target object; and The target object in the to-be-detected image of the target frame image is detected to obtain the throwing position of the target object in the target frame image.

6. The method according to claim 3, wherein: Determining the landing position of the target object according to the throwing position includes: Determine the vertical speed of the target object in each frame of image according to the throwing position; Determining the landing condition and landing frame of the target object according to the vertical speed of the target object in each frame image; and The throwing position of the target object in the landing frame is determined as the landing position.

7. The method according to claim 6, wherein: Determining the landing condition and landing frame of the target object according to the vertical speed of the target object in each frame image includes: Comparing the vertical speeds of the target object in each frame image, when the vertical speed of the Nth frame image is opposite to the vertical speed of the N-1th frame image, determining that the landing condition of the target object is a ground bounce, and determining the N-1th frame image as a landing frame, wherein N is a positive integer greater than 1; or When the vertical speeds of M consecutive frames of images starting from the Nth frame are all less than a preset threshold, it is determined that the landing condition of the target object is ground rolling, and the Nth frame of image is determined to be a landing frame, wherein M is a positive integer greater than 2.

8. The method according to claim 1, wherein: Determining the throwing distance of the target object based on the landing position and the preset calibration area and using perspective transformation includes: Determining whether the landing position is located within the preset calibration area, wherein the preset calibration area is used to identify a valid throwing area of ​​the target object and includes a plurality of preset calibration blocks; When the landing position is not within the preset calibration area, it is determined that the target object is out of bounds and the target person has no throwing score; When the landing position is within the preset calibration area, determining the preset calibration block where the target object is located according to the landing position; Determine a mapping matrix of the preset calibration block according to the image position information and the actual position information of the preset calibration block, wherein the image position information identifies the position of the preset calibration block in the image, and the actual position information identifies the position of the preset calibration block in the throwing field; The mapping matrix is ​​used to perform a perspective transformation on the center point of the landing position to obtain a relative distance of the target object in the preset calibration block; and the throwing distance of the target object is determined based on the relative distance and the preset throwing circle.

9. An electronic device, comprising: processor; as well as A memory storing program instructions for evaluating a throwing motion, wherein when the program instructions are executed by the processor, the electronic device executes the method according to any one of claims 1-8.

10. A computer-readable storage medium storing program instructions for evaluating a throwing motion, wherein when the program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images