Method and device for evaluating running exercise and computer storage medium

Abstract

The invention discloses a method and device for evaluating running exercise and a computer storage medium. The method comprises the steps that human body frames and face frames of multiple athletes in a current frame image are obtained; constructing a binding relationship between the human body frame and the human face frame and a human face tracking relationship between the human face frame and a historical human face frame; determining a target athlete hitting a line based on the human body frame, and taking the acquisition time of the current frame image as the line hitting time of the target athlete; based on the binding relationship and the face tracking relationship, determining a plurality of target face images, meeting a preset quality condition, of the target athlete; and performing face recognition on the plurality of target face images to obtain identity information of the target athlete, and determining an evaluation result based on the line collision time and the identity information. By utilizing the scheme disclosed by the invention, the purchase, maintenance and update cost of equipment can be reduced, and the influence of external factors on the competition state and competition experience of athletes can be reduced.

Description

Technical Field

[0001] The present disclosure generally relates to the field of video processing technology. More specifically, the present disclosure relates to a method, an electronic device and a computer-readable storage medium for evaluating a running motion. Background Art

[0002] The traditional method of running performance evaluation and recording relies on manual timing, which relies on human reaction, resulting in insufficient accuracy and a large consumption of manpower, so intelligent evaluation and recording of running performance is needed. Thanks to the development of science and technology, especially the widespread application of software and hardware facilities, the traditional manual timing recording method is gradually being replaced by intelligent technology, and the evaluation of running performance can be achieved in an automated and intelligent way.

[0003] However, existing automated and intelligent monitoring methods usually require athletes to wear specific monitoring equipment or wear clothes with number tags in order to accurately identify the athletes, which undoubtedly increases the cost of equipment acquisition, maintenance and update. At the same time, the presence of monitoring equipment or number tags may distract athletes and distract them from the game. Especially at critical moments, this distraction may lead to wrong decisions or deformed movements, thus affecting the athletes' game status and game experience.

[0004] In view of this, there is an urgent need to provide a solution for evaluating running sports in order to reduce the cost of equipment purchase, maintenance and update, and reduce the impact of external factors on athletes' competition status and competition experience. Summary of the invention

[0005] In order to at least solve one or more of the technical problems mentioned above, the present disclosure proposes a solution for evaluating running motion in multiple aspects.

[0006] In a first aspect, the present disclosure provides a method for evaluating running motion, comprising: obtaining a current frame image of a motion phase of the running motion, wherein the current frame image contains multiple athletes; detecting the bodies and faces of the multiple athletes in the current frame image respectively to obtain body frames and face frames of the multiple athletes; constructing a binding relationship between the body frame and the face frame according to a preset binding strategy, and constructing a face tracking relationship between the face frame and a historical face frame using a preset tracking technology, wherein the historical face frame is a face frame in each frame image before the current frame image; determining a target athlete who hits the finish line in the current frame image based on the body frame, and using the acquisition time of the current frame image as the hit-line time of the target athlete; determining multiple target face images of the target athlete that meet preset quality conditions based on the binding relationship and the face tracking relationship; and performing face recognition on the multiple target face images to obtain identity information of the target athlete, and determining an evaluation result of the target athlete based on the hit-line time and the identity information.

[0007] In some embodiments, the preset binding strategy is to bind a face frame located within a preset range above the human body frame to it, and constructing a binding relationship between the human body frame and the face frame according to the preset binding strategy includes: determining whether the face frame is located within the preset range above the human body frame; when the face frame is located within the preset range above the human body frame, constructing a binding relationship between the human body frame and the face frame.

[0008] In some embodiments, based on the binding relationship and the face tracking relationship, determining multiple target face images of the target athlete that meet preset quality conditions includes: determining whether there is a target face frame bound to the target body frame of the target athlete in the current frame image based on the binding relationship; when the target face frame exists in the current frame image, obtaining multiple target face frames from the target face frame and historical face frames with face tracking relationships therewith; selecting K target face images that meet preset quality conditions from multiple target face images surrounded by the multiple target face frames, wherein K is an integer greater than 0; sorting the K target face images according to quality parameters to obtain a target face image sequence; and obtaining the first N target face images from the target face image sequence, wherein N is an integer less than or equal to K.

[0009] In some embodiments, the preset tracking technology is also used to establish a human body tracking relationship between the human body frame and the historical human body frame, and the historical human body frame is a human body frame in each frame image before the current frame image; the method also includes: when the target human face frame does not exist in the current frame image, obtaining M historical human body frames from historical human body frames that have a human body tracking relationship with the target human body frame, wherein M is an integer greater than 0; determining P historical human face frames bound to the M historical human body frames, wherein P is an integer less than or equal to M; determining the P historical human face frames The number Q of target face images that meet the preset quality conditions among the enclosed P target face images is determined, and whether the number Q is equal to 0; when Q is equal to 0, the identity information of the target athlete is determined to be the preset identity information; when Q is not equal to 0, the Q target face images are sorted according to the quality parameter to obtain a target face image sequence, and it is determined whether Q is less than N; when Q is less than N, the Q target face images in the target face image sequence are used as multiple target face images that meet the preset quality conditions; and when Q is greater than or equal to N, the first N target face images are obtained from the target face image sequence.

[0010] In some embodiments, performing face recognition on N target face images of the target athlete to obtain the identity information of the target athlete includes: obtaining the first target face image of the N target face images as the face image to be recognized; performing face recognition on the face image to be recognized to obtain face recognition information and confidence of the face image to be recognized; determining whether the face image to be recognized is the last one of the N target face images; when the face image to be recognized is not the last one of the N target face images, determining whether the confidence is greater than a preset threshold; when When the confidence level is greater than or equal to the preset threshold, the facial recognition information is used as the identity information of the target athlete; when the confidence level is less than the preset threshold, a target facial image after the facial image to be recognized is obtained as a new facial image to be recognized, and the facial recognition of the facial image to be recognized is continued to be performed to obtain the facial recognition information and confidence level of the facial image to be recognized; and when the facial image to be recognized is the last one of the N target facial images, the facial recognition information of the target facial image with the largest confidence level is used as the identity information of the target athlete.

[0011] In some embodiments, after determining the evaluation result of the target athlete based on the finish line time and the identity information, the method further includes: determining a violation determination result of the target athlete, wherein the violation determination result is any one of compliance of action, false start and crossing the line; determining whether the evaluation result is valid based on the violation determination result; when the evaluation result is valid, outputting the evaluation result externally; and when the evaluation result is invalid, outputting an invalid result prompt externally.

[0012] In some embodiments, determining the violation result of the target athlete includes: obtaining a video to be evaluated of the target athlete in the preparation stage of a running exercise; detecting the feet of the target athlete in each frame of the video to be evaluated to obtain the foot position and foot speed of the target athlete in each frame; and determining the violation result of the target athlete based on the foot position and foot speed.

[0013] In some embodiments, determining the violation result of the target athlete based on the foot position and foot speed includes: determining whether the foot position of the target athlete crosses the position of a preset starting line; when the foot position does not cross the position of the preset starting line, the violation result can be determined to be a compliant action; when the foot position crosses the position of the preset starting line, determining whether the foot speed is greater than the preset speed; when the foot speed is greater than or equal to the preset speed, determining the violation result to be a false start; and when the foot speed is less than the preset speed, determining the violation result to be crossing the line.

[0014] In a second aspect, the present disclosure provides an electronic device, comprising: a processor; and a memory storing program instructions for evaluating running motion, wherein when the program instructions are executed by the processor, the method described in the first aspect and its multiple embodiments are implemented.

[0015] In a third aspect, the present disclosure provides a computer-readable storage medium having stored thereon program instructions for evaluating running motion, wherein when the program instructions are executed by a processor, the method described in the first aspect and its multiple embodiments are implemented.

[0016] Through the scheme for evaluating running provided above, the athlete's body and face can be detected respectively to obtain the athlete's body frame and face frame, and the binding relationship between the athlete's body frame and face frame can also be constructed. Later, when determining the athlete's evaluation score, it can be determined whether the athlete hits the line based on the body frame, and the athlete can be identified based on the face frame bound to the body frame to determine the athlete's true identity, thereby obtaining the athlete's evaluation score. Using the scheme disclosed in this disclosure, athletes do not need any wearable devices such as number plates, bracelets, and specific clothing when running, thereby reducing the cost of equipment purchase, maintenance, and updating, and can reduce the impact of external factors on the athlete's competition status and competition experience. At the same time, when the scheme disclosed in this disclosure identifies the athlete, a face image that meets the preset quality conditions is used for face recognition. On the one hand, it can ensure the accuracy of the athlete's identity information, and on the other hand, it can reduce the number of identity recognition failures, which helps to improve the evaluation efficiency of running.

[0017] Furthermore, in some embodiments, the disclosed solution can determine whether an athlete has violated a rule to obtain a violation determination result, and can determine whether the athlete's evaluation score is valid based on the violation determination result. Specifically, when it is determined that an athlete has a false start or crosses the line, the athlete's evaluation score can be determined to be invalid to ensure the fairness of the game. Furthermore, in some embodiments, pixel-level segmentation of the target person's feet can be achieved through key point detection, partial foot image capture, and foot segmentation. The pixel-level foot segmentation result can more accurately capture the athlete's crossing-the-line behavior, which is more helpful to ensure the fairness of the game.

[0018] In actual running, there are cases where the athlete lowers his head or is deliberately blocked from the moment he appears on the video screen until the end of the run, resulting in the inability to find a face frame that corresponds to the athlete's body frame and meets the predetermined quality conditions, and thus the athlete's true identity cannot be determined. For this situation, the scheme disclosed herein determines the athlete's identity information as preset identity information and records the evaluation results. Subsequently, the athlete's true identity can be determined through manual judgment, and the evaluation results can be assigned to the athlete. Based on this, the scheme disclosed herein can effectively avoid the loss of results, prevent the athlete from running in vain because the evaluation results are not recorded, and help improve the reliability of the method for evaluating running sports. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] By reading the detailed description below with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0020] Figure 1 is an exemplary flow chart of a method for evaluating a running motion according to some embodiments of the present disclosure;

[0021] Figure 2 It is shown Figure 1 An exemplary flow chart of a step in the method;

[0022] Figure 3 It is shown Figure 1 an exemplary flow chart of another step in the method;

[0023] Figure 4 is an exemplary flow chart showing a method for evaluating a running motion according to other embodiments of the present disclosure;

[0024] Figure 5 is a block diagram showing an exemplary structure of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0026] It should be understood that when the terms "first", "second", "third" and "fourth" are used in the claims, the specification and the drawings of the present application, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the specification and claims of the present disclosure indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0027] It should also be understood that the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. As used in this disclosure and claims, the singular forms of "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this disclosure and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.

[0028] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0029] The specific implementation of the present disclosure is described in detail below with reference to the accompanying drawings.

[0030] Overview of this Disclosure

[0031] In the process of obtaining the solution disclosed herein, as mentioned in the above background technology description, the applicant found that the equipment cost and competition experience of the existing running sports evaluation methods in the industry are not ideal. Specifically, existing monitoring methods usually require athletes to wear specific monitoring equipment or wear clothes with number tags in order to accurately identify the identity of the athletes, which increases the cost of equipment purchase, maintenance and update. At the same time, the presence of monitoring equipment or number tags may distract the athletes, causing them to be distracted during the game, affecting the athletes' game state and game experience.

[0032] In this regard, the applicant has found through research that the athlete's true identity can be determined by detecting and binding the athlete's body frame and face frame to avoid the need for athletes to wear wearable devices such as number plates, bracelets and specific clothing when running, thereby reducing the cost of equipment purchase, maintenance and update, and reducing the impact of external factors on athletes' competition status and competition experience. Based on this, the applicant further discovered that when identifying athletes, a face image that corresponds to the athlete's body frame and meets the preset quality conditions can be used for face recognition, thereby ensuring the accuracy of the athlete's identity information. On the other hand, it can reduce the number of identity recognition failures, which helps to improve the evaluation efficiency of running sports. For ease of understanding, before introducing the scheme of this disclosure in detail, the technical terms involved in this disclosure are introduced first, as follows:

[0033] Object detection: Object detection is an important task in the field of computer vision. Its purpose is to identify and locate specific objects, such as people, cars, and animals, in images or video frames.

[0034] Keypoint detection: Keypoint detection is a technology in the field of computer vision that is used to identify and locate specific significant feature points in an image or video frame, which are usually features such as the hands and feet of the human body. Keypoints are stable and distinguishable points in an image that can be re-identified and matched even after the image has been transformed, such as rotation, scaling, or partial occlusion.

[0035] Semantic segmentation: Semantic segmentation is an important task in the field of computer vision, which aims to assign each pixel in an image to a specific category label, thereby distinguishing different objects and scenes in the image. In the solution disclosed in this disclosure, the foot is segmented.

[0036] Object tracking: Object tracking is a key technology in the field of computer vision, which involves locating and tracking one or more target objects in a video sequence. The object tracking algorithm needs to maintain the recognition of the target between consecutive video frames. In the disclosed solution, object tracking can associate video-level frame relationships within a video and record trajectories.

[0037] Face recognition: Face recognition is a biometric technology that authenticates and identifies a person based on their facial features, meaning that given a facial image, the true identity of the corresponding athlete can be identified.

[0038] Exemplary Methods

[0039] Next, combine Figure 1 The method 100 for evaluating running exercise according to some embodiments of the present disclosure is exemplarily introduced. It is understandable that the method 100 can be executed by any appropriate device with data processing capability, such as but not limited to a terminal device and a server.

[0040] like Figure 1 As shown, at step S101, method 100 may acquire a current frame image of a motion phase of a running motion.

[0041] In the scheme of the present disclosure, the running motion can be further divided into a preparation phase and a movement phase. In the preparation phase, at least one athlete should be located in the preset starting line, and the body is in a ready position. After the start instruction (or also can be referred to as a start command or a start voice, etc.) sounds, at least one athlete can start the running motion, thereby entering the movement phase, until finally arriving at the preset finish line. Here, the start instruction can be outputted audibly by the execution device of method 100. Attached or optionally, the execution device can also output the preparation instruction audibly to remind the athlete to start the pre-running action preparation.

[0042] Based on this, the current frame image obtained by method 100 may include at least one athlete, and the execution time of step S101 in the present disclosure is currently used as the reference time point. It can be understood that the athlete disclosed in the present disclosure can be used to identify any person participating in running sports, not referring to people who specialize in running sports training or running competitions.

[0043] In the scheme disclosed herein, at least one camera device may be pre-installed at a running venue (e.g., a playground, a competition field, or a sports field, etc.) to capture a video to be evaluated of at least one athlete performing running. Furthermore, high poles may be pre-installed at the running venue, and at least one camera device may be installed at a specified height to capture high-quality video to be evaluated at the running site. The execution device of method 100 may be connected in communication with the at least one camera device to obtain the video to be evaluated of the running movement from the at least one camera device, and the running movement may be evaluated by detecting and analyzing the video to be evaluated.

[0044] Given that running can be further divided into two types according to distance: short-distance running (e.g., 50-meter, 100-meter, or 200-meter running) and long-distance running (e.g., 800-meter or 1000-meter running), the present disclosure can determine the number of cameras to be pre-installed at the running venue based on the specific type of sport.

[0045] In one implementation scenario, when the sport type is sprinting, two camera devices (for distinction, they may be referred to as a first camera device and a second camera device) may be pre-installed at the running field. The first camera device may be installed near a preset starting line to capture a video to be evaluated of at least one athlete in the preparation stage of the running sport. The second camera device may be installed near a preset finish line to capture a video to be evaluated of at least one athlete in the running stage of the running sport.

[0046] In another implementation scenario, when the type of motion is long-distance running, at least three camera devices (in order to distinguish, they can be called the first camera device, the second camera device and at least one other camera device) can be pre-set at the running field. The first camera device can be set up near the preset starting line to collect the video to be evaluated of at least one athlete in the preparation stage of the running motion. The second camera device cooperates with at least one other camera device to collect the video to be evaluated of at least one athlete in the motion stage of the running motion. In this implementation scenario, the second camera device can be set up near the preset finish line, and at least one other camera device can be set up between the preset starting line and the preset finish line according to the preset interval. It is understandable that those skilled in the art can select the specific value of the preset interval according to actual needs, and this disclosure does not specifically limit this. Preferably, the preset interval can be 200 meters.

[0047] In the scheme disclosed herein, the start acquisition time and the end acquisition time of at least one camera device may be the same or different, and the start acquisition time and the end acquisition time of the at least one camera device may be controlled by the aforementioned execution device. In actual operation, after receiving the start work instruction input by the user, the execution device may control at least one camera device to start acquisition. Furthermore, after receiving the end work instruction input by the user, the execution device may control at least one camera device to end acquisition.

[0048] In view of the different installation positions of at least one camera device, the time when the athlete enters the camera device's field of view is also different, and the execution device can control them to start collecting at different times. Specifically, for a camera device installed near a preset starting line, the execution device can control it to start collecting after receiving a start work instruction input by the user, or after completing the output of a preparation instruction. For other camera devices, the execution device can control them to start collecting after receiving a start work instruction input by the user, after completing the output of a preparation instruction, or after completing the output of a start instruction. It can be seen that the present disclosure does not make specific limitations in this regard.

[0049] Next, at step S102, method 100 can detect the bodies and faces of multiple athletes in the current frame image respectively to obtain body frames (or body frames or body enclosing frames) and face frames (or face frames or face enclosing frames) of multiple athletes. In one embodiment, the yolov7 target detection model can be used to detect the bodies and faces of multiple athletes respectively to obtain multiple body frames and multiple face frames in the current frame image.

[0050] Next, at step S103, method 100 can construct a binding relationship between a human body frame and a face frame according to a preset binding strategy, and construct a face tracking relationship between a face frame and a historical face frame using a preset tracking technology. Here, the historical face frame is a face frame in each frame image before the current frame image. In the disclosed embodiment, each frame image and the current frame image belong to the video to be evaluated, but the acquisition time of each frame image is earlier than the current frame image. In the disclosed scheme, the athlete's human body frame is used to determine whether the athlete hits the finish line, and the athlete's face frame is used to determine the athlete's identity information. By constructing a binding relationship between a human body frame and a face frame according to a preset binding strategy, the human body frame and face frame belonging to the same athlete can be bound.

[0051] In view of the fact that the human face is usually located in the upper and middle position of the entire human body, the preset binding strategy disclosed herein can be to bind the face frame located in the preset range above the human body frame to it. Specifically, when the binding relationship between the human body frame and the face frame is constructed according to the preset binding strategy, it can be determined first whether the face frame is located in the preset range above the human body frame. When it is determined that the face frame is located in the preset range above the human body frame, the binding relationship between the human body frame and the face frame can be constructed. It is understandable that those skilled in the art can select the value of the preset range according to actual needs, and the present disclosure does not make specific limitations in this regard. In particular, when the athlete is bowing his head or deliberately blocked from the entire process from appearing in the video screen to the end of the run, it will not be possible to find a face frame in the preset range in the middle and upper part of the athlete's human body frame to bind it.

[0052] In the scheme disclosed herein, in order to achieve continuous attention to the athlete's running behavior, after obtaining the athlete's body frame and face frame, method 100 can perform associated tracking on the athlete's body frame and face frame. Specifically, method 100 can use a preset tracking technology to construct a face tracking relationship between a face frame and a historical face frame, and can also use a preset tracking technology to construct a body tracking relationship between a body frame and a historical body frame. Similar to the aforementioned historical face frame, the historical body frame here is the body frame in each frame image before the current frame image. It can be understood that the preset tracking technology of the embodiment of the present disclosure can be an existing tracking technology such as a centroid tracking algorithm or a ByteTrack target tracking algorithm, or it can be any tracking technology that may appear in the future. As long as it can achieve associated tracking of the body frame and the face frame, it can be used in the scheme disclosed herein, and the present disclosure does not make specific limitations on this.

[0053] Next, at step S104, method 100 can determine the athlete (for distinction, can be referred to as the target athlete) who has hit the line (or can also be referred to as touching the line) and whose running distance is the preset distance in the current frame image based on the human body frame, and use the acquisition time of the current frame image as the target athlete's hitting the line time. In the solution disclosed in the present invention, for athletes whose running distance is not the preset distance, the running distance can be used as their current running distance, the acquisition time of the current frame image can be used as their current running time, and the athlete's current running distance and current running time are recorded.

[0054] In the disclosed solution, hitting the finish line means that the torso of the athlete (excluding the head, neck, arms, legs, hands and feet) hits (or can also be called touching) the preset finish line. Specifically, method 100 can determine the position of the torso of the athlete based on the human body diagram surrounded by the human body frame. Next, by comparing the position of the torso with the position of the preset finish line in the current frame image, it can be determined whether the athlete hits the finish line.

[0055] Further, it can be determined whether the running distance of the athlete who hits the finish line is a preset distance. When the running distance of the athlete who hits the finish line is the preset distance, it can be determined that the athlete has reached the finish line and completed the running exercise. Optionally, it can also be determined whether the number of laps run by the athlete who hits the finish line is a preset number of laps. When the number of laps run by the athlete who hits the finish line is the preset number of laps, it can also be determined that the athlete has reached the finish line and completed the running exercise.

[0056] It is understood that the preset distance is a dynamically adjustable distance that varies depending on the running exercise performed by the athlete. In one example, when the running exercise performed by the athlete is a 100-meter run, the preset distance is 100 meters. In another example, when the running exercise performed by the athlete is a 1000-meter run, the preset distance is 1000 meters. Similarly, the preset number of laps is also a dynamically adjustable number that varies depending on the running exercise performed by the athlete. Assume that the circumference of the runway is 200 meters. In one example, when the running exercise performed by the athlete is an 800-meter run, the preset number of laps is 4 laps. In another example, when the running exercise performed by the athlete is a 3000-meter run, the preset number of laps is 15 laps.

[0057] Next, at step S105, the method 100 may determine a plurality of target face images of the target athlete that meet the preset quality conditions based on the binding relationship and the face tracking relationship. Figure 2 The step S105 is described in detail.

[0058] In the scheme disclosed in the present invention, quality parameters may include occlusion, face size, and face orientation. The occlusion can be used to identify whether the target face image is occluded by other face images; the face size can be used to identify the size of the target face image in the current frame image; the face orientation can be used to measure whether the target athlete is in profile or lowering his head. Based on this, the preset quality conditions may be unobstructed, the face size is within the preset size range, and the face orientation is frontal. It is understandable that those skilled in the art may select the preset size range according to actual needs, and the present disclosure does not make specific limitations in this regard.

[0059] In the disclosed solution, a pre-trained detection model can be used to detect the occlusion, face size and face orientation of the target face image, or a logical judgment method can be used to determine the occlusion, face size and face orientation of the target face image. Here, the logical judgment method for determining the occlusion, face size and face orientation of the target face image is introduced as follows:

[0060] The occlusion conditions disclosed herein may include unoccluded and occluded. Specifically, when the size of the portion of the target face image occluded by other face images is less than or equal to a preset occlusion size, the occlusion condition of the target face image can be determined to be unoccluded. Conversely, when the size of the portion of the target face image occluded by other face images is greater than the preset occlusion size, the occlusion condition of the target face frame can be determined to be occluded. It is understandable that those skilled in the art can select a specific value of the preset occlusion size according to actual needs, and the disclosure does not make specific limitations in this regard.

[0061] As those skilled in the art know, face recognition technology relies on the extraction and matching of facial features, and these features are usually most obvious and stable on a face image within a preset size range. If the face image is too large or too small, it may affect the accuracy of feature extraction and reduce the accuracy of face recognition. Therefore, in order to ensure the accuracy of face recognition of the target athlete, the face size of the target face image can be limited to a preset size range.

[0062] In the disclosed solution, the target athlete's front face or profile face can be determined by the vector relationship between the nose and the two eyes. Specifically, when the vectors of the nose and the two eyes are orthogonal, the ratio of the distances from the two eyes to the intersection point can be used to determine whether the target athlete is front face or profile face. Whether the target athlete is lowering his head can be determined by the proportional relationship between the distance from the nose to a specific point and the distance from the specific point to another point.

[0063] Finally, at step S106, method 100 may perform face recognition on multiple target face images of the target athlete to obtain the identity information of the target athlete, and determine the evaluation result of the target athlete based on the finish line time and the identity information.

[0064] In the scheme disclosed herein, the identity information of the target athlete may include, but is not limited to, name and nationality. In order to obtain the identity information of the target athlete, in the scheme disclosed herein, face recognition technology may be used to authenticate and identify the target athlete based on the facial feature information of the target athlete to obtain the identity information of the target athlete. It should be noted that the face recognition technology here may be any existing or future face recognition technology, as long as it can identify the real identity information of the target athlete, it can be used in the scheme disclosed herein, and this disclosure does not make any specific restrictions on this.

[0065] Additionally or optionally, in order to improve the accuracy of face recognition, a larger and more stable face recognition model can be used to authenticate and identify the target athlete to obtain the identity information of the target athlete. Further, multiple face recognition models can be used to authenticate and identify the target athlete, and the identity information of the target athlete can be determined based on multiple face recognition results obtained by the multiple face recognition models.

[0066] In the disclosed solution, the evaluation result of the target athlete is determined based on the time of the line-crossing and the identity information. Specifically, the evaluation score of the target athlete can be determined based on the time of the line-crossing and the time when the start command sounds. Then, the evaluation result of the target athlete is formed based on the evaluation score and the identity information. Further, the method 100 can also output the evaluation result. Additionally or optionally, the method 100 can also output the evaluation result in a visual or audible manner.

[0067] Combination of the above Figure 1 A method 100 for evaluating running is described. The method 100 can detect the body and face of an athlete respectively to obtain the body frame and face frame of the athlete, and can also construct a binding relationship between the body frame and face frame of the athlete. Subsequently, when determining the evaluation results of the athlete, it can be determined whether the athlete has hit the line based on the body frame, and the athlete can be identified based on the face frame bound to the body frame to determine the identity information of the athlete, thereby obtaining the evaluation results of the athlete. Using the scheme disclosed in this disclosure, athletes do not need any wearable devices such as number plates, bracelets, and specific clothing when running, which truly realizes the "come and go" and can reduce the impact of external factors on the athlete's competition status and competition experience. At the same time, when the scheme disclosed in this disclosure identifies the athlete, multiple face images that meet the preset quality conditions are used for face recognition, which can ensure the accuracy of the athlete's identity information on the one hand, and reduce the number of identity recognition failures on the other hand, which helps to improve the evaluation efficiency of running.

[0068] In actual running, if the athlete wears a number plate, the athlete's number plate can also be identified to obtain the athlete's number information. Furthermore, the athlete's number information can be associated with his or her identity information to achieve dual authentication of the athlete's identity. In this way, the accuracy of the athlete's identity recognition can be further improved, and the reliability of running sports evaluation can be improved.

[0069] Figure 2 Shows Figure 1 An exemplary flow chart of step S105 in the method. It is understood that the following Figure 1 The description is a specific implementation of the aforementioned step S105. Figure 1 The features described in can be applied analogously here.

[0070] like Figure 2 As shown, at step S1051, method 100 can determine whether there is a target face frame bound to the target human frame in the current frame image based on the aforementioned binding relationship. When it is determined that there is a target face frame in the current frame image, method 100 can proceed to step S1052, step S1053, step S1054 and step S1055 for continued execution. When it is determined that there is no target face frame in the current frame image, method 100 can proceed to step S1056, step S1057 and step S1058 for continued execution.

[0071] At step S1052, method 100 may obtain multiple target face frames from the target face frames and the historical face frames having a face tracking relationship therewith. Then, at step S1053, the method may select K target face images that meet the aforementioned preset quality conditions from the multiple target face images surrounded by the multiple target face frames, where K is an integer greater than 0. Next, at step S1054, method 100 may sort the K target face images according to the aforementioned quality parameters to obtain a sequence of target face images. Further, at step S1055, method 100 may obtain the first N target face images from the sequence of target face images, where N is an integer less than or equal to K. Preferably, N is 5.

[0072] In the scheme disclosed herein, when the K target face images are sorted according to the aforementioned quality parameters, the target face image that is completely unobstructed and whose face size is the maximum value within the preset size range is arranged at the front of the target face image sequence; the target face image whose obstructed part has a preset obstruction size and whose face size is the minimum value within the preset size range is arranged at the end of the target face image sequence.

[0073] At step S1056, method 100 may obtain M historical human frames from historical human frames that have a human tracking relationship with the target human frame, and determine P historical face frames bound to the M historical human frames, where M is an integer greater than 0, and P is an integer less than or equal to M.

[0074] Then, at step S1057, method 100 may determine the number Q of target face images that meet the preset quality condition among the P target face images surrounded by the P historical face frames. Next, at step S1058, multiple target face images that meet the preset quality condition may be determined based on the number Q. Specifically, as follows:

[0075] When Q is not equal to 0, the Q target face images are first sorted according to the aforementioned quality parameters to obtain a target face image sequence. Then, it is determined whether Q is less than N. When Q is less than N, the Q target face images in the target face image sequence are used as multiple target face images that meet the preset quality conditions. On the contrary, when Q is greater than or equal to N, the first N target face images are obtained from the target face image sequence as the N target face images that meet the preset quality conditions.

[0076] When Q is equal to 0, it means that the target athlete is looking down or deliberately blocked from appearing in the video screen to the end of running. The method 100 does not detect the face frame of the target athlete throughout the whole process. In this case, the upper half of the target human body frame can be intercepted and saved as the face frame of the target athlete, and the identity information of the target athlete is determined to be the preset identity information. Further, the real identity of the target athlete can be determined by manual judgment, and the evaluation result can be assigned to the target athlete. Specifically, the method 100 can output the video to be evaluated in a visual manner. The evaluation personnel can view the running video of the target athlete through the visual interface, and judge its real identity, so that the evaluation result can be assigned to the target athlete.

[0077] Based on this, the solution disclosed in this disclosure can effectively avoid the loss of scores, prevent athletes from running in vain because the evaluation scores are not recorded, and help improve the reliability of the method for evaluating running. It is understandable that those skilled in the art can set the preset identity information according to actual needs, such as "unknown athlete" or "unknown person", etc., and this disclosure does not make specific limitations on this.

[0078] After obtaining N target face images of the target athlete, face recognition can be performed on the N target face images to obtain the identity information of the target athlete. Figure 3 , which shows Figure 1 An exemplary flow chart of step S106 in the method. It can be understood that the following Figure 1 The description is a specific implementation of the aforementioned step S106. Figure 1 The features described in can be applied analogously here.

[0079] At step S1061, method 100 may obtain the first target face image among N target face images as the face image to be identified. Then at step S1062, method 100 may perform face recognition on the face image to be identified to obtain face recognition information and confidence of the face image to be identified. Next, at step S1063, method 100 may determine whether the face image to be identified is the last one among the N target face images. When the face image to be identified is not the last one, method 100 may proceed to step S1064 to continue execution. Conversely, when the face image to be identified is the last one, method 100 may proceed to step S1067 to continue execution.

[0080] At step S1064, the method 100 may further determine whether the confidence is greater than a preset threshold. When the confidence is greater than or equal to the preset threshold, the method 100 may proceed to step S1065 to continue execution. On the contrary, when the confidence is less than the preset threshold, the method 100 may proceed to step S1066 to continue execution. At step S1065, the method 100 may use the face recognition information as the identity information of the target athlete. At step S1066, the method 100 may obtain the next target face image of the face image to be identified as a new face image to be identified, and continue to execute the aforementioned step S1062. In addition, at step S1067, the method 100 may use the face recognition information of the target face image with the greatest confidence as the identity information of the target athlete.

[0081] In actual sports, athletes may violate the rules. In this case, in order to ensure the fairness of the competition, it is necessary to determine whether the athlete has violated the rules, determine the violation result of the athlete, and determine whether the athlete's evaluation results are valid based on the violation result. For details, please refer to Figure 4 , which shows an exemplary flow chart of a method 400 for evaluating running sports according to other embodiments of the present disclosure. It is understood that the method 400 can be executed by any appropriate device with data processing capabilities, such as but not limited to a terminal device and a server.

[0082] like Figure 4 As shown, at step S401, method 400 may determine the violation determination result of the target athlete. Here, the violation determination result may be any one of a legal action, a false start, and a line crossing.

[0083] In the scheme disclosed herein, when determining the violation determination result of the target athlete, the target athlete's video to be evaluated in the preparation stage of the running movement can be obtained first. Then, the feet of the target athlete in each frame of the video to be evaluated can be detected to obtain the foot position and foot speed of the target athlete in each frame. Then, the violation determination result of the target athlete can be determined based on the foot position and foot speed.

[0084] In one embodiment, the violation determination result of the target athlete is determined based on the foot position and foot speed. Specifically, it can include: determining whether the foot position of the target athlete crosses the position of the preset starting line; when the foot position does not cross the position of the preset starting line, the violation determination result can be determined as compliance; when the foot position crosses the position of the preset starting line, it can be further determined whether the foot speed is greater than the preset speed; when the foot speed is greater than or equal to the preset speed, the violation determination result can be determined as a false start; and when the foot speed is less than the preset speed, the violation determination result is determined as crossing the line. It can be understood that those skilled in the art can select the specific value of the preset speed according to actual needs, and this disclosure does not specifically limit this.

[0085] In the scheme disclosed herein, when obtaining the foot position and foot velocity of the target athlete in each frame image, the key point detection model can be used to perform key point detection on the target athlete in each frame image to obtain the key point detection result of the target athlete in each frame image. The key point detection model disclosed herein can adopt the HRNet top-down key point detection model, and the key point detection result can include the foot key points of the target athlete.

[0086] After obtaining the key point results of the target athlete, the foot partial image of the target person in each frame image can be intercepted according to the key point detection results. Here, the foot partial image is the minimum rectangular image containing the foot of the target person. The size of the minimum rectangular image can be determined by the position of the foot key point. Then, the foot partial image can be segmented using a segmentation model to obtain the foot position of the target athlete in each frame image. Finally, the foot rate of the target athlete in each frame image can be determined according to the foot position. Here, the foot rate of the target athlete in each frame image is determined according to the foot position. Specifically, the foot rate can be determined according to the change in the foot position between two consecutive frame images.

[0087] In the solution disclosed herein, the segmentation model may adopt the PPHumanSeg segmentation model, which may perform pixel-level segmentation of the foot along the contour or edge of the foot to obtain the pixel-level foot position. Based on the pixel-level foot position, it is possible to determine whether any edge position of the foot has crossed the line, thereby more accurately capturing the target athlete's crossing the line behavior, thereby ensuring the fairness of the game.

[0088] Next, at step S402, the method 400 may determine whether the evaluation result is valid according to the violation determination result. When the evaluation result is valid, the method 400 may execute step S403. Conversely, when the evaluation result is invalid, the method 400 may execute step S404. Specifically, when the violation determination result is that the action is compliant, the evaluation result may be determined to be valid; when the violation determination result is false start or crossing the line, the evaluation result may be determined to be invalid.

[0089] Finally, at step S403, the method 400 may output the evaluation result to the outside. At step S404, the method 400 may form an invalid result prompt based on the identity information and output the invalid result prompt to the outside.

[0090] Combined with the above description, it can be seen that the violation determination result of the disclosed embodiment is obtained by detecting and analyzing the target athlete's video to be evaluated in the preparation stage of the running movement, and does not rely on the video to be evaluated in the running stage. Based on this, in the scheme disclosed in the present invention, the violation determination can be performed in real time, and when the violation determination result is a false start or crossing the line, a violation prompt is output in real time to remind the target athlete to interrupt the running and start the next run again. Compared with the method of notifying the athlete of the invalid result after the athlete completes the running movement, the real-time determination method can further reduce the occurrence of athletes running in vain.

[0091] Next, combine Figure 5 An electronic device 500 provided in the embodiment of the present disclosure is exemplarily introduced. Figure 5 As shown, the electronic device 500 of the embodiment of the present disclosure may include a processor 501 , a memory 502 , and a communication bus 503 .

[0092] In the process of a specific embodiment, the processor 501 may be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing image processing device (DSPD), a programmable logic image processing device (PLD), a field programmable gate array (FPGA), a CPU, a controller, a microcontroller, and a microprocessor. It is understandable that for different devices, the electronic device used to implement the function of the processor may also be other, which is not specifically limited in this embodiment.

[0093] In the embodiment of the present disclosure, the communication bus 503 is used to realize the connection and communication between the processor 501 and the memory 502; the memory 502 stores program instructions for evaluating the running movement; when the processor 501 executes the program instructions stored in the memory 502, the present disclosure is realized in combination with the attached Figures 1 to 4 A method for evaluating a running sport is described.

[0094] Combination of the above Figure 5 A device for evaluating running motion that can be used to perform the present disclosure is described. It should be understood that the device structure or architecture here is merely exemplary, and the implementation method and implementation entity of the present application are not limited thereto, but can be changed without departing from the spirit of the present application. It is understandable that the description of each embodiment of the present disclosure emphasizes the differences between the various embodiments, and the same or corresponding parts can be referenced to each other. For the purpose of brevity, the present disclosure will not repeat them one by one.

[0095] According to the above description in combination with the accompanying drawings, those skilled in the art can also understand that the embodiments of the present application can also be implemented by a software program. Therefore, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores program instructions for evaluating running movements, and the program instructions can be used to implement the present disclosure in combination with the accompanying drawings. Figures 1 to 4 A method for evaluating a running sport is described.

[0096] It should be noted that although the operations of the method of the present application are described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flow chart can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0097] Although multiple embodiments of the present disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art may think of many changes, modifications, and alternatives without departing from the thought and spirit of the present disclosure. It should be understood that in the process of practicing the present disclosure, various alternatives to the embodiments of the present disclosure described herein may be adopted. The attached claims are intended to define the scope of protection of the present disclosure, and therefore cover equivalents or alternatives within the scope of these claims.

[0098] The collection and acquisition of various data in this application complies with relevant laws and regulations and is authorized by the data provider. Any organization or individual that needs to obtain external data must obtain authorization in accordance with the law and ensure data security. It is not allowed to illegally collect, use, process, or transmit unauthorized or unprotected data, or to illegally buy, sell, provide, or disclose unauthorized or unprotected data.

Claims

1. A method for evaluating running, comprising: Acquire a current frame image of the running motion phase, wherein the current frame image includes a plurality of athletes; Respectively detecting the bodies and faces of the multiple athletes in the current frame image to obtain the body frames and face frames of the multiple athletes; Constructing a binding relationship between the human body frame and the face frame according to a preset binding strategy, and constructing a face tracking relationship between the face frame and a historical face frame using a preset tracking technology, wherein the historical face frame is a face frame in each frame image before the current frame image; Determine the target athlete who hits the finish line in the current frame image based on the human body frame, and use the acquisition time of the current frame image as the target athlete's finish line time; Based on the binding relationship and the face tracking relationship, determining a plurality of target face images of the target athlete that meet preset quality conditions; and Performing face recognition on the multiple target face images to obtain identity information of the target athlete, and determining an evaluation result of the target athlete based on the finish line time and the identity information.

2. The method according to claim 1, wherein: The preset binding strategy is to bind the face frame located in a preset range above the human body frame to it, and the step of constructing the binding relationship between the human body frame and the face frame according to the preset binding strategy includes: Determine whether the face frame is located within a preset range above the body frame; When the face frame is located within a preset range above the human body frame, a binding relationship between the human body frame and the face frame is established.

3. The method according to claim 1, wherein: Based on the binding relationship and the face tracking relationship, determining a plurality of target face images of the target athlete that meet a preset quality condition comprises: Determine whether there is a target face frame bound to a target body frame of the target athlete in the current frame image based on the binding relationship; When the target face frame exists in the current frame image, obtaining multiple target face frames from the target face frame and historical face frames having a face tracking relationship with the target face frame; Selecting K target face images that meet a preset quality condition from the multiple target face images surrounded by the multiple target face frames, where K is an integer greater than 0; Sorting the K target face images according to the quality parameter to obtain a target face image sequence; and The first N target face images are obtained from the target face image sequence, wherein N is an integer less than or equal to K.

4. The method according to claim 3, wherein: The preset tracking technology is also used to establish a human body tracking relationship between the human body frame and a historical human body frame, wherein the historical human body frame is a human body frame in each frame image before the current frame image; the method further includes: When the target face frame does not exist in the current frame image, M historical human frames are obtained from historical human frames that have a human body tracking relationship with the target human body frame, wherein M is an integer greater than 0; Determine P historical face frames bound to the M historical human frames, where P is an integer less than or equal to M; Determine the number Q of target face images that meet a preset quality condition among the P target face images surrounded by the P historical face frames, and determine whether the number Q is equal to 0; When Q is equal to 0, the identity information of the target athlete is determined to be the preset identity information; When Q is not equal to 0, sorting the Q target face images according to the quality parameter to obtain a target face image sequence, and determining whether Q is less than N; When Q is less than N, the Q target face images in the target face image sequence are used as a plurality of target face images that meet a preset quality condition; and When Q is greater than or equal to N, the first N target face images are obtained from the target face image sequence.

5. The method according to claim 3 or 4, wherein: Performing face recognition on N target face images of the target athlete to obtain the identity information of the target athlete includes: Obtaining the first target face image among the N target face images as the face image to be recognized; Performing face recognition on the face image to be recognized to obtain face recognition information and confidence of the face image to be recognized; Determine whether the face image to be recognized is the last one of the N target face images; When the face image to be identified is not the last one of the N target face images, determining whether the confidence level is greater than a preset threshold; When the confidence level is greater than or equal to the preset threshold, using the face recognition information as the identity information of the target athlete; When the confidence level is less than the preset threshold, obtaining a target face image after the face image to be identified as a new face image to be identified, and continuing to perform face recognition on the face image to be identified to obtain face recognition information and confidence level of the face image to be identified; and When the face image to be identified is the last one of the N target face images, the face recognition information of the target face image with the highest confidence is used as the identity information of the target athlete.

6. The method according to claim 1, wherein: After determining the evaluation result of the target athlete based on the finish line time and the identity information, the method further includes: Determining a violation result of the target athlete, wherein the violation result is any one of a legal action, a false start, and crossing the line; Determining whether the evaluation result is valid according to the violation determination result; When the evaluation result is valid, outputting the evaluation result externally; and When the evaluation result is invalid, an invalid result prompt is output externally.

7. The method according to claim 6, wherein: The violation determination results for the target athlete include: Obtaining a video to be evaluated of the target athlete in the preparation stage of a running exercise; Detecting the feet of the target athlete in each frame image of the video to be evaluated to obtain the foot position and foot speed of the target athlete in each frame image; and A violation determination result of the target athlete is determined based on the foot position and the foot velocity.

8. The method according to claim 7, wherein: Determining the violation result of the target athlete according to the foot position and the foot speed includes: Determining whether the foot position of the target athlete has crossed a preset starting line; When the foot position does not cross the preset starting line, determining the violation determination result as compliance with the action; When the foot position passes the preset starting line, determining whether the foot speed is greater than a preset speed; When the foot speed is greater than or equal to the preset speed, determining that the violation result is a false start; and When the foot speed is less than the preset speed, the violation determination result is determined to be crossing the line.

9. An electronic device, comprising: processor; as well as A memory storing program instructions for evaluating running motion, wherein when the program instructions are executed by the processor, the electronic device executes the method according to any one of claims 1-8.

10. A computer-readable storage medium storing program instructions for evaluating running motion, wherein when the program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images