Digital sports artificial intelligence judgment method, system and program product
By image processing and human posture recognition on boxing match videos, and combining tracking algorithms to calculate scores, the misjudgment and unobjective problems caused by referees' naked eye recognition in the existing technology are solved, and real-time and objective competition judgment results are achieved.
Patent Information
- Application Number
- CN202411607219.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-12
- Publication Date
- 2025-03-04
AI Technical Summary
The judgment of the outcome of the existing boxing competition depends on the referee's naked eye to identify it, which is prone to misjudgment and misjudgment, and the judgment result is not objective enough.
By obtaining the video of the boxing match, disassembly into continuous frame images, standardized processing is performed and the trained pose estimation model is imported for human pose skeleton recognition. Combined with the tracking algorithm, draw the athlete's outline and bones, and calculate the score based on the relative positions of the fist and the opponent's bones to obtain the game judgment result.
Real-time and objective game judgment results are achieved, reducing the possibility of misjudgment and misjudgment, and improving the fairness of the game results.
Smart Images
Figure CN119559550A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of digital sports technology, and more specifically, to a digital sports artificial intelligence refereeing method, system and program product. Background Art
[0002] At present, the outcome of a boxing match depends entirely on the subjective judgment of the referee. Relying solely on the referee's visual identification is prone to misjudgment and wrong judgment. In addition, because boxing is a highly professional sport, the professional level of referees varies, and the results of the match are easily controversial, and the judgment results are not objective enough.
[0003] Therefore, the present application provides a digital sports artificial intelligence refereeing method, system and program product to solve the above-mentioned problems. Summary of the invention
[0004] The purpose of this application is to provide a digital sports artificial intelligence referee method, system and program product to solve the problem that the existing boxing match victory and defeat determination relies on the referee's naked eye recognition, which is prone to misjudgment and wrong judgment, and the judgment result is not objective; intelligent judgment of boxing matches is achieved through human posture recognition and tracking.
[0005] The present application first provides a digital sports artificial intelligence referee method, including: obtaining a video of an athlete performing a boxing match, disassembling the video to obtain continuous frame images; standardizing the image to obtain a standard image; importing the standard image into a trained posture estimation model to perform human posture skeleton recognition, mapping the output of the posture estimation model into a picture frame, a recognition rate and a classification, and filtering out low-confidence and non-human picture frames according to the recognition rate and the classification; filtering the remaining picture frames according to a tracking algorithm to obtain picture frames representing athletes and corresponding athlete IDs; drawing the outline and skeleton of each athlete and marking the athlete ID according to the picture frames representing the athletes and the corresponding athlete IDs, and setting the skeleton at the end of the arm as a fist to obtain a drawn image; saving the drawn image generated in real time as an RTSP stream, calculating whether a score is scored according to the relative distance between the athlete's fist and the opponent's athlete's skeleton in the RTSP stream, and obtaining a match judgment result.
[0006] The above technical solution is adopted to decompose the video into continuous frame images, import the images into the trained posture estimation model, identify the human posture skeleton on the image, obtain the rectangular frame containing the human body and the skeleton composed of key points, and use the tracking algorithm to track the frame representing the athlete and the corresponding athlete ID in the image; draw the outline and skeleton of the athlete, set the arm end skeleton as a fist, and obtain the drawn image; generate an rtsp stream based on the drawn image, calculate whether this frame is scored based on the relative position of the fist and the opponent's skeleton, and continuously calculate whether the score is scored based on the real-time rtsp stream to obtain the game judgment result. Through real-time recording, real-time recognition and real-time calculation, real-time and objective game judgment results are obtained.
[0007] In a possible implementation, the method further includes: saving the real-time generated drawing image as an rtsp stream, and uploading the stream to a front end for display.
[0008] In a possible implementation, the image is standardized to obtain a standard image, including: setting the length and width of the image to a standard length and a standard width, and converting the image into a grayscale image to obtain a standard image.
[0009] In a possible implementation, the remaining frames are screened according to a tracking algorithm to obtain frames representing athletes and corresponding athlete IDs; including: sorting the remaining frames of the current frame image and calculating boundary values for each frame; screening frames according to low confidence and high confidence, selecting a frame that is closest to a frame representing an athlete in a previous frame image from the frames as the frame representing the athlete in the current frame image and marking the athlete ID; updating the frame representing the athlete in the current frame image to a global ID recognition group for continuous tracking of the frame representing the athlete in the next frame image.
[0010] In a possible implementation, if the current frame image is the first frame, the athlete ID is initialized.
[0011] In a possible implementation, the outline and skeleton of each athlete are drawn and the athlete ID is marked according to a picture frame representing the athlete and a corresponding athlete ID, and the skeleton at the end of the arm is set as a fist to obtain a drawn image; including: obtaining a picture frame representing the athlete and the corresponding athlete ID according to a tracking algorithm; calculating the length, width and position of the picture frame representing the athlete in the image; drawing the outline and skeleton of the athlete according to the length, width and position of the picture frame representing the athlete in the image, and selecting different colors for representation according to the athlete ID; marking the athlete ID and confidence on the athlete outline to obtain a drawn image.
[0012] In a possible implementation, the pose estimation model is a yolov8n-pose model.
[0013] In one possible implementation, the tracking algorithm is a bytetrack algorithm.
[0014] The present application also provides a digital sports artificial intelligence referee system, which is used to implement a digital sports artificial intelligence referee method as described above, including: a camera capture module, which is used to obtain a video of an athlete performing a boxing match, and disassemble the video to obtain a continuous frame image; an image recognition module, which is used to standardize the image to obtain a standard image, import the standard image into a trained posture estimation model to perform human posture bone recognition, map the output of the posture estimation model into a picture frame, recognition rate and classification, and screen out low-confidence and non-human picture frames according to the recognition rate and classification; an image tracking module, which is used to screen the remaining picture frames according to the tracking algorithm to obtain a picture frame representing the athlete and the corresponding athlete ID; an image drawing module, which is used to draw the outline and skeleton of each athlete and mark the athlete ID according to the picture frame representing the athlete and the corresponding athlete ID, and at the same time set the skeleton at the end of the arm to a fist to obtain a drawn image; a judgment output module, which is used to save the drawn image generated in real time as an rtsp stream, calculate whether a score is scored according to the relative distance between the athlete's fist and the opponent's athlete's skeleton in the rtsp stream, and obtain a match judgment result.
[0015] The present application also provides a program product, including a computer program, which implements the above-mentioned digital sports artificial intelligence refereeing method when executed by a processor.
[0016] Compared with the existing technology, the present application has the following beneficial effects: through real-time recording, real-time identification and real-time calculation, the real-time referee's game judgment results are obtained, which can be further combined with multiple different camera perspectives to comprehensively draw calculation conclusions to achieve fair, open, just and accurate game result judgments; through digital technology, the current sports event refereeing system is changed to make sports competitions more transparent and the game results are completely objective, fair, open and just. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:
[0018] Figure 1 A flowchart of the method for artificial intelligence refereeing in digital sports;
[0019] Figure 2 Flow chart for disassembling a video to obtain continuous frame images;
[0020] Figure 3 Flowchart for image normalization;
[0021] Figure 4A flowchart for tracking algorithm processing;
[0022] Figure 5 Flowchart generated for drawing images;
[0023] Figure 6 Flowchart displayed for rtsp stream upload;
[0024] Figure 7 This is a structural diagram of the digital sports artificial intelligence referee system. DETAILED DESCRIPTION
[0025] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with examples and drawings. The illustrative implementation scheme of the present application and its description are only used to explain the present application and are not intended to limit the present application.
[0026] First, the application scenario of the present application is described. The present application is for the recognition and analysis of motion videos, which has always been a hot topic and difficulty in digital image processing and pattern recognition. Currently, in practical applications, the recognition and analysis methods of motion videos are mostly targeted at specific objects, such as the motion trajectory of a ball in tennis, in order to achieve the purpose of auxiliary judgment. However, methods targeting human motion are relatively lacking. Since human motion in sports is different from the usual behavior of fixed objects, there are many particularities and large variations, so special methods and systems need to be designed for identification.
[0027] In practice, the most famous system for analyzing sports based on human movements is the Kinect device used by Microsoft for the XBOX360 game console. However, this device uses an infrared dot matrix projector and sensor to obtain a three-dimensional depth image of the human body, which has great limitations in use and can only be used in a small indoor space. For most sports venues, such as stadiums and gymnasiums, the space range is large, and it is difficult for infrared dot matrix projection equipment to work effectively. Ordinary optical cameras can easily obtain intuitive sports images under most environmental conditions. Unlike infrared dot matrix projection equipment, a single ordinary optical camera can only obtain a two-dimensional image, and it is necessary to detect the image area where the human body is located, which is more difficult than the Kinect device in terms of algorithm.
[0028] Therefore, the digital sports artificial intelligence referee method, system and program product provided by the present invention are intended to perform real-time analysis of the scores of boxing matches based on two-dimensional images of video data.
[0029] See also Figure 1As shown, the present application provides a digital sports artificial intelligence referee method. The method includes: S1, obtaining a video of an athlete performing a boxing match, disassembling the video to obtain a continuous frame image; S2, standardizing the image to obtain a standard image; S3, importing the standard image into a trained posture estimation model for human posture skeleton recognition, mapping the output of the posture estimation model into a picture frame, recognition rate and classification, and filtering out low confidence and non-human picture frames according to the recognition rate and classification; S4, screening the remaining picture frames according to the tracking algorithm to obtain a picture frame representing the athlete and the corresponding athlete ID; S5, according to the picture frame representing the athlete and the corresponding athlete ID, draw the outline and skeleton of each athlete and mark the athlete ID, and set the skeleton at the end of the arm as a fist to obtain a drawn image; S6, saving the real-time generated drawn image as an rtsp stream, calculating whether to score according to the relative distance between the athlete's fist and the opponent's athlete's skeleton in the rtsp stream, and obtaining the result of the game judgment.
[0030] Specifically, the acquired video can be an MP4 video file or an RTSP video stream file, the video is disassembled into continuous frames, and the video screen is parsed frame by frame into an image; the image is standardized and converted into a format acceptable to the posture estimation model; the image is imported into the trained posture estimation model, the human posture skeleton on the image is identified, and a rectangular picture frame containing a human body and a skeleton composed of continuous key points are obtained, and at the same time, the picture frames with low confidence and the picture frames that are not classified as human are screened out; the human picture frame with relatively high confidence and picture frame size is locked to calculate the tracking target according to the tracking algorithm, and the picture frame representing the tracking target (athlete) in the current frame image is tracked in combination with the recognition of the previous frame image, and the athlete ID corresponding to the picture frame is calculated. According to the output of the human posture estimation model, the outline and skeleton of the athlete are drawn, and the skeleton at the end of the arm is set to a fist to obtain a drawn image. According to the drawn image, an RTSP stream is generated, and whether this frame is scored is calculated according to the relative position of the fist and the skeleton of the opponent, and whether the score is continuously calculated according to the real-time RTSP stream to obtain the result of the game judgment.
[0031] The improvement of this embodiment is that, through real-time recording, real-time recognition and real-time calculation, the real-time referee's game judgment results are obtained, which can be further combined with multiple different camera perspectives to comprehensively draw calculation conclusions to achieve fair, open, just and accurate game result judgments.
[0032] Step S1 is to obtain a video of an athlete performing a boxing match and disassemble the video into continuous frame images. Figure 2 As shown, Figure 2The flowchart of disassembling a video to obtain continuous frame images is shown in FIG. Specifically, after the camera is turned on, the camera captures the game screen of the player (athlete) in real time, and the camera processes the video into an rtsp video stream and transmits it to the background, which decomposes the rtsp video stream into images frame by frame for subsequent processing.
[0033] Step S2 is to perform standardization processing on the image to obtain a standard image. In a possible implementation, performing standardization processing on the image to obtain a standard image includes: setting the length and width of the image to a standard length and standard width, and converting the image into a grayscale image to obtain a standard image. Figure 3 As shown, Figure 3 Flowchart for normalizing an image. Specifically, the image is cropped to 640 pixels by 640 pixels in length and width; the image is reduced from a three-dimensional color image to a one-dimensional grayscale image to facilitate the calculation of the pose estimation model. The effect is that the frame image is preprocessed and reduced to a grayscale image that meets the model input size, so that each frame image can be subsequently recognized and predicted using the human pose estimation model.
[0034] Step S3 is to import the standard image into the trained posture estimation model for human posture skeleton recognition, map the output of the posture estimation model into picture frames, recognition rates and classifications, and screen out low confidence and non-human picture frames according to the recognition rates and classifications. Specifically, the human posture skeleton on the image is recognized by the posture estimation model, and the 3D array output by the model is mapped into picture frames, recognition rates and classifications, and the output 3D array is disassembled into multiple one-dimensional arrays. The various values of the array are converted into the position, length, width, recognition and classification of the picture frame according to the training set configuration, and the picture frames with confidence lower than the threshold and non-human classification are screened out according to the recognition rates and classifications.
[0035] Step S4 is to screen the remaining frames according to the tracking algorithm to obtain frames representing athletes and corresponding athlete IDs. In a possible implementation, the remaining frames are screened according to the tracking algorithm to obtain frames representing athletes and corresponding athlete IDs; including: sorting the remaining frames of the current frame image, calculating the boundary value of each frame; screening the frames according to low confidence and high confidence, selecting the frame closest to the frame representing the athlete in the previous frame image from the frames as the frame representing the athlete in the current frame image and marking the athlete ID; updating the frame representing the athlete in the current frame image to the global ID recognition group, for continuous tracking of the frame representing the athlete in the next frame image.
[0036] See also Figure 4 As shown, Figure 4This is a flowchart of the tracking algorithm processing. Specifically, the frames returned by the posture estimation model (here, the frames remaining after the initial screening of low confidence and non-human classification) are sorted according to the confidence of the frames or other standards; the boundary value of each frame is calculated; the frames with confidence lower than the set threshold are removed, and the frames with higher confidence are selected from the remaining frames. The frames in the current frame are matched with the frames representing the player in the previous frame using a data association algorithm (such as the Hungarian algorithm, Kalman filter, etc.), and the frames that best match the frames representing the players in the previous frame are selected as the frames representing the players in the current frame image, and the player ID in the current frame image is updated according to the player ID in the previous frame, and this information is stored in the global ID recognition group for tracking the players in the next frame image.
[0037] Furthermore, if the current frame image is the first frame, the athlete ID is initialized. If the current frame image is not the first frame, the athlete ID of the current frame image is obtained according to the previous frame image.
[0038] Step S5 is to draw the outline and skeleton of each athlete and mark the athlete ID according to the picture frame representing the athlete and the corresponding athlete ID, and set the skeleton at the end of the arm as a fist to obtain a drawn image. In a possible implementation, according to the picture frame representing the athlete and the corresponding athlete ID, draw the outline and skeleton of each athlete and mark the athlete ID, and set the skeleton at the end of the arm as a fist to obtain a drawn image; including: obtaining the picture frame representing the athlete and the corresponding athlete ID according to the tracking algorithm; calculating the length, width and position of the picture frame representing the athlete in the image; drawing the outline and skeleton of the athlete according to the length, width and position of the picture frame representing the athlete in the image, and selecting different colors for representation according to the athlete ID; marking the athlete ID and confidence on the athlete outline to obtain a drawn image.
[0039] See also Figure 5 As shown, Figure 5 Flowchart for drawing image generation. Specifically, the playerbox calculated by the bytetrack algorithm is labeled, the length, width and position of the playerbox in the image are calculated, the playerbox outline is drawn with different colors according to the player ID corresponding to the playerbox, and the player ID and confidence are labeled on the playerbox outline to obtain the drawn image.
[0040] Step S6 is to save the real-time generated drawing image as an rtsp stream, calculate whether a score is scored based on the relative distance between the athlete's fist and the opponent's athlete's skeleton in the rtsp stream, and obtain the game judgment result.
[0041] Specifically, the score of each frame of the drawn image is calculated. When the fist of one player touches the body of the other player, the score of the player is calculated according to the bone part hit, and the scores of all frames of the drawn image are summarized to obtain the judgment result of the game.
[0042] In a possible implementation, the method further includes: S7, saving the real-time generated drawing image as an rtsp stream, and uploading it to the front end for display. Figure 6 As shown, Figure 6 The flowchart for uploading the display for rtsp stream. Specifically, the drawn image is saved as rtsp stream, the backend port is opened, the video AP is exposed to the frontend, the frontend accesses the API, and the rtsp video stream is continuously played on the web page to clearly show the athlete's game score.
[0043] In a possible implementation, the pose estimation model is a yolov8n-pose model.
[0044] In one possible implementation, the tracking algorithm is a bytetrack algorithm.
[0045] Specifically, this embodiment uses the yolov8 model for human posture recognition and the bytetrack tracking algorithm for human body tracking.
[0046] See also Figure 7 As shown, the present application provides a digital sports artificial intelligence referee system, which is used to implement a digital sports artificial intelligence referee method as described above, and the system includes: a camera capture module, which is used to obtain a video of an athlete performing a boxing match, and disassemble the video to obtain a continuous frame image; an image recognition module, which is used to standardize the image to obtain a standard image, import the standard image into a trained posture estimation model for human posture bone recognition, and map the output of the posture estimation model into a picture frame, a recognition rate and a classification, and screen out low-confidence and non-human picture frames according to the recognition rate and the classification; an image tracking module, which is used to screen the remaining picture frames according to the tracking algorithm to obtain a picture frame representing the athlete and a corresponding athlete ID; an image drawing module, which is used to draw the outline and skeleton of each athlete and mark the athlete ID according to the picture frame representing the athlete and the corresponding athlete ID, and at the same time set the skeleton at the end of the arm to a fist to obtain a drawn image; a judgment output module, which is used to save the drawn image generated in real time as an rtsp stream, and calculate whether a score is scored according to the relative distance between the athlete's fist and the opponent's athlete's skeleton in the rtsp stream to obtain a match judgment result.
[0047] In a possible implementation, the determination output module is further used to save the real-time generated drawing image as an rtsp stream and upload it to the front end for display.
[0048] It should be noted that the digital sports artificial intelligence referee system is used to implement the above-mentioned digital sports artificial intelligence referee method, which corresponds one-to-one to the above-mentioned method and has the same technical effect as the above-mentioned method, so it will not be elaborated on.
[0049] Finally, the present application also provides a program product, including a computer program, which implements the above-mentioned digital sports artificial intelligence refereeing method when executed by a processor.
[0050] Specifically, the program should run in the following environment. For the computer hardware environment, a high-end microcomputer or dedicated server should be selected, including: CPU: 8 cores or above; Memory (RAM): >= 64M; Hard disk: 100g or above; Display: high-resolution color display, graphics card memory 16g or above, high performance is preferred; For the computer software environment, java / python / vue and some assembly languages should be used for compilation and generation of corresponding executable file formats, and run on Windows / MacOS / Linux systems loaded with jdk, python, and node environments.
[0051] The computer hardware may be composed of a computer CPU, a camera, a GPU, etc. The CPU controls the camera to capture video. The CPU breaks the captured video into frames for processing. The GPU predicts the image, outputs the predicted rtsp stream, and outputs the predicted calculated score to obtain the result of the game.
[0052] In addition, this program requires data and database integrity testing, interface testing, integration testing, functional testing, user interface testing, performance evaluation and load testing until a high degree of recognition is achieved.
[0053] If there is no one in the video when the program is running, it will idle and wait for operation.
[0054] It should be noted that the program itself is easy to operate, takes up few resources, and maintenance only requires regular backup of the used folders to ensure that data is not lost unnecessarily.
[0055] For example, some function designs involved in the program are given:
[0056]
[0057]
[0058]
[0059]
[0060]
[0061] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A digital sports artificial intelligence refereeing method, characterized in that: include: Obtain a video of an athlete performing a boxing match, and disassemble the video to obtain continuous frame images; The image is standardized to obtain a standard image; Importing the standard image into a trained posture estimation model to perform human posture skeleton recognition, mapping the output of the posture estimation model into picture frames, recognition rates, and classifications, and filtering out low-confidence and non-human picture frames based on the recognition rates and classifications; The remaining frames are screened according to the tracking algorithm to obtain frames representing athletes and corresponding athlete IDs; According to the picture frame representing the athlete and the corresponding athlete ID, the outline and skeleton of each athlete are drawn and the athlete ID is marked, and the skeleton at the end of the arm is set to a fist to obtain a drawn image; The real-time generated drawing image is saved as an rtsp stream, and the score is calculated based on the relative distance between the athlete's fist and the opponent's athlete's skeleton in the rtsp stream to obtain the game judgment result.
2. A digital sports artificial intelligence refereeing method according to claim 1, characterized in that: Also includes: The real-time generated drawing image is saved as an rtsp stream and uploaded to the front end for display.
3. A digital sports artificial intelligence refereeing method according to claim 1, characterized in that: The image is standardized to obtain a standard image; including: setting the length and width of the image to a standard length and a standard width, and converting the image into a grayscale image to obtain a standard image.
4. A digital sports artificial intelligence refereeing method according to claim 1, characterized in that: The remaining frames are screened according to the tracking algorithm to obtain frames representing athletes and corresponding athlete IDs; including: Sort the remaining frames of the current frame image and calculate the boundary value of each frame; Filter the frames according to the low confidence and high confidence, select the frame that is closest to the frame representing the athlete in the previous frame image, use it as the frame representing the athlete in the current frame image, and mark the athlete ID; The picture frame representing the athlete in the current frame image is updated to the global ID recognition group, so as to continuously track the picture frame representing the athlete in the next frame image.
5. A digital sports artificial intelligence refereeing method according to claim 4, characterized in that: If this frame is the first frame, initialize the athlete ID.
6. A digital sports artificial intelligence refereeing method according to claim 1, characterized in that: According to the picture frame representing the athlete and the corresponding athlete ID, draw the outline and skeleton of each athlete and mark the athlete ID, and set the skeleton at the end of the arm to a fist to obtain a drawn image; including: A picture frame representing an athlete and a corresponding athlete ID obtained according to the tracking algorithm; Calculate the length, width, and position of the frame representing the athlete in the image; Draw the outline and skeleton of the athlete according to the length, width and position of the picture frame representing the athlete in the image, and select different colors according to the athlete ID; The athlete ID and confidence level are marked on the athlete outline to obtain a drawn image.
7. A digital sports artificial intelligence refereeing method according to any one of claims 1-6, characterized in that: The pose estimation model is the yolov8n-pose model.
8. A digital sports artificial intelligence refereeing method according to any one of claims 1-6, characterized in that: The tracking algorithm is the bytetrack algorithm.
9. A digital sports artificial intelligence referee system, characterized in that: A method for implementing a digital sports artificial intelligence refereeing method as claimed in any one of claims 1 to 8, comprising: A camera capture module is used to obtain a video of an athlete performing a boxing match and to disassemble the video to obtain continuous frame images; An image recognition module is used to standardize an image to obtain a standard image, import the standard image into a trained posture estimation model to perform human posture skeleton recognition, map the output of the posture estimation model into picture frames, recognition rates and classifications, and screen out low-confidence and non-human picture frames based on the recognition rates and classifications; An image tracking module, used to screen the remaining frames according to the tracking algorithm to obtain the frames representing the athletes and the corresponding athlete IDs; An image drawing module, used for drawing the outline and skeleton of each athlete and marking the athlete ID according to the picture frame representing the athlete and the corresponding athlete ID, and setting the skeleton at the end of the arm as a fist to obtain a drawn image; The judgment output module is used to save the real-time generated drawing image as an rtsp stream, calculate whether a score is scored based on the relative distance between the athlete's fist and the opponent's athlete's skeleton in the rtsp stream, and obtain the game judgment result.
10. A program product comprising a computer program, characterized in that When the computer program is executed by a processor, a digital sports artificial intelligence refereeing method as described in any one of claims 1 to 8 is implemented.