A real scene assisted penalty method and system

Abstract

This invention provides a real-scene assisted judgment method and system. The method includes: acquiring video data collected in real time by a camera device of a tennis court, a tennis ball, and a human body located within the court; monitoring tennis ball landing events in the video data and recording their occurrence time; acquiring multiple target image frames within a preset time period before and after the occurrence time in the video data; extracting the outline of the moving tennis ball from the multiple target image frames; drawing the extracted tennis ball outline onto a real-scene map of the court to form a real-scene dynamic video and / or real-scene static image corresponding to the tennis ball landing event, including the tennis ball's trajectory; and generating a judgment result based on the real-scene dynamic video and / or real-scene static image, including a determination of whether the tennis ball went out of bounds and the real-scene dynamic video and / or real-scene static image. This invention effectively improves the accuracy and realism of motion analysis using ordinary cameras.

Description

Technical Field

[0001] This invention relates to the field of sports equipment technology, and in particular to a real-scene assisted judgment method and system. Background Technology

[0002] "Hawk-Eye," also known as the real-time replay system, is a powerful tool to enhance the accuracy of referee calls. While the underlying technology is not complex, it is highly sophisticated. The system consists of eight or ten high-speed cameras, four computers, and a large screen. The system operates in several steps: First, the computer calculates and divides the three-dimensional space of the playing field into measurements measured in millimeters. Then, the high-speed cameras simultaneously capture basic data about the tennis ball's trajectory from different angles. Next, the computer calculates and fits this data to a curve on a three-dimensional coordinate system, generating a 3D model. Finally, real-time imaging technology clearly displays the tennis ball's trajectory and landing point on the large screen. From data acquisition to result presentation, the entire process takes no more than 10 seconds.

[0003] In summary, professional Hawk-Eye systems use a fitted image during playback, requiring multiple high-speed cameras to work together to capture every frame of the tennis ball, which is expensive. If ordinary cameras are used, the frame rate is insufficient to accurately determine the ball's landing point, failing to achieve the accuracy of professional Hawk-Eye systems. Therefore, professional Hawk-Eye systems are difficult to popularize in the field of mass fitness. Summary of the Invention

[0004] In view of the above problems, the present invention is proposed to provide a real-scene assisted judgment method and system that solves or at least partially solves the above technical problems.

[0005] One aspect of the present invention provides a real-scene assisted penalty method, the method comprising:

[0006] Acquire video data in real time from the camera device, capturing images of the tennis court and the tennis balls and people located within the tennis court.

[0007] Monitor tennis ball landing events in the video data and record the time points when these events occur in the video data;

[0008] Acquire multiple target image frames from the video data that are located within a preset time period before and after the occurrence time point;

[0009] The outline of the moving tennis ball is extracted from each of the multiple target image frames;

[0010] The tennis ball outline extracted from the multiple target image frames is drawn on a preset real-world scene map to form a real-world dynamic video and / or real-world static image corresponding to the tennis ball landing event, including the tennis ball's trajectory.

[0011] The system performs auxiliary judgment based on the real-scene dynamic video and / or real-scene static image and generates a judgment result, which includes the determination of whether the tennis ball is out of bounds and the real-scene dynamic video and / or real-scene static image.

[0012] Furthermore, the method also includes: pre-generating a real-world image of the site;

[0013] Pre-generate site renderings, including:

[0014] A background model of a tennis court was constructed using a Gaussian mixture model.

[0015] The preset scene image and background model are matched in pixel ratio. The foreground image in the scene image is detected and removed from the tennis court scene image to obtain a scene image that is consistent with the actual tennis court scene.

[0016] Further, the step of extracting the tennis ball outline from the plurality of target image frames includes:

[0017] Extract the initial mask image corresponding to each target image frame, which only describes the pixels of moving objects in the current image;

[0018] Morphological operations are used to enhance the edges of moving objects in each initial mask image to obtain a standard mask image with smoothed contours of moving objects for each target image frame.

[0019] Tennis outline detection is performed on the standard mask image of each target image frame to obtain a target mask image containing only tennis outline for each target image frame.

[0020] Further, the step of extracting an initial mask image corresponding to each target image frame, which only describes the pixels of moving objects in the current image, includes:

[0021] A global difference image corresponding to each target image frame is obtained by performing a difference operation between each target image frame and the actual scene image of the site.

[0022] The local difference image corresponding to each target image frame is obtained by performing a difference operation on the previous image frame adjacent to the current target image frame;

[0023] The global difference image and local difference image corresponding to each target image frame are integrated to obtain the initial mask image corresponding to the current target image frame, which only describes the pixels of the moving object.

[0024] Furthermore, the edge enhancement processing of moving objects in each initial mask image through morphological operations includes:

[0025] The first dilation operation expands the moving objects in each initial mask image outward to enhance the visibility and edges of the moving objects. The moving objects are then eroded by the opening operation to remove noise from the image. Finally, the moving objects are expanded outward by the second dilation operation to restore their size.

[0026] Furthermore, the tennis ball contour detection of the standard mask image for each target image frame includes:

[0027] For each target image frame, perform boundary tracking on the moving objects in the standard mask image to find the closed region corresponding to the moving object, store the contour and / or the pixels within the contour corresponding to the closed region, and obtain the contour information of all moving objects in the image.

[0028] The contour information of each moving object is identified according to at least one preset tennis ball feature recognition rule, and the tennis ball contour in the standard mask image of each target image frame is detected.

[0029] Furthermore, the tennis ball feature recognition rules include:

[0030] Based on the contour information of the moving objects, detect whether the contour area corresponding to each moving object meets the preset area threshold range; and / or

[0031] Based on the contour information of the moving objects, detect whether the contour shape of each moving object meets the preset shape parameter requirements; and / or

[0032] Vector relationship detection is performed on the contour information of the moving object and the tennis ball contour detected in at least one adjacent consecutive target image frame to determine whether the vector relationship of multiple consecutive contours conforms to the motion trajectory.

[0033] Furthermore, the step of performing boundary tracking on moving objects in the standard mask image to find the closed region corresponding to the moving object includes:

[0034] Starting with each pixel in the standard mask image, chained encoding is used to traverse along the boundary of the moving object to find adjacent contour pixels until a closed region corresponding to the moving object is found.

[0035] Furthermore, the tennis ball outline extracted from the multiple target image frames is drawn onto a preset real-world court image, including:

[0036] Each target image frame's standard mask image is subjected to a bitwise AND operation with the actual court view image to merge the tennis ball outline from each standard mask image into the actual court view image, resulting in a continuous sequence of actual court images containing only the tennis ball. Based on this sequence, a real-world dynamic video, including the tennis ball's trajectory, is generated corresponding to the tennis ball landing event; and / or

[0037] Perform a bitwise AND operation between the standard mask images of each target image frame and the same real-world image of the court to merge the tennis ball outlines in each standard mask image into a single real-world image of the court, thereby obtaining a real-world static image that includes the trajectory of the tennis ball, corresponding to the tennis ball landing event.

[0038] Another aspect of the present invention provides a real-scene assisted judgment system, the system comprising:

[0039] The cloud processing system, client devices, central control unit, display device, and at least one camera device;

[0040] A camera device is used to film the tennis court and the tennis balls and people located in the tennis court to capture real-time footage.

[0041] The central control unit is used to store the video streams captured by the camera device;

[0042] The client device is used to trigger auxiliary penalty events based on user requests and send the auxiliary penalty requests to the cloud processing system.

[0043] The cloud processing system is used to send a penalty processing command to the central control unit when it receives an auxiliary penalty request from a client device.

[0044] The central control unit is also used to execute the real-scene assisted judgment method as described above and send the judgment result to the cloud processing system when it receives the judgment processing command sent by the cloud processing system;

[0045] The cloud processing system is also used to push the judgment results to on-site display devices and / or client devices for display.

[0046] The real-scene assisted judgment method and system provided in this invention captures video footage using a common camera device and extracts the trajectory of a tennis ball. It then accurately plots the positional information of the tennis ball before and after landing onto a real-scene map of the court, ultimately generating a real-scene dynamic video and / or a real-scene static image containing the tennis ball's trajectory. This effectively improves the accuracy and realism of motion analysis, assists in determining the precise location of the tennis ball's landing point, and facilitates judgment. It solves the technical problems of existing Hawk-Eye systems, including those using high-speed cameras, being too expensive to be widely adopted in the field of mass fitness, and the inability to accurately determine the ball's landing point due to insufficient frame rate when using ordinary cameras for assisted judgment, thus failing to achieve the accuracy of professional Hawk-Eye systems.

[0047] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0048] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. In the drawings:

[0049] Figure 1 This is a flowchart of the real-scene assisted judgment method according to an embodiment of the present invention;

[0050] Figure 2 This is a schematic diagram of the actual site view proposed in the embodiments of the present invention;

[0051] Figure 3 This is a schematic diagram of the initial mask image proposed in an embodiment of the present invention;

[0052] Figure 4 This is a schematic diagram of the standard mask image proposed in an embodiment of the present invention;

[0053] Figure 5 This is a schematic diagram of the target mask image proposed in an embodiment of the present invention;

[0054] Figure 6 This is a schematic diagram of a real-scene static image proposed in an embodiment of the present invention. Detailed Implementation

[0055] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0056] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the meaning consistent with their meaning in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined.

[0057] This invention provides a real-scene assisted judgment method, such as... Figure 1 As shown, the real-scene assisted judgment method proposed in this invention includes the following steps:

[0058] S1. Acquire video data in real time of the tennis court and the tennis balls and people located in the tennis court using a camera device.

[0059] This embodiment of the real-scene assisted penalty method is applied to a real-scene assisted penalty system. The system includes a cloud processing system, client devices, and on-site devices. The on-site devices include a central control unit, a display device, at least one camera, and some network auxiliary facilities. The display device can be a large screen. The camera can be a webcam, specifically a regular camera or a high-speed camera; this invention does not limit the choice. If there are at least two cameras, they can be installed on both sides of the field; if there is only one camera, it can be installed on only one side of the field. The real-scene assisted penalty method in this embodiment is executed by the central control unit.

[0060] S2. Monitor the tennis ball landing event in the video data and record the time point when the tennis ball landing event occurs in the video data.

[0061] S3. Obtain multiple target image frames from the video data that are located within a preset time period before and after the time point of occurrence.

[0062] S4. Extract the tennis ball outline of the moving tennis ball from the multiple target image frames respectively.

[0063] S5. The tennis ball outline extracted from the multiple target image frames is drawn on a preset real-world scene map to form a real-world dynamic video and / or real-world static image corresponding to the tennis ball landing event, including the tennis ball's trajectory.

[0064] S6. Assist in judging and generate a judgment result based on the real-scene dynamic video and / or real-scene static image. The judgment result includes a determination of whether the tennis ball is out of bounds and the real-scene dynamic video and / or real-scene static image. In this embodiment, the judgment result includes a direct determination of whether the tennis ball is out of bounds: in or out, and an indirect result: the real-scene dynamic video and / or real-scene static image of the most recent time the ball landed.

[0065] In this implementation, the central control unit can determine the occurrence of a tennis ball landing event through real-time calculations. Specifically, the identification of tennis ball landing events adopts a video analysis method. Based on a preset dataset, an efficient temporal neural network (such as LSTM or GRU) is used to dynamically learn the trajectory of the tennis ball on the court. This allows the system to monitor tennis ball landing events in real-time video data by capturing the time and spatial location information of the landing point, significantly improving the accuracy of tennis ball position determination in complex scenarios. In practical applications, during a match, whenever the central control unit detects a ball landing event, it records the time point and extracts the tennis ball's position information (i.e., the tennis ball outline) from dozens or more frames before and after that time point in the locally stored video stream. This information is then depicted onto a real-world image of the court, forming a complete real-world dynamic video and / or real-world static image to assist the referee in determining the accurate location of the tennis ball's landing and making a ruling.

[0066] The real-scene assisted judgment method provided by this invention captures video footage using a common camera device and extracts the trajectory of a tennis ball. It accurately depicts the positional information of the tennis ball before and after landing on a real-scene map of the court, ultimately generating a real-scene dynamic video and / or real-scene static image containing the trajectory of the tennis ball. This effectively improves the accuracy and realism of motion analysis and can assist in determining the precise location of the tennis ball's landing for judgment. It solves the technical problems of existing Hawk-Eye systems, including those using high-speed cameras, being too expensive to be widely adopted in the field of mass fitness, and the inability to accurately determine the landing point of the ball when using ordinary cameras for assisted judgment due to insufficient frame rate, thus failing to achieve the accuracy of professional Hawk-Eye systems.

[0067] The real-scene assisted judgment method provided in this embodiment of the invention further includes the step of pre-generating a real-scene image of the tennis court. Pre-generating the real-scene image specifically includes: constructing a background model of the tennis court using a Gaussian mixture model; performing pixel matching between a preset scene image and the background model; detecting foreground images in the scene image and removing the foreground images from the tennis court scene image to obtain a real-scene image consistent with the actual tennis court scene, such as... Figure 2 As shown.

[0068] Specifically, this embodiment is based on the Gaussian Mixture Model (GMM), using multiple Gaussian distributions to model the color value of each pixel in the actual tennis court scene. Each pixel value in each frame of the image is compared with the Gaussian components. Through analysis over a period of time (a video frame sequence), moving objects (including people, balls, etc.) are detected and removed. Finally, a clean background image that is completely consistent with the actual court scene is retained. This image is used for subsequent frame difference calculations and for drawing the tennis ball outlines detected later on this actual court scene image.

[0069] In one embodiment of the present invention, step S4, which involves extracting the tennis ball outline from the plurality of target image frames, specifically includes the following steps (not shown in the accompanying drawings):

[0070] S41. Extract the initial mask image corresponding to each target image frame, which only describes the pixels of moving objects in the current image.

[0071] S42. Perform edge enhancement processing on the moving objects in each initial mask image through morphological operations to obtain a standard mask image with smoothed contours of the moving objects corresponding to each target image frame.

[0072] S43. Perform tennis ball contour detection on the standard mask image of each target image frame to obtain a target mask image containing only tennis ball contours for each target image frame.

[0073] Further, step S41 extracts an initial mask image corresponding to each target image frame that only describes the pixels of moving objects in the current image. Specifically, this includes: performing a difference operation on each target image frame and the actual scene image to obtain a global difference image corresponding to each target image frame; performing a difference operation on each target image frame and the previous image frame adjacent to the current target image frame to obtain a local difference image corresponding to each target image frame; and integrating the global difference image and the local difference image corresponding to each target image frame to obtain an initial mask image corresponding to the current target image frame that only describes the pixels of moving objects.

[0074] Specifically, the image is represented in the computer as a multidimensional array, with each element corresponding to the color value (RGB) of a pixel. By calculating the absolute difference between each pixel, the final output image's pixel value is the absolute difference between the pixel values ​​at corresponding positions in the two input images, thus obtaining an initial mask image that only describes the pixels of the moving object.

[0075] The specific formula is: output[i,j]=|src1[i,j]-src2[i,j]|. Where i and j are the pixel values ​​at corresponding positions in the two input images.

[0076] In this embodiment, to achieve the best dynamic recognition effect throughout the entire environmental cycle, the frame difference implementation method has been optimized: it involves two frame difference methods. First, a global difference is obtained by calculating the frame difference between the current target image frame and the background frame (i.e., the actual scene image). Then, a local difference is obtained by calculating the frame difference between the current target image frame and the adjacent previous image frame. These two differences are then integrated to obtain an initial mask image corresponding to the current target image frame that only describes the pixels of the moving object. Figure 3 As shown, this can enhance the robustness of detection and reduce the impact of noise.

[0077] Furthermore, step S42, which involves performing edge enhancement processing on the moving objects in each initial mask image through morphological operations, includes: expanding the moving objects in each initial mask image outward through a first dilation operation to enhance the visibility and edges of the moving objects; eroding the moving objects through an opening operation to remove noise from the image; and then expanding the moving objects outward through a second dilation operation to restore the size of the moving objects.

[0078] Specifically, the initial mask image obtained in step S14 cannot be directly output as the result because there is still a lot of noise between pixels and objects are hollow or disconnected, requiring further processing. This embodiment performs edge enhancement processing on moving objects in each initial mask image through the following operations:

[0079] c1. The dilation operation expands a moving object outward. Specifically, it uses a structuring element (kernel) to cover each pixel region in the image, enhancing the visibility of objects and their edges, especially when there are small gaps or noise in the image that need to be filled. It also helps to connect separated parts of the image, such as helping to connect disconnected objects.

[0080] c2. The opening operation also uses a structuring element (kernel) to first erode the object, removing small noise from the image. Then, it restores the object's size through dilation, effectively cleaning up the object's contour and making it smoother. This yields a standard mask image with smoothed moving object contours for each target image frame, such as... Figure 4 As shown.

[0081] In this process, through continuous parameter testing and performance optimization, the present invention has constructed different structural elements (cores) for morphological operations. Optionally, the structural elements can be 5*5, 3*3 (unit: pixel), etc.

[0082] Further, step S43, which involves detecting tennis ball contours in the standard mask image of each target image frame, includes: performing boundary tracking on the moving objects in the standard mask image of each target image frame to find the closed regions corresponding to the moving objects; storing the contours and / or pixels within the contours corresponding to the closed regions to obtain the contour information of all moving objects in the image; and recognizing the contour information of each moving object according to at least one preset tennis ball feature recognition rule to detect the tennis ball contours in the standard mask image of each target image frame. In this embodiment, performing boundary tracking on the moving objects in the standard mask image to find the closed regions corresponding to the moving objects specifically includes: starting from each pixel in the standard mask image, using chain coding to traverse along the boundary of the moving object to find adjacent contour pixels until the closed region corresponding to the moving object is found. The tennis ball feature recognition rules include: detecting whether the contour area of ​​each moving object meets a preset area threshold range based on the contour information of the moving object; and / or detecting whether the contour shape of each moving object meets preset shape parameter requirements based on the contour information of the moving object; and / or performing vector relationship detection on the contour information of the moving object and the tennis ball contour detected in at least one adjacent consecutive target image frame to determine whether the vector relationship of multiple consecutive contours conforms to the motion trajectory.

[0083] Specifically, the initial mask image obtained in step S14 above consists of the pixels of all moving objects in a certain video frame. After morphological operations, these pixels have been classified into specific moving objects, including athletes, tennis rackets, and tennis balls. In this step, the contours of corresponding objects are found based on the standard mask image. A contour is the boundary of regions with the same color or intensity in the image, and contours that do not meet the criteria for the target object are filtered out. In this embodiment, contours that do not meet the tennis ball feature recognition rules are specifically filtered out to further determine which contour corresponds to the tennis ball.

[0084] d1. Contour Finding: First, the standard mask image is scanned for boundary tracing. Starting from each pixel in the image, chained encoding is used to traverse along the contour boundaries, searching for adjacent contour pixels. After point description and finding closed regions, these contours and the pixels within them are stored. This yields the contour information of all objects in the image.

[0085] d1. Filtering: To find information about tennis balls, we filter these outlines using specific tennis ball feature recognition rules. For example, outline area feature recognition rules (if it's too large or too small, it's not a tennis ball); outline shape feature recognition rules (if the aspect ratio is too large, it's not a tennis ball); and vector relationship feature recognition rules (vectors represent quantities with magnitude and direction). By checking whether the vector relationships of multiple consecutive outlines conform to a motion trajectory, we finally obtain a target mask image containing only tennis ball outlines, such as... Figure 5 As shown.

[0086] In this embodiment of the invention, step S5, in which the tennis ball outline extracted from multiple target image frames is drawn onto a preset real-world court image, specifically includes: performing a bitwise AND operation between the standard mask image of each target image frame and the real-world court image to merge the tennis ball outline in each standard mask image into the real-world court image, obtaining a continuous real-world image sequence containing only tennis balls; generating a real-world dynamic video containing the tennis ball's trajectory corresponding to the tennis ball landing event based on the real-world image sequence; and / or performing a bitwise AND operation between the standard mask image of each target image frame and the same real-world court image to merge the tennis ball outline in each standard mask image into a single real-world court image, obtaining a real-world static image containing the tennis ball's trajectory corresponding to the tennis ball landing event.

[0087] Specifically, in one embodiment, the mask image containing only the tennis ball outline obtained in step S4 can be merged into a pre-obtained image using a bitwise AND operation. Figure 2 The actual scene image shown is a picture of a tennis ball. This process is repeated over a period of time, and the image is plotted against the pre-defined image. Figure 2 The actual court view shown can be used to obtain a continuous video sequence of frames. These frames form a real-world dynamic video corresponding to the tennis ball landing event, including the tennis ball's trajectory. The information in this video helps users accurately observe the tennis ball's landing and determine whether it has gone out of bounds.

[0088] Specifically, in another embodiment, the mask images containing only the tennis ball outlines of each target image frame obtained in step S4 can be bitwise ANDed with the same real-world court image to merge the tennis ball outlines in each standard mask image into a single real-world court image, resulting in a real-world static image including the tennis ball's trajectory corresponding to the tennis ball landing event, such as... Figure 6 As shown in the image, the information within this still image helps users accurately observe the tennis ball as it hits the ground and determine whether it is out of bounds.

[0089] The real-scene assisted judgment method provided in this invention uses real-scene video footage of tennis movement captured by a real surveillance camera. Utilizing advanced computer vision technologies such as background subtraction, morphological processing, and contour detection, it identifies and tracks the tennis ball's trajectory in real time. Unlike traditional simulated scene techniques, this method automatically analyzes data from the real court, extracts the tennis ball's positional information before and after its landing, and accurately plots it onto a real-scene map of the court. This ultimately generates a real-scene dynamic video and a real-scene static image containing the tennis ball's trajectory, effectively improving the accuracy and realism of the motion analysis and assisting referees in their decisions.

[0090] This invention captures video footage from a regular surveillance camera and extracts the trajectory of a tennis ball. It then plots the ball's position before and after landing against a real-world image of the court, assisting referees in determining the precise location of the ball's landing and making informed decisions. While regular surveillance cameras may have insufficient frame rates and miss crucial points of the ball's landing, this invention, by depicting the complete trajectory of the tennis ball, provides referees with sufficient information to determine the ball's position relative to the court boundaries upon landing, thus enabling accurate real-world-assisted decision-making.

[0091] For the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0092] Another embodiment of the present invention provides a real-scene assisted judgment system, the system including a cloud processing system, client devices, and field devices. The field devices include a central control unit, a display device, at least one camera, and some network auxiliary facilities, wherein:

[0093] The camera device, implemented using a webcam, can be either a standard webcam or a high-speed webcam; this invention does not limit the choice. If the webcam includes at least two cameras, it can be installed on both sides of the court; if there is only one webcam, it can be installed on only one side of the court. The camera device is used to film the tennis court and the tennis balls and people located within the court to capture real-time footage.

[0094] A central control unit is used to store the video streams captured by the camera device. Preferably, the central control unit is used to locally store the video streams captured by the camera device, such as a webcam.

[0095] A client device is a terminal device with referee-side tools installed, such as a referee mini-program or application software. This terminal device can be a mobile phone, laptop, tablet, etc. The client device is used to trigger auxiliary ruling events based on user requests and send these requests to the cloud processing system.

[0096] The cloud-based processing system is responsible for scheduling field equipment. Specifically, when it receives an auxiliary judgment request from a client device, it sends a judgment processing command to the central control unit.

[0097] The central control unit is also used to execute the real-scene assisted judgment method as described in any of the above embodiments when it receives a judgment processing command sent by the cloud processing system, for example... Figure 1 The steps S1-S6 are shown. The judgment result is then sent to the cloud processing system.

[0098] The cloud processing system is also used to push the judgment results to on-site display devices and / or client devices for display.

[0099] In one specific embodiment, the scheduling process of the real-scene assisted judgment system is as follows: When a user initiates assisted judgment through a mini-program entry, the request is sent to the cloud processing system. The cloud processing system notifies the central control unit to process the judgment. After the central control unit completes the processing, it sends the judgment result to the cloud processing system, which then pushes the judgment result to the on-site large screen or mobile mini-program for display.

[0100] For the system implementation, since its actual scene-assisted judgment process is basically similar to that of the method implementation, the description is relatively simple. For relevant parts, please refer to the description of the method implementation and it has the corresponding technical effects.

[0101] The real-scene assisted judgment method and system provided in this invention captures video footage using a common camera device and extracts the trajectory of a tennis ball. It then accurately plots the positional information of the tennis ball before and after landing onto a real-scene map of the court, ultimately generating a real-scene dynamic video and / or a real-scene static image containing the tennis ball's trajectory. This effectively improves the accuracy and realism of motion analysis, assists in determining the precise location of the tennis ball's landing point, and facilitates judgment. It solves the technical problems of existing Hawk-Eye systems, including those using high-speed cameras, being too expensive to be widely adopted in the field of mass fitness, and the inability to accurately determine the ball's landing point due to insufficient frame rate when using ordinary cameras for assisted judgment, thus failing to achieve the accuracy of professional Hawk-Eye systems.

[0102] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, any of the claimed embodiments can be used in any combination.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method of real-time assisted adjudication, the method comprising: The method comprises: acquiring video data collected by a camera in real time for a tennis court and a tennis ball and a human body located in the tennis court; monitoring a tennis ball landing event in the video data and recording a time point at which the tennis ball landing event occurs in the video data; acquiring a plurality of target image frames located within a preset time period before and after the time point in the video data; extracting a tennis ball contour of a moving tennis ball from each of the plurality of target image frames; drawing the tennis ball contour extracted from each of the plurality of target image frames on a preset court real scene map to form a real scene dynamic video and / or a real scene static image including a tennis ball movement track corresponding to the tennis ball landing event; performing auxiliary judgment according to the real scene dynamic video and / or the real scene static image and generating a judgment result, the judgment result including a judgment result of whether the tennis ball is out of court and the real scene dynamic video and / or the real scene static image.

2. The method of claim 1, wherein, The method further comprises: pre-generating the court real scene map; The pre-generating of the court real scene map comprises: constructing a background model of a tennis court real scene by using a Gaussian mixture model; performing pixel matching on a preset live scene image and the background model, detecting a foreground image in the live scene image, and removing the foreground image from the tennis court scene image to obtain a court real scene map consistent with the tennis court real scene.

3. The method of claim 1, wherein, The extracting of the tennis ball contour of the moving tennis ball from each of the plurality of target image frames comprises: extracting an initial mask image corresponding to each target image frame and describing only moving object pixels in the current image; performing edge enhancement processing on the moving object in each initial mask image by morphological operation to obtain a standard mask image corresponding to each target image frame and smooth the moving object contour; performing tennis ball contour detection on the standard mask image of each target image frame to obtain a target mask image corresponding to each target image frame and containing only the tennis ball contour.

4. The method of claim 3, wherein, The extracting of the initial mask image corresponding to each target image frame and describing only the moving object pixels in the current image comprises: performing difference operation on each target image frame and the court real scene map to obtain a global difference image corresponding to each target image frame; performing difference operation on each target image frame and a previous image frame adjacent to the current target image frame to obtain a local difference image corresponding to each target image frame; integrating the global difference image and the local difference image corresponding to each target image frame to obtain the initial mask image corresponding to the current target image frame and describing only the moving object pixels.

5. The method of claim 3, wherein, The edge enhancement processing on the moving object in each initial mask image by morphological operation comprises: extending the moving object outward in each initial mask image by first dilation operation to enhance the visibility of the moving object and the edge of the moving object, corroding the moving object by opening operation to remove noise in the image, and extending the moving object outward by second dilation operation to restore the size of the moving object.

6. The method of claim 3, wherein, The tennis ball contour detection on the standard mask image of each target image frame comprises: boundary tracking is performed on the moving object in the standard mask image of each target image frame to find a closed region corresponding to the moving object, and a contour corresponding to the closed region and / or a pixel point in the contour is stored to obtain contour information of all moving objects in the image; The contour information of each moving object is identified according to a preset at least one tennis ball feature identification rule, and a tennis ball contour in the standard mask image of each target image frame is detected.

7. The method of claim 6, wherein, The tennis ball feature identification rule includes: whether the contour area corresponding to the moving object meets a preset area threshold range is detected according to the contour information of the moving object; and / or whether the contour shape corresponding to the moving object meets a preset shape parameter requirement is detected according to the contour information of the moving object; and / or vector relationship detection is performed on the contour information of the moving object and the tennis ball contour detected in at least one adjacent continuous target image frame to determine whether the vector relationship of the continuous contours meets a motion trajectory.

8. The method of claim 6, wherein, The boundary tracking is performed on the moving object in the standard mask image to find a closed region corresponding to the moving object, including: starting from each pixel in the standard mask image, chain coding is used to traverse along the boundary of the moving object to find adjacent contour pixels until the closed region corresponding to the moving object is found.

9. The method of claim 6, wherein, The tennis ball contour extracted from the plurality of target image frames is drawn on a preset court real scene map, including: bitwise AND operation is performed on the standard mask image of each target image frame and the court real scene map respectively to merge the tennis ball contour in each standard mask image into the court real scene map to obtain a continuous real scene image sequence including only the tennis ball, and a real scene dynamic video including a tennis ball motion trajectory corresponding to a tennis ball landing event is generated according to the real scene image sequence; and / or bitwise AND operation is performed on the standard mask image of each target image frame and the same court real scene map to merge the tennis ball contour in each standard mask image into the court real scene map to obtain a real scene static image corresponding to the tennis ball landing event and including the tennis ball motion trajectory.

10. A live action assisted adjudication system, comprising: The system includes: a cloud processing system, a client device, a central control machine, a display device, and at least one camera device; the camera device is used to capture the tennis court and the tennis ball and human body located in the tennis court to collect live pictures in real time; the central control machine is used to store the video stream collected by the camera device; the client device is used to trigger an auxiliary penalty event according to a user request and send an auxiliary penalty request to the cloud processing system; the cloud processing system is used to send a penalty processing command to the central control machine when receiving the auxiliary penalty request sent by the client device; the central control machine is also used to execute the real scene auxiliary penalty method according to any one of claims 1-9 and send the penalty result to the cloud processing system when receiving the penalty processing command sent by the cloud processing system; the cloud processing system is also used to push the penalty result to the display device and / or the client device on the scene for display.

Images