A method, system, device and computer storage medium for detecting a cue sport event

Abstract

The application discloses a billiards event detection method, system, device and computer storage medium, relates to the technical field of computer vision and video analysis, and obtains a target video stream of a billiards table; determines a pocket area in the target video stream; for a frame image in the target video stream, determines pixel features of the pocket area in the frame image; according to the pixel features, determines local texture complexity of pixels in the pocket area; according to the local texture complexity, determines a ViBe detection threshold of the pixels in the pocket area; based on the ViBe detection threshold, identifies a pixel category of the pocket area through a ViBe algorithm, and the pixel category is used for representing whether the pixel belongs to a background or a billiard ball; and performs billiards event detection according to the pixel category. The application can keep stable pixel segmentation in scenes such as strong inverse light, weak light and color temperature mutation, reduces false detection caused by background texture fluctuation, and thus can perform billiards event detection according to accurate pixel categories, and improves detection accuracy.

Description

Technical Field

[0001] This application relates to the fields of computer vision and video analysis technology, and more specifically, to a billiards event detection method, system, device, and computer storage medium. Background Technology

[0002] Currently, event annotation in billiards often relies on manual viewing of video recordings, which is inefficient and highly subjective. To reduce manual labor, methods such as frame differencing, optical flow, and static background modeling can be used to process billiards videos for event detection. However, these methods are sensitive to the complex lighting changes in billiard halls, making accurate event detection difficult.

[0003] In conclusion, how to accurately detect billiards incidents is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of this application is to provide a billiards event detection method, which can solve, to some extent, the technical problem of how to accurately detect billiards events. This application also provides a billiards event detection system, electronic device, and computer-readable storage medium.

[0005] To achieve the above objectives, this application provides the following technical solution:

[0006] A billiards event detection method includes:

[0007] Obtain the target video stream of the pool table;

[0008] Determine the pocket opening region in the target video stream;

[0009] For a frame image in the target video stream, determine the pixel features of the bag opening region in the frame image;

[0010] Based on the pixel features, determine the local texture complexity of the pixels in the bag opening region;

[0011] Based on the local texture complexity, determine the visual background extractor detection threshold for pixels in the bag opening region;

[0012] Based on the detection threshold of the visual background extractor, the pixel category of the pocket opening area is identified by the visual background extractor algorithm. The pixel category is used to characterize whether the pixel belongs to the background or a billiard ball.

[0013] Billiards event detection is performed based on the pixel category.

[0014] Preferably, after identifying the pixel category of the bag opening region using the visual background extractor algorithm, the method further includes:

[0015] In response to updating the sample pool of the visual background extractor algorithm, the background pixels in the bag opening region are determined according to the pixel category;

[0016] In the sample pool of the visual background extractor algorithm, the first sample is randomly selected according to the first probability;

[0017] Update the first sample based on the pixel features of the background pixels;

[0018] In the sample pool of the visual background extractor algorithm, determine the neighboring samples of the first sample;

[0019] According to the second probability, the pixel features of the background pixel are updated to the neighborhood sample;

[0020] Specifically, a predetermined number of frames of images preceding the target video stream are randomly sampled in the neighborhood to obtain the initial samples in the sample pool of the visual background extractor algorithm.

[0021] Preferably, the step of detecting billiard events based on the pixel category includes:

[0022] Based on the pixel category, the pocket area containing the billiard ball is selected as the area to be processed;

[0023] Perform a morphological closing operation on the region to be processed to obtain the closed region.

[0024] Perform connectivity analysis on the closed operation region to obtain the initial connected region;

[0025] The initial connected regions are filtered by area, verified by location, and eliminated by boundary to obtain candidate connected regions.

[0026] Based on the aspect ratio, circularity, convex hull area ratio, and set color of the billiard ball's circumscribed rectangle, candidate connected regions are filtered to obtain the target connected region containing the billiard ball.

[0027] Based on the target connected region, generate a valid sphere object for the current frame image;

[0028] Billiards event detection is performed based on the valid sphere objects in the current frame.

[0029] Preferably, the step of detecting billiard events based on valid ball objects in the current frame includes:

[0030] Based on the valid ball objects in the current frame, determine the state information of the billiard ball in the pocket area of ​​the frame image. The state information includes no ball state, present state, stable state, or disappeared state.

[0031] If the state information of the billiard ball is in the disappearing state in the first consecutive number of frames and the direction of movement is into the pocket, then the billiard ball into the pocket event is triggered and the billiard ball into the pocket information is recorded.

[0032] In the target video stream, the last shot before the billiard ball is pocketed is traced, and the time interval between the last shot and the pocketing is calculated.

[0033] If the time interval is within a set time window, a billiards pocketing event is generated based on the billiards pocketing information;

[0034] The initial state of the billiard ball in the pocket area is "no ball". When the billiard ball appears in the pocket area, the state changes from "no ball" to "present". If the billiard ball is in the "present" state in the second consecutive number of frames, the state changes from "present" to "stable"; otherwise, the state changes back to "no ball". When the billiard ball disappears from the pocket area, the state changes to "disappeared".

[0035] Preferably, the step of generating a billiards pocketing event based on the billiards pocketing information includes:

[0036] Filter out accidental cue touches on the billiard ball pocketing information to obtain the initial pocketing information;

[0037] The initial bagging information is filtered by occlusion to obtain candidate bagging information;

[0038] The pixel brightness channel corresponding to the candidate bag entry information is enhanced to obtain the processed bag entry information;

[0039] The bagging information is filtered for retained balls to obtain the target bagging information;

[0040] A billiard ball pocketing event is generated based on the target pocketing information.

[0041] Preferably, after generating the billiard ball pocketing event based on the target pocketing information, the method further includes:

[0042] The event of a billiard ball being pocketed is encapsulated to obtain standardized record data containing metadata;

[0043] Analyze the standardized recorded data to determine the last visible frame before the billiard ball is pocketed;

[0044] Generate the motion vector of the billiard ball in the last visible frame;

[0045] Based on the motion vector, the target moment when the edge of the billiard ball contacts the boundary of the pocket area is generated;

[0046] The sub-frame timestamp is calculated based on the video frame rate to generate the timestamp of the billiard ball being pocketed.

[0047] The target video stream is edited based on the timestamp of the ball being pocketed to generate a video of the ball being pocketed.

[0048] Preferably, the step of editing the target video stream based on the pocket timestamp to generate a billiards pocket video includes:

[0049] Based on the timestamp of the ball entering the pocket, the target video stream is cropped for a set duration to generate the initial video of the billiard ball entering the pocket.

[0050] In response to slow-motion replay of a billiard ball being pocketed, frames are interpolated from the initial video of the billiard ball being pocketed based on the pocketing timestamp to generate candidate videos of the billiard ball being pocketed.

[0051] Dynamically render candidate videos of billiard balls being pocketed to generate a rendered video of billiard balls being pocketed.

[0052] Multiple billiard ball pocketing rendering videos with time intervals less than a set interval are merged to generate a target billiard ball pocketing video.

[0053] The dynamic rendering includes adding a billiard ball pocketing event label, adding a timer, and highlighting the pocket opening with a blinking highlight box.

[0054] A billiards event detection system, comprising:

[0055] The billiards video stream acquisition module is used to acquire the target video stream of the billiards table;

[0056] The pocket opening region determination module is used to determine the pocket opening region in the target video stream;

[0057] A pixel feature determination module is used to determine the pixel features of the bag opening region in the frame image of the target video stream.

[0058] The local texture complexity determination module is used to determine the local texture complexity of pixels in the bag opening region based on the pixel features.

[0059] The detection threshold determination module is used to determine the visual background extractor detection threshold of pixels in the bag opening area based on the local texture complexity.

[0060] The pixel classification module is used to identify the pixel category of the pocket opening area based on the detection threshold of the visual background extractor and through the visual background extractor algorithm. The pixel category is used to characterize whether the pixel belongs to the background or a billiard ball.

[0061] The event detection module is used to detect billiard events based on the pixel category.

[0062] An electronic device, comprising:

[0063] Memory, used to store computer programs;

[0064] A processor, configured to implement the steps of any of the above-described billiard event detection methods when executing the computer program.

[0065] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the billiards event detection methods described above.

[0066] This application provides a billiards event detection method, which involves: acquiring a target video stream of a billiard table; determining the pocket region in the target video stream; for each frame image in the target video stream, determining the pixel features of the pocket region in the frame image; determining the local texture complexity of the pixels in the pocket region based on the pixel features; determining the visual background extractor detection threshold for the pixels in the pocket region based on the local texture complexity; identifying the pixel category of the pocket region using a visual background extractor algorithm based on the visual background extractor detection threshold, where the pixel category is used to characterize whether the pixel belongs to the background or a billiard ball; and performing billiards event detection based on the pixel category. In this application, it is necessary to determine the pixel features of the pocket opening region in the frame image, and then determine the local texture complexity of the pixels in the pocket opening region based on the pixel features. Since the local texture complexity reflects the degree of change in grayscale values ​​within the pixel's neighborhood, and the degree of change in grayscale values ​​reflects the background characteristics of the pocket opening region, determining the visual background extractor detection threshold for pixels in the pocket opening region based on the local texture complexity can make the visual background extractor detection threshold compatible with the background characteristics of the pocket opening region. In this way, when performing pixel classification based on the visual background extractor algorithm, this application can maintain stable pixel segmentation in scenarios such as strong backlight, low light, and sudden color temperature changes, reducing false detections caused by background texture fluctuations. Thus, billiards event detection can be performed based on accurate pixel categories, improving the accuracy of billiards event detection. The billiards event detection system, electronic device, and computer-readable storage medium provided in this application also solve the corresponding technical problems. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0068] Figure 1 A flowchart of a billiards event detection method provided in this application embodiment;

[0069] Figure 2 This is a flowchart of the improved ViBe algorithm in this application;

[0070] Figure 3 A complete flowchart for handling billiard ball pocketing events;

[0071] Figure 4 This is a flowchart for detecting billiard events based on pixel categories;

[0072] Figure 5 A flowchart for sphere detection and multi-feature verification;

[0073] Figure 6 This is a flowchart for detecting billiard events based on valid sphere objects in the current frame;

[0074] Figure 7 This is a flowchart for determining the bag entry event based on a state machine.

[0075] Figure 8 A flowchart for generating a billiards pocketing event based on billiards pocketing information;

[0076] Figure 9 The flowchart for anti-interference optimization processing;

[0077] Figure 10 A flowchart for event output and highlight generation;

[0078] Figure 11 A flowchart for generating the bagging video;

[0079] Figure 12 This is a schematic diagram of the structure of a billiards event detection system provided in an embodiment of this application;

[0080] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0081] Figure 14 This is another structural schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0082] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0083] Please see Figure 1 , Figure 1 A flowchart of a billiards event detection method provided in an embodiment of this application.

[0084] This application provides a billiards event detection method, which may include the following steps:

[0085] Step S101: Obtain the target video stream of the pool table.

[0086] In practical applications, the target video stream of the billiard table can be acquired first, so that billiard event detection can be performed by processing the target video stream. It should be noted that the video stream of the billiard table refers to the video stream obtained by capturing video of the billiard table during billiard movements.

[0087] In an exemplary embodiment, during the process of acquiring the target video stream of the billiard table, the original video stream of the billiard table can first be acquired through a camera. The original video stream supports multiple video input sources, such as RTSP streams, local video files, and real-time camera captures. Then, image denoising and color enhancement processing are performed on the original video stream. For example, Gaussian filtering is applied for image denoising, with a kernel size of 3×3 and a standard deviation σ of 1.0. Color enhancement is then performed using Contrast Limited Adaptive Histogram Equalization (CLAHE). The video resolution is automatically detected and the processing parameters are adapted. The video resolution can be 720p, 1080p, 4K, etc., to obtain the target video stream.

[0088] Step S102: Determine the pocket opening region in the target video stream.

[0089] In practical applications, considering that the main event in billiards is the ball getting into a pocket, which means the ball passes through the pocket area and falls into the pocket, we can first determine the pocket area in the target video stream so that we can analyze the trajectory of the billiard ball in the pocket area to detect the event of the ball getting into a pocket.

[0090] In an exemplary embodiment, during the process of determining the pocket opening region in the target video stream, considering that the pocket opening region is located on a pool table, the pool table in the target video stream can be determined first. For example, multi-scale Hough line detection can be used, with a threshold of 150, a minimum line length of 100 pixels, and a maximum line spacing of 10 pixels. Then, RANSAC (Random Sampler Detection) can be used. Consensus (a random sampling consensus algorithm) fits a quadrilateral and filters the most likely boundaries of the billiard table. A world coordinate system is established with the center of the billiard table as the origin (0,0), the longer side as the X-axis, and the shorter side as the Y-axis. This facilitates determining the pocket coordinates and calculating the perspective transformation matrix based on the billiard table. Then, the pocket area is determined on the billiard table. For example, based on the fitted quadrilateral boundaries of the billiard table and the aspect ratio of a standard billiard table (usually 2:1), the theoretical coordinates of the six pockets (four corner pockets and two middle pockets) in the image are calculated. The theoretical coordinates are then fine-tuned through template matching or edge detection to obtain the precise pocket coordinates. Finally, the pocket area used to capture the billiard balls is determined based on the pocket coordinates.

[0091] Step S103: For a frame image in the target video stream, determine the pixel features of the bag opening region in the frame image.

[0092] In practical applications, the trajectory of a billiard ball can be reflected by its position in each frame of the target video stream. Therefore, each frame of the target video stream can be processed to detect billiard ball events. During this process, if the billiard ball moves to the pocket area, some pixels in the pocket area will belong to the billiard ball, while the rest will belong to the billiard table background. Therefore, it is necessary to determine the pixel features of the pocket area in the frame image so that billiard ball events can be detected based on the pixel features in the future.

[0093] In an exemplary embodiment, pixel features can be flexibly determined according to the application scenario. For example, pixel features can be RGB values ​​or HSV (Hue-Saturation-Value) values. However, considering the influence of lighting on the environment where the billiard table is located, the V channel (Value / brightness) of the HSV color space can be chosen as the pixel feature to enhance robustness to changes in lighting. Therefore, the pixel features in this application can be scalar. , Represents the coordinates of a pixel.

[0094] Step S104: Determine the local texture complexity of pixels in the bag opening region based on pixel features.

[0095] Step S105: Determine the ViBe detection threshold of pixels in the bag opening region based on the local texture complexity.

[0096] Step S106: Based on the ViBe detection threshold, identify the pixel category of the pocket opening area using the ViBe algorithm. The pixel category is used to characterize whether the pixel belongs to the background or the billiard ball.

[0097] In practical applications, the ViBe algorithm (Visual Background Extractor), a motion target detection algorithm based on background modeling, can be used to detect billiard balls in the pocket area. However, the detection threshold used by the ViBe algorithm is a static threshold, which cannot handle scenarios such as strong backlighting, low light, and sudden color temperature changes, leading to a decrease in billiard ball detection accuracy. To solve this problem, this application determines the local texture complexity of pixels in the pocket area based on pixel features. , , , express The average brightness value of the pixels in the pocket opening region and its 8 neighboring pixels; based on the local texture complexity, determine the ViBe detection threshold of the pixels in the pocket opening region. The higher the local texture complexity, the higher the ViBe detection threshold, for example... , The basic detection threshold can be an empirical constant, such as... , It is the texture sensitivity coefficient, which can be an empirical constant, such as... To ensure that a larger threshold is used in areas with complex textures (such as near the edge of the billiard table) and reduce false detections caused by texture fluctuations; based on the ViBe detection threshold, the pixel category in the pocket area is identified by the ViBe algorithm. The pixel category is used to characterize whether the pixel belongs to the background or the billiard ball, reducing the false detection rate by about 30% and achieving an F1-score of over 95%.

[0098] In an exemplary embodiment, during the process of identifying the pixel category of the bag opening region using the ViBe algorithm, a "ghosting" phenomenon occurs due to the untimely updating of the background model. To address this issue, after identifying the pixel category of the bag opening region using the ViBe algorithm, this application can further update the sample pool of the ViBe algorithm in response to the pixel category, thereby determining the background pixels in the bag opening region based on the pixel category. In the sample pool of the ViBe algorithm, a first sample is randomly selected according to a first probability, where the first probability can be... Etc.; Based on the pixel features of the background pixels, update the first sample. Assuming the index of the first sample is z, then apply the pixel features of the background pixels. replace In the sample pool of the ViBe algorithm, determine the neighborhood samples of the first sample. According to the second probability, the pixel features of the background pixels are updated to the neighboring samples. The second probability can be... To better adapt to smooth background changes, such as completing model recovery within 0.5 seconds after removing the occlusion of the pocket area, meeting the requirements of continuous detection scenarios; in particular, to avoid ViBe mistakenly constructing the ball as background when it is present in the first frame, thus failing to detect its disappearance in subsequent frames, a set number of frames in the target video stream can be randomly sampled in the neighborhood to obtain the initial samples in the sample pool of the ViBe algorithm. For example, an 8-neighborhood spatiotemporal random sampling initialization mechanism can be used to construct the initial sample pool using the median information of the first 5 frames of the target video stream, so that the stationary billiard ball is identified as the foreground at the system startup stage, improving the recall rate of pocket detection by 23.6%.

[0099] It is understandable that the improved ViBe algorithm of this application can be as follows: Figure 2 As shown, the improved ViBe algorithm can detect the appearance and disappearance of the ball more stably and accurately in the pocket area, providing a reliable basis for subsequent billiards event judgment, including the following process:

[0100] 1) Background model initialization: For each bag opening area (size is) (pixels), initialize a three-dimensional number z represents the sample pool index (usually) (N is the sample pool size, commonly 20), each sample Store the feature vector of a pixel;

[0101] 2) Neighborhood random sampling initialization: To avoid contamination of the initial background model by foreground objects (such as the ball in the initial frame), random sampling is performed in the first M frames of the target video stream before the ball moves to the pocket area (such as...). In a frame image, for each pixel Randomly select N sample values ​​from its current frame and its spatiotemporal neighborhood (8 neighboring pixels in the current frame and previous frames) and fill them in. Specifically, for the t-th frame, the number of pixels... Initialization sample From the set Randomly selected from, among which And to ensure that the coordinates do not exceed the boundaries, To effectively utilize spatiotemporal redundancy information, smooth the instantaneous foreground in the initial frame, and construct a more reliable initial background model;

[0102] 3) Adaptive foreground detection: For each pixel of the current frame image t Calculate the local texture complexity : Dynamically adjust the foreground detection threshold based on local texture complexity ,

[0103] 4) Foreground detection and determination: Calculate the current pixel value Calculate the distance to N samples in the sample pool, and count those less than the dynamic threshold. Number of samples , , It is an indicator function; it returns 1 if the condition is true and 0 otherwise. ( To minimize the number of matching samples, such as If a pixel is found to be in the background, it is considered to be in the foreground representing a billiard ball; otherwise, it is considered to be in the foreground representing a billiard ball.

[0104] 5) Spatiotemporal random update strategy: In order to enable the background model to adapt to slowly changing backgrounds (such as gradual lighting changes), the background sample pool is updated, for example, for pixels identified as background. Randomly select an index z from the sample pool with probability p, and use the current pixel value replace ; and with a certain probability Update the current pixel value to its neighboring pixels. In the sample pool; for background pixels The update formula can be:

[0105] ;

[0106] in, Indicates generation Random numbers within the interval, i.e. Indicates the generating interval A random integer within the range.

[0107] Step S107: Detect billiard events based on pixel category.

[0108] In practical applications, the pixel category in the pocket opening region of a frame image reflects whether the pixel belongs to the background or a billiard ball. It is easy to understand that if all pixels belong to the background, there is no billiard ball in the pocket opening region. If some pixels belong to the billiard ball, there is a billiard ball in the pocket opening region, and subsequent detection of the ball entering the pocket can be performed. Therefore, billiard ball event detection can be performed based on pixel category.

[0109] In an exemplary embodiment, during the process of detecting billiard ball events based on pixel categories, considering that billiard balls have certain characteristics, detection can also be performed based on these characteristics to ensure that the billiard ball is truly located in the pocket area. Considering that during the process of a billiard ball entering a pocket, the billiard ball exhibits a state machine mechanism in the pocket area, consisting of a No-Ball State, an Appeared State, a Stable State, and a Disappeared State, the pocketing event can also be determined based on the state machine to initially filter accurate billiard ball pocketing events. Considering that interference may exist during the billiard ball pocketing process, anti-interference processing can also be performed to obtain more accurate billiard ball pocketing events. Finally, operations such as outputting the billiard ball pocketing event and replaying highlights can be performed to enhance the user experience. The entire process can be as follows: Figure 3 As shown, it can guarantee an end-to-end accuracy of >98% for bag entry events, a time positioning accuracy of ±10ms, and a processing latency of <80ms at 1080p@30fps, meeting the requirements for live streaming.

[0110] This application provides a billiards event detection method, which involves: acquiring a target video stream of a billiard table; determining the pocket region in the target video stream; for each frame image in the target video stream, determining the pixel features of the pocket region in the frame image; determining the local texture complexity of the pixels in the pocket region based on the pixel features; determining the visual background extractor detection threshold for the pixels in the pocket region based on the local texture complexity; identifying the pixel category of the pocket region using a visual background extractor algorithm based on the visual background extractor detection threshold, where the pixel category is used to characterize whether the pixel belongs to the background or a billiard ball; and performing billiards event detection based on the pixel category. In this application, it is necessary to determine the pixel features of the pocket opening region in the frame image, and then determine the local texture complexity of the pixels in the pocket opening region based on the pixel features. Since the local texture complexity reflects the degree of change of gray value in the neighborhood of the pixel, and the degree of change of gray value reflects the background characteristics of the pocket opening region, if the visual background extractor detection threshold of the pixels in the pocket opening region is determined based on the local texture complexity, the visual background extractor detection threshold can be adapted to the background characteristics of the pocket opening region. In this way, if pixel classification is performed based on the visual background extractor algorithm, this application can maintain stable pixel segmentation in scenarios such as strong backlight, low light, and sudden color temperature changes, reduce false detections caused by background texture fluctuations, and thus perform billiards event detection based on accurate pixel categories, thereby improving the detection accuracy of billiards events.

[0111] Based on the above embodiments, in the process of billiards event detection according to pixel category, real billiard balls can be screened first based on their characteristics to eliminate noise, debris, or foreign object interference, such as... Figure 4 As shown, the following steps may be included:

[0112] Step S201: Based on the pixel category, the pocket area containing the billiard ball is selected as the area to be processed.

[0113] In practical applications, only the pocket area with the pixel category "billiard ball" contains a billiard ball. Therefore, we can first select the pocket area containing the billiard ball as the area to be processed based on the pixel category.

[0114] Step S202: Perform morphological closing operation on the region to be processed to obtain the closed region.

[0115] Step S203: Perform connectivity analysis on the closed operation region to obtain the initial connected region.

[0116] In practical applications, considering that if a real billiard ball appears in the pocket area, there is a connected component between the real billiard ball and the pocket area, morphological closing operations can be performed on the region to be processed, for example, by using a 3×3 elliptical structuring element. The closed computation region is obtained by first expanding and then eroding. , , Indicates expansion. Indicates corrosion. This represents the binary foreground mask generated by ViBe, used to fill small holes inside the foreground region and connect broken edges; it performs connected component analysis on the closed region, such as connected component labeling, to obtain the initial connected region. Specifically, for... Performing 8-neighborhood connectivity analysis yields several initial connected regions. Each region contains a set of pixels. .

[0117] Step S204: Perform area filtering, location verification, and boundary removal on the initial connected regions to obtain candidate connected regions.

[0118] In practical applications, after obtaining the initial connected region, area filtering, location verification, and boundary removal can be performed on the initial connected region to obtain candidate connected regions.

[0119] In an exemplary embodiment, during the area filtering process, the area of ​​each connected component can be calculated. (i.e., number of pixels), the target area is set based on empirical values ​​such as the projected area of ​​the billiard ball in 720p–1080p images. The initial connected regions that satisfy the target region are retained for filtering candidate connected regions.

[0120] In an exemplary embodiment, the extracted ROI mask of the pool table quadrilateral can be used during the location verification process. Verify whether the centroid of the connected component lies within the valid region. ,like If the condition is met, the connected component is removed, and the remaining connected components are retained for filtering candidate connected components.

[0121] In an exemplary embodiment, during the boundary culling process, if the connected components... Any pixel in the middle satisfies ,or ,or ,or If W and H are the image width and height, then it is considered to be in contact with the image boundary, and is determined to be a truncation target, which is then removed. The remaining connected regions are retained for screening candidate connected regions.

[0122] Step S205: Based on the aspect ratio, circularity, convex hull area ratio, and set color of the circumscribed rectangle of the billiard ball, the candidate connected regions are filtered to obtain the target connected region containing the billiard ball.

[0123] In practical applications, the aspect ratio, circularity, convex hull area ratio, and color of the circumscribed rectangle of a billiard ball have distinct characteristics. Therefore, candidate connected regions can be filtered based on the aspect ratio, circularity, convex hull area ratio, and set color of the circumscribed rectangle of the billiard ball to obtain the target connected region containing the billiard ball.

[0124] In an exemplary embodiment, during the filtering process based on the aspect ratio of the circumscribed rectangle, the minimum circumscribed rectangle (MBR) can be calculated to obtain the width. and height The aspect ratio should be close to 1 (for a circle). The candidate connected regions that meet the conditions are retained for use in filtering the target connected regions.

[0125] In the exemplary embodiment, during the screening process based on roundness, roundness is a key indicator for measuring how closely a shape approximates a circle: ,in The perimeter of the connected region can be obtained through contour tracing. Although a perfect circle has a perimeter of 1.0, billiard balls typically have a perimeter ≥0.75 due to perspective distortion; therefore, a threshold is set to [value missing]. The candidate connected regions that meet the conditions are retained for use in filtering the target connected regions.

[0126] In an exemplary embodiment, during the filtering process based on the convex hull area ratio, the convex hull area can be calculated. , will satisfy Candidate connected regions are retained for filtering target connected regions to exclude non-convex foreign objects such as stars and crosses.

[0127] In an exemplary embodiment, during the color-based filtering process, color semantic matching can be performed in the HSV color space, such as matching the corresponding areas of the original image. Converting from RGB to HSV: Construct a billiards color dictionary; for candidate connected regions, extract their H-channel histograms. (e.g., 32 bins), and calculate a reference histogram for each standard color. Similarity between: Where B is the bin number of the histogram; choose to make Minimum color category As candidate colors; introduce regional average brightness Dynamically adjust color matching tolerance : Only when When the color match is successful, it is determined that the color match is successful.

[0128] In specific application scenarios, the billiards color dictionary can be:

[0129] color_dict = {

[0130] 1: {'name': 'Yellow', 'H_range': [50, 70], 'S_min': 100, 'V_min':150},

[0131] 2: {'name': 'Blue', 'H_range': [100, 130], 'S_min': 120, 'V_min':100},

[0132] 3: {'name': 'Red', 'H_range': [0, 10] ∪ [170, 180], 'S_min':100, 'V_min': 120},

[0133] 4: {'name': 'purple', 'H_range': [130, 150], 'S_min': 100, 'V_min':80},

[0134] 5: {'name': 'Orange', 'H_range': [10, 25], 'S_min': 120, 'V_min':180},

[0135] 6: {'name': 'Green', 'H_range': [70, 90], 'S_min': 100, 'V_min':100},

[0136] 7: {'name': 'brown', 'H_range': [25, 40], 'S_min': 80, 'V_min':80},

[0137] 8: {'name': 'Black', 'H_range': [0, 180], 'S_min': 0, 'V_max':50},

[0138] 9: {'name': 'White', 'H_range': [0, 180], 'S_max': 30, 'V_min':200}

[0139] }

[0140] Step S206: Generate valid sphere objects for the current frame image based on the target connected region.

[0141] In practical applications, if a candidate connected region passes both geometric and color verification, the output is the target connected region. Based on this target connected region, a valid sphere object for the current frame image is then generated. Attributes can include position (centroid coordinates), area / radius, color category (e.g., "red"), and confidence level (which can be set to...). ), etc. The complete process is as follows: Figure 5 As shown, this application achieves a sphere recognition accuracy of over 96% through the aforementioned triple filtering and verification mechanism (shape → geometry → color), which is far superior to single color or shape methods. This significantly improves the accuracy and robustness of sphere detection and provides high-quality input for subsequent state machine event determination.

[0142] Step S207: Detect billiard event based on the valid ball objects in the current frame.

[0143] In practical applications, after generating valid sphere objects, billiards events can be detected based on the valid sphere objects in the current frame.

[0144] Based on the above embodiments, during the billiards event detection process based on the valid ball objects in the current frame, in order to accurately identify whether the billiards ball has entered the pocket, a four-state transition model can be used to track each detected billiards ball and determine its state transition based on the changes in consecutive frames. Furthermore, a spatiotemporal correlation verification method can be combined to ensure the logical coherence between the shot and the pocketing event, such as... Figure 6 As shown, the following steps may be included:

[0145] Step S301: Based on the valid ball objects in the current frame, determine the state information of the billiard ball in the pocket area of ​​the frame image. The state information includes no ball state, present state, stable state, or disappeared state.

[0146] In practical applications, this application sets up a four-state transition model for the billiard ball. The initial state of the billiard ball in the pocket area is "no ball," indicating that the ball is not currently in the field of view. When the billiard ball appears in the pocket area, the state changes from "no ball" to "present," confirming the appearance of a new target and avoiding noise interference. If the billiard ball remains in the "present" state for a second consecutive number of frames, the state changes from "present" to "stable." The stable state indicates that the billiard ball is a real and stable object, allowing continuous monitoring of its position and color features. Otherwise, the state changes back to "no ball." When the billiard ball disappears from the pocket area, the state changes to "disappeared." This disappearance state is derived from the stable state; for example, if a billiard ball that was initially in a stable state is not detected for three consecutive frames, it is considered to have entered the disappearance state. In this way, only billiard balls that have undergone the complete cycle of "no ball → present → stable → disappear" are considered to have entered the pocket, effectively filtering out transient noise (such as flying insects or flashes of light), reducing the false alarm rate to below 1.2%. Therefore, it is necessary to determine the state information of the billiard ball in the pocket area of ​​the frame image based on the valid ball objects in the current frame. The state information includes the state of no ball, the state of presence, the stable state, or the state of disappearance, so as to detect the billiard ball entering the pocket event based on the state information.

[0147] Step S302: In response to the fact that the state information of the billiard ball in the first consecutive number of frames is in the disappearing state and the direction of movement is into the pocket, the billiard ball into the pocket event is triggered, and the billiard ball into the pocket information is recorded.

[0148] In practical applications, if the state information of the billiard ball is in a missing state in the first consecutive number of frames and the direction of movement is towards the pocket, a billiard ball pocketing event is triggered, and the billiard ball pocketing information is recorded. The first number can be 3, which can be flexibly determined according to the application scenario to effectively filter out false positives caused by momentary occlusion.

[0149] In specific application scenarios, for each billiard ball, a counter can be maintained to record the number of consecutive undetected frames. When the count reaches 3, a pocketing event is triggered, and the relevant billiard ball pocketing information (such as timestamp, position, etc.) is recorded.

[0150] Step S303: In the target video stream, trace back to the last shot before the billiard ball is pocketed, and calculate the time interval between the last shot and the pocketing.

[0151] Step S304: In response to the time interval being within the set time window, a billiards ball pocketing event is generated based on the billiards ball pocketing information.

[0152] In practical applications, since a true billiards ball pocketing event is caused by a shot, and the time interval between the shot and the pocketing is not long, the last shot before the pocketing event can be traced in the target video stream. The time interval between the last shot and the pocketing can be calculated. For example, a timer can be started simultaneously with the detection of a shot in the target video stream, and all possible target balls can be monitored. For each ball, the time interval from the shot to its final disappearance (i.e., pocketing) can be recorded. If the time interval is within a set time window, the ball is considered pocketed due to this shot, and a pocketing event is generated based on the pocketing information. If the time interval is outside the set time window, the ball is considered not pocketed due to this shot, and the operation can be terminated. The set time window can be a reasonable maximum delay based on ball speed and distance, such as 0.2–3 seconds. This not only detects "ball disappearance" but also confirms "legal pocketing," providing a reliable data source for advanced applications such as automatic scoring and foul penalties. The complete process can be as follows: Figure 7 As shown. It should be noted that, during this process, video frame information can be combined to further analyze the changes in the ball's trajectory before and after the shot, and the pocketing event can be detected based on the trajectory changes to further enhance the accuracy of the pocketing event.

[0153] In the process described above, during the generation of billiard ball pocketing events based on billiard ball pocketing information, various interfering factors that may occur in the actual environment can be filtered to improve the robustness and accuracy of billiard ball event detection, such as... Figure 8 As shown, the following steps may be included:

[0154] Step S401: Filter out cue mis-touches in the billiard ball pocketing information to obtain initial pocketing information.

[0155] In practical applications, if the cue accidentally touches the ball, it can also result in the ball being pocketed. However, this is not considered a true pocketing and can be filtered out. In other words, the information about balls being pocketed needs to be filtered for accidental cue touches. For example, this could be done by identifying linear objects like the cue based on their aspect ratio and setting a threshold. This method distinguishes the cue stick from other round objects. For areas suspected of being cues, they are directly excluded from the ball candidate list, eliminating more than 99% of cue stick false detections and obtaining initial pocketing information free of cue stick mis-hitting.

[0156] Step S402: Filter the initial bag entry information by occlusion to obtain candidate bag entry information.

[0157] In practical applications, considering that human hands can obscure the ball and cause it to disappear, potentially leading to misjudgments of a potted ball, it's necessary to filter the initial potting information to obtain candidate potting information. Taking human hand obscuring as an example, a skin tone range can be defined in the HSV color space: By using color segmentation technology to identify human hands in images, if a hand is found to be obscuring the ball, the detection results of the relevant area can be temporarily ignored or other compensation measures can be taken. Furthermore, once the obstruction (such as a hand) is removed, a local background model reset mechanism can restore normal monitoring within 1-2 frames without requiring a global restart, ensuring the continuity of billiards movement and event detection.

[0158] Step S403: Enhance the pixel brightness channel corresponding to the candidate bag entry information to obtain processed bag entry information.

[0159] In practical applications, to enhance image contrast, reduce the impact of lighting changes, maintain consistency of sphere features under different lighting conditions, and improve color matching and shape recognition, it is also necessary to enhance the pixel brightness channels corresponding to candidate pocket information. For example, the brightness channel V can be separated in the HSV space, and the contrast-limited adaptive histogram equalization (CLAHE) algorithm can be applied to adjust the brightness. This includes relaxing the color threshold in low light and increasing CLAHEclip_limit at high resolutions to obtain processed pocket information, thereby reducing light and shadow artifacts through shadow suppression. This allows for plug-and-play operation in different brands of cameras and different billiard hall environments, eliminating the need for manual parameter tuning and improving deployment efficiency by more than 5 times.

[0160] Step S404: Filter the retained balls from the processed bag information to obtain the target bag information.

[0161] In practical applications, to avoid misclassifying invalid pocketing events as successful pocketing events, ball retention filtering can be applied to the pocketing information. For example, a specific detection area can be set around the pocket opening, continuously monitoring whether any billiard balls remain within this area for more than 30 frames. If a billiard ball is found to remain near the pocket opening for an extended period without moving or disappearing completely, it is marked as an invalid pocketing event and deleted, and the remaining pocketing information is used as the target pocketing information. The complete process is as follows: Figure 9 As shown, during this process, the next behavior of the stuck ball can be predicted by analyzing its speed and direction, further reducing the error rate.

[0162] Step S405: Generate a billiard ball pocketing event based on the target pocketing information.

[0163] In practical applications, this solution can obtain accurate target pocketing information, and if a billiards pocketing event can be generated based on the target pocketing information, it can effectively improve the accuracy and reliability of billiards pocketing event detection, while reducing false alarms and missed alarms caused by environmental factors.

[0164] Based on the above embodiments, to facilitate users in viewing user pocketing events or watching exciting shot pocketing processes, after generating billiards pocketing events based on target pocketing information, event output and exciting segment generation can also be performed, such as... Figure 10 As shown, the following steps may be included:

[0165] Step S501: Encapsulate the billiard ball pocketing event to obtain standardized record data containing metadata.

[0166] In practical applications, to facilitate data processing, the billiard ball pocketing event can be encapsulated to obtain standardized record data containing metadata. The format of the standardized record data can be JSON, etc. JSON can be batch-output as .jsonl (one event per line) or stored in a database to support subsequent retrieval and analysis. The metadata fields can be:

[0167] {

[0168] "event_id": "E20251202_214703_P3_R",

[0169] "event_type": "POCKET",

[0170] "timestamp_ms": 125670,

[0171] "pts": 125670000,

[0172] "frame_index": 3770,

[0173] "pocket_id": 3,

[0174] "pocket_position": {

[0175] "x": 1820,

[0176] "y": 540

[0177] },

[0178] "ball": {

[0179] "color_name": "Red",

[0180] "color_id": 3,

[0181] "confidence": 0.96,

[0182] "radius_px": 32

[0183] },

[0184] "associated_shot": {

[0185] "shot_id": "S20251202_214658",

[0186] "time_diff_ms": 7200

[0187] },

[0188] "video_clip_path": " / clips / E20251202_214703_P3_R.mp4"

[0189] }

[0190] Wherein, `event_id` represents a globally unique event ID, consisting of time + pocket number + color abbreviation; `timestamp_ms` represents the event occurrence time (milliseconds), based on the system clock or video stream start time; `pts` (PresentationTimestamp) represents the original timestamp of the video frame (in microseconds), used for precise synchronization; `pocket_id` represents the pocket number (1–6), defined according to the standard pool table order (e.g., 1 / 2 is the top corner pocket, 3 / 4 is the bottom corner pocket, 5 / 6 is the middle pocket); `ball.confidence` represents the color matching confidence level (…). (From Bhattacharyya distance); associated_shot represents the associated shot event ID and time difference (used for causal verification); video_clip_path represents the storage path of automatically generated highlight clips.

[0191] Step S502: Analyze the standardized recorded data to determine the last visible frame before the billiard ball is pocketed.

[0192] In practical applications, events can be located on a frame-by-frame basis (e.g., a temporal resolution of only ±16.7ms at 30fps) for video extraction, but this cannot meet the requirement of "accuracy down to the instant of impact." To avoid this, this application employs a sub-frame-level temporal refinement strategy for video generation, which involves analyzing standardized recorded data to determine the last visible frame before the ball is pocketed. For example, the state machine can be used to determine that the pocketing occurs on the [frame number missing in original text]. Frames (disappearing frames) and Between frames (last visible frame), the last visible frame is determined.

[0193] Step S503: Generate the motion vector of the billiard ball in the last visible frame.

[0194] In practical applications, after determining the last visible frame, the motion vector of the billiard ball in the last visible frame can be generated through reverse interpolation of the optical flow trajectory. For example, for the billiard ball in... Frame position The motion vector was calculated using the Lucas-Kanade sparse optical flow. (pixels / frame).

[0195] Step S504: Based on the motion vector, generate the target moment when the edge of the billiard ball contacts the boundary of the pocket area.

[0196] In practical applications, the target moment when the edge of the billiard ball contacts the boundary of the pocket can be generated based on the motion vector, allowing for video editing based on this target moment. Specifically, the pocket area can be defined as a circular Region of Interest (ROI). Assuming the billiard ball moves at a constant speed, its trajectory is: Solve for the target time. Make Even if the edge of the billiard ball touches the edge of the pocket.

[0197] Step S505: Calculate the subframe timestamp of the target time based on the video frame rate to generate the timestamp of the billiard ball being pocketed.

[0198] In practical applications, subframe timestamps can be calculated for the target moment based on the video frame rate to generate the timestamp of the billiard ball being pocketed. Specifically, assuming the video frame rate is fps, the pocket timestamp representing the precise timestamp of the event is: ,in For the first The PTS time (ms) of the frame. At 1080p@30fps, this solution can control the event localization error within ±8–10ms, which is significantly better than whole-frame localization.

[0199] Step S506: Edit the target video stream based on the pocket timestamp to generate a video of a billiard ball being pocketed.

[0200] In practical applications, the target video stream can be edited based on the timestamp of the ball being pocketed to generate a video of the billiard ball being pocketed. Specifically, a video clip of a set duration can be extracted from the target video stream based on the timestamp of the ball being pocketed to generate the initial video of the billiard ball being pocketed. The set duration can be... The video, totaling 5 seconds, covers the entire process from shot preparation, cue stroke, ball roll to pocketing; in response to slow-motion replay of the ball pocketing, frames are interpolated from the initial video of the ball pocketing based on the pocketing timestamp, for example, for... Within a given timeframe, optical flow frame interpolation (such as DAIN or FILM algorithms) is used to generate 60fps slow-motion footage, creating candidate videos highlighting the moment the ball is pocketed. If slow-motion playback is not performed on the pocketed ball, the initial pocketed video is used as a candidate video. These candidate videos are then dynamically rendered to generate a rendered pocketed video. Multiple rendered pocketed videos with time intervals less than a set interval are merged to generate the target pocketed video. For example, if multiple pocketed events have a time interval of less than 4 seconds, they are merged into a single long clip to avoid fragmentation. Afterward, lossless editing and encoding can be performed using FFmpeg for easy transmission. The complete process is as follows: Figure 11 As shown, this application can not only output highly semantic and accurate structured event data, but also realize the automated content generation capability of "detection as production", which greatly improves the intelligent processing level of billiards sports videos.

[0201] In an exemplary embodiment, dynamic rendering may include adding a billiard ball pocketing event label, adding a timer, and flashing a highlight box to mark the pocket opening where the billiard ball is pocketed (for 0.5 seconds). For example, an event label (such as "red ball into bottom pocket") may be added to the upper left corner of the video screen, and a game timer (if the input includes OSD) may be added to the lower right corner.

[0202] Please see Figure 12 , Figure 12 This is a schematic diagram of the structure of a billiards event detection system provided in an embodiment of this application.

[0203] This application provides a billiards event detection system, which may include:

[0204] The billiards video stream acquisition module 101 is used to acquire the target video stream of the billiards table.

[0205] The pocket opening region determination module 102 is used to determine the pocket opening region in the target video stream;

[0206] The pixel feature determination module 103 is used to determine the pixel features of the bag opening region in the frame image of the target video stream.

[0207] The local texture complexity determination module 104 is used to determine the local texture complexity of pixels in the bag opening region based on pixel features.

[0208] The detection threshold determination module 105 is used to determine the ViBe detection threshold of pixels in the bag opening region based on the local texture complexity.

[0209] The pixel classification module 106 is used to identify the pixel category of the pocket opening area based on the ViBe detection threshold and the ViBe algorithm. The pixel category is used to characterize whether the pixel belongs to the background or the billiard ball.

[0210] Event detection module 107 is used to detect billiard events based on pixel category.

[0211] The billiards event detection system provided in this application embodiment may further include:

[0212] The first sample update module is used to determine the background pixels in the bag opening region after the pixel classification module identifies the pixel category in the bag opening region through the ViBe algorithm, in response to updating the sample pool of the ViBe algorithm; the first sample is randomly selected from the sample pool of the ViBe algorithm according to the first probability; and the first sample is updated according to the pixel features of the background pixels.

[0213] The neighborhood sample update module is used to determine the neighborhood samples of the first sample in the sample pool of the ViBe algorithm; and update the pixel features of the background pixels to the neighborhood samples according to the second probability.

[0214] In this process, a predetermined number of frames of images preceding the target video stream are randomly sampled in the neighborhood to obtain the initial samples in the sample pool of the ViBe algorithm.

[0215] This application provides a billiards event detection system, wherein the event detection module may include:

[0216] The connected region determination submodule is used to identify the pocket area containing the billiard ball as the region to be processed based on the pixel category; perform morphological closing operation on the region to be processed to obtain the closed operation region; and perform connected component analysis on the closed operation region to obtain the initial connected region.

[0217] The region verification submodule is used to perform area filtering, location verification, and boundary removal on the initial connected regions to obtain candidate connected regions.

[0218] The billiards verification submodule is used to filter candidate connected regions based on the aspect ratio, circularity, convex hull area ratio, and set color of the billiards' circumscribed rectangle, and to obtain the target connected region containing the billiards.

[0219] The sphere object generation submodule is used to generate valid sphere objects for the current frame image based on the target connected region; and to perform billiards event detection based on the valid sphere objects for the current frame.

[0220] This application provides a billiards event detection system, in which a ball object generation submodule may include:

[0221] The state machine submodule is used to determine the state information of the billiard ball in the pocket area of ​​the frame image based on the valid ball objects in the current frame. The state information includes no ball state, present state, stable state, or disappeared state.

[0222] The pocketing filtering submodule is used to trigger a pocketing event and record the pocketing information when the state information of the billiard ball is in the disappearing state and the direction of movement is into the pocket in the first consecutive number of frames of images.

[0223] The spatiotemporal correlation verification submodule is used to trace the last shot in the target video stream before the billiard ball pocketing event, and to count the time interval between the last shot and the pocketing of the billiard ball; if the time interval is within a set time window, a billiard ball pocketing event is generated based on the billiard ball pocketing information.

[0224] The initial state of the billiard ball in the pocket area is "no ball". When the billiard ball appears in the pocket area, the state changes from "no ball" to "present". If the billiard ball is in the "present" state in the second consecutive number of frames, the state changes from "present" to "stable"; otherwise, the state changes back to "no ball". When the billiard ball disappears from the pocket area, the state changes to "disappeared".

[0225] This application provides a billiards event detection system, in which a spatiotemporal correlation verification submodule may include:

[0226] The filtering submodule is used to filter out cue mis-touch information for billiard ball pocketing information to obtain initial pocketing information; to filter out occlusions in the initial pocketing information to obtain candidate pocketing information; to enhance the pixel brightness channel corresponding to the candidate pocketing information to obtain processed pocketing information; to filter out lingering balls in the processed pocketing information to obtain target pocketing information; and to generate a billiard ball pocketing event based on the target pocketing information.

[0227] The billiards event detection system provided in this application embodiment may further include:

[0228] The encapsulation module is used to encapsulate the billiard ball pocketing event after the filtering submodule generates the event based on the target pocketing information, and obtain standardized record data containing metadata.

[0229] The last visible frame determination module is used to analyze standardized recorded data to determine the last visible frame before the billiard ball is pocketed.

[0230] The target time determination module is used to generate the motion vector of the billiard ball in the last visible frame; based on the motion vector, the target time when the edge of the billiard ball contacts the boundary of the pocket is generated.

[0231] The pocket timestamp generation module is used to calculate the sub-frame timestamp of the target moment based on the video frame rate and generate the pocket timestamp of the billiard ball.

[0232] The pocketing video generation module is used to edit the target video stream based on the pocketing timestamp to generate a video of a billiard ball being pocketed.

[0233] This application provides a billiards event detection system. The pocketing video generation module can be used to: extract a target video stream for a set duration based on the pocketing timestamp to generate an initial video of a billiards pocket; in response to slow-motion playback of a billiards pocketing, interpolate frames from the initial video based on the pocketing timestamp to generate candidate videos of a billiards pocket; dynamically render the candidate videos of a billiards pocket to generate a rendered video of a billiards pocket; and merge multiple rendered videos of a billiards pocket with a time interval less than a set interval to generate a target video of a billiards pocket.

[0234] The dynamic rendering includes adding a billiard ball pocketing event label, adding a timer, and highlighting the pocket opening with a blinking highlight box.

[0235] This application also provides an electronic device and a computer-readable storage medium, both of which have the corresponding effects of the billiards event detection method provided in the embodiments of this application. Please refer to... Figure 13 , Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0236] This application provides a fan speed processing device, including a memory 201 and a processor 202. The memory 201 stores a computer program, and the processor 202 executes the computer program to implement the steps of the billiards event detection method described in any of the above embodiments.

[0237] Please see Figure 14Another electronic device provided in this application embodiment may further include: an input port 203 connected to the processor 202 for transmitting commands input from the outside to the processor 202; a display unit 204 connected to the processor 202 for displaying the processing results of the processor 202 to the outside; and a communication module 205 connected to the processor 202 for enabling communication between the electronic device and the outside. The display unit 204 may be a display panel, a laser scanning display, etc.; the communication method adopted by the communication module 205 includes, but is not limited to, Mobile High-Definition Link (MHL), Universal Serial Bus (USB), High-Definition Multimedia Interface (HDMI), wireless connection: Wireless Fidelity (WiFi), Bluetooth communication technology, Bluetooth Low Energy communication technology, and communication technology based on IEEE 802.11s.

[0238] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the billiards event detection method described in any of the above embodiments.

[0239] The computer-readable storage media involved in this application include random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs (compact disc read-only memory), or any other form of storage media known in the art.

[0240] This application provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the billiards event detection method described in any of the above embodiments.

[0241] For descriptions of relevant parts of the billiards event detection system, electronic device, and computer-readable storage medium provided in the embodiments of this application, please refer to the detailed descriptions of the corresponding parts in the billiards event detection method provided in the embodiments of this application, which will not be repeated here. Furthermore, parts of the technical solutions provided in the embodiments of this application that are consistent with the implementation principles of corresponding technical solutions in the prior art have not been described in detail to avoid excessive elaboration.

[0242] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0243] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method of cue sport event detection, characterized by, The method comprises: acquiring a target video stream of a billiard table; determining a pocket area in the target video stream; for a frame image in the target video stream, determining a pixel feature of the pocket area in the frame image; determining a local texture complexity of a pixel in the pocket area according to the pixel feature; determining a visual background extractor detection threshold of the pixel in the pocket area according to the local texture complexity; identifying a pixel category of the pocket area by a visual background extractor algorithm based on the visual background extractor detection threshold, the pixel category being used to represent whether the pixel belongs to a background or a billiard ball; performing billiard event detection according to the pixel category; wherein the billiard event detection according to the pixel category comprises: taking the pocket area containing the billiard ball as a to-be-processed area according to the pixel category; performing a morphological closing operation on the to-be-processed area to obtain a closing operation area; performing a connected domain analysis on the closing operation area to obtain an initial connected domain; performing area filtering, position verification and boundary rejection on the initial connected domain to obtain a candidate connected domain; screening the candidate connected domain according to a length-width ratio of a circumscribed rectangle of the billiard ball, a circular degree, a convex hull area ratio and a set color to obtain a target connected domain containing the billiard ball; generating an effective ball object of the current frame image based on the target connected domain; and performing billiard event detection according to the effective ball object of the current frame.

2. The method of claim 1, wherein, After the pixel category of the pocket area is identified by the visual background extractor algorithm, the method further comprises: in response to updating a sample pool of the visual background extractor algorithm, determining a background pixel in the pocket area according to the pixel category; randomly selecting a first sample from the sample pool of the visual background extractor algorithm according to a first probability; updating the first sample according to the pixel feature of the background pixel; determining a neighborhood sample of the first sample in the sample pool of the visual background extractor algorithm; updating the pixel feature of the background pixel to the neighborhood sample according to a second probability. wherein a domain random sampling is performed on a set number of frame images before the target video stream to obtain initial samples in the sample pool of the visual background extractor algorithm.

3. The method of claim 1, wherein, The billiard event detection according to the effective ball object of the current frame comprises: determining state information of the billiard ball in the pocket area in the frame image according to the effective ball object of the current frame, the state information including a no-ball state, an appearance state, a stable state or a disappearance state; in response to the state information of the billiard ball in a first number of continuous frame images being the disappearance state and the movement direction being into the pocket, triggering a billiard pocketing event and recording billiard pocketing information; tracing back to a last hitting action moment before the billiard pocketing event in the target video stream and counting a time interval from the last hitting action moment to the pocketing; in response to the time interval being within a set time window, generating the billiard pocketing event based on the billiard pocketing information. The initial state information of the billiard ball in the pocket area is a no-ball state; when the billiard ball appears in the pocket area, the state information changes from the no-ball state to the appearance state; when the state information of the billiard ball in the continuous second number of frame images is the appearance state, the state information changes from the appearance state to the stable state, otherwise the state information changes to the no-ball state; when the billiard ball disappears in the pocket area, the state information changes to the disappearance state.

4. The method of claim 3, wherein, The billiard ball pocket information is based on the generation of the billiard ball pocket event, which includes: The ball club is filtered to obtain the initial pocket information. The initial pocket information is filtered to obtain the candidate pocket information. The pixel brightness channel corresponding to the candidate pocket information is enhanced to obtain the processed pocket information. The target pocket information is obtained by filtering the retention ball of the processed pocket information. The billiard ball pocket event is generated based on the target pocket information.

5. The method of claim 4, wherein, After the billiard ball pocket event is generated based on the target pocket information, it further includes: The billiard ball pocket event is encapsulated to obtain standardized record data containing metadata; The last visible frame before the billiard ball is pocketed is determined by analyzing the standardized record data; The motion vector of the billiard ball in the last visible frame is generated; According to the motion vector, the target time when the billiard ball edge contacts the pocket area boundary is generated; The pocket timestamp of the billiard ball is calculated according to the video frame rate. The target video stream is clipped based on the pocket timestamp to generate the billiard ball pocket video.

6. The method of claim 5, wherein, The target video stream is clipped based on the pocket timestamp to generate the billiard ball pocket video, which includes: The target video stream is set for a certain length of video cutting based on the pocket timestamp to generate the billiard ball pocket initial video; In response to slow motion playback of the billiard ball pocket, the billiard ball pocket initial video is inserted based on the pocket timestamp to generate the billiard ball pocket candidate video; The billiard ball pocket candidate video is dynamically rendered to generate the billiard ball pocket rendering video; The billiard ball pocket target video is generated by merging multiple billiard ball pocket rendering videos with a time interval less than a set interval. Dynamic rendering includes adding a billiard ball pocket event label, adding a timer, and marking a high-light box with a flashing highlight on the pocket position of the billiard ball.

7. A cue sport event detection system characterized by, It includes: A billiard table video stream acquisition module is used to acquire a target video stream of a billiard table. A pocket area determination module is used to determine the pocket area in the target video stream. A pixel feature determination module is used to determine the pixel features of the pocket area in the frame image of the target video stream. A local texture complexity determination module is used to determine the local texture complexity of the pixels in the pocket area based on the pixel features. A detection threshold determination module is used to determine the visual background extractor detection threshold of the pixels in the pocket area based on the local texture complexity. A pixel classification module is used to identify the pixel categories of the pocket area based on the visual background extractor detection threshold through a visual background extractor algorithm, and the pixel categories are used to represent whether the pixels belong to the background or the billiard ball. An event detection module is used to detect the billiard ball event based on the pixel categories. The event detection module includes: The communication region determination submodule is configured to: take a pocket region containing a billiard ball as a to-be-processed region according to the pixel category; perform a morphological closing operation on the to-be-processed region to obtain a closing operation region; and perform a connected domain analysis on the closing operation region to obtain an initial connected region; The region verification submodule is configured to: perform area filtering, position verification and boundary elimination on the initial connected region to obtain a candidate connected region; The billiard ball verification submodule is configured to: perform screening on the candidate connected region according to a billiard ball circumscribed rectangle aspect ratio, a circular degree, a convex hull area ratio and a set color to obtain a target connected region containing a billiard ball; The ball object generation submodule is configured to: generate an effective ball object of the current frame image based on the target connected region; and perform billiard ball event detection according to the effective ball object of the current frame.

8. An electronic device, comprising: The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the billiard ball event detection method in any one of claims 1 to 6. The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the billiard ball event detection method in any one of claims 1 to 6. ​ 9. A computer-readable storage medium, characterized in that, ​

Images