A billiards out-of-bounds judgment method based on target detection and heterogeneous parallel computing
By combining target detection and heterogeneous parallel computing with RFID sensor data and Socket communication, the accuracy and speed issues of billiards out-of-bounds judgment in complex environments are solved, achieving fast and accurate billiards out-of-bounds judgment and player identification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NORTHEASTERN UNIV AT QINHUANGDAO
- Filing Date
- 2025-05-27
- Publication Date
- 2026-06-19
AI Technical Summary
Existing methods for judging whether a ball is out of bounds are insufficient in terms of accuracy and processing speed in complex environments, especially when there is multi-ball interaction, high ball speed, and large changes in lighting, making it difficult to make quick and accurate judgments.
The method adopts a target detection and heterogeneous parallel computing approach. It uses semi-automatic model training, real-time image acquisition and preprocessing, heterogeneous parallel computing for image processing, billiard ball out-of-bounds judgment and player identification, combined with RFID sensor data for judgment, and realizes data interaction between systems through Socket communication.
It improves the accuracy and processing speed of billiard ball out-of-bounds judgment, enables rapid response and subsequent correction at critical moments, reduces dataset annotation costs, and improves model accuracy and robustness.
Smart Images
Figure CN122244744A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of sports and computer vision technology, and in particular to a method and device for determining whether a billiard ball is out of bounds based on target detection, heterogeneous parallel computing and RFID technology. Background Technology
[0002] Existing methods for determining whether a billiard ball is out of bounds largely rely on traditional computer vision techniques, such as color recognition, morphological processing, and edge detection. However, these methods often perform poorly in complex environments, especially with multi-ball interaction, high ball speeds, and significant changes in lighting conditions, where accuracy drops drastically. Furthermore, current technologies suffer from significant limitations in processing speed; traditional methods are computationally inefficient and cannot quickly process large amounts of image data. During billiards play, especially at high speeds, delays can lead to missed crucial moments, making timely and accurate judgments difficult. Therefore, improving recognition accuracy, increasing processing speed, and compensating for missed keyframes through subsequent correction functions are pressing issues that current technologies need to address. Consequently, current mainstream algorithms still face dual challenges in terms of accuracy and real-time performance in billiards recognition and out-of-bounds determination, requiring further research and improvement. Summary of the Invention
[0003] This invention provides a billiards out-of-bounds judgment method based on target detection and heterogeneous parallel computing. It performs real-time detection of the billiards ball, determines whether the billiards ball is out of bounds or in the hole, records the player who is currently hitting the ball, and sends the detection information to other programs.
[0004] This invention adopts the following technical solution: a method for determining whether a billiard ball is out of bounds based on target detection and heterogeneous parallel computing, comprising the following steps:
[0005] S1: Semi-automatic model training: Construct a semi-automatic model training process of "labeling-data augmentation-training-pseudo-labeling-human verification-data augmentation-training" to train the table corner detection model and the billiard ball / glove detection model;
[0006] S2: Real-time image acquisition and preprocessing: Real-time panoramic video stream of the tabletop is acquired through a wide-angle camera on the top of the billiard table, and the acquired images are preprocessed by distortion correction, size scaling and color space conversion in sequence;
[0007] S3: Image processing based on heterogeneous parallel computing: Under the GPU-CPU heterogeneous computing architecture, a task grouping parallel strategy is adopted. The subsequent processing of image acquisition and model inference results is handled by the CPU, while image preprocessing and model inference are handled by the GPU. The two groups of tasks run in parallel on different computing resources, realizing pipelined parallel processing of image acquisition and result processing and image preprocessing and model inference, effectively improving the overall processing efficiency and response speed of the system.
[0008] S4: Billiards out-of-bounds judgment and player identification: After the billiards ball stabilizes, the current player's identity is identified; the visual detection results and RFID sensor data are combined to determine whether the billiards ball has gone out of bounds.
[0009] S5: System Communication: Through the Socket communication interface, this system enables data interaction and command transmission between itself and external systems.
[0010] Preferably, in step S1, the specific implementation method of semi-automatic model training is as follows:
[0011] S1.1 Select a portion of samples from the collected images, manually annotate them using the image annotation tool LabelImg, and expand the sample size through data augmentation. Specifically, the annotated data for table corners is used to train the table corner detection model, and the annotated data for billiard balls and gloves of different colors is used to train the billiard ball / glove detection model.
[0012] S1.2. The initial model obtained from the above training is used to automatically label the remaining unlabeled images. After the automatic labeling results are manually reviewed and corrected, data augmentation is performed to form a new training set.
[0013] S1.3. Continue training the table corner detection model and the billiard ball / glove detection model using the updated training set to further improve the detection accuracy and robustness of the models;
[0014] S1.4 The above data augmentation methods include rotation, translation, flipping, scaling, perspective transformation, HSV color gamut change, Mosaic, MixUp, Copy-Paste, local random erasure, and brightness change.
[0015] Preferably, the preprocessing operation in step S2 is accelerated by parallel computing based on the CUDA architecture.
[0016] Preferably, the image processing method based on heterogeneous parallel computing in step S3 is specifically implemented as follows:
[0017] S3.1 The CPU is responsible for image acquisition and subsequent processing of model inference results;
[0018] S3.2, GPU is responsible for image preprocessing and model inference tasks;
[0019] The S3.3 GPU and CPU run simultaneously to achieve pipelined parallel processing of image acquisition and result processing with image preprocessing and model inference, effectively improving the overall processing efficiency and response speed of the system.
[0020] Preferably, in step S4, the specific implementation method of determining the stability of the billiard ball is as follows:
[0021] S4.1 Billiard table background modeling: Before the game, the empty billiard table image is preprocessed, and the corner detection model is used to obtain the coordinates of the four corners to form the effective area of the billiard table surface, which is then saved as the background image of the table.
[0022] S4.2 Occlusion Detection: Using OpenCV, the region of interest is set as the effective area of the desktop, and the billiard ball detection box area in the current frame is removed. The image of this area is subjected to difference, binarization, and erosion operations with the background image. If there are non-zero pixels in the final image, it is determined that there is an occlusion; otherwise, it is an unoccluded state.
[0023] S4.3. In an unobstructed state, if the displacement of each billiard ball in multiple consecutive frames is less than a preset threshold, it is determined to be "stable billiard ball"; otherwise, it is "unstable".
[0024] Preferably, in step S4, the method for identifying the player is as follows: after confirming that the billiard ball is in a stable state, if the current player's glove is not detected in the effective area of the billiard table in multiple consecutive frames, the current player's identity is recorded as unknown; if the current player is unknown, but a glove of a certain color is detected in the effective area of the billiard table in multiple consecutive frames, the current player is recorded as the wearer of that color glove.
[0025] Preferably, in step S4, the method for detecting billiard balls going out of bounds is as follows: the billiard balls have built-in RFID tags, and when a billiard ball enters the pocket, it is recorded that the ball has entered the pocket; if a billiard ball is detected outside the valid area of the billiard table, it is recorded that the billiard ball is out of bounds; after the billiard balls stabilize, the detected billiard balls are compared with the billiard balls detected when the billiard balls stabilized last time. If a billiard ball is detected to have disappeared and it is not recorded that the billiard ball has entered the pocket or has gone out of bounds, it is recorded that the billiard ball is out of bounds.
[0026] Preferably, in step S5, the detection results are sent to other programs via Socket communication for further processing or decision-making. Simultaneously, other programs can also send control commands to this system via the Socket interface, thereby enabling operation control of this system.
[0027] Preferably, before the match, players wear gloves of different colors and place them on the billiard table. The system assigns a unique ID to each glove within the valid area of the billiard table as a player's identification.
[0028] Preferably, the semi-automatic model training in step S1 and the image processing method based on heterogeneous parallel computing in step S3 are not only applicable to the judgment of billiard ball going out of bounds, but can also be applied to various scenarios that require real-time image analysis and decision-making.
[0029] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:
[0030] 1. By constructing a semi-automatic model training system consisting of "labeling-data augmentation-training-pseudo-labeling-human verification-data augmentation-training", the cost of dataset labeling is reduced, and the accuracy and robustness of the model are improved through data augmentation.
[0031] 2. Through the image processing mechanism of heterogeneous parallel computing, parallel computing is performed on the GPU-CPU heterogeneous computing architecture to realize pipelined parallel processing of image acquisition and result processing with image preprocessing and model inference, thereby improving the system running speed.
[0032] 3. Through image processing and target detection, the system can identify the current player hitting the ball, determine if there are any obstructions on the table, determine if the ball is stationary, if the ball is out of bounds, and if the ball has entered the pocket.
[0033] 4. The system uses a Socket communication interface to enable data interaction and command transmission with other programs, facilitating integration into other systems. Attached Figure Description
[0034] Figure 1 This is a flowchart of the steps in the billiard ball out-of-bounds judgment method based on target detection and heterogeneous parallel computing of the present invention.
[0035] Figure 2 This is a flowchart illustrating the semi-automatic training process of the model of this invention.
[0036] Figure 3 This is a flowchart of the image processing based on heterogeneous parallel computing according to the present invention;
[0037] Figure 4 This is a flowchart illustrating the processing of the target detection results of the present invention.
Claims
1. A method for determining whether a billiard ball is out of bounds based on target detection and heterogeneous parallel computing, characterized in that, Includes the following steps: S1. Semi-automatic model training: Construct a semi-automatic model training process of "labeling-data augmentation-training-pseudo-labeling-human verification-data augmentation-training" to train the table corner detection model and the billiard ball / glove detection model; S2. Real-time image acquisition and preprocessing: The panoramic video stream of the tabletop is acquired in real time through a wide-angle camera on the top of the billiard table, and the acquired images are preprocessed in sequence by distortion correction, size scaling and color space conversion. S3. Image processing based on heterogeneous parallel computing: Under the GPU-CPU heterogeneous computing architecture, a task grouping parallel strategy is adopted. The subsequent processing of image acquisition and model inference results is handled by the CPU, while image preprocessing and model inference are handled by the GPU. The two groups of tasks run in parallel on different computing resources, realizing pipelined parallel processing of image acquisition and result processing and image preprocessing and model inference, effectively improving the overall processing efficiency and response speed of the system. S4. Billiards out-of-bounds judgment and player identification: After the billiards ball stabilizes, the current player's identity is identified; the visual detection results and RFID sensor data are combined to determine whether the billiards ball has gone out of bounds. S5. System Communication: Through the Socket communication interface, data interaction and command transmission between this system and external systems are realized.
2. The billiard ball out-of-bounds detection method based on target detection and heterogeneous parallel computing according to claim 1, characterized in that: In step S1, the specific implementation method of semi-automatic model training is as follows: S1.1 Select a portion of samples from the collected images, manually annotate them using the image annotation tool LabelImg, and expand the sample size through data augmentation. Specifically, the annotated data for table corners is used to train the table corner detection model, and the annotated data for billiard balls and gloves of different colors is used to train the billiard ball / glove detection model. S1.
2. The initial model obtained from the above training is used to automatically label the remaining unlabeled images. After the automatic labeling results are manually reviewed and corrected, data augmentation is performed to form a new training set. S1.
3. Continue training the table corner detection model and the billiard ball / glove detection model using the updated training set to further improve the detection accuracy and robustness of the models; S1.4 The above data augmentation methods include rotation, translation, flipping, scaling, perspective transformation, HSV color gamut change, Mosaic, MixUp, Copy-Paste, local random erasure, and brightness change.
3. The method for determining whether a billiard ball is out of bounds based on target detection and heterogeneous parallel computing according to claim 1, characterized in that: The preprocessing operation described in step S2 is accelerated by parallel computing based on the CUDA architecture.
4. The method for determining whether a billiard ball is out of bounds based on target detection and heterogeneous parallel computing according to claim 1, characterized in that, The image processing method based on heterogeneous parallel computing in step S3 is specifically implemented as follows: S3.1 The CPU is responsible for image acquisition and subsequent processing of model inference results; S3.2, GPU is responsible for image preprocessing and model inference tasks; The S3.3 GPU and CPU run simultaneously to achieve pipelined parallel processing of image acquisition and result processing with image preprocessing and model inference, effectively improving the overall processing efficiency and response speed of the system.
5. The method for determining whether a billiard ball is out of bounds based on target detection and heterogeneous parallel computing according to claim 1, characterized in that: In step S4, the specific implementation method for determining the stability of the billiard ball is as follows: S4.1 Billiard table background modeling: Before the game, the empty billiard table image is preprocessed, and the corner detection model is used to obtain the coordinates of the four corners to form the effective area of the billiard table surface, which is then saved as the background image of the table. S4.2 Occlusion Detection: Using OpenCV, the region of interest is set as the effective area of the desktop, and the billiard ball detection box area in the current frame is removed. The image of this area is subjected to difference, binarization, and erosion operations with the background image. If there are non-zero pixels in the final image, it is determined that there is an occlusion; otherwise, it is an unoccluded state. S4.
3. In an unobstructed state, if the displacement of each billiard ball in multiple consecutive frames is less than a preset threshold, it is determined to be "stable billiard ball"; otherwise, it is "unstable".
6. The method for determining whether a billiard ball is out of bounds based on target detection and heterogeneous parallel computing according to claim 1, characterized in that: In step S4, the method for identifying the player is as follows: after confirming that the billiard ball is in a stable state, if the current player's glove is not detected in the effective area of the billiard table in multiple consecutive frames, the current player's identity is recorded as unknown; if a glove of a certain color is detected in the effective area of the billiard table in multiple consecutive frames, the current player is recorded as the wearer of that color glove.
7. The method for determining whether a billiard ball is out of bounds based on target detection and heterogeneous parallel computing according to claim 1, characterized in that: In step S4, the method for detecting billiard balls going out of bounds is as follows: the billiard ball has an RFID tag built into it, and when a billiard ball enters the pocket, it will be recorded that the ball has entered the pocket. If a ball is detected to be outside the valid area of the billiard table, then the ball is recorded as out of bounds. After the billiards ball stabilizes, the detected billiards ball is compared with the billiards ball detected when the billiards ball stabilized last time. If a billiards ball is detected to have disappeared and there is no record of the billiards ball entering the pocket or going out of bounds, the billiards ball is recorded as going out of bounds.
8. The method for determining whether a billiard ball is out of bounds based on target detection and heterogeneous parallel computing according to claim 1, characterized in that: In step S5, the detection results are sent to other programs via Socket communication for further processing or decision-making. Simultaneously, other programs can also send control commands to this system through the Socket interface, thereby enabling control over the system's operation.
9. Pre-competition check-in of athlete information, characterized by: Before the match, players wear gloves of different colors and place them on the pool table. The system assigns a unique ID to each glove within the valid area of the pool table as a player's identification.
10. The method for determining whether a billiard ball is out of bounds based on target detection and heterogeneous parallel computing according to claim 1, characterized in that, The semi-automatic model training in step S1 and the image processing method based on heterogeneous parallel computing in step S3 are not only applicable to judging whether a billiard ball is out of bounds, but can also be applied to various scenarios that require real-time image analysis and decision-making.