Billiard intelligent judging system based on image recognition

Abstract

The application discloses a billiards intelligent referee system based on image recognition, which comprises a signal acquisition module, a signal preprocessing module, a light source separation module, an initialization adaptive calibration module, a daily adaptive calibration module and a referee module. The application realizes the standardization processing of the identified image of the intelligent referee system in the billiards competition through the secondary calibration of the light condition, improves the accuracy of the image recognition, and thus improves the accuracy of the rule judgment. The U-Net network based on the spatial pyramid pooling realizes the separation of the light source, obtains the intensity components and colors of the natural light source, the fixed artificial light source and the changeable artificial light source, and is used for the subsequent image calibration. The improved SARIMA-LSTM model is used for the change prediction of the natural light, and the daily natural light calibration is carried out based on the prediction result, so that the standardization of the image is further improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an intelligent billiards referee system based on image recognition. Background Art

[0002] The development of billiards refereeing systems is closely tied to the need for standardized tournaments. Early decisions relied on manual visual inspection, which was susceptible to perspective errors. In the late 20th century, electronic scoreboards and high-speed cameras were introduced to assist with the determination of controversial balls. Snooker tournaments, for example, employ multi-angle replay systems to verify shot sequences and fouls. In the 21st century, intelligent upgrades have emerged: laser sensors monitor the stationary position of the ball, and some tournaments are piloting automated scoring systems. The World Snooker Federation and other organizations have developed technical standards to ensure that refereeing equipment is coordinated with table parameters, ensuring accurate judging and fair competition.

[0003] However, in actual use, each store has large differences in the installation height, left and right deviation, strong and weak light environment, etc. of the pool table. The actual complex lighting environment interferes with the color of the photographed billiard balls and affects the detection of the target object, resulting in a certain gap in accurate detection. Summary of the Invention

[0004] (1) Technical problems solved

[0005] In order to solve the above technical problems, the present invention provides a billiards intelligent referee system based on image recognition.

[0006] (2) Technical solution

[0007] In order to solve the above-mentioned technical problems and achieve the purpose of the invention, the present invention is implemented through the following technical solutions:

[0008] An intelligent billiard referee system based on image recognition includes a signal acquisition module, a signal preprocessing module, a light source separation module, an initialization adaptive calibration module, a daily adaptive calibration module, and a referee module; wherein,

[0009] The signal acquisition module includes two high-speed image sensors and several light sensors. The two high-speed image sensors collect images of the billiard table, and the several light sensors collect light changes around the billiard table.

[0010] The signal preprocessing module is used to denoise the image signal collected by the image sensor and remove abnormal points in the data of ambient light brightness and energy ratio of each band collected by the light sensor;

[0011] A light source separation module is used to separate the light source by combining the image signal collected by the image sensor and the signal collected by the light sensor;

[0012] Initialize the adaptive calibration module, which is used to initialize and calibrate the collected image;

[0013] A daily adaptive calibration module, which is used to perform daily standardized image recalibration based on the prediction of natural light changes;

[0014] The referee module establishes three-dimensional spatial coordinate fusion based on standardized images to perform intelligent refereeing during billiard games.

[0015] Furthermore, the two high-speed image sensors are placed above the billiard table, and the light sensors are evenly arranged around the billiard table.

[0016] Furthermore, the light source separation module separates and obtains the intensity components and colors of the natural light source, the fixed artificial light source and the variable artificial light source.

[0017] Furthermore, the light source separation module establishes a separation model based on the U-Net network architecture with spatial pyramid pooling, inputs the extracted image features and the time domain features and frequency domain features of the light sensor data, and outputs the intensity components and colors of natural light sources, fixed artificial light sources, and changing artificial light sources.

[0018] Furthermore, the initialization adaptive calibration module includes extracting light source characteristics of natural light sources, fixed artificial light sources and changing artificial light sources based on the separation results, and the light source characteristics include light attributes and time dynamics, wherein the light attributes include the intensity components and colors of each type of light source; the time dynamics are the time-varying characteristics of the light source intensity, including periodic characteristics, trend characteristics, and change rate.

[0019] Furthermore, the initialized adaptive calibration module also includes constructing an initialized adaptive calibration model, which calculates calibration parameters based on an improved generative adversarial network. The model input is light source features, and the output is adaptive calibration parameters, including calibration parameters of brightness and color corresponding to different points on the image.

[0020] Furthermore, the loss function of the improved adversarial generative network discriminator is:

[0021]

[0022] Where: is the number of cores; x i and x j are the i-th and j-th true samples in X respectively, and They are The i-th and j-th generated samples in The number of samples.

[0023] Furthermore, the prediction of changes in natural light in the daily adaptive calibration module is based on the SARIMA-LSTM model, which includes a SARIMA submodule and an LSTM submodule. The final prediction result is obtained by fusing the first prediction result obtained by the SARIMA submodule and the second prediction result obtained by the LSTM submodule based on the weighting method.

[0024] Furthermore, the SARIMA-LSTM model training includes constructing a first training data set and a second training data set corresponding to the SARIMA submodule and the LSTM submodule; wherein the first training data set is selected according to the periodic laws of seasonal periodicity and daily periodicity, and the second training data set is selected based on non-periodic factors.

[0025] Furthermore, the training data set is clustered according to dates with a yearly cycle and hours with a daily cycle, and the clustering results are uniformly sampled to obtain the first training data set. Data clustering is performed based on weather and curtain status, and uniform sampling is performed in each category to obtain the second training data set.

[0026] (3) Beneficial effects

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] (1) The present invention achieves standardized processing of the recognition image of the intelligent referee system in billiards competitions through secondary calibration of the lighting conditions, thereby improving the accuracy of image recognition and thus improving the accuracy of rule judgment.

[0029] (2) The present invention realizes the separation of light sources based on the U-Net network of spatial pyramid pooling, and obtains the intensity components and colors of natural light sources, fixed artificial light sources and variable artificial light sources for subsequent image calibration.

[0030] (3) The present invention predicts the changes in natural light based on the improved SARIMA-LSTM model, and performs daily natural light calibration based on the prediction results, further improving the standardization of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0032] Figure 1 1 is a schematic structural diagram of a billiards intelligent referee system based on image recognition according to an embodiment of the present application;

[0033] Figure 22 is a schematic diagram of a natural light change prediction process according to an embodiment of the present application. DETAILED DESCRIPTION

[0034] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0035] The following describes the embodiments of the present disclosure through specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0036] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The illustrations only show components related to the present disclosure and are not drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.

[0037] See also Figure 1 A billiards intelligent referee system based on image recognition includes a signal acquisition module, a signal preprocessing module, a light source separation module, an initialization adaptive calibration module, a daily adaptive calibration module, and a referee module; wherein,

[0038] The signal acquisition module includes two high-speed image sensors and several light sensors. The two high-speed image sensors collect images of the billiard table surface, and the several light sensors collect changes in the light environment around the billiard table surface, detecting the intensity of the ambient light around the billiard table surface and the energy proportion of each band over a period of time. Specifically, the two high-speed image sensors are placed above the billiard table, and the light sensors are evenly arranged around the billiard table.

[0039] The signal preprocessing module is used to denoise the image signal collected by the image sensor and remove abnormal points in the data of ambient light brightness and energy proportion of each band collected by the light sensor.

[0040] The light source separation module combines the image signal collected by the image sensor with the signal collected by the light sensor to perform light source separation, obtaining the intensity components and colors of natural light sources, fixed artificial light sources, and variable artificial light sources. Fixed artificial light sources are generally fixed lights with essentially constant brightness and position. Variable artificial light sources generally arise from accidental changes in the external environment, such as brief illumination from flashlights, mobile phones, and other light sources.

[0041] This module includes the following steps:

[0042] a. Extracting image features and light sensor data features, optionally using depthwise separable convolution for image feature extraction; extracting light sensor data features, including performing frequency domain transformation on the light sensor data to extract frequency domain features;

[0043] b. Build and train a separation model that takes as input the extracted image features and the time and frequency domain features of the light sensor data, and outputs the intensity components and colors of natural light sources, fixed artificial light sources, and variable artificial light sources;

[0044] Specifically, the separation model is established based on a U-Net network architecture with spatial pyramid pooling, and a spatial pyramid pooling layer is set at the network input. The architecture introduces a pyramid pooling mechanism and utilizes a U-Net supervised classification network to achieve rapid classification of complex signals. Data of any dimension can be input without prior cropping and scaling operations. In addition, to improve training efficiency, the present invention adopts an Adam optimization algorithm to dynamically control the learning rate, and dynamically adjusts the learning rate by calculating the first-order moment estimate and the second-order moment estimate of the learning rate gradient function, thereby improving the training efficiency of the model and avoiding the phenomenon that the model performance fluctuates too much due to an excessively large learning rate in the late training period.

[0045] c. Input the preprocessed image and light sensor signal into the separation model to obtain the intensity components and colors of natural light sources, fixed artificial light sources, and variable artificial light sources.

[0046] Initialize the adaptive calibration module, which is used to initialize and calibrate the collected image;

[0047] During the initial installation of the equipment, since each store has different requirements for the height, left and right deviation, and strong and weak light environments of the pool table, an initialization automatic adaptive calibration is performed to ensure daily use. The initialization adaptive calibration step generally requires calibration when the machine is installed and after a long time interval, and this step is not performed during daily use.

[0048] This module performs the following steps:

[0049] a. Extract light source features for natural light sources, fixed artificial light sources, and variable artificial light sources based on the separation results. The light source features include light attributes and temporal dynamics. Light attributes include the intensity components and color of each type of light source; temporal dynamics are the time-varying characteristics of light source intensity, including periodic characteristics, trend characteristics, and change rate;

[0050] b. Constructing an initialized adaptive calibration model;

[0051] The initialized adaptive calibration model calculates calibration parameters based on an improved generative adversarial network. The model input is light source features, and the output is adaptive calibration parameters, including calibration parameters for brightness and color corresponding to different points on the image. This removes the influence of changing artificial light sources, compensates natural light sources, and fixes artificial light sources to set standard values, thereby obtaining a standard image under standard lighting conditions.

[0052] The improved generative adversarial network improves the objective function of the generative adversarial network;

[0053] In order to enhance the learning of details in real samples, the present invention improves the loss function of the discriminator. The loss function of the improved discriminator is:

[0054]

[0055] Where: is the number of cores; x i and x j are the i-th and j-th true samples in X respectively, and They are The i-th and j-th generated samples in The number of samples.

[0056] After improvement, by minimizing The first one significantly improves the feature difference of real samples in the discriminator output D(x), enabling the discriminator to capture the detailed features of real data more accurately. This improvement not only enhances the discriminator's sensitivity to sample differences and provides more directional gradient feedback for the generator, but also effectively reduces the computational complexity of network training by simplifying the loss function structure, thereby achieving higher training efficiency while improving the quality of generated samples.

[0057] c. Calibrate the image according to the calibration parameters to obtain a first calibration image.

[0058] The daily adaptive calibration module is used to perform daily standardized image recalibration based on the prediction of natural light changes.

[0059] Since natural light conditions in daily use will change with changes in weather, seasons, and lighting conditions throughout the day, the present invention performs daily standardized image optimization based on changes in natural light, and compensates for the image according to the difference between natural light and the set standard light, thereby obtaining an optimized standardized image.

[0060] This module includes predicting the change of natural light and dynamically adjusting the first standardized image according to the difference between the predicted result of the change of natural light and the standard natural light to obtain the second standardized image.

[0061] The prediction of natural light changes includes the following steps:

[0062] a. Data preprocessing: Cleaning the raw sequence data, including handling missing values ​​and outliers, to ensure the data meets the model's input requirements. Model input data includes weather, time, and curtain status; time includes date and hour; and the output is the predicted natural light intensity.

[0063] b. Establish a SARIMA-LSTM model, which includes a SARIMA submodule and an LSTM submodule. The SARIMA submodule is biased towards processing data with high periodic correlation, while the LSTM submodule is biased towards processing non-periodic related data. Therefore, when training the submodules, the training data for the two are selected according to different schemes.

[0064] c. Construction of training datasets: Constructing the first and second training datasets corresponding to the SARIMA submodule and LSTM submodule, respectively. The first training dataset is selected based on seasonal periodicity and daily periodicity, respectively, and the second training dataset is selected based on non-periodic factors. Specifically, the dataset is clustered according to dates with a yearly cycle and hours with a daily cycle, and the clustering results are uniformly sampled to obtain the first training dataset, ensuring the integrity of the periodic data in the first training dataset. Data clustering is performed based on weather and curtain status, and uniform sampling is performed in each category to obtain the second training dataset.

[0065] d. Prediction result fusion

[0066] The final prediction result is obtained by fusing the first prediction result obtained by the SARIMA submodule and the second prediction result obtained by the LSTM submodule based on the weighted method.

[0067] The referee module establishes three-dimensional spatial coordinate fusion based on the second standardized image to perform intelligent refereeing during the billiards game.

[0068] In this implementation, secondary calibration of lighting conditions is used to standardize the images used by the intelligent referee system in billiards matches, improving image recognition accuracy and, consequently, the accuracy of rule judging. A U-Net network based on spatial pyramid pooling separates light sources, obtaining the intensity components and colors of natural, fixed, and variable artificial light sources for subsequent image calibration. Natural light changes are predicted using an improved SARIMA-LSTM model, and daily natural light calibration is performed based on the predicted results, further improving image standardization.

[0069] The embodiments described above are merely descriptions of preferred implementations of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should fall within the scope of protection determined by the claims of the present invention.

Claims

1. A billiards intelligent referee system based on image recognition, characterized in that: It includes signal acquisition module, signal preprocessing module, light source separation module, initialization adaptive calibration module, daily adaptive calibration module and referee module; among them, The signal acquisition module includes two high-speed image sensors and several light sensors. The two high-speed image sensors collect images of the billiard table, and the several light sensors collect light changes around the billiard table. The signal preprocessing module is used to denoise the image signal collected by the image sensor and remove abnormal points in the data of ambient light brightness and energy ratio of each band collected by the light sensor; A light source separation module is used to separate the light sources by combining the image signal collected by the image sensor and the signal collected by the light sensor; the light source separation module separates the intensity components and colors of the natural light source, the fixed artificial light source, and the variable artificial light source; Initialize the adaptive calibration module, which is used to initialize and calibrate the collected image; A daily adaptive calibration module is used to perform daily standardized image recalibration based on natural light change predictions. The natural light change predictions in the daily adaptive calibration module are based on a SARIMA-LSTM model, which includes a SARIMA submodule and an LSTM submodule. The final prediction result is obtained by fusing a first prediction result obtained by the SARIMA submodule and a second prediction result obtained by the LSTM submodule using a weighting method. A training dataset is constructed, where a first training dataset and a second training dataset are constructed corresponding to the SARIMA submodule and the LSTM submodule, respectively. The first training dataset is selected based on seasonal periodicity and daily periodicity, respectively, and the second training dataset is selected based on non-periodic factors. The referee module establishes three-dimensional spatial coordinate fusion based on standardized images to perform intelligent refereeing during billiard games.

2. The billiards intelligent referee system according to claim 1, characterized in that: The two high-speed image sensors are placed above the billiard table, and the light sensors are evenly arranged around the billiard table.

3. The billiards intelligent referee system according to claim 2, characterized in that: The light source separation module establishes a separation model based on the U-Net network architecture with spatial pyramid pooling. It inputs the extracted image features and the time domain features and frequency domain features of the light sensor data, and outputs the intensity components and colors of natural light sources, fixed artificial light sources, and changing artificial light sources.

4. The billiards intelligent referee system according to claim 1, characterized in that: The initialization adaptive calibration module includes extracting light source characteristics of natural light sources, fixed artificial light sources and variable artificial light sources based on the separation results. The light source characteristics include light attributes and temporal dynamics, wherein the light attributes include the intensity components and colors of each type of light source; the temporal dynamics are the time-varying characteristics of the light source intensity, including periodic characteristics, trend characteristics, and change rate.

5. The billiards intelligent referee system according to claim 4, characterized in that: The initialized adaptive calibration module also includes constructing an initialized adaptive calibration model, which calculates calibration parameters based on an improved adversarial generative network. The model input is light source features, and the output is adaptive calibration parameters, including calibration parameters of brightness and color corresponding to different points on the image; the improved adversarial generative network improves the loss function of the discriminator.

6. The billiards intelligent referee system according to claim 5, characterized in that: The loss function of the improved adversarial generative network discriminator is: Where: is the kernel function; x i and x j are the i-th and j-th true samples in X respectively, and They are The i-th and j-th generated samples in The number of samples.

7. The billiards intelligent referee system according to claim 6, characterized in that: The first training data set is selected based on the periodic laws of seasonal periodicity and daily periodicity, and the second training data set is selected based on non-periodic factors.

8. The billiards intelligent referee system according to claim 1, characterized in that: The training data set is clustered according to the date with a year cycle and the hour with a day cycle, and the clustering results are evenly sampled to obtain the first training data set. The data is clustered based on weather and curtain status, and evenly sampled in each category to obtain the second training data set.

Images