Intelligent billiard judgment system based on image recognition

Abstract

The invention discloses an intelligent billiard judgment system based on image recognition, and the system comprises a signal collection module, a signal preprocessing module, a light source separation module, an initialization adaptive calibration module, a daily adaptive calibration module, and a judgment module. The recognition image of the intelligent judgment system in the billiard match is standardized, the accuracy of image recognition is improved, and therefore the accuracy of rule judgment is improved. The U-Net network based on spatial pyramid pooling realizes light source separation, and obtains intensity components and colors of a natural light source, a fixed artificial light source and a variable artificial light source for subsequent image calibration. And natural light change prediction is carried out based on the improved SARIMA-LSTM model, daily natural light calibration is carried out based on a prediction result, and the standard of the image is further improved.

Description

Technical Field

[0001] The invention relates to the field of image processing, and in particular to an intelligent billiards referee system based on image recognition. Background Art

[0002] The development of billiard referee system is closely related to the need for competition standardization. In the early days, it relied on manual visual judgment, which was easily affected by perspective error. In the late 20th century, electronic scoreboards and high-speed cameras were introduced to assist in the judgment of controversial balls. For example, snooker events use a multi-angle replay system to verify the order of shots and fouls. Intelligent upgrade in the 21st century: laser sensors monitor the static position of the ball, and some events try out automatic scoring systems. The World Billiards Federation and other organizations have formulated technical standards to ensure that referee equipment and table parameters are coordinated to achieve accuracy in judgment and fairness in events.

[0003] However, in actual use, each store has large differences in the installation height, left and right deviation, strong and weak light environment, etc. of the pool table. The actual complex light environment interferes with the color of the photographed billiard balls and affects the detection of the target object, resulting in a certain gap in accurate detection. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In order to solve the above technical problems, the present invention provides a billiards intelligent referee system based on image recognition.

[0006] (II) Technical solution

[0007] In order to solve the above-mentioned technical problems and achieve the purpose of the invention, the present invention is implemented by the following technical solutions: A billiards intelligent referee system based on image recognition includes a signal acquisition module, a signal preprocessing module, a light source separation module, an initialization adaptive calibration module, a daily adaptive calibration module, and a referee module; wherein, The signal acquisition module includes two high-speed image sensors and several light sensors. The two high-speed image sensors are used to collect images of the billiard tabletop, and the several light sensors are used to collect light changes in the surrounding environment of the billiard tabletop. A signal preprocessing module is used to denoise the image signal collected by the image sensor and remove abnormal points from the data of ambient light brightness and energy proportion of each band collected by the light sensor; A light source separation module, which is used to separate the light source by combining the image signal collected by the image sensor and the signal collected by the light sensor; Initialize the adaptive calibration module, which is used to initialize and calibrate the acquired image; A daily adaptive calibration module, which is used to perform daily standardized image recalibration based on the prediction of natural light changes; The referee module establishes three-dimensional spatial coordinate fusion based on standardized images to perform intelligent refereeing during billiard games.

[0008] Furthermore, the two high-speed image sensors are placed above the billiard table, and the light sensors are evenly arranged around the billiard table.

[0009] Furthermore, the light source separation module separates and obtains intensity components and colors of the natural light source, the fixed artificial light source and the variable artificial light source.

[0010] Furthermore, the light source separation module establishes a separation model based on the U-Net network architecture with spatial pyramid pooling, inputs the extracted image features and the time domain features and frequency domain features of the light sensor data, and outputs the intensity components and colors of natural light sources, fixed artificial light sources, and changing artificial light sources.

[0011] Furthermore, the initialization adaptive calibration module includes extracting light source characteristics of natural light sources, fixed artificial light sources and variable artificial light sources based on the separation results, and the light source characteristics include light attributes and time dynamics, wherein the light attributes include intensity components and colors of each type of light source; the time dynamics are time-varying characteristics of the light source intensity, including periodic characteristics, trend characteristics, and change rate.

[0012] Furthermore, the initialized adaptive calibration module also includes constructing an initialized adaptive calibration model, which calculates calibration parameters based on an improved generative adversarial network. The model input is light source features, and the output is adaptive calibration parameters, including calibration parameters of brightness and color corresponding to different points on the image.

[0013] Furthermore, the loss function of the improved adversarial generative network discriminator is:

[0014] Where: is the number of cores; x i and x j are the i-th and j-th true samples in X respectively, and They are The i-th and j-th generated samples in The number of samples.

[0015] Furthermore, the prediction of changes in natural light in the daily adaptive calibration module is based on a SARIMA-LSTM model, which includes a SARIMA submodule and an LSTM submodule. The final prediction result is obtained by fusing the first prediction result obtained by the SARIMA submodule and the second prediction result obtained by the LSTM submodule based on a weighting method.

[0016] Furthermore, the SARIMA-LSTM model training includes constructing a first training data set and a second training data set corresponding to the SARIMA submodule and the LSTM submodule; wherein the first training data set is selected according to the periodic laws of seasonal periodicity and the periodicity of one day, respectively, and the second training data set is selected based on non-periodic factors.

[0017] Furthermore, the training data set is clustered according to dates with a year cycle and hours with a day cycle, and the clustering results are evenly sampled to obtain a first training data set. Data clustering is performed based on weather and curtain status, and even sampling is performed in each category to obtain a second training data set.

[0018] (III) Beneficial effects

[0019] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention realizes standardization of the recognition image of the intelligent referee system in billiards competition through secondary calibration of the lighting conditions, thereby improving the accuracy of image recognition and thus improving the accuracy of rule judgment.

[0020] (2) The present invention realizes the separation of light sources based on the U-Net network of spatial pyramid pooling, and obtains the intensity components and colors of natural light sources, fixed artificial light sources and variable artificial light sources for subsequent image calibration.

[0021] (3) The present invention predicts the changes in natural light based on the improved SARIMA-LSTM model, and performs daily natural light calibration based on the prediction results, thereby further improving the standardization of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 It is a structural schematic diagram of a billiards intelligent referee system based on image recognition according to an embodiment of the present application; Figure 2 It is a schematic diagram of the natural light change prediction process according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0024] The following describes the embodiments of the present disclosure through specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0025] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0026] See also Figure 1 A billiards intelligent referee system based on image recognition includes a signal acquisition module, a signal preprocessing module, a light source separation module, an initialization adaptive calibration module, a daily adaptive calibration module, and a referee module; wherein, The signal acquisition module includes two high-speed image sensors and several light sensors. The image of the billiard table is collected by the two high-speed image sensors, and the light changes in the environment around the billiard table are collected by the several light sensors, so as to detect the intensity of the ambient light around the billiard table and the energy proportion of each band over a period of time. Specifically, the two high-speed image sensors are placed above the billiard table, and the light sensors are evenly arranged around the billiard table.

[0027] The signal preprocessing module is used to denoise the image signal collected by the image sensor and remove abnormal points from the data of the ambient light brightness and the energy proportion of each band collected by the light sensor.

[0028] The light source separation module is used to separate the light sources by combining the image signal collected by the image sensor and the signal collected by the light sensor, and obtain the intensity components and colors of the natural light source, the fixed artificial light source and the variable artificial light source. The fixed artificial light source is generally a fixed light, whose brightness and position remain basically unchanged; the variable artificial light source generally comes from the accidental external environment changes on site, such as the relatively short-term illumination of light sources such as flashlights and mobile phones.

[0029] This module includes the following steps: a. Extracting image features and light sensor data features, optionally using depthwise separable convolution to extract image features; extracting light sensor data features, including performing frequency domain transformation on light sensor data to extract frequency domain features; b. Construct and train a separation model, wherein the model inputs the extracted image features and the time domain features and frequency domain features of the light sensor data, and outputs the intensity components and colors of natural light sources, fixed artificial light sources, and variable artificial light sources; Specifically, the separation model is established based on a U-Net network architecture of spatial pyramid pooling, and a spatial pyramid pooling layer is set at a network input. The architecture introduces a pyramid pooling mechanism and utilizes a U-Net supervised classification network to achieve rapid classification of complex signals, and data of any dimension can be input without the need for cutting and scaling operations in advance. In addition, in order to improve the training efficiency, the present invention adopts an Adam optimization algorithm to dynamically control the learning rate, and dynamically adjusts the learning rate by calculating a first-order moment estimation and a second-order moment estimation of a learning rate gradient function, thereby improving the training efficiency of the model and avoiding the phenomenon that the model performance fluctuates too much due to an excessively large learning rate in the later stage of training.

[0030] c. Input the preprocessed image and light sensor signal into the separation model to obtain the intensity components and colors of natural light sources, fixed artificial light sources, and variable artificial light sources.

[0031] Initialize the adaptive calibration module, which is used to initialize and calibrate the acquired image; During the initial installation of the equipment, since each store has different installation heights, left and right deviations, strong and weak light environments, etc. for the pool table, an initialization automatic adaptive calibration is performed to ensure daily use; the initialization adaptive calibration step generally requires calibration when the machine is installed and after a long time interval, and this step is not performed in daily use.

[0032] This module performs the following steps: a. Extracting the light source characteristics of natural light sources, fixed artificial light sources and variable artificial light sources according to the separation results, wherein the light source characteristics include light attributes and time dynamics, wherein the light attributes include the intensity components and colors of each type of light source; the time dynamics are the time-varying characteristics of the light source intensity, including periodic characteristics, trend characteristics, and change rate; b. Constructing an initialization adaptive calibration model; The initialized adaptive calibration model calculates calibration parameters based on an improved generative adversarial network. The model input is light source features, and the output is adaptive calibration parameters, including calibration parameters of brightness and color corresponding to different points on the image, so as to remove the influence of changing light sources of artificial light sources, compensate natural light sources and fix artificial light sources to set standard values, and obtain a standard image under standard lighting conditions.

[0033] The improved generative adversarial network improves the objective function of the generative adversarial network; In order to enhance the learning of details in real samples, the present invention improves the loss function of the discriminator. The loss function of the improved discriminator is:

[0034] Where: is the number of cores; x i and x j are the i-th and j-th true samples in X respectively, and They are The i-th and j-th generated samples in The number of samples.

[0035] After improvement, by minimizing The first one significantly improves the feature difference of real samples in the discriminator output D(x), enabling the discriminator to capture the detailed features of real data more accurately. This improvement not only enhances the sensitivity of the discriminator to sample differences and provides a more directional gradient feedback for the generator, but also effectively reduces the computational complexity of network training by simplifying the loss function structure, thereby achieving higher training efficiency while improving the quality of generated samples.

[0036] c. Calibrate the image according to the calibration parameters to obtain a first calibration image.

[0037] The daily adaptive calibration module is used to perform daily standardized image recalibration based on the prediction of changes in natural light.

[0038] Since in daily use, natural light conditions will change with changes in weather, seasons, and lighting conditions within a day, the present invention performs daily standardized image optimization based on changes in natural light, and compensates for the image according to the difference between natural light and set standard light, thereby obtaining an optimized standardized image.

[0039] The module includes predicting the change of natural light and dynamically adjusting the first standardized image according to the difference between the predicted result of the change of natural light and the standard natural light to obtain the second standardized image.

[0040] The prediction of natural light changes includes the following steps: a. Data preprocessing: cleaning the original sequence data, including processing missing values ​​and outliers, to ensure that the data meets the input requirements of the model; model input data includes weather, time, and curtain status; time includes date and hour; output is the predicted natural light intensity; b. Establish a SARIMA-LSTM model, which includes a SARIMA submodule and an LSTM submodule, wherein the SARIMA submodule tends to process data with high periodic correlation, and the LSTM submodule tends to process non-periodic related data. Therefore, when training the submodules, the training data of the two are selected according to different schemes.

[0041] c. Construction of training data set: constructing the first training data set and the second training data set corresponding to the SARIMA submodule and the LSTM submodule respectively; wherein the first training data set is selected according to the periodic laws of seasonal periodicity and the periodicity of one day, and the second training data set is selected based on non-periodic factors. Specifically, the data set is clustered according to dates with a year as the cycle and hours with a day as the cycle, and the clustering results are uniformly sampled to obtain the first training data set, thereby ensuring the integrity of the periodic data of the first training data set; data clustering is performed based on weather and curtain status, and uniform sampling is performed in each category to obtain the second training data set.

[0042] d. Prediction result fusion

[0043] The final prediction result is obtained by fusing the first prediction result obtained by the SARIMA submodule and the second prediction result obtained by the LSTM submodule based on the weight method.

[0044] The referee module establishes three-dimensional spatial coordinate fusion based on the second standardized image to perform intelligent refereeing during the billiards game.

[0045] In this embodiment, by secondary calibration of the lighting conditions, the recognition image of the intelligent referee system in the billiards game is standardized, the accuracy of image recognition is improved, and thus the accuracy of rule judgment is improved. The U-Net network based on spatial pyramid pooling realizes the separation of light sources, and obtains the intensity components and colors of natural light sources, fixed artificial light sources, and variable artificial light sources for subsequent image calibration. Based on the improved SARIMA-LSTM model, the change of natural light is predicted, and daily natural light calibration is performed based on the prediction results, which further improves the standardization of the image.

[0046] The embodiments described above are only descriptions of the preferred implementation modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A billiards intelligent referee system based on image recognition, characterized in that: It includes signal acquisition module, signal preprocessing module, light source separation module, initialization adaptive calibration module, daily adaptive calibration module and referee module; among them, The signal acquisition module includes two high-speed image sensors and several light sensors. The two high-speed image sensors are used to collect images of the billiard tabletop, and the several light sensors are used to collect light changes in the surrounding environment of the billiard tabletop. A signal preprocessing module is used to denoise the image signal collected by the image sensor and remove abnormal points from the data of ambient light brightness and energy proportion of each band collected by the light sensor; A light source separation module, which is used to separate the light source by combining the image signal collected by the image sensor and the signal collected by the light sensor; Initialize the adaptive calibration module, which is used to initialize and calibrate the acquired image; A daily adaptive calibration module, which is used to perform daily standardized image recalibration based on the prediction of natural light changes; The referee module establishes three-dimensional spatial coordinate fusion based on standardized images to perform intelligent refereeing during billiard games.

2. The billiards intelligent referee system according to claim 1, characterized in that: The two high-speed image sensors are placed above the billiard table, and the light sensors are evenly arranged around the billiard table.

3. The billiards intelligent referee system according to claim 1, characterized in that: The light source separation module separates and obtains intensity components and colors of natural light sources, fixed artificial light sources and variable artificial light sources.

4. The billiards intelligent referee system according to claim 3, characterized in that: The light source separation module establishes a separation model based on the U-Net network architecture with spatial pyramid pooling. It inputs the extracted image features and the time domain features and frequency domain features of the light sensor data, and outputs the intensity components and colors of natural light sources, fixed artificial light sources, and variable artificial light sources.

5. The billiards intelligent referee system according to claim 1, characterized in that: The initialization adaptive calibration module includes extracting light source characteristics of natural light sources, fixed artificial light sources and variable artificial light sources according to the separation results, and the light source characteristics include light attributes and time dynamics, wherein the light attributes include intensity components and colors of each type of light source; the time dynamics are the time-varying characteristics of the light source intensity, including periodic characteristics, trend characteristics, and change rate.

6. The billiards intelligent referee system according to claim 5, characterized in that: The initialized adaptive calibration module also includes constructing an initialized adaptive calibration model, wherein the initialized adaptive calibration model calculates calibration parameters based on an improved generative adversarial network, wherein the model input is light source features, and the output is adaptive calibration parameters, including calibration parameters of brightness and color corresponding to different points on the image; the improved generative adversarial network improves the loss function of the discriminator.

7. The billiards intelligent referee system according to claim 6, characterized in that: The loss function of the improved adversarial generation network discriminator is: Where: is the number of cores; x i and x j are the i-th and j-th true samples in X respectively, and They are The i-th and j-th generated samples in The number of samples.

8. The billiards intelligent referee system according to claim 1, characterized in that: The prediction of changes based on natural light in the daily adaptive calibration module is based on a SARIMA-LSTM model, wherein the model includes a SARIMA submodule and an LSTM submodule, and a first prediction result obtained by the SARIMA submodule and a second prediction result obtained by the LSTM submodule are fused based on a weighting method to obtain a final prediction result; The SARIMA-LSTM model training includes constructing a first training data set and a second training data set corresponding to the SARIMA submodule and the LSTM submodule.

9. The billiards intelligent referee system according to claim 8, characterized in that: The first training data set is selected according to the periodic laws of seasonal periodicity and daily periodicity, and the second training data set is selected based on non-periodic factors.

10. The billiards intelligent referee system according to claim 1, characterized in that: The training data set is clustered according to dates with a year cycle and hours with a day cycle, and the clustering results are evenly sampled to obtain the first training data set. Data clustering is performed based on weather and curtain status, and even sampling is performed in each category to obtain the second training data set.

Images