A billiard ball recognition method based on deep learning
By improving the YOLOv8 network structure, combined with deep learning and data enhancement technology, the problem of low accuracy in billiard ball recognition in complex environments is solved, and high-precision billiard ball recognition and goal time judgment are achieved.
Patent Information
- Application Number
- CN202410841553.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-27
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-06-27
AI Technical Summary
The existing billiards recognition methods are difficult to achieve high-precision recognition in complex environments, especially for small targets such as billiards, which have poor results.
Using a deep learning-based method, the YOLOv8 network structure is improved, and the SPD-Conv module is added before the backbone network, the Transformer attention mechanism and AGBiFPN feature fusion module are introduced, and the NWD Loss loss function is used to enhance feature extraction and multi-layer feature fusion capabilities. , combined with data enhancement technology, a high-precision billiards recognition model is established.
Accurate detection of billiard ball goals in complex contexts is achieved, the calculation amount and parameter amount are reduced, the feature extraction ability of small targets and the accuracy of label allocation is improved, and the accuracy and accuracy of billiard ball recognition is significantly improved.
Smart Images

Figure CN118397401B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of sports and computer-aided equipment, and in particular to a billiard ball recognition method based on deep learning. Background Art
[0002] Existing billiard ball recognition methods usually use Hough transform, HSV color recognition, edge detection and other methods to recognize billiard balls, which often cannot achieve good results in complex and changeable environments in reality. In recent years, the rapid development of deep learning has driven the progress of machine vision research. Deep learning algorithms based on convolutional neural networks have been widely used in the field of target detection. However, these algorithms are mainly designed for large and medium-sized targets such as people and vehicles. For small targets with fewer features such as billiard balls, it is difficult to extract enough feature information and the effect is not good.
[0003] Therefore, the current mainstream algorithm is difficult to achieve ideal accuracy in billiard ball recognition and needs further research and improvement. Summary of the invention
[0004] The problem to be solved by the present invention is to provide a billiard ball recognition method based on deep learning, which can recognize billiard balls with high precision, judge the moment of goal scoring, and automatically generate high-quality wonderful moment videos.
[0005] The present invention adopts the following technical solution: a billiard ball recognition method based on deep learning, comprising the following steps:
[0006] S1: Collect billiard table pictures in real environment, manually mark balls of different colors, build billiard dataset, perform data enhancement on the billiard dataset, and divide the enhanced billiard dataset into training set, validation set and test set;
[0007] S2: Improve the YOLOv8 network structure as the baseline network: For the backbone network, add an SPD-Conv (Space-to-Depth Convolution) module before each layer of the C2f module; introduce the Transformer attention mechanism at the bottom of the backbone network to enhance the feature map representation capability; for the neck network, introduce the feature fusion module AGBiFPN (Attention Guided Bidirectional Feature Pyramid Network); for the head network, add a small target detection head P2, and use NWD Loss (Normalized Wasserstein Distance Loss) instead of the YOLOv8 loss function; use the improved YOLOv8 network structure to train the billiards dataset and obtain a billiards recognition deep learning model;
[0008] S3: Use a camera to collect real-time billiard tabletop images, and input the images into the above-mentioned billiard recognition model for detection.
[0009] S4: According to the detection result of step S3 and the billiards motion characteristics, the billiards goal moment is determined.
[0010] Preferably, in step S1, the collected billiard tabletop pictures are billiard pictures under different scenes, different angles and different illuminations, and the labelImg labeling tool is used to manually label 16 different ball colors.
[0011] The data enhancement methods include rotation, translation, flipping, cropping, scaling, Gaussian blur, color gamut change, and Mosaic data enhancement methods;
[0012] Among them, Gaussian blur is used to generate pictures with different blur levels to improve the detection accuracy of blurred data; color gamut change is used to generate pictures of different colors, including adjusting brightness, saturation, contrast, and white balance; Mosaic selects four random pictures, each of which retains the corresponding anchor frame, and splices the four pictures together to obtain a new picture to enrich the background of billiard recognition.
[0013] Preferably, in step S2, the YOLOv8 network structure is used as the reference network. For the backbone network, an SPD-Conv module is added before each C2f module. The SPD-Conv module is an improvement on the traditional Conv layer, and is composed of an SPD layer and a non-strided convolution layer. It can reduce the loss of small target information and improve the accuracy of feature extraction.
[0014] Preferably, in step S2, the Transformer attention mechanism is introduced at the bottom of the backbone network. As a method to enhance the representation capability of feature maps, the Transformer attention mechanism includes two sublayers: a multi-head attention layer and an MLP. Multi-head attention helps the current node not only focus on the current pixel, but also obtain contextual semantics; at the same time, since the final network resolution of the backbone network is low, applying the Transformer to its feature map can reduce computational and memory costs.
[0015] Preferably, in step S2, for the neck network, a feature fusion module AGBiFPN is introduced, and the AGBiFPN module is intended to fuse multi-scale features to prevent the loss of semantic details related to small targets, so that the model can better detect targets.
[0016] BiFPN introduces bidirectional feature propagation, i.e., top-down and bottom-up feature flow, which enables more comprehensive and rich information transfer and fusion between different levels. A dynamic sparse attention mechanism is introduced on top of BiFPN to extract information from very small target layers, directly skipping the calculation of the structure of the least relevant area to efficiently generate target features with strong discriminative ability. Then, a top-down path is added to transfer high-level semantic feature information to guide subsequent network modules to perform feature fusion and generate features with stronger discriminative ability.
[0017] Preferably, in step S2, for the head network, NWD Loss is used instead of the loss function of YOLOv8; the IoU (Intersection over Union)-based indicator is very sensitive to the position deviation of small targets. When used in an Anchor-based detector, it will lead to inaccurate label assignment and seriously reduce the detection performance.
[0018] NWD Loss models the BBox (Bounding Box) as a two-dimensional Gaussian distribution, and then uses the Gaussian distribution of the Normalized Wasserstein distance to calculate the similarity between them.
[0019] Preferably, the deep model for billiard recognition inputs the training set into the improved YOLOv8 network structure to perform target detection model training to obtain a deep learning model for billiard recognition, inputs the verification set into the deep learning model for verification, and optimizes the deep learning model and training parameters according to the results obtained by the verification to obtain a trained deep learning model for billiard recognition.
[0020] Preferably, in step S3, the collected image is a real-time image of the billiard tabletop. After being recognized by the improved YOLOv8 network, the center point coordinates, length, width of each billiard detection frame and the category of the billiard ball in each detection frame are returned.
[0021] Preferably, in step S4, the billiard ball goal moment is determined by the following method:
[0022] S4.1. Based on the improved YOLOv8 network structure recognition results, calculate the center coordinates of the billiard ball, compare them with the coordinates of the six holes on the billiard table, calculate the distance, and the minimum distance is recorded as ;
[0023] S4.2. Taking the hole as the center, divide the area around the hole into two areas. The larger area is area L, with a radius of , the smaller area is area S, and the radius is ,radius ;
[0024] S4.3. When the billiard ball is not in area L or area S, set ; When the ball is in area L, record the distance from the billiard ball to the hole ; When the ball is in area S, record the distance from the ball to the hole ;in, and Respectively represent the distance from the billiard ball to the hole in areas L and S;
[0025] S4.4. When the billiard ball moves from a distance to the hole, it first enters area L and then area S. When the ball enters area L, recording begins. , when the ball enters area S, record , after the ball falls into the pocket, and Zero at the same time;
[0026] S4.5. According to and The numerical value changes to calculate the goal moment.
[0027] The technical solution of the present invention also provides: an electronic device, comprising:
[0028] one or more processors;
[0029] a storage device having one or more programs stored thereon;
[0030] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the above-mentioned billiard ball recognition methods based on deep learning.
[0031] The technical solution of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps in any of the above-mentioned billiard ball recognition methods based on deep learning are implemented.
[0032] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:
[0033] 1. Compared with the original YOLOv8 target detection network, the improved YOLOv8 network proposed in the present invention can realize accurate detection of billiard targets under complex backgrounds, and has lower computational complexity and parameter quantity. First, the present invention uses the SPD-Conv module to replace the Conv module of the backbone network, which reduces the loss of fine-grained information, enhances the ability to extract shallow information, and adds the Transformer attention mechanism, which effectively improves the network's ability to extract features of small targets. Secondly, the AGBiFPN feature fusion module is added to the neck network to further enhance the network's multi-layer feature fusion capability. Finally, NWD Loss is used instead of the loss function of YOLOv8 to improve the accuracy of label assignment.
[0034] 2. The present invention collects billiard pictures under complex lighting conditions and complex background conditions as training data, uses data enhancement to obtain more comprehensive and higher-quality data, and achieves high-accuracy billiard recognition through deep learning methods, with higher precision and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 It is a flowchart of the steps of the billiard ball recognition method based on deep learning of the present invention;
[0036] Figure 2 Improved YOLOv8 neural network model structure diagram for the present invention;
[0037] Figure 3 This is a structural diagram of the SPD-Conv module of the present invention;
[0038] Figure 4 This is the structure diagram of the Transformer module of the present invention;
[0039] Figure 5 This is a structural diagram of the AGBiFPN module of the present invention;
[0040] Figure 6 This is a schematic diagram of billiard ball recognition results according to an embodiment of the present invention;
[0041] Figure 7 Schematic diagram of goal determination according to an embodiment of the present invention. DETAILED DESCRIPTION
[0042] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the application is further elaborated in detail below in conjunction with the accompanying drawings. The described embodiments are only a part of the embodiments involved in the present invention. All non-innovative embodiments of other researchers in the field on this embodiment belong to the protection scope of the present invention. At the same time, for the step numbering in the embodiment of the present invention, it is only set for the convenience of explanation, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.
[0043] The invention discloses a billiard ball recognition method based on deep learning. The method collects billiard ball table video through a camera, uses a YOLO (You Only Look Once) model to perform high-precision recognition, and finally determines the goal moment to automatically generate a high-quality wonderful moment video.
[0044] In one embodiment of the present invention, a billiard ball recognition method based on deep learning is as follows: Figure 1 As shown, the steps are as follows:
[0045] S1: Collect billiard table images in real environment, manually annotate 16 balls of different colors, build billiard dataset, and use Mosaic data enhancement method to enhance the dataset;
[0046] S2: Improve the YOLOv8 network structure as the baseline network, such as Figure 2 As shown:
[0047] For the backbone network, an SPD-Conv (Space-to-Depth Convolution) module is added before each layer of the C2f module, and the Transformer attention mechanism is introduced at the bottom; for the neck network, the feature fusion module AGBiFPN (Attention Guided Bidirectional Feature Pyramid Network) is introduced; for the head network, a small target detection head P2 is added, and NWD Loss (Normalized Wasserstein Distance Loss) is used instead of the original loss function.
[0048] The improved YOLOv8 network structure is used to train the billiards dataset to obtain a deep learning model for billiards recognition.
[0049] S3: Use a camera to collect real-time billiard tabletop images, and input the images into the above-mentioned billiard recognition model for detection;
[0050] S4: Based on the improved YOLOv8 network detection results and combined with billiard motion characteristics, the billiard goal moment is determined.
[0051] Preferably, the pictures collected in step S1 of this embodiment are billiard pictures in different scenes, different angles and different illuminations, and are manually labeled using the labelImg labeling tool.
[0052] The data enhancement methods used in step S1 include: rotation, translation, flipping, cropping, scaling, Gaussian blur, color gamut change, Mosaic, etc.
[0053] Among them, Gaussian blur is used to generate images with different blur levels, thereby improving the detection accuracy of blurred data. Color gamut change is used to generate images with different colors, including adjusting brightness, saturation, contrast, white balance, etc. Mosaic selects four random images, retains the corresponding anchor frame for each image, and splices the four images together to obtain a new image to enrich the background of billiard recognition.
[0054] Furthermore, the billiards data set is divided. In this embodiment, 60% of the data is used as a training set, 20% of the data is used as a validation set, and 20% of the data is used as a test set.
[0055] Preferably, the SPD-Conv module in step S2 of this embodiment is an improvement on the traditional Conv layer, and is composed of an SPD layer and a non-strided convolution layer, which can reduce the loss of small target information and improve the accuracy of feature extraction. The SPD-Conv module structure is as follows: Figure 3 shown.
[0056] The SPD component generalizes the image conversion technique to downsampling feature maps within and across CNNs, considering any size The original feature map , S represents the length and width of the graph, Represents the channel of the graph, cutting out a series of sub-feature graphs , x , y Indicates the relative coordinates of the components of the sub-feature map in the original feature map, Depend on X Part of Composition, of which represents the downsampling scale factor, Starting from 0, all and integer;
[0057] ;
[0058] Each sub-feature map is scaled by a factor rightX Downsampling:
[0059] ;
[0060] Next, these sub-feature maps are connected along the channel dimension to obtain an intermediate feature map , The spatial dimension is reduced by a factor of , the channel dimension is increased by a scaling factor , the original feature map Convert to an intermediate feature map:
[0061] ;
[0062] After the SPD feature transformation layer, a non-strided convolution layer with a C2 filter is added. The use of non-strided convolution can preserve all the discriminative feature information as much as possible and further transform it:
[0063]
[0064] in, is the non-strided convolutional layer filter, , Represents the final feature map after SPD-Conv.
[0065] Furthermore, this embodiment introduces the Transformer module at the bottom of the backbone network as a method to enhance the feature graph representation capability, such as Figure 4 As shown in the figure, the Transformer module contains two sub-layers: multi-head attention layer and MLP.
[0066] Multi-head attention helps the current node not only focus on the current pixel, but also obtain contextual semantics. At the same time, since the final network resolution of the backbone network is low, applying Transformer to its feature map can reduce computational and memory costs.
[0067] Furthermore, the AGBiFPN module in step S2 aims to fuse multi-scale features to prevent the loss of semantic details related to small targets, so that the model can better detect targets.
[0068] like Figure 5As shown in the figure, BiFPN introduces bidirectional feature propagation, i.e., top-down and bottom-up feature flow, which enables more comprehensive and rich information transfer and fusion between different levels. A dynamic sparse attention mechanism is introduced on top of BiFPN to extract information from very small target layers, directly skipping the calculation of the structure of the least relevant area to efficiently generate target features with strong discriminative ability. Then, a top-down path is added to transfer high-level semantic feature information to guide subsequent network modules to perform feature fusion and generate features with stronger discriminative ability.
[0069] Furthermore, in step S2 of this embodiment, NWD Loss is used instead of the loss function of YOLOv8. Because the area occupied by the billiard ball in the image is very small, the IoU (Intersection over Union) indicator uses the overlap area of the anchor to evaluate the result, which is very sensitive to the position deviation of small targets. A slight position deviation will cause a huge change in the estimation result, resulting in inaccurate label assignment and seriously reducing the detection performance.
[0070] NWD Loss models the BBox (Bounding Box) as a two-dimensional Gaussian distribution, and then uses the Gaussian distribution of the Normalized Wasserstein distance to calculate the similarity between them.
[0071] definition and is the bounding box and Gaussian distribution, yes and The second-order Wasserstein distance between them introduces the NWD Loss as follows:
[0072]
[0073]
[0074]
[0075] Where C is a constant, Respectively represent the center coordinates of BBox, Respectively represent the length and width of the BBox, and the superscript T represents transposition. represents the normalized Wasserstein distance, Indicates NWD Loss.
[0076] Then, the training set is input into the improved YOLOv8 network structure to train the target detection model and obtain a deep learning model for billiards recognition. The verification set is input into the deep learning model for verification. According to the verification results, the deep learning model and training parameters are optimized to obtain the trained deep learning model for billiards recognition.
[0077] Collect real-time images of the billiard table, and after improved YOLOv8 network recognition, return the center point coordinates, length, width, and category of each billiard ball detection frame, such as Figure 6 shown.
[0078] Finally, according to the detected position of the billiard ball, the goal time is determined. According to the detection results of the improved YOLOv8 network, the center coordinates of the billiard ball are calculated, and compared with the coordinates of the six holes of the billiard table, the distance is calculated. The smallest distance is recorded as .
[0079] Taking the hole as the center, the area near the hole is divided into two areas: Figure 7 As shown, the larger area is area L, the smaller area is area S, and its radius .
[0080] When the billiard ball is not in either of the two regions, set , when the ball is in area L, record the distance from the billiard ball to the hole , when the ball is in area S, record the distance from the ball to the hole ,in and Respectively represent the distance from the billiard ball to the hole in areas L and S.
[0081] When the ball moves from a distance to the hole, it first enters area L and then area S. When it enters area L, it starts recording , the ball enters area S, and recording begins , after the billiard ball falls into the pocket, and Zero at the same time.
[0082] according to and The moment of goal is calculated based on the numerical change of .
[0083] Furthermore, in this embodiment, Anaconda is used to build a virtual environment for training the model on a GPU server, and the training set is input into the improved YOLOv8 network structure for target detection model training. After the training is completed, a deep learning model for billiard recognition is obtained, and the verification set is input into the deep learning model for billiard recognition for verification. According to the results obtained by the verification, the deep learning model and training parameters are optimized, and finally the deep learning model for billiard recognition with the best effect is obtained.
[0084] Preferably, the model training parameters of this embodiment are set as follows: the initial learning rate is 0.01, the momentum is set to 0.937, the weight decay is set to 0.0005, the training threshold is 0.2, the input image size is normalized to 640×640, the batch size is 32, the number of iterations is 2000, the number of early stopping waits is set to 100, and the remaining parameters remain the default settings of YOLOv8.
[0085] After verification, the improved YOLOv8 network proposed in this embodiment can achieve accurate detection of billiard targets under complex backgrounds, and has lower calculation amount and parameter amount, compared with the original YOLOv8 target detection network. The specific results are compared in the following table:
[0086]
[0087] As can be seen from the table, this embodiment collects billiard pictures under complex lighting conditions and complex background conditions as training data, and uses data enhancement to obtain more comprehensive and higher-quality data. The deep learning method is used to achieve high-accuracy billiard recognition, which has higher precision and accuracy than the existing methods.
[0088] In summary, the billiard ball recognition method based on deep learning of the present invention uses the SPD-Conv module to replace the Conv module of the backbone network, reduces the loss of fine-grained information, enhances the ability to extract shallow information, and adds the Transformer attention mechanism to effectively improve the network's ability to extract features of small targets; secondly, an AGBiFPN feature fusion module is added to the neck network to further enhance the network's multi-layer feature fusion capability; finally, NWD Loss is used instead of the loss function of YOLOv8 to improve the accuracy of label assignment.
[0089] Furthermore, the disclosure of the present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.
[0090] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.
[0091] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.
[0092] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Python, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect through the Internet). In some embodiments, by using the state information of the computer-readable program instructions to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit may execute the computer-readable program instructions, thereby implementing various aspects of the present disclosure.
[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.
Claims
1. A billiard ball recognition method based on deep learning, It is characterized in that The steps include: S1. Collect billiard table pictures in a real environment, mark balls of different colors, establish a billiard dataset, perform data enhancement on the billiard dataset, and divide the enhanced billiard dataset into training set, validation set and test set; The collected billiard tabletop images include real-life billiard images in different scenes, angles, and light intensities. The labelImg annotation tool is used to manually annotate balls of different colors, establish a billiard dataset, and use the Mosaic data enhancement method to enhance the data of the billiard dataset. S2. Improve the YOLOv8 network structure as the baseline network: For the backbone network, add an SPD-Conv module before each layer of the C2f module; introduce the Transformer attention mechanism at the bottom of the backbone network to enhance the feature map representation capability. The Transformer attention mechanism includes two sublayers: a multi-head attention layer and an MLP. The multi-head attention layer helps the current node to obtain contextual semantics based on the current pixel; For the neck network, introduce the feature fusion module AGBiFPN to fuse multi-scale features to prevent the loss of semantic details related to small targets; The feature fusion module AGBiFPN introduces a dynamic sparse attention mechanism on top of BiFPN to extract tiny information of the target layer, skip calculating the structure of irrelevant areas, and generate target features; A top-down path is added to the neck network to transfer high-level semantic feature information, guiding the subsequent network modules to perform feature fusion and generate features with stronger discrimination ability. For the head network, a small target detection head P2 is added, and NWDLoss is used instead of the loss function of YOLOv8. The billiards dataset is trained using the improved YOLOv8 network structure to obtain a deep learning model for billiards recognition. S3, using a camera to collect real-time billiard table images, inputting the billiard recognition deep learning model for detection, and obtaining the center point coordinates, length, width of each billiard detection frame and the category of the billiard ball in each detection frame after improved YOLOv8 network structure recognition; S4, according to the detection result of step S3, combined with the billiards motion characteristics, determine the billiards goal moment, the method is as follows: S4.
1. Based on the improved YOLOv8 network structure recognition results, calculate the center coordinates of the billiard ball, compare them with the coordinates of the six holes on the billiard table, calculate the distance, and the minimum distance is recorded as ; S4.
2. Taking the hole as the center, divide the area around the hole into two areas. The larger area is area L, with a radius of , the smaller area is area S, and the radius is ,radius ; S4.
3. When the billiard ball is not in area L or area S, set ; When the ball is in area L, record the distance from the billiard ball to the hole ; When the ball is in area S, record the distance from the ball to the hole ; in, and Respectively represent the distance from the billiard ball to the hole in areas L and S; S4.
4. When the billiard ball moves from a distance to the hole, it first enters area L and then area S. When the ball enters area L, recording begins. , when the ball enters area S, record , after the ball falls into the pocket, and Zero at the same time; S4.
5. According to and The change in value determines the goal frame and calculates the goal moment in this game.
2. The billiard ball recognition method based on deep learning according to claim 1, Features: In step S1, the data enhancement method includes: rotation, translation, flipping, cropping, scaling, Gaussian blur, color gamut change, and Mosaic; Among them, the Gaussian blur is used to generate pictures with different blurriness to improve the accuracy of blur data detection; the color gamut change is used to generate pictures with different colors, including: adjusting brightness, saturation, contrast, and white balance; the Mosaic selects four random pictures, each picture retains the corresponding anchor frame, and splices the four pictures together to obtain a new picture to enrich the billiard recognition background.
3. The billiard ball recognition method based on deep learning according to claim 1, Features: In step S2, the YOLOv8 network structure is used as the benchmark network. For the backbone network, an SPD-Conv module is added before each C2f module to reduce the information loss of small targets and improve the accuracy of feature extraction. The SPD-Conv module consists of an SPD layer and a non-strided convolution layer, and downsamples the feature maps inside and throughout the CNN as follows: For the size The original feature map , S represents the length and width of the graph, Represents the channel of the graph, cutting out a series of sub-feature graphs , x , y Indicates the relative coordinates of the components of the sub-feature map in the original feature map, Depend on X Part of Composition, of which represents the downsampling scale factor, Starting from 0, all and integer; ; Each sub-feature map is scaled by a factor right X Downsampling: ; Connect the sub-feature maps along the channel dimension to obtain the intermediate feature map , The spatial dimension is reduced by a factor of , The channel dimension is increased by a scaling factor , the original feature map Convert to an intermediate feature map: ; After the SPD feature transformation layer, add a The non-strided convolutional layer of the filter retains all the discriminative feature information and is further transformed as follows: ; in, is the non-strided convolutional layer filter, , Represents the final feature map after SPD-Conv.
4. The billiard ball recognition method based on deep learning according to claim 3, Features: In step S2, the BiFPN introduces bidirectional feature propagation, including top-down and bottom-up feature flows, to transfer and fuse information between different levels.
5. The billiard ball recognition method based on deep learning according to claim 4, Features: In step S2, for the head network, NWD Loss is used instead of the loss function of YOLOv8, and IoU uses the overlapping area of BBox to calculate the similarity; the NWD Loss models the BBox as a two-dimensional Gaussian distribution, and uses the Gaussian distribution of the Normalized Wasserstein distance to calculate the similarity, as follows: ; ; ; in, and is the bounding box and Gaussian distribution, yes and The second-order Wasserstein distance between ; C is a constant, Respectively represent the center coordinates of BBox, Respectively represent the length and width of the BBox, and the superscript T represents transposition. represents the normalized Wasserstein distance, Indicates NWD Loss.
6. An electronic device, It is characterized in that include: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the billiard ball recognition method based on deep learning as described in any one of claims 1 to 5.
7. A computer-readable storage medium, It is characterized in that A computer program is stored thereon, and when the program is executed by a processor, the steps in the billiard ball recognition method based on deep learning described in any one of claims 1 to 5 are implemented.
