Machine learning driven object detection system and method

Abstract

Embodiments of the present application disclose systems and methods for game monitoring. In particular, embodiments of the present application relate to systems and methods for game monitoring based on a machine learning process configured to analyze captured images to identify or detect game objects and game activities to monitor the game.

Description

Technical Field

[0001] The present application relates to a system and method for machine learning-driven object detection, and also relates to applying such object detection to monitoring table games, and more particularly to a system and method for monitoring events in table games in a gaming venue. Background Art

[0002] Systems used to monitor and manage table games have proven expensive to install and maintain and have failed to achieve a truly useful level of accuracy. Other options, including sensors in the casino's tokens and other offline yield management solutions, have proven ineffective and expensive to implement. Casino operators' reliance on random sampling often provides an inaccurate picture of activity and placement levels within the casino and can be difficult to record and report. Casino operating environments are fast-paced, with a high level of visual and auditory noise and distractions, a potential lack of order in the placement of cards and tokens on the tables, and drastic changes in lighting conditions.

[0003] The present application is intended to address or improve one or more shortcomings or deficiencies associated with machine learning-driven object detection in the prior art, or at least provide a useful alternative.

[0004] Throughout the specification, the word "comprise" or other variations of tense will be understood to imply the inclusion of stated elements, integers or steps, or groups of elements, plural integers or plural steps, but not the exclusion of any other elements, integers or steps, or groups of elements, plural integers or plural steps.

[0005] In this specification, a statement that an element may be “at least one” of a list of options should be understood to mean that the element may be any one of the listed options, or may be any combination of two or more of the listed options.

[0006] Any discussion of documents, acts, materials, devices, articles or the like included in this specification is not to be construed as an admission that any or all of these matters form part of the prior art base or were common general knowledge in the field relevant to the present application as it existed before the priority date of each claim of this application. Summary of the Invention

[0007] The present application discloses a game monitoring system, including:

[0008] at least one camera configured to capture an image of the gaming surface; and

[0009] A computing device in communication with the at least one camera, the computing device configured to analyze captured images of the gaming surface to automatically apply a machine learning process to identify gaming objects, gaming activities, and players in the captured images.

[0010] The machine learning process may be implemented using one or more neural networks. The one or more neural networks may include one or more deep neural networks. The one or more deep neural networks may include one or more convolutional neural networks. The one or more neural networks may include a faster region-based convolutional neural network. At least one of the one or more convolutional neural networks may include a region generation network. At least one of the one or more convolutional neural networks may include an object detection network.

[0011] The at least one camera is configured to capture high-resolution images.

[0012] Game objects may include playing cards or place markers.Game objects may include one or more stacks of one or more placement objects.

[0013] The computing device is further configured to automatically identify and predict the value of each stack of one or more placed objects by: identifying one or more first regions of interest associated with a game object in the captured image using a trained first region generation network; identifying a first region of interest subset associated with a single stack of one or more placed objects using a trained first object detection network; identifying one or more second regions of interest in each identified first region of interest subset using a trained second region generation network, the one or more second regions of interest associated with portions of an edge pattern located on each of the placed objects that constitute part of the single stack of the one or more placed objects; identifying a value pattern in each of the one or more second regions of interest using a trained second object detection network; and predicting a total placement value of the single stack of the one or more placed objects in each of the first region of interest subsets using the identified value pattern and a lookup table.

[0014] The system further includes associating each of the one or more first areas of interest with a placement area identifier.

[0015] The computing device may be further configured to identify game starts and ends based on game start and end trigger configurations stored in a data store accessible to the computing device.

[0016] The present application discloses a method, including:

[0017] The neural network system is trained to process captured images of the gaming table, identify gaming placement objects in the captured images, and calculate values ​​for the identified placement objects in the captured images.

[0018] The present application discloses a method, including:

[0019] processing a captured image of the gaming table through the trained neural network to identify gaming placement objects in the captured image;

[0020] identifying a value associated with each game placement object identified in the captured image; and

[0021] A game placement value is determined based on the identification value of the identified game placement object.

[0022] The processing step may identify a gaming placement object in at least one of a plurality of unique placement areas on the gaming table. The determining step may include determining a total gaming placement for each unique placement area of ​​a gaming object identified in the captured image.

[0023] Embodiments of the present application disclose a method including using a trained neural network to identify placement object values ​​of placed objects in a captured image of a gaming table.

[0024] An embodiment of the present application relates to a computing device configured to execute a neural network system for game object recognition, and comprising:

[0025] at least one processor;

[0026] a memory accessible by the at least one processor storing code for executing:

[0027] A placement object region generation network (RPN) receives image data from a captured image of the gaming table, and

[0028] A placement object detection network receives an output of the placement object RPN;

[0029] The placed object detection network detects one or more placed objects in the captured image based on an output of the placed object detection network.

[0030] The computing device further comprises:

[0031] A Region Proposal Network (RPN) for receiving image data from a captured image of the gaming table;

[0032] a game table object detection network for receiving an output of the game table RPN;

[0033] wherein the gaming table object detection network detects one or more gaming objects in the captured image based on an output of the gaming table object detection network, wherein the one or more gaming objects are distinct from the one or more placed objects. The computing device of an embodiment of the present application is further configured to determine illumination of an indicator on a transaction device on the gaming surface.

[0034] The embodiment of the present application discloses that the at least one camera and the computing device are part of a smartphone.

[0035] An embodiment of the present application discloses that the one or more convolutional neural networks include a convolutional neural network for performing image segmentation to determine the outline of the game object in the captured image.

[0036] The embodiment of the present application discloses that the convolutional neural network used to perform image segmentation is Mask R-CNN.

[0037] The embodiment of the present application discloses that the one or more convolutional neural networks include a game object classifier neural network, which is configured to classify the game object into the determined outline. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a block diagram of a game monitoring system provided by an embodiment of the present application;

[0039] Figure 2 This is a schematic diagram of a system for automatic tabletop game recognition provided by an embodiment of the present application, which forms Figure 1 Part of the game monitoring system;

[0040] Figure 3 This is a schematic diagram of a system for automatic tabletop game recognition provided by an embodiment of the present application, which forms Figure 1 Part of the game monitoring system;

[0041] Figure 4 is a block diagram of a computing device provided in an embodiment of the present application;

[0042] Figure 5 This is a block diagram of a message proxy server provided in an embodiment of the present application;

[0043] Figure 6 This is a block diagram of a database server provided in an embodiment of the present application;

[0044] Figure 7 This is a flowchart of a method for detecting the start and end of a game provided by an embodiment of the present application;

[0045] Figure 8is a composite flow chart and block diagram illustrating the operation of a neural network module provided by an embodiment of the present application;

[0046] Figure 9 This is a flowchart of a method for training a neural network provided in an embodiment of the present application;

[0047] Figure 10 This is a flow chart of a method for object detection provided by an embodiment of the present application;

[0048] Figure 11 is an example image of a stack of Coins game objects;

[0049] Figure 12 is an example image showing a stack of gaming chip game objects with a region of interest detected in a placed object image;

[0050] Figure 13 is a flowchart of a process for detecting non-placed objects provided by an embodiment of the present application;

[0051] Figure 14 This is an image of a stack of detected placed objects provided by an embodiment of the present application. DETAILED DESCRIPTION

[0052] Embodiments of the present application generally relate to systems and methods for machine learning-driven object detection. Some embodiments apply such object detection to monitoring tabletop gaming. Some embodiments relate to systems and methods for monitoring events during tabletop gaming at a gaming venue. Embodiments of the present application relate to improvements and / or adaptations of the systems, methods, and techniques described in commonly owned International Patent Application No. PCT / AU2017 / 050452, filed May 16, 2017, the entire contents of which are incorporated herein by reference.

[0053] Game Monitoring System

[0054] Figure 1 1 is a block diagram of a game monitoring system 100 provided in an embodiment of the present application. System 100 may include multiple game monitoring setups 105, a game monitoring infrastructure 115, and a database client 180. Game monitoring setups 105 include a gaming environment 110, a camera 120, and a computing device 130. System 100 is suitable for installation and operation in one or more gaming rooms of a gaming venue, such as a casino. Each gaming room has one or more gaming tables located therein, and some or each of these tables may constitute part of a corresponding game monitoring setup 105.

[0055] A gaming venue may have multiple gaming environments, such as areas or rooms where tabletop games are played, and to monitor each of those gaming environments, there may be multiple game monitoring setups 105. Multiple game monitoring setups 105 may be coupled to or linked to a common game monitoring infrastructure 115 using network links 147. Network links 147 may include a link 117 between a computing device 130 and a message broker server 140, and a link 167 between a computing device 130 and a neural network manager server 160. The game monitoring infrastructure 115 may also be coupled to or linked to game monitoring setups 105 in two or more different gaming venues. In some embodiments, a gaming venue may have a large number of gaming environments 110, and multiple game monitoring infrastructures 115 may be coupled to different subsets of the game monitoring setups 105 in the same venue.

[0056] The game monitoring infrastructure 115 includes a message broker server 140, a neural network manager server 160, and a database server 150. The message broker server 140 can be connected to multiple computing devices 130 via a bidirectional network link 117. A link 127 can be provided between the message broker server 140 and the database server 150 to facilitate the transfer of data or instructions. A network link 137 can be provided between the database server 150 and the neural network manager server 160. The computing devices 130 and the monitoring infrastructure 115 of the system 100 are separate computing systems, but are described herein as computing devices because they collaborate to perform various functions as described herein and form part of the same computer architecture of the system 100.

[0057] Each of server 140, server 150, and server 160 may be a standalone server or a different virtual server on one or more physical servers, or may be implemented as a cloud computing service. Each of server 140, server 150, and server 160 may also be implemented as a network of more than one server configured to handle higher performance or high availability requirements. Database client 180 may be an end-user computing device or an interface for relaying data to other end-user computing devices or other databases, and may be connected to database server 150 via network link 157.

[0058] Game Environment

[0059] The configuration of the gaming environment 110 may vary depending on the specific game being played, but most games monitored by any embodiment have some common elements. Figure 2 A portion of a game monitoring system 200 according to some embodiments is shown. The system can detect the start and end of a particular game, the placement of one or more stacks of objects or gaming chips, and the value of the objects or gaming chips placed in a single stack.

[0060] The gaming environment 110 includes a gaming surface or table 210 on which the game is played. The gaming surface 210 typically comprises a substantially horizontal plane upon which various gaming objects, such as cards 211 or gaming coins 213, or other objects detectable by the gaming monitoring system 100, may be placed. The camera 120 may be mounted on a support or pillar 220 at a height sufficient to position the camera 120 above any obstructions within the camera's field of view and tilted to direct the camera's field of view slightly downward toward the gaming table 210. Obstructions may be temporary, such as a dealer or participant playing at the table, or passersby. The camera 120 and computing device 130 may be positioned adjacent to other display screens located on the support or pillar on the gaming table 210.

[0061] Camera 120 is positioned to provide a better cross-sectional view of one or more stacks of placement objects while maintaining a reasonable viewing angle for detecting the playing surface and the player's cards. An example of a perspective view of camera 120 is the image frame shown in FIG. In the image frame, the placement objects and playing cards are visible across the entire playing surface. Also visible in the image frame are designated placement areas or placement zones on the playing surface where one or more placement objects may be placed according to the rules of the particular game being played on the gaming table.

[0062] Figure 3 A portion of a game monitoring system 300 according to some embodiments is shown. System 300 includes two cameras 120 and 320. Cameras 120 and 320 are mounted on opposite lateral ends / sides of gaming surface 210 so as to capture images of gaming table 210 and gaming objects on the gaming table from both lateral ends. Cameras 120 and 320 may have the same physical setup and / or the same configuration. Using two cameras 120 and 320 in system 100 can improve the accuracy of gaming object recognition by allowing for processing of separate sets of images of the entire gaming surface of gaming table 210 captured by the two cameras. Processing separate sets of images allows system 100 to more accurately account for inter-ship occlusion situations, where gaming chips or stacks of gaming chips are hidden from view by one camera but not the other.

[0063] In other embodiments, each camera can be configured to monitor approximately half of the gaming surface (or a portion of gaming table 210 that is less than the entirety of the gaming surface), thereby improving the performance or speed of object detection. In other embodiments, the game monitoring system can have more than two cameras. Camera 320 can be supported on a support structure 330, such as a pillar, column, gantry, wall, or other support. Camera 320 can be connected to computing device 130 via a communication link 310 that allows captured images to be transferred from camera 320 to computing device 130 and instructions to be transferred from computing device 130 to camera 320.

[0064] In some embodiments, computing device 130 may be in the form of a smartphone. Camera 120 may be embedded in computing device 130 in the form of a smartphone camera. Computing device 130 in the form of a smartphone should have the necessary hardware configuration to implement the various machine learning processes according to various embodiments. For example, a Google Pixel 2 phone or a phone with equivalent technical specifications can be used as a smartphone to provide computing device 130 and camera 120. When computing device 130 is implemented as a smartphone, a communication interface provided by the smartphone can be used to facilitate communication with neural network manager server 160 and message broker server 140. The communication interface used in the smartphone can be a cellular communication interface or a Wi-Fi communication interface provided by the smartphone. Using a smartphone as computing device 130 and camera 120 simplifies the implementation of game monitoring system 105 by utilizing off-the-shelf devices that can be configured to provide a portion of game monitoring system 105. A smartphone also simplifies the interface between camera 120 and computing device 130. In a smartphone, computing device 130 and camera 120 are part of a single physical unit and are pre-configured to communicate with each other. In some instances, more than one camera may be required in the game monitoring setup 105 , and two smartphones may be used to provide the two cameras 120 and 320 , with each camera having its own computing device 130 .

[0065] Game participants include players who may place bets and dealers who conduct the game. To place bets or conduct the game, players or dealers use objects described as game objects. Game objects can include cards 211 with specific shapes and specific markings to identify them, game coins or placement objects 213, or other such objects that can specify the number of bets a player can place during the game, or can include other objects with different shapes that can specify the outcome of the game, such as position markers or carts used in roulette. The game is played through a series of game events, including the start of the game, player placements during the game, intermediate results during the game, and the end of the game that determines the final outcome of the game. During the game, a player can place bets by placing his or her placement object 213 (i.e., placing game coins or game coins) in a placement area or placement area designated for placement. The game coins or placement objects can be arranged in groups or stacks in the placement area on the gaming surface 210. A single group or stack of placement objects can include a common color or denomination of placement objects (related to the bet value) or can include a combination of placement objects of two or more colors or denominations.

[0066] For example, camera 120 and camera 320 can be mounted at a distance of approximately 0 to 4 meters (optionally approximately 2 to 3 meters) from the near edge of the gaming table and can be elevated to approximately 0 to 3 meters (optionally approximately 1 to 2 meters) above the tabletop level. For example, camera 120 and camera 320 can be tilted downward at an angle of approximately 15 to 45 degrees from the horizontal. For example, camera 120 and camera 320 can be adapted to capture images at a high resolution, such as 720p resolution (images of up to 1280 x 720 pixels), 1080p resolution (images of up to 1920 x 1080 pixels), or 4k resolution (images of up to 4096 x 2160 pixels). For example, the camera can continuously capture images at a rate of 30 frames per second (fps), 60 fps, or 90 fps. The camera can transmit the captured images to computing device 130 via communication link 107, which can be in the form of a USB cable or a wireless communication link. An example of a suitable camera for each of camera 120 and camera 320 is the BRIO 4k webcam camera from Logitech.

[0067] computing device

[0068] The data generated by the camera 120 is received by the computing device 130 via a communication port 590. The port 590 may be in the form of a USB port or a wireless adapter that is coupled to the camera 120 to receive captured images or send commands to start or stop capturing images. The hardware components 510 of the computing device 130 include a memory 514, a processor 512, and other components necessary for the operation of the computing device. The memory 514 stores the necessary software modules 520, which include: an image processing library 522; a camera application program interface (API) 524; a runtime environment driver 526; a neural network module 528; a gaming activity detection module 532; and a message generator module 534.

[0069] The image processing library 522 is a set of routines for performing basic image processing operations, such as thresholding and warping operations on images and other routines necessary to pre-process the images before providing them as input to the neural network module 528. OpenCV is an example of an image processing library that can be used. The camera API 524 is a set of routines that enables the computing device 130 to establish a communication channel with one or more cameras 120. The camera API 424 enables data generated by the camera 120 to be received and processed by the neural network module 528.

[0070] The message generator module 534 generates messages based on instructions from the neural network module 528, which are passed to the message broker server 140. The message generator module can be based on a standard messaging system such as RabbitMQ or Kafka. Based on the message broker configuration 546 stored in the configuration module 540, the message generator module 534 can pass messages to the message broker server 140 via the communication port 590 and the network link 117. The configuration module 540 also includes a game start and end trigger configuration 544. The game start and end trigger configuration 544 includes details of a specific game event on a specific desktop, which specifies the start and end of the game. The components of the configuration module 540 can be stored in the memory 514 in the form of one or more configuration files. For example, the configuration file can be stored in XML format.

[0071] Message Broker Server

[0072] Message broker server 140 implements a message broker service and listens for messages from multiple computing devices 130 via network link 117. Message broker server 140 can be located in the same premises as computing device 130 within a public local area network, or can be located off-site (remote) but still communicate via network link 117 established between the two premises to facilitate the transmission of messages and data. Message broker server 140 can be centralized and connected to computing devices 130 in multiple gaming locations to provide centralized message broker services.

[0073] The message broker server 140 has hardware components 610, including memory 614, a processor 612, and other necessary hardware components for server operation. A message queue module 620 implements a queue to receive, interpret, and process messages from the plurality of configuration devices 130. Messages are received via a communication port 690, which may be in the form of a network adapter or other similar port that enables bidirectional transmission of data and instructions to and from the message broker server 140. The message queue module 620 may be implemented using a message broker package such as RabbitMQ or Kafka. Upon receiving a message containing transaction information regarding a gaming event occurring at a gaming table, the message queue module 620 initiates a database parsing module 630. The database parsing module 630 parses the message received by the message queue module 620 into a database query, which is primarily executed on the database server 150 via the network link 127.

[0074] Database server

[0075] The database server 150 receives game event data from the message broker server 140 and serves as a repository for the database client 180 to provide access to the game event data monitored by the game monitoring system 100. The database server 150 has a hardware component 710 that includes a memory 714, a processor 712, and other necessary hardware components for server operation. The communication port 790 can be in the form of a network adapter or other similar port that enables bidirectional transmission of data and instructions to and from the database server 150 via one or more network links. The database module 720 can be implemented using a database management system such as MySQL, Postgres, or Microsoft SQL Server.

[0076] Game event data 724 includes transaction data representing game events that occurred at a gaming table or table. Records comprising game event data may include a timestamp identifying the time of the game event; a unique identifier of the gaming table where the game event occurred; an identifier of the nature of the game event, such as a place; intermediate outcomes in the game; the final outcome of the game; an identifier of the place area associated with the game event; a prediction of the place value associated with the area of ​​interest; and other relevant attributes representing the game event.

[0077] The table configuration data 722 includes: a unique identifier for the gaming table and associated computing device 130; the nature of the game start and end triggering events, whether detecting the start of a game by placing a card on the gaming surface or placing a specific gaming object on a specific area of ​​interest; and other relevant data necessary to represent the parameters relied upon by the game monitoring system 100. In some embodiments, the table configuration data 722 and the game event data 724 may be stored in separate database servers to enable greater scalability and manageability of the game monitoring system 100.

[0078] Game activity detection

[0079] In some embodiments, camera 120 may be a high-resolution camera that can generate large amounts of data in real time. Storing and processing all of this data generated by a high-resolution camera may present significant challenges in terms of obtaining sufficient storage and processing capacity to store and process the acquired data. Furthermore, processing large amounts of data through deep neural networks may require significant processing by other processing cores or a graphics processing unit (GPU).

[0080] To address the aforementioned challenges, the game monitoring system is configured to detect the start and end of a game in the game environment 110 and capture high-resolution images only after the game begins. In some embodiments, the captured high-resolution images can be processed by the neural network module 528 in substantially real time to identify game objects and estimate the value of the placed objects. In other embodiments, the captured high-resolution images can be stored in the computing device 130 and processed by the neural network module 528 in a non-real-time manner.

[0081] Figure 7 Flowchart 800 illustrates a process for detecting the start and end of a game in some embodiments. Detection of the start and end of a game activity occurs in near real time within the gaming environment 110. Contour detection techniques can be used to detect the start and end of a game activity. Before the contour detection techniques can be applied, multiple image preprocessing steps are applied to the images captured by the camera 120. These image preprocessing steps improve the performance and accuracy of the process or algorithm implementing the contour detection techniques.

[0082] At step 810, camera 120 and camera 320 may acquire input image frames. The input images do not necessarily need to be high-resolution images. Some embodiments utilize a card detection process to determine the start and end times of a card game. This is useful, for example, for determining table utilization and dealer efficiency. Furthermore, storage and processing of high-resolution images can be avoided until a game is detected to have begun, and can be stopped once the game is determined to have ended, thereby improving computational efficiency of image processing. According to some embodiments, for example, high-resolution images include images with a resolution of 720 x 480, 1920 x 1080, or 3840 x 2160. According to some embodiments, high-resolution images include images with a resolution greater than 720 x 480, greater than 1920 x 1080, or greater than 3840 x 2160.

[0083] For embodiments employing the playing card detection process, one image pre-processing technique that may be employed is thresholding at step 820. One of several thresholding techniques, such as global thresholding or adaptive thresholding or binarization using the maximum inter-class variance method (also known as "Otsu's method"), may be employed to segment the image into a binary image having pixels representing either black or white portions of the input image.

[0084] Following the thresholding operation, a morphological transformation at step 830 can be applied to the output image of the thresholding operation. Morphological transformations enhance the features to be detected in the image and improve the performance and accuracy of the contour detection process. The erosion operation applied at step 832 and the dilation operation applied at step 834 are examples of morphological transformations that can be applied during the image preprocessing stage. Both the erosion and dilation processes require two inputs: image data in matrix form captured by camera 120 and a structuring element, or kernel, that determines the nature of the morphological operation performed on the input image. The kernel can be square or circular, has a defined center, and can be operated on by traversing the input image.

[0085] The morphological transformation of erosion involves sharpening foreground objects in an image by traversing the image using a kernel. The value of a single pixel is retained as 1 or a value corresponding to white only when all values ​​corresponding to the kernel are 1 or a value corresponding to white. Kernels of size 3x3, 5x5, or other sizes can be used to perform the erosion operation. The erosion operation erodes the boundaries of the foreground object. The erosion operation can be performed by a predefined library in the image processing library 522.

[0086] To implement erosion, a kernel is slid across the image (like a 2D convolution). A single pixel (1 or 0) in the original image is considered 1 only if all pixels under the kernel are 1, otherwise it is eroded (set to zero).

[0087] The process of dilation is the inverse of erosion. For example, in a dilation operation using a 3x3 square kernel, the pixel in the center of the kernel can be retained as a value of 1, or any value corresponding to white in the corresponding kernel, or a value corresponding to white. As a result of dilation, features in the image become more continuous and larger. The dilation operation can be performed by a predefined library in the image processing library 522.

[0088] Applying the thresholding technique to an image produces a binary image. To further enhance the features present in the image, morphological transformations of erosion and dilation are applied. The technical benefits of morphological transformations are that they help reduce noise from the image, isolate individual elements, and combine different elements into an image.

[0089] An image contour consists of a curve connecting all consecutive points along the boundary of an object represented in an image. Contours are a useful tool for shape analysis, as well as object detection and recognition. Contour approximation is used in applications to approximate the similarity of a specific shape to a desired shape. The desired shape can be, for example, a polygon, circle, or ellipse. For better accuracy and performance, contour detection can be performed on a binary image after edge detection has already been performed.

[0090] Edge detection, applied at step 840, is an image processing technique used to find object boundaries within an image. It involves detecting discontinuities in brightness within an input image. Among several edge detection techniques, Canny edge detection is a popular multi-stage edge detection algorithm or process that may be implemented by some embodiments.

[0091] Some or all of the steps of edge detection can be performed by routines available in the image processing library 522. For example, if the OpenCV library is used, the Canny edge detection function call can be used. Other alternative methods of edge detection can also be used as an alternative to the Canny edge detection to obtain the same result of edge identification in the input image.

[0092] After applying an edge detection operator to the input image to identify edges, the contour detection process of step 850 can be applied to the results of the edge detection operation to approximate the similarity of shapes in the image to a particular model shape, such as a polygon or circle.

[0093] Contour approximation approximates the contour shape to another shape (polygon) with a smaller number of vertices, according to the accuracy specified in the embodiment.Some embodiments may implement the Douglas-Peak algorithm for contour approximation.

[0094] The contour approximation operation can be performed by calling pre-packaged functions in the image processing library 522 in the game monitoring module 928. For example, if OpenCV is used to implement the contour approximation process, the function "findContours" or "drawContours" or "approxPolyDP" can be called to implement the process.

[0095] In some embodiments, the start of a game can be detected at step 860 by detecting the first presence of a card on the gaming table. To detect the presence of a card, the outline identified at the conclusion of the outline approximation step is analyzed. This analysis includes calculating the area of ​​the outline, identifying the number of vertices, and the angle formed by the outline's edges. To identify a card, in some embodiments, the following criteria may be applied: an outline area of ​​40 to 70 square centimeters or 50 to 60 square centimeters; four vertices after approximation; and an angle close to 90 degrees. If one or more of the identified outlines is identified as a card, the gaming activity detection module 532 signals the start of a game. The signal identifying the start of a game can be used by the game monitoring system 100 as a trigger to initiate the capture and storage of high-resolution images by the camera 120. These criteria may be specifically calibrated or adjusted based on the angle or position of the camera 120 relative to the gaming surface or the nature of the cards being used.

[0096] The specific properties of the events that define the game start and end triggers can be stored in the game start and end trigger configuration 544 and used as a reference by the game activity detection module 532 to predict whether the game on the table has started or ended. For example, for a table designated for a card game, the presence of one or more cards in the image frame can be considered to indicate the start of the game. Similarly, after the game starts, at step 880, the game activity detection module 532 can consider the absence of any cards in the image frame to indicate the end of the game. For games that are not based on cards, such as roulette, other game objects such as carts can be used as game start and end triggers. The specific shape and properties of the game objects that initiate the game start or end triggers can be stored in the game start and end trigger configuration 544 of the configuration module 540 of the computing device 130.

[0097] Once the gaming activity detection module 532 identifies the start of gaming and the camera 120 begins capturing high-resolution images at step 870, the neural network module 528 can use them for object detection and placement value estimation processes. However, before the neural network module 528 can accurately perform these operations, the neural networks undergo necessary training in order to calibrate, configure, or weight them to optimally perform the desired operations based on the specific gaming environment.

[0098] Neural Network Module

[0099] To detect game objects on the gaming table and predict the value of placed objects, the game monitoring system 100 relies on a trained machine learning process to perform its functions. In some embodiments, the machine learning process can employ one or more neural networks. In some embodiments, the one or more neural networks can include one or more deep learning neural networks. In some embodiments, the one or more deep learning neural networks can be convolutional neural networks (CNNs).

[0100] A CNN as implemented in some embodiments may include multiple layers of neurons whose structures and operations may differ from one another. The first layer of a CNN may be a convolutional layer of neurons. The convolutional layer of neurons performs the function of extracting features from an input image while preserving the spatial relationships between pixels of the input image. The output of the convolution operation may include a feature map of the input image. The convolution operation is performed using a filter or kernel matrix, and the weights in the filter or kernel matrix are obtained or calibrated by the process of training the CNN described later.

[0101] After the convolution layer, in some embodiments, the CNN implements a pooling layer or a rectified linear unit (ReLU) layer or both. The pooling layer reduces the dimensionality of each feature map while retaining the most important feature information. The ReLU operation introduces nonlinearity in the CNN because most real data to be learned from the input image is nonlinear. The CNN may include multiple convolution, ReLU and pooling layers, where the output of the previous pooling layer may be provided as input to the subsequent convolution layer. Having a large number of neuron layers is the reason why CNN is described as a deep learning algorithm or technique. The last layer or the last multiple layers of the CNN may be a traditional multilayer perceptron neural network, which uses the high-level features extracted by the convolution layer and the pooling layer to produce the output. The design of the CNN is inspired by the patterns and connectivity of neurons in the visual cortex of animals. The basis of the CNN design is one of the reasons why CNN may be chosen to perform the function of object detection in images.

[0102] The neural network module 428 can be in the form of a convolutional neural network, such as a region-based convolutional neural network (R-CNN) or a faster region-based convolutional neural network (Faster R-CNN). Some embodiments may use Resnet-101 or a single shot detector (SSD) as the basic feature extractor of Faster R-CNN. The neural network module 428 can be based on other deep learning methods or other machine learning methods. The following section of this application describes object detection and training methods for some embodiments based on the Faster R-CNN neural network training process, but should not be understood as limiting the application of other suitable machine learning or deep learning methods to other embodiments.

[0103] Figure 8 Flowchart 900 in FIG. 1 illustrates at a high level the operation of the neural network module 528 according to some embodiments. The neural network module 528 trains and maintains a table region profiling network (RPN) 920, a table object detection network 930, a placed object RPN 940, and a placed object detection network 950. The object detection and placement value prediction processes or methods described herein can be performed using a neural network model trained according to the Faster R-CNN training process. This training process includes two high-level steps or modules. The first step involves identifying or isolating various proposed regions of interest in the input image using a sufficiently deep convolutional neural network (also referred to herein as a region profiling network (RPN)). This first step is performed by the table RPN 920 and the placed object RPN 940. The second step involves using the proposed regions of interest identified by the RPNs to perform object detection based on the Faster R-CNN object detector. This step is performed by the table object detection network 930 and the placed object detection network 950.

[0104] RPN 920 and RPN 940 can take an image as input and produce one or more object proposals as output. Each object proposal can include coordinates on the image that define the rectangular boundaries of the region of interest and the detected object, and an associated object score that reflects the likelihood that a class of objects may be present in the region of interest. The object class may include playing cards, placed objects, or players, or other objects of interest that the RPN may be trained to detect.

[0105] The regions of interest identified by the RPN in object proposals may overlap, or one region of interest may be completely surrounded by another region of interest. The regions of interest can have varying aspect ratios to better approximate the shape of the object identified by the object proposal.

[0106] Training neural network modules

[0107] Before using RPN or Fast R-CNN to perform object detection on a gaming table or table, the neural network must be trained using a large training dataset. One or more of several known supervised training methods can be used to train the relevant neural network. The training dataset may include several images in which the boundaries of regions of interest and the identity of objects in each region of interest may have been manually identified and recorded. The boundaries of the region of interest can be recorded by the coordinates of four points of a rectangle that defines the region of interest.

[0108] Figure 9Flowchart 1000 illustrates a training technique or process for a CNN that may be implemented by some embodiments. The training method may be implemented to train an RPN such as the gaming table RPN 920 or the placed object RPN 940. The training method may be implemented to train an object detection network such as the gaming table object detection network 930 or the placed object detection network 950.

[0109] Examples of suitable hardware and software that can be used to perform method 1000 for training, testing, and running deep learning object detection are shown below.

[0110] hardware:

[0111] a.CPU is quad-core Intel i7 7700k.

[0112] b. Memory is 16GB.

[0113] C. The GPU is dual GTX 1080Ti, each with 12GB of memory.

[0114] Software: The Tensorflow framework can be used to complete the training and inference process.

[0115] a.Tensorboard: used to monitor network training and evaluation.

[0116] b. Models: Tensorflow open source community-driven model GitHub repository.

[0117] C. Pretrained Models: TensorFlow provides several pretrained models. These models are trained on large datasets with thousands of different categories, ranging from airplanes to dogs. Some popular datasets include MSCOCO and PascalVOC. This allows you to fine-tune the model to any task. This process is called transfer learning.

[0118] d. Google Cloud Engine: If all local resources are being used, you can deploy your training job on Google Cloud Engine.

[0119] The information required to train a Tensorflow Object Detection (OD) model (as an example model) is:

[0120] Image data;

[0121] The height, width, and depth of the image;

[0122] The object name (card, coin, person) and bounding box coordinates (xmin, ymin, xmax, ymax) in the image; and

[0123] Other parameters like hard objects, segmentation etc. can be used but are mainly used for database evaluation.

[0124] The PascalVOC format is a suitable XML format for packaging OD information for a single image.

[0125] As a first step 1010, in some embodiments, a CNN can be initialized using parameters or weights that can be randomly generated by drawing from a Gaussian distribution. Alternatively, in some embodiments, a previously trained CNN can be used to initiate training. As an example, to train the RPN, one or more ground truth regions or boxes can be identified in all training images 1020. The ground truth regions or boxes identify objects and their boundaries in the training images. The training images can be passed as input to the initialized RPN to obtain potential regions of interest as output.

[0126] Based on the output of the RPN or CNN, a loss function or error function may be calculated at step 1030. The output of the loss function may show the difference between the ground truth boxes or regions in the input image and the region proposals made by the RPN or CNN. The output of the loss function may be used at step 1040 to calculate a stochastic gradient descent with respect to the weights in the RPN or CNN. The error gradient may be backpropagated through the RPN or CNN to adjust the weights to minimize the calculated error function or loss function. This process may continue using multiple (large) input images until the training dataset is exhausted at step 1050.

[0127] Relying on the above-mentioned error or loss function and the principles of backpropagation and stochastic gradient descent, the RPN or CNN can be trained end-to-end to improve its accuracy by optimizing the error or loss function. After multiple training cycles with a large number of training data sets, the RPN or CNN can perform at an acceptable accuracy level and can then be incorporated into the neural network module 528.

[0128] The table object detection network 930 and the placed object detection network 950 can be trained according to the same principles identified with respect to the CNN or RPN, except that the object detection network 930 and the object detection network 950 accept the identified region of interest as input and output the probability of the presence of a class of objects in the object of interest region.

[0129] In addition, the object detection network 930 and the object detection network 950 can be trained in conjunction with the table RPN 920 and the placed object RPN 940 to allow the convolutional layers to be shared between the two networks, thereby improving the efficiency and accuracy of the neural network module 528. Joint training can include alternating training of the two networks, relying on the output of one network as the input of the other network. Another option for joint training can include merging the two networks to form a single network and relying on backpropagation and stochastic gradient distribution to change the weights of the entire network in each training iteration.

[0130] To prepare large datasets for training machine learning or neural network algorithms, regions of interest can be manually drawn or identified in images captured from the game at the gaming table. Regions of interest can be manually labeled with associated identifiers, such as placed objects, characters, cards, or other game objects, for example, by using an annotation or labeling tool as shown. An example of a suitable annotation tool is the "Labellmg" tool, accessible via GitHub, which provides annotation XML files for each file in the PascalVOC format. Additionally, additional parameters, such as those related to difficult or segmented objects, can be identified by manually labeling using the labeling tool relative to the regions of interest.

[0131] The image annotation process in the training dataset can be improved by leveraging the output produced by a previously trained neural network or a neural network trained on manually annotated images. The output produced by a previously trained neural network or a neural network trained on manually annotated images can be manually modified to correct any errors in the identification of regions of interest and the identities of objects within them. The corrected output can be used as input to the next iteration of neural network training to further improve the accuracy of the results. This feedback loop can be repeated using several different datasets to obtain a robust neural network that can reliably identify objects under varying conditions.

[0132] Further robustness to the training of the neural network can be achieved by applying data augmentation or other techniques to the training data, such as: randomly flipping the input image horizontally; randomly changing the brightness of the input image; randomly scaling the size of the training image by setting a scaling factor; converting random color images to grayscale; or randomly introducing jitter or variation in the object box size of the object input area used as RPN input. One or more of these data augmentation processes can be applied in training the game table RPN 920 and / or the object placement RPN 940.

[0133] Placement object value prediction

[0134] Figure 10Flowchart 1100 in FIGURE 1 illustrates an overall game object detection and placed object value prediction process or method, according to some embodiments. The method can be applied to images captured in near real time or stored from previously played games. A first step 1102 involves retrieving an image frame that can be used as input for the trained table RPN 920. At step 1106, the trained table RPN 920 identifies regions of interest in the input image. As an example, if an image frame is used as input, the image frame shown in the screenshot can be generated as output, displaying suggested regions of interest.

[0135] The table RPN 920 can only identify no more than a certain number of regions of interest. This constraint is necessary to maintain adequate performance of the table RPN 920 without over-configuring the computing device 130 in terms of processing power. In some embodiments, this constraint can be set to identify a maximum number of regions of interest in the range of, for example, 30 to 70. In other embodiments, this constraint can be set to identify a maximum of 40 to 60 regions of interest, for example.

[0136] Once the proposed regions of interest are identified, the portion of the image corresponding to each region of interest is provided as input to the table object detection network 930 at step 1110. The table object detection network 930 detects players within the regions of interest. Regions of interest are determined by the table object detection network 930 to contain card objects. Regions of interest are determined by the placed object detection network 950 to contain placed objects. Regions of interest containing placed objects proceed to step 1124 at step 1114. Regions of interest in which non-placed objects are detected proceed to step 1120, where the nature of the detected objects and their coordinates are recorded.

[0137] The placement object can comprise a stack of a game coin or multiple game coins. The stack of multiple game coins can include game coins of various denominations (that is, various associated game coin values). In most gaming venues, the denomination of game coins or the placement object is specified or identified with the color of the game coins, and is also specified or identified by the specific pattern on the game coin edge. The edge pattern on the game coin or the placement object can be symmetrically positioned at multiple (e.g., 4) different points around or around the circumference of the game coin or other placement objects. The edge pattern of the game coin of a specific value can include stripes or edge patterns of specific colors different from the color or edge pattern of the game coins of different values.

[0138] The systems and techniques described herein assume that gaming objects in the form of gaming coins will have a generally circular outline in plan view and a generally uniform depth or thickness in side view, such that each gaming coin resembles a short cylinder (or a cylinder whose height is much less than its diameter).

[0139] The game monitoring system 100, and in particular the neural network module 528, may be configured to detect or recognize specific edge patterns of gaming chips in a gaming venue in which the system 100 is to be deployed.

[0140] To predict the value of a stack of placed objects (coins) on a gaming table, it is necessary to identify the surface pattern of each individual coin, such as the pattern on the outer cylindrical (ring-shaped) edge. Furthermore, for the top coin in the stack, the top surface markings corresponding to or defining the coin's value can also be identified as one or more regions of interest (ROIs), allowing these regions to be used to verify the value of the coin determined based on the detected edge patterns. Top and edge pattern recognition can be accomplished by a placed object RPN 940 and a placed object detection network 950. The placed object RPN 940 and placed object detection network 950 can be initialized and trained using techniques similar to those used for the gaming table RPN 920 and gaming table object detection network 930. However, the placed object RPN 940 is trained in step 1124 to identify regions of interest in the placed object image frame, including regions of interest that cover or bound the edge pattern portion of each individual placed object in the stack, as well as regions of interest that cover or bound the surface of the top placed object in the stack.

[0141] Placed object RPN 940 proposes a region of interest for a portion of an edge pattern on a single placed object, and in step 1128, placed object detection network 950 identifies a value pattern for the proposed region of interest. This value pattern can be a value associated with a specific portion of the edge pattern on a placed object, whose boundaries are defined by the region of interest proposed by placed object RPN 940. Neural network module 528 can include a value pattern lookup table 1150 that records the denominations associated with specific value patterns detected by placed object detection network 950 (including the top surface markings of the top placed object). The contents of value pattern lookup table 1150 can vary between different gaming locations to reflect the different designs and edge patterns on placed objects or tokens in different locations. For example, image frame 1200 can be the input image for placed object RPN 940. Edge patterns 1210 and 1215 can reflect the value associated with a specific placed object that is part of a series of placed objects in image frame 1200. Figure 12 The output image produced by the object placement RPN 940 may have different regions of interest 1310 and 1315 defining part of the boundary of the edge pattern.

[0142] The region of interest identified by the placed object RPN 940 can only be rectangular, and the edges of the rectangle must be parallel to the edges of the input image. However, a placed object, such as a token, is somewhat circular (when placed on a table in an elevated and tilted position, e.g., from cameras 120 and / or 320), and if the entire edge pattern of a placed object is contained within the rectangular region of interest, the rectangular (defined) region of interest may include edge patterns of other vertically or horizontally adjacent placed objects. This can reduce the performance accuracy of the placed object detection network 950, as the isolation of the object to be identified within the proposed region of interest is crucial for accurate object detection. To overcome this issue, in step 1128, the placed object RPN 940 is trained to identify the ends of each visible edge pattern, rather than targeting the entire edge pattern of the placed object for object detection. For example, regions of interest 1310 and 1315 identified in image frame 1300 confine or cover only a portion of the edge pattern on the placed object. Such edge patterns are distinct, spaced around the circumference of the token, and separated by non-patterned edge regions. Thus, what is detected as an edge pattern may be a transition from a patterned area to a non-patterned area along the edge of the gaming chip.

[0143] In step 1132, the detected value pattern (which may be a portion of the edge pattern of the placed object and the value associated with the edge pattern) is compared to the values ​​in the value pattern lookup table 1150 to predict the value of each individual placed object in the single stack of placed objects. Processing performed by the neural network module 528 or another software module 520 sums the associated gaming coin values ​​for each detected value pattern to generate a value determination result. In step 1132, this result is used to predict the value of the entire stack or stacks of placed objects (gambling coins).

[0144] In step 1136, the neutral network module 528 checks whether the next image frame in the series represents the end of game play. If the end of game play is detected, then at step 1140, the stored observations regarding game objects, the number and / or positions of players, and the predicted value of placed objects are reported to the database server 150 via the message broker server 140. If the end of game play is not detected, the entire process 1100 continues by returning to step 1102 to process the next image frame.

[0145] When a trained neural network is deployed in computing device 130 via neural network module 528, it is not necessarily static or fixed. The deployed neural network can undergo subsequent training based on data recorded on database server 150, which is obtained through actual operation of game monitoring system 100. Neural network manager server 160 can access the game object detection data and placed object value prediction data in database server 150 via network link 137. Based on this data and additional correction data that can be provided to neural network manager server 160 over time, the deployed neural network module 528 can be further trained. If further training produces a neural network that outperforms the currently deployed set of neural networks, neural network manager server 160 can replace the deployed neural network with the better-performing neural network obtained after further training. This feedback can further improve the accuracy or performance of game monitoring system 100.

[0146] In some embodiments, the gaming environment 110 may include a device, such as a card delivery box or a card shuffling device or a card transaction device (232). Typically, the card delivery box or card shuffling device or transaction device includes a mechanism for verifying the authenticity of the cards being processed by the device. A verification mechanism may be in place to detect or prevent card swapping, that is, a player replacing real cards with counterfeit cards to influence the outcome of the game. The card delivery box or shuffling device or transaction device may also verify the dealer's dealing process by tracking the order in which the cards are dealt. The card delivery box or shuffling device may rely on a unique code printed on each card. This unique code is read by the card delivery box or shuffling device, and counterfeit cards detected during the dealing process are confiscated. The card delivery box or shuffling device typically includes an indicator light, such as an LED light (234). The illumination of the indicator light may indicate a discrepancy or unexpected card or result in the game. The indicator light is typically located on the upper portion of the card delivery box or shuffling device visible to the dealer and the general public in the gaming area.

[0147] In some embodiments, camera 120 can be positioned to capture the illumination of an indicator light located on a card delivery box or shuffling device. Computing device 130 can be configured to evaluate or determine the illumination of the indicator light to identify the occurrence of a discrepancy, as indicated by the cards or the delivery box of a shuffling or transaction device. The evaluation or determination can be based on a predetermined set or area of ​​pixels covering the signal in the image captured by camera 120. Computing device 130 can communicate and log the occurrence of a discrepancy via message broker 140, thereby enabling casino monitoring agencies to respond to the discrepancy.

[0148] Mask R-CNN is a convolutional neural network that provides a framework for segmenting or masking object instances. This segmentation or masking of object instances allows for the identification of objects and all pixels associated with the identified object in an image. Pixels identified by a trained Mask R-CNN do not necessarily have a predetermined rectangular shape. Pixels identified by a trained Mask R-CNN closely predict the outline of the identified object. One of the advantages of Mask R-CNN is its ability to more accurately identify overlapping objects.

[0149] Mask R-CNN has a similar structure to the Faster R-CNN described above. However, in addition to the structure included in Faster R-CNN, Mask R-CNN also includes a branch or mask branch that performs instance segmentation or masking and outputs a mask to identify whether each pixel in the input image is an identified region of interest or object. In some embodiments, the detection of the mask can occur in parallel with the identification of the object. The mask branch can include a separate fully convolutional neural network that is applied to each identified region of interest, thereby producing a segmentation mask at the pixel level in the form of a binary mask that identifies whether the pixel is part of the detected object.

[0150] Figure 13 Flowchart 1400 is shown for masking and detecting non-placed objects such as playing cards according to some embodiments. Flowchart 1400 includes Figure 10 14. In addition, flowchart 1400 includes steps 1102, 1106, 1110, and 1114 of flowchart 1100. In addition, flowchart 1400 includes steps for image segmentation and object detection of non-placed objects. In step 1410, the regions of interest identified in step 1110 are processed by a region of interest alignment neuron layer to improve the alignment of the boundaries of the identified regions of interest, thereby improving subsequent steps of image segmentation or mask processing. In step 1420, the aligned regions of interest are processed by the trained Mask R-CNN. After processing by the trained Mask R-CNN, an output is generated in step 1430 in the form of a binary segmentation mask for each non-placed object identified in step 1114. For each identified object, the output can be in the form of a binary segmentation mask, where each binary segmentation mask represents a set of pixels in the captured image that are associated with the identified object.

[0151] At step 1440, the region of the captured image corresponding to the identified binary segmentation mask is processed by the trained object classifier. The trained object classifier further classifies the identified object within each identified binary segmentation mask. For example, the object classifier may classify the object as the Ace of Spades within the identified binary segmentation mask. Information regarding the identified and classified object may be stored by computing device 130, or alternatively, computing device 130 may send this information to message broker server 140 via communication link 117. Information about the cards displayed on the playing surface allows for verification of past game events with recorded game outcomes.

[0152] The object classification of step 1440 may be performed using a capsule neural network or an initial neural network or a deep neural network trained using a residual learning framework.

[0153] Figure 14 Screenshot 1500 illustrates the results of placed object region detection in accordance with some embodiments. The placed object RPN 940 of some embodiments may be additionally trained to detect top placed objects 1520 and base regions 1530 for a placed object stack or region. For example, in screenshot 1500, the top placed object 1520 and placed object base region 1530 are detected by the placed object RPN 940. The placed object RPN 940 also detects edge patterns 1510 of placed objects that are part of a stack of placed objects. Top placed object 1520 and placed object base region 1530 serve as anchor points for the entire placed object detection process, thereby improving the accuracy and performance of the entire placed object detection process and providing a means for validating the results.

[0154] Those skilled in the art will appreciate that various changes and / or modifications may be made to the above embodiments without departing from the broad general scope of the present disclosure. Therefore, the present embodiments should be considered in all aspects as illustrative rather than restrictive.

Claims

1. A game monitoring system, comprising: at least one camera configured to capture an image of the playing surface; and a computing device in communication with the at least one camera, the computing device configured to analyze captured images of the gaming surface to automatically apply a machine learning process to identify gaming objects, gaming activities, and players in the captured images, The game object includes one or more stacks of one or more placement objects, The computing device is further configured to automatically identify and predict a value for each stack of one or more placement objects by: identifying one or more first regions of interest associated with a game object in the captured image using the trained first region generation network; identifying, using the trained first object detection network, a first subset of regions of interest within the one or more first regions of interest associated with a single stack of one or more placed objects; identifying, using the trained second region generation network, one or more second regions of interest in each of the identified subset of first regions of interest, the one or more second regions of interest being associated with portions of an edge pattern located on each of the placement objects comprising part of the single stack of the one or more placement objects; identifying a value pattern in each of the one or more second regions of interest using a trained second object detection network; as well as A total placement value for a single stack of the one or more placement objects in each of the first subset of regions of interest is predicted using the identified value pattern and a lookup table.

2. The system of claim 1, wherein the machine learning process is implemented via one or more neural networks.

3. The system of claim 2, wherein the one or more neural networks comprise one or more deep neural networks.

4. The system of claim 3, wherein the one or more deep neural networks comprise one or more convolutional neural networks.

5. The system of claim 4, wherein at least one of the one or more convolutional neural networks comprises a region generation network.

6. The system of claim 4, wherein at least one of the one or more convolutional neural networks comprises an object detection network.

7. The system of any one of claims 1 to 6, wherein the at least one camera is configured to capture high-resolution images.

8. The system of any one of claims 1 to 6, wherein the game objects include at least one of playing cards and position markers.

9. The system of claim 1, further comprising associating each of the one or more first areas of interest with a placement area identifier.

10. The system of any one of claims 1 to 6, wherein the computing device is configured to identify game starts and ends based on game start and end trigger configurations stored in a data store accessible to the computing device.

11. The system of claim 2, wherein the one or more neural networks comprise a faster region-based convolutional neural network.

12. The game monitoring system of any one of claims 1 to 6, wherein the computing device is further configured to determine illumination of an indicator on a transaction device on the gaming surface.

13. The game monitoring system of any one of claims 1 to 6, wherein the at least one camera and the computing device are part of a smartphone.

14. The game monitoring system of claim 4, wherein the one or more convolutional neural networks include a convolutional neural network for performing image segmentation to determine outlines of game objects in the captured image.

15. The game monitoring system of claim 14, wherein the convolutional neural network for performing image segmentation is Mask R-CNN.

16. The game monitoring system of claim 14, wherein the one or more convolutional neural networks include a game object classifier neural network configured to classify the game object as having a determined outline.

17. A method for monitoring a game, comprising: training a neural network system to: process a captured image of a gaming table, identify gaming objects in the captured image, and calculate a value for the identified gaming objects in the captured image, The game object includes one or more stacks of one or more placement objects, The method includes using a trained neural network to identifying one or more first regions of interest associated with a game object in the captured image using the trained first region generation network; identifying, using the trained first object detection network, a first subset of regions of interest within the one or more first regions of interest associated with a single stack of one or more placed objects; identifying, using the trained second region generation network, one or more second regions of interest in each of the identified subset of first regions of interest, the one or more second regions of interest being associated with portions of an edge pattern located on each of the placement objects comprising part of the single stack of the one or more placement objects; identifying a value pattern in each of the one or more second regions of interest using a trained second object detection network; as well as predicting a total placement value for a single stack of the one or more placement objects in each of the first subset of regions of interest using the identified value patterns and a lookup table, Thereby, a total placement value of a single stack of the one or more placement objects is automatically identified and predicted.

18. A method for monitoring a game, comprising: processing a captured image of the gaming table through the trained neural network to identify gaming objects in the captured image; identifying a value associated with each game object identified in the captured image; as well as Determining a game placement value based on the identification value of the identified game object, The game object includes one or more stacks of one or more placement objects, The method includes using a trained neural network to identifying one or more first regions of interest associated with a game object in the captured image using the trained first region generation network; identifying, using the trained first object detection network, a first subset of regions of interest within the one or more first regions of interest associated with a single stack of one or more placed objects; identifying, using the trained second region generation network, one or more second regions of interest in each of the identified subset of first regions of interest, the one or more second regions of interest being associated with portions of an edge pattern located on each of the placement objects comprising part of the single stack of the one or more placement objects; identifying a value pattern in each of the one or more second regions of interest using a trained second object detection network; as well as predicting a total placement value for a single stack of the one or more placement objects in each of the first subset of regions of interest using the identified value patterns and a lookup table, Thereby, a total placement value of a single stack of the one or more placement objects is automatically identified and predicted.

19. The method of claim 18, wherein the processing step comprises: A gaming object is identified in at least one of a plurality of unique placement areas on the gaming table.

20. The method of claim 19, wherein the determining step comprises: A total game placement value is determined for each unique placement area of ​​a game object identified in the captured image.

21. A game monitoring system comprising at least one computer processor and memory configured to perform the method of any one of claims 17 to 20.

22. A tangible computer-readable memory storing program code which, when executed by a computer system, causes the computer system to perform the method according to any one of claims 17 to 20.

Images