Ball tracking in a physical mini-golf game for computer visualisation

Abstract

A method and a system are provided for ball tracking in a physical mini-golf (crazy golf) game for computer visualisation. The method comprises obtaining synchronised camera data from multiple cameras positioned to capture movement of a ball on a course from a tee-off position to the hole cup. A ball is identified 305 and linked to a registered player at the tee-off position by capturing a repeating polygonal pattern upon the ball using pattern recognition. The method further comprises tracking the identified ball using the synchronised camera data to provide three-dimensional (3D) coordinates 307 and generating events (e.g. identification of legal shots 308 and determining when a ball is in the cup 309) relating to the ball during the movement on the course. The method provides the 3D coordinates of the ball tracking and the events to a computer visualisation system for display 310. The method may use irradiating the course hole with infrared light 302 to facilitate infrared image acquisition. The 3D coordinates for ball position may be provided by using a supplied 3D model of each course hole, or by using triangulation using two or more calibrated cameras. An associated system and computer program product for ball tracking is also described.

Description

FIELD OF THE INVENTION This invention relates to ball tracking in ball sports and, in particular, ball tracking in a physical mini-golf game for computer visualisation. BACKGROUND TO THE INVENTION Miniature golf, referred to herein as mini-golf, is an offshoot of the sport of golf that uses courses made up of short course holes from a tee-off area to a putting hole. Mini-golf focuses on the putting aspect. Due to the shorter course holes, mini-golf courses can be provided indoors or in areas of limited space and provide an entertainment activity for all levels of skill. Mini-golf providers, along with the leisure industry as a whole, have started to integrate technology to create a more immersive experience for their players. Players have the physical experience of playing a physical mini-golf game with the integration of technology to provide more tracking and scoring features. One example, is a mini-golf experience that leverages Radio Frequency Identification (RFID) technology to track shot data and use this to generate a digital scoring experience for customers. In another field of development, computer vision provides methods of acquiring, processing, analysing, and understanding digital images. This may involve extracting high-dimensional data from the real world in order to produce numerical or symbolic information. The preceding discussion of the background to the invention is intended only to facilitate an understanding of the present invention. It should be appreciated that the discussion is not an acknowledgment or admission that any of the material referred to was part of the common general knowledge in the art as at the priority date of the application. SUMMARY OF THE INVENTION According to an aspect of the present invention there is provided a method for ball tracking in a physical mini-golf game for computer visualisation, comprising: obtaining camera data from multiple cameras positioned to capture movement of a ball on a course hole from a tee-off position to a cup of a physical mini-golf game course; identifying a ball at the tee-off position by capturing a pattern by a camera with the pattern provided on the ball to link the ball to a registered player; tracking the identified ball using the camera data to provide three-dimensional coordinates of a movement of a ball; and generating events relating to the ball during the movement on the course hole from the tee-off position to the cup; wherein the method provides the three-dimensional coordinates of the ball tracking and the events to a computer visualisation system for display. Generating events may include: identifying legal shots of the identified ball by analysing ball velocity; and determining that the identified ball is in the cup. The method may include providing at least two overhead cameras on the course hole for ball movement capture and at least one close-up camera at the tee-off position for ball identification. The method may include shining infrared light source over the course hole to illuminate the course hole in the infrared spectrum and obtaining infrared camera data. Identifying legal shots may include distinguishing between legal and illegal shots by monitoring ball movement and tracking a golf club head, wherein a legal shot is where the golf club head is at the ball at the moment where the ball transitions from a stationary state to a moving state. Tracking the identified ball may include tracking in occluded areas in the course hole by extrapolating the tracking through the occluded area. Tracking the identified ball using the camera data to provide three-dimensional coordinates may include a first tracking method that uses a three-dimensional model of the course hole and at least one camera calibrated to the three-dimensional model coordinate system. Tracking the identified ball using the camera data to provide three-dimensional coordinates may include a second tracking method that calculates three-dimensional coordinates through triangulation using two more calibrated cameras and without use of a three-dimensional model of the course hole. The first and second tracking methods may be used independently or in a combined manner. The first and second tracking methods may be continually used in combination with the results of the first and second tracking methods combined. The method may include auto-calibrating the multiple cameras for the course hole using a cup as a calibrating feature of the course hole. The method may use machine learning modelling for object detection of a ball and a golf club in the camera data. The method may include training the machine learning modelling including: using training datasets of camera data with annotations identifying balls and clubs in the camera data. The method may include training the machine learning modelling using three-dimensional course hole models used by the computer visualisation system. Providing the three-dimensional coordinates of the ball tracking and the events to a computer visualisation system may use a publish / subscribe messaging system including queuing data and events to provide ordered delivery. The method may include receiving the three-dimensional coordinates of the ball tracking and the events at a computer visualisation system for: mapping the coordinates to a three-dimensional model of the course hole; identifying a ball as relating to a registered player; and scoring the events of the course hole for the registered player. The method may include providing course hole metadata at the computer visualisation system to be used to identify events in a game. According to another aspect of the present invention there is provided a system for ball tracking in a physical mini-golf game for computer visualisation, the system comprising, comprising: a ball tracking and event generating system including a memory for storing computer-readable program code and a processor for executing the computer-readable program code and configured to carrying out the steps of: obtaining synchronised camera data from multiple cameras positioned to capture movement of a ball on a course hole from a tee-off position to a cup of a physical mini-golf game course; identifying a ball at the tee-off position by capturing a pattern by a camera with the pattern provided repeatedly on the ball to link the ball to a registered player; tracking the identified ball using the synchronised camera data to provide three-dimensional coordinates of a movement of a ball; and generating events relating to the ball during the movement on the course hole from the tee-off position to the cup; and an adapter interface including a memory for storing computer-readable program code and a processor for executing the computer-readable program code and configured to provide the three-dimensional coordinates of the ball tracking and the events to a computer visualisation system for display. The system may include a physical camera configuration for providing at least two overhead cameras on the course hole for ball movement capture and at least one close-up camera at the tee-off position for ball identification. The physical camera configuration may include an infrared light source positioned over the course hole to illuminate the course hole in the infrared spectrum for obtaining infrared camera data. The system may include a mini-golf ball having a pattern provided repeatedly on the surface of the ball and the pattern configured to be captured by a camera regardless of the position of the ball in relation to the camera and providing an identifier to link the ball to a registered player. The pattern may be included in each of repeating polygons on a surface of the ball and wherein each of the polygons is divided into a pattern of geometric areas that divide the polygon into equal parts and each of the geometric areas is coloured black or white to represent binary 1 or 0 to provide an identification code. The system may include a decoding component configured to decode the pattern provided repeatedly on the surface of the ball to obtain a numerical identifier. The system may include a computer visualisation system including a memory for storing computer-readable program code and a processor for executing the computer-readable program code and configured to receive the three-dimensional coordinates of the ball tracking and the events and to: map the coordinates to a three-dimensional model of the course hole for visualisation of the ball in a three-dimensional visualisation of the hole; identify a ball as relating to a registered player; and score the events of the course hole for the registered player. The computer visualisation system may be configured to determine an optimum positioning of the multiple cameras to capture camera data covering the course hole using the three-dimensional model of the course hole and virtual cameras. The adapter interface may use a publish / subscribe messaging system including queuing data and events to provide ordered delivery. According to a further aspect of the present invention there is provided a computer program product for ball tracking in a physical mini-golf game for computer visualisation comprising a computer-readable medium having stored computer-readable program code for performing the steps of: obtaining camera data from multiple cameras positioned to capture movement of a ball on a course hole from a tee-off position to a cup of a physical mini-golf game course; identifying a ball at the tee-off position by capturing a pattern by a camera with the pattern provided repeatedly on the ball to link the ball to a registered player; tracking the identified ball using the synchronised camera data to provide three-dimensional coordinates of a movement of a ball; and generating events relating to the ball during the movement on the course hole from the tee-off position to the cup; wherein the method provides the three-dimensional coordinates of the ball tracking and the events to a computer visualisation system for display. Further features provide for the computer-readable medium to be a non-transitory computer-readable medium and for the computer-readable program code to be executable by a processing circuit. Embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS In the drawings: Figure 1A and 1B are block diagram of an example embodiment of a system in accordance with the disclosure; Figure 2 is a block diagram of an example implementation of a system in accordance with the disclosure; Figure 3 is a flow diagram of an example embodiment of a method in accordance with the disclosure; Figures 4A and 4B are flow diagrams of other example embodiments of a method in accordance with the disclosure; Figures 5A and 5B are diagrams showing an example embodiment of a ball identification pattern; Figure 6 is a schematic diagram of a machine learning system as implemented by aspects of the disclosure; and Figure 7 illustrates an example of a computing device in which various aspects of the disclosure may be implemented. DETAILED DESCRIPTION WITH REFERENCE TO THE DRAWINGS A method and a system for ball tracking in a physical mini-golf game are provided. A ball tracking and event generating system for mini-golf is described that combines a physical camera configuration for capturing data of a ball on a course hole from a tee-off position to a cup of a physical mini-golf game course. The output of the tracking and event generating system is data that can be used for a computer visualisation system displayed to a player for video playback, enhanced game features, and scoring. The computer visualisation system (190) may include processing of the output of the tracking and event generating system (150) in association with a three-dimensional model (191) of the course hole to provide a three-dimensional visualisation (192) of the game. Referring to Figure 1A, a block diagram (100) shows an example embodiment of the described system. At a physical location (110) of the physical mini-golf course hole (112), a hardware system (130) is provided for the course hole (112). The system is described for a single course hole and may be expanded and scaled to multiple course holes. The hardware system (130) includes a physical camera configuration for providing at least two course cameras (132) in the form of cameras positioned on the course hole (112) for recording frame data with a number of frames per time unit during a playing of a course hole. At least some of the course cameras (132) may be positioned overhead of the course hole. The course cameras (132) may be synchronised so that the camera data is captured at the same time. The course cameras (132) may also be calibrated for a location as described further below to provide accurate location in the space. The at least two course cameras (132) are positioned to capture movement of the ball (120) as it moves (114) through the physical course hole (112) from a tee-off position to a cup into which the ball is received at the end of the course hole. The number of cameras needed and their positions required are arranged to ensure that the course cameras (132) have as much visibility of the ball as possible. In some holes, only two course cameras (132) may be needed. In more complex holes, three or more course cameras (132) may be needed and some cameras made need to be positioned under obstacles of the course (i.e. under a Ferris wheel). The physical camera configuration also includes at least one camera (131) for ball identification that captures an identifying pattern on a ball. This may capture a stationary image. The ball identification camera (131) may be at the tee-off position (113) and may be a close-up camera. Alternatively, one of the course cameras (132) may be used to capture the pattern for the ball identification. The physical camera configuration may be maintained on site at the physical location (110) of the physical course hole (112) or with remote desktop management. The camera configuration may be provided such that as many cameras are provided as necessary within a minimum distance of the ball and so that the ball is not occluded or only minimally occluded from all cameras at any time. The camera configuration may include one or more cameras that record the playing of a course hole for playback at the physical location (110). The two or more course cameras (132) are used to calculate a three-dimensional tracking of a ball as described further below. This data may be stored so that the path of the ball can be shown on the 3D replica of the hole using the 3D course model (191). This data may also be stored and used for playback at a future point in time to give players an opportunity to replay their shots (i.e. hole in one). As course cameras (132) are being used, the physical raw footage may be stored and saved to give an option to reuse the footage. This may be via live streaming at the venue (i.e. showing feeds of various players on a big central screen) or having action playbacks (i.e. when someone makes a hole in one showing their playback and video feed on a central screen as a live action playback). The computer visualisation system (190) may be configured to determine an optimum positioning of the multiple course cameras (132) to capture camera data covering the course hole by using the three-dimensional model (191) of the course hole and virtual cameras. The physical camera configuration may include a lighting system (134) that includes an infrared light source positioned over the course hole (112) to illuminate the course hole (112) in the infrared spectrum for obtaining infrared camera data. At least some of the cameras (131, 132) may be infrared capturing cameras. Golf balls or mini-golf balls (120) may be provided for playing the physical mini-golf game, with each ball (120) having a pattern (121) provided on the surface of the ball and the pattern configured to be captured by a camera regardless of the position of the ball in relation to the camera. The pattern (121) may be provided repeatedly over the surface of the ball (121). The pattern (121) may be linked to an identifier of a registered player. The pattern (121) may be generated by a visual coding system that gives a large number of unique patterns. Further details of an example of the visual coding are provided in relation to Figure 5A and 5B described below. The system includes a data processing system (140) for obtaining camera data from the cameras (131, 132) and processing this by a tracking and event generating system (150) and sending messages via an adapter interface (160) to the computer visualisation system (190). The tracking and event generating system (150) is configured for identifying a ball (120) at the tee-off position of the hole by capturing a pattern by a camera to link the ball to a registered player and tracking the identified ball to provide three-dimensional coordinates and to generate events relating to the playing of the ball (120) on the course hole. The events that are generated may relate to legal shots used for scoring the play. The tracking and event generating system (150) may include a computer vision artificial intelligence (Al) processor peripheral. The computer visualisation system (190) may be provided in a server environment (180), which may be a local or cloud environment provided via a network (170). The computer visualisation system (190) may include a three-dimensional model of the course hole (191) used to provide a three-dimensional visualisation (192) of the course hole (112) in combination with the ball tracking and events. The computer visualisation system (190) may include a game logic component (193) for providing the game aspects of the visualisation. The server environment (190) may also include other functionality in the form of game systems (181) including one or more of: a booking system, a check-in system, a player / course allocation system, a player registration, a ball scanning and allocation system, a game management dashboard, a core gameplay system, a leaderboard, and a post-game system. The server environment (190) may also include secure player data storage (182) where customer data may be securely stored (182) and shared with security requirements. The data processing system (140) and the server environment (180) may each include a processor for executing the functions of components described below, which may be provided by hardware or by software units executing on the system. The software units may be stored in a memory component and instructions may be provided to the processor to carry out the functionality of the described components. In some cases, for example in a cloud computing implementation, software units arranged to manage and / or process data on behalf of the system may be provided remotely. Some or all of the components may be provided by a software application downloadable onto and executable on a mobile device. Referring to Figure 1B, the system components of the example embodiment of Figure 1A are shown with details of the data processing aspects. The hardware system at the course hole (130) provides real-time camera data inputs (158) to the data processing system (140). The ball tracking and event generating system (150) is configured to carrying out the steps of: obtaining camera data from the multiple cameras positioned to capture movement of a ball from a course from tee-off position to a cup of a physical mini-golf game; identifying a ball at the tee position of the hole by capturing a pattern by a camera with the pattern provided repeatedly on the ball to link the ball to a registered player; tracking the identified ball using the camera data to provide three-dimensional coordinates; and generating events relating to the ball during the movement on the course from the tee position to the cup. Generating events may include identifying legal shots of the identified ball by analysing ball velocity and determining that the identified ball is at the cup. Generating events may include events of one or more of the following: a legal shot, an illegal shot, a ball out of bounds, a ball in cup, a ball interacting with a feature of the course (such as a hazard, booster, interactive element, etc.), and a ball moving through a virtual feature mapped onto the course (i.e. using projection mapping). The ball tracking and event generating system (150) may include a computer vision system (152) including: a machine learning object detection component (154), a tracking component (153), a ball identification component (155), an event component (156), and an occluded area component (157). The machine learning object detection component (154) may include using computer vision Al models such as an arrangement of deep neural networks that are trained using a training system (151). The training system (151) may include training datasets of camera inputs. The computer vision Al models may include object detector models trained with data enabling them to detect golf balls and golf clubs to be used to detect the ball and club in the camera data. Video data to be used for training data is generated and then annotated by human annotators. This annotated data is then used to train object detection models (for example, off-the-shelf object detection models trained with the annotated data), using standard methods to train object detectors. The training system (151) may include a 3D model training (171). The 3D course hole models (191) used by the computer visualisation system (190) may be used to train the computer vision system (152) prior to needing real world video footage. The tracking component (153) may be used to track the balls from the camera data in three-dimensional space. The tracking component will align every individual frame, at up to 70 frames per second, across all of the cameras when calculating ball tracking and behaviour. When a ball is detected on more than one camera, mathematical calculations are done using triangulation to calculate the position of the ball in three-dimensional space. Two tracking methods are used to calculate the three-dimensional position. In a first tracking method, an accurate 3D model (191) of the course hole is combined x and y coordinates of the ball and then use the 3D model to calculate the z coordinate. In a second tracking method, accurate calibration of the cameras, together with synchronised frames from at least two cameras are used to calculate the intersection point of what the cameras are seeing and therefore x, y, z coordinates are calculated. To track the ball, both methods are continually used and then the result of those two methods are combined to give a more accurate and robust result. The occluded area component (157) may provide an additional element of ball tracking where the captured camera data has no visual of the ball. Examples of this will be where a ball travels through the internal system of the hole, or there is a large object on the course occluding the ball (i.e. a Ferris wheel). In these cases, a combination of the 3D course hole model (191) and a physics engine may be used to extrapolate the moment of the ball through the ‘hidden’ areas of the hole. This may take the last co-ordinate of the computer vision system and try to determine the location of the ball to see if it as gone into a ‘hidden’ area. The event component (156) may determine ball events. Events may include stroke detection, legal shot detection, hole completion detection, ball zone detection, etc. Zone detection may be by detecting a ball entering a zone by comparing the position of the ball to pre-defined zone coordinates. Stroke detection may be carried out by comparing the coordinates of the ball as well as the club when the ball velocity is detected to change from standstill (velocity = 0) to motion. If a club is detected next to the ball at that moment, a valid stroke is detected, If not, an invalid stroke is detected. The ball tracking and event generating system (150) may take real-time camera inputs (158) and provide real-time or near real-time outputs of tracking and events (159). The ball identification component (155) may process a captured identifier pattern on a ball and may provide ball identification with the output data. The tracking and event generating system (150) may detect a ball in three-dimensional (3D) space and send the 3D coordinates of that ball plus the time stamp for that detection. This may be for each frame of the captured image data from the cameras. The 3D coordinates are provided whether the ball is in motion or standing still so that a stream of coordinates is sent continually. The cameras (131, 132) may be calibrated such that the tracking and event generating system (150) knows where the cameras are situated and orientated in a real world, 3D space. The calibration may allow the tracking and event generating system (150) to determine the physical distance between two or more points in space from a camera image alone, or where a particular point on a camera image exists in physical space. When a golf ball is detected on two or more cameras, the position of a golf ball in 3D space may be determined by triangulation. The tracking and event generating system (150) may detect events such as legal shots of the ball and determining when the identified ball is in the cup (115). Additionally, events such as stroke detection, the moment a club strikes the ball, may be performed by comparing the coordinates of the ball, as well as the club, when the ball velocity is detected to change from a standstill to moving. The ball velocity may be determined by evaluating the ball position over time. If a club was detected next to the ball at that moment, a valid stroke may be detected, else an invalid stroke may be detected. The data processing system (140) also includes an adapter interface (160) configured to provide the three-dimensional coordinates of the ball tracking and the events to a computer visualisation system (190) for display to a player. The adapter interface (160) may use a publish / subscribe messaging system (161) including queuing data and events to provide ordered delivery. The messaging system (161) provides a fault tolerant interface that takes all the events from the computer vision machine learning algorithms in real-time and ensures that every message is processed by the downstream adapter. This ensures that even if there is a network drop or power failure the events will not be lost and will be processed once the environment is back in working order. The messaging system (161) may include a backend with a database that stores all of these messages and the messages are sent to a message broker which uses the messaging publish / subscribe architecture so that subscribers can subscribe to the relevant messages for their further processing. The adapter interface (160) may also include an event interpreter (162). The adapter interface (160) provides functionality to provide the hardware output to the visualisation system (190). The event interpreter (162) takes the real-time outputs (159) of tracking and events and interprets them in a way that the in-game computer visualisation system (190) can use. The protocol between the data processing system (140) at the hardware sensing side and the receiving computer visualisation system (190) at a server environment (180) for visualisation includes publish / subscribe messages relating to the following. 1. Real-time ball detection stream including time and 3D coordinates. The receiving side computer visualisation system has a 3D model of the hole to which the 3D coordinates can be mapped. 2. Ball events including when the ball is stationary, when the ball is moving for different reasons (being picked up), and stroke detection. 3. Cup events including when the ball is entering or exiting. 4. Special events where the ball interacts with special features on the course. 5. Ball ID at tee-off position. This may be received from cameras less than a threshold distance, such as approximately 50cm from ball, and ball having ID pattern / number on all surfaces. The ball is identified at the tee-off position and then tracked as it keeps moving, so it does not need to be re-identified. 5. Exception events such as ball out of bounds. 6. System events, such as errors, warnings, device reboot, etc. The adapter interface (160) on the sensing side may send event and ball location information. All state and logic may be on the receiver side. The receiver side has a 3D model (191) with 3D visualisation (192) and may update in real time. The computer visualisation system (190) may subscribe to update and displays the 3D coordinates to visualise the ball. The computer visualisation system (190) may also provide scoring and leader board information. The computer visualisation system (190) at the receiving side is configured to receive the three-dimensional coordinates of the ball tracking and the events and to: map the coordinates to a three-dimensional model (191) of the course of the hole; identify a ball as relating to a registered player; and score the events of the course for the registered player. The computer visualisation system (190) may include a visual user interface and a scoring system. The computer visualisation system (190) may include a game logic component (193). The game logic component (193) will use the tracking and events as sent by the adapter interface (160) and the three-dimensional course hole model (191) to simulate the ball in the course hole including going through the inner sections of the course. Where the ball may be occluded, the game logic may use the last co-ordinates from the computer vision before the delay and the first co-ordinate after the delay to determine where the ball travelled and display this for the player, using the 3D model replica of the hole. The computer visualisation system (190) may include a state component (197) for maintaining state in the form of a player’s session of the tracked balls and events. Once a player has registered on a course and been allocated a unique ball the game logic will start a new game session. This game session maintains the state in the form of a player’s session. On each hole the computer vision software will respond to ball events that take place and the game logic component (193) will link this to the game session, group, player, hole, shot and events. All of this will aid in the scoring logic that is performed by the game logic component (193). The computer visualisation system (190) may include a hole metadata component (195) with metadata (183) stored for each hole. Some areas of the course holes may have special meanings, i.e. Tee off, Boosters, Hazards, Final Cup, Special Objects. These may be created as metadata that lives with each hole and will be stored in a database. The metadata may be used by the game logic component (193) to identify special events. The metadata (183) may also be used by the computer vision system (152) as part of the training process. The computer visualisation system (190) may be configured to determine an optimum positioning of the multiple cameras to capture camera data covering the course using the three-dimensional model of the course and virtual cameras. This may be used during a set-up of the cameras for a cup. The computer visualisation system (190) may visualise the course hole in 3D and uses the 3D messages from the adapter interface (160). The real-time detection streams to as real-time as possible, updating the position of the ball. The computer visualisation system (190) subscribes and gets the messages from the messaging system database to directly get the messages and display them. The computer visualisation system (190) may include a scoring system that subscribes to and processes ball events, whether a legal shot happened or not, and when the ball is in the cup. The scoring system may provide a leader board given a continuous update of the scoring based on the received events. The server environment (180) may include a registration system for on-site registration, identifying the players based on their booking reference, name, etc. and setting up the game by allocating each player to a ball with a visually unique marking or pattern. The in-game computer visualisation system (190) tracks the player journey, calculates scores, shows leader board, and includes user experience elements, such as game modes, animations, sounds, etc. The computer visualisation system (190) may include a number of discrete models each solving specific requirements. Various game systems (181) are included in the customer journey. These may include the following systems. A booking system may manage booking games, managing bookings and pre-registering players to make the registration prior to playing more efficient. A check-in system may inform the system that a player has arrived and this may confirm their game, put their game on the notice board with the starting time and may register any players who did not pre-register in the booking system. A game management dashboard may display the upcoming games, which course a game will be played on, and may notify players when their game is ready so that they can proceed to play their game. A registration system may allocate a ball for each player. The registration system may also link players who are in groups larger than four players into teams of up to four players and register any players who have not pre-registered or checked in. A game play system may manage all in game events, from identifying the ball when it is placed at the starting tee, to counting shots, ball tracking and scoring. A leaderboard system may keep track of player rankings within their smaller groups and for all players in ae booking. A post-game system may send information (for example, via an email) to players with key metrics from their game and may include an ability to replay their hole-inOones by replaying the ball tracking data on the 3D hole in their browser. Referring to Figure 2, an example implementation (200) of the described system is shown. In the physical space (220) of the physical mini-golf game, a registration system (230) may be provided with a registration application (231). A game system (240) may also be provided at the physical space with a gameplay monitoring application (241) with an event simulator (243) for simulating the events. The registration application (231) and the gameplay monitoring application (241) may both send and receive data via REST endpoints to an application programming interface (API) (211) on a virtual private cloud (210) where the API (211) is in communication with a database (212) and which may send messages via an email system (213). A tracking and event generating system (150) may receive camera frame data from the multiple overhead cameras (132) of the course hole and may receive a numerical identifier of the ball from a processor (222) for ball ID decoding of data from a ball identification camera (131). The output of the tracking and event generating system (150) may be sent using a message broker (242) to the gameplay monitoring application (241). The game system (240) may output a display (225) of the visualisation. Referring to Figure 3, a flow diagram (300) shows an example embodiment of the method for ball tracking in a physical mini-golf game. The method is described for a single course hole and may be expanded and scaled to multiple course holes. The method may provide (301) at least two cameras on the course hole and at least one additional close-up camera at the tee-off position. The method may include determining an optimum positioning of the multiple cameras to capture camera data covering the course. The method may shine (302) an infrared light source over the course hole to illuminate the course in the infrared spectrum and may obtain infrared camera data. The method may calibrate (303) the multiple cameras for the course. The method may obtain (304) camera data from the multiple cameras positioned to capture movement of a ball from a course from tee position to a cup of a physical mini-golf game. The camera data from multiple cameras may be synchronised. The method may identify (305) a ball at the tee position of the hole by capturing a pattern by a camera with the pattern provided repeatedly on the ball to link the ball to a registered player. The method may detect (306) a ball and a club in the camera data. Machine learning modelling may be used for object detection of the ball and club. The method may include training the machine learning modelling including: using training datasets of camera data with annotations identifying balls and clubs in the camera data. The method may track (307) the identified ball using the camera data to provide three-dimensional coordinates. Tracking the identified ball may use the camera data to provide three dimensional (3D) coordinates includes a combination of a first tracking method and a second tracking method as explained further below. The tracking may account for obstacles or occluded areas in the course. The method may identify (308) legal shots of the identified ball by analysing ball velocity. Identifying legal shots may include distinguishing between legal and illegal shots by monitoring ball movement and tracking a golf club head, wherein a legal shot is where the golf club head is at the ball at the moment where the ball transitions from a stationary state to a moving state. The method may determine (309) that the identified ball is in the cup. The method may generate (310) events relating to the ball during the movement on the course from the tee position to the cup. The method may provide the three-dimensional coordinates of the ball tracking and the events from the input camera data. The method may provide (311) the three-dimensional coordinates of the ball tracking and the events to a computer visualisation system for display. Providing the three-dimensional coordinates of the ball tracking and the events to a computer visualisation system may use a publish / subscribe messaging system including queuing data and events to provide ordered delivery. The method may include interpreting events for use by the computer visualisation system including states linked to ball identification and registration of players. The computer visualisation system may receive the three-dimensional coordinates of the ball tracking and the events at a computer visualisation system for: mapping the coordinates to a three-dimensional model of the course of the hole; identifying a ball as relating to a registered player; and scoring the events of the course for the registered player. Referring to Figures 4A and 4B, flow diagrams (400, 450) show an example embodiment of the method at each phase. In a physical phase (410), the method may position (411) cameras at a course hole and may provide (412) lighting for the course hole. The lighting may include infrared lighting to illuminate the course hole to enable the cameras to capture images in dark or strobe lighting situations. The camera positioning (411) may use the computer visualisation 3D modelling of the course hole to place virtual cameras to test camera positions to ensure an optimised coverage of the cameras to track a ball on the course hole. The computer visualisation may provide a simulation of the course hole enabling the virtual cameras to be moved around until there is a desired orientation for each course hole. This may be an automated process that may be applied to each course hole to obtain optimised camera positions. In the physical phase (410), the method may calibrate (413) the cameras for the course hole. Calibration may use a known pattern provided at the course hole, such as white and black squares on a flat surface placed in different places and captured by the cameras and compared. A calibration software recognises the pattern and the exact corners of each block and that gives it the size on which to calibrate. When there is a known 3D model of the course hole, automated calibration may be carried out without using a known calibration pattern. An element of the course hole, such as the cup, may be used for the automated calibration of the cameras. Features, such as known dimensions and shape, of the cup may be selected to use for the calibration. The method may synchronise (414) the cameras for the course hole. This may be carried out using known synchronisation functions of cameras. In a game phase of processing (420), the method may include the use of a computer vision system including Al modelling for object detection. The computer vision Al modelling system may be trained as explained further below. The training may include calibration of the models to work in natural light and infrared light to accommodate for any lighting in the environment that could affect the accuracy of the computer vision. The computer vision system may include shot tracking, player tracking, event tracking, and positional tracking. A bespoke computer vision modelling system is trained to solve each of the tracking requirements. The method may capture and identify (421) a ball at the tee-off area. The method may also capture (422) whether the ball is moving or at rest. The method may identify (423) ball events including legal and illegal shots. The method may track (424) the ball using tracking methods and may also process (425) for occluded areas of a course hole. The method may identify (426) when the ball is in key areas of the course hole and may capture (427) the ball at the cup. In Figure 4B, an adapter phase (430) may include receiving (431) computer vision events into a messaging queue, interpreting (432) the events for use by a visualisation phase, and publishing events for the visualisation phase to subscribe to. The visualisation phase (440) may include a player booking (441) a game and registering (442) for the game including identifying players. The method may allocate (443) a ball identifier to a player. The visualisation phase (440) may receive (444) interpreted computer vision events and may link (445) the interpreted events to the players using the ball identifier. The method may track (446) a player’s journey through the game and may provide (447) visualisation of the ball play and scoring. The computer vision system may include multiple models and additional processing that is combined to generate the required outputs. The Al modelling system identifies a golf ball as it moves along a course. The computer vision system can identify when a ball is moving and when a ball has come to rest. Using the two or more calibrated cameras, the modelling system can track the (x,y,z) coordinates of the ball in real-time. The modelling system can cater for obstacles and occlusions, this may be achieved both within the training of the model as well as having multiple cameras pointed at the course. The computer vision system may be built in a way that the models can auto-calibrate. Camera outputs may have a maximum number of frames per second, and the Al modelling system may detect at close to the same frames per second with training. The Al modelling system may detect the ball whenever visible, and immediately after it has been hidden and becomes visible again. Positional tracking Positional tracking may include being able to track the (x,y,z) coordinates of the ball in three-dimensional space in a way that can be tracked by visualisation software and displayed on a screen to the player showing them an augmented, computer-generated trajectory of the shot that they have just played. The tracking component (153) may be used to track the balls from the camera data in three-dimensional space. When a ball is detected on more than one camera, mathematical calculations are done using triangulation to calculate the position of the ball in three-dimensional space. Two tracking methods are used to calculate the three-dimensional position. In a first tracking method, an accurate 3D model of the course hole is combined with accurate calibration of the cameras in 3D space allowing the cameras to measure the x and y coordinates of the ball and then use the 3D model to calculate the z coordinate. In a second tracking method, accurate calibration of the cameras, together with synchronised frames from at least two cameras are used to calculate the intersection point of what the cameras are seeing and therefore x, y, z coordinates are calculated. To track the ball, both methods are continually used and then the result of those two methods are combined to give a more accurate and robust result. The first positional tracking approach may calculate the (x,y,z) coordinates of the ball using a 3D model of the course hole. Hole dimensions may be measured (x,y,z), create a 3D model of hole and the camera is calibrated to the hole’s 3D model coordinate system. The camera’s coordinates (x,y) are converted to real world coordinates (x,y) and the real world elevation (z) is calculated. Only one camera is required. However, a 3D model is required for each hole and it is assumed that the ball stays on the surface of the hole. Manual calibration is required when the camera moves. A second positional tracking approach may calculate the (x,y,z) coordinates through triangulation which is independent of course hole. This method may calibrate two or more cameras according to the real world and each other and may use triangulation to calculate the ball position. This has the advantage of being independent of the hole model. There may also be automated calibration when cameras move. The ball needs to be visible to two cameras at all times for this to be effective. Accurate 3D ball tracking is achieved with multiple cameras that are successfully calibrated and that have knowledge of world ground. The positional tracking may use only one or a combination of the two tracking approaches. The first positional tracking approach may be used with the 3D model of the hole where two cameras do not capture the ball or where improved ball localisation is needed. The second approach may be used with no 3D course model needed. The two methods may be fused using a probability measure. The accuracy of positional tracking may be solved with smoothing and frame rate increases as required. Motion filters may be used to improve detection and jerkiness. Tracking in occluded areas The tracking may provide an additional element of ball tracking where the captured camera data has no visual of the ball. Examples of this will be where a ball travels through the internal system of the hole, or there is a large object on the course occluding the ball (i.e. a Ferris wheel). In these cases, the 3D course hole model may be used to extrapolate the moment of the ball through the ‘hidden’ areas of the hole. This may take the last co-ordinate of the computer vision system and try to determine the location of the ball to see if it as gone into a ‘hidden’ area. Player tracking Player tracking may include being able to identify the player by the use of unique ball markings at the tee-off and then tracking the ball on the course. A computer vision model may be trained to identify a unique pattern such as a shape or number of the ball and use information from a registration system to identify the player who is at the hole. A time stamp identifying when the player initiated their first shot is used to provide a capability to limit the player’s time at each hole. Once the ball is identified (and therefore the player), knowing which player to assign strokes to comes down to tracking the ball accurately until it is in the cup. Shot tracking Shot tracking may include tracking a legal shot, i.e. knowing when a ball has been hit and when it has come to rest and being able to determine if this is a legal shot. A legal stroke may be detected by keeping track of a “motion” state of the ball when stationary or moving. A stationary ball has low speed for long time and a moving ball has high speed for any amount of time. If state changes from stationary to moving, a stroke has occurred. This may be augmented by detecting a club head and if the club was near the ball, it was a legal stroke. Sudden ball acceleration from still-standing with club head nearby will equate to a legal stroke. Shot tracking also may include one or more of the following: tracking an illegal shot, i.e. knowing if a ball has been kicked or picked up and moved to a new location on the course; tracking a ball through obstacles, i.e. knowing that mini-golf courses can have areas that will be hidden from view, can the computer vision still track a legal shot if a ball goes through an area of the course which is hidden from the camera; and tracking through an occlusion, i.e. still able to track a ball when a player is standing over the ball taking a shot or a person is standing between the camera and the ball. Tracking key events Tracking key events may include determining when a ball has landed in the final cup, marking the end of the players session for that course hole. Tracking key events may also include knowing if a player’s ball has gone into a specific area of the course. This could be a Hazard (penalty area), Booster (fast track to hole in one) or any other aspect of the course that has a bearing on the score. An extension of the shot and position tracking computer vision model may allow for the identification of key areas of the course that have a special meaning. These can be determined by the game system in real time to affect the score. Ball identification patterns A ball may be covered by repeating polygons. Each or a selection of the polygons may be divided into a pattern of geometric areas that divide the polygon into equal parts. Each of the geometric areas may be coloured black or white to represent binary 1 or 0 to provide an identification code. The codes can be decoded regardless of orientation of the ball. The code is decoded in software by recognizing a polygon and finding the outer border. The polygon is then flattened by projecting a spherical surface onto a flat plane, rotated and compared with a set of canonical codes until a match is found. The numerical identifier of the canonical code is known, and that therefore gives the numerical identifier for that ball. Figures 5A and 5B show an example embodiment of a ball identifier pattern in which hexagons with internal kite shapers are used. Figure 5A shows a single hexagon (500) and Figure 5B shows a flattened net design (520) that can be used to cover a ball. In this example, a ball may be covered by a combination of hexagons (500) and pentagons (530), like a soccer ball. Each hexagon (500) is the polygon that is coded and the geometric areas are four-sided polygons, specifically kites (510). Each hexagon (500) on a specific ball contains a specific code that can be decoded to a number (the identifier of the ball). The code is made up of 18 kites, that divide the hexagon into 18 equal parts. Each kite can be 1 of 2 colours: black or white, representing a binary 1 or 0. The code is decoded in software by recognising a hexagon (500) and finding the outer border (502). The hexagon (500) is then flattened by projecting a spherical surface onto a flat plane, rotated and compared with a set of canonical codes until a match is found. The numerical identifier of the canonical code is known, and that therefore gives the numerical identifier for that ball. Machine Learning Aspects An embodiment of the trained computer vision Al models may be an arrangement of artificial neural networks (ANNs). Further embodiments of ANNs may include convolutional neural networks suitable for computer vision applications, or an arrangement of recurrent neural networks, such as long-shortterm memory (LSTM) networks suitable for time series applications. An ANN consists of interconnected units, commonly referred to as neurons, as they are inspired by and resemble neurons of the brain. The units consist of nodes and edges forming a connected network. ANN’S are commonly configured into multiple rows, referred to as layers, to form a layered structure with an input at the first layer, and an output at the final layer. The layers between the first and final layer are the hidden layers. Each node in the ANN receives a signal from one or more nodes in the preceding layer, starting from the input layer. The output of a node is computed by an activation function, which is a nonlinear function of the sum of the inputs into each node in each layer. The output value of each node in the preceding layer is multiplied by a weighting value, which determines the strength of each nodes output value. Finally, the value that is determined at the final layer is the output of the ANN. For regression type ANNs, the output may contain only a single node with a value, or many nodes. Alternatively, classification type ANNs, the output may include multiple nodes, where each node provides the probability of a classification type. Furthermore, more complex ANNs are better suited to specific tasks. Computer vision tasks frequently use convolutional neural networks (CNNs). A CNN is particularly suited to imagebased tasks, where image data is often structured as a two-dimensional data structure. In addition to the weights and activation functions of a regular ANN, a CNN applies a filter (or a kernel) onto a two-dimensional data structure to reduce the size of the hidden layers in the neural network, thereby reducing the number of weights within the neural network. Creation and training an ANN may be readily available through software libraries such as PyTorch™ or Tensorflow™. These libraries contain tutorials and templates for the creation of ANNs. The terms machine learning, deep learning and Al may be used interchangeably throughout this disclosure. Deep learning is considered a sub-branch of machine learning, which itself is considered a sub-branch of artificial intelligence. The trained computer vision Al models may be configured to run on an off-the-shelf small, powerful computer. An example embodiment of an off-the-shelf computer may be a Raspberry Pi ™, a NVIDIA Jetson Nano ™ developer kit, or a standard personal computer (PC) including a graphical processing unit (GPU). Off-the-shelf computers are designed to perform specific computational tasks, which may include running multiple neural networks in parallel for applications including image classification, object detection, segmentation, and speech processing. The off-the-shelf computer may run without the need for large computing resources in a server environment. Running the computer vision Al models on a computer at the data processing system reduces the amount of data transmitted over a network. Training deep neural networks may be a computationally intensive and time consuming. The training system may be a large computing infrastructure or a cloud computing infrastructure that can be accessed over a network. These resources allow for dynamic computing resources to be dedicated to training a deep neural network, after which the trained model can be downloaded to run on a separate application. An embodiment of this disclosure does not rely on any specific infrastructure for training the deep neural networks. The training system may train the modelling system for ball and club detection. For example, this may be by using a target ball with stickers that moves through the camera frames. The modelling system may include an object detector model. The object detector model may be trained to detect golf balls and clubs that are used to strike a golf ball. The 3D course hole model may also be used in the training. The training datasets of camera inputs may include camera or video data of a golf ball or other objects that are required to be detected by the modelling system. The training datasets may be manually annotated by human operators or using automated annotation tools to annotate balls or clubs in the camera inputs. The annotated training datasets may be used with standard methods to train an object detection model that enables the modelling system to perform ball tracking and generating events. The object detection model may be an off-the-shelf detection model. The models are trained using large datasets of video and frame grabs from those videos. This data is annotated and stored on an online store and the training may then happen in the cloud. The trained models are then deployed to the on-site computer to do the detections. Figure 6 illustrates a general overview of a training and use of machine learning models (614). The training involves a data preparation process (611) that formats a raw incoming data (610) to prepare training data (612). A training process (613) receives the prepared training data (612) and iteratively updates a machine learning model until a predefined quality criteria and accuracy criteria are achieved. A machine learning model (614) is output at the end of the training process (613) to be used in the runtime process (622). When the model (614) is in use, an input (621) is received into the runtime process (622) that uses the model (614) to obtain an output (623) that may be used in a downstream process (624). The computation of the runtime process (622) is often referred to as ‘inference’. The training process (613) may be computationally demanding and time consuming. To successfully train a machine learning model, very large datasets are often used which are stored on databases. The training process may be performed on a large computing cluster which may access the database to obtain the training data when required. Additionally, the trained machine learning model (614) may be stored on the database. The runtime process (622) may run on an end user device by downloading the machine learning model (614) over a network from a database, or the runtime process (622) may run on a large computing infrastructure such as a computing cluster. An example embodiment of interacting with the machine learning model (614) may include an end user device, such as a mobile device or a computer which may obtain or be the source of the input data (621), transmit the input data (621) over a network to a computing cluster to perform the runtime process (622). Alternatively, an end user device may obtain the machine learning model from a database over a network and store the machine learning model locally on the device. The end user device may obtain an input data (621) and perform the runtime process (622) locally on the device to obtain an output (632). By performing the runtime process (622) locally on the device, the input data (621) does not need to be transmitted over a network, reducing bandwidth usage. This may be referred to as ‘on-the-edge’ computing. A machine learning model (614) may be trained using supervised leaning. Supervised learning requires a first set of input data (610) with known output data, also referred to as training data (612). The data preparation process (611) may involve labelling the data to provide a known output for the raw incoming data (610), thereby generating the training data (612). A second set of input data with known output data may be used to test the machine learning model during and after training, also referred to as validation data. The task of the training process (613) is to minimize the difference (or error) between the output of the final layer of the machine learning model (614) and the known output data of the training data (612). The training procedure modifies the machine learning model (613) such that the difference is minimized. Figure 7 illustrates an example of a computing device (700) in which various aspects of the disclosure may be implemented. The computing device (700) may be embodied as any form of data processing device including a personal computing device (e.g. laptop or desktop computer), a server computer (which may be self-contained, physically distributed over a number of locations), a client computer, or a communication device, such as a mobile phone (e.g. cellular telephone), satellite phone, tablet computer, personal digital assistant or the like. Different embodiments of the computing device may dictate the inclusion or exclusion of various components or subsystems described below. The computing device (700) may be suitable for storing and executing computer program code. The various participants and elements in the previously described system diagrams may use any suitable number of subsystems or components of the computing device (700) to facilitate the functions described herein. The computing device (700) may include subsystems or components interconnected via a communication infrastructure (705) (for example, a communications bus, a network, etc.). The computing device (700) may include one or more processors (710) and at least one memory component in the form of computer-readable media. The one or more processors (710) may include one or more of: CPUs, graphical processing units (GPUs), microprocessors, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs) and the like. In some configurations, a number of processors may be provided and may be arranged to carry out calculations simultaneously. In some implementations various subsystems or components of the computing device (700) may be distributed over a number of physical locations (e.g. in a distributed, cluster or cloud-based computing configuration) and appropriate software units may be arranged to manage and / or process data on behalf of remote devices. The memory components may include system memory (715), which may include read only memory (ROM) and random access memory (RAM). A basic input / output system (BIOS) may be stored in ROM. System software may be stored in the system memory (715) including operating system software. The memory components may also include secondary memory (720). The secondary memory (720) may include a fixed disk (721), such as a hard disk drive, and, optionally, one or more storage interfaces (722) for interfacing with storage components (723), such as removable storage components (e.g. magnetic tape, optical disk, flash memory drive, external hard drive, removable memory chip, etc.), network attached storage components (e.g. NAS drives), remote storage components (e.g. cloud-based storage) or the like. The computing device (700) may include an external communications interface (730) for operation of the computing device (700) in a networked environment enabling transfer of data between multiple computing devices (700) and / or the Internet. Data transferred via the external communications interface (730) may be in the form of signals, which may be electronic, electromagnetic, optical, radio, or other types of signal. The external communications interface (730) may enable communication of data between the computing device (700) and other computing devices including servers and external storage facilities. Web services may be accessible by and / or from the computing device (700) via the communications interface (730). The external communications interface (730) may be configured for connection to wireless communication channels (e.g., a cellular telephone network, wireless local area network (e.g. using Wi-Fi™), satellite-phone network, Satellite Internet Network, etc.) and may include an associated wireless transfer element, such as an antenna and associated circuitry. The computer-readable media in the form of the various memory components may provide storage of computer-executable instructions, data structures, program modules, software units and other data. A computer program product may be provided by a computer-readable medium having stored computer-readable program code executable by the central processor (710). A computer program product may be provided by a non-transient or non-transitory computer-readable medium, or may be provided via a signal or other transient or transitory means via the communications interface (730). Interconnection via the communication infrastructure (705) allows the one or more processors (710) to communicate with each subsystem or component and to control the execution of instructions from the memory components, as well as the exchange of information between subsystems or components. Peripherals (such as printers, scanners, cameras, or the like) and input / output (I / O) devices (such as a mouse, touchpad, keyboard, microphone, touch-sensitive display, input buttons, speakers and the like) may couple to or be integrally formed with the computing device (700) either directly or via an I / O controller (735). One or more displays (745) (which may be touch-sensitive displays) may be coupled to or integrally formed with the computing device (700) via a display or video adapter (740). The foregoing description has been presented for the purpose of illustration; it is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Persons skilled in the relevant art can appreciate that many modifications and variations are possible in light of the above disclosure. Any of the steps, operations, components or processes described herein may be performed or implemented with one or more hardware or software units, alone or in combination with other devices. Components or devices configured or arranged to perform described functions or operations may be so arranged or configured through computer-implemented instructions which implement or carry out the described functions, algorithms, or methods. The computer-implemented instructions may be provided by hardware or software units. In one embodiment, a software unit is implemented with a computer program product comprising a non-transient or non-transitory computer-readable medium containing computer program code, which can be executed by a processor for performing any or all of the steps, operations, or processes described. Software units or functions described in this application may be implemented as computer program code using any suitable computer language such as, for example, Java™, C++, or Perl™ using, for example, conventional or object-oriented techniques. The computer program code may be stored as a series of instructions, or commands on a non-transitory computer-readable medium, such as a random access memory (RAM), a read-only memory (ROM), a magnetic medium such as a hard-drive, or an optical medium such as a CD-ROM. Any such computer-readable medium may also reside on or within a single computational apparatus, and may be present on or within different computational apparatuses within a system or network. Flowchart illustrations and block diagrams of methods, systems, and computer program products according to embodiments are used herein. Each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, may provide functions which may be implemented by computer readable program instructions. In some alternative implementations, the functions identified by the blocks may take place in a different order to that shown in the flowchart illustrations. Some portions of this description describe the embodiments of the invention in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations, such as accompanying flow diagrams, are commonly used by those skilled in the data processing arts to convey the substance of their work effectively to others skilled in the art. These operations, while described functionally, computationally, or logically, are understood to be implemented by computer programs or equivalent electrical circuits, microcode, or the like. The described operations may be embodied in software, firmware, hardware, or any combinations thereof. The language used in the specification has been principally selected for readability and instructional purposes, and it may not have been selected to delineate or circumscribe the inventive subject matter. It is therefore intended that the scope of the invention be limited not by this detailed description, but rather by any claims that issue on an application based hereon. Accordingly, the disclosure of the embodiments of the invention is intended to be illustrative, but not limiting, of the scope of the invention set forth in any accompanying claims. Finally, throughout the specification and any accompanying claims, unless the context requires otherwise, the word ‘comprise’ or variations such as ‘comprises’ or ‘comprising’ will be understood to imply the inclusion of a stated integer or group of integers but not the exclusion of any other integer or group of integers. 14 03 25

Claims

1. A method for ball tracking in a physical mini-golf game for computer visualisation,5 comprising:obtaining camera data from multiple cameras positioned to capture movement of a ball on a course hole from a tee-off position to a cup of a physical mini-golf game course;identifying a ball at the tee-off position by capturing a pattern by a camera with the pattern provided repeatedly on the ball to provide a visual coding to link the ball to a10 registered player, wherein the pattern is a polygon divided into a visual coding of geometric areas that divide the polygon into parts with each geometric area coloured to represent a binary code;tracking the identified ball using the camera data to provide three-dimensional coordinates of a movement of a ball; and15 generating events relating to the ball during the movement on the course hole fromthe tee-off position to the cup;wherein the method provides the three-dimensional coordinates of the ball tracking and the events to a computer visualisation system for display.20 2. The method of claim 1, wherein generating events includes:identifying legal shots of the identified ball by analysing ball velocity; and determining that the identified ball is in the cup.

3. The method of claim 1 or claim 2, including providing at least two overhead cameras 25 on the course hole for ball movement capture and at least one close-up camera at the tee-off position for ball identification.

4. The method of any of claims 1 to 3, including shining infrared light source over the course hole to illuminate the course hole in the infrared spectrum and obtaining infrared 30 camera data.

5. The method of any of the preceding claims, wherein identifying legal shots includes distinguishing between legal and illegal shots by monitoring ball movement and tracking a golf club head, wherein a legal shot is where the golf club head is at the ball at the moment 35 where the ball transitions from a stationary state to a moving state.

6. The method of any of the preceding claims, wherein tracking the identified ball includes tracking in occluded areas in the course hole by extrapolating the tracking through the occluded area.

7. The method of any of the preceding claims, wherein tracking the identified ball using the camera data to provide three-dimensional coordinates includes a first tracking method that uses a three-dimensional model of the course hole and at least one camera calibrated to the three-dimensional model coordinate system.

8. The method of claim 7, wherein tracking the identified ball using the camera data to provide three-dimensional coordinates includes a second tracking method that calculates three-dimensional coordinates through triangulation using two more calibrated cameras and without use of a three-dimensional model of the course hole.

9. The method of claim 8, wherein the first and second tracking methods are continually used in combination with the results of the first and second tracking methods combined.

10. The method of any of the preceding claims, including auto-calibrating the multiple cameras for the course hole using a cup as a calibrating feature of the course hole.

11. The method of any of the preceding claims, using machine learning modelling for object detection of a ball and a golf club in the camera data.

12. The method of any of the preceding claims, wherein providing the three-dimensional coordinates of the ball tracking and the events to a computer visualisation system uses a publish / subscribe messaging system including queuing data and events to provide ordered delivery.

13. The method of any of the preceding claims, including receiving the three-dimensional coordinates of the ball tracking and the events at a computer visualisation system for: mapping the coordinates to a three-dimensional model of the course hole;identifying a ball as relating to a registered player; and scoring the events of the course hole for the registered player.

14. The method of any of the preceding claims, including providing course hole metadata at the computer visualisation system to be used to identify events in a game.

15. A system for ball tracking in a physical mini-golf game for computer visualisation, the system comprising, comprising:a ball tracking and event generating system including a memory for storing computer-readable program code and a processor for executing the computer-readable5 program code and configured to carrying out the steps of:obtaining synchronised camera data from multiple cameras positioned to capture movement of a ball on a course hole from a tee-off position to a cup of a physical mini-golf game course;identifying a ball at the tee-off position by capturing a pattern by a camera10 with the pattern provided repeatedly on the ball to provide a visual coding to link theball to a registered player, wherein the pattern is a polygon divided into a visual coding of geometric areas that divide the polygon into parts with each geometric area coloured to represent a binary code;tracking the identified ball using the synchronised camera data to provide15 three-dimensional coordinates of a movement of a ball; andgenerating events relating to the ball during the movement on the course hole LO from the tee-off position to the cup; andan adapter interface including a memory for storing computer-readable program code CO and a processor for executing the computer-readable program code and configured to ^^20 provide the three-dimensional coordinates of the ball tracking and the events to a computer visualisation system for display.1—16. The system of claim 15, including:a physical camera configuration for providing at least two overhead cameras on the25 course hole for ball movement capture and at least one close-up camera at the tee-off position for ball identification.

17. The system of claim 15 or 16, wherein the physical camera configuration includes an infrared light source positioned over the course hole to illuminate the course hole in the30 infrared spectrum for obtaining infrared camera data.

18. The system of any of claims 15 to 17, including:a mini-golf ball having a pattern provided repeatedly on the surface of the ball to provide a visual coding and the pattern configured to be captured by a camera regardless of35 the position of the ball in relation to the camera and the visual coding providing an identifier to link the ball to a registered player, wherein the pattern is a polygon divided into a visualcoding of geometric areas that divide the polygon into parts with each geometric area coloured to represent a binary code.

19. The system of claim 18, wherein the polygon is a hexagon with geometric areas in 5 the form of kites, wherein each of the geometric areas is coloured black or white to represent binary 1 or 0 to provide an identification code.

20. The system of claim 18 or claim 19, wherein the system includes a decoding component configured to decode the pattern provided repeatedly on the surface of the ball 10 to obtain a numerical identifier.

21. The system of any of claims 15 to 20, including a computer visualisation system including a memory for storing computer-readable program code and a processor for executing the computer-readable program code and configured to receive the three-15 dimensional coordinates of the ball tracking and the events and to:map the coordinates to a three-dimensional model of the course hole for visualisation LO of the ball in a three-dimensional visualisation of the hole;identify a ball as relating to a registered player; andCO score the events of the course hole for the registered player.O2022. The system of claim 21, wherein the computer visualisation system is configured to "1” determine an optimum positioning of the multiple cameras to capture camera data covering the course hole using the three-dimensional model of the course hole and virtual cameras.25 23. The system of any of claims 15 to 22, wherein the adapter interface uses apublish / subscribe messaging system including queuing data and events to provide ordered delivery.

24. A computer program product for ball tracking in a physical mini-golf game for30 computer visualisation comprising a computer-readable medium having stored computer-readable program code for performing the steps of:obtaining camera data from multiple cameras positioned to capture movement of a ball on a course hole from a tee-off position to a cup of a physical mini-golf game course;identifying a ball at the tee-off position by capturing a pattern by a camera with the35 pattern provided repeatedly on the ball to provide a visual coding to link the ball to a registered player, wherein the pattern is a polygon divided into a visual coding of geometricareas that divide the polygon into parts with each geometric area coloured to represent a binary code;tracking the identified ball using the synchronised camera data to provide three-dimensional coordinates of a movement of a ball; and5 generating events relating to the ball during the movement on the course hole fromthe tee-off position to the cup;wherein the method provides the three-dimensional coordinates of the ball tracking and the events to a computer visualisation system for display.

Images