Autonomous activity monitoring system and method

The system addresses the challenge of recording recreational athletic activities by using AI-enabled remote cameras to automatically capture, process, and enhance videos, providing users with detailed performance analysis without manual setup or third-party assistance.

JP2024167229AActive Publication Date: 2024-12-03HOLE IN ONE MEDIA INC
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2024132090
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2019-05-13
Filing Date
2024-08-08
Publication Date
2024-12-03
Estimated Expiration
2040-05-13

AI Technical Summary

Technical Problem

Recreational athletes face challenges in recording their activities due to the lack of widespread video recording and broadcasting, as existing systems like body-worn cameras are limited in capturing body movements and require manual setup or third-party assistance, which is impractical for unpredictable outdoor locations.

Method used

A system that includes remote cameras with AI logic to automatically detect and record athletic activities, process videos, and add graphical elements, using GPS, facial recognition, and object detection to identify users and activities, and communicate processed videos through a mobile app.

Benefits of technology

Enables automatic and efficient recording of athletic activities, allowing users to analyze their performance with enhanced video features, reducing the need for manual setup and third-party involvement.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024167229000001_ABST
    Figure 2024167229000001_ABST
Patent Text Reader

Abstract

To improve a video recording system for an athletic event.SOLUTION: A system 100 may include AI logic configured to identify a user recorded within a video frame captured by cameras 102, 104, 106, 108. The system may also detect and identify a user when the user is located within a predetermined area. The system may include a video processing engine configured to process images within the video frame to identify the user and may modify and format the video upon identifying the user and the activity. The system may include a communication module to communicate formatted video to a remote video processing system 118, which may further process the video and allow access to a mobile app of the user.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of previously filed U.S. Provisional Patent Application No. 62 / 847,052 (filed May 13, 2019), the entire contents of which are incorporated herein by reference in their entirety.

[0002] Technical Field The present disclosure is directed to video recording of activities. More specifically, the present disclosure is directed to automatically monitoring and recording athletic activities such as golfing using video cameras and computer processing. [Background technology]

[0003] Background technology Amateur and professional athletes participate in a variety of athletic events and competitions. In professional sports, it is common for the competitions to be recorded and broadcast on television and / or the Internet. Broadcasters often work with the athletes or the governing bodies of the athletic events to provide access to the events and the equipment necessary to record and broadcast the events, such as multiple cameras and microphones distributed in fixed or mobile locations. These arrangements are common in most professional spectator sports, including golf, tennis, football, baseball, basketball, soccer, hockey, auto racing, and cycling. These arrangements are also common in collegiate athletics, where events are often covered with the same or similar types of access and cameras. Summary of the Invention [Problem to be solved by the invention]

[0004] For other athletic events, such as typical recreational activities undertaken by amateur, non-college athletes, video recording and / or broadcasting is not common due to the lack of widespread public demand to watch recreational golf, tennis, basketball, etc. However, participating athletes often desire to have their activities recorded, similar to the recordings offered at professional events. Such recordings can be viewed at a later time for entertainment or evaluation purposes. For example, a golfer may desire to have his swing recorded so that the mechanics of his swing can be analyzed for possible improvements. Similarly, a golfer may wish to record his shots. A basketball player may similarly desire to record his shooting motion and resulting shots, or his movements on the court, in order to evaluate the effectiveness of a particular offensive play or defensive position.

[0005] However, recreational athletes must make their own arrangements to record the event. Athletes may need to provide and configure their own recording equipment and configure their recording equipment in an effective manner to ensure the event is adequately recorded. Alternatively, athletes may need to secure a third-party service that can handle the recording equipment to obtain different views and to accommodate player movements.

[0006] In some cases, athletes may equip themselves with body-mounted cameras to record certain aspects of an event. For example, a head-mounted camera may be used to record a particular cycling route. Rock climbers may similarly use body-mounted cameras to record a particular climbing route. However, these body-mounted cameras usually have limitations, such as the inability to record specific body movements of the athlete. Additionally, head-mounted cameras are typically oriented according to the position of the user's head, so they may only record a limited area that corresponds to what the athlete is looking at. In many cases, the cameras do not even adjust to eye movements or changes in focus, so they cannot record what the athlete is focusing on.

[0007] Some athletic locations, such as basketball courts and indoor tennis courts, where the playing field and area are known and constant, may have pre-installed recording equipment. However, this is typically not possible for many outdoor recreational activities, as there are an infinite number of locations where an athlete may be. For example, on a golf course, it is difficult to reliably predict the path of a golf ball or the location of a golfer on the course, as each location depends on the previous location and the flight of the ball from that previous location. For runners, cyclists, climbers, etc., locations can be even less predictable. Thus, to record these types of activities, recreational athletes typically require another individual to follow the activity and record said activity to provide a usable recording.

[0008] In view of the above, an improvement can be made to an athletic event recording system. [Means for solving the problem]

[0009] Summary of the Invention One aspect of the present disclosure is to provide a system for automatically recording an athletic event.

[0010] Yet another aspect of the present disclosure is to provide a system for automatically processing recordings of athletic events and displaying the recordings with additional information.

[0011] Yet another aspect of the present disclosure is to provide a system for automatically recording golf shots and processing images of the golf shots.

[0012] In view of these and other aspects, a system for automatically recording and processing activities is provided. The system includes a first remote camera disposed at a first geographic location to record video of a first predetermined activity. The system further includes a first processor and memory operatively associated with and in communication with the first remote camera. The first processor and memory are proximate to the first geographic location. The system further includes a local video processing engine associated with the first processor and memory. The local video processing engine is configured to process frames of video captured by the first remote camera of a first user participating in the first predetermined activity. The first processor is further configured to modify the video upon identifying the first user and the first predetermined activity. The system further includes a communication module capable of communicating the formatted video to a remote video processing system. The remote video processing system is configured to further process the formatted video and enable access of the processed video to the first user.

[0013] In one aspect, the processor is configured to detect the location of the first user using one or more of GPS location, facial recognition, object recognition, clothing recognition, motion detection, RFID detection, Bluetooth detection, Wi-Fi detection, radar detection, and heat detection.

[0014] In one aspect, the system includes artificial intelligence (AI) logic accessible to a first processor, the artificial intelligence (AI) logic configured with logic to identify one or more users captured in video frames captured by a first remote camera.

[0015] In one aspect, the first processor is configured to identify a first user, automatically record the first user when the first user is located within a first predetermined area, associate a first recording of the first user with the first user, associate a second recording of the first user with the first user, and process the first and second recordings of the first user to define a formatted video associated with the first user.

[0016] In one aspect, the first processor is configured to detect and identify a second user within the first predetermined area.

[0017] In one aspect, the first processor is configured to automatically record a second user, and the first processor is configured to process the recording associated with the second user.

[0018] In one aspect, the first processor is configured to determine whether the first or second user is to be recorded.

[0019] In one aspect, the first processor is configured to send a message to the first user or the second user indicating that the first user or the second user will be recorded.

[0020] In one aspect, the first processor is configured to generate a location grid over at least a portion of the first geographic location.

[0021] In one aspect, the first processor is configured to detect a location of a first user within a location grid and is further configured to detect a location of an object associated with the first user within the location grid.

[0022] In one aspect, the first processor is configured to monitor a location of the first user while the first user is located within the first geographic location.

[0023] In one aspect, the first processor is configured to add a graphical element to the first processed recording based on data associated with the recording.

[0024] In one aspect, the graphical elements may include one or more graphical elements added to one or more frames of the video, the graphical elements including text information indicating a location in the video, one or more names of the locations where the video was recorded, the name of the first user, the date the video was recorded, a logo or other marketing imagery associated with the locations, colored traces created to indicate graphic lines between frames of moving objects, graphical objects, and augmented reality elements.

[0025] In one aspect, the colored tracing includes creating a trace for one or more of a flight path of a golf ball, a running path of a football player, a flight path of a football, a running path of a soccer player, a flight path of a soccer ball, a path of a downhill skier or snowboarder, a swimming path of a swimmer, a swimming path of a fish caught by the first user, a navigation path of a catamaran or a racing boat, and a cycling path of a mountain bike or a racing bike.

[0026] In another aspect, a system for automatically recording and processing activities is provided. The system includes a first remote camera positioned at a first geographic location to record video of a first predetermined activity. The system further includes a first processor and memory operatively associated with and in communication with the first remote camera. The first processor and memory include: , located near the first geographic location. The system further includes artificial intelligence (AI) logic accessible to the processor. The artificial intelligence (AI) is configured with logic for identifying a user recorded within a video frame captured by the first remote camera. The system further includes a local video processing engine associated with the first processor and the memory. The local video processing engine is configured to process images within the video frames to identify the first user. The first processor is further configured to modify the video upon identifying the first user and the first predetermined activity. The system further includes a communication module capable of communicating the formatted video to a remote video processing system. The remote video processing system is configured to further process the formatted video and enable access to the mobile app for the identified first user.

[0027] In one aspect, the AI ​​logic may be configured to recognize a golfer, a golf ball, a shirt, a shirt color, pants, pants color, a skirt, a skirt color, a hat, a hat color, a golf glove, a golf shoe, a golf cart, one or more people riding in a golf cart, a golf tee, a golf club, an iron, a driver, a utility club, a putter, a wedge, a logo on a golf ball, a male, a female, a child, a youth, a logo on a shirt, a caddie, a facilitator, a brand, a left handed golfer, a right handed golfer, a visor, glasses, sunglasses, a beverage, a tee box, a tee box The logic can identify one or more of the following: color of the field, trees, fairways, cart paths, greens, pins, holes, sand bunkers, water hazards, grass hazards, woods, out of bounds, rough, first cut of the green, second cut of the green, birds, insects, animals, distance from tee to pin, distance from tee to front of the green, distance from tee to center of the green, distance from tee to back of the green, red stakes, white stakes, yellow stakes, changes in elevation, clouds, rain, snow, fog, mist, mud, wind, green topography, or cut of a hole.

[0028] In one aspect, the AI ​​logic further includes logic capable of identifying activities including one or more of a golf activity, a football activity, a soccer activity, a lacrosse activity, a baseball activity, a basketball activity, a tennis activity, a pickleball activity, a bean bag toss activity, a bowling activity, a billiards activity, a swimming activity, a diving activity, a racing activity, a hockey activity, a field hockey activity, a disc golf activity, a rugby activity, a skiing activity, a snowboarding activity, a cycling activity, a fishing activity, a boating activity, and a sports activity.

[0029] In one aspect, the remote video processing system further includes a remote video processing management system in communication with the first processor. The remote video processing management system is configured to receive the first series of recorded videos and process the first series of recorded videos to create AI logic. The processing of the first series of recorded videos includes tagging one or more uniquely identified objects in one or more frames of each of the recorded videos. The tagging includes tags for identifying a user and user activity in the recorded videos. The process further includes creating AI logic including a neural network of the tagged objects and delivering the AI ​​logic to the first processor for use in processing the videos at the predetermined location. The process further includes receiving the formatted video from the communication module and modifying the formatted video based on usage of the mobile app.

[0030] In one aspect, the system includes a remote video processing management system in communication with the video processing system, the remote video processing management system including a network processor coupled to the video processing system and to cloud storage for storing the processed video received from the first remote camera, an artificial intelligence (AI) enabled graphics processing engine configured to access the formatted video and process the formatted video, The system further includes a graphics asset accessible to the AI-enabled graphics processing engine for being added to one or more frames of the generated video, a format manager configured to format the formatted video based on a delivery location for the formatted video, and a delivery manager accessible to the network processor, the delivery manager configured to deliver the formatted video to the delivery location.

[0031] In one aspect, the system includes a mobile app associated with a delivery manager, the mobile app including content provided by the delivery manager, the content including one or more formatted videos output by a remote video processing management system, a list of videos created using one or more activities, a list of locations where the one or more videos were recorded, social media functionality for sharing the formatted videos, and a virtual coaching functionality configured to allow a coach to access and comment on the first user's formatted videos.

[0032] In one aspect, the processor is configured to identify a second user recorded within a video frame captured by a first remote camera. The processor is configured to extract a video frame from the first video, the video frame including the second user. The processor is configured to combine the extracted video frame into a second formatted video, the second formatted video including the second user. The communication module transmits the second formatted video to a remote video processing system. The remote video processing system is configured to further process the video and enable access to the mobile app for the identified second user.

[0033] In one aspect, the remote video processing management system is further configured to identify a second user who is in the vicinity of the first user during the recorded activity, initiate access to the first user of the second video generated by the remote video processing system, and enable the first user to access the second video using the mobile app.

[0034] In another aspect, a method for automatically recording and providing video is provided. The method includes recording video of a predetermined activity using a first remote camera located at a first geographic location. The method further includes processing the video at the first geographic location. The processing includes identifying a first user performing the predetermined activity, extracting image frames from the video including the first user during the predetermined activity, and merging the extracted image frames to generate a formatted video. The method includes outputting the formatted video to a remote video processing system for further processing.

[0035] In one aspect, the method further includes identifying a second user performing a predetermined activity in the video, extracting additional image frames including the second user performing the predetermined activity, merging the additional image frames to generate a second formatted video, and outputting the second formatted video to a remote video processing system.

[0036] In one aspect, the method includes providing a second remote camera at a first geographic location, identifying the first geographic location as a golf hole of the golf course, establishing a first geofence around a tee box of the golf hole, establishing a second geofence around a green of the golf hole, detecting when a first user is within the first geofence, activating recording of the first remote camera and the second remote camera in response to identifying the first user within the first geofence to record the predetermined activity, detecting when the first user is within the second geofence, detecting when the first user leaves the second geofence, and disabling recording of the predetermined activity when the first user leaves the second geofence.

[0037] In one aspect, the first geofence includes a first geofence radius and the second geofence includes a second geofence radius that is different from the first geofence radius.

[0038] In one aspect, the first geofence includes a first geofence size and the second geofence includes a second geofence size that is different from the first geofence size.

[0039] In one aspect, the method includes identifying the predetermined activity as a golf activity; extracting a first image frame to identify a golf ball at a first location in the first image frame; extracting a second image frame at a later time period than the first image frame; determining if the golf ball has moved to another location in the second image frame; drawing a colored line from within the second image frame extending to the first location; repeating drawing the line in subsequent frames until the golf ball is no longer visible; estimating where the golf ball will land; and drawing a colored line to the estimated location where the golf ball will land.

[0040] In one aspect, the method further includes creating AI logic using previously recorded activity to identify the specific activity, identifying the specific activity using the AI ​​logic, identifying a first user performing the specific activity, initiating extraction of image frames of the specific activity, and combining the extracted image frames of the specific activity.

[0041] In one aspect, creating the AI ​​logic further includes identifying golfers holding a golf club in previously recorded video, tagging golfers holding a particular outfit, golf club, and golf ball, repeating the identifying and tagging steps for a number of previously recorded activities and image frames, generating the AI ​​logic using the tagged images, and using the AI ​​logic at the golf course with the first remote camera.

[0042] In another aspect, a method of automatically recording an athletic performance is provided, the method including detecting, by a processor, that at least one player is located within a predetermined area, identifying a first player of the at least one player, automatically recording a performance of the first player with at least one camera operatively coupled to the processor and defining a first recording, automatically storing the first recording in a database operatively coupled to the processor, automatically correlating the first recording with the first player, and automatically processing the first recording and defining a first processed recording.

[0043] In one embodiment, the at least one camera includes a first camera and a second camera. In another aspect, the method includes identifying, with a processor, a first geographic location of the first camera and a second geographic location of the second camera.

[0044] In another aspect, the method includes identifying, with a processor, the predetermined region. In another aspect, the method includes defining a location grid within a given region.

[0045] In another aspect, the method includes detecting an object associated with the first player at a grid location within the location grid.

[0046] In another aspect, the method includes, in response to detecting an object at a grid location, automatically adding a graphic associated with the location to the first processed video recording. Includes.

[0047] In another embodiment, the identity of the first player is determined via image recognition. In another aspect, the method includes communicating, by a processor, to a mobile device associated with the first player a signal instructing the player to perform.

[0048] In another aspect, the method includes automatically transmitting the first processed recording to the first user.

[0049] BRIEF DESCRIPTION OF THE DRAWINGS Other aspects of the present disclosure will be readily appreciated as the same becomes better understood by reference to the following detailed description when considered in conjunction with the accompanying drawings. [Brief description of the drawings]

[0050] [Figure 1] 1 is a block diagram illustrating a video processing system for detecting and recording activities according to one aspect of the present disclosure. [Diagram 2] FIG. 1 is a block diagram illustrating an AI-enabled camera for use with a video processing system, according to one aspect of the disclosure. [Figure 3A] FIG. 1 is a block diagram illustrating an AI-enabled video processing system for local video processing, according to one aspect of the present disclosure. [Figure 3B]1 is a flow diagram of a method for local video processing according to an aspect of the present disclosure. [Figure 4A] FIG. 1 is a block diagram illustrating an AI-enabled video processing system for remote video processing, according to one aspect of the present disclosure. [Figure 4B] FIG. 1 is a flow diagram of a method for processing video using AI-enabled far-field video processing, according to one aspect of the present disclosure. [Diagram 5] 1 is a user interface illustrating a mobile device application according to one aspect of the present disclosure. [Figure 6] FIG. 1 is a block diagram illustrating an example of an AI-enabled video recording system deployed on a golf course, according to one aspect of the present disclosure. [Figure 7] 1 illustrates a method of an embodiment of an athletic activity monitoring system according to an embodiment of the present disclosure. [Figure 8] 1 illustrates a method of an embodiment of an athletic activity monitoring system according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0051] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following description in combination with the figures is provided to aid in understanding the teachings disclosed herein. The following description focuses on specific implementations and embodiments of the teachings. This focus is provided to help explain the teachings and should not be construed as a limitation on the scope or applicability of the teachings. However, other teachings can certainly be utilized in this application. The present teachings can also be utilized in other applications and in several different types of architectures, such as distributed computing architectures, client / server architectures, middleware server architectures and related components.

[0052] Devices or programs that are in communication with each other need not be in continuous communication with each other unless specifically specified, and furthermore, devices or programs that are in communication with each other may communicate directly or indirectly through one or more intermediaries.

[0053] The embodiments described below in part describe a distributed computing solution that manages all or part of the communication interactions between network elements. In this context Thus, a communication interaction may be an attempt to transmit information, a transmission of information, a request for information, a receipt of information, a receipt of a request for information, or any combination thereof. Thus, a communication interaction may be unidirectional, bidirectional, multidirectional, or any combination thereof. In some situations, a communication interaction may be relatively complex and involve two or more network elements. For example, a communication interaction may be a "conversation" or a series of related communications between a client and a server, in which each network element sends and receives information to and from the other elements. A communication interaction between network elements is not necessarily limited to only one particular form. A network element may be a node, hardware, software, firmware, middleware, another component of a computing system, or any combination thereof.

[0054] For purposes of this disclosure, an athletic monitoring and recording system may include any means or collection of means operable to compute, classify, process, transmit, receive, search, originate, switch, store, display, manifest, detect, record, play, handle, or utilize any form of information, intelligence, or data for business, scientific, control, entertainment, or other purposes. For example, an athletic monitoring and recording system may be a personal computer, a PDA, a consumer electronic device, a smart phone, a cellular or mobile phone, a set-top box, a digital media subscriber module, a cable modem, a fiber optic enabled communication device, a media gateway, a home media management system, a network server or storage device, a switch router, a wireless router, other network communication device, or other suitable device and may vary in size, shape, performance, functionality, and price.

[0055] The system may include memory, one or more processing resources or controllers, such as a central processing unit (CPU), hardware or software control logic, etc. Additional components of the system include one or more storage devices, one or more wireless, wired, or any combination thereof, communication ports for communicating with external devices and various input / output (I / O) devices, such as a keyboard, mouse, and video display. The athletic monitoring and recording system may also include one or more buses operable to transmit communications between the various hardware components.

[0056] In the following description, a flow-charted methodology or algorithm may be described as a series of sequential actions. Unless otherwise indicated, the order of the actions and the parties performing the actions may be freely modified without departing from the scope of the teachings. Actions may be added, removed, or modified in some manner. Similarly, actions may be reordered, looped, etc. Additionally, while a process, method, algorithm, or the like may be described in a sequential manner, such a process, method, algorithm, or any combination thereof may be operable to be performed in an alternative order. Additionally, some actions within a process, method, or algorithm may be performed simultaneously, at least at some point (e.g., actions performed in parallel), and may also be performed in whole, in part, or in any combination thereof.

[0057] As used herein, the terms "comprises," "comprising," "includes," "including," "has," "having," or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus that includes a list of features is not necessarily limited to only those features, but may include other features not expressly listed or inherent in such process, method, article, or apparatus. Further, unless expressly stated to the contrary, "or" refers to an inclusive or and not an exclusive or. For example, condition A or B may be any of the following: A is true (or exists) and B is false (or does not exist), A is false (or does not exist) and B is true (or exists), and both A and B are true (or exist).

[0058] Also, the use of "a" or "an" is used to describe elements and components described herein. This is done merely for convenience and to give a general sense of the scope of the invention. This description should be read to include one or at least one, and the singular includes the plural and vice versa unless otherwise clearly meant. For example, where a single device is described herein, a plurality of devices can be used in place of the single device. Similarly, where a plurality of devices is described herein, a single device can be used in place of the devices.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention belongs. Although methods and materials similar or equivalent to those described herein can be used to practice or test embodiments of the present invention, suitable methods and materials are described below. All publications, patent applications, patents, and other references mentioned herein are incorporated by reference in their entirety unless a specific section is cited. In case of conflict, the present specification, including definitions, will control. Additionally, the materials, methods, and examples are illustrative only and are not intended to be limiting.

[0060] To the extent not described herein, many of the details regarding specific materials, process acts, and circuits are conventional and can be found in textbooks and other sources in the computing, electronics, and software arts.

[0061] Within the description, the use of artificial intelligence (AI) or AI logic, machine learning, and neural networks are also used. AI or AI logic includes several categories of techniques that allow computers to mimic human capabilities. AI techniques or logic include machine learning, speech and language processing, expert systems, and robotics. Machine learning is a subset of AI that allows computers to improve tasks through experience. Machine learning includes traditional statistically-based approaches such as regression analysis, and newer techniques such as deep learning. Deep learning uses large amounts of historical data to train multi-level neural networks to draw conclusions about new data. Throughout the specification, the description uses AI logic that deploys deep learning in the form of neural networks to identify classes of objects, object locations in video images and segments. Deep learning is also used to identify characteristic activities or sub-activities in video images and segments. In some forms, statistically-based machine learning is used to characterize the movement or direction of objects in video images and segments.

[0062] Next, exemplary aspects of the autonomous recording and processing system will be described more completely. Each of these exemplary aspects is provided so that this disclosure will be thorough and will fully convey the scope of the concepts, features, and advantages of the present invention to those skilled in the art. To this end, numerous specific details are shown, such as examples of specific components and methods related to the system, to provide a thorough understanding of each aspect related to the present disclosure. However, as will be apparent to those skilled in the art, it is not necessary to use all of the specific details set forth herein, and the exemplary aspects can be embodied in many different forms, and therefore should not be interpreted or understood to limit the scope of the disclosure.

[0063] Various aspects of the present disclosure generally involve hardware, software, and / or hardware components distributed across various systems. "com. NVIDIA Corporation" may refer to a set of hardware, software, modules, etc. that can be used to facilitate the features and functionality described herein, including an NVIDIA Jetson TX2 computer, a 256-core NVIDIA Pascal GPU architecture with 256-core NVIDIA CUDA, and a dual-core NVIDIA Denver 2 64-bit CPU and a quad-core ARM Cortex-A57 MPCore with 8GB 128-bit LPDDR4 memory and 32GB The software includes, but is not limited to, a Linux operating system with applications programmed in Python, OpenCV image processing libraries, AWS Greengrass ML model development and execution, video editing software using OpenCV image processing libraries and Python programming. Various cloud services can be used for storing and transmitting the video, as well as AWS S3 and AWS Glacier for video storage, and AWS CloudFront for content delivery and distribution. Cloud services for processing and editing the video may include Python and OpenCV running on AWS EC2 servers. Cloud services for converting video from one format to another may include AWS Elemental MediaConvert. Cloud services for generating neural networks and AI may include AWS SageMaker for building, training, tuning, and evaluating machine learning models, including Keras / TensorFlow development frameworks, and Sagemaker NEO for preparing models for deployment on local computers.

[0064] The cameras for use in the systems described herein can be HD cameras or 4K cameras.

[0065] The 4K camera may be a Hanwha model PNP-9200RH, whose specifications and operating manual are incorporated herein by reference. The Hanwha camera, model PNO-9200RH, is a 4K PTZ camera with the following specifications:

[0066] Imaging device sensor: 1 / 2.5” 8MP CMOS Resolution: 3,840(H) x 2,160(V), 8.00M pixels Focal length (zoom ratio): 4.8-96mm (optical 20x) Angle of view H / V: 65.1 degrees (wide) - 3.8 degrees (telephoto) / 38.4 degrees (wide) - 2.2 degrees (telephoto) Autofocus & Auto Iris Infrared lighting 120dB Dynamic Range Pan range / speed: 360° / 400° / sec Tilt range / speed: 190° / 300° / sec 16x digital zoom Application Programming Interface: ONVIF Profile S / G Video Compression Format: H.265 / H.264, MJPEG Maximum frame rate in all resolutions H.265 / H.264: 30fps Audio input selectable (microphone input / line input) Ethernet: 10 / 100 BASE-T Operating temperature / humidity: -58°F~+131°F / less than 90%RH Ingress Protection: IP66 / Vandal Resistant IK10 Input voltage: 24V AC Power consumption: 90W (Heater on, IR on) The camera may be an HD camera capable of recording in high resolution. The camera may be a Hanwha HD 1080p PTZ camera, model number XNP-6321H, with specifications and operating manuals which are incorporated herein by reference. The Hanwha camera, model XNP-6321H, is an HD 1080p PTZ camera with the following specifications:

[0067] Imaging device: 1 / 2.8” 2.4M CMOS Resolution: 1,981(H)x1,288(V), 2.55M Focal length (zoom ratio): 4.44-142.6mm (32x optical) View angle H / V: 61.8 degrees (wide) - 2.19 degrees (tele) / 36.2 degrees (wide) - 1.24 degrees (tele) Autofocus & Auto Iris IR Illumination 150dB Dynamic Range Pan range / speed: 360° / 700° / sec Tilt range / speed: 210° / 700° / sec Digital zoom: 32x Application Programming Interface: ONVIF Profile S / G Video Compression Format: H.265 / H.264, MJPEG Maximum frame rate H.265 / H.264: 60fps in all resolutions Audio input selectable (microphone input / line input) Ethernet: 10 / 100 BASE-T Operating temperature / humidity: -31°F~+131°F / less than 90%RH Ingress Protection: IP66 / Vandal Resistant IK10 Input voltage: 24V AC or POE+ Power Consumption: Max. 24W (Heater off), Max. 65W (Heater on, 24V AC) Referring now to FIG. 1, a block diagram illustrating a video processing system for detecting and recording activities is provided. The video processing system is generally illustrated in system 100. For purposes of illustration, various embodiments and aspects of system 100 are described and illustrated herein, with various system modules distributed across different interconnected systems, hardware, and software, which communicate with each other in wired or wireless manners both locally and remotely via the Internet / Cloud. Various functions of system 100 described herein can be accomplished using a computer including a processor and a non-transitory computer-readable medium or memory, and instructions stored on the computer are executed by the processor. System 100 can function automatically according to rules set forth in various algorithms. It will be further understood that various processors, memories, and instructions can be distributed between various systems and via the Cloud, and some instructions or processing can occur remotely and be communicated between systems or modules.

[0068] According to one aspect, the system 100 can include various combinations of additional cameras, generally denoted as a first remote camera 102, a second remote camera 104, a third remote camera 106, or an Nth remote camera 108. The system 100 also includes a network switch 110 that connects one or more of the remote cameras 102-108 to an AI-enabled video processor 112. The system 100 includes a non-transitory memory 111 connected to the AI-enabled processor 112. The remote cameras 102-108 can be operatively connected to the network switch 110 and controlled by the AI-enabled video processor 112. In other aspects, the cameras 102-108 can operate independently of on-board capabilities for recording video, as described herein.

[0069] The system 100 may also include a modem 114, such as a cellular or hardwired modem, configured to communicate or transmit data over a network, such as the Internet / Cloud 116. The modem 114 may be a 3G, 4G, 5G, or or other communications standards. In another embodiment, modem 114 may be a wired modem capable of communicating using a broadband connection such as Ethernet, or via a fiber optic connection, or various combinations thereof.

[0070] According to one aspect, the modem 114 may be configured to communicate raw video data captured by the remote cameras 102-108 for further processing, or the modem 114 may be configured to communicate processed video created by the AI-enabled processor 112. Thus, according to one aspect, the modem 114 may be configured to communicate with or operatively communicate with the Internet / Cloud 116.

[0071] According to further aspects, the system 100 may further include a remote video processing management system 118 connected to the modem 114 via the Internet / Cloud 116. The remote video processing management system 118 may automatically process the videos and manage distribution of the created video files. The system 100 may further include a content management distribution system 120 in communication with the remote video processing management system 118. The content management distribution system 120 may be configured to further control distribution of the videos created by the system 100.

[0072] For example, in response to receiving the raw video data at the remote video processing management system 118, the system 100 may be configured to automatically process the video according to predetermined instructions. In response to processing the video, the content management and delivery system 120 may receive the processed video and, in response, transmit or make the processed video available to end users. Thus, the system 100 may use video processing located near the remote camera to detect video activity and process the video in one or more forms, either locally to the camera, in a cloud service, or a combination thereof.

[0073] According to one aspect, the system 100 can be used to detect golfers in a golf environment and record and communicate the golfer's activities in the form of processed video. During use of the system 100 in a golf environment, the system 100 can be configured to automatically capture video data of the golfer via the cameras 102-108. In response to capturing the video data, the system 100 can receive the video data from the cameras 102-108 at the AI-enabled processor 112. The AI-enabled processor 112 can automatically process the video data and create a processed video. For example, the AI-enabled processor 112 can be used to detect golfers using image data from the video and a neural network created to detect people in the video or image frames. Upon detecting a person, a neural network (NN) can be created to identify other elements of the golfer, such as the golfer's clothing, shoes, hat, golf club, golf ball, or various other aspects or combinations of aspects specific to golfing activities. Upon identifying a golfer, the system 100 can capture and process video for that particular golfer. In response to creating the processed video, the AI-enabled processor 112 can transmit the video data. The processing of the video data can occur locally or remotely. For example, the AI-enabled processor 112 can be part of the camera 102-108, can be in a computer or processing system in communication with the camera 102-108, or can be in the cloud.

[0074] According to another embodiment, the system 100 may be configured to detect the presence of one or more golfers within a predetermined area associated with the cameras 102-108. For example, the system 100 may detect the presence of one or more golfers within a predetermined area associated with the cameras 102-108 via a signal transmitted to the AI-enabled processor 112. The system 100 is configured to detect when a golfer has reached a hole on a golf course that has been designed and calibrated. The system 100 is further configured to detect when one or more golfers have completed and left a hole. In this manner, a limited amount of video can be captured and recorded only when a golfer is present, thereby reducing the amount of memory and processing required to store, process, and communicate the video.

[0075] According to further embodiments described in more detail below, a golfer may associate himself with a transmitter, such as a mobile device, a GPS or location services enabled device such as a smartphone or tablet, or other GPS or location services enabled device, an RFID device, a Bluetooth enabled device, or any combination thereof, that can communicate with system 100 to indicate the golfer's presence. In another form, system 100 may be configured to communicate with golf carts that have a GPS or location services tracking system integrated within the golf cart to detect the location of the golf cart and the golfer.

[0076] Thus, for example, prior to automatically recording video with cameras 102-108, system 100 may receive a proximity signal from a device associated with a golfer. In response to receiving the proximity signal, system 100 may automatically begin recording the hole via the cameras.

[0077] According to another aspect of the disclosure, one or more of the remote cameras 102-108 may include GPS or location services positioning capabilities, such as a GPS or location services device installed at one or more of the remote cameras 102-108, such that the GPS coordinates of the remote cameras 102-108 may be known. The GPS or location services device may be used to detect and transmit to the system 100 the specific GPS coordinates of each camera 102-108, thereby providing a reference point to the system 100 providing the system 100 with location data for each remote camera 102-108 such that other objects on the golf hole and within the field of view of the remote cameras 102-108 may be detected by the cameras 102-108. Thus, the system 100 may triangulate to determine the location of the objects relative to the remote cameras 102-108.

[0078] According to another embodiment, during an initial setup phase of the system 100 at a golf course, the remote cameras 102-108 may be installed at fixed locations determined by the installer of the golf hole depending on the particular layout of the golf hole. As is typical, each golf hole is unique and has a unique layout between the tee box and the green and the areas in between including different grass cuts (fairway, first cut, rough, fringe, etc.). In one embodiment, the first camera 102 may be located behind the tee box and the second camera 104 may be located behind the green. When installed, the remote cameras 102-108 are typically located in positions that are not in the expected path of ball flight. In other words, the first remote camera 102 may be located behind the golfer as he stands on the tee box and facing the green, and the second remote camera 104 may be located beyond the green in a direction relative to the tee box. Thus, the remote cameras 102-108 are positioned such that impact from a golf ball is unlikely. It will be appreciated that the system 100 may include multiple different cameras in a variety of different locations.

[0079] 2, a block diagram is provided illustrating an AI-enabled camera for use in a video processing system. The AI-enabled camera, generally designated 200, includes an on-board processor 204 and one or more sensors 206, 208, 210, which may be configured to send signals and / or video to the processor 204.

[0080] The sensors 206-210 may include optical sensors and may include various types or combinations of sensors in any form, including but not limited to optical, motion, infrared, radar or Doppler sensors, Bluetooth sensors, Wi-Fi sensors, RFID sensors, or various combinations thereof. The camera 202 may further include a memory 212 in communication with the processor 204 and an embedded AI module 214 in communication with the memory 212. The AI ​​module 214 may be in the form of software stored in the memory 212 and executed by the processor 204.

[0081] The camera 202 may further include a communication module 216 in communication with the processor 204 and a power module 218. The communication module 216 may be wired or wireless communication, such as fiber optics, Ethernet, power over Ethernet (POE), etc. The wireless communication may be Wi-Fi or other 802.11 communication, Bluetooth, cellular communication, or various combinations thereof. The power module 218 may be configured to provide power to the camera 202. The power module 208 may be in the form of a battery or a hardwired input connected to a power grid or other power source. According to one aspect, power may be provided using a POE connection sufficient to power the camera 200 over a given distance. Thus, with reference to the automatic video recording and processing steps described herein, the camera 202 may perform these steps, and in response to creating the processed video, the camera 202 may transmit the processed video to an end user.

[0082] The camera 202 may further include a control module 220 in communication with the camera sensors 206-210. The control module 220 may be in communication with the processor 204, receive signals from the sensors 206-210, and provide signals to the processor 204 to control the camera 202. The control module 220 may be configured to pan, tilt, or zoom the camera 202. The camera 202 may be used with any other camera described herein. The various cameras described herein may include some or all of the functionality associated with the camera 202. For example, the various cameras described herein may include pan, tilt, and zoom functionality but may not include on-board AI processing, for example. For example, in one embodiment, in response to detection of one or more golfers, the camera 202 may generate a control signal for the control module 220 to automatically tilt / pan, or zoom the camera 202.

[0083] According to one embodiment, the camera 202 may be a Hanwha 4K camera, model PNP-9200RH, having specifications and operating manuals incorporated herein by reference. The camera 202 as a Hanwha camera, model PNO-9200RH, is a 4K PTZ camera with the following specifications:

[0084] Imaging device sensor: 1 / 2.5” 8MP CMOS Resolution: 3,840(H) x 2,160(V), 8.00M pixels Focal length (zoom ratio): 4.8-96mm (optical 20x) Angle of view H / V: 65.1 degrees (wide) - 3.8 degrees (telephoto) / 38.4 degrees (wide) - 2.2 degrees (telephoto) Autofocus & Auto Iris Infrared lighting 120dB Dynamic Range Pan range / speed: 360° / 400° / sec Tilt range / speed: 190° / 300° / sec 16x digital zoom Application Programming Interface: ONVIF Profile S / G Video Compression Format: H.265 / H.264, MJPEG Maximum frame rate in all resolutions H.265 / H.264: 30fps Audio input selectable (microphone input / line input) Ethernet: 10 / 100 BASE-T Operating temperature / humidity: -58°F~+131°F / less than 90%RH Ingress Protection: IP66 / Vandal Resistant IK10 Input voltage: 24V AC Power consumption: 90W (Heater on, IR on) Alternatively, camera 202 may be provided as an HD camera capable of recording in high resolution. Thus, camera 202 may include a Hanwha HD 1080p PTZ camera, model number XNP-6321H, with specifications and operating manuals incorporated herein by reference. Camera 202 as a Hanwha camera, model XNP-6321H, is an HD 1080p PTZ camera with the following specifications:

[0085] Imaging device: 1 / 2.8” 2.4M CMOS Resolution: 1,981(H)x1,288(V), 2.55M Focal length (zoom ratio): 4.44-142.6mm (32x optical) View angle H / V: 61.8 degrees (wide) - 2.19 degrees (tele) / 36.2 degrees (wide) - 1.24 degrees (tele) Autofocus & Auto Iris IR Illumination 150dB Dynamic Range Pan range / speed: 360° / 700° / sec Tilt range / speed: 210° / 700° / sec Digital zoom: 32x Application Programming Interface: ONVIF Profile S / G Video Compression Format: H.265 / H.264, MJPEG Maximum frame rate H.265 / H.264: 60fps in all resolutions Audio input selectable (microphone input / line input) Ethernet: 10 / 100 BASE-T Operating temperature / humidity: -31°F~+131°F / less than 90%RH Ingress Protection: IP66 / Vandal Resistant IK10 Input voltage: 24V AC or POE+ Power Consumption: Max. 24W (Heater off), Max. 65W (Heater on, 24V AC) Thus, the camera 202 may be embodied as a variety of different types of cameras and may be deployed and used in conjunction with the various video processing systems and methods described herein.

[0086] Referring now to FIG. 3A, a block diagram illustrating an AI-enabled video processing system is provided. The embedded AI video processing system, generally shown as system 300, includes a processor 302 and a memory 303. System 300 may include an NVIDIA JetsonTX2 system for processing and controlling a local video camera. Processor 302 may include a dual-core NVIDIA Denver 2 64-bit CPU and a quad-core ARM® Cortex®-A57MP Core, and memory 303 may include 8GB 128-bit LPDDR4 memory and may also include 32GB eMMC storage. System 300 may also include an AI-enabled graphics processor 316, which may include a 256-core NVIDIA Pascal™ GPU architecture with 256-core NVIDIA CUDA. Operating software for system 300 includes Li The system 300 may include an Intel NVIDIA NVIDIA GeForce GTX 1080 ...

[0087] The system 300 further includes a remote camera interface 304 in communication with the processor 302. The remote camera interface 304 may be connected to the network switch 110 shown in FIG. 1, or other interface / communication mechanism connecting the camera processor 302. The system 300 further includes a power module 306, which may be configured to provide power to the processor 302 and other components of the system 300. The power module 306 may be in the form of a battery or may be a hardwired connection to another power source, such as a power grid or an existing power source. The remote camera interface 304 may be used to provide power to a remote camera connected to the remote camera interface 304. According to one embodiment, the remote camera interface 304 may be an Ethernet interface, and each camera (not shown) may be powered and controlled using a PoE connection. Other forms of connection may also be used, including but not limited to fiber optics, coaxial cable, twisted pair, single strand, or various combinations thereof.

[0088] The system 300 may further include a communication module 308 connected to the processor 302 and a modem, such as a cellular modem 310, configured to transmit data. The modem 310 may be the modem 114 of FIG. 1, or other forms of modem as appropriate. The modem 310 may communicate with a wired or wireless network that may connect the Internet or cloud-based services. The communication module 308 may be used to determine a location or address for communication via the modem 310, and may further receive data or instructions for the processor 302 via the modem 310.

[0089] Additionally, according to one aspect, the system 300 may further include an AI-enabled digital video recorder 312 and a local video processing engine 314, each of which may be connected to the processor 302 and may receive control signals from the processor 302. The video recorder 312 and the processing engine 314 may combine to receive raw video, store the raw video, and then automatically process the raw video to detect particular objects and create particular types of videos described herein.

[0090] In use, the system 300 can configure and control the camera using the remote camera interface 304 to capture and record video. The AI-enabled graphics processor 316 can perform object detection on the recorded video to identify predefined activities. If the predefined activities are valid, the system 300 can process the video into segments for the detected user. The system 300 can edit the segments as needed and combine the segments into a video file uniquely for the detected user. The video file can then be uploaded to the internet or cloud using the communication module 308 and cell modem 310. In this manner, local video processing using AI capabilities can be deployed to specific locations to reduce the overall file size of the video file, generate user-specific content, and efficiently communicate the video file to the internet or cloud for quick access.

[0091] According to further aspects, the system 300 provides AI video processing locally near the geographic location of installed remote cameras to provide automated monitoring and processing of desired activities. Thus, automatic detection and video recording / processing may be performed by the system 300, and the processed video may be transmitted to an end user. The processed video may be received and forwarded to the end user via an intermediate server or other communication device.

[0092] However, in alternative embodiments, system 300 can use remote AI video processing, as described further below. Even when processing video remotely, the video can be recorded locally by one or more remote cameras. The recorded video can then be communicated by system 300 to a remote processing system (not explicitly shown), where additional graphics can be automatically edited and incorporated using a remote video processing system, such as remote video processing system 400 described below.

[0093] FIG. 3B is a flow diagram of a method of local video processing according to one embodiment of the present disclosure. The method may be used to perform the method described in FIG. 3B by one or more of a system, device, processor, module, software, firmware, or other various forms of processing. Furthermore, FIG. 3B may be implemented as in various portions of FIG. 1-FIG. 3A, FIG. 4-FIG. 8, and in some embodiments may be modified to include various functions, uses, and features described therein.

[0094] The method generally begins at step 301 and can be used at a variety of geographic locations where a predetermined activity is performed. As an example, the method can be used for local processing and video capture at a golf course as described below, although other activities can be described in connection with the method of FIG. 3B. Once a presence is detected, the method proceeds to step 303. For example, a golfer can be detected as he approaches a tee box in connection with playing a golf hole. Detection can be done in a variety of ways, including, but not limited to, using reception of a signal from another transmitter, such as GPS or RFID, Wi-Fi, Bluetooth, location based services in a mobile device, or other sensors to detect presence, including motion detection, RFID detection, Bluetooth detection, Wi-Fi detection, radar detection, temperature or heat sensing, or other various sensing technologies. In addition to detecting the presence of individual golfers, similar transmitters may be provided on golf carts, etc., to indicate the presence of one or more golfers. It will be appreciated that other detection mechanisms can also be used.

[0095] Upon sensing a presence, the method proceeds to step 305 to determine if the user is valid. For example, the detected presence may be a deer on the golf course or may be a jogger or pedestrian. Thus, various types of techniques may be used to detect the golfer at step 305. For example, in one form, a golfer location service on a mobile device having an application for recording video may be detected. For example, a geofence may be placed on a tee box, and when a valid golfer approaches the tee box, the geofence is triggered to validate the user. Other forms of validation may also be used, such as AI logic that includes facial recognition of the golfer and is detected by a camera at the location. In another form, the AI ​​logic may include an object recognition neural network that can identify objects unique to the golfer. For example, the AI ​​logic may identify a person and a golf club on the tee box. Other forms of object recognition may be used to identify the golfer, as described herein.

[0096] If the user is not valid, the method proceeds to step 303 until another presence is detected. If a valid user is detected at step 305, the method proceeds to step 307 to begin recording the activity. For example, multiple cameras may be present at a golf hole or other activity location and may be used to record the activity. As such, once a valid user is detected, a remote camera associated with the geographic location may begin recording the event. In one form, the recording can also be initiated by an individual. For example, a golfer may have a mobile device with an app associated with a geolocated camera. Thus, the golfer can step onto the green and begin recording the golfing activity. In this manner, the recording can occur automatically or with the user initiating the recording.

[0097] The method then proceeds to identify the user at step 309. For example, at step 309, a golfer may be detected and at block 309, the user may be identified during the recording. For example, the method may use image processing to identify the clothing the user is wearing, or in other forms, AI logic may be used to identify a particular user using facial recognition. Once the user is identified, the method may proceed to step 311 and identify an activity. For example, a golfer holding a golf club may be used to identify the activity in one or more frames of the video segment. In some forms, other individuals holding golf clubs who do not subscribe to the services provided by the method of FIG. 3B may appear at the tee box. As such, if an invalid user is swinging a golf club, the method will not consider the activity to be valid and proceed to step 303. In other forms, a maintenance worker may be present at the tee box and working on the tee box. The method will identify that activity and at step 313, using AI logic with valid activity, will dismiss the activity as invalid. The method will then proceed to step 303 until another presence is detected. If valid activity is detected at step 313, the method proceeds to step 315 and continues recording video.

[0098] As recording continues, the method checks at step 317 to determine if the user is still present at the geographic location. For example, the user may be a golfer and may hit the ball multiple times before completing a hole. Thus, when the user leaves a predetermined area of ​​the geographic location, such as the green, recording ends. If the user has not exited or moved away, the remote camera continues recording. If the user is no longer present or is not detected at step 317, the method proceeds to step 319 where video is processed for the user.

[0099] At block 319, local processing of the video can include various AI logic processing, image processing, formatting and compression techniques as described herein. According to one aspect, the video can be segmented into portions that include only the identified user. The segmentation can occur using AI logic or image processing to verify the user within the segment. Once the segment is created, additional information can be added to the video segment, such as a ball trace or predefined graphics. The video segments can then be merged and formatted into a format that can be used by the end user. According to another aspect, the final video can be compressed before being communicated from the local video processor. In other formats, various portions of the processing can be added or removed depending on the desire of the method of performing the local processing.

[0100] Once the video has been processed, the method proceeds to step 321 to detect if another user is present. For example, the video may show a snowboarder descending a trail with several friends. The method then detects another valid user in the video and extracts a segment of the video where the second snowboarder is present. Once segments have been extracted for the additional user, the method may proceed to step 323 to process the video for the second user. For example, a processed video for multiple snowboarders may include a trace or colored line detailing where the second snowboarder descended relative to the first snowboarder. In this manner, multiple users may be detected using the same video and a unique video segmented for that user. Although shown as only a second user, it should be understood that the video can be processed to detect multiple additional users as needed or desired. Upon processing the second user video, the method proceeds to step 325 and communicates the formatted video and proceeds to step 327 where the method ends.

[0101] 4A, a block diagram is provided illustrating an AI-enabled video processing system for remote use according to one aspect of the disclosure. The AI-enabled video processing system, generally shown as a remote video processing and management system or remote processing system 400, includes a network processor 402 connected to a cloud storage and services 404, which is connected to a remote video interface 406 that can communicate video from a remote camera (not explicitly shown). The network processor 402 can access various modules for managing and processing video received from the remote video interface 406. For example, the network processor can access a remote video manager 416, a content manager 418, a profile manager 420, a format manager 422, and an output manager 424. Each enumerated manager can be provided as a software module or program interface that can be accessed as needed by the network processor 402.

[0102] The network processor 402 may also be realized as a cloud service that may be deployed using Amazon Cloud Services, IBM Cloud Services, Microsoft Cloud Services, or various combinations thereof. The cloud storage and services 404 and the delivery manager / communications 408 may also include various types of cloud storage and delivery services with different storage capabilities and accessibility. For example, some content may be stored for immediate access, while other forms of content can be stored for delayed access using deep storage techniques. This allows for flexible access to content such as videos while reducing the overall cost per user. For example, if a user chooses to pay for long-term storage, the image processing system can change the type of storage per rate. Thus, the cloud storage and services 404 may include various different types of online services and, according to one aspect, may include Amazon Web Services (AWS) Glacier for storing videos in the cloud. Additionally, the content manager 418 and the delivery manager / communications 408 may utilize AWS Cloudfront as a content delivery service to deliver videos to end users.

[0103] The remote system 400 may also include an AI-enabled graphics processing engine or GPU 410. The GPU 410 may include various types of AI-enabled processors, and in one form may include one or more NVIDIA V100 Tensor Cores capable of AI processing for generating, developing, and training machine learning (ML) for the AI ​​logic 412. Including the GPU, the AI ​​logic 412 can be created, modified, distributed, and used by the system 400 or other AI-enabled processors described herein. According to one embodiment, the GPU 410 and / or network processors can also utilize additional software and services to create the AI ​​logic 412. For example, the GPU 410 can use AWS SageMaker to build, train, tune, and evaluate ML models. Sagemaker supports many ML development frameworks, and according to one embodiment, may use Keras / TensorFlow. Additionally, the system 300 can use Sagemaker NEO to prepare the AI ​​logic 412 models for deployment to remote processors, as shown in Figures 1-3 and 6.

[0104] According to one embodiment, the GPU 410 can access graphic assets 414 that can be added to the video being processed using the network processor 402. The GPU 410 can also access AI logic 412, which can include various stored AI-enabled logic designed to automate various aspects of processing video autonomously in the remote processing system 400 and local processing system 300, camera 200, or various other processing systems and devices provided herein. For example, the AI ​​logic 412 can be created using various videos created during a particular activity, such as a golf activity, a football activity, a soccer activity, a baseball activity, a basketball activity, a skiing activity, a snowboarding activity, a cycling activity, a fishing activity, a boating activity, a general sporting activity, or various other types of non-sporting activities that are pre-determined to occur in a geographic location. The AI ​​logic 412 can process image data previously recorded within the video frames and can be used to tag objects within the video that are significant to or related to the activity. For example, the AI ​​logic 412 may be used to tag a soccer ball being used within a soccer field, but may not be used to tag a bird flying above the soccer field. Additionally, soccer player numbers and names may be processed and tagged in a manner that helps identify videos in which particular players may be present and processed accordingly. AI logic 412 created for a particular activity may be shared with a local video processing system or camera as described herein. In other forms, the AI ​​logic 412 may be stored locally on the remote video processing system 400 for use or distribution as needed.

[0105] According to further aspects, the system 400 can be used to post the processed video received from the remote video interface 406. For example, the video can be modified or edited to add additional assets 414 or can be formatted using a specific format provided by the format manager 422. As such, the system 400 can employ additional software for post-processing and editing, including the use of Python and OpenCV for editing the video on an AWS EC2 web server. The system 400 can also utilize AWS Elemental MediaConvert to convert or format the video before distribution using the distribution manager / communications 408.

[0106] Referring now to FIG. 4B, a flow diagram of a method for processing video using AI-enabled remote video processing according to one embodiment of the present disclosure is shown. The method may be used by one or more of a system, device, processor, module, software, firmware, or other various forms of processing to perform the method described in FIG. 4B. Furthermore, FIG. 4B may be implemented as within various portions of FIGS. 1-3B and 5-8, and in some embodiments may be modified to include various functions, uses, and features described therein.

[0107] The method generally begins at step 401. At step 403, once video is received from a remote video source, the method proceeds to step 405 where activity within the video is identified. The various activities described herein can be stored within various AI logics created using machine learning as neural networks. Portions of the video can be compared to the AI ​​logic, and if no activity is detected, the method proceeds to block 407 where the video can be processed to identify new activities. In some forms, the processing can include tagging or identifying objects within the video that are specific to the activity and can be used by machine learning for one or more activities. Once the video has been processed, the method proceeds to block 409 where it is determined whether a new activity needs to be created within the neural network. For example, the AI ​​logic described herein can be compared to the AI ​​logic. While various activities as described may be identified, in some forms sub-activities within the activity category may also be created. Examples of such activities may include, in one form of golf activity, a golfer slicing or hooking the ball, a golfer swinging a club, a golfer high-fiving another golfer, a golfer making a hole-in-one, or various other activities or sub-activities that may be created. If an activity needs to be created, the method proceeds to step 411 to identify an object or set of objects that may be used and present within an image frame of the video. The method proceeds to step 423 to label the identified object and then to block 425 to add the object or frame to the AI ​​logic of the activity. In some forms, if the activity exists, the object may be added to the neural network of the activity, and in other forms, if the activity does not exist and a neural network is not available, the method may generate a new neural network and machine learning instance to be used within the AI ​​logic. The method then proceeds to step 417 to process the AI ​​logic and proceeds to step 429 to determine if the activity is valid and can be released within the AI ​​logic. For example, the accuracy of the neural network may include a dependency on the number of objects identified and provided to the machine learning instance for that activity. If there is only one instance, the AI ​​logic may fail. As additional objects are identified and used within the machine learning instance, the likelihood that the AI ​​logic will identify the activity increases statistically. If additional objects for the activity are required, the method proceeds to step 401 until additional video is received. If the activity is enabled at step 419, the machine learning instance may enable the AI ​​logic for that activity at step 421 and proceed to step 423 to distribute the AI ​​logic to various locations as needed. The method then proceeds to step 425 and ends.

[0108] If activity is identified in step 405, the method proceeds to step 427 to determine whether the video is valid for output or storage. For example, a local video processor may have processed enough of the video for distribution. Thus, a remote video processor, such as system 400, may use the data provided with the video to detect whether the video requires additional processing. If the video is valid for output, the method may proceed to step 429 to format the video using a format manager. For example, the video may need to be formatted for output to a mobile device or application that has a specific format, file size, and other specifications required in connection with posting the video. Videos provided to various locations and applications include Facebook, YouTube, Instagram, Snapchat, and other applications. Each utilized app may require its own format for publishing to a particular network. As such, the format manager may determine one or more locations for the video and format it accordingly. In other formats, the video may be processed and distributed to a network location with a high resolution or 4K video output on a fixed output device, such as a particular monitor. Various types of formats may be used to output the video to various destinations. Once the video is formatted, the method proceeds to step 431 where a distribution manager is used to distribute the formatted video. For example, the video may be a single instance that is distributed to a cloud storage account configured to store the video. However, in other formats, the video may be formatted into multiple formats, thus creating multiple videos that may require distribution. Thus, in step 431, the videos are distributed to their destinations. The method then proceeds to step 425 where it ends.

[0109] If the video is not valid for output at step 427, the method proceeds to process the video. For example, the method considers three different types of processing that may be used to process the video: , which are provided only as a reference to illustrate the processing of the video, and are not in any particular order. At block 433, the method determines whether one or more user processes need to be performed. For example, a local video processor may have provided information about a particular user recorded in the video. That information may then be used to identify the user in the video. Various types of identification may be used, such as facial recognition, geofencing, GPS or location based services location identification, grid identification, manual input from the user's mobile app, or various other triggers that may be used to identify a particular user in the video. The method may also use AI logic to identify a particular user and provide characteristics, details, and / or objects of that user along with the video. Upon identifying the user, the method proceeds to step 437 to extract segments of the video related to the user. For example, the identified user may be a football player with a particular jersey number and name. The method may identify all segments of the video in which the football player is present and extract those segments from the other players. In another form, a golfer may be playing a hole on a golf course with other players, and the video may include numerous other shots and activities made by the other golfers. Thus, the method can identify specific users and activities in various segments of the video and filter out segments that do not include the specific user. In this manner, a video of only the specific golfer can be created. Once the video segments are extracted, the method proceeds to step 439 to determine whether the end of the video has been reached. If not, the method proceeds to step 437 and repeats. If the video has ended, the method proceeds to step 441 to determine whether the video should be processed for a new user. For example, as previously discussed, multiple golfers or players may be part of the same captured video. Thus, if desired, a new user can be identified at block 435 and the method can proceed as described above.In this manner, multiple segments specific to a particular user can be extracted from a single video, thereby reducing the number of video uploads required for processing. For example, at a football field, a single video can be uploaded and the method can extract video footage of each player to create unique video segments for each player that can be provided to each player, their teammates, coaches, etc. While multiple users may be detected in block 441, the method may not want to extract video segments for all users and may include profiles from a content manager to extract only segments for a particular user.

[0110] If at step 441 the method determines that no additional user segments should be extracted, the method may proceed to step 443. At step 443, the method determines whether the segments require further processing. For example, if only a single segment of video is extracted, no additional processing may be required to combine the segments. If at step 443 the video segments require additional processing, the method proceeds to step 445 to combine each user's segments into a single video. For example, the segments may be extracted and saved as part of the video or video segments. At step 445, the segments may be combined to create a single composite video for one user. Once the segments are combined, the method proceeds to step 447 to combine any remaining users' video segments to create a video unique to each user. Thus, each individual participant may have their own video using the segments created for their unique experience.

[0111] Having processed the video as necessary, the method proceeds to step 449 to determine whether effects need to be added to the video. For example, a content manager, such as content manager 418 of FIG. 4 or another autonomous content manager, may generate a video of a golfer playing a particular golf hole at a resort, such as Omni Barton Creek. A content manager may store an introductory video of a drone flyover of a golf hole being played and may add the introductory video to the user's video segment. In another form, an animated graphic showing the distance to the hole may be depicted from a "top-down" view of the hole from the tee box to the green. Other effects may also include adding audio or additional captured video of the user and other players in action. In one form, a portion of a segment of video may be identified or tagged to add a tracer to the ball's movement as part of the effects creation. In another example, AI logic may be used to detect that the ball is positioned around the green in a location that the golfer does not want. In that case, an augmentation effect may be added to the video when the ball goes into woods, sand traps, water hazards, etc. The augmentation effect may include an animated video overlay. For example, an animation of the Loch Ness Monster stealing a golf ball as it goes into a water hazard may be added to the video segment. Other animations may also be used and added as needed or desired. In this manner, augmented reality may be added to the user's video. According to a further aspect, the video may add a ball trace effect to a shot made by the golfer. For example, the method may be used to identify a golf ball in frames of a video and add a colored tracing line to each frame to indicate the path of the ball. If the video contains a video segment of a ball entering a green, a tracing may be added to the video when the ball lands on the green. In some cases, AI logic or image processing may be used to identify the location of the ball in the frames and in some cases, the video may be reversed after the ball is placed on the green. For example, as a user approaches a golf ball on the green, AI logic or image processing may identify the user and add effects or other content before the user picks up or addresses the ball.In this manner, through reverse processing of the video data, the ball can be traced back to a previous frame or segment, and the video can be altered accordingly for that particular user. In another form, the effects can include audio effects, music, or sounds added to the video. For example, music can be added to all or a portion of the video and can include various audio levels. Unique sounds can also be added to the video based on what is happening in the video. For example, a user can drive a ball into a forest, and a "chainsaw" sound, clapping sounds, laughter, clapping sounds, or other sound effects can be added to the video segment. In another form, AI logic or image processing can be used to identify when the golf ball goes into the cup and a "ball dropping into the cup" sound effect can be added. The effects can be predetermined based on the activity or sub-activity identified by the AI ​​logic. In this manner, the method can access labels in the video segment and automatically add desired effects to portions or segments of the video.

[0112] After adding effects as necessary, the method proceeds to step 453 to determine whether graphics need to be added to the video. If graphics do not need to be added, the method proceeds to step 429 and ends. If additional graphics are to be added, the method proceeds to step 455 to obtain content or graphics to add from an asset resource, such as assets 414, using content manager 418 of FIG. 4, or using other assets or content resources as needed or desired. The assets or graphics may include one or more graphics to add to a video image or video segment. For example, the graphics may include the name of the golfer, the date, the golf course, the hole number, the distance to the hole, the club used by the golfer, the current weather conditions, the maximum altitude of the ball or the maximum altitude after the ball was hit, the speed of the ball after it was hit, the curvature of the ball in flight, the maximum distance the ball has traveled, the current stroke or number of strokes made, the par for the hole, the current progress of other players currently being played, the current time of the golfer ... The information may include information such as player information, or other player or course information as needed or desired. According to another aspect, the golf course may include graphic assets to be added to the segments of the video, such as the name of the golf course, a logo of the golf course, age or establishment, the name of the current professional, the name of the owner, or various other types of marketing assets or graphics that the golf course may wish to add to the video segments. Although described as adding assets for the golf industry, other graphic assets can be added to the video as needed or desired. Once the graphic assets are obtained, the method proceeds to step 457 to modify the segment to add assets or graphics to the particular video image or segment. The method then proceeds to step 429 where the method ends.

[0113] The method of FIG. 4B can be modified as needed to combine or remove various portions as needed. For example, once an activity or sub-activity is identified in step 405, the method can be used to segment the video and further process the video segments to identify sub-activities. The segments can be labeled as having that sub-activity, and the labels can be used to further process the segments, add effects, add graphics, or other various types of processing of the video segments. In this manner, an automated process using AI logic can efficiently edit and process the video without the need for an individual to manually modify and edit the video.

[0114] Thus, the system 100 can ultimately capture, process, and store video of the activity for later provision to a user, such as via a user app 500.

[0115] Referring to FIG. 5, a schematic diagram of a user app 500 and / or user phone 501 is shown illustrating various screens and sub-screens presented by the app 500 for visualization by the user. The app may include multiple soft buttons 514, 516, 518, 520, and 522, corresponding respectively to user's videos, user's locations, user's friends, user's coaching, and others. By selecting button 514 ("V" for video), a list of videos categorized by event type may be displayed. As shown in FIG. 5, the categories may include golf courses 502, skiing 504, football practice 506, fishing 508, and soccer 510.

[0116] Multiple location categories can be displayed by selecting button 516. As shown in the example of Figure 5, the locations may include a first location 524, a second location 526, a third location 528, a fourth location 530, and a fifth location 532, respectively. The locations shown include, for example, a golf course, a ski resort, a beach location, and a sports field.

[0117] By selecting button 518, a list of the user's friends on the app may be displayed, including a first friend category 534, a second friend category 536, a third friend category 538, a fourth friend category 540, and a fifth friend category 542. By selecting button 520, coaching comments may be listed according to categories, including a first category 544, a second category 546, a third category 548, and a fourth category 550.

[0118] It will be appreciated that additional buttons and corresponding sub-screens can be used for other groups. Within each category, the app may display a quantity of videos in the category. The quantity may represent a total number of videos, a total number of unwatched videos, or some other measurement. In addition to organizing and displaying videos for the user, the app 500 may also display the videos on the phone 501's software. The system 100 may communicate directly or indirectly with other aspects of the system 100 via software / hardware.

[0119] Referring now to FIG. 6, a block diagram illustrating an example of an AI-enabled video recording system disposed on a golf course is provided, according to one aspect of the disclosure. The AI-enabled video recording system, generally shown as system 600, is used to detect, record, and process golfing activity at a golf hole having a tee box 602 and a green 604. System 600 includes cameras 606, 608, 610 that may be disposed adjacent to tee box 602, with camera 606 disposed behind tee box 602 and cameras 608 and 610 disposed on opposite sides of tee box 602. Cameras 614, 616, 618 may be disposed adjacent to green 604, with camera 616 disposed behind green 604 (and facing tee box 602) and cameras 614 and 618 disposed on opposite sides of green 604. Each camera is in communication with a camera interface 624, which may include network switch 110 of FIG. 1, remote camera interface 304 of FIG. 3, or other interface capable of connecting the remote cameras to processor 611. The interface 624 is connected to or integrated with the processor 611 and the AI ​​video processing engine 622. The processing engine 622 may be the processing system 112 or the system 300 described above and may include the modem 310 or the modem 114.

[0120] The processing engine 622 may be connected to a golf course irrigation power interface 620, which may provide electrical power, although other forms of power may be provided to the power processor 611 and the processing engine 622. The processing engine 622 may communicate with a remote golf course video processing management system 628 (which may be system 400). The video processing management system 628 may communicate with a mobile app 630, which may be the mobile app 500 described above. The mobile app 630 may be installed on a mobile device 631. The mobile device 631 may be a mobile phone, tablet, smart watch, golf cart, pull-art, push cart, powered "follow me" cart, or any other mobile device. It will be appreciated that the mobile app 630 may also be installed / embodied / accessible on other devices, such as a conventional computer, an internet browser, etc. The mobile app 630 may also include other features and functions as described below. It will be appreciated that the various above-mentioned systems may be integrated in whole or in part into the system 600 to enable local and / or remote processing of the video captured by the camera. Such processing may be accomplished automatically based on data received by system 600 and determined using video or image processing and / or artificial intelligence. Moreover, various aspects and uses of system 600 may be implemented as methods and software usable by system 600 or various components within system 600. As such, the description of FIG. 6, or elements thereof, may be deployed as a method.

[0121] Various cameras have been described above with reference to different views and aspects of the cameras. For purposes of description, each of the above cameras may be referred to generally as one or more cameras 601, which are generally shown in FIG. 6. It will be understood that reference to camera 601 may also refer to cameras 102-108, camera 202, cameras 606-618, or other camera references in this description.

[0122] During initial setup, the camera 601 can be utilized to record or perform a 3D scan of the hole / golf course so that different aspects of the course can be determined through image processing in the processor 611. For example, the processor 611 can determine the location and shape of water hazards or sand traps / bunkers, as well as the location of trees, different cuts of grass, etc. , structures, etc. The results of the 3D scan may be stored in the processor 611 for a particular hole and may be used as a reference for later processing of the ball flight.

[0123] As part of the initial setup, the height of the cameras 601 may be determined and input to the processor 611. This information may not be readily obtained from the GPS or location services location information of the cameras 601. The height of the cameras 601 may be entered manually or may be detected using other sensors that utilize other measurement methods such as lasers. The distance between the cameras 601 may also be measured, either by GPS coordinates or other measurement methods.

[0124] Additionally, with the determined positions for each of the cameras 601 and a 3D scan of the hole, the setup may include establishing a geofence 636 or other predefined boundary assigned to the hole. The geofence 636 may take the form of a bounding box or a complex curvature surrounding the hole and may be created with reference to the established GPS coordinates of the cameras 601. The geofence 636 may be utilized to detect when a golfer enters the hole by detecting whether the golfer's position is inside or outside the boundary of the geofence 636.

[0125] Detection of the golfer may be based on the detected or transmitted location of the golfer, or through detection by the camera 601, or through other detection methods such as sensors in combination with image processing software. According to another embodiment, detection of the golfer may be based on receiving a signal from a GPS or location based service or another transmitter such as RFID, Wi-Fi, Bluetooth, etc., and transmitted via the mobile device 631 or other transmitting device. In addition to detecting the presence of individual golfers, similar transmitters may be provided on golf carts, etc., to indicate the presence of one or more golfers. As mentioned above, the mobile device 631 may be a phone associated with the golfer, or may be another device such as a golf cart. It will be appreciated that other detection mechanisms may also be used.

[0126] In addition to the 3D scan of the hole, the initial setup may include pinpointing the exact location of various objects on the course via other marking methods. In one embodiment, the installer may travel to the location of a particular object and mark the object at a particular GPS or location-based service location, thereby providing an additional reference point for the object. This may be repeated to mark various objects around the course. For example, FIG. 6 shows a hazard adjacent to a green 604.

[0127] Thus, system 600 may receive location information corresponding to camera 601 and surrounding objects. In response to receiving this location information, system 600 may define a geofence 636. In accordance with the definition of geofence 636, system 600 may automatically detect and / or determine when a golfer is within geofence 636 and the golfer's position relative to camera 601.

[0128] The system 600 may further define a location grid 650. The location grid 650, partially shown in FIG. 6, may be limited to the area within the geofence 636 or may extend beyond the geofence 636. Preferably, the geofence 636 is defined to be a space large enough to encompass a significant portion of the area in which the golf ball and golfer are likely to be while playing a hole / course. Of course, the ball flight of a golf ball is unpredictable, especially for recreational golfers, and may vary depending on the location of the ball. It will be appreciated that the golf ball, or the golfer who hits the ball, may eventually travel outside the geofence 636 while playing the hole / course.

[0129] The grid 650 may be defined by multiple grid boxes, such as a 3' x 3' box that are repeated throughout or substantially throughout the hole. Each grid box has a fixed position relative to the camera 601 and the grid boxes may be utilized to provide the processor 611 with information regarding the particular position of the ball or golfer while the hole is being played. The position of the ball or golfer relative to a particular grid box may be used during image processing by the processor 611 to provide specially tailored video to the golfer.

[0130] Thus, in response to receiving the position information of the camera 601 and the position information of the environmental objects, the system 600 automatically defines the position grid 650. According to the definition of the position grid 650, the system 600 can automatically detect the position of the golfer within the position grid 650.

[0131] The system 600 may include one or more mobile devices 631, which may include applications and associated memory / processors, and thus may be referred to as mobile computing devices. For purposes of explanation, the mobile computing devices 631 are referred to as mobile devices 631. The mobile devices 631 have been described above as providing location and detection capabilities to the golfer. However, the mobile devices 631 may also provide other communication and control capabilities. The mobile devices 631 are configured to communicate with the processor 611 to provide the processor 611 with various information regarding the golfer, enabling the processor 611 to appropriately monitor and record the golfer and golf shots. The mobile devices 631 may include GPS or location services capabilities, thereby indicating the location of the mobile device 631 and the golfer in possession of the mobile device 631. The mobile devices 631 may also communicate with a cloud or remotely based system. It will be understood that any device with GPS or location services location capabilities, such as a GPS or location services watch, including those with yardage capabilities, may be used for purposes consistent herein. Various functions of the mobile device 631 described herein may also be provided by multiple mobile devices 631. For example, one mobile device 631 may be used for location determination, while another mobile device 631 may be used to communicate with a cloud / remote system to provide or receive other information.

[0132] In one aspect, in the case of a group of golfers, each golfer may have their own mobile device 631 in specific communication with the processor 611. The communication between the mobile device 631 and the processor 611 may be direct or may be through another communication relay. With the mobile device 631 in communication with the processor 611, the processor 611 may determine the specific location of each mobile device 631 and each golfer, thereby determining when one or more golfers enter a predefined geofence 636 where monitoring and recording should begin.

[0133] For example, when a group of golfers and their mobile devices 631 enter a geofence 636, their mobile devices 631 detecting their location can determine that the coordinates of the mobile devices 631 are within the predefined geofence 636. The coordinates of the geofence 636 can be communicated and stored in the mobile devices 631 such that the mobile devices 631 can determine that the mobile devices 631 are within the predefined geofence 636. Thus, the mobile devices 631 can then communicate to the processor 611 that the mobile devices 631 are within the geofence 636. In this embodiment, the processor 611 does not actively monitor for the presence of a golfer. Rather, the processor 611 receives a signal from the mobile device 631.

[0134] In another aspect, the processor 611 can actively monitor the presence of a golfer or mobile device 631. For example, the mobile device 631 or other device can send a "ping" at a predetermined interval. The processor 611 can "listen" to the ping, and when the golfer and mobile device 631 come within a predetermined range of the processor 611, the processor 611 can determine that the golfer and mobile device 631 have arrived at the hole / course and can begin monitoring and recording.

[0135] System 600 is configured to recognize each particular golfer participating and to identify when each golfer is participating. Typically, golfers take turns taking shots. While camera 601 records the shots as they occur, processor 611 is configured to determine which shots belong to which golfer so that particular shots can be associated with the correct golfer. Thus, processor 611 is configured to identify the golfer prior to each shot.

[0136] In one aspect, a golfer can indicate via an associated mobile device 631 that he or she is the one about to take their shot. The golfer can indicate via a button displayed on the mobile device 631 that it is their next shot. Alternatively, another golfer in the group can indicate via their mobile device 631 that a particular golfer in the group is about to take their shot. The processor 611 can then associate the resulting shot with the indicated golfer. This type of golfer identification can be described as manual golfer identification.

[0137] In another embodiment, the identification of the golfer immediately prior to hitting a shot can be performed automatically by the processor 611 and associated processors and software. In one embodiment, facial recognition software can be used. In this approach, prior to hitting a shot, the golfer stands in front of one camera 601, which captures an image of the golfer, and it can be determined through the captured image which golfer in the group of golfers matches the recognized face. The golfer can record his or her face prior to the round, allowing the processor 611 to access a database of faces of golfers expected to participate.

[0138] In another aspect, golfers can be identified by the clothes they are wearing. Typically, each golfer's clothes are unique within the group. For example, pants, shirt, hat, shoes, etc. can be recorded by each golfer and then detected by camera 601 before the golfer hits a shot.

[0139] In another aspect, the processor 611 may utilize the location data of the mobile device 631 to determine which golfer is about to take a shot. For example, if one golfer is standing on the tee box and addressing the ball and another golfer is standing outside the tee box or has not addressed the ball, the processor 611 may determine the particular golfer about to take a shot based on the location of the mobile device 631.

[0140] In another embodiment, the golfer may carry a remote identifier 633 on his / her person, such as in a pocket, clipped to a belt, fastened to a hat, etc. The remote identifier may be in the form of a GPS transmitter or an RFID tag. In the case of a GPS transmitter, the remote identifier 6 33 may actively transmit golfer location data to the processor 611, which receives the transmission and detects the golfer's location. In the case of an RFID tag, the remote identifier 633 may be detected by the processor 611, which may transmit a radio frequency to identify the golfer's location relative to the processor 611 to determine each golfer's location.

[0141] In another embodiment, system 600 can store the ending location of the ball for each shot and can piece together subsequent shots played from the stored location with the previous shot. Thus, even if a particular golfer cannot be identified at one time, system 600 can still determine that a shot belongs to that golfer based on where the shot began relative to where the previous shot ended. In one embodiment, such a determination can be made based on the location within the video frame or the location within grid 650.

[0142] Therefore, with the above in mind, system 600 is configured to identify individual golfers before they take their first shot from the tee box, as well as each subsequent shot. Cameras 601 installed on the hole record each shot and store each golfer's shots in a recording database along with each golfer's unique identifier based on each golfer's identification so that each golfer's shots can later be provided to the correct golfer.

[0143] As is typical, the resulting shot from each golfer will be unique. It will be appreciated that there are many factors that affect the outcome of the shot, such as the golfer's swing, positioning, wind speed and direction, etc. Thus, the resulting position of each golfer's shot may be at various locations on the hole, typically closer to the green than the tee box. Thus, the recording of each golfer's subsequent shots may include recordings from multiple cameras.

[0144] The camera 601 may be configured to include zoom capabilities, including one or both of optical and digital zoom. The camera 601 may be further configured with tilt and pan capabilities such that the camera 601 may be aimed at an identified target. The cameras 601 may be pointed and zoomed at each golfer attempting a shot, including the first shot and subsequent shots up to and including the final shot of the hole.

[0145] In another embodiment, one or more cameras 601 may have a fixed viewpoint without tilt, pan, or optical zoom. In this embodiment, camera 601 may be configured to capture everything within its field of view. The video may be provided or analyzed as a whole, or a segment / window of view may be separated or cropped to isolate a particular golfer or shot being played. Thus, the same video image may be used for multiple golfers who are within the field of view of camera 601 at the same time.

[0146] After the first tee shot, the system 600 is configured to determine and identify which golfer will take the next shot. Typical golf etiquette and rules dictate that golfers take their shots as determined by which golfer's ball is farthest from the pin. Recreational and professional golfers typically follow this rule, but exceptions are common, especially in recreational environments. Many golf courses encourage players to play "ready golf," in which the first golfer who is ready to hit takes the next shot, even if that golfer's ball is closer to the hole than the other golfers. This practice typically results in more efficient completion of holes and pace of play, allowing golfers to complete holes faster and allowing players behind to play faster. This allows the racing golfer to have the opportunity to play the hole sooner.

[0147] Thus, the system 600 is configured to determine which golfer will take the next shot so that the camera 601 can be aimed and focused on the correct golfer to record the next shot.

[0148] For each shot that each golfer takes, camera 601 records the shot and, based on the recorded video, determines the location of each ball within location grid 650. System 600 identifies the golfer for each shot, thereby correlating the location of the ball with the golfer who hit the ball. Thus, system 600 can determine, for each golfer, where the next shot will occur. Similarly, system 600 can determine, for each ball on location grid 650, which golfer will hit each ball. Thus, system 600 determines both the location of the ball and the identity of the golfer.

[0149] The system 600 may be configured to control which golfer will take the next shot or to identify which golfer is about to take the next shot. In one embodiment, the processor 611 may communicate with the golfer to inform the golfer that it is his / her turn to take the next shot. In one embodiment, an alert or signal may be sent to the golfer's mobile device 631. The alert may be in the form of an audible alert, a visual alert (such as a graphic representation on the screen of the mobile device), a tactile alert (such as by activating a vibration function of the mobile device 631), or a message (such as an SMS text message). In this approach, the golfer may know that the processor 611 will indicate which golfer is going to take their shot. The processor 611 may simultaneously send an alert to each golfer indicating the order of play, so that the golfers are informed of the next golfer to take the shot after the current golfer has finished.

[0150] As described above, the processor 611 is configured to monitor and detect the position of each golfer, so that by controlling which golfer is about to take a shot, the camera 601 can be aimed at that golfer to record the shot. The camera 601 can also zoom in on the golfer about to take the next shot.

[0151] In another aspect, the processor 611 can determine which golfer is about to take the next shot based on the golfer's movement relative to the ball. For example, if one of a group of golfers is located within a predetermined distance and the remaining golfers are located further away, the processor 611 can determine that the golfer closest to the ball is the golfer about to take the shot. In response to this determination, the processor 611 can direct the camera to be pointed at this golfer, and the camera 601 can zoom in on the golfer.

[0152] The relative distances between the golfers and the locations of their respective golf balls may be determined using grid 650. For example, if a golfer is located within the same grid square or an adjacent grid square as a previously determined location of the ball, processor 611 may determine that this golfer is the golfer about to strike. Processor 611 may compare the relative distances between the golfers and their respective ball locations and determine that the golfer closest to the ball is the golfer about to strike. To assist processor 611 in making the determination, golfers may be instructed to keep a predetermined distance from the ball when not about to strike.

[0153] Thus, system 600 may be configured to signal to the golfers which golfer should hit next and / or system 600 may be configured to determine which golfer has decided to hit next based on the golfers' positions relative to their respective ball locations. In both cases, system 600 may be configured to focus camera 601 on the correct golfer so that footage of the golfers and the next shot is properly recorded and stored.

[0154] This process may be repeated for each successive shot being played on a hole by various golfers on the hole. In some cases, the same golfer may take multiple shots before another golfer takes the next shot. This process may be repeated until the hole is completed.

[0155] During the play of a hole, the camera 601 may be configured to record at all times and the processor 611 may be configured to tag specific times of the recording to correspond to various golfers and shots being taken so that the recording can be segmented and stitched together according to the identity of each golfer and each shot taken, or recording may be started and stopped for each shot taken, with each recording tagged and later stitched together for each individual golfer.

[0156] The system 600 may include local video storage (not explicitly shown in FIG. 6) in communication with the processor 611, such as the AI-enabled digital video recorder 312 or memory 303 shown in FIG. 3. Alternatively, remote video storage may be used, such as cloud storage and services 404 in FIG. 4. The processor 611 may communicate with the video storage via Wi-Fi, cellular data, wireless communication, etc. The processor 611 may also communicate via a communication cable with a remote server or other communication device in communication with the video storage.

[0157] The processor 611 may further include or communicate with an image processing system / module 112, 300, 118, 400, which communicates with a video storage of recorded video and stitches together the various recordings of each shot assigned to each golfer. With reference to FIG. 6, the processor 611 communicates with a local AI video processing system / engine 622 and a remote processing system 628. The video processing system / engine 622 may include its own database for storing and formatting the video recordings. The image processing system / module 622, 628, 112, 300, 118, 400 may be included in the processor 611 or may be a separate module as shown in the various figures. The image processing system / module 622, 628, 112, 300, 118, 400 may begin processing images immediately after the completion of each shot and add the assembled recording for each additional shot or process after the video recording is finished.

[0158] System 600 may be further configured to determine based on the golfer's location data (via mobile device 631 or other location mechanism) and the boundaries of the geofence 636 when the golfer has left the hole. In response to determining that the golfer has left the hole, processor 611 may provide the golfer with one or more recordings that have been stitched or segmented, assembled or combined, processed, formatted, and communicated. In one aspect, each golfer may receive a recording of a particular shot on mobile device 631 or other device. In another aspect, each golfer may receive all recordings of the group and select which recording to watch.

[0159] In a preferred embodiment, the video recording is provided to the golfer after the completion of a hole and after the golfer has left the hole to prompt the golfer to leave the hole so that the subsequent golfer can play the hole. However, in another approach, the video recording or a portion of the video recording may be provided to the golfer immediately after the completion of each shot. In another embodiment, the video recording may be provided to the golfer after the golfer has finished the round and entered another part of the golf course, such as a pro shop, restaurant, bar, clubhouse, locker room, or parking lot.

[0160] In one embodiment, the system 600 may be configured to automatically upload assembled recordings of golfers to the Internet in addition to providing the recordings to individual golfers. Alternatively, the recordings may be uploaded to the Internet instead of being provided directly to the golfers.

[0161] In one embodiment, the recordings may be uploaded to a specific account associated with the golfer that corresponds to each golfer's unique recording. For example, the recordings may be automatically uploaded to one of the golfer's social media accounts. Each golfer may have a user account associated with the system 600. For example, via an app 630 installed on the golfer's mobile device 631, the golfer may enter various identifying data such as the golfer's name, address, email address, payment information, photo, and other identifying characteristics of the golfer. The golfer may also provide social media accounts and permission for the app installed on the mobile device 631 to post information to the linked social media accounts. In one embodiment, the golfer may enter their biographical information and social media accounts into a database associated with the processor 611 rather than entering this information into the app installed on the mobile device 631.

[0162] In one aspect, the golfer can choose whether or not to upload the recording to a social media account. The golfer can choose whether or not the recording is automatically uploaded before or after recording. The golfer can also choose to manually control whether or not the recording is automatically uploaded.

[0163] The inclusion of multiple cameras 601, in this case cameras 601 positioned near both the green and the tee box, allows each shot to be recorded from multiple angles. Thus, image processing software can stitch together the multiple angles of each shot. The system 600 can determine whether to display only one angle depending on the distance from the camera 601 and the ability of the camera 601 to zoom in on a particular golfer. In some cases, the golfer may be behind a structure or other obstacle, resulting in one angle being preferable over another. In some cases, the golfer may be too far away from one of the cameras 601 for a desired recording. For example, when a golfer is on or near the putting green, a camera 601 near the tee box may not provide a desired angle. However, when a golfer is hitting from the tee box, the golfer may be far away, but the flight of the ball may cause the ball to land near the camera 601 on the green, and therefore angles from both the tee box and the green may be used. In other cases, the golfer may be approximately halfway between the tee and the green, and therefore angles from both cameras 601 may be desirable.

[0164] The above aspects provide the ability to perform "unmanned" or autonomous video capture of golfers, similar to the video recording of golfers competing on television.

[0165] The system 600 may further be configured to add graphical elements to the recordings of golf shots for entertainment and evaluation purposes.

[0166] In one aspect, the system 600 may be configured to analyze images captured by the camera 601 and processed by the video processing system / module / engine 622, 628 (or 112, 118, 300, 400) to provide additional image enhancements to the recorded video. For example, the system 600 may include "tracer" software that provides a visual display of the flight of the ball. For example, in many television broadcasts of professional golf events, as the golfer hits the ball, a colored line tracks the ball, leaving a colored path of the ball's flight on the broadcast image. The curvature of the path is displayed on the broadcast image showing how the ball traveled, hooked, sliced, faded, etc. The tracer software may determine the speed of the ball, the distance the ball traveled, the apex or height of the ball during the shot, or other aspects.

[0167] These images, including the tracer included in the video or recording, provide a more robust description of the ball's flight path than would normally be possible for a typical viewer, especially if the ball traveled a relatively large distance from the camera. In a broadcast without a tracer, it may be difficult to figure out the ball's path later in its flight, and the broadcast often switches to another view showing the ball's landing area, leaving the viewer with incomplete information about how the ball traveled. Thus, the video processing system 622, 628 (or 112, 118, 300, 400) associated with the system 600 can provide the user with more complete information related to the golf shot, as compared to images without a tracer.

[0168] In addition to tracking the path of the ball, other graphical representations can be added to the flight of the ball. For example, the flight path can be color coded to indicate a particular outcome depending on the speed, distance, or flight of the ball. For example, if the ball travels above a certain speed, a red or "intense" color can be applied to indicate high speed or a frame graphic can be added as the tail of the ball. Similarly, if the ball flight is within what is considered "straight," a green color can be applied to the ball flight to indicate the absence of a hook or slice. Conversely, if the ball flight is not straight, another color, such as yellow or red, can be applied to the ball flight to indicate a less than ideal shot. It will be appreciated that these graphical additions based on the flight of the ball can be tailored to provide a variety of colors or graphical representations. In one aspect, the golfer can indicate, via the mobile device 631, the type of indicator they would like to display. The type of indicator can be switched so that multiple types of indicators can be applied to the same recording.

[0169] Graphic elements can also be added to the video recording based on where the ball lands at the end of each shot. As described above, the system 600 can include a location grid 650 associated with the topography of the hole. For example, each grid square of the location grid 650 can correlate to a topographical aspect of the hole. The selected squares can be associated with a bunker, a water hazard, out of bounds, in the woods, the fairway, the rough, the green, etc. The system 600, the camera 601, and the processor 611 can determine, based on the video recording and the flight of the ball, where the ball finished within the grid 650 and therefore the type of location where the ball finished. Based on the location of the ball, the system 600 can add graphic elements.

[0170] For example, if the ball is determined to have landed in a water hazard, a graphic element such as a sea monster can be added to the recording of the ball's position. A golfer, a snorkeler, etc. may be added to the location of the ball. If the ball is determined to have landed in a bunker or sand trap, a beach ball or beach umbrella may be added to the recording of the ball's location. If the ball is determined to have landed in the rough, an illustration of a lawnmower may be added to the ball's location. If the ball is determined to have landed in a forest, a squirrel or bear may be added that appears to approach the ball and then run away. If the ball lands on a green, sometimes referred to as a "dance floor," an illustration of a dancer or disco ball may be added. It will be appreciated that various other types of ball locations and corresponding animations or illustrations may be applied based on location. The system 600 may include multiple animations for the same type of location such that each instance may be a relatively unique animation. Animations may be randomly assigned based on location type or may be rotated for an identified golfer such that repeated animations are limited.

[0171] The location grid 650 may also be used for additional purposes. In one aspect, the location grid 650 may be used to help a golfer find his or her ball. Often, golfers have difficulty seeing the flight of the ball after a shot and may not know where the ball is. The processor 611 may provide the golfer with ball location data so that the golfer does not have to spend additional time looking for where the ball will land. Thus, the golfer may be able to complete a hole in a more efficient manner and improve their pace of play.

[0172] Similarly, based on the location of the ball within the grid and the locations of other features of the hole relative to the location grid 650, the system 600 can provide the golfer with additional information regarding the next shot. For example, the processor 611 can communicate to the golfer the distance to the pin at the end of the shot so that the golfer can consider which club to use for the next shot before arriving at the location of the ball. The system 600 can provide other distance-based information, such as distances to other features of the hole, such as the distance needed to clear a water hazard or bunker, or distances to particular areas of the green.

[0173] As described above, the location grid 650 may be overlaid on a hole, including the green 604. Thus, the system 600 may also be configured to operate as a virtual caddie to assist the golfer with putting. The system 600 may include various information about the green stored in the memory 303, the cloud storage and services 404, or other database storage in communication with the system 600. For example, various undulations of the surface of the green may be stored and referenced to determine the expected break for a particular putt. For example, the processor 611 may tell the golfer that the next putt will curve 3 inches to the left. The processor 611 may indicate whether the putt is uphill or downhill. Additionally, the processor 611 may store various video recordings of putts made from various locations on the green and aggregate these putts to use artificial intelligence to determine how the putt travels from a particular location on the green. For example, temperature, wind, humidity, and grain direction may affect how a putt travels, but may not be readily apparent from the shape of the green and may change with time and weather conditions. By analyzing the results of putts recorded from different locations, processor 611 can utilize this data to update its recommendations for different locations. In one aspect, the location grid 650 associated with the green may have smaller grid squares to provide more accurate location data for providing virtual caddie assistance.

[0174] The system 600 also automatically identifies the brand being used by the golfer. Similar to the facial recognition described above, the processor 611 may identify, via the camera 601, the brand of golf clubs the golfer is using or the brand of clothing the golfer is wearing. The processor 611 may utilize image recognition capabilities to identify logos, patterns, trademarks, etc. associated with different brands. In response to identifying the brand associated with each golfer, the processor 611 may be configured to communicate with the golfer to provide information regarding the brand, such as new products, product offerings, or alternative products.

[0175] The system 600 may also include the ability to integrate various software applications that may be used by a golfer or golf course owner. The mobile device 631 may include a user app 630 that may operate to indicate the user's location relative to a golf hole, the user app may be executed manually or automatically. As described above, the system 600 may automatically determine that a golfer has arrived at a particular golf hole. The user app 630 may also be used to manually signal to the processor 611 that the golfer has arrived at a hole. Similarly, the user app 630 may notify the golfer that he or she has arrived at a hole where recording is available. The user app 630 may provide other functionality, such as allowing the golfer to opt out of having their shots recorded.

[0176] The owner of the golf course where the system 600 is used may also have a dedicated application that interfaces with the processor 611. This application may be referred to as a course app 632. The course app 632 may be configured to communicate with the user apps 630. The course app 632 may enable the golf course to register the golf course as a course that includes recording capabilities. The user apps 630 may receive information from the various golf courses that register and provide the end user with information regarding available courses that offer recording capabilities.

[0177] The course app 632 can communicate with the user app 630 to notify the course that an interested user is onsite. Alerts can be provided via the course app 632 that a user or group of users has arrived and that the user is interested in using the technology. Similarly, the user app 630 can notify a golfer that a nearby course or the course the golfer is preparing to play includes a recording feature.

[0178] The User Apps 630 may also provide the ability to request a recording midway through a round. In another approach, the Course App 632 may be used by the golf course during golfer check-in to provide a service to interested golfers. However, golfers may change their mind about whether to record a shot. Thus, providing the User Apps 630 with the ability to request a recording or to decline a previously requested recording may ensure that the user's needs are met.

[0179] The User Apps 630 may also provide mechanisms to pay for certain features of a shot before or after the shot is recorded. For example, if a shot is recorded, the recording and image processing has the ability to provide the above-mentioned tracer technology to the shot to show the path of the ball. This may be considered an added feature, where the user may choose whether or not to apply the tracer technology after hitting the shot. However, in another aspect, the tracer technology may be applied regardless of user input. The User Apps 630 may provide the ability to turn the tracer on or off on demand.

[0180] The User App 630 can communicate with other User Apps 630 for other golfers. For example, each member of a particular group of golfers may activate their User App 630, allowing each golfer to record their shots. Each golfer's shots may be aggregated by one or more User Apps 630 to provide each golfer with a composite of the shots. The shots may be overlaid on top of each other or displayed one after the other. The User Apps 630 may provide other features among the group of golfers, such as each golfer's location on the course, each golfer's distance to the hole, etc., which may provide a desirable advantage among the group for competitive or entertainment purposes.

[0181] According to one aspect, the user app 630 can provide further camera / recording control by the golfer to adjust the recording as needed. In one aspect, the golfer can use the user app 630 to activate video recording using a record button displayed via the app 630 or otherwise provided on the mobile device 631, enabling the camera 601 to record the golfer. In a related aspect, a stop button can be provided as well, which disables the camera 601 while the golfer is, for example, reading the green.

[0182] In another embodiment, the recording of the golfer may be automatically stopped according to predetermined programming. In one embodiment, geofencing 636 and / or location sensing technology may be used in combination with the manual activation described above to turn off the camera 601 if the user forgets to stop recording.

[0183] It should be noted that not all of the activities described in the general description or examples above are required, some of the specific activities may not be required, and one or more additional activities may be performed in addition to the activities described. Furthermore, the order in which the activities are listed is not necessarily the order in which they are performed.

[0184] The specifications and illustrations of the embodiments described herein are intended to provide a general understanding of the structure of the various embodiments. The specifications and illustrations are not intended to serve as an exhaustive and comprehensive description of all elements and features of the apparatus and systems that use the structures or methods described herein. Upon reviewing this disclosure, many other embodiments may be apparent to those skilled in the art. Other embodiments may be used and derived from this disclosure, such that structural substitutions, logical substitutions, or other changes may be made without departing from the scope of this disclosure. Thus, this disclosure should be considered illustrative rather than limiting.

[0185] Certain features are described herein for clarity in the context of separate embodiments, but may also be provided in combination in a single embodiment. Conversely, various features that are, for brevity, described in the context of a single embodiment, may also be provided separately or in any subcombination. Further, references to values ​​stated in ranges include all values ​​within that range.

[0186] Benefits, other advantages, and solutions to problems have been described above with respect to specific embodiments. However, the benefits, advantages, solutions to problems, and features that may cause or make more noticeable the benefits, advantages, or solutions should not be construed as critical, necessary, or essential to any or all of the claims.

[0187] The subject matter disclosed above should be considered as illustrative and not limiting, and the appended claims are intended to cover all such modifications, extensions, and other embodiments that are within the scope of the invention. Accordingly, to the maximum extent permitted by law, the scope of the invention should be determined by the broadest permissible interpretation of the following claims and their equivalents, and should not be limited or constrained by the foregoing detailed description. It shall be so.

[0188] The above system 600 may be embodied as a group of related components that are controllable by the processor 611 and its associated software. Various aspects of the system 600 and its use may also be embodied as a method that utilizes the above-mentioned functionality to automatically provide the above-mentioned benefits to an end user. The system 600 may include various associated software modules that may be implemented using the processor 611 or remotely in communication with the processor 611. The modules or methods may include various artificial intelligence and machine learning and image processing to automatically process various images and provide a desired output to an end user. Figures 7 and 8 below show examples that provide methods that may be used to automatically record and process videos of golf activities in accordance with aspects of the present disclosure.

[0189] 7 illustrates one embodiment of a method 700 associated with the above system. In one embodiment, a method 700 for automatically recording athletic performance is provided. The method 700 includes detecting, by a processor, at step 702, that at least one player is located within a predetermined area; identifying a first player of the at least one player, at step 704; automatically recording the performance of the first player by at least one camera operably coupled to the processor and defining a first recording, at step 706; automatically saving the first recording in a database operably coupled to the processor, at step 708; automatically correlating the first recording with the first player, at step 710; and automatically processing the first recording and defining a first processed recording, at step 712.

[0190] 8 illustrates one embodiment of a method 800 for automatically recording and providing video in accordance with aspects of the system described above. Method 800 may include, at step 802, recording video of a predetermined activity using a first remote camera located at a first geographic location, and, at step 804, processing the video at the first geographic location. Processing may include, at step 806, identifying a first user performing the predetermined activity, at step 808, extracting image frames from the video including the first user during the predetermined activity, and, at step 810, merging the extracted image frames to generate a formatted video. Method 800 may also include, at step 812, outputting the formatted video to a remote video processing system for further processing.

[0191] It will be appreciated that various other additional method steps may be included in the above method or the above method may be modified according to the functionality of the system 100 described above.

[0192] It will be appreciated that such aspects and embodiments are more than abstract ideas executed by a computer or other controller: the above aspects are performed automatically based on various inputs that are not easily accessible or determinable, and the resulting end product cannot otherwise be provided in the same automated manner.

[0193] Although only a few exemplary embodiments have been described in detail above, those skilled in the art will readily appreciate that many modifications are possible to the exemplary embodiments without substantially departing from the novel teachings and advantages of the disclosed embodiments. Accordingly, all such modifications are intended to be included within the scope of the disclosed embodiments as defined in the following claims. In the claims, means-plus-function clauses are intended to cover the structures described herein as performing the recited function, as well as structural equivalents and equivalent structures.

Claims

1. 1. A system for automatically recording and processing activities, comprising: a first remote camera positioned at a first geographic location for recording video of a first predetermined activity; a first processor and memory operatively associated with and in communication with the first remote camera, the first processor and memory being located near the first geographic location; The system further comprises a local video processing engine associated with the first processor and memory, the local video processing engine configured to process frames of video captured by the first remote camera of a first user participating in the first predetermined activity; The first processor is further configured to modify the video upon identifying the first user and the first predetermined activity; The system further comprises a communications module capable of communicating the formatted video to a remote video processing system, the remote video processing system configured to further process the formatted video and make the processed video accessible to the first user.

2. The processor, Geofence, GPS location, Face recognition, Object Recognition, clothing recognition, Motion detection, RFID detection, Bluetooth detection, Wifi detection, Radar sensing, and heat sensing The system of claim 1 , configured to detect the location of the first user using one or more of:

3. 3. The system of claim 2, further comprising artificial intelligence (AI) logic accessible to the first processor, the artificial intelligence (AI) logic configured with logic to identify one or more users recorded in video frames captured by the first remote camera.

4. The first processor, Identifying the first user; automatically recording the first user when the first user is located within a first predetermined area; Associating a first recording of the first user with the first user; Associating a second recording of the first user with the first user; The system of claim 2 , configured to process the first and second recordings of the first user and define the formatted video associated with the first user.

5. The first processor detects and identifies a second user within the first predetermined area. The system of claim 2 , configured to:

6. The system of claim 5 , wherein the first processor is configured to automatically record the second user, and the first processor is configured to process recordings associated with the second user.

7. The system of claim 5 , wherein the first processor is configured to determine which of the first and second users is to be recorded.

8. 6. The system of claim 5, wherein the first processor is configured to send a message to the first user or the second user indicating that the first user or the second user will be recorded.

9. The system of claim 1 , wherein the first processor is configured to generate a location grid over at least a portion of the first geographic location.

10. 10. The system of claim 9, wherein the first processor is configured to detect a location of the first user within the location grid, and further configured to detect a location of an object associated with the first user within the location grid.

11. The system of claim 1 , wherein the first processor is configured to monitor a location of the first user while the first user is located within the first geographic location.

12. The system of claim 1 , wherein the first processor is configured to add graphical elements to the first processed recording based on data associated with the recording.

13. The graphical elements may include one or more graphical elements added to one or more frames of the video, the graphical elements comprising: text information indicating a location of said video; One or more names of locations where said video was recorded; The name of the first user; The date the video was recorded; a logo or other marketing image associated with said location; A colored trace created to show the graphic lines between frames of a moving object, Graphic objects, and Augmented reality elements The system of claim 12 , comprising:

14. The colored traces are The flight path of a golf ball Football player running path, Football flight path, Soccer player running path, Soccer ball flight path, Path for downhill skiers or snowboarders, The swimmer's swimming path, A swimming path of a fish caught by the first user; the route of a catamaran or race boat; and Mountain bike or racing bike cycling route 14. The system of claim 13, further comprising: creating a trace for one or more of the following:

15. 1. A system for automatically recording and processing activities, comprising: a first remote camera positioned at a first geographic location for recording video of a first predetermined activity; a first processor and memory operatively associated with and in communication with the first remote camera, the first processor and memory being located near the first geographic location; The system further comprises artificial intelligence (AI) logic accessible to the processor, the AI ​​logic configured with logic to identify a user recorded within a video frame captured by the first remote camera; The system further comprises a local video processing engine associated with the first processor and memory, the local video processing engine configured to process images in the video frames to identify the first user; The first processor is further configured to modify the video upon identifying the first user and the first predetermined activity; The system further comprises a communications module capable of communicating the formatted video to a remote video processing system, the remote video processing system configured to further process the formatted video and enable access to a mobile app for the identified first user.

16. The AI ​​logic is: golfer, Golf balls, shirt, Shirt color, pants, Pants color, skirt, Skirt color, hat, hat color, Golf gloves, Golf shoes, Golf carts, One or more people in a golf cart; Golf tee, Golf clubs, Iron, driver, Utility Club, putter, Wedge, Golf ball logo, male, woman, child, Young people, shirt logo, caddy, facilitator, brand, Left-handed golfer, Right-handed golfer, visor, glasses, sunglasses, drink, Tea box, Tea box color, Trees, fairway, Cart path, green, pin, hole, Sand bunker, Water hazards, Glass Hazard, forest, Out of bounds, Rough, First cut of the green, The second cut of the green, bird, insect, animal, Distance from tee to pin Distance from the tee to the front of the green Distance from the tee to the center of the green Distance from the tee to the back of the green, Red stake, white stake, yellow stake, Change in altitude, cloud, rain, snow, fog, haze, mud, wind, Green form, or Cutting the hole The system of claim 15 , further comprising logic capable of identifying one or more of:

17. The AI ​​logic is: Golf activities, Football activities, Soccer activities, Lacrosse activities, Baseball activities, Basketball activities, Tennis activities, Pickleball activities, Bean bag toss activity, Bowling activities, Billiards activities, swimming activities, Diving activities, Racing activities, Hockey activities, Field hockey activities, Disc Golf Activities, Rugby activities, Skiing activities, Snowboarding activities, Cycling activities, Fishing activities, Boating activities, and Sports Activities The system of claim 16 , further comprising logic capable of identifying an activity that includes one or more of:

18. The remote video processing system further includes a remote video processing management system in communication with the first processor, the remote video processing management system configured to receive a first series of recorded videos and process the first series of recorded videos to create the AI ​​logic; processing the first series of recorded videos includes tagging one or more uniquely identified objects within one or more frames of each of the recorded videos, the tagging including tags for identifying users and user activities within the recorded videos; The process further includes creating the AI ​​logic including a neural network of tagged objects and delivering the AI ​​logic to the first processor for use in processing videos at the predetermined location; 20. The system of claim 17, wherein the process further comprises receiving the formatted video from the communication module and modifying the formatted video based on usage of the mobile app.

19. and a remote video processing management system in communication with the video processing system, the remote video processing management system comprising: a network processor coupled to the video processing system and to cloud storage for storing processed video received from the first remote camera; an artificial intelligence (AI) enabled graphics processing engine configured to access the formatted video and process the formatted video; a graphics asset accessible to the AI-enabled graphics processing engine that is added to one or more frames of the formatted video; a format manager configured to format the formatted video based on a delivery location of the formatted video; 16. The system of claim 15, further comprising: a distribution manager accessible to the network processor, the distribution manager configured to distribute the formatted video to the distribution locations.

20. and a mobile app associated with the delivery manager, the mobile app including content provided by the delivery manager, the content comprising: one or more formatted videos output by the remote video processing management system; a list of videos created using one or more activities; a list of locations where the one or more videos were recorded; social media functionality for sharing the formatted video; and a virtual coaching feature configured to allow a coach to access and comment on the formatted video of the first user.

21. the processor is configured to identify a second user recorded within the video frames captured by the first remote camera; The processor is configured to extract a video frame from the first video, the video frame including the second user; the processor is configured to combine the extracted video frames into a second formatted video, the second formatted video including the second user; 16. The system of claim 15, wherein the communications module is configured to communicate the second formatted video to the remote video processing system, the remote video processing system configured to further process the video and enable access to a mobile app for the identified second user.

22. The remote video processing management system further comprises: identifying the second user in proximity to the first user during the recorded activity; initiating access to the first user of a second video generated by the remote video processing system; 22. The system of claim 21, configured to enable the first user to access the second video using the mobile app.

23. 1. A method for automatically recording and providing video, comprising: recording video of a predetermined activity using a first remote camera located at a first geographic location; and processing the video at the first geographic location, the processing comprising: identifying a first user performing the predetermined activity; extracting image frames from the video including the first user during the predetermined activity; and merging the extracted image frames to generate a formatted video. The method further includes outputting the formatted video to a remote video processing system for further processing.

24. identifying a second user performing the predetermined activity in the video; Extracting additional image frames including the second user performing the predetermined activity. and, merging the additional image frames to generate a second formatted video; 24. The method of claim 23, further comprising the step of: outputting the second formatted video to the remote video processing system.

25. providing a second remote camera at the first geographic location; identifying the first geographic location as a golf hole of a golf course; placing a first geofence around a tee box of the golf hole; placing a second geofence around the green of the golf hole; detecting when a first user is within the first geofence; activating recording of the first remote camera and the second remote camera in response to identifying the first user within the first geofence to record the predetermined activity; Detecting when the first user is within the second geofence; detecting when the first user leaves the second geofence; and disabling recording of the predetermined activity when the first user leaves the second geofence.

26. the first geofence includes a first geofence radius; 26. The method of claim 25, wherein the second geofence comprises a second geofence radius that is different from the first geofence radius.

27. the first geofence includes a first geofence size; 26. The method of claim 25, wherein the second geofence comprises a second geofence size that is different from the first geofence size.

28. identifying the predetermined activity as a golfing activity; extracting a first image frame and identifying a golf ball at a first position within the first image frame; extracting a second image frame at a later time period than the first image frame; determining whether the golf ball has moved to another location within the second image frame; drawing a colored line extending from within the second image frame to the first location; repeating drawing the line in subsequent frames until the golf ball is no longer visible; estimating where the golf ball will land; and drawing a colored line to an estimated location where the golf ball will land.

29. creating AI logic using previously recorded activity to identify specific activities; using said AI logic to identify said particular activity; identifying the first user performing the particular activity; commencing extraction of image frames of said particular activity; and combining the extracted image frames of the particular activity.

30. The step of creating the AI ​​logic includes: identifying a golfer holding a golf club within a previously recorded video; tagging said golfer with a particular outfit, golf clubs, and golf balls; repeating the identifying and tagging steps for a number of previously recorded activities and image frames; generating said AI logic using said tagged images; and using the AI ​​logic at a golf course having the first remote camera.

Citation Information

Patent Citations

  • Hole-in-one confirmation system and camera of the same

    JP2013248381A

  • Image processing device, image processing method, and virtual golf simulation device using the same

    JP2013523253A

  • Systems and methods for event timing and photography

    JP2017504372A

  • Image Enhancement in Sports Recordings

    US20080199043A1

  • System and Method for Creating and Sharing an Event Using a Social Network

    US20150006639A1