Systems and methods for automatic target identification, classification, and scoring

Abstract

The system is configured to receive images of targets located downrange, identify the targets in the images, classify the targets, and determine the location of a scoring region. The system is further configured to determine the likelihood of an actual hit from a projectile on the target and score the actual projectile hit. The system uses machine vision and machine learning models to classify the targets and determine the hit score.

Description

[Technical Field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 406,208, filed September 13, 2022, entitled "SYSTEMS AND METHODS FOR AUTOMATED TARGET IDENTIFICATION, CLASSIFICATION, AND SCORING," the contents of which are incorporated herein by reference in their entirety. [Background technology]

[0002] Target shooting is enjoyed by millions of people annually, and many reports indicate that the number of people who regularly engage in target shooting has increased over the past decade and continues to grow. In the United States alone, it is estimated that over 52 million people regularly shoot targets. There are many different types of recreational shooting activities, ranging from simple plinking with a handgun or rifle against paper or steel targets, to skilled long-range rifle shooting competitions, which require advanced training and skill, to fun, fast-paced pistol shooting against popping or stationary targets, or shotgun shooting against skeet, trap, or sporting clays. Aside from recreational shooting, an increasing number of target shooters practice as part of their professions, such as law enforcement, the military, and security guards. The number of people at shooting ranges continues to grow, and these participants, whether for sport, recreation, self-defense, or public defense, want to improve their skills. However, at crowded shooting ranges that allow paper targets, the need to cold-call the range to move downrange to set up, check, score, and replace targets can be disruptive. Similarly, in any of a number of shooting competitions, the range must be cold before competitors are allowed to move downrange to view, score, and replace targets.

[0003] The effort required to score and change targets can be disruptive to other shooters on the range, and in some cases, participants must walk 100, 300, 600, or even 1,000 yards or more to view, score, and change targets. This is very time-consuming, especially in mixed ranges where a single shooter wants to safe the range to change targets, requiring the remaining shooters to stop firing, safe their weapons, and wait while the other shooter proceeds downrange.

[0004] Scoring a target is a relatively simple process, as many targets include a scoring ring. Participants can visually inspect the target, identify where the projectile penetrated the target, and assign a score based on the number of penetrations and the score for each penetration. However, when steel targets are utilized, scoring these shots is extremely difficult, if not impossible, because shooting at a steel target is primarily a binary effect—either a satisfying ping if the projectile hits its mark or silence if the shooter misses. Most steel targets are not replaced when dents or holes form in the steel target, making it very difficult to locate new dents from existing dents and determining the location of hits on a steel target.

[0005] It would be advantageous if a system were capable of automatically scoring shooting targets, whether replaceable paper targets or more permanent targets such as steel targets. This would provide, among other things, increased efficiency in practice or competition. These and other advantages will be readily apparent from the following disclosure. Summary of the Invention

[0006] One or more computer systems may be configured to perform specific operations or actions by installing software, firmware, hardware, or a combination thereof that causes the system to perform the actions during operation. One or more computer programs may be configured to perform specific operations or actions by including instructions that, when executed by a data processing device, cause the device to perform the actions. One general aspect includes a method for automatically scoring a shooting target. The method also includes receiving one or more images of the target, generating a bounding box around the target, classifying the target, determining a scoring ring for the target, receiving one or more additional images of the target, determining a difference between a short-term moving average and a long-term moving average based on the received images of the target, filtering the difference between the short-term moving average and the long-term moving average to determine an impact, and executing a hit classifier that receives the difference and outputs an impact score. Other embodiments of this aspect include corresponding computer systems, devices, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0007] Implementations may include one or more of the following features. In the method, receiving one or more images is performed by a video capture device associated with the mobile computing device. The method may be performed on a smartphone. Determining a difference between the short-term moving average and the long-term moving average may include convolving the difference image with an impact kernel. The impact kernel may be a 5x5 uniform weight square kernel. The impact classifier may be configured to replace a window around a potential impact in the long-term moving average image with the short-term moving average that includes the potential impact. The method may include replacing a window around a potential impact in the long-term moving average image with the short-term moving average that includes the potential impact over at least five subsequent frames. The window may be at least a 10x10 pixel window. Generating a bounding box around the target may include iteratively modifying one or more corners of the bounding box to generate a modified bounding box and projecting the modified bounding box onto a reference target image. The method may include determining corners of the modified bounding box by a hill-climbing technique. The method may include determining coordinates of the impact and determining that the coordinates of the impact are in a scoring ring. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.

[0008] One general aspect includes a system for automatically scoring a target, including a computing device having one or more processors, the one or more processors configured with instructions, which may include an image analysis module configured to receive one or more images from an imaging sensor associated with the computing device, a target classifier configured to determine one or more boundaries of the target and determine geometric boundaries of one or more scoring regions, an impact kernel configured to compare two or more images and determine a likely impact, a machine learning model configured to receive data associated with the likely impact and determine a likelihood of an actual impact, and a scoring module configured to score the impact if the likelihood of the actual impact exceeds a threshold. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0009] Implementations may include one or more of the following features: In this system, the computing device is a smartphone. The target classifier may be configured to apply object detection to one or more images and determine corners of the target. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium. [Brief explanation of the drawings]

[0010] The accompanying drawings are part of this disclosure and are incorporated herein. The drawings illustrate examples of embodiments of the present disclosure and, together with the description and claims, serve to explain, at least in part, various principles, features, or aspects of the present disclosure. Specific embodiments of the present disclosure are described more fully below with reference to the accompanying drawings. However, various aspects of the present disclosure may be implemented in many different forms and should not be construed as limited to the implementations set forth herein. Like numbers refer to similar, but not necessarily the same or identical, elements throughout. [Figure 1] 1 illustrates a system configured for automated scoring of shooting targets, according to some embodiments. [Figure 2] 1 shows a sample process flow for classifying and scoring targets, according to some embodiments. [Figure 3] 1 illustrates a sample process flow for identifying and classifying targets, according to some embodiments. [Figure 4] 1 illustrates a sample process flow for registering targets and determining scoring hits, according to some embodiments. [Figure 5] 5A, 5B, and 5C show a method for initializing a target scoring system to identify a target, according to some embodiments. [Figure 6] 1 illustrates a sample process flow for detecting impact of a target, according to some embodiments. [Figure 7] 1 illustrates a sample process flow for scoring target impacts, according to some embodiments. [Figure 8] 1 illustrates a sample user interface for automated target scoring in a software application, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0011] According to some embodiments, a system is described that can quickly identify targets, classify targets, including identifying target scoring rings, and score target hits at a shooting range. In some cases, the system is stored and executed on a consumer mobile computing device (e.g., an iPhone, tablet, phone, video camera). In some cases, the system includes a video camera device pointed at a target of interest, and the system is configured to identify targets, classify targets, determine shots landed on the target, and score the shots landed on the target. In some cases, the system is configured to prompt the shooter regarding a shooting stage. For example, the system may be configured for use in a CMP high-powered rifle competition, where the system may prompt the user that the current stage requires firing 20 shots downrange from an off-hand position within a 20-minute time frame. In some cases, the system knows the number of shots expected during the shooting stage (called a firing sequence) and may prompt the user with information related to the current shooting stage, such as the number of shots, the time frame, and the shooting location. In some cases, the shooter may input information associated with the shooting stage, such as the number of shots the system should expect, the firearm to be used, and the distance to the target, among other things. In some cases, the system is started and stopped manually, identifying shots on target only during the time the system is started.

[0012] In some cases, the described systems operate in near real time on a single consumer recording device, such as a mobile phone, using only a modest amount of training data. As used herein, the terms "real time" or "near real time" are broad terms and, in the context of this disclosure, relate to receiving input data, processing the input data, and outputting the results of data analysis with little or no human-perceived latency. In other words, a system such as that described herein that outputs analyzed data within less than one second is considered near real time. A system operating in real time or near real time may limit the amount of computation for machine learning, or at least training models, that the method can use to characterize specific target acquisition, classification, and scoring.

[0013] Previous approaches to automated target scoring have relied on acoustic triangulation, optical triangulation, and piezoelectric sensor triangulation. Acoustic triangulation has been attempted by using acoustic chamber targets that use the Mach waves of a projectile to determine the location of the projectile as it passes through the target. Acoustic triangulation automated scoring systems operate by using microphones to measure the sound waves of the projectile as it passes through the target. The sound of the projectile passing through the target from multiple audio sensors (e.g., microphones) can then be used to determine the location where the projectile passed through the target.

[0014] Optical triangulation automated scoring systems use three or more lasers, such as infrared lasers, to triangulate the projectile's position as it passes through the target. Piezoelectric sensor triangulation systems rely on an array of piezoelectric sensors on a plate that sense vibrations caused by the projectile impacting the target.

[0015] FIG. 1 illustrates a system 100 configured to automatically identify targets, classify targets, determine target impacts, and score shot strings. The system 100 may include a computing resource 102, which may be a mobile computing device associated with a shooting range participant and may include any one or more of several mobile computing devices, such as a smartphone, tablet computer, laptop computer, or other suitable computing device. The computing resource 102 typically includes one or more processors 104 and a memory 106 that stores one or more modules 108. The modules 108 may store instructions that, when executed, cause the one or more processors 104 to perform various operations. The computing resource 102 may further include data storage, which may be remote storage, such as a remote server or cloud-based storage system, or local storage, or a combination thereof. The data storage may store data on previous engagement (DOPE), which can enable data tracking over time, as well as comparison data between different shooters, different firearms, different ammunition, different targets, different environments, etc.

[0016] The storage system can further enable historical trend analysis, which can be used to show a shooter's performance over time, including tracking improvements. Data storage can also be analyzed to provide performance predictions, rankings, and social features, among other benefits.

[0017] The system may incorporate one or more imaging sensors 110, such as any suitable video camera. In some cases, the imaging sensor 110 may be associated with a computing resource 102. For example, in some embodiments, the computing resource 102 may be a smartphone with a built-in camera 110.

[0018] Camera 110 may be aimed to capture an image of target 112. The target may be located at any distance from the shooter, and camera 110 may be aimed and / or zoomed to capture an image of the target. In some embodiments, the camera may be coupled to a lens, such as a spotting scope or camera lens, allowing the camera to capture a closer image of the target through optical or digital zoom.

[0019] The computing resources 102 may include, among other things, instructions (e.g., modules 108) that enable the computing device to initialize 118 targets, detect 114 target impacts, and score 116 target impacts. In some embodiments, the step of identifying targets is optional, and in some cases, the system may be configured to detect scoring rings and not perform the additional step of identifying or classifying targets.

[0020] FIG. 2 illustrates a decision tree 200 configured to detect, identify, and classify targets. According to some embodiments, the system does not know the type of target before the system begins searching for the target. For example, in some conventional systems, the scoring system may be pre-programmed with the target against which the shooter will aim. This allows the system to easily understand the size and shape of the target and the location and boundaries of each scoring ring or zone. In the illustrated embodiment, the system is configured to automatically determine and determine the scoring rings and zones without a priori data regarding the type of target. For example, the system may have one or more video capture devices that may be integrated into one or more mobile computing devices. As used herein, a mobile computing device may be one or more of a mobile phone, smartphone, tablet, laptop, personal digital assistant, smart glasses, body cam, wearable computing device, or any other computing device that a user may carry to a shooting range.

[0021] The mobile device may activate the camera and capture one or more frames of the target 202. The computing device may have instructions to analyze the one or more frames and identify the target within the one or more frames by using any suitable image analysis algorithm. If the target is detected and classified in block 204, the target is registered with the system 206, and a scoring ring and area are determined. The system may capture additional image frames containing the target and look for differences between one frame and the next that may correlate with impact on the target. The frames may be compared, and a moving average may be generated in block 208. Moving averages are fundamental mathematical and statistical techniques applied in image analysis and machine learning for a variety of purposes, including noise reduction, feature extraction, and trend analysis. They involve calculating the average value of pixel intensities or other data points within a moving window or kernel across an image or dataset. Moving averages can be used to extract meaningful features from an image. For example, important information can be highlighted by sliding a small window over the image and calculating the average pixel value within that window. For example, in edge detection, a moving average can highlight areas with abrupt changes in pixel intensity, helping to identify the edges or boundaries of the target and scoring area. Edge detection can also be used to identify bullet impacts on the target.

[0022] In some instances, moving averages are used in time series data analysis. As one example, moving averages can be used to establish a baseline behavior of a system to detect anomalies. Data points that deviate significantly from this baseline can be flagged as anomalies or outliers. These anomalies can be further analyzed to determine target impact.

[0023] Once the sequential moving averages are generated, they may be combined into a long-term moving average. At block 210, the moving average image may be compared to the long-term moving average to determine differences from one frame to a subsequent frame that indicate changes to the target that are most likely associated with impact on the target.

[0024] The impacts are selected and classified in block 212. For example, the system determines the boundaries of a scoring ring, determines the location of each impact, and associates the location of each impact with a score for the impact.

[0025] Returning to block 204, if a target has not been previously detected and classified, for example, if the shooter is initializing the system or has replaced a target, the system determines whether a target has been detected in block 214. If not, the system proceeds to detect the target in block 216. If a target has been detected, the system classifies the target in block 218, for example, by identifying the target boundaries, scoring ring boundaries, and scoring ring values.

[0026] If the system does not detect the target, the system may capture one or more additional image frames and analyze the one or more additional image frames to determine that the target is located within the field of view of the imaging device. Once the target is detected, the system may classify the target and determine its size, as well as the relative positions and sizes of the scoring rings or regions.

[0027] FIG. 3 further illustrates the initial steps a system may take to identify and classify a target 300 by analyzing one or more image frames. Object detection is a computer vision technique that involves identifying and locating multiple objects within an image or video stream. Unlike image classification, which determines the presence of a single object class in an entire image, object detection provides a more granular understanding by not only recognizing objects but also specifying their location through bounding boxes. In some embodiments, object detection algorithms typically output bounding boxes that enclose detected objects. These bounding boxes consist of the coordinates (x, y) of the object's upper-left corner and dimensions (width and height) that define the object's spatial extent within the image.

[0028] In block 302, the system applies object detection to one or more images of the target to search for the target. In some embodiments, the object detection model is generic with respect to the target, allowing the system to detect any target regardless of size or shape. In block 304, if the target is found in the same location in subsequent images (e.g., two or more images, three or more images, four or more images, etc.), the system assumes it has located the target and defines a bounding box around the target. In some cases, finding the target in the same location in subsequent images includes determining a moving average of the images to determine the target's location, size, and shape.

[0029] At block 306, the targets are optionally classified. In addition to locating objects, the system may be configured to detect objects and classify each detected object into a predetermined class or category. This allows the system to distinguish between different object types, such as circular targets, oval targets, rectangular targets, silhouette targets, etc.

[0030] A target classifier may be applied to the images within the bounding box, thus allowing the system to determine which reference target image to apply.

[0031] In block 308, the system registers the target to the reference target image. In some cases, this involves applying a contrast adjustment to the image. This may also involve iteratively modifying the initial bounding box, such as by adjusting its corners and then projecting the adjusted bounding box onto the reference target image. The difference between the two may be applied as a score, and a hill-climbing technique may be applied to find the optimal corner, which can be correlated with the initial position of the image. Hill-climbing is an optimization algorithm used to find the local maximum (or minimum) of a given objective function. By iteratively taking small steps in the direction of higher values, the algorithm determines the highest and lowest values, which can then be used to determine the boundary of the target and / or the boundary of the target's scoring zone. In some cases, object detection is combined with semantic segmentation to provide pixel-level object masks. This allows for a more accurate understanding of object boundaries in the image, such as target boundaries and scoring ring boundaries.

[0032] In block 310, the system initializes and registers the target and begins looking for impacts over the following moving average.

[0033] FIG. 4 illustrates a process 400 for registering a target 300 and determining target impacts. In block 402, the target may be re-registered, such as by performing a hill-climbing technique to search for a new set of best corners for the target. In some cases, the hill-climbing search uses mean-squared distance in a perceptual space technique. For example, mean-squared distance, also known as mean-squared perceptual error, is a metric used to measure the similarity or dissimilarity between two data points containing perceptual data, such as target corners. In some cases, for each data point, relevant perceptual features (in this case, target corners, edges, scoring rings, etc.) are extracted. Features can be visual descriptors that can be represented as vectors of perceptual features. These feature vectors capture relevant information about each data point in a more concise and informative form.

[0034] The mean squared distance between two points (represented as their respective feature vectors) is generated by determining the squared difference between corresponding features and calculating the average of these squared differences. The resulting mean squared distance provides a quantitative measure of the dissimilarity between two data points in perceptual space.

[0035] In block 404, the system may apply a transformation matrix that may be used to map the set of corners to an image. In some cases, the image to which the coordinates are mapped has dimensions of 160 pixels, and in other cases, it may be less than 160 pixels.

[0036] In block 406, the moving average image is updated, with the long-term moving average being, in some cases, about 10 seconds or more and the short-term moving average being about 0.1 seconds. In some cases, the video camera may capture more than 30 frames per second. For the short-term moving average, this equates to averaging about 3 frames to determine the short-term moving average.

[0037] In block 408, the system determines the difference between the moving averages. For example, a long-term moving average may be associated with a static target that has not changed for 10 seconds or so and compared to a short-term moving average that reflects changes in the image. Thus, the difference between the short-term and long-term moving averages highlights changes to the image, such as a bullet hitting the target. The system may convolve any difference image with a simple impact kernel, which may be a 5x5 uniformly weighted square kernel, to find the location of the largest block in the difference image. A kernel typically refers to a convolution filter that may be used to process and modify pixel values ​​for feature extraction, etc. The square kernel may be convolved (or moved) across the image, and at each location, the kernel's value may be multiplied by the pixel values ​​in the corresponding neighborhood, and the results may be summed to generate a new pixel value in the output image. Of course, the size of the kernel may be varied to adjust the range of the neighborhood considered during the convolution, and may include any of the full set of aperture kernels with non-uniform weights, and may have any suitable size.

[0038] The convolution returns a set of potential hits on the target. The set of potential hits may be further filtered, such as by using simple statistics of a window surrounding the flagged difference. In some cases, the window is chosen to be a 16x16 window with the difference in the center of the window. Of course, other window sizes are entirely reasonable, and the pixel values ​​described herein are merely illustrative of some embodiments. The system may also apply some business rules to the windowed differences; for example, the system should not detect multiple hits in the exact same location.

[0039] In block 410, a target impact is determined. In some cases, this is accomplished by passing the filtered set of differences to a hit classifier for scoring. If the difference score exceeds a threshold, the location is marked as a hit, and another window may be placed around the hit. In some cases, a 10x10 window is placed around the hit location in the long-term average, and the short-term average is used over 5-10 subsequent frames. This ensures that the same hit is not detected again. Thus, the difference is windowed with a first window, and if the difference exceeds a threshold score, the difference is windowed with a second window smaller than the first window. The windowed difference associated with the short-term moving average may be added to a long-term moving average of at least 5 frames, or at least 6 frames, or at least 10 frames, or at least 12 frames, or at least 15 frames or more. In some cases, if the difference score falls below the threshold, the difference is marked as a false hit, and the system does not need to evaluate and classify it again.

[0040] According to some embodiments, the system may receive audio data associated with a shot being fired and determine that a shot has been fired based on the audio data. In some cases, the audio data is correlated with a target image, and the system can convolve a differential target image in response to audio data indicating that a shot has been fired. In some cases, the system may not need to continuously convolve a differential image. In this case, the system can determine that a shot has been fired through the audio data, then update a short-term moving average and convolve the differential image to look for the shot. In some cases, the system is configured to distinguish shots fired by a user who has set their sights on a target from other shooters on the shooting range. In this way, the system can know when a shooter of interest will fire a shot, even if other active shooters are on the shooting range.

[0041] In some cases, the audio data may be used in impact detection, such as by correlating the audio of a fired shot with impacts appearing on a target image.

[0042] 5A-5C illustrate and describe initializing the scoring system by identifying and classifying targets. In some cases, the system can automatically determine the target boundaries, but in some embodiments, user input may define the target boundaries. For example, using a human-computer interface (e.g., a touchscreen, mouse, stylus, touchpad, etc.), a human can draw a boundary around a target to help the system identify it. However, in many embodiments, the system uses machine vision to identify the target and its boundaries. FIG. 5A shows an image 500 captured by a camera associated with the system. The image may include a target stand 502, a target 504, a target fixation clip 506, and other features within the field of view. The system may determine an initial bounding box 508 around the identified target, such as by a trained target detection model. In some embodiments, a user may define the initial bounding box using the human-computer interface, such as by drawing on a computer display. The human-computer interface may be any suitable interface, such as a touchscreen, pen, mouse, trackball, etc. The initial bounding box may not exactly match the edges and corners of the target, especially if the bounding box is user-defined. The initial bounding box and target image are sometimes referred to as an initialization frame. The initialization frame may be converted to Lab color space, which includes components for lightness, a green-to-red axis, and a blue-to-yellow axis, to produce perceptual uniformity. In some cases, the luminance channel is equalized via contrast-limited adaptive histogram equalization (CLAHE).

[0043] 5B shows a target whose coordinates have been determined as described above, often representing a quadrilateral that can be projected onto a reference target image 510. The reference target image 510 may also be converted to Lab color space, and a squared difference (in Lab space) may be generated between the projection and the target. A hill-climbing algorithm may be applied to the coordinates, where a possible delta is a small change to the coordinate, and a better solution may be determined by the squared perceptual difference.

[0044] 5C shows the best coordinates, such as determined by minimum difference over several random restarts of the hill climbing algorithm. These coordinates may then be used to apply an updated bounding box 512. Thus, even if the target image is distorted, such as when the viewing angle from the camera makes the target appear as a parallelogram rather than a rectangle, the initial bounding box may be modified to match the shape of the target as presented in the image captured by the camera.

[0045] In some embodiments, the system may define the edges of a target through image analysis. However, in some cases, the edges of the target are irrelevant; only the scoring rings are important. Thus, in some cases, the system is configured to identify the scoring rings and is not concerned with the target boundary. Additionally, the system only needs to identify the scoring rings, rather than classifying the target. For example, the system may determine through one or more machine learning models that a target represents a central bullseye target with a series of scoring rings. The system may assign a score value to each ring, such as 10 points for the bullseye, 9 points for the next larger ring, and so on. Similarly, the system may identify a target with five bullseye-sized circles spaced throughout the target and assign a value of 10 points to each of these scoring rings. One or more of the multiple bullseye-sized rings may have radially spaced larger scoring rings that may be assigned a smaller value than the bullseye-sized rings. Thus, the system need only omit the step of classifying the target and focus solely on the size and location of the scoring rings.

[0046] 6 illustrates and describes impact detection and scoring of detected impacts. A machine learning model may be implemented to determine whether a difference between image frames is likely a projectile impact on the target 504. The target 504 may be re-registered, such as by applying a hill-climbing technique using possible corner coordinates, as in the target initialization step. The differences are windowed (602a, 602b, 602c) by comparing the short-term moving average difference with the long-term moving average to generate a difference image 612 between the current target and the long-term target average.

[0047] The difference image may be convolved with an impact kernel (e.g., a windowed kernel that scans the image difference). Any point that is several standard deviations away from the long-term exponential moving average (EMA) of the maximum convolution value is flagged as a possible impact 604a, 604b, 604c.

[0048] The possible hits 604a-604c are fed to a machine learning model (e.g., a classifier) ​​606 that determines whether the difference is likely to be an actual hit. If the difference is above a threshold, the system marks the difference as an actual hit 608. However, if the difference is below the threshold, the system marks the difference as a false hit 610.

[0049] FIG. 7 illustrates and explains scoring of shots on a target 504. Different scoring zones may be determined by the system based on computer vision, by referencing registered targets from previous shooting sessions, by retrieving stored target models from a known target database, or in some other manner. Scoring zones 702 on the target may be represented as connected regions of one or more simple shapes (e.g., ellipses, rectangles, circles, triangles, etc.). Coordinates of detected shots 704 may be normalized and transformed to axes implied by a reference image. In other words, the shots may be overlaid on a reference image, which can be used for the shot's coordinates. The coordinates may be Cartesian coordinates expressed as x, y values. In some cases, the coordinates may be radial coordinates, representing the shot as an angle and distance from, for example, the center of the target. The system can then determine whether the shot is entirely within a single scoring zone or penetrates a scoring zone boundary, allowing the system to accurately score the shot. The system may use simple geometric shapes to determine whether a significant portion of a given impact falls within any of the simple shapes of each target zone.

[0050] In some embodiments, the system uses the coordinates of the impacts for further analysis. For example, by generating and storing the coordinates for a given firing string, grouping can be quantified and used as a measure of improvement over time. Similarly, a shooter's MOA (moment of angle) is a measure of group size in inches and arc minutes and can be determined from center to center and edge to edge. Additionally, grouping may be used to define pose, grip, or motion errors during the firing string. Grouping may be quantified, including group size, group rotation, or other metrics.

[0051] Scoring can be quantified by any suitable metric. In some cases, scoring is point-based, with points awarded for each zone on the target being added to or subtracted from an initial amount. In some cases, missed shots or extra shots fired are scored as negative or higher values, depending on the scoring type. In some cases, timed scoring is used, where total time is reflected in the score, with time increased as a penalty for misses. In some cases, group size is used to determine scoring, with extra or missed shots penalizing group size. Of course, other metrics and combinations of metrics may be determined by the system to score a particular firing string.

[0052] The system may be configured to return the coordinates of impact and the time each shot occurred for each shot in a firing string, so that the firing string can be ranged using some metric that combines position and time. For example, the time between shots may be measured, or shots after a buzzer or other initiation signal may be tracked and stored along with an accuracy metric.

[0053] In some cases, once impacts are identified, the system may draw a bounding box around one or more of the impacts. When a firing string ends, the system may draw a bounding box containing each of the shots in the group and determine a metric based on the bounding box to determine a score.

[0054] 8, which illustrates a user interface 800 of a system developed and operable in accordance with some of the embodiments described herein, the system may be configured to determine a bounding box 802 that may pass through the center of the outermost impact or along the edge of the impact. The system may determine any of several metrics, such as, but not limited to, group size 804, overall group width 806, group height 808, bounding box rotation angle, MOA, elevation offset 810, windage offset 812, and may further determine shot distance 814, which may be manually entered or determined based on the detected projectile's time of flight.

[0055] For example, the system may be configured to register the sound of gunfire, shock waves from projectile or gunpowder explosions, movement of the firearm or shooter, or some other indicator that a shot has been fired. The system may then detect when impact on the target occurs, determine the ammunition's time of flight, and determine the target distance based on the firearm, ammunition, and / or charge. This process can be performed in near real time by a simple consumer-grade mobile computing device. In some cases, the mobile computing device may utilize the zoom capabilities of an integrated image capture device. In some cases, an external zoom lens may be used to capture image frames by the mobile computing device. For example, a mobile phone may be coupled to a spotting scope, which provides optical zoom through the spotting scope, allowing the mobile computing device to capture clearer images of targets that may be downrange. Some mobile computing devices may rely on digital zoom to capture one or more images of targets located downrange.

[0056] The system may further determine and display the number of shots fired in the current shooting string 816, the average split time for each shot 818, which may be useful in timed shooting competitions. The system may further show the score 820 associated with each shot and the cumulative score 822 for the shooting string.

[0057] The system may include one or more processors and one or more computer-readable media that may store various modules, applications, programs, or other data. The computer-readable media may include instructions that, when executed by the one or more processors, cause the processors to perform the operations described herein for the system.

[0058] In some implementations, the processor(s) may include a central processing unit (CPU), a graphical processing unit (GPU), both a CPU and a GPU, a microprocessor, a digital signal processor, or other processing units or components known in the art. Alternatively or additionally, the functions described herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems-on-chips (SoCs), complex programmable logic devices (CPLDs), etc. Additionally, each of the processor(s) may possess its own local memory, which may also store program modules, program data, and / or one or more operating systems. One or more control systems, computer controllers, and remote controls may include one or more cores.

[0059] Embodiments may be provided as a computer program product including a non-transitory machine-readable storage medium having stored thereon instructions (in compressed or uncompressed format) used to program a computer (or other electronic device) to perform the processes or methods described herein. Computer-readable media may include volatile and / or non-volatile memory, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Machine-readable storage media include, but are not limited to, hard drives, floppy disks, optical disks, CD-ROMs, DVDs, read-only memory (ROM), random-access memory (RAM), EPROMs, EEPROMs, flash memory, magnetic or optical cards, solid-state memory devices, or other types of media / machine-readable media suitable for storing electronic instructions. Furthermore, embodiments may be provided as a computer program product including a transitory machine-readable signal (in compressed or uncompressed format). Examples of machine-readable signals include, but are not limited to, signals (including signals downloaded over the Internet or other networks) that can be configured to be accessed by a computer system or machine that hosts or executes a computer program, whether or not modulated using a carrier wave.

[0060] Those skilled in the art will recognize that any process or method disclosed herein can be modified in many ways. The process parameters and order of steps described and / or illustrated herein are given by way of example only and can be changed as desired. For example, although the steps illustrated and / or described herein may be shown or described in a particular order, these steps do not necessarily have to be performed in the order shown or described.

[0061] The various exemplary methods described and / or illustrated herein may omit one or more of the steps described or illustrated herein or may include additional steps in addition to those disclosed. Furthermore, the steps of any method as disclosed herein may be combined with any one or more steps of any other method as disclosed herein.

[0062] This disclosure describes exemplary embodiments and is therefore not intended to limit the scope of the embodiments of the present disclosure and the appended claims in any way. The embodiments are described above using functional components that illustrate implementations of specified functions and their relationships. The boundaries of these functional components have been arbitrarily defined herein for the convenience of description. Alternative boundaries may be defined to the extent that the specified functions and their relationships are appropriately performed.

[0063] The foregoing description of specific embodiments sufficiently clarifies the general nature of the embodiments of the present disclosure so that others, by applying the knowledge of those skilled in the art, can easily modify and / or adapt such specific embodiments for various applications without undue experimentation and without departing from the general concept of the embodiments of the present disclosure. Therefore, such adaptations and modifications are intended to be within the meaning and range of equivalents of the disclosed embodiments, based on the teaching and guidance presented herein. The phrases or terms used herein are for the purpose of description, not limitation, and should be interpreted by those skilled in the art in light of the teaching and guidance presented herein.

[0064] The breadth and scope of embodiments of the present disclosure should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.

[0065] In particular, conditional language such as "can," "could," "might," or "may," unless expressly stated otherwise or understood otherwise within the context in which it is used, is generally intended to convey that certain implementations include certain features, elements, and / or operations, but may not include them in other implementations. Thus, such conditional language is generally not intended to imply that features, elements, and / or operations are somehow required in one or more implementations, or that one or more implementations necessarily include logic for determining whether those features, elements, and / or operations should be included or performed in any particular implementation, with or without user input or prompting.

[0066] Unless otherwise noted, as used herein, the terms "connected to" and "coupled to" (and their derivatives) should be interpreted as allowing both direct and indirect (i.e., via other elements or components) connections. Additionally, as used herein, the terms "a" or "an" should be interpreted as meaning "at least one of." Finally, for ease of use, as used herein, the terms "including" and "having" (and their derivatives) should be interpreted as open-ended and not excluding additional components.

[0067] This specification and the accompanying drawings disclose examples of systems, apparatus, devices, and techniques that may provide systems and methods for determining the acoustic signature of a fired firearm. For purposes of describing various features of the present disclosure, it is, of course, not possible to describe every conceivable combination of elements and / or methodologies, but those skilled in the art will recognize that many further combinations and permutations of the disclosed features are possible. Accordingly, various modifications can be made to the present disclosure without departing from the scope or spirit of the disclosure. Moreover, other embodiments of the present disclosure will be apparent from consideration of the specification and accompanying drawings, as well as from practice of the disclosed embodiments presented herein. The examples presented in this specification and the accompanying drawings are to be considered in all respects as illustrative and not restrictive. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

[0068] Those skilled in the art will understand that in some implementations, the functionality provided by the processes and systems described above may be provided in alternative ways, for example, divided among more software programs or routines or integrated into fewer programs or routines. Similarly, in some implementations, the illustrated processes and systems may provide more or less functionality than described, such as when other illustrated processes, respectively, lack or include such functionality, or when the amount of functionality provided is varied. Additionally, while various operations may be shown as being performed in a particular way (e.g., sequentially or simultaneously) and / or in a particular order, those skilled in the art will understand that in other implementations, these operations may be performed in other orders and in other ways. Those skilled in the art will also understand that the data structures described above may be structured differently, for example, by dividing a single data structure into multiple data structures or by combining multiple data structures into a single data structure. Similarly, in some implementations, the illustrated data structures may store more or less information than described, such as when other illustrated data structures, respectively, lack or include such information, or when the amount or type of information stored is varied. The various methods and systems shown in the figures and described herein represent example implementations. In other implementations, the methods and systems may be implemented in software, hardware, or a combination thereof. Similarly, in other implementations, the order of any method may be changed, and various elements may be added, rearranged, combined, omitted, modified, etc.

[0069] From the foregoing, it will be understood that, although specific implementations have been described herein for illustrative purposes, various modifications may be made without departing from the spirit and scope of the appended claims and the elements described therein. In addition, while certain aspects are presented below in certain claim forms, the inventors contemplate various aspects in any available claim form. For example, while only some aspects may currently be described as being embodied in a particular configuration, other aspects may likewise be so embodied. Various modifications and changes may be made, as would be apparent to one skilled in the art having the benefit of this disclosure. All such modifications and changes are intended to be encompassed, and therefore the above description should be regarded in an illustrative rather than a limiting sense.

Claims

1. 1. A method for automatically scoring a shooting target, comprising: receiving one or more images of a target; generating a bounding box around the target; classifying the target; determining a scoring ring for said target; receiving one or more additional images of the target; determining a difference between a short-term moving average and a long-term moving average based on the received image of the target; filtering the difference between the short-term moving average and the long-term moving average to determine impact; executing a hit classifier that receives the difference and outputs a score for the hit; A method comprising:

2. The method of claim 1 , wherein the receiving of the one or more images is performed by a video capture device associated with a mobile computing device.

3. The method of claim 1 , wherein the method is performed on a smartphone.

4. The method of claim 1 , wherein determining the difference between the short-term moving average and the long-term moving average comprises convolving the difference image with an impact kernel.

5. The method of claim 4 , wherein the impact kernel is a 5×5 uniform weight square kernel.

6. The method of claim 1 , wherein the impact classifier is configured to replace a window around a potential impact in a long-term moving average image with the short-term moving average that includes the potential impact.

7. 7. The method of claim 6, further comprising replacing the window around a potential impact in the long-term moving average image with the short-term moving average that includes the potential impact over at least five subsequent frames.

8. The method of claim 6 , wherein the window is at least a 10×10 pixel window.

9. 2. The method of claim 1 , wherein generating a bounding box around the target comprises iteratively modifying one or more corners of the bounding box to generate a modified bounding box, and projecting the modified bounding box onto a reference target image.

10. The method of claim 9 , further comprising determining corners of the modified bounding box by a hill-climbing technique.

11. The method of claim 1 , further comprising determining a coordinate of the impact and determining that the coordinate of the impact is in a scoring ring.

12. 1. A system for automatically scoring targets, comprising: Computing device having one or more processors wherein the one or more processors are configured with instructions, the instructions comprising: an image analysis module configured to receive one or more images from an imaging sensor associated with the computing device; a target classifier configured to determine one or more boundaries of the target and to determine geometric boundaries of one or more scoring regions; an impact kernel configured to compare two or more images and determine likely impacts; a machine learning model configured to receive data associated with the potential impact and determine a likelihood of actual impact; a scoring module configured to determine a score for the impact if the likelihood of the actual impact exceeds a threshold; Including, the system.

13. The system of claim 12 , wherein the computing device is a smartphone.

14. The system of claim 12 , wherein the target classifier is configured to apply object detection to the one or more images to determine corners of the target.

15. The system of claim 12 , wherein the scoring module is configured to determine coordinates of the impact and determine that the impact is within a scoring zone of the target.

16. The system of claim 15 , wherein the scoring module is configured to convolve an impact kernel over the difference image to identify the location of the impact.

Images