System and method for automatic target recognition, classification, and scoring

Abstract

The system is configured to receive an image of a target located in a target region, identify the target in the image, and classify the target to determine a location of a scored region. The system is further configured to determine a likelihood of an actual impact of the fly on the target, and score the actual fly impact. The system classifies targets and determines impact scores using machine vision and machine learning models.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 406,208, filed on September 13, 2022, entitled “SYSTEMS AND METHODS FOR AUTOMATEDTARGET IDENTIFICATION, CLASSIFICATION, AND SCORING,” the contents of which are incorporated herein by reference in their entirety.

[0003] background

[0004] According to many reports, millions of people enjoy target shooting every year, and the number of people who regularly shoot targets has increased over the past decade and continues to increase. In the United States alone, it is estimated that more than 52 million people regularly shoot targets. There are many different types of recreational shooting activities, from simple casual shooting (plinking) with a pistol or rifle pointed at a paper or steel target, to skilled long-range rifle shooting competitions that require a high degree of self-discipline and skill, to fun and fast-paced pistol shooting pointed at pop-up targets or fixed targets, or shotgun shooting pointed at skeet, trap, sporting clays, etc. In addition to recreational shooting, more and more target shooters practice as part of their profession, such as law enforcement officers, military personnel, and security personnel.

[0005] The number of people attending shooting ranges continues to increase, and these participants, whether for sport, recreation, personal defense, or public defense, are eager to improve their skills. However, at busy shooting ranges that allow paper targets, requiring the range to stop shooting activity (go cold) in order to move downrange to set up, check, score, or change targets can be disruptive. Similarly, at many shooting competitions, the range must stop shooting activity before participants are allowed to move downrange to check, score, and change their targets during any round.

[0006] The amount of work required to score and change targets can be disruptive to other shooters on the range and, in some cases, require participants to walk 100, 300, 600, 1,000 yards or more to inspect, score, and change targets. This is very time-consuming, especially when only one shooter on a busy range wishes to make the range safe for a target change, and the remaining shooters must stop firing, secure their weapons, and wait while one person makes their way to the range.

[0007] Scoring targets is a relatively simple process, as many targets include scoring rings. Participants can visually inspect the target and see where the projectile penetrates the target, assigning a score based on the number of penetrations and the score for each penetration. However, when using steel targets, these shots are difficult, if not impossible, to score, as shooting at steel targets is largely a binary experience: either a satisfying ping when the projectile strikes the target, or silence when the shooter misses. It is difficult to determine where a hit was made on a steel target, as most steel targets are not replaced, and even if a dent or hole is made in a steel target, it is difficult to distinguish the new dent from the pre-existing one.

[0008] It would be advantageous if the system could automatically score shooting targets, whether replaceable paper targets or more permanent targets (such as steel targets). This would provide greater efficiency in practice or competition, etc. These and other benefits will become apparent from the following disclosure.

[0009] Overview

[0010] A system having one or more computers can be configured to perform specific operations or actions by installing software, firmware, hardware, or a combination thereof on the system, wherein the software, firmware, hardware, or a combination thereof causes the system to perform the actions. One or more computer programs can be configured to perform specific operations or actions by including instructions that, when executed by a data processing device, cause the device to perform the actions. One general aspect includes a method for automatically scoring a shooting target. The method also includes receiving one or more images of a target; generating a bounding box around the target; classifying the target; determining a scoring ring for the target; receiving one or more additional images of the target; determining a difference between a short-term moving average and a long-term moving average based on the received images of the target; filtering the difference between the short-term moving average and the long-term moving average to determine an impact; and executing an impact classifier that receives the difference and outputs an impact score. Other embodiments of this aspect include corresponding computer systems, devices, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0011] Implementations may include one or more of the following features. In the method, receiving one or more images is performed by a video capture device associated with a mobile computing device. The method may be performed on a smartphone. Determining the difference between the short-term moving average and the long-term moving average may include convolving the difference image with an impact kernel. The impact kernel may be a 5×5, uniformly weighted square kernel. The impact classifier may be configured to replace a window around a potential impact in the long-term moving average image with a short-term moving average that includes the potential impact. The method may include replacing the window around the potential impact in the long-term moving average image with a short-term moving average that includes the potential impact for at least 5 subsequent frames. The window may be at least a 10×10 pixel window. Generating a bounding box around the target may include iteratively modifying one or more corners of the bounding box to generate a modified bounding box, and projecting the modified bounding box onto a reference target image. The method may include determining the corners of the modified bounding box by a hill climbing technique. The method may include determining the coordinates of the impact and determining that the coordinates of the impact are within the scoring ring. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.

[0012] One general aspect includes a system for automatically scoring an object, the system comprising a computing device having one or more processors configured with instructions, the instructions including: an image analysis module configured to receive one or more images from an imaging sensor associated with the computing device; an object classifier configured to determine one or more boundaries of the object and determine geometric boundaries of one or more scoring regions; an impact kernel configured to compare two or more images and determine a possible impact; a machine learning model configured to receive data associated with a possible impact and determine the likelihood of an actual impact; and a scoring module configured to determine a score for the impact if the likelihood of the actual impact exceeds a threshold. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0013] Implementations may include one or more of the following features. In the system, the computing device is a smartphone. The object classifier may be configured to apply object detection to one or more images and determine corner points of an object. Implementations of the described technology may include hardware, methods or processes, or computer software on a computer-accessible medium. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings are part of the present disclosure and are incorporated into this specification. The accompanying drawings illustrate examples of embodiments of the present disclosure and, in conjunction with the description and claims, serve to at least partially explain various principles, features, or aspects of the present disclosure. Certain embodiments of the present disclosure are described more fully below with reference to the accompanying drawings. However, various aspects of the present disclosure may be implemented in many different forms and should not be construed as limited to the embodiments set forth herein. The same numbers always refer to similar, but not necessarily identical or identical, elements.

[0016] Figure 1 A system configured for automatically scoring shooting targets according to some embodiments is shown;

[0017] Figure 2 shows an example process flow for classifying and scoring targets according to some embodiments;

[0018] Figure 3 shows an example process flow for identifying and classifying objects according to some embodiments;

[0019] Figure 4 shows an example process flow for registering targets and determining scoring hits according to some embodiments;

[0020] Figure 5A 、 Figure 5B and Figure 5C A method for initializing a target scoring system to identify a target according to some embodiments is shown;

[0021] Figure 6 shows an example process flow for detecting an impact on a target according to some embodiments;

[0022] Figure 7 shows an example process flow for scoring impacts on a target according to some embodiments; and

[0023] Figure 8 An example user interface for automatic goal scoring in a software application is shown, according to some embodiments.

[0024] Detailed description

[0025] According to some embodiments, a system is described that can quickly identify targets, classify targets, including identifying a scoring ring for a target, and score impact hits on targets at a shooting range. In some cases, the system is stored on and executed on a consumer mobile computing device (e.g., an iPhone, tablet, phone, or camera). In some cases, the system includes a camera device pointed at a target of interest and is configured to identify the target, classify the target, determine the impact of the shot on the target, and score the impact on the target. In some cases, the system is configured to prompt the shooter about the shooting phase. For example, the system can be configured for use during a CMP high-powered rifle competition and can prompt the user that the current phase requires firing 20 shots from an off-hand position into a target area within a 20-minute window. In some cases, the system knows how many shots are expected during the shooting phase (called a string of fire) and can prompt the user with information associated with the current shooting phase, such as the number of shots, timeframe, and shooting posture. In some cases, the shooter may input information associated with the shooting phase, such as the number of shots the system should expect, the firearm used, the distance to the target, etc. In some cases, the system is manually activated and deactivated, and shots on target are recognized only during the time the system has been activated.

[0026] In some cases, the described systems operate in near real time on a single consumer device of record, such as a mobile phone, using only a modest amount of training data. As used herein, the terms "real time" or "near real time" are broad terms and, in the context of the present disclosure, relate to receiving input data, processing the input data, and outputting the results of the data analysis with little to no human-perceptible delay. In other words, a system as described herein that outputs analysis data in less than one second is considered to be near real time. Systems that operate in real time or near real time can limit the amount of computation used for machine learning, or at least training models, which the method can use to characterize specific target acquisition, classification, and scoring.

[0027] Existing methods for automatic target scoring rely on acoustic triangulation, optical triangulation, and piezoelectric sensor triangulation. Acoustic triangulation has been attempted using an acoustic chamber target, which uses the Mach waves of a missile to determine the position of a missile as it passes through a target. An acoustic triangulation automatic scoring system operates by using microphones to measure the sound waves of a missile as it passes through a target. Thus, the sound of a missile passing through a target from multiple audio sensors (e.g., microphones) can be used to determine the position of a missile passing through a target.

[0028] Optical triangulation automated scoring systems use three or more lasers, such as infrared lasers, to triangulate the position of a projectile as it passes through a target. Piezoelectric sensor triangulation systems rely on an array of piezoelectric sensors on a board that sense the vibrations caused by the projectile striking the target.

[0029] Figure 1 A system 100 is shown, configured to automatically identify targets, classify targets, determine impacts on targets, and score shooting strings. System 100 may include a computing resource 102, which may be a mobile computing device associated with a participant at a shooting range and may include any one or more of a variety of mobile computing devices, such as a smartphone, tablet, laptop, or other suitable computing device. Computing device 102 typically includes one or more processors 104 and a memory 106 storing one or more modules 108. Modules 108 may store instructions that, when executed, cause one or more processors 104 to perform various actions. Computing resource 102 may also include a data storage device, which may be a remote storage device (such as a remote server or cloud-based storage system), a local storage device, or a combination thereof. The data storage device may store data on past target hits (DOPE) (which may allow for data tracking over time) as well as comparative data between different shooters, different firearms, different ammunition, different targets, different environments, and the like.

[0030] The storage system may also allow for historical trend analysis, which can be used to show a shooter's performance over time, including tracking improvements. The data store may also be analyzed to provide performance predictions, rankings, social features, and other benefits.

[0031] The system may include one or more imaging sensors 110, such as any suitable camera. In some cases, the imaging sensor 110 may be associated with the computing resource 102. For example, in some embodiments, the computing resource 102 may be a smartphone with a built-in camera 110.

[0032] The camera 110 can be pointed to capture an image of a target 112. The target can be located at any distance from the shooter, and the camera 110 can be aimed and / or zoomed to capture an image of the target. In some embodiments, the camera can be coupled to a lens (such as a spotting scope or a camera lens) to allow the camera to obtain a closer view of the target through optical zoom or digital zoom.

[0033] The computing resource 102 may include instructions (e.g., module 108) that allow the computing device to initialize a target 114, detect an impact on the target 116, and score the impact on the target 118. In some embodiments, the step of identifying the target is optional, as in some cases the system is configured to detect a scoring ring and may not perform the additional step of identifying or classifying the target.

[0034] Figure 2 A decision tree 200 is shown that is configured to detect, identify, and classify targets. According to some embodiments, before the system begins searching for a target, the system does not know the type of target. For example, in some existing systems, the scoring system can be pre-programmed with the target that the shooter will aim at. This makes it easy for the system to understand the size and shape of the target, as well as the location and boundaries of each scoring ring or area. In the illustrated embodiment, the system is configured to automatically determine and determine the scoring ring and area without prior data about the target type. For example, the system can have one or more video capture devices that can be integrated into one or more mobile computing devices. As used herein, a mobile computing device can be one or more of the following: a mobile phone, a smart phone, a tablet computer, a laptop computer, a personal digital assistant, smart glasses, a body camera (body cam), a wearable computing device, or some other computing device that a user can carry to a shooting range.

[0035] The mobile device may activate its camera and capture one or more frames of a target 202. The computing device may be instructed to analyze the one or more frames using any suitable image analysis algorithm to identify the target in the one or more frames. If the target is detected and classified at block 204, the target is registered with the system 206, and scoring rings and regions are determined. The system may capture additional image frames containing the target and look for differences from one frame to the next that may be associated with an impact on the target. These frames may be compared, and at block 208, a moving average may be generated. Moving averages are a fundamental mathematical and statistical technique used in image analysis and machine learning for various purposes, including noise reduction, feature extraction, and trend analysis. They involve calculating the average of pixel intensities or other data points within a moving window or kernel across an image or dataset. Moving averages can be used to extract meaningful features from an image. For example, by sliding a small window across an image and calculating the average pixel value within that window, important information can be highlighted. For example, in edge detection, a moving average can emphasize areas of sudden changes in pixel intensity, helping to identify the edges or boundaries of targets and scoring regions. Edge detection can also be used to identify impacts on a target.

[0036] In some examples, moving averages are used in time series data analysis. For example, to detect anomalies, moving averages can be used to establish a baseline behavior for a system. Any data point that deviates significantly from this baseline can be flagged as an anomaly or outlier. These anomalies can then be further analyzed to determine their impact on the target.

[0037] As the sequence moving averages are generated, they can be combined into a long-term moving average. At block 210, the moving average image can be compared to the long-term moving average to determine a difference from one frame to a subsequent frame that indicates a change in the target that is most likely associated with an impact on the target.

[0038] At block 212, the impacts are selected and classified. For example, the system determines the boundaries of the scoring ring and determines the location of each impact and associates the location of each impact with a score for that impact.

[0039] Returning to block 204, in the event that a target has not been previously detected and classified (such as when the shooter initialized the system or changed targets), the system determines whether a target has been detected at block 214. If a target has not been detected, the system continues to detect targets at block 216. If a target has been detected, the system classifies the target at block 218, for example, by identifying the boundaries of the target, the boundaries of the scoring ring, and the values ​​of the scoring ring.

[0040] In the event that the system has not yet detected the target, the system can capture one or more additional image frames and analyze the one or more additional image frames to determine that the target is within the field of view of the imaging device. Once the target is detected, the system can classify the target to determine the size of the target and the relative position and size of the scoring ring or area.

[0041] Figure 3 Further described are the initial steps that the system can take to identify and classify a target 300 by analyzing one or more image frames. Object detection is a computer vision technique that involves identifying and locating multiple objects in an image or video stream. Unlike image classification, which determines whether a single object category is present in an entire image, object detection provides a more refined understanding by not only identifying the object but also specifying its location via a bounding box. In some embodiments, the object detection algorithm typically outputs a bounding box that surrounds the detected object. These bounding boxes consist of the coordinates (x, y) of the upper left corner of the object and the dimensions (width and height) that define the spatial extent of the object in the image.

[0042] At block 302, the system applies object detection to one or more images of a target and searches for the target. In some embodiments, the object detection model is generic for the target, which allows the system to detect any target regardless of size or shape. At block 304, if the target is found in the same location in subsequent images (e.g., 2 or more images, 3 or more images, 4 or more images, etc.), the system assumes that the system has located the target and defines a bounding box around the target. In some cases, finding the target in the same location in subsequent images includes determining a moving average of the images to determine the target location, size, and shape.

[0043] At block 306, the objects are optionally classified. In addition to locating objects, the system can be configured to detect objects and classify each detected object into a predefined class or category. This allows the system to distinguish between different object types, such as circular objects, oval objects, rectangular objects, silhouette objects, or other types of objects.

[0044] The object classifier can be applied to the image within the bounding box. Thus, the system determines which reference object image to apply.

[0045] At box 308, the system aligns the target with the reference target image. In some cases, this involves applying contrast adjustment to the image. This can also involve iteratively modifying the initial bounding box, such as by adjusting the corners of the bounding box and then projecting the adjusted bounding box onto the reference target image. The difference between the two can be applied as a score, and a hill climbing technique can be applied to find the best corner, which can be associated with the initial position of the image. Hill climbing is an optimization algorithm for finding the local maximum (or minimum) of a given objective function. By iteratively taking small steps in the direction to produce higher values, the algorithm determines the highest value, the lowest value, and can therefore be used to determine the boundary of the target and / or the boundary of the scoring area of ​​the target. In some cases, object detection is combined with semantic segmentation to provide pixel-level object masks. This allows a more accurate understanding of object boundaries within the image, such as target boundaries and scoring ring boundaries.

[0046] At box 310, the system has been initialized and registered with the target and begins looking for shocks in the subsequent moving average.

[0047] Figure 4A process 400 of registering a target 300 and determining an impact on the target is described. At block 402, the target can be re-registered, such as by performing a hill climbing technique to search for a new optimal set of corner points for the target. In some cases, the hill climbing search uses mean squared distance in a perceptual space technique. For example, mean squared distance (also known as mean squared perceptual error) is a metric used to measure the similarity or dissimilarity between two data points involving perceptual data (e.g., target corners). In some cases, for each data point, relevant perceptual features are extracted, in this case, the features are target corners, edges, scoring rings, etc. The features can be visual descriptors, which can be represented as vectors of perceptual features. These feature vectors capture the relevant information for each data point in a simplified and more informative form.

[0048] The mean squared distance between two points (represented by their respective feature vectors) is generated by determining the squared differences between corresponding features and calculating the mean of these squared differences. The resulting mean squared distance provides a quantitative measure of the dissimilarity between two data points in the perceptual space.

[0049] At block 404, the system can apply a transformation matrix that can be used to map the set of corner points to an image. In some cases, the image onto which the coordinates are mapped has a size of 160 pixels, and in some cases less than 160 pixels.

[0050] At block 406, the moving average image is updated, and in some cases, the long-term moving average is about 10 seconds or longer, while the short-term moving average is about 0.1 seconds. In some cases, the camera can capture more than 30 frames per second. For the short-term moving average, this is equivalent to averaging about 3 frames to determine the short-term moving average.

[0051] At block 408, the system determines the difference between the moving averages. For example, a long-term moving average would be associated with a static target that has not changed for about 10 seconds and can be compared to a short-term moving average that reflects changes in the image. Thus, the difference between the short-term and long-term moving averages will highlight changes in the image, such as impacts on the target. The system can convolve any difference image with a simple impact kernel (in some cases, a 5×5, uniformly weighted square kernel) and find the maximal block-wise locations in the difference image. A kernel generally refers to a convolution filter that can be used to process and modify pixel values, such as for feature extraction. The square kernel can be convolved (or moved) across the image, and at each location, the kernel value can be multiplied by the pixel value in the corresponding neighborhood, and the results can be summed to produce a new pixel value in the output image. Of course, the kernel size can be varied to adjust the extent of the neighborhood considered during the convolution and can include any of a set of aperture kernels with non-uniform weights and can have any suitable size.

[0052] The convolution returns a set of potential impacts on the target. This set of potential impacts can be further filtered, such as by using simple statistics on a window around the marked difference values. In some cases, the window is chosen to be 16×16, with the difference value in the middle of the window. Of course, other window sizes are entirely possible, and the pixel values ​​described herein are merely illustrative of some embodiments. The system can also apply business rules to the windowed difference values, for example, the system should not detect multiple impacts at the exact same location.

[0053] At block 410, an impact on the target is determined. In some cases, this is accomplished by passing the filtered set of differences to an impact classifier for scoring. If the difference score is above a threshold, the location is marked as an impact, and another window can be placed around the impact. In some cases, a 10×10 window is placed around the impact location in a long-term average, with the short-term average being for 5 to 10 subsequent frames. This ensures that the same impact is not detected again. Thus, the difference is windowed by a first window, and if the difference exceeds a threshold score, the difference is windowed by a second window that is smaller than the first window. The windowed difference associated with the short-term moving average can be added to a long-term moving average for at least 5 frames, or at least 6 frames, or at least 10 frames, or at least 12 frames, or at least 15 frames, or more. In some cases, when the difference score is below a threshold, the difference is marked as a false impact, and the system will not need to evaluate and classify it again.

[0054] According to some embodiments, the system may receive audio data associated with a shot fired and determine that a shot has been fired based on the audio data. In some cases, the audio data is associated with a target image, and the system may convolve a difference target image in response to the audio data indicating that a shot has been fired. In some cases, the system may not need to continuously convolve the difference image. In such cases, the system may determine that a shot has been fired based on the audio data, then update a short-term moving average and convolve the difference image to look for the shot. In some cases, the system is configured to distinguish shots fired by a user aiming at a target from other shooters at the shooting range. In this way, the system can know when a shooter of interest has fired a shot, even if there are other active shooters at the shooting range.

[0055] In some cases, audio data may be used for impact detection, such as by correlating the audio of a fired shot with an impact appearing on a target image.

[0056] Figures 5A-5C Initializing the scoring system by identifying and classifying an object is shown and described. In some cases, the system can automatically determine the boundaries of the object, while in some embodiments, user input can define the boundaries of the object. For example, using a human-computer interface (e.g., a touch screen, mouse, stylus, touchpad, etc.), a human can draw a boundary around the object to help the system identify the object. However, in many embodiments, the system uses machine vision to identify the object and its boundaries. Figure 5A An image 500 captured by a camera associated with the system is shown. The image may include a target support 502, a target 504, a target retaining clip 506, and other features within the field of view. The system may determine an initial bounding box 508 around the identified target, such as by using a trained target detection model. In some embodiments, a user may define the initial bounding box, for example, by drawing on a computer display using a human-machine interface. The human-machine interface may be any suitable interface, and in some cases may be a touch screen, pen, mouse, trackball, etc. The initial bounding box may not exactly conform to the edges and corners of the target, particularly in those cases where the bounding box is defined by the user. The initial bounding box and target image may be referred to as an initialization frame. The initialization frame may be converted to a Lab color space comprising the following components: luminance, a green to red axis, and a blue to yellow axis to generate perceptual uniformity. In some cases, the luminance channel is equalized via contrast-limited adaptive histogram equalization (CLAHE).

[0057] Figure 5BA target is shown, where the coordinates of the target are determined as described above, and the coordinates often imply a quadrilateral, which can be projected onto a reference target image 510. The reference target image 510 can also be converted to Lab color space, and a squared difference can be generated (in Lab space) between the projection and the target. A hill climbing algorithm can be applied to the coordinates, where the likely delta is a small change in the coordinates, and a better solution can be determined by squaring the perceptual difference.

[0058] Figure 5C The optimal coordinates, determined, for example, by minimizing the difference from several random restarts of a hill climbing algorithm, are shown. The coordinates can then be used to apply an updated bounding box 512. Thus, even in cases where the target image is skewed, such as where the camera's perspective appears to display the target as a parallelogram rather than a rectangle, the initial bounding box can be modified to conform to the shape of the target as presented in the image captured by the camera.

[0059] In some embodiments, the system can define the edges of the target through image analysis; however, in some cases, the edges of the target are irrelevant and only the scoring rings are important. Therefore, in some cases, the system is configured to identify the scoring rings and not focus on the target boundaries. Furthermore, the system may not need to classify the target, but only need to identify the scoring rings. For example, the system may determine, through one or more machine learning models, that the target represents a central bullseye target with sequential scoring rings. The system can assign a score value to each ring, such as a bullseye with a score of 10, the next larger ring with a score of 9, and so on. Similarly, the system can identify a target with five bullseye-sized circles spaced throughout the target and assign a value of 10 to each of these scoring rings. One or more of the multiple bullseye-sized rings may have radially spaced larger scoring rings, which may be assigned a smaller value than the bullseye-sized rings. Thus, the system can omit the target classification step and focus solely on the size and position of the scoring rings.

[0060] Figure 6 Impact detection and scoring of detected impacts are shown and described. A machine learning model can be executed to determine whether a difference between image frames is likely an impact of a missile on a target 504. Target 504 can be re-registered, for example, by applying a hill climbing technique to possible corner coordinates as in the target initialization step. The short-term moving average difference is compared to the long-term moving average, and the difference is windowed 602a, 602b, 602c, generating a difference image 612 between the current target and the long-term target average.

[0061] The difference image can be convolved with a shock kernel (e.g., a windowed kernel that is scanned over the image differences). Any point that exceeds the long-term exponential moving average (EMA) of the maximum convolution value by a few standard deviations is marked as a possible shock 604a, 604b, 604c.

[0062] The possible shocks 604a-604c are fed into a machine learning model (e.g., a classifier) ​​606, which determines whether the difference is likely to be an actual shock. If the difference is above a threshold, the system labels the difference as an actual shock 608. However, if the difference is below the threshold, the system labels the difference as a false shock 610.

[0063] Figure 7 Scoring of impacts on a target 504 is shown and described. Different scoring zones can be determined by the system based on computer vision, by referencing registered targets from previous shooting campaigns, by retrieving stored target models from a database of known targets, or in some other manner. Scoring zones 702 on the target can be represented as the union of one or more simple shapes (e.g., ellipse, rectangle, circle, triangle, etc.). The coordinates of detected impacts 704 can be normalized and converted to the axes implied by the reference image. In other words, the impact can be superimposed on the reference image, and the reference image can be used to determine the coordinates of the impact. The coordinates can be Cartesian coordinates expressed as x and y values. In some cases, the coordinates can be radial coordinates, which represent the impact as an angle and distance (e.g., the distance from the center of the target) by expressing the coordinates in (r, θ) coordinate space. The system can then determine whether the impact is completely within a single scoring zone or intrudes into the scoring zone boundary, which allows the system to accurately score the impact. The system can use simple geometry to determine whether any significant portion of a given impact is within any simple shape in each target zone.

[0064] In certain embodiments, the coordinates of the impact are used for further analysis by the system. For example, by generating and storing the coordinates of a given shooting string, the spread (grouping) can be quantified, which can be used as a measure of improvement over time. Similarly, the shooter's angular momentum (moment of angle, MOA) can be determined, which is a measure of the spread size from center to center and edge to edge in inches and minutes of anangle. Spread can also be used to define posture, grip or motion errors during the shooting string. Spread can be quantified, including the size of the spread, the rotation of the spread or other metrics.

[0065] Scoring can be quantified using any suitable metric. In some cases, scoring is based on points, with each zone of the target receiving points added or subtracted from the initial amount. In some cases, missed shots or extra shots fired are scored as negative or higher values, depending on the type of scoring. In some cases, timed scoring is used, where the total time is reflected in the score, and misses can be penalized by increasing the time. In some cases, spread size is used to determine the score, and extra shots or missed shots may penalize spread size. Of course, other metrics and combinations of metrics can be determined by the system used to score a particular shot string.

[0066] The system can be configured to return, for each shot in a string, the coordinates of the impact and the time at which the shot occurred in such a way that multiple metrics combining position and time can be used to range the string. For example, the time between shots can be measured, or the shots following a buzzer or other activation signal can be tracked and stored with a measure of accuracy.

[0067] In some cases, when a hit is identified, the system can draw a bounding box around one or more hits. When the shot string is complete, the system can draw a bounding box containing each shot within the spread and determine a score based on the bounding box metrics. Figure 8 As shown, Figure 8 A user interface 800 of a system developed and operable according to some embodiments described herein is shown, which can be configured to determine a bounding box 802, which can pass through the center of the outermost impact or along the edge of the impact. The system can determine any of a number of metrics, such as, but not limited to, spread size 804, total spread width 806, spread height 808, bounding box rotation angle, MOA, elevation offset 810, windage offset 812, and can further determine a shot distance 814, which can be manually entered or determined based on the time of flight of the detected projectile.

[0068] For example, the system can be configured to record the sound of a gunshot, the shockwave of a projectile or powder detonation, the movement of the firearm or shooter, or some other indication that a shot has been fired. The system can then detect when the impact on the target occurs and determine the time of flight of the ammunition and, based on the firearm, ammunition, and / or powder load, the target distance. This process can be accomplished in near real time using a simple consumer-grade mobile computing device. In some cases, the mobile computing device can utilize the zoom feature of a built-in image capture device. In some cases, an external zoom lens can be used by the mobile computing device to acquire image frames. For example, a mobile phone can be coupled to a scope that provides optical zoom through the scope to allow the mobile computing device to capture clearer images of targets that may be within the target area. Some mobile computing devices can rely on digital zoom to capture one or more images of targets positioned within the target area.

[0069] The system can further determine and display the number of shots fired in the current shooting string 816, the average split time 818 of each shot, which can be helpful for timing shooting events. The system can also show the score 820 associated with each shot and the cumulative score 822 of the shooting string.

[0070] The system may include one or more processors and one or more non-transitory computer-readable media that may store various modules, applications, programs, or other data. The computer-readable media may include instructions that, when executed by one or more processors, cause the processors to perform the operations described herein for the system.

[0071] In some embodiments, the processor may include a central processing unit (CPU), a graphics processing unit (GPU), both a CPU and a GPU, a microprocessor, a digital signal processor, or other processing units or components known in the art. Alternatively or additionally, the functions described herein may be performed at least in part by one or more hardware logic components. For example, but not limited to, illustrative types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), etc. In addition, each of the processors can have its own local memory, which can also store program modules, program data, and / or one or more operating systems. One or more control systems, computer controllers, and remote controllers may include one or more cores.

[0072] Embodiments can be provided as a computer program product including a non-transitory machine-readable storage medium having instructions (in compressed or uncompressed form) stored thereon that can be used to program a computer (or other electronic device) to perform the process or method described herein. Computer-readable media can include volatile and / or non-volatile memory, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Machine-readable storage media can include, but are not limited to, hard drives, floppy disks, optical disks, CD-ROMs, DVDs, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, flash memories, magnetic or optical cards, solid-state memory devices, or other types of media / machine-readable media suitable for storing electronic instructions. In addition, embodiments can also be provided as a computer program product including transient machine-readable signals (in compressed or uncompressed form). Examples of machine-readable signals, whether or not using carrier modulation, include, but are not limited to, signals that a computer system or a machine hosting or running a computer program can be configured to access, including signals downloaded via the Internet or other networks.

[0073] Those skilled in the art will recognize that any process or method disclosed herein may be modified in a variety of ways. The process parameters and step sequences described and / or illustrated herein are provided as examples only and may be changed as needed. For example, although the steps illustrated and / or described herein may be shown or discussed in a particular order, the steps do not necessarily need to be performed in the order shown or discussed.

[0074] The various exemplary methods described and / or shown herein may also omit one or more steps described or shown herein, or include additional steps in addition to these disclosed steps. In addition, the steps of any method as disclosed herein may be combined with any one or more steps of any other method as disclosed herein.

[0075] The present disclosure sets forth example embodiments and, therefore, is not intended to limit the scope of the embodiments of the present disclosure and the appended claims in any way. The embodiments have been described above with the help of functional building blocks that illustrate the implementation of specific functions and their relationships. For ease of description, the boundaries of these functional building blocks have been arbitrarily defined herein. Alternative boundaries may be defined to the extent that the specified functions and their relationships are appropriately performed.

[0076] The foregoing description of specific embodiments will fully reveal the general nature of the embodiments of the present disclosure so that others can, by applying the knowledge of one of ordinary skill in the art, readily modify and / or adapt various applications of such specific embodiments without departing from the general concepts of the embodiments of the present disclosure, without undue experimentation. Therefore, such adaptations and modifications are intended to be within the meaning and range of equivalents of the disclosed embodiments, based on the teachings and guidance presented herein. The wording or terminology herein is for the purpose of description and not for the purpose of limitation, so that the terms or wording of this specification will be interpreted by one of ordinary skill in the relevant art in light of the teachings and guidance presented herein.

[0077] The breadth and scope of embodiments of the present disclosure should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.

[0078] Conditional language, such as "can," "could," "might," or "may," among others, unless specifically stated otherwise or understood otherwise in the context as used, is generally intended to convey that certain embodiments may include certain features, elements, and / or operations, while other embodiments do not. Thus, such conditional language is generally not intended to imply that features, elements, and / or operations are in any way required for one or more embodiments, or that one or more embodiments must include logic for determining, with or without user input or prompting, whether such features, elements, and / or operations are included in or to be performed in any particular embodiment.

[0079] Unless otherwise specified, the terms "connected to" and "coupled to" (and their derivatives) as used in the specification should be interpreted as allowing both direct and indirect connections (i.e., via other elements or components). Furthermore, the terms "a" or "an" as used in the specification should be interpreted as meaning "at least one of." Finally, for ease of use, the terms "including" and "having" (and their derivatives) as used in the specification do not exclude additional components and should be interpreted as open-ended.

[0080] The specification and drawings disclose examples of systems, devices, apparatus, and techniques that can provide systems and methods for determining the acoustic signature of a fired firearm. Of course, it is not possible to describe every conceivable combination of elements and / or methods for the purpose of describing the multiple features of the present disclosure, but one of ordinary skill in the art recognizes that many additional combinations and permutations of the disclosed features are possible. Therefore, various modifications may be made to the present disclosure without departing from the scope or spirit of the present disclosure. In addition, other embodiments of the present disclosure may be apparent from consideration of the specification and drawings, as well as from practice of the disclosed embodiments as presented herein. The examples set forth in the specification and drawings should be considered in all respects to be illustrative and not restrictive. Although specific terms are employed herein, these terms are used in a general and descriptive sense only and are not intended to be limiting.

[0081] Those skilled in the art will appreciate that, in some embodiments, the functionality provided by the above-described processes and systems can be provided in alternative ways, such as by being split between more software programs or routines or by being merged into fewer programs or routines. Similarly, in some embodiments, the processes and systems shown can provide more or less functionality than described, such as when other illustrated processes lack or include such functionality, or when the amount of functionality provided is changed. In addition, although various operations can be illustrated as being performed in a particular manner (e.g., serially or in parallel) and / or in a particular order, those skilled in the art will appreciate that, in other embodiments, operations can be performed in other orders and in other ways. Those skilled in the art will also appreciate that the data structures discussed above can be constructed in different ways, such as by splitting a single data structure into multiple data structures or by merging multiple data structures into a single data structure. Similarly, in some embodiments, the data structures shown can store more or less information than described, such as when other illustrated data structures lack or include such information, or when the amount or type of stored information is changed. The various methods and systems shown in the accompanying drawings and described herein represent example embodiments. In other embodiments, the methods and systems can be implemented in software, hardware, or a combination thereof. Similarly, in other embodiments, the order of any method may be changed, and various elements may be added, reordered, combined, omitted, modified, etc.

[0082] In light of the foregoing, it should be understood that although specific embodiments have been described herein for illustrative purposes, various modifications may be made without departing from the spirit and scope of the appended claims and the elements described therein. In addition, although certain aspects are presented above in the form of certain claims, the inventors envision multiple aspects of any available claim form. For example, although only some aspects may be described as embodied in a specific configuration at present, other aspects may also be embodied in this manner. As will be apparent to those skilled in the art who benefit from this disclosure, various modifications and changes may be made. It is intended to include all such modifications and changes, and therefore, the above description should be considered illustrative rather than restrictive.

Claims

1. A method for automatically scoring a shooting target, comprising: receiving one or more images of a target; generating a bounding box around the object; categorizing the target; determining a scoring ring for the objective; receiving one or more additional images of the target; determining a difference between a short-term moving average and a long-term moving average based on the received image of the target; filtering a difference between the short-term moving average and the long-term moving average to determine a shock; as well as A shock classifier is executed that receives the difference and outputs a score for the shock.

2. The method according to claim 1, wherein Receiving the one or more images is performed by a video capture device associated with the mobile computing device.

3. The method according to claim 1, wherein The method is executed on a smart phone.

4. The method according to claim 1, wherein Determining the difference between the short term moving average and the long term moving average includes convolving the difference image with a shock kernel.

5. The method according to claim 4, wherein The impact kernel is a 5×5, uniformly weighted square kernel.

6. The method according to claim 1, wherein The shock classifier is configured to replace a window surrounding a potential shock in a long-term moving average image with the short-term moving average that includes the potential shock.

7. The method of claim 6, further comprising replacing a window around a potential impact in the long-term moving average image with the short-term moving average for at least 5 subsequent frames that includes the potential impact.

8. The method according to claim 6, wherein: The window is at least a 10×10 pixel window.

9. The method according to claim 1, wherein Generating a bounding box around the object includes iteratively modifying one or more corner points of the bounding box to generate a modified bounding box, and projecting the modified bounding box onto a reference object image.

10. The method of claim 9, further comprising determining corner points of the modified bounding box by a hill climbing technique.

11. The method of claim 1 , further comprising determining coordinates of the impact and determining that the coordinates of the impact are within a scoring ring.

12. A system for automatically scoring a target, comprising: A computing device having one or more processors configured with instructions comprising: an image analysis module configured to receive one or more images from an imaging sensor associated with the computing device; an object classifier configured to determine one or more boundaries of the object and to determine geometric boundaries of one or more scoring regions; an impact kernel configured to compare two or more images and determine a possible impact; a machine learning model configured to receive data associated with the possible impact and determine a likelihood of an actual impact; and A scoring module is configured to determine a score for the impact if the likelihood of the actual impact exceeds a threshold.

13. The system according to claim 12, wherein: The computing device is a smartphone.

14. The system according to claim 12, wherein: The object classifier is configured to apply object detection to the one or more images and determine corner points of the object.

15. The system according to claim 12, wherein: The scoring module is configured to determine the coordinates of the impact and to determine that the impact is within a scoring zone of the target.

16. The system according to claim 15, wherein: The scoring module is configured to convolve an impact kernel on the difference image to identify the location of the impact.

Images