An automatic target reporting system and method based on dual photoelastic point positioning technology

Abstract

The present application relates to the technical field of automatic target reporting, and particularly relates to an automatic target reporting system and method based on double photoelastic point positioning technology. The present application first acquires an infrared thermal imaging image of a positioning target collected by an infrared thermal imager and a visible light image collected by a visible light camera, and pre-processes the infrared thermal imaging image and the visible light image to obtain a pixel point matrix corresponding to the infrared thermal frame image and the visible light frame image. Then, based on the pixel point matrix, the present application performs pixel difference processing on two continuous infrared thermal frame images to obtain a difference image. Then, the present application performs binarization on the difference image to obtain a binarized image, and removes dark points of the pixel point matrix on the binarized image to obtain a connected domain. The present application can utilize thermal imaging characteristics and time sequence continuous frames to identify and analyze suspicious bullet points, and can not only distinguish the sequence of different bullet points, but also detect overlapping bullet points.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic target reporting, in particular to an automatic target reporting system and method based on dual-optical point positioning technology. BACKGROUND

[0002] Live ammunition shooting of light weapons as an important part of military basic training is crucial to improve the combat skills and reaction ability of soldiers. With the continuous progress of intelligent technology, the application of automatic target reporting systems has gradually become popular. They not only improve the safety and efficiency of shooting training, but also greatly improve the accuracy and scientificity of training.

[0003] Video target is a new automatic target reporting technology that uses high-definition cameras to capture target images and determines the point of impact through image processing technology. However, the traditional video target only uses visible light cameras to collect images of the target, and the identification of the bullet point does not have a time sequence characteristic. In the case of continuous shooting, it is impossible to distinguish the position of the newly added bullet point, especially when the bullet points coincide, it is impossible to detect the later coinciding bullet points. SUMMARY

[0004] Therefore, the present application provides an automatic target reporting system and method based on dual-optical point positioning technology to solve the problem of traditional video target identification of bullet points without time sequence characteristics.

[0005] In a first aspect, the present application provides an automatic target reporting method based on dual-optical point positioning technology, comprising the following steps:

[0006] S1, obtaining the infrared thermal imaging image of the positioning target collected by the infrared thermal imager and the visible light image collected by the visible light camera, and pre-processing the infrared thermal imaging image and the visible light image to obtain the pixel point matrix corresponding to the infrared thermal frame image and the visible light frame image;

[0007] S2, based on the pixel point matrix, performing pixel difference processing on the two consecutive infrared thermal frame images to obtain a difference image;

[0008] S3, binarizing the difference image to obtain a binary image, and removing dark points of the pixel point matrix on the binary image to obtain a connected domain;

[0009] S4, removing the pixel points in the connected domain that do not meet the preset circularity shape and the preset area threshold;

[0010] S5, based on the area brightness gradient information in the visible light image corresponding to the connected domain, obtaining the area with positive brightness gradient from the center to the periphery, and correspondingly removing the pixel points of the pixel point matrix to obtain a target connected domain;

[0011] S6. Calculate the target shooting score information based on the positional relationship between the target connected domain and the pre-set target corner feature markers in the visible light camera.

[0012] In one optional implementation, binarizing the difference image to obtain a binarized image includes:

[0013] Calculate the average pixel value of the infrared thermal frame image at the next moment and the average pixel value of the infrared thermal frame image at the previous moment by binarizing the two consecutive infrared thermal frame images corresponding to the difference image according to the Otsu method.

[0014] The difference between the average pixel value of the infrared thermal frame image at the next moment and the average pixel value of the infrared thermal frame image at the previous moment is calculated and used as the binarization threshold.

[0015] Binarization is performed on the difference image based on the binarization threshold to obtain a binarized image.

[0016] In one alternative implementation, it further includes:

[0017] S7, acquire the infrared thermal frame image from step S2 and the infrared thermal frame image that is the interval frame between them, and perform pixel difference;

[0018] S8. Repeat steps S3 to S5 to obtain the target connected component.

[0019] S9. Compare the target connected regions obtained in step S5 with those obtained in step S8, retain the connected regions at the same location, and calculate the shooting score based on the positional relationship between the retained connected regions at the same location and the visible light camera markers.

[0020] In an optional implementation, the method further includes: automatic calibration of the extrinsic parameter matrices of the infrared thermal imager and the visible light camera, including:

[0021] The marker points on the target surface in the visible light image are obtained by corner detection;

[0022] The infrared thermal imaging image of the target surface corresponding to the corner detection mark point is obtained by artificially heating the mark point and the infrared thermal imaging image after artificial heating, and then the infrared mark point point is obtained by differential processing.

[0023] Match corner detection marker points and infrared marker points;

[0024] Calculate the transformation matrix between the pixel coordinates of the corner detection marker and the infrared marker, and label it as the extrinsic transformation matrix between the infrared thermal imager and the visible light camera;

[0025] The target surface markers in the visible light image are obtained by corner detection and matched with the corresponding markers in the visible light camera.

[0026] The transformation matrix between the pixel coordinates of the target surface marker and the pixel coordinates of the visible light camera marker is calculated, and the calculated transformation matrix is ​​superimposed on the extrinsic transformation matrix of the infrared thermal imager and the visible light camera for automatic calibration of the extrinsic transformation matrix of the infrared thermal imager and the visible light camera.

[0027] In one alternative implementation, it further includes:

[0028] Corner detection is performed on two consecutive visible light frames to obtain the tracked corner points;

[0029] Forward optical flow tracking is performed on the tracking corner points in two consecutive visible light frames to obtain optical flow tracking point pairs;

[0030] Perform reverse optical flow tracking on the obtained optical flow tracking point pairs to obtain reverse tracking points, and discard optical flow tracking point pairs in which the error between the tracking corner point and the corresponding reverse tracking point is greater than a preset error;

[0031] Calculate the sum of pixel errors for all optical flow tracking point pairs. If the sum of pixel errors is greater than a preset scene threshold, the scene is determined to have changed. If the sum of pixel errors is less than or equal to the preset scene threshold, the scene is determined not to have changed. The preset scene threshold is the average value of the sum of optical flow tracking point errors for a number of consecutive visible light frames.

[0032] Secondly, the present invention provides an automatic target reporting system based on dual-optical bullet point positioning technology, comprising:

[0033] The image acquisition module is used to acquire infrared thermal imaging images of the positioning target collected by the infrared thermal imager and visible light images collected by the visible light camera, and to preprocess the infrared thermal imaging images and visible light images to obtain the pixel matrix corresponding to the infrared thermal frame image and the visible light frame image.

[0034] The infrared differential image acquisition module is used to perform pixel differential processing on two consecutive infrared thermal frame images based on the pixel matrix to obtain a differential image;

[0035] The connected component acquisition module binarizes the difference image to obtain a binarized image, and removes dark points from the pixel matrix on the binarized image to obtain the connected components.

[0036] The pixel removal module removes pixels in the connected component that do not conform to the preset roundness shape and preset area threshold.

[0037] The target connected component acquisition module acquires the regional brightness gradient information in the visible light image corresponding to the connected component of the visible light image, removes regions with positive brightness gradients from the center to the surrounding areas, and removes the corresponding pixels in the pixel matrix to obtain the target connected component.

[0038] The target shooting performance information acquisition module calculates target shooting performance information based on the positional relationship between the target connected domain and the preset target corner feature markers in the visible light camera.

[0039] Thirdly, the present invention provides an automatic target reporting device based on dual-optical bullet point positioning technology, comprising:

[0040] Target positioning;

[0041] A dual-light bullet point acquisition device includes an infrared thermal imager and a visible light camera. The infrared thermal imager is suitable for obtaining infrared thermal imaging images of the positioning target, and the visible light camera is suitable for obtaining visible light images of the positioning target. The visible light camera is equipped with several marker points based on the corner features of the target surface.

[0042] The processor is connected to both an infrared thermal imager and a visible light camera to execute the target information obtained by the automatic target reporting method based on dual-light bullet point positioning technology.

[0043] A display is connected to the processor.

[0044] In one alternative embodiment, the display includes a handheld display and a viewing display, both connected to the processor. The handheld display is adapted to display infrared thermal imaging images and visible light images, and the viewing display is adapted to display target information.

[0045] Fourthly, the present invention provides a computer device, comprising:

[0046] The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes these computer instructions to perform the aforementioned automatic target reporting method based on dual-optical bullet point positioning technology.

[0047] Fifthly, the present invention provides a computer-readable storage medium storing computer instructions, the computer instructions being used to cause a computer to execute the above-described automatic target reporting method based on dual-optical bullet point positioning technology.

[0048] This invention first acquires infrared thermal imaging images of the target from an infrared thermal imager and visible light images from a visible light camera, and preprocesses these images to obtain pixel matrices corresponding to the infrared thermal and visible light frames. Then, based on the pixel matrices, pixel difference processing is performed on two consecutive infrared thermal frames to obtain a difference image. Next, the difference image is binarized to obtain a binarized image, and dark points in the pixel matrix are removed from the binarized image to obtain connected components. Then, pixels in the connected components that do not conform to a preset roundness shape and a preset area threshold are removed. Next, based on the regional brightness gradient information in the visible light image corresponding to the connected components, regions with positive brightness gradients from the center outwards are obtained, and the corresponding pixels in the pixel matrix are removed to obtain the target connected component. Finally, based on the positional relationship between the target connected component and preset target corner feature markers in the visible light camera, the target shooting score is calculated.

[0049] This invention utilizes thermal imaging characteristics and sequential frames to identify and analyze suspicious bullet points, distinguishing not only the order of different bullet points but also detecting overlapping bullet points. Through dual-light fusion technology, bullet points can be accurately detected, maintaining high positioning accuracy even in complex environments.

[0050] Traditional automatic target reporting systems may rely on a single type of sensor, while this invention, by combining infrared and visible light images, can overcome the limitations of a single sensor, thereby achieving more accurate bullet point detection and positioning. Attached Figure Description

[0051] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0052] Figure 1 This is a schematic diagram of the automatic target reporting method according to an embodiment of the present invention;

[0053] Figure 2 This is a structural block diagram of the automatic target reporting device according to an embodiment of the present invention;

[0054] Figure 3 This is a block diagram of the automatic target reporting system according to an embodiment of the present invention;

[0055] Figure 4 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention;

[0056] Figure 5This is a matching diagram of visible light feature corner points and thermal imaging hot spots in an embodiment of the present invention;

[0057] Figure 6 This is a schematic diagram of optical flow tracking point pairs according to an embodiment of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] Live-fire exercises with small arms are a crucial component of basic military training, vital for improving soldiers' combat skills and responsiveness. With the continuous advancement of intelligent technology, the application of automatic target reporting systems has become increasingly widespread. These systems not only enhance the safety and efficiency of shooting training but also significantly improve its accuracy and scientific rigor.

[0060] Currently, automatic target reporting systems are mainly classified into traditional photoelectric targets, acoustic-electric targets, double-electrode short-circuit targets, and video targets. Traditional photoelectric targets detect the impact point using photoelectric sensors, offering advantages such as fast response speed and high reporting accuracy, but they are costly and complex to maintain. Acoustic-electric targets use sound and electrical signals to determine the impact point, offering lower cost and high accuracy, but requiring professional operation. Double-electrode short-circuit targets detect the projectile's impact position through an electrode grid on the target surface, offering high accuracy, but are costly and have a shorter lifespan.

[0061] Video targets represent an emerging automatic target reporting technology. They utilize high-definition cameras to capture images of the target surface and determine the bullet impact point through image processing. This method is low-cost, highly practical, has great development potential, and is suitable for complex outdoor environments, adapting to changes in natural lighting conditions. Video target systems improve the automation of target reporting by analyzing the acquired video sequences in real time for shot detection and bullet hole location, and are applicable to complex outdoor environments under natural lighting conditions. However, traditional video targets only use visible light cameras to acquire images of the target surface, making them highly susceptible to changes in ambient lighting, the chosen algorithm, and threshold parameters. When the scene changes, the selected algorithm and parameters may not meet the actual accuracy requirements, necessitating readjustment and testing. Furthermore, traditional video targets lack temporal characteristics in bullet point recognition, making it impossible to distinguish the location of newly added bullet points during continuous firing, especially when bullet points overlap, making it impossible to detect later overlapping bullet points.

[0062] The following is combined Figures 1 to 6The following describes embodiments of the present invention.

[0063] Example 1

[0064] This invention provides an automatic target reporting method based on dual-optical bullet point positioning technology, such as... Figure 1 As shown, it includes the following steps:

[0065] S1. Acquire the infrared thermal image of the target from an infrared thermal imager and the visible light image from a visible light camera. Preprocess the infrared thermal and visible light images to obtain the corresponding pixel matrices. Preprocessing includes denoising, alignment, and registration of the infrared thermal and visible light images. Due to environmental factors or external interference, the acquired images may contain noise. Filtering algorithms (such as Gaussian filtering and mean filtering) can be used to remove image noise to improve the accuracy of subsequent analysis. There may be viewing angle differences between the infrared thermal and visible light images; therefore, registration (image alignment) is required to ensure spatial alignment. Feature point matching or global optimization algorithms can be used to solve the image alignment problem. Preprocessing also includes processing the infrared thermal imaging image and the visible light image into video frames to obtain infrared thermal frame images and visible light frame images. It should be noted that the infrared thermal frame images and the visible light frame images are obtained based on the same positioning target. Each of them yields two pixel matrixes, and the two pixel matrixes can correspond one-to-one. Therefore, a pixel matrix that can characterize the pixels of the infrared thermal imaging image and the visible light image can be obtained.

[0066] S2. Based on the pixel matrix, perform pixel difference processing on two consecutive infrared thermal frame images to obtain difference images, which can be denoted as the k-th frame and the (k+1)-th frame infrared thermal frame images. For infrared thermal imaging images, heat changes are detected by the difference between consecutive frames. For example, when a bullet hits a target, the target area experiences instantaneous thermal changes. By differencing adjacent frame images, regions of abrupt thermal changes can be captured; these regions may be the point of impact.

[0067] S3. Binarize the difference image to obtain a binarized image, and remove the dark points of the pixel matrix on the binarized image to obtain the connected components.

[0068] S4. Remove pixels in connected components that do not conform to the preset roundness shape and preset area threshold. This can be done by performing a roundness check on the connected component, specifically by roundness fitting, to exclude pixels that do not conform to the preset roundness shape, including strip-shaped connected components. It can also exclude connected components with small areas (approximately 5*5 pixels) or excessively large areas (approximately 100*100 pixels).

[0069] S5. Based on the regional brightness gradient information in the visible light image corresponding to the connected component, obtain the region with a positive brightness gradient from the center to the surrounding area, and remove the corresponding pixels in the pixel matrix to obtain the target connected component; the real bullet points present a unimodal distribution similar to a normal distribution, that is, the pixel temperature gradually decreases from the center of the connected component to the surrounding area, that is, the gradient is negative, and the connected components with positive gradient values ​​are removed.

[0070] By calculating the brightness gradient of each connected region, regions with abnormal gradients (such as regions with rising gradients) are eliminated. Specifically, the true point of impact should be a heat source whose temperature gradually decreases around the center point, therefore its brightness gradient should be negative.

[0071] S6. Calculate the target shooting score information based on the positional relationship between the target connected domain and the pre-set target corner feature markers in the visible light camera.

[0072] In thermal imaging images, the heat changes at the bullet points decay over time. By performing differential processing on consecutive frames, it is possible to distinguish between new and old bullet points.

[0073] This invention performs differential calculations on two consecutive thermal imaging frames to determine whether the detected bullet point region appeared in the previous frame. If a region already appeared in the previous frame and its temperature value has decreased, it can be determined that the region is an old bullet point.

[0074] New bullet points have a higher temperature, which gradually decreases over time. By comparing the temperature information in the current frame's thermal imaging image with that in previous frames, it can be determined whether a bullet point is newly appeared or an older bullet point that has cooled down.

[0075] In one optional implementation, binarizing the difference image to obtain a binarized image includes:

[0076] The average pixel value of the infrared thermal frame image at the next moment and the average pixel value of the infrared thermal frame image at the previous moment are obtained by calculating the average pixel value of the two consecutive infrared thermal frame images corresponding to the difference image after binarization using the Otsu method. It should be noted that the present invention may not directly perform binarization using the Otsu method, but instead calculate the threshold of the Otsu method from the difference image, and then use the threshold of the Otsu method to remove pixels with pixels lower than the threshold of the Otsu method, and then calculate its average pixel value.

[0077] The difference between the average pixel value of the infrared thermal frame image at the next moment and the average pixel value of the infrared thermal frame image at the previous moment is calculated and used as the binarization threshold.

[0078] Binarization is performed on the difference image based on the binarization threshold to obtain a binarized image.

[0079] By setting a dynamic threshold for each frame of the differential image, the system can identify high-temperature areas (i.e., the point of impact) in the image. Specifically, the dynamic threshold is determined by the brightness distribution of the image (e.g., using the "Otsu method" for automatic thresholding), ensuring stable identification of the point of impact under different ambient lighting conditions.

[0080] The formula used is: Td = HM(k+1) - HM(k), where HM(k) is the average value of the bright area in the k-th frame, i.e., the average pixel value (HighMean) of the bright area after binarization, and also the average pixel value after binarization using Otsu's method. Td is the difference threshold, and Td is the dynamic threshold. After binarization, pixels with increased temperature can be identified. The main factor causing the increase in pixel temperature is the heat generated by friction between the bullet and the target when the bullet passes through it. Pixels with increased temperature (i.e., pixel differences greater than the binarization threshold) are marked as 255, and other pixels are marked as 0. Therefore, the area formed by pixels with increased temperature is considered a suspicious bullet point.

[0081] In one alternative implementation, it further includes:

[0082] S7, acquire the infrared thermal frame image from step S2 and the infrared thermal frame image that is the interval frame between them, and perform pixel difference;

[0083] S8. Repeat steps S3 to S5 to obtain the target connected component.

[0084] S9: Compare the target connected components obtained in step S5 with those obtained in step S8, retain connected components at the same location, and calculate the shooting score based on the positional relationship between the retained connected components at the same location and the visible light camera marker. To prevent false detections due to noise or target surface jitter in single-frame recognition, subtract the k-frame and k+2-frame and repeat steps S3 to S5. Compare and verify the two target connected components obtained; only those that pass the verification are ultimately output as bullet points. It should be noted that the interval frame can be represented as infrared thermal image of k-frame and k+2-frame, or as infrared thermal image of k-frame and k+3-frame, etc. The specific number of interval frames is not limited to this in this embodiment.

[0085] By performing differential calculations on consecutive frames, the location of the same impact point can be confirmed at multiple time points. To avoid misjudgments due to factors such as lens shake and changes in lighting, the identified connected components are verified in multiple frames. If a region remains consistent across consecutive frames, that region can be identified as the impact point.

[0086] In an optional implementation, the method further includes: automatic calibration of the extrinsic parameter matrices of the infrared thermal imager and the visible light camera, including:

[0087] The marker points on the target surface in the visible light image are obtained by corner detection;

[0088] The infrared thermal imaging image of the target surface corresponding to the corner detection mark point is obtained by artificially heating the mark point and the infrared thermal imaging image after artificial heating, and then the infrared mark point point is obtained by differential processing.

[0089] Match corner detection marker points and infrared marker points;

[0090] The transformation matrix between the pixel coordinates of the corner detection markers (specifically, the pixel coordinates of the corner detection markers in the visible light image) and the infrared markers (specifically, the pixel coordinates of the infrared markers in the infrared thermal image) is calculated and denoted as the extrinsic parameter transformation matrix between the infrared thermal imager and the visible light camera. This can be considered as a process of establishing a one-to-one correspondence between the pixel matrix of the infrared thermal image and the pixel matrix of the visible light image.

[0091] It is important to note that the extrinsic parameter transformation matrix from the target surface to the infrared thermal imager comprises two parts: the transformation matrix between the target surface and the visible light plane, and the transformation matrix between the visible light plane and the thermal imaging plane. The extrinsic parameter transformation matrix from the target surface to the infrared thermal imager can be obtained from these transformation matrices.

[0092] The conversion parameter between the thermal imaging plane and the visible light plane is called the dual-light camera extrinsic parameter T. cR Bullet mark pixels identified using infrared thermal imager ρ R Using the extrinsic parameters T of the dual-light camera cR Transform to the visible light image plane and obtain the corresponding pixel point ρ. c .

[0093] Visible light cameras extract target markers using corner detection. Infrared thermal imagers require manual marking of target markers with hot stamping, followed by differential processing of the images before and after marking. This allows for the creation of eight pairs of matching points with the visible light camera, such as... Figure 5 The image shows the matching diagram between visible light feature corner points and thermal imaging hotspots. The extrinsic parameters T of the two-light camera can be solved based on the spatial coordinate transformation relationship and singular value decomposition. cR :

[0094]

[0095] The conversion matrix between visible light and the target surface can be calibrated in real time using an automatic correction method to ensure stable accuracy, as shown below.

[0096] The target surface markers in the visible light image are obtained by corner detection and matched with the corresponding markers in the visible light camera.

[0097] The transformation matrix between the pixel coordinates of the target surface marker (specifically, the pixel coordinates of the target surface marker in the visible light image) and the marker of the visible light camera (the specific position is set by the factory) is calculated, and the calculated transformation matrix is ​​superimposed on the extrinsic transformation matrix of the infrared thermal imager and the visible light camera for automatic calibration of the extrinsic transformation matrix of the infrared thermal imager and the visible light camera. This can be regarded as automatic calibration of the extrinsic transformation matrix from the target surface to the infrared thermal imager.

[0098] Before leaving the factory, the extrinsic parameters of the infrared thermal imager and the visible light camera can be calibrated to obtain the extrinsic parameters between the two lights, which is the transformation matrix between the visible light plane and the thermal imaging plane, and also the extrinsic parameter transformation matrix between the infrared thermal imager and the visible light camera. After factory installation, the relative pose of the infrared thermal imager and the visible light camera does not change, meaning the transformation matrix between the visible light plane and the thermal imaging plane remains unchanged. Therefore, during automatic calibration, it is only necessary to recalibrate the transformation matrix between the target surface and the visible light plane, and then superimpose the transformation matrix between the target surface marker points and the visible light camera marker points onto the transformation matrix between the visible light plane and the thermal imaging plane.

[0099] In one alternative implementation, it further includes:

[0100] Corner detection is performed on two consecutive visible light frames to obtain the tracked corner points;

[0101] Forward optical flow tracking is performed on the corner points in two consecutive visible light frames to obtain optical flow tracking point pairs. Forward optical flow tracking is based on the assumption of pixel movement in the image, assuming that the movement of pixels in consecutive frames is smooth within a small range. By calculating the motion vector (optical flow) of each pixel in the image, the motion of objects or targets can be tracked, for example, by extracting feature points from the current image frame, which are usually corner points or edge points (e.g., through Harris corner detection or the Shi-Tomasi algorithm).

[0102] Reverse optical flow tracking is performed on the obtained optical flow tracking point pairs to obtain reverse tracking points. Optical flow tracking point pairs where the error between the tracking corner point and the corresponding reverse tracking point is greater than a preset error are discarded. Reverse optical flow tracking involves tracing back through feature points in the current image to the corresponding position in the previous frame to verify whether the feature points are correctly matched. Through reverse optical flow tracking, an attempt is made to map the feature points in the current image to their corresponding positions in the previous frame. That is, given a point in the current image, its position in the previous frame is calculated using reverse optical flow, and it is checked whether it is within the feature point set of the previous frame.

[0103] Calculate the sum of pixel errors for all optical flow tracking point pairs. If the sum of pixel errors is greater than a preset scene threshold, the scene is determined to have changed. If the sum of pixel errors is less than or equal to the preset scene threshold, the scene is determined not to have changed. The preset scene threshold is the average value of the sum of optical flow tracking point errors for a number of consecutive visible light frames.

[0104] Considering that during operation, there may be cases where the lens is accidentally touched, leading to false alarms or missed alarms on the imaging surface, a real-time self-checking mechanism can be used to detect whether the scene inside the lens has changed, thereby avoiding false alarms and missed alarms.

[0105] The scene change self-checking algorithm relies on the original scene image stored from the previous automatic calibration to compare the current scene image pixel by pixel to determine whether the scene has changed and then decide to recalibrate automatically. The comparison method can use LK optical flow tracking. First, Harris corner point extraction is performed on the original scene image to detect the edge parts of the scene in the image, thereby obtaining key corner points. Then, optical flow tracking is used to perform optical flow matching on feature points in two frames.

[0106] To prevent mismatches, optical flow tracking is performed in reverse using tracking points in the current scene image to obtain reverse tracking points in the original scene. The errors of corner points and reverse tracking points are compared, and optical flow tracking points with larger errors are removed from the discrimination queue.

[0107] A pair of optical flow tracing points contains corner points H within the original scene image. r and the tracking point H of the current scene image. c The scene has changed, determined by the sum of the pixel L2 errors between these two points. The sum of pixel L2 errors is:

[0108]

[0109] If the sum of errors is greater than the preset scene threshold, the scene is considered to have changed; if the sum of errors is less than or equal to the preset scene threshold, the scene is considered to have changed.

[0110] Among them, u c and v c Tracking point H in the current scene image c pixel coordinates, u r and v r H is the corner point within the original scene image. r The pixel coordinates.

[0111] To increase the reliability of the judgment, the threshold thr is calculated based on the average error between consecutive frames of the scene image. Given a set of 5 consecutive frames F = {F1, F2, F3, F4, F5}, the sum of the optical flow point errors L2 between each consecutive frame is calculated, and the average of this sum is the threshold thr.Figure 6 As shown.

[0112]

[0113] Where thr is the preset scene threshold.

[0114] This invention can perform four parts: automatic parameter calibration, bullet point position calculation, abnormal situation judgment, and shooting performance evaluation.

[0115] This invention takes into account the stability of accuracy, as well as the cost and simplicity of debugging and maintenance. It can also perform automatic parameter calibration. After the infrared-visible light extrinsic matrix and lens distortion parameters are calibrated once at the factory, it enters the real-time automatic calibration stage, which monitors the shift of the conversion matrix between the visible light camera plane and the target surface in real time, thereby ensuring the confidence of accuracy during use and the convenience of debugging when used again.

[0116] To avoid false alarms and missed alarms, anomaly detection can be performed. By using time-series differential verification and optical flow tracking to detect changes, the elimination of differential anomalies in infrared thermal imaging can be ensured, and the occurrence of anomalies can be identified and corrected in a timely manner.

[0117] This invention obtains reliable and accurate bullet point coordinate records after steps such as correction, positioning, and anomaly elimination. It can be combined with the shooting performance evaluation module to estimate the circular error of the bullet point, thereby intuitively evaluating the shooting level of the trainee.

[0118] Example 2

[0119] This invention provides an automatic target reporting system based on dual-optical bullet point positioning technology, such as... Figure 3 As shown, it includes:

[0120] The image acquisition module 201 is used to acquire the infrared thermal imaging image of the positioning target collected by the infrared thermal imager and the visible light image collected by the visible light camera, and to preprocess the infrared thermal imaging image and the visible light image to obtain the pixel matrix corresponding to the infrared thermal frame image and the visible light frame image.

[0121] The infrared differential image acquisition module 202 is used to perform pixel differential processing on two consecutive infrared thermal frame images based on the pixel matrix to obtain a differential image.

[0122] The connected component acquisition module 203 binarizes the difference image to obtain a binarized image, and removes dark points from the pixel matrix on the binarized image to obtain the connected components.

[0123] The pixel removal module 204 removes pixels in the connected component that do not conform to the preset roundness shape and preset area threshold.

[0124] The target connected component acquisition module 205 acquires the regional brightness gradient information in the visible light image corresponding to the visible light image connected component, removes regions with positive brightness gradients from the center to the surrounding areas, and removes the corresponding pixels of the pixel matrix to obtain the target connected component.

[0125] The target shooting performance information acquisition module 206 calculates the target shooting performance information based on the positional relationship between the target connected domain and the preset target corner feature markers in the visible light camera.

[0126] Example 3

[0127] This invention provides an automatic target reporting device based on dual-optical bullet point positioning technology, such as... Figure 2 As shown, it includes:

[0128] Target positioning;

[0129] A dual-light bullet point acquisition device includes an infrared thermal imager and a visible light camera. The infrared thermal imager is suitable for obtaining infrared thermal imaging images of the positioning target, and the visible light camera is suitable for obtaining visible light images of the positioning target. The visible light camera is equipped with several marker points based on the corner features of the target surface.

[0130] The processor is connected to both an infrared thermal imager and a visible light camera to execute the target information obtained by the automatic target reporting method based on dual-light bullet point positioning technology. The processor can be an analysis host.

[0131] A display is connected to the processor.

[0132] In one alternative embodiment, the display includes a handheld display and a viewing display, both connected to the processor. The handheld display is adapted to display infrared thermal imaging images and visible light images, and the viewing display is adapted to display target information.

[0133] The handheld display can be a tablet device, which is located on the same wireless local area network as the dual-light bullet point acquisition unit and the analysis host. The original thermal imaging images and visible light camera image video streams are pushed to the handheld display for online display of the live shooting situation and results.

[0134] The monitor receives and analyzes the target bullet point coordinates and scores output by the host, and displays the target surface bullet point position, number of bullets, and total target score in real time.

[0135] This invention first acquires infrared thermal imaging images of the target from an infrared thermal imager and visible light images from a visible light camera, and preprocesses these images to obtain pixel matrices corresponding to the infrared thermal and visible light frames. Then, based on the pixel matrices, pixel difference processing is performed on two consecutive infrared thermal frames to obtain a difference image. Next, the difference image is binarized to obtain a binarized image, and dark points in the pixel matrix are removed from the binarized image to obtain connected components. Then, pixels in the connected components that do not conform to a preset roundness shape and a preset area threshold are removed. Next, based on the regional brightness gradient information in the visible light image corresponding to the connected components, regions with positive brightness gradients from the center outwards are obtained, and the corresponding pixels in the pixel matrix are removed to obtain the target connected component. Finally, based on the positional relationship between the target connected component and preset target corner feature markers in the visible light camera, the target shooting score is calculated.

[0136] This invention utilizes thermal imaging characteristics and sequential frames to identify and analyze suspicious bullet points, distinguishing not only the order of different bullet points but also detecting overlapping bullet points. Through dual-light fusion technology, bullet points can be accurately detected, maintaining high positioning accuracy even in complex environments.

[0137] Traditional automatic target reporting systems may rely on a single type of sensor, while this invention, by combining infrared and visible light images, can overcome the limitations of a single sensor, thereby achieving more accurate bullet point detection and positioning.

[0138] This invention enables real-time detection and positioning of live projectiles and provides auxiliary analysis of their trajectories. It features high precision, robustness, low maintenance requirements, flexible operation, and strong environmental adaptability. Compared to traditional photoelectric targets, which suffer from high maintenance costs and long setup times, this invention utilizes a dual-light camera as its primary acquisition device. Compared to photoelectric transceiver sensors, this camera has lower power consumption and system complexity, resulting in lower usage and maintenance costs. Furthermore, the dual-light target features automatic calibration and self-testing upon startup, requiring virtually no setup. Unlike acoustic-electric targets, which are affected by fundamental principles where any interference—including bullet shape, angle, and motion—distorts and disturbs the sound waves, leading to decreased accuracy, this invention primarily analyzes thermal and visible light images, maintaining stable, high-precision positioning unaffected by these factors. Compared to disposable targets like the double-electrode short-circuit target, which has higher usage costs and lower accuracy, this invention offers sustainable use and significantly higher accuracy. Compared to traditional video targets that only use visible light cameras for identification and cannot distinguish between new and old bullet points and overlapping bullet points, this invention uses thermal imaging characteristics and sequential continuous frames to identify and track suspicious bullet points. It can not only distinguish the order of different bullet points but also detect overlapping bullet points. Furthermore, it analyzes the trajectory of bullet points to discover problems in the user's shooting. The overall system capability has reached the international leading level.

[0139] This invention also provides a computer device having the above-described features. Figure 3 The automatic target reporting system shown is based on dual-optical bullet point positioning technology.

[0140] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 4 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 4 Take a processor 10 as an example.

[0141] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0142] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0143] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0144] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0145] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0146] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor central control systems, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the methods shown in the above embodiments are implemented.

[0147] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0148] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. An automatic target reporting method based on a dual photoelastic point positioning technique, characterized in that, The method comprises the following steps: S1, acquiring an infrared thermal imaging image of a positioning target collected by an infrared thermal imager and a visible light image collected by a visible light camera, and pre-processing the infrared thermal imaging image and the visible light image to obtain a pixel point matrix corresponding to the infrared thermal frame image and the visible light frame image, the infrared thermal frame image and the visible light frame image being obtained based on the same positioning target, two pixel point matrices being obtained respectively, and the two pixel point matrices corresponding to each other one by one; S2, performing pixel difference processing on two continuous infrared thermal frame images based on the pixel point matrix to obtain a difference image; S3, performing binarization on the difference image to obtain a binarized image, and removing dark points of the pixel point matrix on the binarized image to obtain a connected domain; S4, removing the pixel points in the connected domain that do not conform to a preset circularity shape and a preset area threshold; S5, based on the area brightness gradient information in the visible light image corresponding to the connected domain, obtaining an area with positive brightness gradient from the center to the periphery, and correspondingly removing the pixel points corresponding to the pixel point matrix to obtain a target connected domain; S6, based on the target connected domain and the position relationship between the preset target surface corner feature marker points in the visible light camera, calculating shooting score information; The binarization of the difference image to obtain the binarized image comprises: respectively calculating the pixel mean values of the two continuous infrared thermal frame images corresponding to the difference image after binarization according to the Otsu method, to obtain the pixel mean value of the infrared thermal frame image at the next moment and the pixel mean value of the infrared thermal frame image at the previous moment; calculating the difference between the pixel mean value of the infrared thermal frame image at the next moment and the pixel mean value of the infrared thermal frame image at the previous moment as a binarization threshold value; based on the binarization threshold value, performing binarization processing on the difference image to obtain a binarized image; wherein, the threshold value of the Otsu method is calculated from the difference image, and the pixel points with pixel values lower than the threshold value of the Otsu method are removed by using the threshold value of the Otsu method, and then the average pixel value is calculated; By setting a dynamic threshold value for each frame of difference image, the system identifies the high temperature area in the image, i.e. the impact point. The dynamic threshold value is determined by the brightness distribution of the image.

2. The automatic scoring method based on the dual photoelastic point positioning technology according to claim 1, characterized in that, Further comprising: S7, acquiring the infrared thermal frame image in step S2 and the infrared thermal frame image which is an interval frame of the infrared thermal frame image in step S2 and performing pixel difference processing; S8, repeating steps S3 to S5 to obtain a target connected domain; S9, comparing the target connected domain obtained in step S5 and the target connected domain obtained in step S8, retaining the connected domains at the same position, and based on the position relationship between the retained connected domains at the same position and the marker points in the visible light camera, calculating the shooting score.

3. The automatic scoring method based on the dual photoelastic dot positioning technology according to claim 1, characterized in that, Further comprising: automatic calibration of the extrinsic matrix of the infrared thermal imager and the visible light camera, comprising: obtaining the marker points of the target surface in the visible light image through corner point detection; acquiring the infrared thermal imaging image of the artificial hot spot on the position corresponding to the marker points of the corner point detection on the target surface and the infrared thermal imaging image after the artificial hot spot, performing difference processing to obtain infrared marker points; matching the marker points of the corner point detection and the infrared marker points; calculating the conversion matrix between the pixel coordinates of the marker points of the corner point detection and the infrared marker points, and marking it as the extrinsic conversion matrix of the infrared thermal imager and the visible light camera; Obtaining the target surface mark point in the visible light image through corner point detection, and matching the mark point in the visible light camera; Calculating the conversion matrix between the pixel coordinates of the target surface mark point and the pixel coordinates of the mark point in the visible light camera, and superimposing the calculated conversion matrix to the external parameter conversion matrix of the infrared thermal imager and the visible light camera, for automatic calibration of the external parameter conversion matrix of the infrared thermal imager and the visible light camera.

4. The automatic scoring method based on the dual photoelastic point positioning technology according to claim 3, characterized in that, Also includes: Respectively, the corner point detection is carried out on two continuous visible light frame images, and the tracking corner point is obtained; The forward optical flow tracking is carried out on the tracking corner points in the two continuous visible light frame images, and the optical flow tracking point pair is obtained; The reverse optical flow tracking is carried out on the obtained optical flow tracking point pair, and the reverse tracking point is obtained, and the optical flow tracking point pair whose tracking corner point and corresponding reverse tracking point error is greater than the preset error is eliminated; Calculating the total pixel error of all optical flow tracking point pairs, if the total pixel error is greater than the preset scene threshold, it is determined that the scene changes; If the total pixel error is less than or equal to the preset scene threshold, it is determined that the scene does not change, wherein the preset scene threshold is the average value of the total error of the continuous visible light frame image optical flow tracking points.

5. An automatic scoring system based on the dual photoelastic point positioning technique, characterized in that, Including: The image acquisition module is used for acquiring the infrared thermal imaging image of the positioning target collected by the infrared thermal imager and the visible light image collected by the visible light camera, and pre-processing the infrared thermal imaging image and the visible light image to obtain the pixel point matrix corresponding to the infrared thermal frame image and the visible light frame image, the infrared thermal frame image and the visible light frame image are obtained based on the same positioning target, two pixel point matrices are obtained respectively, and the two pixel point matrices correspond to each other; The infrared differential image acquisition module is used for carrying out pixel differential processing on two continuous infrared thermal frame images based on the pixel point matrix to obtain a differential image; The connected domain acquisition module obtains a binary image by binarizing the differential image, and eliminates the dark points of the pixel point matrix on the binary image to obtain a connected domain; The pixel point elimination module eliminates the pixel points in the connected domain which do not conform to the preset circularity shape and the preset area threshold; The target connected domain acquisition module obtains the area brightness gradient information in the visible light image corresponding to the visible light image connected domain, eliminates the area whose brightness gradient from the center to the periphery is positive, and correspondingly eliminates the pixel points of the pixel point matrix to obtain a target connected domain; The target shooting information acquisition module calculates the target shooting information based on the positional relationship between the target connected domain and the preset target surface corner feature mark point in the visible light camera; The binarization of the differential image to obtain a binary image includes: Respectively calculating the pixel mean values of the two continuous infrared thermal frame images corresponding to the differential image after Otsu binarization, obtaining the pixel mean value of the next infrared thermal frame image and the pixel mean value of the previous infrared thermal frame image; Calculating the difference between the pixel mean value of the next infrared thermal frame image and the pixel mean value of the previous infrared thermal frame image as a binarization threshold; Binarizing the differential image based on the binarization threshold to obtain a binary image; The threshold value of the Otsu method is calculated from the difference image, and the pixel points below the threshold value of the Otsu method are removed by the threshold value of the Otsu method, and then the average pixel value is calculated; By setting a dynamic threshold value for each frame of difference image, the system identifies the high temperature area in the image, i.e. the impact point, and the dynamic threshold value is determined by the brightness distribution of the image.

6. An automatic target reporting device based on the dual photoelastic point positioning technology, characterized in that, It comprises: a positioning target; a dual-optical-spot collector comprising an infrared thermal imager and a visible light camera, the infrared thermal imager being adapted to obtain an infrared thermal imaging image of the positioning target, and the visible light camera being adapted to obtain a visible light image of the positioning target, wherein the visible light camera is provided with a plurality of mark points based on the corner features of the target surface; a processor connected with the infrared thermal imager and the visible light camera respectively, for obtaining the shooting information obtained by the automatic scoring method based on the dual-optical-spot positioning technology according to any one of claims 1-4; a display connected with the processor.

7. The automatic scoring device based on the dual photoelastic point positioning technology according to claim 6, characterized in that, The display comprises a handheld display and a viewing display connected with the processor respectively, the handheld display is adapted to display the infrared thermal imaging image and the visible light image, and the viewing display is adapted to display the shooting information.

8. A computer device, comprising: It comprises: a memory and a processor, which are connected with each other in communication, the memory stores computer instructions, and the processor executes the computer instructions to perform the automatic scoring method based on the dual-optical-spot positioning technology according to any one of claims 1-4.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, which are used to make the computer execute the automatic scoring method based on the dual-optical-spot positioning technology according to any one of claims 1-4.

Images