Automatic target scoring system and method based on double photoelastic point positioning technology

Abstract

The invention relates to the technical field of automatic target scoring, in particular to an automatic target scoring system and method based on a double-photoelastic-point positioning technology. The method comprises the following steps: acquiring an infrared thermal imaging image of a positioning target acquired by an infrared thermal imager and a visible light image acquired by a visible light camera, and preprocessing the infrared thermal imaging image and the visible light image to obtain pixel point matrixes corresponding to an infrared thermal frame image and a visible light frame image; based on the pixel matrix, performing pixel difference processing on two continuous infrared thermal frame images to obtain a difference image; and binarizing the difference image to obtain a binarized image, and eliminating dark spots of the pixel matrix on the binarized image to obtain a connected domain. According to the method, suspicious bullet points can be identified and analyzed by using thermal imaging characteristics and sequential continuous frames, so that the precedence order of different bullet points can be distinguished, and overlapped bullet points can be detected.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic target marking, and in particular to an automatic target marking system and method based on dual photo-bullet point positioning technology. Background Art

[0002] As an important part of basic military training, light weapons live-fire shooting is crucial to improving soldiers' combat skills and reaction capabilities. With the continuous advancement of intelligent technology, the application of automatic target reporting systems has gradually become popular. They have not only improved the safety and efficiency of shooting training, but also greatly improved the accuracy and scientific nature of training.

[0003] Video target is a new automatic target reporting technology, which uses high-definition cameras to capture the target surface image and uses image processing technology to determine the impact point. However, traditional video targets only use visible light cameras to collect images of the target surface, and the identification of bullet points does not have a time sequence characteristic. In the case of continuous shooting, it is impossible to distinguish the location of newly added bullet points, especially when the bullet points overlap, it is impossible to detect the bullet points that overlap later. Summary of the invention

[0004] In view of this, the present invention provides an automatic target reporting system and method based on dual-light bullet point positioning technology to solve the problem that the recognition of bullet points by traditional video targets does not have a timing characteristic.

[0005] In a first aspect, the present invention provides an automatic target reporting method based on dual photo-bomb point positioning technology, comprising the following steps:

[0006] S1, obtaining an infrared thermal imaging image of a positioning target collected by an infrared thermal imager and a visible light image collected by a visible light camera, and preprocessing the infrared thermal imaging image and the visible light image to obtain a pixel matrix corresponding to the infrared thermal frame image and the visible light frame image;

[0007] S2, based on the pixel matrix, performing pixel difference processing on two consecutive infrared thermal frame images to obtain a differential image;

[0008] S3, binarizing the difference image to obtain a binarized image, and removing dark points of the pixel matrix on the binarized image to obtain a connected domain;

[0009] S4, removing the pixel points in the connected domain that do not conform to the preset roundness shape and the preset area threshold;

[0010] S5. Based on the regional brightness gradient information in the visible light image corresponding to the connected domain, obtain the region with a positive brightness gradient from the center to the surrounding area, and remove the pixel points corresponding to the pixel matrix to obtain the target connected domain;

[0011] S6. Calculate the shooting performance information based on the positional relationship between the target connected domain and the target surface corner feature marker points preset in the visible light camera.

[0012] In an optional implementation, binarizing the difference image to obtain a binarized image includes:

[0013] Respectively calculating the pixel mean of two consecutive infrared thermal frame images corresponding to the differential image after binarization according to the Otsu method, to obtain the pixel mean of the infrared thermal frame image at the next moment and the pixel mean of the infrared thermal frame image at the previous moment;

[0014] Calculate the difference between the pixel mean of the infrared thermal frame image at the next moment and the pixel mean of the infrared thermal frame image at the previous moment as the binarization threshold;

[0015] The differential image is binarized based on the binarization threshold to obtain a binarized image.

[0016] In an optional implementation, it also includes:

[0017] S7, obtaining the infrared thermal frame image in step S2 and the infrared thermal frame image as an interval frame therewith and performing pixel difference;

[0018] S8, repeating steps S3 to S5 to obtain the target connected domain;

[0019] S9, comparing the target connected domain obtained in step S5 with the target connected domain obtained in step S8, retaining the connected domain at the same position, and calculating the shooting score based on the positional relationship between the retained connected domain at the same position and the visible light camera marker point.

[0020] In an optional implementation, the method further includes: automatically calibrating the external parameter matrices of the infrared thermal imager and the visible light camera, including:

[0021] Obtain the landmark points of the target surface in the visible light image through corner point detection;

[0022] Obtain an infrared thermal imaging image of a position on the target surface corresponding to the corner point detection mark point, perform differential processing on the infrared thermal imaging image after the artificial hot spot, and obtain the infrared mark point;

[0023] Match corner detection marker points and infrared marker points;

[0024] The conversion matrix between the pixel coordinates of the corner detection marker points and the infrared marker points is calculated and marked as the external parameter conversion matrix of the infrared thermal imager and the visible light camera;

[0025] Obtain target surface marker points in the visible light image through corner point detection, and match them with the marker points in the visible light camera;

[0026] The conversion matrix between the pixel coordinates of the target surface marking points and the pixel coordinates of the visible light camera marking points is calculated, and the calculated conversion matrix is ​​superimposed on the extrinsic parameter conversion matrix of the infrared thermal imager and the visible light camera for automatic calibration of the extrinsic parameter conversion matrix of the infrared thermal imager and the visible light camera.

[0027] In an optional implementation, it also includes:

[0028] Perform corner point detection on two consecutive visible light frame images respectively to obtain tracking corner points;

[0029] Perform forward optical flow tracking on the tracking corner points in two consecutive visible light frame images to obtain optical flow tracking point pairs;

[0030] Perform reverse optical flow tracking on the obtained optical flow tracking point pairs to obtain reverse tracking points, and eliminate the optical flow tracking point pairs whose errors between the tracking corner points and the corresponding reverse tracking points are greater than a preset error;

[0031] Calculate the sum of pixel errors of all pairs of optical flow tracking points. If the sum of pixel errors is greater than a preset scene threshold, it is determined that the scene has changed. If the sum of pixel errors is less than or equal to the preset scene threshold, it is determined that the scene has not changed. The preset scene threshold is the average sum of the errors of optical flow tracking points of several consecutive visible light frame images.

[0032] In a second aspect, the present invention provides an automatic target reporting system based on dual photo-bomb point positioning technology, comprising:

[0033] An image acquisition module is used to acquire an infrared thermal imaging image of the positioning target acquired by the infrared thermal imager and a visible light image acquired by the visible light camera, and pre-process the infrared thermal imaging image and the visible light image to obtain a pixel matrix corresponding to the infrared thermal frame image and the visible light frame image;

[0034] The infrared differential image acquisition module is used to perform pixel differential processing on two consecutive infrared thermal frame images based on a pixel matrix to obtain a differential image;

[0035] A connected domain acquisition module is used to binarize the difference image to obtain a binary image, and remove dark points of the pixel matrix on the binary image to obtain a connected domain;

[0036] A pixel removal module removes pixels in the connected domain that do not conform to a preset roundness shape and a preset area threshold;

[0037] The target connected domain acquisition module obtains the regional brightness gradient information in the visible light image corresponding to the connected domain of the visible light image, and removes the area with positive brightness gradient from the center to the surrounding area, and correspondingly removes the pixel points corresponding to the pixel matrix to obtain the target connected domain;

[0038] The shooting result information acquisition module calculates the shooting result information based on the positional relationship between the target connected domain and the preset target surface corner feature marker points in the visible light camera.

[0039] In a third aspect, the present invention provides an automatic target reporting device based on dual light bullet point positioning technology, comprising:

[0040] Positioning target;

[0041] A dual-photoelastic point collector, comprising an infrared thermal imager and a visible light camera, wherein the infrared thermal imager is suitable for obtaining an infrared thermal imaging image of a positioning target, and the visible light camera is suitable for obtaining a visible light image of the positioning target, wherein the visible light camera is provided with a number of marker points based on the corner point features of the target surface;

[0042] A processor, connected to the infrared thermal imager and the visible light camera respectively, for executing the target shooting information obtained by the automatic target reporting method based on the dual light bullet point positioning technology;

[0043] A display is connected to the processor.

[0044] In an optional embodiment, the display includes a handheld display and a viewing display respectively connected to the processor, the handheld display is suitable for displaying infrared thermal imaging images and visible light images, and the viewing display is suitable for displaying target shooting information.

[0045] In a fourth aspect, the present invention provides a computer device, comprising:

[0046] A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the above-mentioned automatic target reporting method based on the dual light bullet point positioning technology by executing the computer instructions.

[0047] In a fifth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions are used to enable a computer to execute the above-mentioned automatic target reporting method based on the dual photo-bullet point positioning technology.

[0048] The present invention first obtains an infrared thermal imaging image of a positioning target collected by an infrared thermal imager and a visible light image collected by a visible light camera, and pre-processes the infrared thermal imaging image and the visible light image to obtain a pixel matrix corresponding to the infrared thermal frame image and the visible light frame image; then, based on the pixel matrix, pixel difference processing is performed on two consecutive infrared thermal frame images to obtain a differential image; then, the differential image is binarized to obtain a binary image, and dark points of the pixel matrix are eliminated on the binary image to obtain a connected domain; then, pixel points in the connected domain that do not conform to a preset roundness shape and a preset area threshold are eliminated; then, based on regional brightness gradient information in the visible light image corresponding to the connected domain, an area with a positive brightness gradient from the center to the surrounding area is obtained, and pixel points corresponding to the pixel matrix are eliminated accordingly to obtain a target connected domain; finally, based on the positional relationship between the target connected domain and a target surface corner point feature marker point preset in the visible light camera, the shooting score information is calculated.

[0049] The present invention can use thermal imaging characteristics and sequential frames to identify and analyze suspicious bullet points, not only distinguishing the order of different bullet points but also detecting overlapping bullet points. Through dual-light fusion technology, bullet points can be accurately detected, and high positioning accuracy can be maintained even in complex environments.

[0050] Traditional automatic target marking systems may rely on a single type of sensor, but the present invention can make up for the limitations of a single sensor by combining infrared and visible light images, thereby achieving more accurate bullet point detection and positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 This is a schematic diagram of the process of the automatic target reporting method according to an embodiment of the present invention;

[0053] Figure 2 This is a structural block diagram of an automatic target reporting device according to an embodiment of the present invention;

[0054] Figure 3 This is a structural block diagram of the automatic target reporting system according to an embodiment of the present invention;

[0055] Figure 4 A schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention;

[0056] Figure 5The embodiment of the present invention is a matching diagram of a visible light feature corner point and a thermal imaging hot spot;

[0057] Figure 6 Schematic diagram of optical flow tracking point pairs according to an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0059] As an important part of basic military training, light weapons live-fire shooting is crucial to improving soldiers' combat skills and reaction capabilities. With the continuous advancement of intelligent technology, the application of automatic target reporting systems has gradually become popular. They have not only improved the safety and efficiency of shooting training, but also greatly improved the accuracy and scientific nature of training.

[0060] At present, the automatic target reporting system is mainly divided into traditional photoelectric targets, acoustic and electric targets, double-layer electrode short-circuit targets and video targets. Traditional photoelectric targets use photoelectric sensors to detect the impact point, which has the advantages of fast response speed and high target reporting accuracy, but its cost is high and maintenance is complicated. Acoustic and electric targets use sound and electrical signals to determine the impact point, which is low in cost and high in accuracy, but requires professional operation. Double-layer electrode short-circuit targets use the electrode grid on the target surface to detect the position where the projectile hits, which has high accuracy, but high cost and short service life.

[0061] Video target is an emerging automatic target reporting technology. It uses a high-definition camera to capture the target surface image and uses image processing technology to determine the impact point. This method is low-cost, highly practical, has great development potential, is suitable for complex outdoor environments, and can adapt to changes in natural lighting conditions. The video target system performs shooting judgment and bullet hole positioning by real-time analysis of the collected video sequence, which improves the automation of target reporting and can be applied to complex outdoor environments under natural lighting conditions. However, traditional video targets only use visible light cameras to collect images of the target surface, which is greatly affected by the site lighting, selected algorithms and threshold parameters. When the scene changes, the selected algorithm and parameters may not meet the actual usage accuracy, and then need to be readjusted and tested. In addition, the recognition of bullet points by traditional video targets does not have a time sequence characteristic. In the case of continuous shooting, it is impossible to distinguish the location of newly added bullet points, especially when the bullet points overlap, it is impossible to detect the bullet points that overlap later.

[0062] Combine the following Figures 1 to 6, describing an embodiment of the present invention.

[0063] Example 1

[0064] The present invention provides an automatic target reporting method based on dual light bullet point positioning technology, such as Figure 1 As shown, the following steps are included:

[0065] S1. Obtain the infrared thermal imaging image of the positioning target collected by the infrared thermal imager and the visible light image collected by the visible light camera, and preprocess the infrared thermal imaging image and the visible light image to obtain the pixel matrix corresponding to the infrared thermal frame image and the visible light frame image; wherein the preprocessing includes denoising, aligning and registering the infrared thermal imaging image and the visible light image, wherein the collected image may contain noise due to environmental factors or external interference. A filtering algorithm (such as Gaussian filtering, mean filtering, etc.) can be used to remove image noise to improve the accuracy of subsequent analysis. There may be a difference in viewing angle between the infrared thermal imaging image and the visible light image, so the two images need to be registered (image alignment) to ensure that they are aligned in space. The image alignment problem can be solved by feature point matching or global optimization algorithm. Preprocessing also includes processing the infrared thermal imaging image and the visible light image into video frames to obtain infrared thermal frame images and visible light frame images. It should be noted that the infrared thermal frame image and the visible light frame image are obtained based on the same positioning target, and two pixel matrixes are obtained respectively, and the two pixel matrixes can correspond one to one, so a pixel matrix that can characterize the pixel points of the infrared thermal imaging image and the visible light image can be obtained.

[0066] S2. Based on the pixel matrix, two consecutive infrared thermal frame images are subjected to pixel difference processing to obtain a difference image, which can be recorded as the infrared thermal frame images of the kth frame and the k+1th frame. For infrared thermal imaging images, the heat change is detected by the difference between consecutive frames. For example, when a bullet hits the target surface, the target surface area will produce an instantaneous heat change. By differentiating adjacent frame images, the areas with sudden heat changes can be captured, which may be the impact points.

[0067] S3, binarizing the difference image to obtain a binarized image, and removing dark points of the pixel matrix on the binarized image to obtain a connected domain;

[0068] S4, removing pixels in the connected domain that do not conform to the preset roundness shape and the preset area threshold; wherein the shape of the connected domain may be checked for roundness, specifically by fitting the connected domain for roundness, removing pixels that do not conform to the preset roundness shape, and removing the strip-shaped connected domain. Connected domains with a small area (about 5*5 pixels) or a large area (about 100*100 pixels) may also be excluded.

[0069] S5. Based on the regional brightness gradient information in the visible light image corresponding to the connected domain, the area with positive brightness gradient from the center to the surrounding area is obtained, and the pixel points corresponding to the pixel matrix are correspondingly eliminated to obtain the target connected domain; the real bullet points present a single-peak distribution similar to the normal distribution, that is, the pixel temperature gradually decreases from the center of the connected domain to the surrounding areas, that is, the gradient is negative, and the connected domain with positive gradient is eliminated.

[0070] By calculating the brightness gradient of each connected domain, those with abnormal gradients (such as areas with rising gradients) are eliminated. Specifically, the real impact point should be a heat source, whose temperature shows a gradually decreasing trend around the center point, so its brightness gradient should be negative.

[0071] S6. Calculate the shooting performance information based on the positional relationship between the target connected domain and the target surface corner feature marker points preset in the visible light camera.

[0072] In thermal imaging images, the heat change of bullet points will decay over time. By performing differential processing on consecutive frame images, new and old bullet points can be distinguished.

[0073] The present invention performs differential calculation on two consecutive frames of thermal imaging images to determine whether the detected bullet point area appears in the previous frame of image. If a certain area has appeared in the previous frame and the temperature value has weakened, it can be determined that the area is an old bullet point.

[0074] The temperature of a new bullet point is high, and as time goes by, the temperature gradually decays. By comparing the temperature information in the current frame of the thermal imaging image with the temperature information in the previous frames, it can be determined whether a bullet point is a new one or an old bullet point whose temperature has decayed.

[0075] In an optional implementation, binarizing the difference image to obtain a binarized image includes:

[0076] Calculate the pixel mean of two consecutive infrared thermal frame images corresponding to the differential image after binarization according to the Otsu method respectively, and obtain the pixel mean of the infrared thermal frame image at the latter moment and the pixel mean of the infrared thermal frame image at the previous moment; it should be noted that the present invention may not directly perform binarization according to the Otsu method, but calculate the threshold of the Otsu method from the differential image, and then use the threshold of the Otsu method to eliminate pixels whose pixels are lower than the threshold of the Otsu method, and then calculate their average pixel value.

[0077] Calculate the difference between the pixel mean of the infrared thermal frame image at the next moment and the pixel mean of the infrared thermal frame image at the previous moment as the binarization threshold;

[0078] The differential image is binarized based on the binarization threshold to obtain a binarized image.

[0079] Among them, by setting a dynamic threshold for each frame of the differential image, the system can identify the high-temperature area in the image (i.e., the impact point). Specifically, the dynamic threshold is determined by the brightness distribution of the image (for example, using the "Otsu method" for automatic threshold segmentation), ensuring that the impact point can be stably identified under different ambient lighting conditions.

[0080] The formula used is: Td = HM(k+1)-HM(k), HM(k) is the mean value of the highlight area in the k-th frame image, that is, the pixel value mean HighMean of the bright area after the image is binarized, and it is also the pixel mean after binarization according to the Otsu method. Td is the differential threshold, and Td is the dynamic threshold. After binarization, the pixels with increased temperature can be reflected, and the main factor for the increase in pixel temperature is the heat generated by the friction between the bullet and the positioning target when it passes through the positioning target. The pixels with increased temperature, that is, the pixel difference is greater than the binarization threshold, are marked as 255, and the other pixels are marked as 0. Therefore, the area formed by the pixels with increased temperature is the suspicious bullet point.

[0081] In an optional implementation, it also includes:

[0082] S7, obtaining the infrared thermal frame image in step S2 and the infrared thermal frame image as an interval frame therewith and performing pixel difference;

[0083] S8, repeating steps S3 to S5 to obtain the target connected domain;

[0084] S9, compare the target connected domain obtained in step S5 with the target connected domain obtained in step S8, retain the connected domain at the same position, and calculate the shooting score based on the positional relationship between the retained connected domain at the same position and the visible light camera marker point. In order to prevent single-frame recognition from false detection due to noise or target surface jitter, the k frame and the k+2 frame are differentiated again and steps S3 to S5 are repeated. The two obtained target connected domains are compared and verified, and only those that pass the verification are finally output as bullet points. It should be noted that the interval frame can be represented as an infrared thermal frame image of the k frame and the k+2 frame, or as an infrared thermal frame image of the k frame and the k+3 frame, etc. The specific number of interval frames is not limited to this in this embodiment.

[0085] By calculating the difference between consecutive frames, we can ensure that the same impact point can be confirmed at multiple time points. To avoid misjudgment due to factors such as lens shake and lighting changes, the identified connected domain is verified in multiple frames. If an area remains consistent in consecutive frames, it can be determined that the area is the impact point.

[0086] In an optional implementation, the method further includes: automatically calibrating the external parameter matrices of the infrared thermal imager and the visible light camera, including:

[0087] Obtain the landmark points of the target surface in the visible light image through corner point detection;

[0088] Obtain an infrared thermal imaging image of a position on the target surface corresponding to the corner point detection mark point, perform differential processing on the infrared thermal imaging image after the artificial hot spot, and obtain the infrared mark point;

[0089] Match corner detection marker points and infrared marker points;

[0090] The conversion matrix between the pixel coordinates of the corner detection marker points (specifically, the pixel coordinates of the corner detection marker points in the visible light image) and the infrared marker points (specifically, the pixel coordinates of the infrared marker points in the infrared thermal imaging image) is calculated and marked as the external parameter conversion matrix of the infrared thermal imager and the visible light camera. It can be considered as a one-to-one correspondence process between the pixel matrix of the infrared thermal imaging image and the pixel matrix of the visible light image.

[0091] It should be noted that the external parameter conversion matrix from the target surface to the infrared thermal imager includes two parts: the conversion matrix between the target surface and the visible light plane and the conversion matrix between the visible light plane and the thermal imaging plane. The external parameter conversion matrix from the target surface to the infrared thermal imager can be obtained from the conversion matrix between the target surface and the visible light plane and the conversion matrix between the visible light plane and the thermal imaging plane.

[0092] The conversion parameter between the thermal imaging plane and the visible light plane is called the bi-optical camera extrinsic parameter T cR , bullet mark pixel points ρ identified by infrared thermal imager R Through the dual-light camera external reference T cR Convert to the visible light image plane and obtain the corresponding pixel point ρ c .

[0093] The visible light camera extracts the target's landmarks by detecting corner points. The infrared thermal imager needs to manually mark the target's landmarks with hot spots, and then perform differential processing on the images before and after the marking, so that eight pairs of matching points can be formed with the visible light camera, such as Figure 5 As shown in the figure, it is the matching diagram of the visible light feature corner point and the thermal imaging hot spot. According to the spatial coordinate transformation relationship and singular value decomposition, the external parameter T of the bi-optical camera can be solved. cR :

[0094]

[0095] For the conversion matrix between visible light and the target surface, the automatic correction method can be used to calibrate the external parameter matrix in real time to ensure the stability of accuracy, as shown below.

[0096] Obtain target surface marker points in the visible light image through corner point detection, and match them with the marker points in the visible light camera;

[0097] The conversion matrix between the pixel coordinates of the target surface marking points (specifically, the pixel coordinates of the target surface marking points in the visible light image) and the visible light camera marking points (whose specific positions are factory settings) is calculated, and the calculated conversion matrix is ​​superimposed on the extrinsic parameter conversion matrix of the infrared thermal imager and the visible light camera for automatic calibration of the extrinsic parameter conversion matrix of the infrared thermal imager and the visible light camera, which can be considered as automatic calibration of the extrinsic parameter conversion matrix from the target surface to the infrared thermal imager.

[0098] Before leaving the factory, the infrared thermal imager and the visible light camera can be calibrated with external parameters to obtain the external parameters between the two lights, that is, the conversion matrix between the visible light plane and the thermal imaging plane, which is also the external parameter conversion matrix between the infrared thermal imager and the visible light camera. After factory installation, the relative posture of the infrared thermal imager and the visible light camera does not change, that is, the conversion matrix between the visible light plane and the thermal imaging plane remains unchanged. Therefore, when performing automatic correction, it is only necessary to recalibrate the conversion matrix between the target surface and the visible light plane, and superimpose the conversion matrix between the target surface mark point and the visible light camera mark point on the conversion matrix between the visible light plane and the thermal imaging plane.

[0099] In an optional implementation, it also includes:

[0100] Perform corner point detection on two consecutive visible light frame images respectively to obtain tracking corner points;

[0101] Perform forward optical flow tracking on the tracking corner points in two consecutive visible light frame images to obtain optical flow tracking point pairs; forward optical flow tracking is based on the assumption that the movement of pixels in the image is smooth within a small range in consecutive frames. By calculating the motion vector (optical flow) of each pixel in the image, the movement of the object or target can be tracked, such as extracting feature points from the current image frame, which are usually corner points or edge points (for example, through Harris corner detection or Shi-Tomasi algorithm).

[0102] Perform reverse optical flow tracking on the obtained optical flow tracking point pairs to obtain reverse tracking points, and eliminate the optical flow tracking point pairs whose error between the tracking corner point and the corresponding reverse tracking point is greater than the preset error; reverse optical flow tracking is to trace back in the previous frame image through the feature points in the current image to verify whether the feature points are correctly matched. Through reverse optical flow tracking, try to map the feature points in the current image to the corresponding positions in the previous frame. That is, given a point in the current image, use reverse optical flow to calculate its position in the previous frame and check whether it is in the feature point set of the previous frame.

[0103] Calculate the sum of pixel errors of all pairs of optical flow tracking points. If the sum of pixel errors is greater than a preset scene threshold, it is determined that the scene has changed. If the sum of pixel errors is less than or equal to the preset scene threshold, it is determined that the scene has not changed. The preset scene threshold is the average sum of the errors of optical flow tracking points of several consecutive visible light frame images.

[0104] Considering that during operation, the lens may be accidentally touched, resulting in false alarms or missed bullet points on the imaging surface, a real-time self-check mechanism can be used to detect whether the scene in the lens has changed, thereby avoiding false alarms and missed alarms.

[0105] The scene change self-check algorithm relies on the original scene image stored in the previous automatic calibration to compare pixels with the current scene image to determine whether the scene has changed and then decide to re-calibrate automatically. The comparison method can use LK optical flow tracking. First, Harris corner point extraction is performed on the original scene image to detect the edge of the scene in the image to obtain key corner points, and then optical flow tracking is used to match the feature points in the two frames.

[0106] To prevent mismatching, the tracking points in the current scene image are used to perform reverse optical flow tracking to obtain reverse tracking points in the original scene. The errors between the corner points and the reverse tracking points are compared, and the optical flow tracking point pairs with large errors are removed from the judgment queue.

[0107] A pair of optical flow tracking points contains the corner point H in the original scene image. r , and the tracking point H of the current scene image c , the sum of the pixel L2 errors between these two points is used to determine whether the scene has changed. The sum of the pixel L2 errors is:

[0108]

[0109] If the sum of the errors is greater than the preset scene threshold, it is considered that the scene has changed; if the sum of the errors is less than or equal to the preset scene threshold, it is considered that the scene has changed.

[0110] Among them, u c and v c is the tracking point H of the current scene image c The pixel coordinates, u r and v r is the corner point H in the original scene image r The pixel coordinates of .

[0111] In order to increase the reliability of the judgment, the value of the threshold thr is calculated by the average error between consecutive frames of the scene image. The set of five consecutive frames of images F = {F1, F2, F3, F4, F5} calculates the sum of the optical flow point errors L2 between each consecutive frame. The average value of the sum is the value of the threshold thr, as shown in Figure 6 shown.

[0112]

[0113] Wherein, thr is the preset scene threshold.

[0114] The present invention can perform four parts: automatic parameter calibration, bullet point position calculation, abnormal situation determination and shooting performance evaluation.

[0115] The present invention takes into account the stability of accuracy as well as the cost and simplicity of debugging and maintenance, and can also perform automatic parameter calibration. After completing a calibration of the infrared-visible light external parameter matrix and lens distortion parameters at the factory, it enters the real-time automatic calibration stage to monitor the conversion matrix offset between the visible light camera plane and the target surface in real time, thereby ensuring the confidence level of accuracy during use and facilitating debugging when used again.

[0116] In order to avoid false alarms and missed alarms, abnormal situation judgment can also be performed. Through time difference verification and optical flow tracking change judgment, the elimination of differential abnormal values ​​in infrared thermal imaging can be ensured, and the occurrence of abnormal situations can be identified and corrected in time.

[0117] The present invention obtains reliable and accurate bullet point coordinate records after the steps of correction, positioning, and abnormality elimination, and can be combined with a shooting performance evaluation module to estimate the circular probability error of the bullet point, thereby intuitively evaluating the shooting level of the trainees.

[0118] Example 2

[0119] The present invention provides an automatic target reporting system based on dual light bullet point positioning technology, such as Figure 3 As shown, including:

[0120] The image acquisition module 201 is used to acquire the infrared thermal imaging image of the positioning target acquired by the infrared thermal imager and the visible light image acquired by the visible light camera, and pre-process the infrared thermal imaging image and the visible light image to obtain the pixel matrix corresponding to the infrared thermal frame image and the visible light frame image;

[0121] The infrared differential image acquisition module 202 is used to perform pixel differential processing on two consecutive infrared thermal frame images based on a pixel matrix to obtain a differential image;

[0122] The connected domain acquisition module 203 binarizes the difference image to obtain a binarized image, and removes dark points of the pixel matrix on the binarized image to obtain a connected domain;

[0123] The pixel removal module 204 removes the pixels in the connected domain that do not conform to the preset roundness shape and the preset area threshold;

[0124] The target connected domain acquisition module 205 acquires the regional brightness gradient information in the visible light image corresponding to the connected domain of the visible light image, removes the region with a positive brightness gradient from the center to the surroundings, and correspondingly removes the pixel points corresponding to the pixel matrix to obtain the target connected domain;

[0125] The shooting result information acquisition module 206 calculates the shooting result information based on the positional relationship between the target connected domain and the preset target surface corner feature marker points in the visible light camera.

[0126] Example 3

[0127] The present invention provides an automatic target reporting device based on dual light bullet point positioning technology, such as Figure 2 As shown, including:

[0128] Positioning target;

[0129] A dual-photoelastic point collector, comprising an infrared thermal imager and a visible light camera, wherein the infrared thermal imager is suitable for obtaining an infrared thermal imaging image of a positioning target, and the visible light camera is suitable for obtaining a visible light image of the positioning target, wherein the visible light camera is provided with a number of marker points based on the corner point features of the target surface;

[0130] A processor, connected to the infrared thermal imager and the visible light camera respectively, for executing the target shooting information obtained by the automatic target reporting method based on the dual light bullet point positioning technology, and the processor may be an analysis host;

[0131] A display is connected to the processor.

[0132] In an optional embodiment, the display includes a handheld display and a viewing display respectively connected to the processor, the handheld display is suitable for displaying infrared thermal imaging images and visible light images, and the viewing display is suitable for displaying target shooting information.

[0133] The handheld display can be a tablet device, which is in the same wireless LAN as the dual optical bullet point collector and analysis host. The original thermal imaging image and visible light camera image video stream are pushed to the handheld display for online display of the actual shooting situation and results.

[0134] The observation display receives the coordinates of the target bullet points and the scores output by the analysis host, and displays the target bullet point positions, the number of bullets and the total score of the target in real time.

[0135] The present invention first obtains an infrared thermal imaging image of a positioning target collected by an infrared thermal imager and a visible light image collected by a visible light camera, and pre-processes the infrared thermal imaging image and the visible light image to obtain a pixel matrix corresponding to the infrared thermal frame image and the visible light frame image; then, based on the pixel matrix, pixel difference processing is performed on two consecutive infrared thermal frame images to obtain a differential image; then, the differential image is binarized to obtain a binary image, and dark points of the pixel matrix are eliminated on the binary image to obtain a connected domain; then, pixel points in the connected domain that do not conform to a preset roundness shape and a preset area threshold are eliminated; then, based on regional brightness gradient information in the visible light image corresponding to the connected domain, an area with a positive brightness gradient from the center to the surrounding area is obtained, and pixel points corresponding to the pixel matrix are eliminated accordingly to obtain a target connected domain; finally, based on the positional relationship between the target connected domain and a target surface corner point feature marker point preset in the visible light camera, the shooting score information is calculated.

[0136] The present invention can use thermal imaging characteristics and sequential frames to identify and analyze suspicious bullet points, not only distinguishing the order of different bullet points but also detecting overlapping bullet points. Through dual-light fusion technology, bullet points can be accurately detected, and high positioning accuracy can be maintained even in complex environments.

[0137] Traditional automatic target marking systems may rely on a single type of sensor, but the present invention can make up for the limitations of a single sensor by combining infrared and visible light images, thereby achieving more accurate bullet point detection and positioning.

[0138] The present invention can detect and locate the bullet points of live ammunition in real time, and assist in analyzing the trajectory of bullet points, and has the characteristics of high precision, high robustness, low maintenance requirements, operating boundaries, strong environmental adaptability, etc. Compared with the high maintenance cost and long debugging time of traditional photoelectric targets, the main acquisition equipment of the present invention is a dual-light camera. Compared with the photoelectric transceiver sensor, the power loss and system complexity are low, so the use loss and maintenance cost are low, and the dual-light target has the function of automatic calibration and self-test when powered on, and basically no debugging is required. Compared with the acoustic and electric targets affected by the basic principles, any interference including the shape, angle, and motion state of the bullet will cause deformation disturbance to the sound wave, which will lead to a decrease in accuracy. The present invention is mainly based on thermal imaging images and visible light images for analysis, and can stably maintain high-precision positioning without being affected by these factors. Compared with the disposable target such as the double-layer electrode short-circuit target, which has a high cost of use and low accuracy, the present invention can be used sustainably and has a much higher accuracy than this target. Compared with traditional video targets that only use visible light cameras for identification and cannot distinguish between new and old bullet points and overlapping bullet points, the present invention uses thermal imaging characteristics and time-series continuous frames to identify and track suspicious bullet points. It can not only distinguish the order of different bullet points but also detect overlapping bullet points, and on this basis analyze the bullet point trajectory to discover problems with the user's shooting. The overall system capability has reached the international leading level.

[0139] The embodiment of the present invention also provides a computer device having the above Figure 3 The automatic target reporting system based on dual photo-bullet point positioning technology is shown.

[0140] See also Figure 4 , Figure 4 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 4 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.

[0141] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0142] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0143] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0144] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0145] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0146] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor central control system or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0147] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0148] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. An automatic target reporting method based on dual photo-bomb point positioning technology, characterized in that: The following steps are involved: S1, obtaining an infrared thermal imaging image of a positioning target collected by an infrared thermal imager and a visible light image collected by a visible light camera, and preprocessing the infrared thermal imaging image and the visible light image to obtain a pixel matrix corresponding to the infrared thermal frame image and the visible light frame image; S2, based on the pixel matrix, performing pixel difference processing on two consecutive infrared thermal frame images to obtain a differential image; S3, binarizing the difference image to obtain a binarized image, and removing dark points of the pixel matrix on the binarized image to obtain a connected domain; S4, removing the pixel points in the connected domain that do not conform to the preset roundness shape and the preset area threshold; S5. Based on the regional brightness gradient information in the visible light image corresponding to the connected domain, obtain the region with a positive brightness gradient from the center to the surrounding area, and remove the pixel points corresponding to the pixel matrix to obtain the target connected domain; S6. Calculate the shooting performance information based on the positional relationship between the target connected domain and the target surface corner feature marker points preset in the visible light camera.

2. The automatic target reporting method based on dual photoelastic point positioning technology according to claim 1 is characterized in that: The step of binarizing the difference image to obtain a binarized image includes: Respectively calculating the pixel mean of two consecutive infrared thermal frame images corresponding to the differential image after binarization according to the Otsu method, to obtain the pixel mean of the infrared thermal frame image at the next moment and the pixel mean of the infrared thermal frame image at the previous moment; Calculate the difference between the pixel mean of the infrared thermal frame image at the next moment and the pixel mean of the infrared thermal frame image at the previous moment as the binarization threshold; The differential image is binarized based on the binarization threshold to obtain a binarized image.

3. The automatic target reporting method based on dual photoelastic point positioning technology according to claim 1 is characterized in that: Also includes: S7, obtaining the infrared thermal frame image in step S2 and the infrared thermal frame image as an interval frame therewith and performing pixel difference; S8, repeating steps S3 to S5 to obtain the target connected domain; S9, comparing the target connected domain obtained in step S5 with the target connected domain obtained in step S8, retaining the connected domain at the same position, and calculating the shooting score based on the positional relationship between the retained connected domain at the same position and the visible light camera marker point.

4. The automatic target reporting method based on dual photoelastic point positioning technology according to claim 1 is characterized in that: Also includes: Automatic calibration of the extrinsic matrix of infrared thermal imagers and visible light cameras, including: Obtain the landmark points of the target surface in the visible light image through corner point detection; Obtain an infrared thermal imaging image of a position on the target surface corresponding to the corner point detection mark point, perform differential processing on the infrared thermal imaging image after the artificial hot spot, and obtain the infrared mark point; Match corner detection marker points and infrared marker points; The conversion matrix between the pixel coordinates of the corner detection marker points and the infrared marker points is calculated and marked as the external parameter conversion matrix of the infrared thermal imager and the visible light camera; Obtain target surface marker points in the visible light image through corner point detection, and match them with the marker points in the visible light camera; The conversion matrix between the pixel coordinates of the target surface marking points and the pixel coordinates of the visible light camera marking points is calculated, and the calculated conversion matrix is ​​superimposed on the extrinsic parameter conversion matrix of the infrared thermal imager and the visible light camera for automatic calibration of the extrinsic parameter conversion matrix of the infrared thermal imager and the visible light camera.

5. The automatic target reporting method based on dual photoelastic point positioning technology according to claim 4 is characterized in that: Also includes: Perform corner point detection on two consecutive visible light frame images respectively to obtain tracking corner points; Perform forward optical flow tracking on the tracking corner points in two consecutive visible light frame images to obtain optical flow tracking point pairs; Perform reverse optical flow tracking on the obtained optical flow tracking point pairs to obtain reverse tracking points, and eliminate the optical flow tracking point pairs whose errors between the tracking corner points and the corresponding reverse tracking points are greater than a preset error; Calculate the sum of pixel errors of all optical flow tracking point pairs. If the sum of pixel errors is greater than the preset scene threshold, it is determined that the scene has changed. If the sum of pixel errors is less than or equal to a preset scene threshold, it is determined that the scene has not changed, wherein the preset scene threshold is an average value of the sum of optical flow tracking point errors of a number of consecutive visible light frame images.

6. An automatic target reporting system based on dual light bullet point positioning technology, characterized in that: include: An image acquisition module is used to acquire an infrared thermal imaging image of the positioning target acquired by the infrared thermal imager and a visible light image acquired by the visible light camera, and pre-process the infrared thermal imaging image and the visible light image to obtain a pixel matrix corresponding to the infrared thermal frame image and the visible light frame image; The infrared differential image acquisition module is used to perform pixel differential processing on two consecutive infrared thermal frame images based on a pixel matrix to obtain a differential image; A connected domain acquisition module is used to binarize the difference image to obtain a binary image, and remove dark points of the pixel matrix on the binary image to obtain a connected domain; A pixel removal module removes pixels in the connected domain that do not conform to a preset roundness shape and a preset area threshold; The target connected domain acquisition module obtains the regional brightness gradient information in the visible light image corresponding to the connected domain of the visible light image, and removes the area with positive brightness gradient from the center to the surrounding area, and correspondingly removes the pixel points corresponding to the pixel matrix to obtain the target connected domain; The shooting result information acquisition module calculates the shooting result information based on the positional relationship between the target connected domain and the preset target surface corner feature marker points in the visible light camera.

7. An automatic target reporting device based on dual light bullet point positioning technology, characterized in that: include: Positioning target; A dual-photoelastic point collector, comprising an infrared thermal imager and a visible light camera, wherein the infrared thermal imager is suitable for obtaining an infrared thermal imaging image of a positioning target, and the visible light camera is suitable for obtaining a visible light image of the positioning target, wherein the visible light camera is provided with a number of marker points based on the corner point features of the target surface; A processor, connected to the infrared thermal imager and the visible light camera, respectively, for executing the target shooting information obtained by the automatic target reporting method based on the dual photo-bullet point positioning technology according to any one of claims 1 to 5; A display is connected to the processor.

8. The automatic target reporting device based on dual photo-bullet point positioning technology according to claim 7 is characterized in that: The display comprises a handheld display and a viewing display respectively connected to the processor, the handheld display is suitable for displaying infrared thermal imaging images and visible light images, and the viewing display is suitable for displaying target shooting information.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the automatic target reporting method based on the dual photo-bullet point positioning technology as described in any one of claims 1 to 5 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the automatic target reporting method based on the dual photoelastic point positioning technology according to any one of claims 1 to 5.

Images