A neural network-based intelligent auxiliary aiming method and system

Abstract

The application discloses an intelligent auxiliary aiming method and system based on a neural network, which comprises the following steps: performing Gaussian filtering processing, gray scale stretching and bilateral filtering separation on an infrared image to obtain a basic layer image; subtracting the basic layer image from the original image after noise reduction to obtain a high-frequency detail layer; performing gamma value greater than 1 gamma change to suppress background noise on the basic layer, and performing gamma value less than 1 change to enhance details on the detail layer, and linearly superimposing to obtain an infrared image with enhanced details; inputting a training set and its labeled information for training; transmitting infrared image data to a YOLOv5 neural network to detect and frame a human target, and obtaining the pixel width and the pixel height of the target; calculating the target detector field of view angle by using the field of view angle of the detector, and calculating the distance of the target distance detector by using the mathematical relationship among the target field of view angle, the target width and the target distance; calculating the bullet landing point information by using the bullet speed, the air resistance and the gravitational acceleration; and performing accurate local magnification processing on the aiming area by using a bilinear interpolation algorithm, and comprehensively displaying the real-time infrared image, the frame selection and distance information of the target detection, the predicted bullet landing point and the locally magnified image on the screen. The application can provide intelligent detection, landing point judgment and local zooming and other auxiliary aiming functions.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, in particular to an intelligent auxiliary aiming method and system based on a neural network. BACKGROUND

[0002] In traditional shooting, the observer needs to observe and record the change parameters of the external environment, such as wind direction, wind power, air temperature, air pressure, humidity, pitch angle and target information, and then adjusts the shooting correction table according to his own experience, accurately reports the shooting correction information to the shooter, and the shooter adjusts the aiming parameters according to the information. For example, the elevation adjustment: according to the target distance, the height of the aiming point is adjusted to compensate for the drop of the bullet in the flight process due to gravity; the windage adjustment: the left and right positions of the aiming point are adjusted to compensate for the influence of wind power on the flight trajectory of the bullet. In addition, there is a micrometer table engraved on the eyepiece, which can determine the approximate distance of the target. This process requires long-term training of the shooter.

[0003] With the development of science and technology, the aiming system also develops towards informationization and intelligence. Compared with traditional aiming, the intelligent auxiliary aiming system combines digital imaging and image processing, and replaces some processes of traditional aiming with electronic methods. The intelligent auxiliary aiming system is composed of a detector, an image acquisition module, an image processing module, a digital adjustment module and a display module. The image processing module pre-processes the target image, then detects the target through an algorithm to help the shooter lock, and the system also has auxiliary functions such as local magnification, which can be adjusted by the shooter according to the actual situation through the digital adjustment module. The combination of infrared and intelligent aiming is a method more suitable for assisting the shooter to aim. SUMMARY

[0004] The purpose of the present application is to provide an intelligent auxiliary aiming method and system based on a neural network.

[0005] The technical solution for achieving the purpose of the present application is as follows: an intelligent auxiliary aiming method based on a neural network, comprising the following steps:

[0006] Step 1: install an infrared detector on the gun body to ensure that the light path center derivation line of the detector is parallel to the barrel, and use the infrared detector to collect real-time scene information;

[0007] Step 2: perform Gaussian filtering on the collected infrared image to preliminarily suppress noise, then perform gray scale stretching on the noise-reduced image to increase its dynamic gray scale range, then perform bilateral filtering to separate the basic layer image, subtract the basic layer image from the original image after noise reduction to obtain the high-frequency detail layer; perform gamma value greater than 1 gamma change to suppress background noise on the basic layer, and perform gamma value less than 1 change to enhance details on the detail layer, and finally linearly superimpose the two layers of images to obtain the infrared image with enhanced details;

[0008] Step 3, select images that meet the application scenario from the KAIST dataset and image data collected by the infrared detector, process the selected images according to step 2 for noise suppression and detail enhancement, remove images with poor effects, and randomly divide the images that pass the quality assessment into two parts, one part as the training set and the other part as the validation set; input the training set and its labeled information into the YOLOv5 neural network for training, use the validation set to evaluate the model performance, and according to the performance feedback of the validation set, fine-tune the network structure and parameters to improve the recognition accuracy of the model for the person in the infrared image. After sufficient training, a neural network model capable of accurately identifying the person in the infrared image is obtained;

[0009] Step 4, after the infrared image data is preprocessed in step 2, it is transmitted to the neural network model, the human target is detected and framed, the pixel width and pixel height of the target are obtained, the target field of view angle is calculated using the field of view angle of the detector, and the target distance from the detector is calculated using the mathematical relationship between the target field of view angle, target width and target distance;

[0010] Step 5, using the bullet speed, air resistance and gravitational acceleration, combined with the target distance calculated in step 4, a mathematical formula for the bullet flight trajectory is established to calculate the bullet landing point information in real time;

[0011] Step 6, after locking the target, a bilinear interpolation algorithm is used to perform accurate local magnification processing on the aiming area, and the real-time infrared image, target detection frame, distance information, predicted bullet landing point and locally magnified image are displayed on the screen to assist aiming.

[0012] Further, in step 2, the collected infrared image is subjected to Gaussian filtering to preliminarily suppress noise, then the denoised image is subjected to gray scale stretching to increase its dynamic gray scale range, and then a bilateral filter is used to separate the base layer image, and the high-frequency detail layer is obtained by subtracting the base layer image from the original denoised image; the base layer is subjected to γ change with a γ value greater than 1 to suppress background noise, and the detail layer is subjected to γ change with a γ value less than 1 to enhance details, and finally the two layers of images are linearly superimposed to obtain an infrared image with enhanced details, wherein:

[0013] The Gaussian filter template formula is:

[0014]

[0015] Where the template coordinates are (x, y), g(x, y) represents the image after Gaussian filtering, σ 2 is the overall variance;

[0016] The bilateral filter uses a two-dimensional Gaussian function to generate a distance template and a one-dimensional Gaussian function to generate a value range template. The generation formula of the distance template coefficient is as follows:

[0017]

[0018] where (k, l) is the center coordinate of the template window; (i, j) is the coordinate of other coefficients of the template window; σ d is the standard deviation of the Gaussian function, and the generation formula of the template coefficient is as follows:

[0019]

[0020] where f(x, y) represents the pixel value of the image at point (x, y); σ τ is the standard deviation of the Gaussian function, and the formula of the template of the bilateral filter is as follows:

[0021]

[0022] The expression of γ change of the base layer and the detail layer images is as follows:

[0023]

[0024] Further, the step 4, the infrared detector real shot data is transmitted to the neural network model after the pre-processing in the step 2, the human body target is detected and framed, the pixel width and the pixel height of the target are obtained, the field of view angle of the target is calculated by using the field of view angle of the detector, the distance between the target and the detector is calculated by using the mathematical relationship between the field of view angle of the target, the width of the target and the distance of the target, wherein:

[0025] The calculation formula of the field of view angle of the detector is as follows:

[0026] θ (x / y) = 2arctan((L (x / y) / 2) / f)

[0027] where θ (x / y) is the horizontal or vertical field of view angle of the detector, L (x / y) is the horizontal or vertical frame size of the sensor of the detector, and f is the focal length of the detector;

[0028] The formula for converting the density of the detector by using the field of view angle of the detector is as follows:

[0029]

[0030] where MIL (x / y) is the horizontal or vertical field of view angle of the detector, and MOA is the angular unit;

[0031] The calculation formula of the distance of the target is as follows:

[0032]

[0033] where d is the distance between the target and the detector, and l(y) For the target height, θ (y) is the target vertical field of view angle on the imaging.

[0034] Further, step 5, using the bullet speed, air resistance, gravity acceleration, combined with the target distance calculated in step 4, the mathematical formula of the bullet flight trajectory is established, and the landing point information of the bullet is calculated in real time, and the specific method is:

[0035] The horizontal displacement of the bullet flying towards the target is:

[0036]

[0037] The calculation formula of the vertical offset of the bullet is:

[0038]

[0039] Where d is the target distance, C is the air resistance coefficient, p is the air density, A is the cross-sectional area of the bullet, v is the bullet speed, m is the mass of the bullet, v 0x ,v 0y The speed of the bullet in the horizontal and vertical directions, and θ is the angle between the bullet and the horizontal target;

[0040] At the same time, the motion in a small time interval is regarded as uniform acceleration, and the Runge-Kutta algorithm principle is used to solve θ and v in each time interval, and this process is continuously carried out until the horizontal displacement reaches d, the corresponding time t is obtained, and finally the vertical displacement d y of the bullet is obtained; then the initial θ is changed to obtain different angles and corresponding vertical displacements d y , d, θ and the corresponding vertical displacement d y are made into a lookup table, and in the actual calculation process, the vertical displacement of the bullet (the landing point of the bullet) can be quickly obtained according to the distance between the target calculated before d and θ. y , and it is converted into a pixel position and marked and displayed with a cross.

[0041] Further, step 6, after locking the target, a bilinear interpolation algorithm is used to accurately magnify the local area, and the real-time infrared image, the target detection frame selection and distance information, the predicted bullet landing point and the magnified image are displayed on the screen to assist aiming, wherein the bilinear interpolation formula is:

[0042] f1(x,y)≈(x2-x)(y2-y)f(x1,y1)+(x-x1)(y2-y)f(x2,y1)

[0043] +(x2-x)(y-y1)f(x1,y2)+(x-x1)(y-y1)f(x2,y2)

[0044] wherein f1(x,y) is the value of the pixel to be filled, (x1,y1), (x2,y2), (x3,y3) and (x4,y4) are the four known pixels around (x,y).

[0045] The intelligent auxiliary aiming system based on a neural network implements the intelligent auxiliary aiming method based on a neural network, and realizes intelligent auxiliary aiming based on a neural network, and comprises:

[0046] Module 1: The collected infrared image is subjected to Gaussian filtering processing to preliminarily suppress noise, and then the image after noise reduction is subjected to gray scale stretching to increase the dynamic gray scale range, and then subjected to bilateral filtering to separate a basic layer image, and the high-frequency detail layer is obtained by subtracting the basic layer image from the original image after noise reduction; the background noise is suppressed by changing the γ value of the basic layer to be greater than 1, and the details are enhanced by changing the γ value of the detail layer to be less than 1, and finally the two layers of images are linearly superimposed to obtain an infrared image with enhanced details;

[0047] Module 2: images conforming to the application scene are selected from the KAIST dataset and infrared detector real shooting data, the selected images are subjected to noise suppression and detail enhancement processing, the images with poor effect are rejected, the images with qualified quality are randomly divided into two parts, one part is used as a training set, and the other part is used as a verification set; the training set and its labeled information are input into the YOLOv5 neural network for training, the model performance is evaluated using the verification set, the network structure and parameters are fine-tuned according to the performance feedback of the verification set, so as to improve the recognition accuracy of the model for the person in the infrared image, and finally after sufficient training, a neural network model capable of accurately recognizing the person in the infrared image is obtained;

[0048] Module 3: the infrared image data is transmitted to the neural network model after preprocessing, the human target is detected and framed, the pixel width and pixel height of the target are obtained, the field of view angle of the target is calculated using the field of view angle of the detector, and the distance of the target from the detector is calculated using the mathematical relationship between the field of view angle of the target, the width of the target and the distance of the target.

[0049] Module 4: the bullet flight trajectory mathematical formula is established by using the bullet speed, air resistance and gravitational acceleration, combined with the calculated target distance, and the landing point information of the bullet is calculated in real time;

[0050] Module 5: after locking the target, the bilinear interpolation algorithm is used to perform accurate local magnification processing on the aiming area, and the real-time infrared image, the frame selection and distance information of the target detection, the predicted bullet landing point and the locally magnified image are comprehensively displayed on the screen to assist aiming.

[0051] Compared with the prior art, the present application has the following advantages: (1) the system function is highly perfect, combining infrared imaging, neural network image processing technology and ballistic trajectory estimation technology, and can intelligently detect, judge the drop point and perform various auxiliary aiming functions such as local zooming; (2) the algorithm is fully hardware-based, improving real-time performance, meeting actual requirements and being easy to operate; and (3) the algorithm has good adaptability and is adjustable, easy to correct and suitable for various shooting instruments. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 is a schematic diagram of the neural network-based intelligent auxiliary aiming system of the present application.

[0053] Figure 2 is a flowchart of the neural network-based intelligent auxiliary aiming system of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0055] The neural network-based intelligent auxiliary aiming system of the present application acquires target images in real time through an infrared detector, transmits the target images to an FPGA platform through a Camera Link protocol, performs filtering and noise reduction on the target images, performs frequency domain detail enhancement processing, transmits the target images to a GPU platform through a UDP protocol, detects and frames a human target using a trained yolov5 neural network model, calculates the target distance using the mil method, judges the bullet drop point, integrates local zooming and other functions, and synchronously outputs the functions to a display device for auxiliary aiming. The specific steps include the following steps:

[0056] Step 1, install the infrared detector on the gun body to ensure that the optical path center of the detector is strictly parallel to the barrel, so that the deviation between the two can be detected and then corrected and repaired; use the infrared detector to acquire actual scene information in real time.

[0057] Step 2, perform Gaussian filtering on the acquired infrared image to preliminarily suppress noise, perform gray scale stretching on the noise-reduced image to increase its dynamic gray scale range, then perform bilateral filtering to separate the base layer image, obtain the high-frequency detail layer by subtracting the base layer image from the original image after noise reduction, suppress background noise by changing the gamma value of the base layer to be greater than 1, enhance the details by changing the gamma value of the detail layer to be less than 1, and finally linearly superimpose the two layers to obtain the detail-enhanced infrared image. The Gaussian filtering template formula is:

[0058]

[0059] where the template coordinates are (x, y), g(x, y) represents the image after Gaussian filtering, and σ 2 is the total variance. The patent adds Gaussian noise with a mean of 0 and a variance of 0.02 to the image.

[0060] The bilateral filter uses a two-dimensional Gaussian function to generate a distance template and a one-dimensional Gaussian function to generate a value domain template. The generation formula of the distance template coefficient is as follows:

[0061]

[0062] where (k, l) is the center coordinate of the template window; (i, j) is the coordinate of other coefficients of the template window; and σd is the standard deviation of the Gaussian function. The generation formula of the value domain template coefficient is as follows:

[0063]

[0064] where f(x, y) represents the pixel value of the image at point (x, y); and σ τ is the standard deviation of the Gaussian function. Multiplying the above two templates obtains the template of the bilateral filter, as shown in the following formula:

[0065]

[0066] The expression of γ change for the base layer and the detail layer images is as follows:

[0067]

[0068] It is defined as: for an input image f(x, y) with M bit gray scale range [0, 2M-1], an output image g(x, y) with N bit gray scale range [0, 2N-1] is wanted to be mapped.

[0069] Step 3, select images that meet the application scene from the KAIST dataset and infrared detector real shooting data, perform noise suppression and detail enhancement processing on the selected images according to step 2, eliminate images with poor effect, and randomly divide the images that meet the quality evaluation into two parts, one part as the training set and the other part as the validation set; input the training set and its labeled information into the YOLOv5 neural network for training, use the validation set to evaluate the model performance, according to the performance feedback of the validation set, fine-tune the network structure and parameters, to improve the recognition accuracy of the model for the characters in the infrared image, and finally after sufficient training, a neural network model capable of accurately recognizing characters in infrared images is obtained.

[0070] Step 4, the infrared detector real data after step 2 pretreatment is transmitted to the neural network model, detects and frames the human target, obtains the pixel width and pixel height of the target, calculates the target field of view angle by using the field of view angle of the detector, and calculates the distance between the target and the detector by using the mathematical relationship between the target field of view angle, the target width (height) and the target distance. The formula for calculating the field of view angle of the detector is:

[0071] θ (x / y) = 2arctan((L (x / y) / 2) / f)

[0072] Wherein θ (x / y) is the horizontal or vertical field of view angle of the detector, L (x / y) is the horizontal or vertical frame size of the detector sensor, and f is the focal length of the detector.

[0073] The formula for converting the detector density by the field of view angle of the detector is:

[0074]

[0075] Wherein MIL (x / y) is the horizontal or vertical field of view angle of the detector, and MOA is the angular unit.

[0076] The formula for calculating the target distance is:

[0077]

[0078] Wherein d is the target distance, l (y) is the target height, generally taking the experience value 1.7, and θ(y) is the vertical field of view angle of the target in imaging.

[0079] Step 5, by using the physical parameters such as bullet speed, air resistance, acceleration of gravity, and combining the target distance calculated in step 4, the mathematical formula of the bullet flight trajectory is established, and the landing point information of the bullet is calculated in real time. The calculation process of the bullet landing point is:

[0080] The horizontal displacement of the bullet flying towards the target is:

[0081]

[0082] The calculation formula of the vertical offset of the bullet is:

[0083]

[0084] Wherein d is the target distance, C is the air resistance coefficient, ρ is the air density, A is the cross-sectional area of the bullet, v is the bullet speed, m is the mass of the bullet, v 0x ,v 0yVx and Vy are the horizontal and vertical components of the velocity of the bullet, θ is the angle between the bullet and the horizontal target.

[0085] At the same time, the movement in a small time interval is regarded as uniform acceleration, and the Runge-Kutta algorithm is used to solve θ and v in each time interval, and this process is continuously carried out until the horizontal displacement reaches d, the corresponding time t is obtained, and finally the vertical displacement d of the bullet is obtained y ; then the initial θ is changed to obtain different angles and corresponding vertical displacements d y , d, θ and corresponding vertical displacement d y are made into a lookup table, and in the actual calculation process, the vertical displacement (the landing point of the bullet) of the bullet can be obtained according to the distance d and θ between the target and the detector calculated before y , which is converted into a pixel position and marked and displayed with a cross.

[0086] Step 6, after locking the target, a bilinear interpolation algorithm is used to perform accurate local magnification processing on the aiming area. In addition, the system also displays the real-time infrared image, the frame selection of target detection and distance information, the predicted landing point of the bullet and the locally magnified image on the screen, which assists in aiming. The formula of the bilinear interpolation local magnification algorithm is:

[0087] f1(x,y)≈(x2-x)(y2-y)f(x1,y1)+(x-x1)(y2-y)f(x2,y1)

[0088] +(x2-x)(y-y1)f(x1,y2)+(x-x1)(y-y1)f(x2,y2)

[0089] Where f1(x,y) is the value of the pixel to be filled, (x1,y1), (x2,y2), (x3,y3) and (x4,y4) are the four known pixels around (x,y).

[0090] Embodiment

[0091] In order to verify the effectiveness of the scheme of the application, the following experiments are carried out.

[0092] In the intelligent auxiliary aiming system based on neural network, the pre-processing of the acquired image is mainly Gaussian filtering and detail enhancement based on bilateral filtering. The image acquired by the infrared imaging system often produces spatial noise due to the complexity of the scene or the problem of the system itself, and needs to be pre-processed to eliminate noise as much as possible. The system uses a Gaussian filter to perform spatial filtering processing on the original image.

[0093] The formula of the Gaussian filter template is:

[0094]

[0095] where (x,y) is the template coordinate, g(x,y) represents the image after Gaussian filtering, σ 2 is the total variance. In this embodiment, Gaussian noise with mean value of 0 and variance of 0.02 is added to the image.

[0096] The infrared image can be generally divided into a low-frequency part and a high-frequency part in the spatial domain. The background and the target with large area, which exhibit the overall visual effect of the image, have strong spatial correlation and the change of pixel gray value is very small and gentle, showing low-frequency characteristics. The edge of the object, the small target and the texture information of the local target in the image, which are isolated and distributed, have very low spatial correlation and the change of gray value is often abrupt, showing high-frequency characteristics. In addition, the details with high local contrast can also be classified into the low-frequency part because they can be clearly perceived. In this regard, the infrared image is divided into two parts, i.e. the high-frequency part and the low-frequency part, so that the two parts are analyzed and processed according to the different frequency characteristics. The image is subjected to gray stretching to increase the dynamic gray range, and then subjected to bilateral filtering to separate the base layer image. The high-frequency detail layer is obtained by subtracting the base layer image from the original image after noise reduction. The base layer is subjected to γ change to suppress the background noise with a γ value greater than 1, and the detail layer is subjected to γ change to enhance the details with a γ value less than 1. Finally, the two images are linearly superimposed to obtain the infrared image with enhanced details.

[0097] The bilateral filter uses a two-dimensional Gaussian function to generate a distance template and uses a one-dimensional Gaussian function to generate a value domain template. The generation formula of the distance template coefficient is as follows:

[0098]

[0099] where (k,l) is the center coordinate of the template window; (i,j) is the coordinate of other coefficients of the template window; σ d is the standard deviation of the Gaussian function. The generation formula of the value domain template coefficient is as follows:

[0100]

[0101] where f(x,y) represents the pixel value of the image at point (x,y); σ τ is the standard deviation of the Gaussian function. The above two templates are multiplied to obtain the template of the bilateral filter, as shown in the following formula:

[0102]

[0103] The expression of γ change for the base layer and the detail layer images is as follows:

[0104]

[0105] It is defined as: for the input image f(x, y) of M bit gray scale range [0, 2M-1], the output image g(x, y) of N bit gray scale range [0, 2N-1] is wanted to be mapped.

[0106] The images conforming to the application scene are selected from the KAIST dataset and the infrared detector real shot data, the selected images are subjected to noise suppression and detail enhancement processing according to step 2, the images with poor effect are rejected, the images qualified in quality assessment are randomly divided into two parts, one part is used as a training set, and the other part is used as a verification set; the training set and its labeled information are input into the YOLOv5 neural network for training, the model performance is evaluated using the verification set, the network structure and parameters are fine-tuned according to the performance feedback of the verification set, so as to improve the recognition accuracy of the model for the person in the infrared image, and finally after sufficient training, a neural network model capable of accurately recognizing the person in the infrared image is obtained.

[0107] The infrared detector real shot data is transmitted to the neural network model after preprocessing, the human target is detected and framed, and the distance of the target from the detector is calculated, the specific method is as follows: first, the field angle of the detector is calculated through the sensor frame and the focal length, then the total pitch of the detector imaging is calculated, the pixel width of the detected target is combined with the conversion relationship between the pitch and the pixel width and the mathematical relationship between the pitch and the target distance, so as to calculate the target distance.

[0108] Considering the air resistance, the bullet flight time, the initial speed of the bullet and the horizontal displacement of the bullet, a mathematical model is established to calculate the bullet flight time, then a mathematical model of the vertical offset of the bullet and the flight time is established, and finally the physical actual displacement is converted into the display pitch, so as to display the bullet landing point.

[0109] After locking the target, the bilinear interpolation algorithm is used to accurately magnify the local area of the aiming area. In addition, the system also displays the real-time infrared image, the frame selection and distance information of the target detection, the predicted bullet landing point and the magnified image on the screen, which assists in aiming.

[0110] The technical features of the above embodiments can be combined in any way. In order to make the description simple, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0111] The above-described embodiments are merely illustrative of several embodiments of the present application, which are described in more detail and in a specific manner, but should not be construed as limiting the scope of the present application. It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A neural network-based intelligent assisted aiming method, characterized in that, The steps are as follows: Step 1, install the infrared detector on the gun body, ensure that the light path center derivative line of the detector is parallel to the barrel, and use the infrared detector to collect real-time scene information; Step 2, the collected infrared image is processed by Gaussian filter to preliminarily suppress noise, then the denoised image is subjected to gray scale stretching to increase its dynamic gray scale range, then bilateral filter is used to separate the basic layer image, the high frequency detail layer is obtained by subtracting the basic layer image from the denoised original image; the background noise is suppressed by changing the gamma value of the basic layer to be greater than 1, and the details are enhanced by changing the gamma value of the detail layer to be less than 1, and finally the two layers of images are linearly superimposed to obtain the infrared image with enhanced details; Step 3, select images that meet the application scene from the KAIST dataset and the collected infrared image data, process the selected images according to step 2 to suppress noise and enhance details, eliminate images with poor effect, and randomly divide the images with qualified quality into two parts, one part as the training set and the other part as the validation set; input the training set and its labeled information into the YOLOv5 neural network for training, use the validation set to evaluate the model performance, and according to the performance feedback of the validation set, fine tune the network structure and parameters to improve the recognition accuracy of the model for the characters in the infrared image, and finally after sufficient training, a neural network model capable of accurately recognizing characters in the infrared image is obtained; Step 4, the infrared image data is preprocessed by step 2 and transmitted to the neural network model, the human target is detected and framed, the pixel width and pixel height of the target are obtained, the target detector field of view angle is calculated by using the field of view angle of the detector, and the target distance from the detector is calculated by using the mathematical relationship between the target field of view angle, the target width and the target distance; Step 5, using the bullet speed, wind resistance and gravity acceleration, combined with the target distance from the detector calculated in step 4, a mathematical formula of the bullet flight trajectory is established, and the landing point information of the bullet is calculated in real time; Step 6, after locking the target, the bilinear interpolation algorithm is used to accurately magnify the local area of the aiming area, and the real-time infrared image, the frame selection and distance information of the target detection, the predicted bullet landing point and the magnified image are displayed on the screen to assist aiming.

2. The neural network-based intelligent assisted targeting method according to claim 1, characterized in that, Step 2, the collected infrared image is processed by Gaussian filter to preliminarily suppress noise, then the denoised image is subjected to gray scale stretching to increase its dynamic gray scale range, then bilateral filter is used to separate the basic layer image, the high frequency detail layer is obtained by subtracting the basic layer image from the denoised original image; the background noise is suppressed by changing the gamma value of the basic layer to be greater than 1, and the details are enhanced by changing the gamma value of the detail layer to be less than 1, and finally the two layers of images are linearly superimposed to obtain the infrared image with enhanced details; The Gaussian filter template formula is: where the template coordinates are (x, y), g(x, y) represents the image after Gaussian filtering, σ 2 is the total variance; The bilateral filter uses a two-dimensional Gaussian function to generate a distance template, and a one-dimensional Gaussian function to generate a value domain template, the generation formula of the distance template coefficient is as follows: Where (k, l) is the center coordinate of the template window; (i, j) is the coordinate of other coefficients of the template window; σ is the standard deviation of the Gaussian function, the generation formula of the value domain template coefficient is as follows: where f(x, y) represents the pixel value of the image at point (x, y); σ τ is the standard deviation of the Gaussian function, and the template formula of the bilateral filter is as follows: The expression of gamma change for the basic layer and detail layer images is as follows: It is defined as: for M bit gray scale range [0, 2M-1] of input image f(x, y), it wants to map to N bit gray scale range [0, 2N-1] of output image m(x, y).

3. The neural network-based intelligent assisted targeting method according to claim 1, characterized in that, Step 4, the infrared detector real shot data after step 2 pretreatment is transmitted to the neural network model, the human body target is detected and framed, the pixel width and pixel height of the target are obtained, the field angle of the target is calculated by using the field angle of the detector, the distance of the target is calculated by using the mathematical relationship between the field angle of the target, the width of the target and the distance of the target, wherein: The formula for calculating the field angle of the detector is: θ (x / y) = 2arctan((L (x / y) / 2) / f) where θ (x / y) is the detector lateral or longitudinal field of view in radian units, L (x / y) is the detector sensor lateral or longitudinal format size, and f is the detector focal length. The formula for calculating the density of the detector by the field angle of the detector is: where MIL (x / y) is the lateral or longitudinal field of view of the detector in mils, and MOA is the angle in minutes of arc. The formula for calculating the distance of the target is: where d is the target-to-detector distance, l (y) is the target height, and θ(y) is the vertical field of view angle occupied by the target in the image.

4. The neural network-based intelligent assisted targeting method according to claim 1, characterized in that, Step 5, the bullet flight orbit mathematical formula is established by using the bullet speed, wind resistance, gravity acceleration, and combining the target distance calculated in step 4, the bullet landing point information is calculated in real time, and the specific method is: The horizontal displacement of the bullet flying to the target is: The formula for calculating the vertical offset of the bullet is: where d is the target distance, C is the air resistance coefficient, p is the air density, A is the bullet cross-sectional area, v is the bullet velocity, m is the bullet mass, where v is the velocity of the bullet, and 9 is the angle between the bullet and the horizontal target. At the same time, the motion in small time interval is regarded as uniform acceleration, and the θ and v in each time interval are solved by Runge-Kutta algorithm, and this process is repeated until the horizontal displacement reaches d, and the corresponding time t is obtained, and finally the vertical displacement d of the bullet is obtained y Then the initial θ is changed to obtain the vertical displacement d corresponding to different angles y The d, θ and the corresponding vertical displacement d y are made into a lookup table, and in the actual calculation process, the vertical displacement d of the bullet can be obtained according to the distance d and θ between the target and the detector y It is converted into pixel position and marked and displayed with a cross.

5. The neural network-based intelligent assisted targeting method according to claim 1, wherein, Step 6, after locking the target, the bilinear interpolation algorithm is used to accurately magnify the local aiming area, the real-time infrared image, the frame selection and distance information of the target detection, the predicted bullet landing point and the magnified image are displayed on the screen, which assists aiming, wherein the bilinear interpolation formula is: f1(x,y)≈(x2-x)(y2-y)f(x1,y1)+(x-x1)(y2-y)f(x2,y1)+(x2-x)(y-y1)f(x1,y2)+(x-x1)(y-y1)f(x2,y2) Wherein f1(x, y) is the value of the pixel to be filled, (x1, y1), (x2, y2), (x3, y3) and (x4, y4) are four known pixels around (x, y).

6. A neural network based intelligent assisted targeting system characterized in that, The neural network based intelligent auxiliary aiming method according to any one of claims 1-5 is implemented, and the neural network based intelligent auxiliary aiming is realized, comprising: Module one, the collected infrared image is subjected to Gaussian filter processing to preliminarily suppress noise, then the image after noise reduction is subjected to gray scale stretching to increase its dynamic gray scale range, then the bilateral filter is used to separate the basic layer image, the high frequency detail layer is obtained by subtracting the basic layer image from the original image after noise reduction; the background noise is suppressed by changing the gamma value of the basic layer to be greater than 1, the details are enhanced by changing the gamma value of the detail layer to be less than 1, and finally the two layers of images are linearly superimposed to obtain the infrared image with enhanced details; Module two, select the image of the application scene from the KAIST dataset and the real data of the infrared detector, process the selected image for noise suppression and detail enhancement, remove the images with poor effect, and randomly divide the images with qualified quality into two parts, one part as the training set and the other part as the verification set; input the training set and its labeled information into the YOLOv5 neural network for training, use the verification set to evaluate the performance of the model, according to the performance feedback of the verification set, fine-tune the network structure and parameters, to improve the recognition accuracy of the model for the characters in the infrared image, finally after sufficient training, a neural network model capable of accurately recognizing characters in the infrared image is obtained; Module three, the infrared image data detector transmits the preprocessed data to the neural network model, detects and frames the human target, obtains the pixel width and pixel height of the target, calculates the target field of view angle using the field of view angle of the detector, and calculates the distance between the target and the detector using the mathematical relationship between the target field of view angle, the target width and the target distance; Module four, using the bullet speed, air resistance and gravitational acceleration, combined with the calculated target distance, a mathematical formula of the bullet flight trajectory is established, and the bullet landing point information is calculated in real time; Module five, after locking the target, a bilinear interpolation algorithm is used to accurately magnify the local area of the aiming area, and the real-time infrared image, the frame selection and distance information of the target detection, the predicted bullet landing point and the magnified image are displayed on the screen to assist aiming.

Images