Intelligent auxiliary aiming method and system based on neural network

Through the intelligent assisted aiming method based on neural network, the infrared detector and YOLOv5 neural network are used to identify the target and calculate the trajectory of the bullet, which solves the complexity of aiming parameter adjustment and the insufficient function of the intelligent assisted aiming system in traditional shooting, and achieves high efficiency , real-time aiming assistance function.

CN119027499AActive Publication Date: 2024-11-26NANJING UNIV OF SCI & TECH
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202410778834.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-17
Publication Date
2024-11-26
Estimated Expiration
2044-06-17

AI Technical Summary

Technical Problem

In traditional shooting, the observation hand needs long training to adjust the aiming parameters to cope with changes in the external environment, and the intelligent assisted aiming system has limited functions, making it difficult to achieve efficient and real-time aiming assistance.

Method used

Using an intelligent assisted aiming method based on neural network, images are collected in real time through infrared detectors, Gaussian filtering and bilateral filtering are performed, combined with YOLOv5 neural network to identify targets, calculate bullet flight trajectory, and local amplification is used to achieve accurate aim.

Benefits of technology

It has achieved a high degree of improvement in system functions, combined with infrared imaging, neural network image processing and ballistic trajectory estimation technology, and has intelligent detection, landing point judgment and local scaling functions, which improves real-time and operation convenience and strong adaptability. Suitable for all kinds of shooting equipment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027499A_ABST
    Figure CN119027499A_ABST
Patent Text Reader

Abstract

The invention discloses an intelligent auxiliary aiming method and system based on a neural network, and the method comprises the steps: carrying out the Gaussian filtering processing, gray stretching and bilateral filtering separation of an infrared image, obtaining a basic layer image, and subtracting the basic layer image from an original image after noise reduction to obtain a high-frequency detail layer; performing gamma change with a gamma value greater than 1 on the base layer to suppress background noise, performing change with a gamma value less than 1 on the detail layer to enhance details, and performing linear superposition to obtain an infrared image with enhanced details; inputting the training set and the annotation information of the training set for training; infrared image data are transmitted to a YOLOv5 neural network, a human-shaped target is detected and selected in a frame mode, and the pixel width and the pixel height of the target are obtained; calculating a target detector field angle by using the detector field angle, and calculating the distance from the target to the detector by using the mathematical relationship among the target field angle, the target width and the target distance; calculating the falling point information of the bullet by using the bullet speed, the wind resistance and the gravitational acceleration; a bilinear interpolation algorithm is adopted to carry out accurate local amplification processing on an aiming area, and a real-time infrared image, frame selection and distance information of target detection, a predicted bullet drop point and a locally amplified image are comprehensively displayed on a screen. According to the invention, various auxiliary aiming functions such as intelligent detection, drop point judgment and local scaling can be provided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an intelligent auxiliary aiming method and system based on a neural network. Background Art

[0002] In traditional shooting, the observer needs to observe and record the changing parameters of the external environment, such as wind direction, wind force, temperature, air pressure, humidity, pitch angle and target information, and then adjust the shooting correction table based on his own experience, accurately report the shooting correction information to the shooter, and the shooter adjusts the aiming parameters based on the information. For example, elevation adjustment: adjust the height of the aiming point according to the target distance to compensate for the falling of the bullet caused by gravity during flight; windage adjustment: adjust the left and right position of the aiming point to compensate for the influence of wind force on the bullet's flight trajectory. In addition, there is a density table engraved on the eyepiece, which can determine the approximate distance of the target. This process requires long-term training for the shooter.

[0003] With the development of science and technology, the aiming system is also developing towards the direction of informatization and intelligence. Compared with traditional aiming, the intelligent assisted aiming system combines digital imaging and image processing, and replaces some processes of traditional aiming with electronic methods. The intelligent assisted aiming system consists of a detector, an image acquisition module, an image processing module, a digital adjustment module and a display module. The image processing module pre-processes the target image, and then detects the target through an algorithm to help the shooter lock it. The system also has auxiliary functions such as local magnification, and the shooter can adjust it according to the actual situation through the digital adjustment module. Combining infrared with intelligent aiming is a method of assisting shooters to aim that is more suitable for actual needs. Summary of the invention

[0004] The purpose of the present invention is to provide an intelligent auxiliary aiming method and system based on neural network.

[0005] The technical solution to achieve the purpose of the present invention is: an intelligent auxiliary aiming method based on a neural network, the steps are as follows:

[0006] Step 1: Install the infrared detector on the gun body, ensure that the optical path center derivative line of the detector is parallel to the gun barrel, and use the infrared detector to collect actual scene information in real time;

[0007] Step 2: Perform Gaussian filtering on the collected infrared image to initially suppress noise, then perform grayscale stretching on the denoised image to increase its dynamic grayscale range, and then perform bilateral filtering to separate the base layer image. Subtract the base layer image from the denoised original image to obtain the high-frequency detail layer. Perform a γ change with a γ value greater than 1 on the base layer to suppress background noise, and perform a γ change with a γ value less than 1 on the detail layer to enhance details. Finally, the two layers of images are linearly superimposed to obtain a detail-enhanced infrared image.

[0008] Step 3: Take images that meet the application scenario from the KAIST dataset and the image data collected by the infrared detector, perform noise suppression and detail enhancement processing on the selected images according to step 2, eliminate images with poor effects, and randomly divide the images that pass the quality assessment into two parts, one as a training set and the other as a verification set; input the training set and its annotation information into the YOLOv5 neural network for training, use the verification set to evaluate the model performance, and fine-tune the network structure and parameters based on the performance feedback of the verification set to improve the model's recognition accuracy for people in infrared images. Finally, after sufficient training, a neural network model that can accurately recognize people in infrared images is obtained;

[0009] Step 4: After the infrared image data is pre-processed in step 2, it is transmitted to the neural network model to detect and select the human-shaped target, obtain the pixel width and pixel height of the target, calculate the target field angle using the detector field angle, and calculate the distance of the target from the detector using the mathematical relationship between the target field angle, target width and target distance;

[0010] Step 5: Using the bullet speed, wind resistance and gravity acceleration, combined with the target distance calculated in step 4, a mathematical formula for the bullet flight trajectory is established to calculate the bullet landing point information in real time;

[0011] Step 6: After locking the target, a bilinear interpolation algorithm is used to accurately enlarge the aiming area. The real-time infrared image, the target detection frame and distance information, the predicted bullet landing point and the locally enlarged image are displayed on the screen to assist aiming.

[0012] Furthermore, in step 2, the collected infrared image is subjected to Gaussian filtering to initially suppress noise, and then the grayscale of the denoised image is stretched to increase its dynamic grayscale range, and then bilateral filtering is performed to separate the base layer image, and the high-frequency detail layer is obtained by subtracting the base layer image from the denoised original image; the base layer is subjected to a γ change with a γ value greater than 1 to suppress background noise, and the detail layer is subjected to a γ change with a γ value less than 1 to enhance details, and finally the two layers of images are linearly superimposed to obtain a detail-enhanced infrared image, wherein:

[0013] The Gaussian filter template formula is:

[0014]

[0015] The template coordinates are (x, y), g(x, y) represents the image after Gaussian filtering, σ 2 is the overall variance;

[0016] The bilateral filter uses a two-dimensional Gaussian function to generate a distance template and a one-dimensional Gaussian function to generate a range template. The generation formula of the distance template coefficient is as follows:

[0017]

[0018] Among them, (k, l) is the center coordinate of the template window; (i, j) is the coordinate of other coefficients of the template window; σ d is the standard deviation of the Gaussian function, and the generation formula of the range template coefficient is as follows:

[0019]

[0020] Among them, f(x,y) represents the pixel value of the image at point (x,y); σ τ is the standard deviation of the Gaussian function, and the template formula of the bilateral filter is as follows:

[0021]

[0022] The expression for changing the gamma of the base layer and detail layer images is:

[0023]

[0024] Furthermore, in step 4, the infrared detector real shot data is transmitted to the neural network model after preprocessing in step 2, and the human-shaped target is detected and selected to obtain the pixel width and pixel height of the target. The target field angle is calculated using the detector field angle, and the distance of the target from the detector is calculated using the mathematical relationship between the target field angle, the target width and the target distance, where:

[0025] The detector field of view angle calculation formula is:

[0026] θ (x / y) =2arctan((L (x / y) / 2) / f)

[0027] where θ (x / y) is the horizontal or vertical field of view of the detector, L (x / y) is the horizontal or vertical frame size of the detector sensor, and f is the focal length of the detector;

[0028] The formula for converting the detector density by the detector field angle is:

[0029]

[0030] Among them, MIL (x / y) It is the horizontal or vertical field of view of the detector, and MOA is the unit of angle.

[0031] The target distance is calculated as:

[0032]

[0033] Where d is the distance between the target and the detector, l(y) is the target height, and θ(y) is the vertical field angle of the target in the imaging.

[0034] Furthermore, in step 5, the bullet speed, wind resistance, and gravity acceleration are used in combination with the target distance calculated in step 4 to establish a mathematical formula for the bullet flight trajectory and calculate the bullet landing point information in real time. The specific method is as follows:

[0035] The horizontal displacement of the bullet flying towards the target is:

[0036]

[0037] The calculation formula for the vertical deflection of the bullet is:

[0038]

[0039] Where d is the target distance, C is the air resistance coefficient, ρ is the air density, A is the cross-sectional area of ​​the bullet, v is the bullet speed, m is the mass of the bullet, and v 0x ,v 0y are the components of the bullet’s velocity in the horizontal and vertical directions, and θ is the angle between the ballistic bullet and the horizontal target;

[0040] At the same time, the movement in the small time interval is regarded as uniform acceleration motion, and the Runge-Kutta algorithm is used to solve the θ and v of each time interval. This process is repeated until the horizontal displacement reaches d, and the corresponding time t is obtained. Finally, the vertical displacement d of the bullet is obtained. y ; Then change the initial θ to get different angles and corresponding vertical displacement d y , replace d, θ and the corresponding vertical displacement d y A lookup table is made. In the actual calculation process, d and θ can be quickly obtained according to the previously calculated distance to the target to obtain the vertical displacement of the bullet (the point where the bullet falls) d y , convert it into pixel position and mark it with a cross.

[0041] Furthermore, in step 6, after locking the target, a bilinear interpolation algorithm is used to accurately enlarge the aiming area locally, and the real-time infrared image, the frame selection and distance information of the target detection, the predicted bullet landing point and the locally enlarged image are comprehensively displayed on the screen to assist aiming, wherein the bilinear interpolation formula is:

[0042] f1(x,y)≈(x2-x)(y2-y)f(x1,y1)+(x-x1)(y2-y)f(x2,y1)

[0043] +(x2-x)(y-y1)f(x1,y2)+(x-x1)(y-y1)f(x2,y2)

[0044] Where f1(x,y) is the value of the pixel to be filled, (x1,y1), (x2,y2), (x3,y3) and (x4,y4) are the four known pixels around (x,y).

[0045] A neural network-based intelligent assisted aiming system implements the neural network-based intelligent assisted aiming method to achieve neural network-based intelligent assisted aiming, comprising:

[0046] Module 1: Gaussian filtering is performed on the collected infrared image to initially suppress noise, and then the grayscale of the denoised image is stretched to increase its dynamic grayscale range. Then, bilateral filtering is performed to separate the base layer image, and the high-frequency detail layer is obtained by subtracting the base layer image from the denoised original image. The base layer is subjected to a γ change with a γ value greater than 1 to suppress background noise, and the detail layer is subjected to a γ change with a γ value less than 1 to enhance details. Finally, the two layers of images are linearly superimposed to obtain a detail-enhanced infrared image.

[0047] Module 2: Select images that meet the application scenario from the KAIST dataset and the infrared detector real-shot data, perform noise suppression and detail enhancement on the selected images, remove images with poor results, and randomly divide the images that pass the quality assessment into two parts, one as a training set and the other as a validation set; input the training set and its annotation information into the YOLOv5 neural network for training, use the validation set to evaluate the model performance, and fine-tune the network structure and parameters based on the performance feedback of the validation set to improve the model's recognition accuracy for people in infrared images. Finally, after sufficient training, a neural network model that can accurately recognize people in infrared images is obtained;

[0048] Module 3: After preprocessing, the infrared image data is passed to the neural network model to detect and select the human-shaped target, obtain the pixel width and pixel height of the target, calculate the target field of view angle using the detector field of view angle, and calculate the distance of the target from the detector using the mathematical relationship between the target field of view angle, target width and target distance;

[0049] Module 4 uses the bullet speed, wind resistance and gravity acceleration, combined with the calculated target distance, to establish a mathematical formula for the bullet's flight trajectory and calculate the bullet's landing point information in real time;

[0050] Module 5, after locking the target, uses a bilinear interpolation algorithm to accurately enlarge the aiming area, and displays the real-time infrared image, the target detection frame and distance information, the predicted bullet landing point and the locally enlarged image on the screen to assist aiming.

[0051] Compared with the prior art, the present invention has the following significant advantages: (1) The system functions are highly improved, combining infrared imaging, neural network image processing technology and ballistic trajectory estimation technology, and can perform various auxiliary aiming functions such as intelligent detection, point of impact judgment and local zooming. (2) The algorithm is fully hardware-based, which improves real-time performance, meets actual requirements and is easy to operate. (3) The algorithm has strong adaptability, is adjustable, and is easy to calibrate, and can be adapted to various shooting equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a schematic diagram of the intelligent auxiliary aiming system based on neural network of the present invention.

[0053] Figure 2 It is a flow chart of the intelligent auxiliary aiming system based on neural network of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0056] The invention discloses an intelligent auxiliary aiming system based on a neural network. The target image is collected in real time by an infrared detector, and is transmitted to an FPGA platform through a Camera Link protocol. After filtering and noise reduction and frequency domain detail enhancement processing are performed on the target image, the image is transmitted to a GPU platform through a UDP protocol. A humanoid target is detected and framed by a trained yolov5 neural network model, and the target distance is calculated in combination with a dense position method, so as to determine the bullet landing point, and the local zoom and other functions are integrated and synchronously output to a display device for auxiliary aiming. The system specifically comprises the following steps:

[0057] Step 1. Install the infrared detector on the gun body, ensure that the optical path center of the detector is strictly parallel to the gun barrel, so that the deviation between the two can be detected through the image and then corrected; use the infrared detector to collect actual scene information in real time.

[0058] Step 2: Perform Gaussian filtering on the collected infrared image to initially suppress noise, then perform grayscale stretching on the denoised image to increase its dynamic grayscale range, and then perform bilateral filtering to separate the base layer image. Subtract the base layer image from the denoised original image to obtain the high-frequency detail layer; perform a γ change with a γ value greater than 1 on the base layer to suppress background noise, and perform a γ change with a γ value less than 1 on the detail layer to enhance the details. Finally, the two layers of images are linearly superimposed to obtain an infrared image with enhanced details. The Gaussian filter template formula is:

[0059]

[0060] The template coordinates are (x, y), g(x, y) represents the image after Gaussian filtering, σ 2 is the overall variance. This patent adds Gaussian noise with a mean of 0 and a variance of 0.02 to the image.

[0061] The bilateral filter uses a two-dimensional Gaussian function to generate a distance template and a one-dimensional Gaussian function to generate a range template. The formula for generating the distance template coefficient is as follows:

[0062]

[0063] Among them, (k, l) is the center coordinate of the template window; (i, j) is the coordinate of other coefficients of the template window; σd is the standard deviation of the Gaussian function. The formula for generating the range template coefficient is as follows:

[0064]

[0065] Among them, f(x,y) represents the pixel value of the image at point (x,y); σ τ is the standard deviation of the Gaussian function. Multiplying the above two templates gives the template of the bilateral filter, as shown in the following formula:

[0066]

[0067] The expression for changing the gamma of the base layer and detail layer images is:

[0068]

[0069] It is defined as: for an input image f(x,y) with an M-bit grayscale range of [0,2M-1], we want to map it to an output image g(x,y) with an N-bit grayscale range of [0,2N-1].

[0070] Step 3. Select images that meet the application scenario from the KAIST data set and the actual data of the infrared detector, perform noise suppression and detail enhancement processing on the selected images according to step 2, eliminate images with poor results, and randomly divide the images that pass the quality assessment into two parts, one as a training set and the other as a validation set; input the training set and its annotation information into the YOLOv5 neural network for training, use the validation set to evaluate the model performance, and fine-tune the network structure and parameters based on the performance feedback of the validation set to improve the model's recognition accuracy of people in infrared images. Finally, after sufficient training, a neural network model that can accurately recognize people in infrared images is obtained.

[0071] Step 4: After the preprocessing in step 2, the infrared detector real shot data is passed to the neural network model to detect and select the human-shaped target, obtain the pixel width and pixel height of the target, calculate the target field angle using the detector field angle, and calculate the distance of the target from the detector using the mathematical relationship between the target field angle, target width (height) and target distance. The detector field angle calculation formula is:

[0072] θ (x / y) =2arctan((L (x / y) / 2) / f)

[0073] where θ (x / y) is the horizontal or vertical field of view of the detector, L (x / y) is the horizontal or vertical frame size of the detector sensor, and f is the focal length of the detector.

[0074] The formula for converting the detector density by the detector field angle is:

[0075]

[0076] Among them, MIL (x / y) It is the horizontal or vertical field of view of the detector, and MOA is the unit of angle.

[0077] The calculation formula of the target distance is:

[0078]

[0079] Where d is the target distance, l (y) is the target height, and the empirical value is generally 1.7. θ(y) is the vertical field angle of the target in the imaging.

[0080] Step 5: Using the physical parameters such as bullet speed, wind resistance, and gravity acceleration, combined with the target distance calculated in step 4, a mathematical formula for the bullet flight trajectory is established to calculate the bullet landing point information in real time. The bullet landing point calculation process is as follows:

[0081] The horizontal displacement of the bullet towards the target is:

[0082]

[0083] The calculation formula for the vertical deflection of the bullet is:

[0084]

[0085] Where d is the target distance, C is the air resistance coefficient, ρ is the air density, A is the cross-sectional area of ​​the bullet, v is the bullet speed, m is the mass of the bullet, and v 0x ,v 0yare the components of the bullet's velocity in the horizontal and vertical directions, and θ is the angle between the ballistic bullet and the horizontal target.

[0086] At the same time, the movement in the small time interval is regarded as uniform acceleration motion, and the Runge-Kutta algorithm is used to solve the θ and v of each time interval. This process is repeated until the horizontal displacement reaches d, and the corresponding time t is obtained. Finally, the vertical displacement d of the bullet is obtained. y ; Then change the initial θ to get different angles and corresponding vertical displacement d y , replace d, θ and the corresponding vertical displacement d y A lookup table is made. In the actual calculation process, the vertical displacement of the bullet (the point where the bullet falls) d can be obtained according to the previously calculated distance d and θ between the target and the detector. y , convert it into pixel position and mark it with a cross.

[0087] Step 6: After locking the target, the bilinear interpolation algorithm is used to accurately zoom in on the aiming area. In addition, the system will also display the real-time infrared image, the target detection frame and distance information, the predicted bullet landing point, and the locally zoomed image on the screen to assist aiming. The formula of the bilinear interpolation local zooming algorithm is:

[0088] f1(x,y)≈(x2-x)(y2-y)f(x1,y1)+(x-x1)(y2-y)f(x2,y1)

[0089] +(x2-x)(y-y1)f(x1,y2)+(x-x1)(y-y1)f(x2,y2)

[0090] Where f1(x,y) is the value of the pixel to be filled, (x1,y1), (x2,y2), (x3,y3) and (x4,y4) are the four known pixels around (x,y).

[0091] Example

[0092] In order to verify the effectiveness of the scheme of the present invention, the following experiment was carried out.

[0093] In the intelligent assisted aiming system based on neural network, the preprocessing of the acquired image is mainly Gaussian filtering and detail enhancement based on bilateral filtering. The images collected by the infrared imaging system often produce spatial noise due to the complexity of the scene or problems of the system itself, and it is necessary to preprocess the image to eliminate the noise as much as possible. This system uses Gaussian filter to perform spatial filtering on the original image.

[0094] The Gaussian filter template formula is:

[0095]

[0096] The template coordinates are (x,γ), g(x,γ) represents the image after Gaussian filtering, σ 2 In this embodiment, Gaussian noise with a mean of 0 and a variance of 0.02 is added to the image.

[0097] Infrared images can generally be divided into low-frequency and high-frequency parts in the spatial domain. The background and large-area targets that show the overall visual effect of the image have strong spatial correlation, and the change of pixel grayscale value is very small and gentle, showing low-frequency characteristics; while the edges of objects in the image, weak targets and local texture information of the target, their pixels are isolated and distributed, with extremely low spatial correlation, and the change of grayscale value is often sudden, showing high-frequency characteristics; in addition, those details with high local contrast can also be merged into the low-frequency part, because they can be clearly perceived. For this reason, this system separates the high-frequency and low-frequency parts of the infrared image, so as to analyze and process them separately according to the different frequency characteristics of each part. This system performs grayscale stretching on the image to increase its dynamic grayscale range, and then performs bilateral filtering to separate the base layer image, and obtains the high-frequency detail layer by subtracting the base layer image from the original image after denoising; the base layer is subjected to a γ change with a γ value greater than 1 to suppress background noise, and the detail layer is subjected to a γ change with a γ value less than 1 to enhance the details, and finally the two layers of images are linearly superimposed to obtain an infrared image with enhanced details.

[0098] The bilateral filter uses a two-dimensional Gaussian function to generate a distance template and a one-dimensional Gaussian function to generate a range template. The formula for generating the distance template coefficient is as follows:

[0099]

[0100] Among them, (k, l) is the center coordinate of the template window; (i, j) is the coordinate of other coefficients of the template window; σ d is the standard deviation of the Gaussian function. The formula for generating the range template coefficient is as follows:

[0101]

[0102] Among them, f(x,y) represents the pixel value of the image at point (x,y); σ τ is the standard deviation of the Gaussian function. Multiplying the above two templates gives the template of the bilateral filter, as shown in the following formula:

[0103]

[0104] The expression for changing the gamma of the base layer and detail layer images is:

[0105]

[0106] It is defined as: for an input image f(x,y) with an M-bit grayscale range of [0,2M-1], we want to map it to an output image g(x,y) with an N-bit grayscale range of [0,2N-1].

[0107] Images that meet the application scenarios are selected from the KAIST dataset and the actual data of infrared detectors. The selected images are subjected to noise suppression and detail enhancement processing according to step 2, and images with poor effects are eliminated. The images that pass the quality assessment are randomly divided into two parts, one as a training set and the other as a validation set. The training set and its annotation information are input into the YOLOv5 neural network for training, and the model performance is evaluated using the validation set. According to the performance feedback of the validation set, the network structure and parameters are fine-tuned to improve the model's recognition accuracy for people in infrared images. Finally, after sufficient training, a neural network model that can accurately recognize people in infrared images is obtained.

[0108] The infrared detector's real-shot data is preprocessed and passed to the neural network model to detect and select the human target, and calculate the distance of the target from the detector. The specific method is: first calculate the detector's field of view angle through the sensor frame and focal length, and then calculate the total density of the detector imaging. The target distance is calculated through the detected target pixel width, combined with the conversion relationship between density and pixel width and the mathematical relationship between density and target distance.

[0109] Taking into account air resistance, a mathematical model is established for the bullet's flight time, initial velocity and horizontal displacement to calculate the bullet's flight time. Then a mathematical model for the bullet's vertical displacement and flight time is established. Finally, the actual physical displacement is converted into a displayable digit, thereby showing the bullet's landing point.

[0110] After locking the target, the system uses a bilinear interpolation algorithm to accurately magnify the aiming area. In addition, the system will also display the real-time infrared image, the target detection frame and distance information, the predicted bullet landing point, and the locally magnified image on the screen to assist aiming.

[0111] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0112] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An intelligent auxiliary aiming method based on a neural network, characterized in that: Here are the steps: Step 1: Install the infrared detector on the gun body, ensure that the optical path center derivative line of the detector is parallel to the gun barrel, and use the infrared detector to collect actual scene information in real time; Step 2: Perform Gaussian filtering on the collected infrared image to initially suppress noise, then perform grayscale stretching on the denoised image to increase its dynamic grayscale range, and then perform bilateral filtering to separate the base layer image. Subtract the base layer image from the denoised original image to obtain the high-frequency detail layer. Perform a γ change with a γ value greater than 1 on the base layer to suppress background noise, and perform a γ change with a γ value less than 1 on the detail layer to enhance details. Finally, the two layers of images are linearly superimposed to obtain a detail-enhanced infrared image. Step 3: Select images that meet the application scenario from the KAIST data set and the collected infrared image data, perform noise suppression and detail enhancement processing on the selected images according to step 2, eliminate images with poor results, and randomly divide the images that pass the quality assessment into two parts, one as a training set and the other as a validation set; input the training set and its annotation information into the YOLOv5 neural network for training, use the validation set to evaluate the model performance, and fine-tune the network structure and parameters based on the performance feedback of the validation set to improve the model's recognition accuracy for people in infrared images. Finally, after sufficient training, a neural network model that can accurately recognize people in infrared images is obtained; Step 4: After the infrared image data is preprocessed in step 2, it is passed to the neural network model to detect and select the human-shaped target, obtain the pixel width and pixel height of the target, calculate the target detector field of view angle using the detector field of view angle, and calculate the distance of the target from the detector using the mathematical relationship between the target field of view angle, target width and target distance; Step 5: Using the bullet speed, wind resistance and gravity acceleration, combined with the distance from the target to the detector calculated in step 4, a mathematical formula for the bullet flight trajectory is established to calculate the bullet landing point information in real time; Step 6: After locking the target, a bilinear interpolation algorithm is used to accurately enlarge the aiming area. The real-time infrared image, the target detection frame and distance information, the predicted bullet landing point and the locally enlarged image are displayed on the screen to assist aiming.

2. The intelligent auxiliary aiming system based on neural network according to claim 1 is characterized in that: Step 2: Perform Gaussian filtering on the collected infrared image to initially suppress noise, then perform grayscale stretching on the denoised image to increase its dynamic grayscale range, and then perform bilateral filtering to separate the base layer image. Subtract the base layer image from the denoised original image to obtain the high-frequency detail layer. Perform a γ change with a γ value greater than 1 on the base layer to suppress background noise, and perform a γ change with a γ value less than 1 on the detail layer to enhance details. Finally, the two layers of images are linearly superimposed to obtain a detail-enhanced infrared image, where: The Gaussian filter template formula is: The template coordinates are (x, y), g(x, y) represents the image after Gaussian filtering, σ 2 is the overall variance; The bilateral filter uses a two-dimensional Gaussian function to generate a distance template and a one-dimensional Gaussian function to generate a range template. The generation formula of the distance template coefficient is as follows: Among them, (k, l) is the center coordinate of the template window; (i, j) is the coordinate of other coefficients of the template window; σ d is the standard deviation of the Gaussian function, and the generation formula of the range template coefficient is as follows: Among them, f(x,y) represents the pixel value of the image at point (x,y); σ τ is the standard deviation of the Gaussian function, and the template formula of the bilateral filter is as follows: The expression for changing the gamma of the base layer and detail layer images is:

3. The intelligent auxiliary aiming system based on neural network according to claim 1 is characterized in that: Step 4: After the preprocessing in step 2, the infrared detector real shot data is transmitted to the neural network model, the human-shaped target is detected and selected, the pixel width and pixel height of the target are obtained, and the target field angle detector is calculated using the detector field angle. The mathematical relationship between the target field angle, target width and target distance is used to calculate the distance of the target from the detector, where: The detector field of view angle calculation formula is: <h2 style=";text-align:left;direction:ltr">θ<h2 style=";text-align:left;direction:ltr"> (x / y) <h2 style=";text-align:left;direction:ltr"> =2arctan((L<h2 style=";text-align:left;direction:ltr"> (x / y) <h2 style=";text-align:left;direction:ltr"> / 2) / f) where θ (x / y) is the horizontal or vertical field of view of the detector, L (x / y) is the horizontal or vertical frame size of the detector sensor, and f is the focal length of the detector; The formula for converting the detector density by the detector field angle is: Among them, MIL (x / y) It is the horizontal or vertical field of view of the detector, and MOA is the unit of angle. The target distance is calculated as: Where d is the distance between the target and the detector, l (y) is the target height, and θ(y) is the vertical field angle of the target in the imaging.

4. The intelligent auxiliary aiming system based on neural network according to claim 1 is characterized in that: Step 5: Use the bullet speed, wind resistance, gravity acceleration, and the target distance calculated in step 4 to establish a mathematical formula for the bullet's flight trajectory and calculate the bullet's landing point information in real time. The specific method is as follows: The horizontal displacement of the bullet towards the target is: The calculation formula for the vertical deflection of the bullet is: Where d is the target distance, C is the air resistance coefficient, ρ is the air density, A is the cross-sectional area of ​​the bullet, v is the bullet speed, m is the mass of the bullet, and v 0x ,v 0y are the components of the bullet’s velocity in the horizontal and vertical directions, and θ is the angle between the ballistic bullet and the horizontal target; At the same time, the movement in the small time interval is regarded as uniform acceleration motion, and the Runge-Kutta algorithm is used to solve the θ and v of each time interval. This process is repeated until the horizontal displacement reaches d, and the corresponding time t is obtained. Finally, the vertical displacement d of the bullet is obtained. y ; Then change the initial θ to get different angles and corresponding vertical displacement d y , replace d, θ and the corresponding vertical displacement d y A lookup table is made. In the actual calculation process, the vertical displacement of the bullet (the point where the bullet falls) d can be obtained according to the previously calculated distance d and θ between the target and the detector. y , convert it into pixel position and mark it with a cross.

5. The intelligent auxiliary aiming system based on neural network according to claim 1 is characterized in that: Step 6: After locking the target, the bilinear interpolation algorithm is used to accurately enlarge the aiming area. The real-time infrared image, the target detection frame and distance information, the predicted bullet landing point and the locally enlarged image are displayed on the screen to assist aiming. The bilinear interpolation formula is: f1(x,y)≈(x2-x)(y2-y)f(x1,y1)+(x-x1)(y2-y)f(x2,y1)+(x2-x)(y-y1)f(x1,y2)+(x-x1)(y-y1)f(x2,y2) Where f1(x,y) is the value of the pixel to be filled, (x1,y1), (x2,y2), (x3,y3) and (x4,y4) are the four known pixels around (x,y).

6. An intelligent auxiliary aiming system based on neural network, characterized in that: Implementing the neural network-based intelligent assisted aiming method according to any one of claims 1 to 5 to realize the neural network-based intelligent assisted aiming comprises: Module 1: Gaussian filtering is performed on the collected infrared image to initially suppress noise, and then the grayscale of the denoised image is stretched to increase its dynamic grayscale range. Then, bilateral filtering is performed to separate the base layer image, and the high-frequency detail layer is obtained by subtracting the base layer image from the denoised original image. The base layer is subjected to a γ change with a γ value greater than 1 to suppress background noise, and the detail layer is subjected to a γ change with a γ value less than 1 to enhance details. Finally, the two layers of images are linearly superimposed to obtain a detail-enhanced infrared image. Module 2: Select images of the application scene from the KAIST dataset and the infrared detector real-shot data, perform noise suppression and detail enhancement on the selected images, remove images with poor results, and randomly divide the images that pass the quality assessment into two parts, one as a training set and the other as a validation set; input the training set and its annotation information into the YOLOv5 neural network for training, use the validation set to evaluate the model performance, and fine-tune the network structure and parameters based on the performance feedback of the validation set to improve the model's recognition accuracy for people in infrared images. Finally, after sufficient training, a neural network model that can accurately recognize people in infrared images is obtained; Module 3, after preprocessing, the infrared image data detector is passed to the neural network model to detect and select the human-shaped target, obtain the pixel width and pixel height of the target, calculate the target field of view angle using the detector field of view angle, and calculate the distance of the target from the detector using the mathematical relationship between the target field of view angle, target width and target distance; Module 4 uses the bullet speed, wind resistance and gravity acceleration, combined with the calculated target distance, to establish a mathematical formula for the bullet's flight trajectory and calculate the bullet's landing point information in real time; Module 5, after locking the target, uses a bilinear interpolation algorithm to accurately enlarge the aiming area, and displays the real-time infrared image, the target detection frame and distance information, the predicted bullet landing point and the locally enlarged image on the screen to assist aiming.

Citation Information

Patent Citations

  • Power equipment infrared thermogram identification method combining deep learning with traditional algorithm

    CN112508073A

  • Intelligent training and detection method for infrared target under cloud background

    CN114022733A

  • Visualization-based shooting aiming method

    CN116086244A

  • Infrared weak and small target detection method based on local contrast mechanism

    CN116189160A

  • Infrared tracking and aiming system based on non-refrigeration type infrared detector

    CN116781996A