An artificial intelligence low-light aiming system and method

Abstract

An artificial intelligence micro-light aiming system and method belong to the technical field of intelligent aiming of military weapons. The system comprises a micro-light lens, a low-illumination image sensor, a CPU, an environment sensing module, a posture sensing module, a wireless transmitting module and a wireless display module. The present application adopts core algorithms such as automatic identification algorithm, automatic tracking algorithm and intelligent ranging algorithm to intelligently identify, track and lock the target. When the aiming scale and the marking scale are displayed on the wireless display module, the trigger is pulled to hit the target, which can improve the first-hit rate of the weapon, reduce the personnel training accumulation and physical condition requirements, reduce the influence of individual subjective factors on the aiming hit rate and increase the aiming safety to realize concealed aiming. The intelligent ranging function can reduce the dependence on the laser ranging module for evaluating the target distance and reduce the product weight. The wireless display module is used to receive images, and the personnel can not use the cheek supporting aiming mode, concealed aiming and reduce the risk of being hit.

Description

Technical Field

[0001] This invention belongs to the field of intelligent aiming technology for military weapons, and in particular relates to an artificial intelligence low-light aiming system and method for daytime and nighttime observation, automatic identification and tracking, intelligent ranging, automatic perception of environmental parameters, automatic generation of aiming point lead, and enabling covert shooting. Background Technology

[0002] With the rapid development of artificial intelligence technology, day and night combat modes are constantly evolving. Military weapons are gradually moving towards intelligent development. Traditional aiming systems typically involve a sight providing a target image, with the soldier resting their chin on their hand and aiming directly. After observing the image, the soldier subjectively judges the target type based on personal experience, estimates and determines the target distance based on combat experience or a rangefinder, and then aims at the target using a mil-dot firing table. This aiming system is limited by subjective factors such as the soldier's training experience and physical condition on the spot, resulting in significant uncertainty in target aiming accuracy. Direct aiming with the chin on the sight can easily expose the target and increase the risk of being hit.

[0003] Therefore, there is an urgent need for a new technical solution to address this problem. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide an artificial intelligence low-light aiming system and method to solve the technical problem that traditional aiming systems rely on subjective aiming and judgment based on personal experience, which has high uncertainty and makes it easy for soldiers to expose the target when aiming directly with their chins on.

[0005] An artificial intelligence low-light aiming system includes a low-light lens, a low-light image sensor, a CPU, an environmental perception module, an attitude perception module, a power supply module, a wireless transmission module, a weapon mounting module, function buttons, and a wireless display module. The weapon mounting module is fixedly connected to the weapon. The low-light lens is connected to the low-light image sensor and is used to receive the visible spectrum of the scene. The low-light image sensor is used to achieve 1×10 Hz illumination during daytime or nighttime. -3The system includes: a visible light signal acquisition module for low-light environments; an environmental perception module for acquiring environmental parameters such as temperature, air pressure, and altitude; an attitude perception module for acquiring weapon attitude; an embedded neural network processor (NPU) on the CPU, which serves as the core calculation module of the system. This core calculation module is connected to the low-light image sensor, environmental perception module, attitude perception module, power supply module, wireless transmission module, and function buttons. The core calculation module locks onto the target via the function buttons and automatically calculates the trajectory and generates marking and aiming reticles based on preset ammunition types, environmental parameters, target distance, and weapon attitude. The wireless transmission module is connected to the wireless display module for signal transmission, transmitting compressed video images wirelessly. The wireless display module receives and decodes the wireless signals for display. When the marking and aiming reticles in the displayed image coincide, the soldier pulls the trigger to fire.

[0006] The function key selection includes a power button, up / down / left / right directional keys, number keys, confirmation key, and aiming key.

[0007] The attitude perception module includes a six-axis sensor and a data processing module.

[0008] The environmental sensing module includes temperature and air pressure sensors and a data processing module. The data processing module obtains the altitude value based on the collected air pressure value using the air pressure-altitude conversion formula. The environmental parameters also include the wind speed value measured by an anemometer, which is manually input to the core calculation module via function keys.

[0009] The core calculation module includes an automatic identification module, an automatic tracking module, and an intelligent ranging module; the automatic identification module automatically identifies targets by detecting environmental parameters; the automatic tracking module tracks selected targets in real time; and the intelligent ranging module measures the distance to selected targets in real time.

[0010] An artificial intelligence low-light aiming method, utilizing the aforementioned artificial intelligence low-light aiming system, includes the following steps, which are performed sequentially:

[0011] Step 1: Train an improved YOLOv5 model network on a PC using deep learning methods based on a manually labeled sample library. The improved YOLOv5 model network replaces the Focus module with a large convolution module to make it more suitable for inference operations of an embedded neural network processor (NPU). Then, use a toolchain to transform the improved YOLOv5 model network and load it onto the CPU of an artificial intelligence low-light aiming system.

[0012] Step 2: Press and hold the function button to turn on the system. The environmental perception module automatically detects environmental parameters, the low-light lens and low-light image sensor collect visible light video signals in low-light environments, and the attitude perception module detects the weapon's attitude values.

[0013] Step 3: The automatic identification module in the core solution module uses the YOLOv5 model network to automatically identify personnel and vehicle targets. The target closest to the center of the field of view is selected by double bounding box by default. Click the up / down / left / right arrow keys of the function button to switch the pre-selected target in the field of view, and click the OK button of the function button to select the target.

[0014] Step 4: The automatic tracking module in the core solution module tracks the target using the Bytetrack tracking algorithm, obtaining the bounding boxes and identities of automatically identified people and vehicles in the video;

[0015] Step 5: The intelligent ranging module in the core calculation module uses an intelligent ranging algorithm to measure the distance to the selected target in real time, lock the target, and obtain the target distance;

[0016] Step 6: Click the aiming button on the function key. The core calculation module automatically calculates the trajectory based on the preset ammunition type firing table, environmental parameters, target distance, and weapon attitude, and generates the target marking reticle and aiming point lead, i.e., the aiming reticle, which is displayed on the wireless display module through the wireless transmission module.

[0017] Step 7: When the markings and aiming reticles on the wireless display module coincide, the soldier can pull the trigger to hit the target.

[0018] In step one, floating-point operations are converted to 8-bit integer operations and an asymmetric floating-point quantization algorithm is used to quantize the model to ensure the computational accuracy of the model.

[0019] When obtaining the bounding boxes and identities of automatically identified people and vehicles in the video in step four, the detection boxes of multiple objects, the corresponding detection scores of each detection box, and the tracking trajectories of each detection box are obtained through the multi-object tracking (MOT) method. The detection boxes are divided into two groups according to the detection scores by setting a threshold I, namely high-scoring detection boxes and low-scoring detection boxes, and identity matching is performed twice. The first time, the high-scoring detection boxes are matched with the tracking trajectories of the previously obtained detection boxes. The second time, the low-scoring detection boxes are matched with the tracking trajectories of the high-scoring detection boxes that did not match in the first time.

[0020] For detection boxes that do not match the tracking trajectory and whose scores are lower than the set threshold I but higher than the set threshold II, a new tracking trajectory is created for them;

[0021] For tracking trajectories that do not match detection boxes, retain n frames. When a detection box matches the tracking trajectory, perform the matching again.

[0022] The low-scoring detection box uses the Bytetrack algorithm, which tracks each tracking trajectory that does not match a high-scoring detection box. The low-scoring detection box is the low-scoring detection box that is restored to the real object and the background is filtered out by using its similarity to the tracking trajectory.

[0023] The intelligent ranging algorithm is as follows: it calculates the distance to the target based on the pinhole imaging principle, and optimizes and compensates for measurement errors through a neural network model to achieve intelligent ranging.

[0024] Through the above design scheme, the present invention can bring the following beneficial effects:

[0025] This invention employs core algorithms such as automatic identification, automatic tracking, and intelligent ranging to intelligently identify, track, and lock onto targets. When the aiming reticle and the marker reticle on the wireless display module coincide, pulling the trigger results in a successful hit. Soldiers no longer need to rely on personal experience to choose the firing timing, thus improving the weapon's first-shot hit probability, reducing the requirements for personnel training and physical condition, minimizing the impact of subjective factors on aiming accuracy, and increasing aiming safety for concealed aiming. The intelligent ranging function reduces reliance on laser rangefinders for target distance assessment, reducing product weight. Using the wireless display module to receive images allows personnel to aim covertly without using a chin-resting aiming method, reducing the risk of being hit. Attached Figure Description

[0026] The present invention will be further described below with reference to the accompanying drawings and specific embodiments:

[0027] Figure 1 This is a structural block diagram of the artificial intelligence low-light aiming system and method of the present invention.

[0028] Figure 2 This is a flowchart of a method in an artificial intelligence low-light aiming system and method of the present invention.

[0029] Figure 3 This is a hardware structure block diagram of an embodiment of an artificial intelligence low-light aiming system and method according to the present invention.

[0030] Figure 4 This is a hardware structure block diagram of the wireless display module in an embodiment of an artificial intelligence low-light aiming system and method of the present invention.

[0031] Figure 5 This is a software functional flowchart of an artificial intelligence low-light aiming system and method according to the present invention.

[0032] Figure 6This is a flowchart of the target recognition algorithm in an artificial intelligence low-light aiming system and method of the present invention.

[0033] Figure 7 This is a schematic diagram of the pinhole imaging principle in an artificial intelligence low-light aiming system and method of the present invention.

[0034] In the diagram, 1-low-light lens, 2-low-illuminance image sensor, 3-core calculation module, 4-environmental perception module, 5-attitude perception module, 6-power supply module, 7-wireless transmission module, 8-function button, 9-weapon connector module, 10-wireless display module, 11-application layer, 12-algorithm layer, 13-operating system layer, 14-data link layer, 15-hardware driver layer, 301-EMMC memory, 302-DDR memory, 303-wifi chip, 304-domestic ARM main control chip, 401-temperature and air pressure sensor, 501-six-axis sensor, 1003-OLED display. Detailed Implementation

[0035] An artificial intelligence low-light aiming system mainly consists of a low-light lens 1, a low-light image sensor 2, a CPU, an environmental perception module 4, an attitude perception module 5, a power supply module 6, a wireless transmission module 7, a weapon connector module 9, function buttons 8, and a wireless display module 10. The CPU serves as the core calculation module 3 of the system. The structural block diagram of the artificial intelligence low-light aiming system is shown below. Figure 1 The low-light lens 1 is mainly used to receive the visible spectrum of the target, and the low-light image sensor 2 is used to achieve 1×10⁻⁶ illumination during daytime or nighttime. -3 Visible light signal acquisition of targets under low illumination conditions (lx). Core calculation module 3, as the main control unit, runs various software function programs, see... Figure 5 The core calculation module 3 performs image analysis and processing on the optical signal, analyzes and calculates the computational parameters, and transmits the image and calculation results in video form via wireless Wi-Fi. The weapon connector module 4 serves as an adapter between the aiming system and the weapon, ensuring a secure connection. Function buttons 8 are used to power on / off the AI ​​aiming system and perform various function operations. The attitude perception module 5, composed of a six-axis sensor 501 and accompanying software, detects the system's attitude angles, providing angular parameters for intelligent aiming. The environmental perception module 4, composed of a temperature and pressure sensor 401 and an adapter program, detects the temperature, air pressure, and altitude of the system's operating environment, providing computational parameters for the combat environment for intelligent aiming. The wireless transmission module 7, mainly composed of a Wi-Fi chip 701 and accompanying software, transmits compressed video images via wireless signals. The power supply module 6 provides electrical energy to ensure system operation. The wireless display module 10 receives, decodes, and displays wireless signals.

[0036] Example:

[0037] The electronic hardware of an artificial intelligence low-light aiming system adopts a completely domestically developed embedded hardware platform framework, mainly consisting of a low-light image sensor 2, an eMMC memory 301, a DDR memory 302, a Wi-Fi chip 303, a domestically produced ARM main control chip 304, a six-axis sensor 501, and a temperature and pressure sensor 401. (See framework details below.) Figure 3 The electronic hardware of the wireless display module 10 mainly consists of a Wi-Fi chip, a domestically produced ARM main control chip, an OLED display 1003, DDR memory, and an eMMC memory. The framework is shown in [reference needed]. Figure 4 .

[0038] The software functional program framework of an artificial intelligence low-light aiming system is shown below. Figure 5 Application layer 11 calls internal programs to implement the system's artificial intelligence aiming based on aiming logic. Algorithm layer 12 mainly includes: image processing algorithms, which use the processor's ISP module to achieve automatic exposure and automatic white balance for the acquired image data; target recognition and tracking algorithms, which use the YOLOv5 neural network algorithm to detect targets and the Bytetrack tracking algorithm to lock and track targets; and intelligent ranging algorithms to calculate target distance in real time. Operating system layer 13 completes the task scheduling, peripheral device configuration, memory management, and isolates the coupling between user space and the underlying driver space of the entire system. Data link layer 14 processes bus data, including data acquisition, data storage, and data analysis. Hardware driver layer 15 is responsible for communicating with hardware devices, including reading sensor data and controlling actuators.

[0039] See the aiming process of the AI ​​aiming system. Figure 2 Upon system startup, the system automatically detects environmental parameters and intelligently identifies personnel and vehicle targets. Targets closest to the center of the field of view are automatically double-framed and tracked. These double-framed targets are locked by an intelligent ranging algorithm, with real-time distance measurement. Target locking: Personnel can choose to set up their weapons and conceal their positions for observation and aiming. After identifying the target, the system locks onto it. Based on preset ammunition types, environmental parameters, target distance, weapon attitude, and automatically calculated trajectory, the system generates a target marking and aiming lead (aiming reticle). The aiming personnel align the aiming reticle with the marking and pull the trigger to hit the target.

[0040] The system employs core algorithms including automatic target recognition, automatic tracking, and intelligent ranging. The target recognition algorithm utilizes deep learning; its flowchart is shown below. Figure 6As shown, an improved YOLOv5 model network was first trained on a PC using a large database of manually labeled samples. The improved model replaced the Focus module with a large convolutional module, making it more suitable for NPU inference computation. Then, the model was converted using a toolchain provided by Rockchip to make it suitable for deployment on the RK hardware platform for object detection. The key challenges lie in recognition accuracy and speed. Embedded platforms lack dedicated graphics cards like those on PCs, have limited computing resources, and operate on 8-bit integer precision, which is inferior to the floating-point precision of PCs. The neural network model was improved based on the current YOLO model to improve efficiency while maintaining accuracy. The preprocessing part was removed from the model and placed in a separate thread on the CPU, accelerated using the embedded platform's RGA hardware. Post-processing was also performed on a separate CPU thread, and inference was run on a separate thread, fully utilizing the CPU and NPU computing power of the embedded platform to significantly improve processing speed. In terms of computational accuracy, when converting the model to the embedding platform, floating-point operations are converted to 8-bit integer operations, which inevitably reduces computational accuracy. However, during the conversion, some training samples are used for data statistical calculations, and an asymmetric floating-point quantization algorithm is used for model quantization, which greatly ensures the computational accuracy of the model.

[0041] The automatic tracking algorithm uses the multi-object tracking (MOT) method to estimate the bounding boxes and identities of objects in the video. Byte retains almost all detection boxes (obtained from YOLO detection results), and divides the detection boxes into two groups based on their detection scores by setting a threshold I. The identities are obtained by associating detection boxes with detection scores higher than the threshold and performing two identity matching processes. The first matching uses the high-scoring detection boxes and the previously obtained detection box tracking trajectories for identity matching. The second matching uses the low-scoring detection boxes and the tracking trajectories that did not match the high-scoring detection boxes in the first matching process for identity matching.

[0042] For detection boxes that do not match the tracking trajectory and whose scores are lower than the set threshold I but higher than the set threshold II, a new tracking trajectory is created for them;

[0043] For tracking trajectories that do not match detection boxes, retain n frames. When a detection box matches the tracking trajectory, perform the matching again.

[0044] In trajectory tracking and identity matching, only Kalman filtering is used to predict the position of the tracking trajectory in the next frame. The IoU between the predicted bounding box and the actual detection box is used as the similarity between the two matches, and the matching is completed using the Hungarian algorithm. Kalman filtering, proposed by Kalman based on linear minimum variance estimation, is mathematically simple and is the optimal linear recursive filtering method, with advantages such as low computational cost, low storage requirement, and high real-time performance. A target state model is used to describe the relationship between the state at a certain moment and the state at previous moments, such as the target's motion model. The state variables can be the target's position and velocity. Generally, increasing the number of state variables increases the computational cost of estimation, and if the state order is higher than the actual number of target states, the estimation accuracy decreases.

[0045] Therefore, the basic model of Kalman filtering is X(k+1)=F(k)X(k)+Γ(k)W(k)

[0046] The observation model is Z(k+1)=H(k+1)X(k+1)+V(k+1)

[0047] Where: X(k+1) is the state vector of the next frame; F(k) is the state transition matrix; X(k) is the state vector; Γ(k) is the process noise distribution matrix; W(k) is the zero-mean, white Gaussian process noise sequence; H(k+1) is the measurement matrix; and V(k+1) is the measurement noise. Kalman filtering utilizes the target's dynamic information to remove the influence of noise and obtain a good estimate of the target's position. Kalman filtering does not require storing past measurement information. When new measurement information is obtained, a new estimate can be obtained using dynamic equations and recursive formulas based on the new data and the previous time-stamped estimate, thus achieving dynamic target tracking.

[0048] The environmental sensing module employs temperature and pressure sensors. Altitude H is calculated using a conversion formula between air pressure P and altitude H, referring to the altitude-pressure conversion table in GJB365.2-1987. The conversion formula is as follows:

[0049]

[0050] During intelligent ranging, based on the pinhole imaging principle, see... Figure 7 According to the formula for similar triangles, we can obtain:

[0051] f / d = w' / w,

[0052] In the formula, F represents the camera focal length; d represents the target distance; w' represents the image size, specifically the pixel width or height occupied by the target in the image * pixel size; and W represents the width or height of the target.

[0053] The distance to typical targets is calculated, and then a large amount of data is used to continuously learn and optimize through a neural network model to make up for measurement errors, thereby achieving intelligent distance measurement of typical targets.

Claims

1. An artificial intelligence-based low-light aiming method, characterized by: An artificial intelligence low-light aiming system is adopted, comprising a low-light lens (1), a low-light image sensor (2), a CPU, an environment perception module (4), an attitude perception module (5), a power supply module (6), a wireless transmission module (7), a weapon connector module (9), function buttons (8), and a wireless display module (10). The weapon connector module (9) is fixedly connected to the weapon. The low-light lens (1) is connected to the low-light image sensor (2) and is used to receive the visible spectrum at the scene. The low-light image sensor (2) is used to achieve 1×10 during daytime or nighttime. -3 The system collects visible light signals in low-light environments; the environment perception module (4) is used to collect environmental parameters such as temperature, air pressure and altitude; the attitude perception module (5) is used to collect weapon attitude; the CPU is equipped with an embedded neural network processor (NPU), which serves as the core calculation module (3) of the system. The core calculation module (3) is connected to the low-light image sensor (2), environment perception module (4), attitude perception module (5), power supply module (6), wireless transmission module (7) and function button (8). The core calculation module (3) locks onto the target through the function button (8) and automatically calculates the trajectory of the locked target based on the preset ammunition table, environmental parameters, target distance and weapon attitude, and generates marking and aiming reticles; the wireless transmission module (7) is connected to the wireless display module (10) for signal transmission. The wireless transmission module (7) is used to transmit the compressed video image to the wireless display module (10) via wireless signal; the wireless display module (10) is used to receive and decode the wireless signal for display. When the marking and aiming reticles overlap in the displayed image, the soldier pulls the trigger to complete the firing. The core calculation module (3) includes an automatic identification module, an automatic tracking module, and an intelligent ranging module; the automatic identification module automatically identifies targets by detecting environmental parameters; the automatic tracking module tracks selected targets in real time; and the intelligent ranging module measures the distance to selected targets in real time. The method includes the following steps: And the following steps are performed in sequence. Step 1: Train an improved YOLOv5 model network on a PC using deep learning methods based on a manually labeled sample library. The improved YOLOv5 model network replaces the Focus module with a large convolution module to make it more suitable for inference operations of an embedded neural network processor (NPU). Then, use a toolchain to transform the improved YOLOv5 model network and load it onto the CPU of an artificial intelligence low-light aiming system. Step 2: Press and hold the function button (8) to turn on the system. The environmental perception module (4) automatically detects environmental parameters, the low-light lens (1) and the low-light image sensor (2) collect visible light video signals in low-light environments, and the attitude perception module (5) detects the attitude values ​​of the weapon. Step 3: The automatic identification module in the core solution module (3) uses the YOLOv5 model network to automatically identify personnel and vehicle targets. The target closest to the center of the field of view is selected by double frame by default. Click the up / down / left / right arrow keys of the function button (8) to switch the pre-selected target in the field of view. Click the OK button of the function button (8) to select the target. Step 4: The automatic tracking module in the core solution module (3) tracks the target using the Bytetrack tracking algorithm to obtain the bounding boxes and identities of automatically identified people and vehicles in the video; When obtaining the bounding boxes and identities of automatically identified people and vehicles in the video, the Multi-Object Tracking (MOT) method is used to obtain the detection boxes of multiple objects, the corresponding detection scores of each detection box, and the tracking trajectories of each detection box. By setting a threshold I, the detection boxes are divided into two groups according to their detection scores: high-scoring detection boxes and low-scoring detection boxes. Identity matching is performed twice. The first time, the high-scoring detection boxes are matched with the tracking trajectories of the previously obtained detection boxes. The second time, the low-scoring detection boxes are matched with the tracking trajectories of the high-scoring detection boxes that did not match in the first match. For detection boxes that do not match the tracking trajectory and whose scores are lower than the set threshold I but higher than the set threshold II, a new tracking trajectory is created for them; For tracking trajectories that do not match detection boxes, retain n frames, and perform matching again when a detection box matches the tracking trajectory; The low-scoring detection box uses the Bytetrack algorithm, which tracks the object by associating it with the tracking trajectory of each high-scoring detection box that does not match it. The low-scoring detection box is the low-scoring detection box that is restored by using its similarity to the tracking trajectory and then filtering out the background. Step 5: The intelligent ranging module in the core calculation module (3) uses the intelligent ranging algorithm to measure the distance to the selected target in real time, lock the target and obtain the target distance; Step 6: Click the aiming key of function button (8). The core calculation module (3) automatically calculates the trajectory according to the preset ammunition type firing table, environmental parameters, target distance, and weapon attitude, and generates the target marking and aiming point lead, i.e., the aiming marking, which is displayed on the wireless display module (10) through the wireless transmission module (7). Step 7: When the markings and aiming reticles on the wireless display module (10) coincide, the soldier can pull the trigger to hit the target.

2. The artificial intelligence low-light aiming method according to claim 1, characterized in that: The function keys (8) include the power button, up / down / left / right directional keys, number keys, confirmation key, and aiming key.

3. The artificial intelligence low-light aiming method according to claim 1, characterized in that: The attitude sensing module (5) includes a six-axis sensor and a data processing module.

4. The artificial intelligence low-light aiming method according to claim 1, characterized in that: The environmental sensing module (4) includes a temperature and air pressure sensor and a data processing module. The data processing module obtains the altitude value by means of the air pressure altitude conversion formula based on the collected air pressure value. The environmental parameters also include the wind speed value measured by the anemometer. The wind speed value is manually input to the core calculation module (3) through the function key (8).

5. The artificial intelligence low-light aiming method according to claim 1, characterized in that: In step one, floating-point operations are converted to 8-bit integer operations and an asymmetric floating-point quantization algorithm is used to quantize the model to ensure the computational accuracy of the model.

6. The artificial intelligence low-light aiming method according to claim 1, characterized in that: The intelligent ranging algorithm is as follows: it calculates the distance to the target based on the pinhole imaging principle, and optimizes and compensates for measurement errors through a neural network model to achieve intelligent ranging.

Images