A Deep Learning-Based Intelligent Target Finding System and Method

Abstract

A deep learning-based intelligent target-finding system and method, involving artificial intelligence, computer vision, and automatic target reporting technologies, solves the problems of insufficient target-finding accuracy, susceptibility to environmental influences, and reliance on manual adjustments in traditional target reporting systems. It is particularly suitable for complex scenarios with multiple targets and multiple cameras. This method is mainly applied in shooting training scenarios for the military and police. By employing deep learning technology to achieve an automated and intelligent target-finding process, it improves the system's positioning accuracy, real-time performance, and environmental adaptability. The intelligent target-finding method of this invention integrates data acquisition, deep learning detection, precise turntable control, and feedback adjustment through multi-module collaboration, realizing intelligent target-finding in multi-target, multi-camera systems. This method significantly improves target-finding speed and efficiency while meeting accuracy requirements, and has broad practical value and promising application prospects.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence, computer vision, and automatic target reporting, and specifically to an intelligent target-finding system and method based on deep learning. Background Technology

[0002] With the development of optoelectronic technology and intelligent recognition algorithms, vision-based non-contact automatic target reporting systems are gradually replacing the high-cost contact-based systems. In vision-based automatic target reporting systems, a one-to-one correspondence between multiple targets and cameras is typically used. In systems with multiple targets and cameras, achieving precise alignment and automatic target finding for each camera at a specific target is crucial for the automated target reporting system. This process, often called "target finding," refers to each camera finding and aligning with a predetermined target in a complex scene, ensuring the target is centered or in an optimal position within its field of view. The accuracy and efficiency of target finding directly affect the performance of the entire reporting system. Traditional target finding techniques often rely on mechanical adjustments, manual calibration, or simple image processing algorithms, such as using color, shape, or edge detection features to locate the target. These methods usually require manual adjustment and pre-calibration, are greatly affected by ambient lighting, target pose, and occlusion factors, and are difficult to automate in dynamic or complex environments. Furthermore, traditional methods often lack effective feature differentiation methods in multi-target scenarios, easily leading to false detections or misalignments, thus affecting the system's stability and accuracy. With the rapid development of deep learning technology, object detection methods based on convolutional neural networks (CNNs) have brought new possibilities to target-finding tasks. Deep learning technology can automatically learn multi-level features of images, enabling cameras to effectively identify and locate specific targets under different lighting conditions and complex backgrounds. How to design an efficient and accurate deep learning target-finding method, so that each camera can accurately find the corresponding target in real-time dynamic conditions and maximize the target area within its field of view, has become a current research hotspot and challenge.

[0003] Therefore, there is an urgent need for an efficient and accurate deep learning-based intelligent target finding method that can improve the real-time performance of the model while ensuring recognition accuracy, support intelligent automatic alignment of multi-camera systems, and adapt to diverse target layouts.

[0004] The intelligent target-finding system and method of this invention are used in target reporting equipment with multiple target positions and multiple cameras for shooting training in the military, police, and other units. The method receives real-time frame images from multiple cameras, extracts semantic information from the images using multiple convolution operations, identifies the targets, and outputs the target miss distance information to the main control system. The main control system then drives the turntable to rotate based on the miss distance information, repeating this process until all target positions appear at the center of the target camera's field of view and maximize the filling of the entire field of view. Summary of the Invention

[0005] This invention provides a deep learning-based intelligent target-finding system and method, aiming to solve the problems of insufficient target-finding accuracy, susceptibility to environmental influences, and reliance on manual adjustments in traditional target-reporting systems. It is particularly suitable for complex scenarios with multiple target positions and multiple cameras. This method is mainly applied in shooting training scenarios such as those used by the military and police. By employing deep learning technology to achieve an automated and intelligent target-finding process, it can improve the system's positioning accuracy, real-time performance, and environmental adaptability.

[0006] A deep learning-based intelligent target-finding system includes a data acquisition module, a feature extraction module, a target detection module, a deviation calculation module, a turntable adjustment module, and a feedback control module.

[0007] The data acquisition module is used to acquire real-time video frame images from each camera and transmit them to the feature extraction module;

[0008] The feature extraction module performs feature extraction on the video frame image to obtain the edge, texture and shape features of the target image and obtain the target feature map.

[0009] The target detection module uses a deep learning-based target detection algorithm to identify the target feature map and obtain the coordinate position of the target in the feature map, i.e., the target center position; and transmits the target center position to the deviation calculation module.

[0010] The deviation calculation module calculates the deviation value of the target center coordinates relative to the center of the camera's field of view, i.e., the miss distance, based on the target center coordinates detected by the target detection module.

[0011] The turntable adjustment module drives the turntable where the camera is located to adjust its direction and angle in real time based on the miss distance information obtained by the deviation calculation module, so as to ensure that the target position is always in the best field of view of the camera.

[0012] The feedback control module is used to continuously adjust and optimize the control parameters of the turntable according to the position information of the target in the current field of view during the process of the camera aligning with the target, until the target is located in the center of the camera's field of view and maximizes the coverage of the entire field of view area; when the camera completes accurate alignment with the target, the feedback control module will output a completion signal.

[0013] This invention also provides a deep learning-based intelligent target finding method, which is implemented by the following steps:

[0014] Step 1: Acquire high-resolution video frame images transmitted in real time by the camera through the data acquisition module;

[0015] Step 2: After preprocessing the high-resolution video frame image obtained in Step 1, a low-resolution video frame image is obtained, and the RGB image is converted to a grayscale image.

[0016] Step 3: Use a CNN network model as a feature extraction module to extract features from the low-resolution video frame image, extract the edge, texture and shape features of the target image, and obtain the target feature map;

[0017] Step 4: Use the target detection module to identify target positions;

[0018] The target position is detected in the target feature map. After the target identification is completed, the target center coordinates are transmitted to the deviation calculation module. The deviation calculation module calculates the offset between the camera field of view center and the target center coordinates, i.e. the miss distance, based on the target center coordinates.

[0019] Step 5: The deviation calculation module transmits the miss amount information to the turntable adjustment module. The turntable adjustment module drives the turntable where the camera is located to adjust its direction and angle in real time according to the miss amount information, so as to ensure that the target position is always in the best field of view of the camera.

[0020] During the process of aligning the camera with the target, the feedback control module continuously adjusts and optimizes the control parameters of the turntable based on the position information of the target in the current field of view until the target is located in the center of the camera's field of view and maximizes the coverage of the entire field of view area; when the camera completes accurate alignment with the target, the feedback control module will output a completion signal.

[0021] The beneficial effects of this invention are:

[0022] I. Intelligent Automatic Alignment. The intelligent target-finding method described in this invention can achieve automated alignment in complex scenes, replacing traditional manual calibration and mechanical adjustment. Through the multi-level feature extraction capabilities of CNN, the system can effectively identify the target position under different lighting, angles, and occlusion conditions, significantly improving target-finding accuracy.

[0023] II. Real-time Feedback and Precise Control. By introducing a deviation calculation and feedback control module, this invention can adjust the camera's field of view in real time, ensuring the target position remains centered. The PID control algorithm ensures a smooth response from the turntable during adjustment, reducing jitter and unnecessary waiting time, thus guaranteeing the system's high efficiency and real-time performance.

[0024] III. Application of Lightweight Detection Model. This invention employs a lightweight target detection model, such as YOLO or MobileNet-SSD, ensuring high detection speed and making the system suitable for embedded platforms or low-power edge computing devices. Furthermore, the lightweight model design reduces system computational overhead, supporting real-time target finding at high frame rates.

[0025] IV. Modular Design, High Applicability. The intelligent target-finding method of this invention, through its modular design, can be flexibly applied to different types of multi-target, multi-camera layouts. Each module operates independently yet collaborates with others. Through a complete process of data acquisition, target detection, deviation calculation, and feedback control, it can be widely applied in scenarios such as military shooting training, security monitoring, and automated target reporting systems.

[0026] V. High Adaptability and Robustness. This invention, through the combination of CNN and PID feedback control, not only improves the system's target-finding accuracy but also enhances its robustness in complex environments. Even under varying ambient lighting conditions, target occlusion, or changes in the field of view, the system can still effectively achieve automatic target alignment and real-time tracking.

[0027] In summary, the intelligent target-finding method described in this invention, through multi-module collaboration, integrates data acquisition, deep learning detection, precise turntable control, and feedback adjustment, achieving intelligent target-finding for multi-target, multi-camera systems. This method significantly improves target-finding speed and efficiency while meeting accuracy requirements, possessing broad practical value and promising application prospects. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of a deep learning-based intelligent target-finding system according to the present invention.

[0029] Figure 2 This is a schematic diagram of the layout of a deep learning-based intelligent target-finding system according to the present invention.

[0030] Figure 3 This is a schematic diagram of the network structure of the feature extraction module and the object detection module;

[0031] Figure 4 This is a schematic diagram of the deviation calculation module;

[0032] Figure 5 This is a flowchart of the feedback control module;

[0033] Figure 6 This is a flowchart of a deep learning-based intelligent target finding method according to the present invention;

[0034] Figure 7 This is a schematic diagram illustrating the target location identification under different environmental conditions using the deep learning-based intelligent target-finding system described in this invention. Detailed Implementation

[0035] Specific Implementation Method 1: Combination Figures 1 to 5This embodiment describes a deep learning-based intelligent target-finding system. The system includes an automatic target-finding device with multiple target positions, multiple cameras (data acquisition modules), and a turntable assembly. The spatial layout of the automatic target-finding device and the multiple target positions is as follows: Figure 2 As shown, the humanoid target and digital display are arranged vertically. The longitudinal distance between the automatic target-finding device and the target supports 10-15m, and the interval between multiple targets supports 2-4m. It also supports the expansion of the number of custom targets, demonstrating the overall layout of a multi-camera, multi-target automated target-finding system. This implementation uses deep learning and feedback control to achieve high-precision, high-real-time automatic target alignment. The automatic target-finding device communicates with the industrial control computer via a switch; the industrial control computer integrates a feature extraction module, a target detection module, a deviation calculation module, and a turntable control module (turntable adjustment module and feedback control module).

[0036] The data acquisition module is used to acquire real-time video frame images from each camera and transmit the real-time image frames to the industrial control computer system according to the set frame rate. The acquired image frames will directly enter the feature extraction module.

[0037] In this embodiment, the camera is mounted on a turntable, allowing for free adjustment of its direction and angle to adapt to different target positions. To improve the stability and adaptability of data acquisition, this embodiment preferably uses a high-sensitivity CMOS or CCD sensor camera to ensure clear target image capture even in complex environments such as low light or strong light, guaranteeing image quality. The acquired video frame images are directly transmitted to subsequent modules to ensure high real-time performance of the system. The data acquisition module is set to a certain frame rate to ensure that the acquired video frames have a sufficient refresh rate for subsequent feature extraction and target detection.

[0038] The feature extraction module uses a convolutional neural network (CNN) to perform multi-level feature extraction on video frame images captured by the camera. First, the video frame images undergo multiple convolutional and activation function processing to extract low-level edge and contour features; then, pooling layers are used to reduce the feature dimensionality and increase the model's computational efficiency. For example... Figure 3 As shown, the feature extraction module contains multiple convolutional and pooling layers, capable of extracting edge, texture, and shape features from the target image. The baseline model used in the target-finding system is YOLOv10. Based on this, customized improvements and modifications were made to suit the device's operating environment and target characteristics. These include the WTB module and the CorrdAtt module. The WTB module is based on wavelet transform convolution operations, which extracts target features more precisely and improves detection accuracy without increasing model complexity. For detailed introductions to other modules, such as the Conv module and the C2f module, please refer to the original YOLOv10 author paper (Link: https: / / arxiv.org / abs / 2405.14458 (Details omitted here.) This feature extraction module can adapt to different lighting, angles, and occlusion conditions, ensuring accurate target identification even in complex backgrounds. Compared to traditional manual feature extraction methods, the CNN feature extraction method used in this module has stronger adaptability and generalization ability, enabling accurate target identification in diverse environments and greatly improving the robustness and accuracy of the system.

[0039] The target detection module uses a deep learning-based target detection algorithm to identify target locations and obtain their coordinates in the image. By combining a bounding box regression algorithm and loss function convergence, the target detection module can accurately predict the bounding box position of the target location in the image. After completing target identification, the detection results (x, y, K) are transmitted to the bias calculation module.

[0040] like Figure 3 As shown, in this embodiment, the target detection module employs a lightweight deep learning model, such as YOLO or MobileNet-SSD, to ensure a balance between detection accuracy and real-time performance. This module utilizes feature maps extracted by CNNs to predict the target's coordinates through bounding box regression. Specifically, this includes input image preprocessing, model forward propagation, bounding box generation, and coordinate extraction. During forward propagation, the target detection module filters background noise based on a set threshold, extracting only high-confidence target regions. The detected target center coordinates and bounding box coordinates are transmitted to the deviation calculation module to calculate the target's offset relative to the camera's field of view center.

[0041] like Figure 4 As shown, Figure 4 This method represents the position of the target position within the camera's field of view. Using the camera's field of view as the origin (0,0) of a two-dimensional coordinate system, with the X-axis ranging from (-0.5, 0.5) and the Y-axis ranging from (-0.5, 0.5), the position of the target center relative to the center of the camera's field of view is represented by this two-dimensional coordinate system. The detection output result is (x, y, K). In the figure, point A has coordinates (x, y), and K = S. 靶标 / S 视场 The coordinates of point A represent the position of the target center in the image within the coordinate system established as shown in the figure. Meanwhile, x and y represent the horizontal and vertical deviation values, respectively. The target-finding task requires the target to occupy 8 / 9 of the field of view, so the K value represents the proportion of the target to the field of view, which is expressed as the ratio of the target area to the field of view area.

[0042] The deviation calculation module calculates the deviation value of the result (x, y, K) detected by the target detection module;

[0043] Based on the target center coordinates provided by the target detection module and the center position of the camera's field of view, this module calculates the horizontal and vertical deviations in real time, while simultaneously recording the area of ​​the target in the image. In other words, the deviation calculation module calculates the offset between the camera's field of view center and the target center, i.e., the miss distance (detection output result (x, y, K)). The miss distance calculation result contains the precise positional offset information of the target relative to the camera, serving as input parameters for the turntable adjustment module. Through precise deviation calculation, this module provides crucial control parameters for subsequent turntable adjustments, ensuring that the target gradually moves to the center of the camera's field of view and achieves maximum coverage.

[0044] The turntable adjustment module aligns the camera with the target based on the miss distance information transmitted by the deviation calculation module. In this embodiment, the control strategy for the turntable adjustment adopts a PID (Proportional-Integral-Derivative) control algorithm. The PID control algorithm enables the camera to move quickly and smoothly to the center of the target position through proportional, integral, and derivative control. Specifically, proportional control is responsible for adjusting the current deviation, integral control is used to accumulate historical deviations, and derivative control suppresses overshoot during the adjustment process. The turntable adjustment module continuously monitors and corrects the motion state of the turntable to avoid jitter caused by over-adjustment, and gradually reduces the adjustment amplitude when the target position is close to the center of the field of view, ensuring that the camera is ultimately accurately aligned with the target. The PID control algorithm ensures that the camera quickly and accurately aligns with the target position by adjusting control parameters (such as the turntable's rotation speed and angle) in real time, achieving the best target-finding effect. The introduction of the PID control algorithm guarantees the system's high response speed and adjustment accuracy.

[0045] The feedback control module is used to continuously adjust and optimize the control parameters of the turntable during the camera's alignment with the target, based on the target's position information in the current field of view, until the target is located in the center of the camera's field of view and maximizes coverage of the entire field of view. Once the camera has accurately aligned with the target, the feedback control module will output a completion signal to mark the completion status of the current target in the multi-target system and proceed to the target-finding process for the next target.

[0046] In this embodiment, the feedback control process ensures that the target position remains centered in the camera's field of view during the target-finding task. The feedback control module plays a real-time adjustment role throughout the entire target-finding process. Specifically, the feedback control module receives status feedback from the turntable adjustment module and monitors changes in the target's position within the camera's field of view. If the target position is not centered or does not maximize field of view coverage, the feedback control module dynamically updates control parameters to further optimize the turntable adjustment. When the camera finally aligns with the target and ensures the target is centered and maximizes coverage, the feedback control module outputs a completion signal, marking the completion of the current target-finding task, and the system automatically transitions to the next target-finding stage. This module ensures that the system can continuously and stably achieve high-precision target-finding in dynamic and complex environments, exhibiting excellent performance, especially in complex scenes with multiple targets and multiple cameras.

[0047] like Figure 5 As shown, the workflow of the feedback control module is as follows:

[0048] Step A: Set the target miss distance X for the (N-1)th time. n-1 The rotation amount Q of the turntable in the (N-1)th iteration n-1 ;

[0049] Step B, the target miss distance for the Nth time is X. n The angle of rotation of the turntable for the Nth time is: Q n= X n* Q n-1 / (X n -X n-1 );

[0050] Step C, the target miss distance for the (N+1)th time is X. n+1 ;

[0051] Step D: The feedback control module determines whether the miss distance meets the requirements, that is: if ,and ,and If the target is found, the search is complete; otherwise, N = N + 1, and return to step A.

[0052] like Figure 7 As shown, the target-finding system described in this embodiment is applied to target identification under different environmental conditions. The test results for the three images, under different test backgrounds, demonstrate that the system can stably identify and locate the target, directly showcasing the system's excellent robustness and strong adaptability.

[0053] Specific Implementation Method Two: Combination Figure 6 and Figure 7This embodiment describes a method for intelligent target finding using a deep learning-based intelligent target finding system as described in Specific Embodiment 1. This method utilizes a deep learning model to automatically identify and locate the target position, and combines this with a turntable control system to achieve precise alignment and maximize field-of-view coverage. The specific process of this method is as follows:

[0054] Step 1: Acquire high-resolution video frame images transmitted in real time by the camera through the data acquisition module;

[0055] Step 2: To improve the computation speed of the neural network, the high-resolution video frame image obtained in Step 1 is preprocessed to obtain a low-resolution video frame image, and the RGB image is converted to a grayscale image.

[0056] Step 3: Use the feature extraction module to extract features from the low-resolution video frame image;

[0057] In this embodiment, a convolutional neural network (CNN) is used to perform multi-level feature extraction on the image, which can extract the edge, texture and shape features of the target image to obtain the target feature map;

[0058] Step 4: Use the target detection module to detect the coordinate position of the target in the image. After the target position recognition is completed, the detection result is transmitted to the deviation calculation module. The deviation calculation module calculates the offset between the center of the camera field of view and the center of the target position based on the target position coordinates, i.e. the miss distance.

[0059] Then, based on the miss distance information, the turntable adjustment module is controlled in real time. The turntable adjustment module drives the two-dimensional turntable mounted on the target camera to adjust the horizontal and vertical orientation in real time, ensuring that the target position is always in the optimal field of view of the camera.

[0060] Step 5: The motion result verification module calculates whether the target center position and the field of view occupied by the target meet the set requirements based on the deviation value after the turntable adjustment, and performs feedback control based on the judgment result. If it does not meet the requirements, the target finding step is re-executed from step 1; if it meets the requirements, the target finding is completed.

[0061] Step 6: After the target search is completed, stop the target search and light up a green light to indicate that the target search is complete.

[0062] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0063] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A deep learning based intelligent targeting system, characterized in that: The system includes a data acquisition module, a feature extraction module, a target detection module, a deviation calculation module, a turntable adjustment module, and a feedback control module; The data acquisition module is used to acquire real-time video frame images from each camera and transmit them to the feature extraction module; The feature extraction module performs feature extraction on the video frame image to obtain the edge, texture and shape features of the target image and obtain the target feature map. The target detection module uses a deep learning-based target detection algorithm to identify the target feature map and obtain the coordinate position of the target in the feature map, i.e., the target center position; and transmits the target center position to the deviation calculation module. The deviation calculation module calculates the deviation value of the target center coordinates relative to the center of the camera's field of view, i.e., the miss distance, based on the target center coordinates detected by the target detection module. The deviation calculation module calculates the deviation values ​​in the horizontal and vertical directions in real time based on the target center coordinates sent by the target detection module and the center position of the camera's field of view, and simultaneously records the area of ​​the target in the image. The turntable adjustment module drives the turntable where the camera is located to adjust its direction and angle in real time based on the miss distance information obtained by the deviation calculation module, so as to ensure that the target position is always in the best field of view of the camera. The feedback control module is used to continuously adjust and optimize the control parameters of the turntable according to the position information of the target in the current field of view during the process of the camera aiming at the target, until the target is located in the center of the camera's field of view and maximizes the coverage of the entire field of view area. Once the camera has accurately aligned with the target, the feedback control module will output a completion signal. The specific steps of the intelligent target-finding method are as follows: Step 1: Acquire high-resolution video frame images transmitted in real time by the camera through the data acquisition module; Step 2: After preprocessing the high-resolution video frame image obtained in Step 1, a low-resolution video frame image is obtained, and the RGB image is converted to a grayscale image. Step 3: Use a CNN network model as a feature extraction module to extract features from the low-resolution video frame image, extract the edge, texture and shape features of the target image, and obtain the target feature map; Step 4: Use the target detection module to identify target positions; The target position is detected in the target feature map. After the target identification is completed, the target center coordinates are transmitted to the deviation calculation module. The deviation calculation module calculates the offset between the camera field of view center and the target center coordinates, i.e. the miss distance, based on the target center coordinates. Step 5: The deviation calculation module transmits the miss amount information to the turntable adjustment module. The turntable adjustment module drives the turntable where the camera is located to adjust its direction and angle in real time according to the miss amount information, so as to ensure that the target position is always in the best field of view of the camera. During the process of the camera aiming at the target, the feedback control module continuously adjusts and optimizes the control parameters of the turntable based on the position information of the target in the current field of view, until the target is located in the center of the camera's field of view and maximizes the coverage of the entire field of view area. Once the camera has accurately aligned with the target, the feedback control module will output a completion signal. The workflow of the feedback control module is as follows: Step A, set the Nth target off-target amount X n-1 , the Nth turntable rotation amount Q n-1 ; Step B, the Nth target off-target amount is X n , the Nth turntable rotation angle is: Q n= X n* Q n-1 / (X n -X n-1 ); Step C, the N+1st target off-target quantity is X n+1 ; Step D, the feedback control module judges whether the off-target amount (x, y, K) meets the requirements, that is: if , and , and , K=S 靶标 / S 视场 ; where x and y represent the horizontal and vertical deviation values, respectively; S 靶标 S is the target area, S 视场 S is the camera field of view area; if yes, the targeting is complete; otherwise, n = n + 1, return to step A. 2.The intelligent targeting system based on deep learning of claim 1, wherein: The camera is mounted on a turntable and its direction and angle can be freely adjusted to adapt to the position of different targets. 3.The intelligent targeting system based on deep learning of claim 1, wherein: The feature extraction module uses a CNN network model and extracts edge, texture and shape features of the target image through multiple convolutional layers, pooling layers and normalization layers. 4.The intelligent targeting system based on deep learning of claim 1, wherein: The target detection module uses a lightweight deep learning model. It uses the target feature map extracted by the feature extraction module, and predicts the target center coordinates and bounding box coordinates through bounding box regression algorithm and loss function convergence, and then transmits them to the deviation calculation module. 5.The intelligent targeting system based on deep learning of claim 1, wherein: The turntable adjustment module uses a PID-based control algorithm to adjust control parameters in real time so that the camera is aligned with the center of the target. 6.The intelligent targeting system based on deep learning of claim 1, wherein: The feedback control module receives status feedback from the turntable adjustment module and monitors the position changes of the target in the camera's field of view. If the target does not reach the center of the field of view or does not maximize the coverage of the field of view, the feedback control module dynamically updates the control parameters to further optimize the turntable adjustment. When the camera finally aligns with the target and ensures that the target is in the center of the field of view and maximizes the coverage, the feedback control module outputs a completion signal to mark the completion of the target-finding task at the current target position, and the system automatically switches to the target-finding process at the next target position.

Images