A small target recognition method under large field of view for a small unmanned optoelectronic reconnaissance system

Abstract

This invention belongs to the field of detection and identification technology for optoelectronic reconnaissance systems and discloses a method for identifying small targets within a large field of view for a small unmanned optoelectronic reconnaissance system. This method places the target identification function of the small unmanned optoelectronic reconnaissance system on a high-performance ground station. On the ground, a YOLOv3 model trained using a SMOTE-optimized dataset is used to identify the target. This model is then used to calculate the target position and upload the data to the sky-side optoelectronic payload. This method effectively improves the recognition accuracy, precision, and speed of small targets within a large field of view. Furthermore, combined with improved target positioning accuracy, it addresses the difficulty of identifying and detecting small targets within a large field of view.

Description

Technical Field

[0001] The present invention belongs to the technical field of detection and identification of optoelectronic reconnaissance systems, and relates to a method for identifying small targets in a large field of view of a small unmanned optoelectronic reconnaissance system. Background Art

[0002] Drones (UAVs) are typically unmanned aerial vehicles controlled by radio remote control and self-contained programmable controls. Compared to manned aircraft, UAVs are often better suited for "dirty" or "dangerous" missions, leading to their widespread application in disaster relief, infectious disease monitoring, power inspections, and wildlife conservation. In practice, operators can leverage the UAV's onboard optoelectronic reconnaissance system, employing target recognition and tracking technology, to continuously lock onto and monitor selected targets, predicting their location and motion, and guiding subsequent decisions.

[0003] With the continuous advancement of technology, the diversification of application scenarios, and the emergence of new user demands, drones and their optoelectronic payloads are facing increasingly stringent performance requirements. Although drones currently come in a wide variety of types and sizes, small drones are still the most widely used, as they are economical and portable. However, this comes with the difficulty of integrating high-performance target detection and recognition modules, which are large, heavy, and power-intensive. This, in turn, restricts the real-time operation of high-performance algorithms in the airborne part of small unmanned optoelectronic reconnaissance systems, resulting in a decline in their overall recognition performance. Consequently, small unmanned optoelectronic reconnaissance systems have very limited ability to recognize small targets within a large field of view. Summary of the Invention

[0004] (1) Purpose of the invention

[0005] The purpose of the present invention is to provide a method for identifying small targets in a large field of view of a small unmanned reconnaissance system in combination with a ground station, so as to effectively solve the effective identification requirements of small unmanned optoelectronic reconnaissance systems for real-time solution and processing of small targets in a large field of view at the sky end.

[0006] (2) Technical solution

[0007] In order to solve the above technical problems, the present invention provides a small target recognition method under a large field of view of a small unmanned optoelectronic reconnaissance system, which comprises the following steps:

[0008] Step 1: The operator controls the flight control module to fly the drone to the reconnaissance area and switches the television sensor in the onboard electro-optical reconnaissance system to wide field of view mode to obtain images and videos of the largest range of the target area. The operator also obtains and records the pitch and azimuth angle information of the television sensor, as well as the drone's GPS and IMU information.

[0009] Step 2: Transmit the information, video, and images acquired in step 1 to the ground station via wireless image and data transmission links;

[0010] Step 3: The operator uses the trained YOLOv3 model at the ground station to analyze the video and images sent back by the TV sensor to identify and locate suspected small targets.

[0011] Step 4: The operator at the ground station combines the TV sensor information transmitted by the sky end with the position information of the UAV to calculate the position of the selected small target;

[0012] Step 5: Upload the suspected small target identified in step 3 and the small target location information obtained in step 4 to the sky terminal via image and data transmission to achieve identification and locking of the small target under a large field of view. Then switch the TV sensor to a small field of view to achieve identification and detection of the suspected small target.

[0013] (3) Beneficial effects

[0014] The above-mentioned technical solution provides a method for identifying small targets in a large field of view of a small unmanned optoelectronic reconnaissance system. By placing the target recognition of the small unmanned optoelectronic reconnaissance system on a high-performance ground station, the target is identified on the ground using the YOLOv3 model trained after optimizing the data set through SMOTE. The target is then uploaded to the optoelectronic payload on the sky side in conjunction with the target position solution, effectively improving the recognition accuracy, recognition precision and recognition speed of small targets in a large field of view. At the same time, combined with the improvement of the target position positioning accuracy, the problem of difficult recognition and detection of small targets in a large field of view is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a system structure block diagram of the present invention;

[0016] Figure 2 is a flow chart of the method of the present invention;

[0017] Figure 3 and Figure 4 This is a schematic diagram of the SMTOTE algorithm of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below with reference to the accompanying drawings and examples.

[0019] This embodiment first provides a small-scale unmanned optoelectronic reconnaissance system with a large field of view and a small target recognition system. Figure 1 As shown, the system includes a sky-side optoelectronic payload, an embedded microprocessor, an image and data transmitter, a flight controller, a ground station module including a workstation and display and control software for operation.

[0020] The drone of the present invention is a micro-drone in a small unmanned optoelectronic reconnaissance system. The AI ​​computing module loaded into the ground station is a dedicated workstation, which calls the weight file trained by the YOLOv3 model on the workstation to detect and identify targets.

[0021] The small target recognition method under a large field of view of a small unmanned optoelectronic reconnaissance system in this embodiment can be used to identify a specific target. The acquired image is processed in advance by the SMOTE algorithm, and the YOLOv3 model is trained to obtain a weight file. The weight file is called to realize the recognition of the specific target. Figure 2 As shown, the method includes the following steps:

[0022] Step 1: The operator controls the flight control module to fly the drone to the reconnaissance area and switches the television sensor in the onboard electro-optical reconnaissance system to wide field of view mode to obtain images and videos of the largest possible range of the target area. The operator also obtains and records the pitch and azimuth angles of the television sensor, as well as the drone's GPS and IMU information. The acquired images and videos are pre-processed, including basic operations such as image distortion correction and video encoding.

[0023] Step 2: Transmit the information obtained in step 1 and the pre-processed video images to the ground station via wireless image transmission and data transmission data link;

[0024] Step 3: The operator uses the trained YOLOv3 model at the ground station to analyze the video images sent back by the TV sensor and identify and locate suspected small targets.

[0025] Step 4: The operator at the ground station combines the TV sensor information transmitted by the sky unit with the position information of the UAV to calculate the position of the selected small target; the position information includes the pitch angle and azimuth angle of the TV sensor and the GPS and IMU information of the UAV;

[0026] Step 5: Upload the suspected small target identified in step 3 and the small target position information obtained in step 4 to the sky end through image transmission and digital transmission, so as to realize the recognition and locking of the small target in a large field of view. Then switch the TV sensor to a small field of view to realize the identification and detection of the suspected small target.

[0027] In steps 1 and 2, the optoelectronic reconnaissance system located in the sky can also switch between visible light and infrared videos according to different scenarios, and compress the videos using H.265 compression technology, and transmit them to the ground end to obtain more than 1,000 images of material.

[0028] In step 3, a data set is created in advance and a YOLOv3 model is trained so that the ground station can quickly and accurately complete real-time detection and recognition of suspected small targets. The process of data set creation, YOLOv3 model training, and target recognition is as follows:

[0029] 1) Obtain videos captured by the sky-side electro-optical reconnaissance system, intercept the videos, and obtain more than 1,000 target images as a dataset;

[0030] 2) Use the SMOTE algorithm to extract and analyze the suspected target features, synthesize the feature information obtained by analysis into new samples in an artificial way, and then insert it into the data set. The algorithm diagram is shown in the figure below. Figure 3 and Figure 4 As shown, the algorithm flow is as follows:

[0031] For a specific minority class sample i, use x i , i∈{1,2,......,T}, represents its eigenvector, and the distance between sample i and each minority class sample point is statistically calculated using methods such as Euclidean distance, and the k closest sample points are found in all the statistical data, and x is used to represent the distance between sample i and each minority class sample point. i(near) ,near∈{1,2,..,T} represents the nearest neighbor sample point.

[0032] Calculate the ratio of large and small samples in the data set, set the sampling rate N according to the calculated value, and then randomly select a sample point. This sample point comes from k neighboring sample points. The generated random number ξ ranges from 0 to 1. Use the following formula to insert the new sample point, x i1 That is, the new image generated.

[0033] x i1 =x i +ξ*(x i(nn) -x i )

[0034] 3) After optimizing the dataset using the SMOTE algorithm, some necessary image processing is performed on the images in the dataset, including blurring, filtering, and brightness linear transformation;

[0035] 4) Add interference noise and perform Gaussian filtering on the image processed in the previous step;

[0036] 5) Adjust the image resolution obtained in the previous step to 416×416;

[0037] 6) Train the YOLOv3 model, including the following:

[0038] The data is trained using the most classic YOLOv3 model and data set enhancement provided by YOLO. By analyzing the morphology of typical targets in the data set, the data enhancement methods used in this model training include: small-angle rotation, flipping, random fusion, grid masking, random ratio combination of multiple images, and mosaic construction. After training, the corresponding weight file yolov3XXX.weights of the model is obtained;

[0039] 7) Call the weight file yolov3XXX.weights in the display and control software to detect and identify suspected small targets in the current input image and obtain their location in the current image.

[0040] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for identifying small targets in a large field of view of a small unmanned optoelectronic reconnaissance system, characterized in that: The following steps are involved: Step 1: The operator controls the flight control module to fly the drone to the reconnaissance area and switches the television sensor in the onboard electro-optical reconnaissance system to wide field of view mode to obtain images and videos of the largest range of the target area. The operator also obtains and records the pitch and azimuth angle information of the television sensor, as well as the drone's GPS and IMU information. Step 2: Transmit the information, video, and images acquired in step 1 to the ground station via wireless image and data transmission links; Step 3: The operator uses the trained YOLOv3 model at the ground station to analyze the video and images sent back by the TV sensor to identify and locate suspected small targets. Step 4: The operator at the ground station combines the TV sensor information transmitted by the sky end with the position information of the UAV to calculate the position of the selected small target; Step 5: Upload the suspected small target identified in step 3 and the small target location information obtained in step 4 to the sky terminal via image and data transmission to achieve identification and locking of the small target under a large field of view. Then switch the TV sensor to a small field of view to achieve identification and detection of the suspected small target.

2. The method for identifying small targets in a large field of view of a small unmanned optoelectronic reconnaissance system according to claim 1, characterized in that: Step 1 also includes preprocessing the acquired images and videos, including image distortion correction and video coding compression operations.

3. The method for identifying small targets in a large field of view of a small unmanned optoelectronic reconnaissance system according to claim 2, characterized in that: In step 1, when obtaining the largest range of images of the target area, the optoelectronic reconnaissance system switches between visible light and infrared videos according to different scenarios, compresses the videos using H.265 compression technology, and transmits them to the ground end, obtaining more than 1,000 images.

4. The method for identifying small targets in a large field of view of a small unmanned optoelectronic reconnaissance system according to claim 3, characterized in that: In step 3, when identifying and locating suspected small targets, a data set is prepared in advance, and the YOLOv3 model is trained, and then the real-time detection and identification of suspected small targets is completed.

5. The method for identifying small targets in a large field of view of a small unmanned optoelectronic reconnaissance system according to claim 4, characterized in that: In step 3, the process of data set preparation is as follows: obtain the video shot by the optoelectronic reconnaissance system on the sky side, intercept the video, and obtain more than 1,000 target images as the data set.

6. The method for identifying small targets in a large field of view of a small unmanned optoelectronic reconnaissance system according to claim 5, characterized in that: In step 3, after obtaining the data set, the SMOTE algorithm is used to extract and analyze the suspected target features, and the feature information obtained from the analysis is artificially synthesized into new samples, which are then inserted into the data set.

7. The method for identifying small targets in a large field of view of a small unmanned optoelectronic reconnaissance system according to claim 6, characterized in that: In step 3, the SMOTE algorithm process is as follows: For the minority class sample i, use x i ,i∈{1,2,......,T}, represents its eigenvector, The distance between sample i and each minority class sample point is counted using the Euclidean distance, and the k closest sample points are found in all the statistical data, and x is used to calculate the distance between sample i and each minority class sample point. i(near) ,near∈{1,2,..,T} represents the nearest neighbor sample point; Calculate the ratio of large and small samples in the data set, set the sampling rate N according to the calculated value, and then randomly select a sample point. This sample point comes from k neighboring sample points. The generated random number ξ ranges from 0 to 1. Use the following formula to insert the new sample point, x i1 That is, the new image generated; x i1 =x i +ξ*(x i(nn) -x i )。 8. The method for identifying small targets in a large field of view of a small unmanned optoelectronic reconnaissance system according to claim 7, characterized in that: In step 3, after optimizing the dataset using the SMOTE algorithm, the images in the dataset are blurred, filtered, and subjected to linear brightness transformation. Interference noise is added to the processed images and Gaussian filtering is performed. The image resolution is adjusted to 416×416.

9. The method for identifying small targets in a large field of view of a small unmanned optoelectronic reconnaissance system according to claim 8, characterized in that: In step 3, the YOLOv3 model training process is: The data was trained using the YOLOv3 model and dataset enhancement provided by YOLO. By analyzing the morphology of typical targets in the dataset, the data enhancement methods used in this model training include: small-angle rotation, flipping, random fusion, grid masking, random ratio combination of multiple images, and mosaic construction. After training, the corresponding weight file yolov3XXX.weights was obtained. Call the weight file yolov3XXX.weights in the display and control software to detect and identify suspected small targets in the current input image and obtain their location in the current image.

10. Application of a small target recognition method in a large field of view of a small unmanned optoelectronic reconnaissance system according to any one of claims 1 to 9 in the field of detection and recognition technology of optoelectronic reconnaissance systems.

Images