Armored target and important part detection method based on deep learning and knowledge reasoning

Abstract

The invention discloses an armored target and important part detection method based on deep learning and knowledge reasoning. The method comprises the following steps: constructing a missile-borne image data set; constructing a deep learning detection model and a critical part auxiliary detection unit; wherein the deep learning detection model is used for detecting an armored target and a critical part in the missile-borne image, and outputting a corresponding target detection frame; when the deep learning detection model cannot generate a target detection frame of the critical part under a preset condition, calling a critical part auxiliary detection unit, and outputting a detection frame coordinate of the critical part through the critical part auxiliary detection unit; training and testing the deep learning detection model by using the image data set, and storing the trained deep learning detection model; and acquiring a missile-borne image of a to-be-detected armored target, performing detection by using the trained deep learning detection model and the critical part auxiliary detection unit, and outputting detection results of the armored target and the critical part.

Description

Technical Field

[0001] The present invention belongs to the technical field of missile-borne image intelligent processing, and specifically relates to a method for detecting armored targets and critical parts (key and important parts) of missile-borne images based on deep learning and knowledge reasoning. Background Art

[0002] In the field of precision guidance, missile-borne image target detection is the core key technology for information processing of optical imaging guided weapons. This technology comprehensively utilizes target and background characteristics, missile-borne sensor image information, and related prior knowledge / data, and completes the detection and identification of threat targets through target detection image processing algorithms, thereby providing important target information for weapon systems to implement precision strikes. Taking the application of anti-armor precision guidance as an example, if armored targets and their key parts on the battlefield can be accurately detected, it means "find and strike", which is directly related to the precision strike effectiveness of anti-armor weapon systems in actual combat. Therefore, it is necessary to carry out research on missile-borne image target detection algorithms, optimize and improve the performance of algorithm applications, so as to meet the development of precision guidance technology and field needs.

[0003] The key to target detection algorithms lies in feature extraction. Traditional target detection algorithms, such as feature matching and template matching, mainly rely on manual design to extract predefined features. This approach makes the algorithm itself have strong limitations and is difficult to adapt to different complex scenarios. In recent years, intelligent detection algorithms based on deep learning models have become the mainstream research direction in the field of target detection. Its outstanding feature is to use deep neural networks to automatically / autonomously learn a large number of data samples from different levels and scales to form efficient and adaptive feature representation capabilities, effectively eliminating the subjectivity and tediousness brought by manual design, making the target detection algorithm processing autonomous and intelligent, so as to more robustly cope with different targets and complex scenarios. For the task of missile-borne image target detection, some classic and general deep learning model architectures and intelligent algorithms have been developed and utilized, such as the YOLO series and the RCNN series. From the perspective of practical application, the high dynamic scale, perspective and complex scene change characteristics of missile-borne images, as well as the requirements for refined recognition capabilities of key target parts and background interference, and design constraints such as real-time information processing and limited computing resources, all put forward higher requirements and new challenges for target intelligent detection algorithms. Summary of the invention

[0004] The purpose of the present invention is to provide a method for detecting armored targets and critical parts based on deep learning and knowledge reasoning. By combining a data-driven deep learning target detection model with knowledge reasoning based on the geometric relationship between armored targets and critical parts imaging, refined detection of armored targets and critical parts in missile-borne images can be achieved, thereby providing technical support for precision guidance information processing.

[0005] In order to achieve the above tasks, the present invention adopts the following technical solutions:

[0006] A method for detecting armored targets and key parts based on deep learning and knowledge reasoning, comprising:

[0007] Obtain missile-borne images of armored targets through target scene simulation or flight tests, annotate the armored targets and key parts in the missile-borne images, and thus construct an image dataset;

[0008] Constructing a deep learning detection model and a key part auxiliary detection unit; wherein the deep learning detection model is used to detect armored targets and key parts in the missile-borne image and output corresponding target detection frames; when the deep learning detection model only detects target detection frames for armored targets under preset conditions but cannot generate target detection frames for key parts, the key part auxiliary detection unit is called to output the detection frame coordinates of the key parts through the unit;

[0009] Using the image data set to train and test the deep learning detection model, and saving the trained deep learning detection model;

[0010] Obtain the missile-borne image of the armored target to be detected, use the trained deep learning detection model and the auxiliary detection unit of key parts to perform detection, and output the detection results of the armored target and key parts.

[0011] Furthermore, in the target scene simulation and flight test, after the missile-borne platform captures the ground armored target, a side image sequence of the armored target should be collected from far to near;

[0012] The acquired missile-borne images are preprocessed and cleaned, and the armored targets and key parts are marked to form a data set with label information; the key parts refer to the key and important parts on the armored targets, which are set and marked according to the detection requirements.

[0013] Furthermore, the data set is divided into a training set and a test set in a ratio of 3:1, and it is ensured that the missile-borne images of the two data sets come from different test or simulation scenarios; the model is first trained with the training set, and then tested with the test set. After meeting the requirements, the trained model is saved.

[0014] Furthermore, the deep learning detection model is:

[0015] Based on the existing Faster RCNN, the backbone network ResNet is replaced with Res2Net with a multi-branch structure, and the following distance is used to measure the position relationship of the target bounding box:

[0016]

[0017] Where N a and N b It describes two candidate target detection boxes A = [x a ,y a ,h a , w a ] and B=[x b ,y b ,h b , w b ] Gaussian distribution model, W(N a , N b ) and D(N a , N b ) represents the distance between two Gaussian distribution models and its exponential normalized form, c is a control parameter less than 1, x a ,y a represents the center point coordinates of the candidate target detection box A, w a ,h a represents the width and height of the candidate target detection box A; ||·||2 represents the l2 norm;

[0018] During the detection process, after the deep learning detection model generates multiple candidate target detection frames with different confidence levels for the armored target, the distance D (N a , N b ), then sort all the calculated distances, select the two candidate target detection frames with the largest distance, and use the one with higher confidence as the final target detection frame of this type, so as to obtain target detection frames for armored targets and key parts.

[0019] Furthermore, when the deep learning detection model generates a target detection frame for the armored target but fails to generate a target detection frame for the critical parts, the target detection frame of the armored target is first judged. When the number of pixels in the target detection frame is not less than a preset number of pixels, the critical parts auxiliary detection unit is used to perform detection of the critical parts.

[0020] Furthermore, the preset number of pixels is 400.

[0021] Furthermore, the modeling process of the key part auxiliary detection unit is as follows:

[0022] Assume the physical size of the armored target is l target × target ×h target , the height of the key part is ρh target , ρ is the high and low control parameter, 0<ρ<1; when the missile-carrying platform approaches the armored target from far to near with a lateral perspective, the target detection frame size of the armored target is hbox × box , the center coordinates of the target box detection box are (x box ,y box ); then according to the optical imaging geometry, w box Relatively fixed and target is proportional to h box Can be regarded as h target and w taxget The sum of the object surface projections; assuming the sight angle of the missile-carrying platform is θ, w target With h target The ratio is b, then:

[0023] h box =κ(aw target sinθ+h target cosθ)

[0024] In the formula, k and a are proportional coefficients, k can be obtained from w box / l target Calculated;

[0025] Therefore, the image center coordinates of the key parts are

[0026] An object detection frame of the key part is generated based on the image center coordinates of the key part and output.

[0027] Furthermore, for different critical parts, different values ​​of ρ can be used to locate the critical parts at different positions on the armored target.

[0028] A terminal device includes a processor, a memory, and a computer program stored in the memory; when the processor executes the computer program, the armored target and key part detection method based on deep learning and knowledge reasoning is implemented.

[0029] A computer-readable storage medium stores a computer program; when the computer program is executed by a processor, the armored target and key part detection method based on deep learning and knowledge reasoning is implemented.

[0030] Compared with the prior art, the present invention has the following technical features:

[0031] 1. Based on the improved Faster RCNN deep learning detection model, the present invention realizes visual detection and recognition of armored targets and key parts; adopts the Res2Net backbone network and normalized distance metric to effectively enhance the detection capability of small targets at a long distance.

[0032] 2. According to the optical imaging geometric relationship between the target and the part, the present invention constructs an auxiliary detection unit for the key parts of the target based on knowledge reasoning, further strengthening the detection and reasoning ability of the model for the key parts of the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of the overall process of the method of the present invention. DETAILED DESCRIPTION

[0034] Referring to the accompanying drawings, the present invention provides a method for detecting armored targets and key parts based on deep learning and knowledge reasoning, comprising:

[0035] Obtain missile-borne images of armored targets through target scene simulation or flight tests, annotate the armored targets and key parts in the missile-borne images, and thus construct an image dataset;

[0036] Constructing a deep learning detection model and a key part auxiliary detection unit; wherein the deep learning detection model is used to detect armored targets and key parts in the missile-borne image and output corresponding target detection frames; when the deep learning detection model only detects the detection frame for the armored target under preset conditions and cannot generate the target detection frame for the key part, the key part auxiliary detection unit is called to output the detection frame coordinates of the key part through the unit to assist the deep learning detection model in performing inference detection when the key part is not detected;

[0037] Using the image data set to train and test the deep learning detection model and the key part auxiliary detection unit, and saving the trained deep learning detection model and the key part auxiliary detection unit;

[0038] Obtain the missile-borne image of the armored target to be detected, use the trained deep learning detection model and the auxiliary detection unit of key parts to perform detection, and output the detection results of the armored target and key parts, that is, the target detection frame and center coordinates.

[0039] The specific implementation process of the present invention is further described below.

[0040] 1. Establishment of image dataset

[0041] In this scheme, missile-borne images are obtained through target scene simulation, flight tests, etc.; wherein, in the target scene simulation and flight tests, after the missile-borne platform captures a ground armored target, a side image sequence of the armored target should be collected from far to near; that is, the collection scene is that the missile-borne platform is above the side of the armored target and approaches the armored target from far to near, and in order to make the features of the armored target in the image richer, the side image of the armored target should be collected.

[0042] The acquired missile-borne images are preprocessed and cleaned, and the armored targets and critical parts are marked to form a data set with label information; in this solution, the critical parts refer to key and important parts, which are set and marked according to the detection requirements.

[0043] In order to evaluate the model detection effect, this scheme divides the data set into a training set and a test set in a ratio of 3:1, and ensures that the missile-borne images of the two data sets come from different test or simulation scenarios; the model is first trained with the training set, and then tested with the test set to meet the requirements, and then the trained model is saved.

[0044] 2. Deep Learning Detection Model

[0045] In this solution, the improved Faster RCNN is used as the deep learning detection model, specifically:

[0046] Based on the existing Faster RCNN, this solution replaces the backbone network ResNet with Res2Net with a multi-branch structure to enhance the network's ability to express and extract multi-scale features and small targets; at the same time, since the original IOU measurement method of the model has unstable defects in the small target detection process, this solution uses the following distance to measure the position relationship of the target bounding box to improve the measurement accuracy and robustness of small target detection. The specific formula is as follows:

[0047]

[0048] Where N a and N b It describes two candidate target detection boxes A = [x a ,y a ,h a , w a ] and B = [x b ,y b ,h b , w b ] Gaussian distribution model, W(N a , N b ) and D(N a , N b ) represents the distance between two Gaussian distribution models and its exponential normalized form, c is a control parameter less than 1, x a ,y a represents the center point coordinates of the candidate target detection box A, w a ,h a represents the width and height of the candidate target detection box A; ||·||2 represents the l2 norm;

[0049] During the detection process, after the deep learning detection model generates multiple candidate target detection frames with different confidence levels for the armored target, the distance D (N) between each two candidate target detection frames A and B is calculated for all candidate target detection frames of each type (such as armored target, critical part 1, critical part 2, etc. in this embodiment). a , N b ), then sort all the calculated distances, select the two candidate target detection frames with the largest distance, and use the one with higher confidence as the final target detection frame of this type.

[0050] Through the above method, target detection frames for armored targets and critical parts can be obtained.

[0051] Since the armored target is relatively large as a whole, and the critical part is a part on the armored target, its size in the image is relatively small; in actual applications, since image acquisition for the armored target is continuous, when the distance to the armored target is far, the critical part cannot generate a corresponding target detection frame because the corresponding pixel information in the target detection frame of the armored target is too little; as the missile-borne platform continues to approach the armored target and continuously acquires images, when the target detection frame for the armored target is not less than 400 pixels, if there is a situation where the deep learning detection model cannot detect the critical part, the critical part auxiliary detection unit is used to assist the deep learning detection model in performing inference detection when the critical part is not detected.

[0052] 3. Auxiliary detection unit for key parts

[0053] When the deep learning detection model generates a target detection frame for the armored target, but does not generate a target detection frame for the key parts, the target detection frame of the armored target is first judged. The preset condition in this scheme means that when the pixel point in the target detection frame is not less than the preset pixel value of 400 (that is, the size of the detection frame is not less than 20*20), according to the optical imaging geometry relationship between the target and the part and the length and width variation law of the target frame, the image coordinate calculation model of the key parts of the armored target based on the imaging geometry knowledge under the lateral perspective is constructed, and the image coordinates of the key parts are solved from the detected target information. The specific modeling and calculation process is as follows:

[0054] Assume the physical size of the armored target is l target × target ×h target (length × width × height), the height occupied by the key parts is ρh target , ρ is the high and low control parameter, 0<ρ<1; when the missile-carrying platform approaches the armored target from far to near with a lateral perspective, the target detection frame size of the armored target is h box × box (height × width), the center coordinate of the target frame detection frame is (xbox ,y box ); then according to the optical imaging geometry, w box Relatively fixed and target is proportional to h box Can be regarded as h target and w target The sum of the object surface projections; assuming the sight angle of the missile-carrying platform is θ, w target With h target The ratio is b, then:

[0055] h box =κ(aw target sinθ+h target cosθ)

[0056] In the formula, k and a are proportional coefficients, k can be obtained from w box / l target It is calculated that a can be simplified to 1;

[0057] Therefore, the image center coordinates of the key parts are It should be noted that when θ is greater than a certain value θ th After that, the target key part will disappear from the field of view.

[0058] An object detection frame of the key part is generated based on the image center coordinates of the key part and output.

[0059] In practical applications, different values ​​of ρ can be used to locate different critical parts on the armored target.

[0060] Example:

[0061] (1) Experimental conditions and related settings

[0062] In this embodiment, a total of 1670 images of armored targets and key parts data sets are constructed, of which 1206 images are used as training samples and 464 images are used as test samples. Set the training parameters: set the learning rate to 0.005, the optimizer to ADAM, the number of training steps to 2000, the number of training rounds to 24, the batch training size to 4, and use the classification model parameters obtained by pre-training the backbone network Res2Net on the imagenet data set as the initialization parameters. The benchmark method selects the YOLOv5 and FasterRCNN target detection models.

[0063] The processor used for the simulation is Xeon(R)CPU E5-2630 v4@2.20GHz×40, memory is 64.00GB, GPU is 8G GeForce RTX Titan, simulation platform is Ubuntu16.04 operating system, using Pytorch deep learning framework and implemented in Python language.

[0064] (2) Experimental results and comparative analysis

[0065] The average accuracy is used as the target detection evaluation index. The present invention and the above two benchmark methods are used to perform simulation tests on the constructed armored target and key parts data set, and the numerical results of the evaluation indexes are compared and analyzed, as shown in Table 1.

[0066] Table 1 Comparison of average precision indicators

[0067]

[0068] It can be seen from Table 1 that the detection accuracy of the present invention for armored targets and key parts is significantly higher than that of the two benchmark methods, which shows that the present invention has relatively superior detection performance.

[0069] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for detecting armored targets and key parts based on deep learning and knowledge reasoning, characterized in that: include: Obtain missile-borne images of armored targets through target scene simulation or flight tests, annotate the armored targets and key parts in the missile-borne images, and thus construct an image dataset; Constructing a deep learning detection model and a key part auxiliary detection unit; wherein the deep learning detection model is used to detect armored targets and key parts in the missile-borne image and output corresponding target detection frames; when the deep learning detection model only detects target detection frames for armored targets under preset conditions but cannot generate target detection frames for key parts, the key part auxiliary detection unit is called to output the detection frame coordinates of the key parts through the unit; Using the image data set to train and test the deep learning detection model and the key part auxiliary detection unit, and saving the trained deep learning detection model and the key part auxiliary detection unit; Obtain the missile-borne image of the armored target to be detected, use the trained deep learning detection model and the auxiliary detection unit of key parts to perform detection, and output the detection results of the armored target and key parts.

2. The armored target and key parts detection method based on deep learning and knowledge reasoning according to claim 1 is characterized in that: In the target scene simulation and flight test, after the missile-borne platform captures the ground armored target, a side image sequence of the armored target should be collected from far to near; The acquired missile-borne images are preprocessed and cleaned, and the armored targets and key parts are marked to form a data set with label information; the key parts refer to the key and important parts on the armored targets, which are set and marked according to the detection requirements.

3. The armored target and key parts detection method based on deep learning and knowledge reasoning according to claim 2 is characterized in that: The dataset is divided into a training set and a test set in a ratio of 3:1, and it is ensured that the missile-borne images of the two datasets come from different test or simulation scenarios; the model is first trained with the training set, and then tested with the test set. After meeting the requirements, the trained model is saved.

4. The armored target and key parts detection method based on deep learning and knowledge reasoning according to claim 1 is characterized in that: The deep learning detection model is: Based on the existing Faster RCNN, the backbone network ResNet is replaced with Res2Net with a multi-branch structure, and the following distance is used to measure the position relationship of the target bounding box: Where N a and N b It describes two candidate target detection boxes A = [x a ,y a ,h a , w a ] and B = [x b ,y b ,h b , w b ] Gaussian distribution model, W(N a , N b ) and D(N a , N b ) represents the distance between two Gaussian distribution models and its exponential normalized form, c is a control parameter less than 1, x a ,y a represents the center point coordinates of the candidate target detection box A, w a ,h a represents the width and height of the candidate target detection box A; ||·||2 represents the l2 norm; During the detection process, after the deep learning detection model generates multiple candidate target detection frames with different confidence levels for the armored target, the distance D (N a , N b ), then sort all the calculated distances, select the two candidate target detection frames with the largest distance, and use the one with higher confidence as the final target detection frame of this type, so as to obtain target detection frames for armored targets and key parts.

5. The armored target and key parts detection method based on deep learning and knowledge reasoning according to claim 1 is characterized in that: When the deep learning detection model generates a target detection frame for an armored target but fails to generate a target detection frame for a critical part, the target detection frame of the armored target is first judged. When the number of pixels in the target detection frame is not less than the preset number of pixels, the critical part auxiliary detection unit is used to detect the critical part.

6. The armored target and key parts detection method based on deep learning and knowledge reasoning according to claim 5 is characterized in that: The preset number of pixels is 400.

7. The armored target and key parts detection method based on deep learning and knowledge reasoning according to claim 1 is characterized in that: The modeling process of the key part auxiliary detection unit is as follows: Assume the physical size of the armored target is l target × target ×h target , the height of the key part is ρh target , ρ is the high and low control parameter, 0<ρ<1; when the missile-carrying platform approaches the armored target from far to near with a lateral perspective, the target detection frame size of the armored target is h box × box , the center coordinates of the target box detection box are (x box ,y box ); then according to the optical imaging geometry, w box Relatively fixed and target is proportional to h box Can be regarded as h target and w target The sum of the object surface projections; assuming the sight angle of the missile-carrying platform is θ, w target With h target The ratio is b, then: h box =κ(aw target sinθ+h target cosθ) In the formula, κ and a are proportional coefficients, and κ can be obtained from w box / l target Calculated; Therefore, the image center coordinates of the key parts are An object detection frame of the key part is generated based on the image center coordinates of the key part and output.

8. The armored target and key parts detection method based on deep learning and knowledge reasoning according to claim 7 is characterized in that: For different critical parts, different values ​​of ρ can be used to locate the critical parts at different positions on the armored target.

9. A terminal device comprising a processor, a memory and a computer program stored in the memory; characterized in that: When the processor executes the computer program, it implements the armored target and critical part detection method based on deep learning and knowledge reasoning according to any one of claims 1 to 8.

10. A computer-readable storage medium, wherein a computer program is stored in the medium; characterized in that: When the computer program is executed by a processor, the armored target and critical part detection method based on deep learning and knowledge reasoning according to any one of claims 1 to 8 is implemented.

Images