Method for identifying weak part of target under complex background by fusing cognitive map

Abstract

The invention discloses a recognition method for a target weak part under a complex background fused with a cognitive map, and the method comprises the steps: obtaining a to-be-recognized input image, inputting the to-be-recognized input image into a trained target weak part recognition model based on visual perception, carrying out the auxiliary reasoning through combining with a spatial position relation knowledge map of the target weak part, and carrying out the recognition of the target weak part. Obtaining a target weak part identification result; wherein the target weak part identification model comprises a backbone network, a multi-scale sensing network and a candidate region generation network; the multi-scale sensing network is used for fusing the multi-scale feature maps extracted by the backbone network to obtain a fused feature map; the candidate region generation network is used for generating a target candidate region according to the fusion feature map so as to realize classification of target weak parts, and a label distribution strategy based on Gaussian distribution measurement is set so as to improve the positive sample preset anchor frame distribution number of the target; and a target weak part identification multi-task loss function is set, and training of an identification model is realized.

Description

Technical Field

[0001] The present invention belongs to the field of image information processing technology, and in particular relates to a method for identifying weak parts of a target in a complex background by integrating cognitive maps, which can be used to obtain the category and location information of the target and its weak parts from images detected by a large number of aircraft. Background Art

[0002] Target recognition technology, as a prerequisite for target tracking, is key for aircraft to track targets. Faced with complex terrestrial background environments and the complex and uncertain state of targets, aircraft must not only have the ability to accurately recognize targets, but also the ability to identify the weak points of targets. However, existing target recognition algorithms mostly focus on the global information of the target, ignore the local characteristics of the target, lack the understanding of typical weak points of targets in complex environments, and cannot meet the current demand for accurate tracking of targets in complex backgrounds. Currently, major domestic research institutions are also conducting in-depth research on related technologies to improve the accuracy of target tracking in complex backgrounds, especially the identification of weak points of targets. Although existing technologies provide solutions for identifying weak points of targets, they rely solely on data-driven intelligent algorithms and have not built a knowledge system between the target and its weak points. As a result, the model lacks the ability to make associative reasoning and the overall level of intelligence is not high. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for identifying weak parts of targets under complex backgrounds that integrates cognitive maps, which comprehensively utilizes the rapidity of data-driven and the logic of knowledge-driven to alleviate the problem of high missed detection rate in identifying weak parts of targets under complex backgrounds and improve the accuracy of identifying weak parts of targets under complex backgrounds.

[0004] In order to achieve the above tasks, the present invention adopts the following technical solutions:

[0005] A method for identifying weak points of a target in a complex background by integrating cognitive maps, including:

[0006] The input image to be identified is obtained and input into a trained target weak part identification model based on visual perception. Assisted reasoning is performed in combination with the target weak part spatial position relationship knowledge graph to obtain the target weak part identification result.

[0007] The target weak part recognition model includes a backbone network, a multi-scale perception network, and a candidate region generation network; the backbone network is used to perform multi-scale feature extraction on the input image to obtain multi-scale feature maps at different stages; the multi-scale perception network is used to further perform multi-scale fusion on the multi-scale feature maps to obtain a fused feature map; the candidate region generation network is used to generate target candidate areas based on the fused feature map to achieve the classification of target weak parts, wherein a label allocation strategy based on Gaussian distribution measurement is provided to increase the number of preset anchor frames allocated to the target positive samples; a multi-task loss function for target weak part recognition is set to achieve the training of the recognition model;

[0008] By analyzing the target structure and weak parts, and determining the position of the weak parts on the target and the positional relationship between different weak parts, a knowledge graph of the spatial positional relationship of the target weak parts is constructed.

[0009] Furthermore, the backbone network adopts ResNet50; for the input training sample, the corresponding multi-scale feature maps P2, P3, P4, and P5 are obtained after four stages of the backbone network;

[0010] The multi-scale perception network further performs multi-scale feature fusion on the multi-scale feature maps P2, P3, P4, and P5 to obtain fused feature maps P2′, P3′, P4′, and P5′, including:

[0011] The feature map P2 is downsampled 8 times through three 3×3 convolutional layers with a step size of 2 to obtain the feature map P 2→5 ; The feature map P3 is downsampled by 4 times through two 3×3 convolutional layers with a step size of 2 to obtain the feature map P 3→5 ; The feature map P4 is downsampled by 2 times through a 3×3 convolution layer with a step size of 2 to obtain the feature map P 4→5 ; And the sampled feature map P 2→5 , P 3→5 , P 4→5 Add element-by-element to the feature map P5 to obtain the fused feature map P5′;

[0012] The feature map P2 is downsampled by 4 times through two 3×3 convolutional layers with a step size of 2 to obtain the feature map P 2→4 ; The feature map P3 is downsampled by 2 times through two 3×3 convolutional layers with a step size of 1 to obtain the feature map P 3→4 ; The feature map P5 is upsampled by 2 times through bilinear upsampling, and the number of channels is adjusted through a 1×1 convolution layer to obtain the feature map P 5→4 ; And the sampled feature map P 2→4 , P 3→4 , P 5→4Add element-by-element to the feature map P4 to obtain the fused feature map P4′;

[0013] The feature map P2 is downsampled by 2 times through a 3×3 convolution layer with a step size of 2 to obtain the feature map P 2→3 ; The feature map P4 is upsampled by 2 times through bilinear upsampling, and the number of channels is adjusted through a 1×1 convolution layer to obtain the feature map P 4→3 ; The feature map P5 is upsampled by 4 times through bilinear upsampling, and the number of channels is adjusted through a 1×1 convolution layer to obtain the feature map P 5→3 ; And the sampled feature map P 2→3 , P 4→3 , P 5→3 Add the element-by-element sum of the feature map P3 to obtain the fused feature map P3′;

[0014] The feature map P3 is upsampled by 2 times through bilinear upsampling, and the number of channels is adjusted through a 1×1 convolution layer to obtain the feature map P 3→2 ; The feature map P4 is upsampled by 4 times through bilinear upsampling, and the number of channels is adjusted through a 1×1 convolution layer to obtain the feature map P 4→2 ; The feature map P5 is upsampled 8 times by bilinear upsampling, and the number of channels is adjusted by 1×1 convolution layer to obtain the feature map P 5→2 ; And the sampled feature map P 3→2 , P 4→2 , P 5→2 Add the feature map P2 element by element to obtain the fused feature map P2′.

[0015] Furthermore, the label assignment strategy based on Gaussian distribution metric replaces the original label assignment strategy based on IoU criterion in the candidate region generation network; specifically, it includes:

[0016] The candidate region generation network automatically sets a series of preset anchor boxes for the fused feature map output by the multi-scale perception network, and outputs the probability of each preset anchor box belonging to the foreground object and its bounding box position offset information; among these preset anchor boxes, the preset anchor boxes with higher correlation with the position of the annotated target bounding box are assigned as positive samples to the candidate region generation network to generate target candidate regions, while the preset anchor boxes containing more background information are assigned as negative samples to the candidate region generation network;

[0017] The normalized Wasserstein distance is designed to measure the position correlation between the position of the annotated target bounding box and the preset anchor box. First, the probability density function of the two-dimensional Gaussian distribution is used to model the annotated target bounding box:

[0018]

[0019] Where x is a random variable, cx, cy, w and h represent the horizontal and vertical coordinates, height and width of the center point of the target respectively;

[0020] Secondly, for two Gaussian distributions N a and N b Modeled target bounding box A=(cx a ,cy a ,w a ,h a ) and B=(cx b ,cy b ,w b ,h b ), its position correlation is measured by the Wasserstein distance between Gaussian distributions, which can be expressed as:

[0021]

[0022] Among them, cx a ,cy a ,w a ,h a Represents a two-dimensional Gaussian distribution N a The horizontal and vertical coordinates, height and width of the center point of the target during modeling;

[0023] The exponential function is used to normalize the value of the Wasserstein distance to obtain the normalized Wasserstein distance metric:

[0024]

[0025] Where C is a constant.

[0026] Furthermore, for each preset anchor box, the normalized Wasserstein distance metric between it and all labeled target bounding boxes is calculated; if the maximum value of the normalized Wasserstein distance metric between the preset anchor box and all labeled target bounding boxes is greater than a first threshold, it is set as a positive sample and assigned to the labeled target corresponding to the maximum value of the normalized Wasserstein distance metric to generate a target candidate area; if the maximum value of the normalized Wasserstein distance metric between the preset anchor box and all labeled target bounding boxes is less than a second threshold, it is set as a negative sample.

[0027] Furthermore, the anchor box classification loss function and the anchor box bounding box position regression loss function are set to enable the candidate region generation network to learn to distinguish the target candidate region from the background region:

[0028] The binary cross entropy loss function is set as the anchor box classification loss function, and L rpn_cls express:

[0029]

[0030] in, is a binary label, the preset anchor box for the positive sample For the preset anchor box of negative samples is the probability that the i-th preset anchor box belongs to the foreground target, and N is the number of preset anchor boxes;

[0031] Set the smooth L1 loss function as the anchor box bounding box position regression loss function, and use L rpn_reg Expressed as:

[0032] L rpn_reg =Smooth L1 (x)

[0033] Among them, Smooth L1 (x) represents the smooth L1 square loss function, Represents the offset t predicted by the preset anchor box of the i-th positive sample i The offset of the preset anchor box of the i-th positive sample relative to the labeled target bounding box The difference.

[0034] Furthermore, a multi-classification loss function for the target candidate region and a bounding box position regression loss function are set;

[0035] The cross entropy loss function is set as the target candidate region multi-classification loss function, and L rcnn_cls express:

[0036]

[0037] Among them, p i (c) represents the true value label corresponding to the i-th target candidate region, represents the classification probability that the i-th target candidate region belongs to category c.

[0038] The smooth L1 loss function is set as the loss function of the target candidate area position regression task, and L rcnn_reg express:

[0039] L rcnn_reg =Smooth L1 (x)

[0040] in, Represents the offset t predicted by the i-th target candidate region i The offset of the i-th target candidate region relative to the labeled target bounding box The difference between

[0041] The multi-task loss function for target weak part identification includes anchor box classification loss function, anchor box bounding box position regression loss function, target candidate area multi-classification loss function and bounding box position regression loss function.

[0042] Furthermore, the positive and negative samples generated by the candidate region generation network are pooled through the region of interest to generate fixed-size target candidate region features, and then pass through two fully connected layers to output the probability of each target candidate region belonging to different target categories and the bounding box position offset information.

[0043] Furthermore, the input image is input into the target weak part recognition model to obtain the target weak part recognition result, including the target weak part bounding box position, category and confidence score in the input image; the target weak part spatial position relationship knowledge graph is used to assist in reasoning, and the obtained target weak part recognition result is corrected to obtain the final recognition result.

[0044] Furthermore, the auxiliary reasoning is specifically as follows:

[0045] First, the constructed knowledge graph of the spatial position relationship of the target weak parts is used to obtain the spatial geometric position distribution relationship between different weak parts and the target, as well as between different weak parts; then, based on the target weak part identification results and combined with the spatial position relationship contained in the knowledge graph of the spatial relationship of the target weak parts, the missed and misdetected target weak parts are inferred and corrected.

[0046] A terminal device includes a processor, a memory, and a computer program stored in the memory; when the processor executes the computer program, the method for identifying weak points of a target under a complex background by integrating cognitive maps is implemented.

[0047] A computer-readable storage medium stores a computer program; when the computer program is executed by a processor, the method for identifying weak points of a target in a complex background by integrating cognitive maps is implemented.

[0048] Compared with the prior art, the present invention has the following technical features:

[0049] 1. Aiming at the high dynamic change characteristics of target scale in the process of identifying weak parts of targets under complex backgrounds, the present invention designs a multi-scale visual perception network to replace ResNet as the backbone network of the present invention. The multi-scale visual perception network fuses network features from different levels, so that the fused features contain both rich detail information and high-level semantic information of the target, which is conducive to the recognition of targets of different scales.

[0050] 2. This invention takes into account that the widely used Intersection of Union (IoU) criterion is very sensitive to positional deviations of small targets. Slight positional perturbations can cause drastic changes in the IoU, which in turn reduces the number of positive sample labels assigned in the weak part identification task, inhibiting the model's learning of the target's weak parts. To address this issue, this invention models the target bounding box as a two-dimensional Gaussian function and uses the normalized Wasserstein distance to measure the positional relationship between bounding boxes. This enhances the stability against perturbations of the small target bounding box, provides more positive sample labels for the target's weak parts, and facilitates the model's learning of the target's weak parts.

[0051] 3. The present invention fully analyzes the positional relationship between the target and the weak parts, as well as between different weak parts, constructs a knowledge graph of the spatial positional relationship of the target weak parts, and uses the knowledge graph of the spatial positional relationship of the target weak parts to assist in reasoning to correct the target weak part identification results, thereby improving the recognition accuracy of the target weak parts. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 Flowchart for the implementation of the present invention;

[0053] Figure 2 This is the overall structural diagram of the model of the present invention;

[0054] Figure 3 This is a schematic diagram of the multi-scale feature fusion module proposed in the present invention;

[0055] Figure 4 Schematic diagram of the knowledge graph of the spatial position relationship of target weak parts constructed by the present invention. DETAILED DESCRIPTION

[0056] The present invention provides a method for identifying weak points of a target in a complex background by integrating cognitive maps, comprising:

[0057] The input image to be identified is obtained and input into a trained target weak part identification model based on visual perception. Assisted reasoning is performed in combination with the target weak part spatial position relationship knowledge graph to obtain the target weak part identification result.

[0058] The target weak part recognition model includes a backbone network, a multi-scale perception network, and a candidate region generation network; the backbone network is used to perform multi-scale feature extraction on the input image to obtain multi-scale feature maps at different stages; the multi-scale perception network is used to further perform multi-scale fusion on the multi-scale feature maps to obtain a fused feature map; the candidate region generation network is used to generate target candidate areas based on the fused feature map to achieve the classification of target weak parts, wherein a label allocation strategy based on Gaussian distribution measurement is provided to increase the number of preset anchor frames allocated to the target positive samples; a multi-task loss function for target weak part recognition is set to achieve the training of the recognition model;

[0059] By analyzing the target structure and weak parts, and determining the position of the weak parts on the target and the positional relationship between different weak parts, a knowledge graph of the spatial positional relationship of the target weak parts is constructed.

[0060] The training and application process of the recognition network will be further described in detail below in conjunction with the embodiments.

[0061] 1. Construct training set and test set.

[0062] The videos of targets in complex backgrounds recorded by the aircraft detection device are divided into 75% of the videos as the training set and the remaining 25% as the test set. The video frame images in the training set and the test set are extracted at intervals of 10 frames to obtain corresponding training samples and test samples. In this example, the training sample contains 1206 video frame images and the test sample contains 400 video frame images.

[0063] 2. Target weak point identification model.

[0064] 2.1 Backbone network.

[0065] In one embodiment of the present invention, the backbone network adopts ResNet50; for the input training sample, the corresponding multi-scale feature maps P2, P3, P4, and P5 are obtained through the four stages of the backbone network, where the feature map sizes of the multi-scale feature maps P2, P3, P4, and P5 are 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image size, respectively.

[0066] 2.2 Multi-scale perception network.

[0067] A multi-scale visual perception network is designed to extract multi-scale features that contain both rich detail information and semantic information. The multi-scale perception network further performs multi-scale feature fusion on the multi-scale feature maps P2, P3, P4, and P5 to fuse feature information from different layers of the network and generate multi-scale fused feature maps P2′, P3′, P4′, and P5′ that contain both rich detail information and semantic information. The details are as follows:

[0068] (1) The feature map P2 is downsampled 8 times through three 3×3 convolutional layers with a step size of 2 to obtain the feature map P 2→5 ; The feature map P3 is downsampled by 4 times through two 3×3 convolutional layers with a step size of 2 to obtain the feature map P 3→5 ; The feature map P4 is downsampled by 2 times through a 3×3 convolution layer with a step size of 2 to obtain the feature map P 4→5 ; And the sampled feature map P 2→5 , P 3→5 , P 4→5 Add the feature map P5 element by element to obtain the fused feature map P5′.

[0069] (2) The feature map P2 is downsampled by 4 times through two 3×3 convolutional layers with a step size of 2 to obtain the feature map P 2→4 ; The feature map P3 is downsampled by 2 times through two 3×3 convolutional layers with a step size of 1 to obtain the feature map P 3→4 ; The feature map P5 is upsampled by 2 times through bilinear upsampling, and the number of channels is adjusted through a 1×1 convolution layer to obtain the feature map P 5→4 ; And the sampled feature map P 2→4 , P 3→4 , P 5→4 Add the feature map P4 element by element to obtain the fused feature map P4′.

[0070] (3) The feature map P2 is downsampled by 2 times through a 3×3 convolution layer with a step size of 2 to obtain the feature map P 2→3 ; The feature map P4 is upsampled by 2 times through bilinear upsampling, and the number of channels is adjusted through a 1×1 convolution layer to obtain the feature map P 4→3 ; The feature map P5 is upsampled by 4 times through bilinear upsampling, and the number of channels is adjusted through a 1×1 convolution layer to obtain the feature map P 5→3 ; And the sampled feature map P 2→3 , P 4→3 , P 5→3 Add the feature map P3 element by element to obtain the fused feature map P3′.

[0071] (4) The feature map P3 is upsampled by 2 times through bilinear upsampling, and the number of channels is adjusted through a 1×1 convolution layer to obtain the feature map P 3→2 ; The feature map P4 is upsampled by 4 times through bilinear upsampling, and the number of channels is adjusted through a 1×1 convolution layer to obtain the feature map P 4→2 ; The feature map P5 is upsampled 8 times by bilinear upsampling, and the number of channels is adjusted by 1×1 convolution layer to obtain the feature map P 5→2 ; And the sampled feature map P3→2 , P 4→2 , P 5→2 Add the feature map P2 element by element to obtain the fused feature map P2′.

[0072] 2.3 Candidate Region Generation Network RPN.

[0073] The candidate region generation network is used to automatically set a series of preset anchor boxes for the fused feature map output by the multi-scale perception network, and output the probability that each preset anchor box belongs to the foreground target and its bounding box position offset information; for these preset anchor boxes, the preset anchor boxes with higher correlation with the position of the marked target bounding box are assigned as positive samples at the candidate region generation network to generate target candidate regions, while the preset anchor boxes containing more background information are assigned as negative samples at the candidate region generation network.

[0074] In this scheme, a label assignment strategy based on Gaussian distribution metric is designed to replace the original label assignment strategy based on the IoU criterion in the candidate region generation network, so as to increase the number of positive sample preset anchor boxes assigned to small objects and promote small object learning.

[0075] (1) The candidate region generation network automatically sets a series of preset anchor boxes for the fused feature map output by the multi-scale perception network, and outputs the probability that each preset anchor box belongs to the foreground target and its bounding box position offset information; for these preset anchor boxes, the preset anchor boxes with higher correlation with the position of the marked target bounding box are assigned as positive samples at the candidate region generation network to generate target candidate regions, while the preset anchor boxes containing more background information are assigned as negative samples at the candidate region generation network.

[0076] (2) Design a normalized Wasserstein distance to measure the positional correlation between the position of the labeled target bounding box and the preset anchor box; specifically, first use the probability density function of the two-dimensional Gaussian distribution to model the labeled target bounding box:

[0077]

[0078] Among them, x is a random variable, cx, cy, w and h represent the horizontal and vertical coordinates, height and width of the center point of the target respectively.

[0079] Secondly, for two Gaussian distributions N a and N b Modeled target bounding box A=(cx a ,cy a ,w a ,h a ) and B=(cx b ,cy b ,w b ,h b), its position correlation is measured by the Wasserstein distance between Gaussian distributions, which can be expressed as:

[0080]

[0081] Among them, cx a ,cy a ,w a ,h a Represents a two-dimensional Gaussian distribution N a The horizontal and vertical coordinates, height and width of the center point of the target during modeling

[0082] The exponential function is used to normalize the value of Wasserstein distance (i.e., the value range is between 0 and 1) to obtain the normalized Wasserstein distance metric (NWD):

[0083]

[0084] Where C is a constant.

[0085] (3) The normalized Wasserstein distance metric (NWD) is used as the basis for assigning preset anchor box labels in the candidate region generation network, replacing the original label assignment strategy based on the IoU criterion. Specifically:

[0086] For each preset anchor box, the normalized Wasserstein distance metric is calculated between it and all labeled target bounding boxes; if the maximum value of the normalized Wasserstein distance metric between the preset anchor box and all labeled target bounding boxes is greater than 0.7, it is set as a positive sample and assigned to the labeled target corresponding to the maximum value of the normalized Wasserstein distance metric to generate the target candidate region; if the maximum value of the normalized Wasserstein distance metric between the preset anchor box and all labeled target bounding boxes is less than 0.3, it is set as a negative sample.

[0087] 2.4 Multi-task loss function for target weak point identification.

[0088] (1) Set the anchor box classification loss function and the anchor box bounding box position regression loss function to enable the candidate region generation network to learn to distinguish the target candidate region from the background region:

[0089] The binary cross entropy loss function is set as the anchor box classification loss function, and L rpn_cls express:

[0090]

[0091] in, is a binary label, the preset anchor box for the positive sample For the preset anchor box of negative samples is the probability that the i-th preset anchor box belongs to the foreground target, and N is the number of preset anchor boxes.

[0092] Set the smooth L1 loss function as the anchor box bounding box position regression loss function, and use L rpn_reg Expressed as:

[0093] L rpn_reg =Smooth L1 (x)

[0094] Among them, Smooth L1 (x) represents the smooth L1 square loss function, Represents the offset t predicted by the preset anchor box of the i-th positive sample i The offset of the preset anchor box of the i-th positive sample relative to the labeled target bounding box The difference.

[0095] (2) Set the target candidate region multi-classification loss function and the bounding box position regression loss function.

[0096] The cross entropy loss function is set as the target candidate region multi-classification loss function, and L rcnn_cls express:

[0097]

[0098] Among them, p i (c) represents the true value label corresponding to the i-th target candidate region, represents the classification probability that the i-th target candidate region belongs to category c.

[0099] The smooth L1 loss function is set as the loss function of the target candidate area position regression task, and L rcnn_reg express:

[0100] L rcnn_reg =Smooth L1 (x)

[0101] in, Represents the offset t predicted by the i-th target candidate region i The offset of the i-th target candidate region relative to the labeled target bounding box The difference.

[0102] The target weak part identification multi-task loss function L is composed of the anchor box classification loss function L rpn_cls , anchor box bounding box position regression loss function L rpn_reg , target candidate region classification loss function L clsAnd the target bounding box position regression loss function L reg composition:

[0103] L=L rpn_cls +L rpn_reg +L rcnn_cls +L rcnn_reg .

[0104] 2.5 Feature extraction of target candidate regions.

[0105] The positive and negative samples generated by the candidate region generation network are pooled through the region of interest (RoI) to generate target candidate region features of a fixed size of 7×7×256, and then pass through two fully connected layers (where the first fully connected layer has an input feature dimension of 12544 and an output feature dimension of 1024, and the second fully connected layer has an input feature dimension of 1024 and an output feature dimension of 1024) to output the probability of each target candidate region belonging to different target categories and the bounding box position offset information.

[0106] 3. Training of target weak point identification model.

[0107] Set training parameters: In this embodiment, the optimizer uses the stochastic gradient descent method with a weight decay of 5e-4 and a momentum of 0.9 as the optimizer. The number of training steps is set to 2000, the number of training rounds is set to 12, the batch training size is 4, the learning rate starts from 0.01, and the learning rate decays to 0.001 and 0.0001 in the 8th and 11th rounds respectively. ResNet pre-trained on the ImageNet dataset is used as the initialization weight of the backbone network in the target weak point recognition model, and the remaining parameters are randomly initialized.

[0108] The training samples are input into the target weak part recognition model, and the optimizer SGD is used to optimize the target weak part recognition multi-task loss function and update the weight parameters. When the number of training rounds reaches 2000, the trained target weak part recognition model is obtained.

[0109] 4. Auxiliary reasoning based on the knowledge graph of the spatial position relationship of the target’s weak points.

[0110] Based on the characteristics of the rigid structure of the target under typical complex backgrounds, the positional relationship between the target and the weak parts, as well as between different weak parts, is analyzed to construct a knowledge graph of the spatial positional relationship of the target's weak parts.

[0111] The test sample is input into the target weak part recognition model to obtain the target weak part recognition result, including the target weak part bounding box position, category and confidence score in the test sample; the target weak part spatial position relationship knowledge graph is used to assist in reasoning, and the obtained target weak part recognition result is corrected to obtain the final recognition result.

[0112] The auxiliary reasoning is specifically as follows:

[0113] First, the constructed knowledge graph of the spatial position relationship of the target weak parts is used to obtain the spatial geometric position distribution relationship between different weak parts and the target, as well as between different weak parts; then, based on the target weak part identification results and combined with the spatial position relationship contained in the knowledge graph of the spatial relationship of the target weak parts, the missed and misdetected target weak parts are inferred and corrected.

[0114] For example, when the target weak point identification model identifies a tank turret, the constructed target weak point spatial relationship knowledge graph shows that the center points of the tank turret and the track are on the same straight line, based on which the center point of the track can be obtained; in addition, the size of the track can be further inferred based on the prior knowledge of the size of the track and turret, and finally the position of the track can be obtained.

[0115] In summary, the present invention constructs a target weak part recognition network based on visual perception, and utilizes the spatial position relationship knowledge graph between the target and its weak parts to realize a target weak part recognition method under complex backgrounds that integrates cognitive graphs. First, in view of the high dynamic change characteristics of the target scale during the target weak part recognition process under complex backgrounds, a multi-scale visual perception network is designed to improve the model's recognition accuracy for multi-scale weak parts. Secondly, considering the small pixel ratio of the target weak part, a label allocation strategy based on Gaussian distribution measurement is designed to enhance the model's feature learning and optimization capabilities for small targets, thereby improving the recognition accuracy of the target weak part. In addition, a knowledge graph of the spatial position of the target weak part is constructed, and the visual information recognition results and knowledge reasoning are combined to alleviate the phenomenon of missed detection and false detection of the target weak part, and further improve the recognition accuracy of the target weak part.

[0116] Comparison and analysis of simulation experiment results:

[0117] 1. Simulation conditions and contents.

[0118] The target tracking video recorded by the aircraft detection device against a complex background is divided into two parts: 75% of the video is used as the training set, and the remaining 25% is used as the test set. Ten-frame videos are extracted from the training and test sets to obtain the corresponding training and test samples. In this example, the training sample contains 1206 video frames, and the test sample contains 400 video frames.

[0119] The processor used for simulation is The system is equipped with a Xeon(R) CPU E5-2630 v4@2.20GHz×40, a memory of 64.00GB, an 8G GeForce RTX Titan GPU, and an Ubuntu 16.04 operating system. The simulation is implemented using the Pytorch deep learning framework and Python.

[0120] 2. Comparison and analysis of simulation experiment results.

[0121] To verify the effectiveness of the present invention, three existing methods are set, among which: existing method 1 is the yolov5 target detection model; existing method 2 is the Faster RCNN target detection model with ResNet101 as the backbone network; existing method 3 is the remote sensing image target detection model of multi-model decision fusion.

[0122] The mean average precision is used as the evaluation index for detecting all target categories, and the average precision is used as the evaluation index for detecting single-category targets. The target detection simulation is performed on the tank target detection dataset constructed by the present invention and three existing methods, and the numerical results of the evaluation indicators of their detection are compared, as shown in Table 1.

[0123] Table 1 Comparison of the numerical results of the evaluation indicators of the present invention and the benchmark model

[0124]

[0125] Table 1 shows a comparison of the numerical results of the evaluation indicators of the present invention and three existing methods. Bold indicates the highest average accuracy of the four methods in detecting targets of this category. From the comparison of the numerical results in Table 1, it can be seen that the mean average accuracy of existing method 1 is 0.799, and the mean average accuracy of the present invention is 0.923. The present invention has improved the mean average accuracy of existing method 1 by 0.124, which is a huge improvement and far exceeds the accuracy of the single-stage target detection model. The mean average accuracy of existing method 2 is 0.823, and the mean average accuracy of existing method 3 is 0.872, which are still far behind the method we proposed. The present invention has achieved the best results on three types of targets and their weak points: tanks, turrets, and tracks.

[0126] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A method for identifying weak points of a target in a complex background by integrating cognitive maps, characterized in that: include: The input image to be identified is obtained and input into a trained target weak part identification model based on visual perception. Assisted reasoning is performed in combination with the target weak part spatial position relationship knowledge graph to obtain the target weak part identification result. The target weak part recognition model includes a backbone network, a multi-scale perception network, and a candidate region generation network; the backbone network is used to perform multi-scale feature extraction on the input image to obtain multi-scale feature maps at different stages; the multi-scale perception network is used to further perform multi-scale fusion on the multi-scale feature maps to obtain a fused feature map; the candidate region generation network is used to generate target candidate areas based on the fused feature map to achieve the classification of target weak parts, wherein a label allocation strategy based on Gaussian distribution measurement is provided to increase the number of preset anchor frames allocated to the target positive samples; a multi-task loss function for target weak part recognition is set to achieve the training of the recognition model; By analyzing the target structure and weak parts, and determining the position of the weak parts on the target and the positional relationship between different weak parts, a knowledge graph of the spatial positional relationship of the target weak parts is constructed.

2. The method for identifying weak points of a target in a complex background by integrating cognitive maps according to claim 1 is characterized in that: The backbone network uses ResNet50. For the input training sample, the corresponding multi-scale feature maps P2, P3, P4, and P5 are obtained through the four stages of the backbone network. The multi-scale perception network further performs multi-scale feature fusion on the multi-scale feature maps P2, P3, P4, and P5 to obtain fused feature maps P2′, P3′, P4′, and P5′, including: The feature map P2 is downsampled 8 times through three 3×3 convolutional layers with a step size of 2 to obtain the feature map P 2→5 ; The feature map P3 is downsampled by 4 times through two 3×3 convolutional layers with a step size of 2 to obtain the feature map P 3→5 ; The feature map P4 is downsampled by 2 times through a 3×3 convolution layer with a step size of 2 to obtain the feature map P 4→5 ; And the sampled feature map P 2→5 , P 3→5 , P 4→5 Add element-by-element to the feature map P5 to obtain the fused feature map P5′; The feature map P2 is downsampled by 4 times through two 3×3 convolutional layers with a step size of 2 to obtain the feature map P 2→4 ; The feature map P3 is downsampled by 2 times through two 3×3 convolutional layers with a step size of 1 to obtain the feature map P 3→4 ; The feature map P5 is upsampled by 2 times through bilinear upsampling, and the number of channels is adjusted through a 1×1 convolution layer to obtain the feature map P 5→4 ; And the sampled feature map P 2→4 , P 3→4 , P 5→4 Add element-by-element to the feature map P4 to obtain the fused feature map P4′; The feature map P2 is downsampled by 2 times through a 3×3 convolution layer with a step size of 2 to obtain the feature map P 2→3 ; The feature map P4 is upsampled by 2 times through bilinear upsampling, and the number of channels is adjusted through a 1×1 convolution layer to obtain the feature map P 4→3 ; The feature map P5 is upsampled by 4 times through bilinear upsampling, and the number of channels is adjusted through a 1×1 convolution layer to obtain the feature map P 5→3 ; And the sampled feature map P 2→3 , P 4→3 , P 5→3 Add the element-by-element sum of the feature map P3 to obtain the fused feature map P3′; The feature map P3 is upsampled by 2 times through bilinear upsampling, and the number of channels is adjusted through a 1×1 convolution layer to obtain the feature map P 3→2 ; The feature map P4 is upsampled by 4 times through bilinear upsampling, and the number of channels is adjusted through a 1×1 convolution layer to obtain the feature map P 4→2 ; The feature map P5 is upsampled 8 times by bilinear upsampling, and the number of channels is adjusted by 1×1 convolution layer to obtain the feature map P 5→2 ; And the sampled feature map P 3→2 , P 4→2 , P 5→2 Add the feature map P2 element by element to obtain the fused feature map P2′.

3. The method for identifying weak points of a target in a complex background by integrating cognitive maps according to claim 1, characterized in that: The label assignment strategy based on Gaussian distribution metric replaces the original label assignment strategy based on IoU criterion in the candidate region generation network; Specifically include: The candidate region generation network automatically sets a series of preset anchor boxes for the fused feature map output by the multi-scale perception network, and outputs the probability of each preset anchor box belonging to the foreground object and its bounding box position offset information; among these preset anchor boxes, the preset anchor boxes with higher correlation with the position of the annotated target bounding box are assigned as positive samples to the candidate region generation network to generate target candidate regions, while the preset anchor boxes containing more background information are assigned as negative samples to the candidate region generation network; The normalized Wasserstein distance is designed to measure the position correlation between the position of the annotated target bounding box and the preset anchor box. First, the probability density function of the two-dimensional Gaussian distribution is used to model the annotated target bounding box: Where x is a random variable, cx, cy, w and h represent the horizontal and vertical coordinates, height and width of the center point of the target respectively; Secondly, for two Gaussian distributions N a and N b Modeled target bounding box A=(cx a ,cy a ,w a ,h a ) and B=(cx b ,cy b ,w b ,h b ), its position correlation is measured by the Wasserstein distance between Gaussian distributions, which can be expressed as: Among them, cx a ,cy a ,w a ,h a Represents a two-dimensional Gaussian distribution N a The horizontal and vertical coordinates, height and width of the center point of the target during modeling; The exponential function is used to normalize the value of the Wasserstein distance to obtain the normalized Wasserstein distance metric: Where C is a constant.

4. The method for identifying weak points of a target in a complex background by integrating cognitive maps according to claim 1 is characterized in that: For each preset anchor box, calculate the normalized Wasserstein distance metric between it and all annotated target bounding boxes; If the maximum value of the normalized Wasserstein distance metric between the preset anchor box and all labeled target bounding boxes is greater than the first threshold, it is set as a positive sample and assigned to the labeled target corresponding to the maximum value of the normalized Wasserstein distance metric to generate a target candidate area; if the maximum value of the normalized Wasserstein distance metric between the preset anchor box and all labeled target bounding boxes is less than the second threshold, it is set as a negative sample.

5. The method for identifying weak points of a target in a complex background by integrating cognitive maps according to claim 1, characterized in that: Set the anchor box classification loss function and the anchor box bounding box position regression loss function so that the candidate region generation network learns to distinguish target candidate regions from background regions: The binary cross entropy loss function is set as the anchor box classification loss function, and L rpn_cls express: in, is a binary label, the preset anchor box for the positive sample For the preset anchor box of negative samples is the probability that the i-th preset anchor box belongs to the foreground target, and N is the number of preset anchor boxes; Set the smooth L1 loss function as the anchor box bounding box position regression loss function, and use L rpn_reg Expressed as: L rpn_reg =Smooth L1 (x) Among them, Smooth L1 (x) represents the smooth L1 square loss function, Represents the offset t predicted by the preset anchor box of the i-th positive sample i The offset t of the preset anchor box of the i-th positive sample relative to the labeled target bounding box i * The difference.

6. The method for identifying weak points of a target in a complex background by integrating cognitive maps according to claim 5, characterized in that: Set the target candidate region multi-classification loss function and the bounding box position regression loss function; The cross entropy loss function is set as the target candidate region multi-classification loss function, and L rcnn_cls express: Among them, p i (c) represents the true value label corresponding to the i-th target candidate region, represents the classification probability that the i-th target candidate region belongs to category c. The smooth L1 loss function is set as the loss function of the target candidate area position regression task, and L rcnn_reg express: L rcnn_reg =Smooth L1 (x) in, Represents the offset t predicted by the i-th target candidate region i The offset of the i-th target candidate region relative to the labeled target bounding box The difference between The multi-task loss function for target weak part identification includes anchor box classification loss function, anchor box bounding box position regression loss function, target candidate area multi-classification loss function and bounding box position regression loss function.

7. The method for identifying weak points of a target in a complex background by integrating cognitive maps according to claim 1, characterized in that: The positive and negative samples generated by the candidate region generation network are pooled into fixed-size target candidate region features through the region of interest, and then passed through two fully connected layers to output the probability that each target candidate region belongs to different target categories and the bounding box position offset information.

8. The method for identifying weak points of a target in a complex background by integrating cognitive maps according to claim 1, characterized in that: Input the input image into the target weak part recognition model to obtain the target weak part recognition result, including the target weak part bounding box position, category and confidence score in the input image; The knowledge graph of the spatial position relationship of the target weak parts is used to assist in reasoning, and the obtained target weak parts identification results are corrected to obtain the final recognition results.

9. The method for identifying weak points of a target in a complex background by integrating cognitive maps according to claim 1, characterized in that: The auxiliary reasoning is specifically as follows: First, the constructed knowledge graph of the spatial position relationship of the target's weak parts is used to obtain the spatial geometric position distribution relationship between different weak parts and the target, as well as between different weak parts. Subsequently, based on the identification results of the target weak parts and combined with the spatial position relationship contained in the knowledge graph of the spatial relationship of the target weak parts, the missed and misdetected target weak parts are inferred and corrected.

10. A terminal device comprising a processor, a memory, and a computer program stored in the memory; characterized in that: When the processor executes the computer program, it implements the method for identifying weak points of a target under a complex background by integrating cognitive maps according to any one of claims 1 to 9.

Images