Multi-category armored car local segmentation method based on improved YOLACT

Abstract

The invention provides a multi-category armored car local segmentation method based on improved YOLACT, and relates to the technical field of target segmentation. The method comprises the following steps: firstly, labeling collected pictures by using Label software, performing data enhancement work on a labeled original picture and a json file, automatically dividing a data set according to a proportion, and constructing a multi-category armored car local segmentation data set; and establishing a multi-category armored car local segmentation detection model based on the improved YOLACT model, and training, testing and evaluating the multi-category armored car local segmentation data set by using the multi-category armored car local segmentation detection model based on the improved YOLACT model. According to the invention, an end-to-end full-convolution real-time segmentation model (YOLACT) is improved, a lightweight YOLACT based on an attention mechanism is provided, and the feature extraction capability of the segmentation model is enhanced, so that the target segmentation precision is improved, and the problem that in a complex and changeable environment, an armored vehicle cannot be subjected to target segmentation is solved. And the armored vehicle local segmentation model is poor in segmentation effect and even has false detection.

Description

Technical Field

[0001] The present invention relates to the field of target segmentation technology, and in particular to a multi-category armored vehicle local segmentation method based on improved YOLACT. Background Art

[0002] With the rapid development of artificial intelligence, deep learning has gradually penetrated the military field, enabling applications such as segmenting armored targets and rapidly retrieving and extracting the type and model of armored vehicles within images. Traditional image semantic segmentation primarily extracts phenotypic information about military equipment through texture, edge, spectrum, and length information. Fine-grained image recognition, or refined classification, involves more detailed sub-category identification of objects within the same category, identifying not only the object's category but also its specific model.

[0003] With the rapid development of deep learning, convolutional neural networks (CNNs) have been widely applied to the field of image instance segmentation. Currently, most research results, both domestically and internationally, focus on coarse-grained segmentation based on the outline and shape of armored vehicles, with less research on the feature recognition of subtle components within images. Convolutional neural networks are a type of deep learning model widely used in fields such as image processing and object detection. Compared to traditional neural networks, CNNs offer superior performance and efficiency when processing images and other two-dimensional data. The core concept of CNNs is to extract local features from the input image through convolution operations and reduce the size of the feature maps through pooling operations. Batch normalization and activation functions are often added to CNNs to enhance the model's nonlinear capabilities and stability.

[0004] The above research and analysis demonstrate that deep learning models can be applied to armored vehicle target recognition tasks. However, current research on fine-grained armored vehicle recognition on land battlefields is very limited. This limitation is reflected in the following: existing technologies only conduct local feature segmentation and fine-grained recognition research on a select number of armored vehicle types, and the local features of various armored vehicle types vary; research on armored vehicle local feature weapon classification is incomplete, mostly focusing on features such as the gun barrel, light machine gun, wheels, turret mount, tracks, and skirts, and the types of local features need to be further expanded; and there are few images of real battlefields, resulting in insufficient generalization capabilities for the algorithms studied. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to address the deficiencies of the above-mentioned prior art, provide a multi-category armored vehicle local segmentation method based on improved YOLACT, improve the end-to-end fully convolutional real-time segmentation model (YOLACT), and propose a lightweight YOLACT based on the attention mechanism to enhance the feature extraction capability of the segmentation model, thereby improving the target segmentation accuracy, and solving the technical problem that the armored vehicle local segmentation model has poor segmentation effect or even false detection in complex and changeable environments.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] A multi-category armored vehicle local segmentation method based on improved YOLACT includes the following steps:

[0008] Step 1: Data construction phase: Use GoPro sports cameras to collect armored vehicle images, build an armored vehicle dataset, and use Labelme software to annotate them;

[0009] Step 2: Data processing: The dataset is annotated with categories and masks. The armored vehicle images in the dataset are batch augmented using random flipping, random scaling, random cropping, and random rotation. The preprocessed dataset is then divided into training, validation, and test sets in a ratio of 8:1:1.

[0010] Step 3: Feature extraction stage: Establish the backbone network of the improved YOLACT multi-category armored vehicle local area segmentation model, replace the Resnet-101 of the backbone network with the lightweight CSPDarknet-53, and introduce the DA attention mechanism module into the feature extraction network of YOLACT, and integrate the DA attention mechanism with the cross-stage partial connection mechanism of the fusion focus technology characteristics in the last layer of the backbone network. By increasing the feature matrix in space and channel reshaping, capturing context dependencies, the model can extract more features. The optimized model strengthens the local and global feature extraction capabilities of key features.

[0011] Step 4: Feature fusion stage: Replace the original PANet model with a feature pyramid model with efficient feature interaction to enhance the utilization of key features, prevent false detection, and improve computing speed.

[0012] Step 5: Construct bilinear attention pooling to multiply the feature map by the attention map generated by the DA attention mechanism element by element, and then further extract more local features through convolution and pooling operations;

[0013] Step 6: Loss function: Replace the original loss function with the FIoU loss function, focusing on the feature learning of the bounding box, while improving the model convergence speed and robustness;

[0014] Step 7: Training phase: Use the training set to train the multi-category armored vehicle local area segmentation model based on the improved YOLACT, and use the validation set to adjust the model parameters to obtain the optimal weights;

[0015] Step 8: Testing phase: Use the test set to evaluate the accuracy and speed of the multi-category armored vehicle local area segmentation model based on the improved YOLACT that obtains the optimal weight.

[0016] Furthermore, in step 1, the armored vehicle dataset was shot using a GoPro sports camera at an actual weapons square, with a shooting distance of approximately 5 to 50 meters. It also includes some pictures taken from the military parade video. The format after annotation using Labelme software is json format, which is converted to txt format.

[0017] Furthermore, in step 3, the DA attention mechanism is integrated with the cross-stage partial connection mechanism of the last layer of the backbone network that integrates the focusing technology characteristics. The specific steps of reshaping the space and channels by adding feature matrices are as follows:

[0018] Step 3.1: Generate a self-attention matrix that models the spatial relationship between any two pixels in its own semantic features;

[0019] Step 3.2: Perform matrix multiplication of the self-attention matrix and the original features;

[0020] Step 3.3: Perform an element-wise sum operation on the result matrix of the multiplication above and the original features to obtain the final representation reflecting the long-distance context.

[0021] Furthermore, in step 4, the FPT efficient feature interaction model is a feature fusion network introduced to address the false detection of local areas of armored vehicles. The output features of different layers in the backbone network are extracted to obtain a corresponding feature pyramid network. According to the rules of the self-transformer, the top-down non-local interactor, and the bottom-up non-local interactor, the feature maps corresponding to the context information of different scales are respectively used. First, a self-attention map is generated using the self-transformer, and then a channel self-attention operation is applied to the deep feature map to obtain a feature map. Finally, a feature map is obtained by applying the bottom-up non-local interactor to the shallow feature. The feature map is reordered according to size, and a splicing operation is performed with the original pyramid feature. The dimension is reduced through the convolution layer to finally obtain the FPT feature.

[0022] Furthermore, the step 5 specifically includes:

[0023] First, bilinear interpolation is used to adjust the attention map to the same scale as the feature map; then each attention map A is used KMultiply the texture feature map F by elements to get the partial texture feature map F k ; Design a standardized average pooling, the formula is as follows:

[0024] (1);

[0025] Among them, F is the feature map extracted by CNN from the image. It refers to the feature map corresponding to the kth attention area, m is the height value of the spatial dimension, and n is the width value of the spatial dimension; It refers to the feature corresponding to the k-th attention area with a width of n and a height of m in the spatial dimension; It refers to the feature vector aggregated by the kth attention region, where k is the index of the attention region; and Represent the height and width of the overall feature space respectively; Represents the Euclidean norm, which calculates the distance between two points;

[0026] By superimposing V k Get the texture feature matrix , and then the texture feature matrix is ​​sent to the classifier to obtain the category information, where R represents the real number field, M is the number of rows of the texture feature matrix, and each row corresponds to the feature vector of an attention area ; Refers to the number of columns of the texture feature matrix, representing the characteristics of each attention area Dimensions, Depends on the number of channels in the last layer of the ResNet101 network used.

[0027] Furthermore, in step 6, the FIoU loss function is as follows:

[0028] (2);

[0029] (3);

[0030] in, It represents the intersection over union (IoU), which is used to measure the degree of overlap between the predicted box and the true labeled box. It is expressed by calculating the ratio of the intersection area of ​​the two boxes to the union area. The value range of IoU is between 0 and 1. The closer it is to 1, the higher the degree of overlap between the two boxes, and the closer it is to 0, the lower the degree of overlap between the two boxes. , by adjusting the lower limit of the scaling factor d and the upper limit of the scaling factor u, FIoU is focused on the multi-scale regression samples.

[0031] Furthermore, in the training of step 7, the evaluation indicators include the precision rate of each category, the recall rate, the precision rate AP, the average precision rate mAP of multiple categories, and the detection frame rate FPS; the precision rate AP represents the average accuracy, which refers to the degree of overlap between the objects detected by the model and the real objects, and is used to measure the detection performance of the model for a single category; the average precision rate mAP of multiple categories represents the mean average precision, which is used to measure accuracy and evaluate the overall effect of the model in complex tasks. The higher the mAP, the better the performance of the model on multiple categories; the detection frame rate FPS stands for frames per second, which is one of the important indicators for measuring system performance. FPS represents the number of image frames processed in one second;

[0032] The formulas for each evaluation index are as follows:

[0033] (4);

[0034] (5);

[0035] (6);

[0036] (7);

[0037] (8);

[0038] Among them, TP is true positive, which means the number of correctly detected targets; FP is false positive, which means the number of non-targets that are mistakenly detected as targets; FN is false negative, which means the number of undetected targets; P(R) represents the Precision-Recall curve, where R represents the recall rate; N represents the total number of categories; AP i It represents the precision of the i-th target, and its value is equal to the area covered by the Precision-Recall function and the coordinate axis; is the preprocessing time for each image, is the inference time for each image, is the non-maximum suppression processing time for each image. The sum of these three times is the time required for each image detection, which is used to evaluate the processing speed of the model.

[0039] The beneficial effects of adopting the above technical solution are: the multi-category armored vehicle local segmentation method based on the improved YOLACT provided by the present invention uses data enhancement and local enhancement to improve the generalization ability of the model; the backbone network is replaced with CSPDarkNet-53, and the DA-Attention mechanism module is introduced into the feature extraction network of YOLACT, and the dimension is reduced by the convolution structure, and the DA-Attention is aligned with the C2f dimension of the C5 layer of the backbone network. The global features are captured through spatial and channel attention and long-distance dependencies are modeled to improve the feature extraction ability of the model; in order to improve the accuracy of the model, FPT is introduced The feature fusion network combines instance-level information and underlying fine-grained semantic information to achieve the effect of local and global fusion; the BAP bilinear attention pool is used to fuse the extracted deep channel features and deep spatial features, so that the channel features and spatial features are associated and interacted, so that the network can learn richer local image features; finally, the learned local features are spliced ​​with the global features extracted by the deep neural network, and the fully connected layer is used to obtain the final image fusion features for fine-grained classification of armored vehicles; high-definition cameras are used to collect image data with high data quality, and some real battlefield shots and video screenshots of military parades are mixed in to improve generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Flowchart of the multi-category armored vehicle local segmentation method based on the improved YOLACT provided by an embodiment of the present invention;

[0041] Figure 2 This is a diagram of the original YOLACT model structure provided by an embodiment of the present invention;

[0042] Figure 3 A structural diagram of an armored vehicle local area segmentation model based on an improved YOLACT model provided in an embodiment of the present invention;

[0043] Figure 4 This is a diagram of the new DA-CSPDarknet-53 structure proposed in an embodiment of the present invention;

[0044] Figure 5 1 is a structural diagram of the FPT feature fusion module used in an embodiment of the present invention;

[0045] Figure 6 It is a structural diagram of the BAP bilinear pooling module provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0047] like Figure 1 As shown, the method of this embodiment is as follows.

[0048] Step 1: Data construction phase. Use GoPro sports cameras to collect armored vehicle images, build an armored vehicle dataset, and use Labelme software to annotate them. The annotated format is JSON, which is then converted to txt format.

[0049] The data used in this example is a self-constructed dataset, captured at a military plaza using a GoPro sports camera at a distance of approximately 5 to 50 meters. In addition, some images are taken from military parade videos. The total number of image samples is 2,000, encompassing images of varying lighting intensities and angles, with some images experiencing background interference and partial occlusion.

[0050] Step 2: Data processing: Batch data augmentation is performed on the armored vehicle images in the dataset using random flipping, random scaling, random cropping, and random rotation. The preprocessed dataset is then divided into training, validation, and test sets.

[0051] First, the dataset was preprocessed. The specific process was as follows: Based on the armored vehicle local region instance segmentation dataset constructed in Step 1, the dataset was annotated with categories and masks. The dataset includes images of four types of armored vehicles: tanks, guns, missiles, and transporters. Tanks include the Type 59 tank and the ZTZ99 tank; guns include the Type 89 120mm self-propelled anti-tank gun, the PLZ-05 self-propelled howitzer, and the PL009 122mm wheeled self-propelled howitzer; missiles include the DF-2 ballistic missile and the HQ-713 air defense missile; and transporters include the Type 63 amphibious armored transporter and the Type 63 armored transporter. The annotated images were directly processed using Torch's DataTransformer to perform batch data augmentation on the images in the dataset by random flipping, scaling, cropping, and rotation. The dataset was then split into training, validation, and test sets in an 8:1:1 ratio.

[0052] Step 3: Feature extraction stage. The original YOLACT model is as follows Figure 2 As shown, this embodiment improves it and establishes an improved YOLACT multi-category armored vehicle local area segmentation model (such as Figure 3As shown in the figure, the model has a total of 22 layers) in the backbone network, replacing the Resnet-101 of the backbone network with the lightweight CSPDarknet-53, and introducing the DA attention mechanism module into the feature extraction network of YOLACT, and integrating the DA attention mechanism with the cross-stage partial connection mechanism C2f that integrates the focus technology characteristics in the last layer of the backbone network. By increasing the feature matrix in space and channel reshaping and capturing contextual dependencies, the model can extract more features. The optimized model enhances the local and global feature extraction capabilities of key features.

[0053] The CSPDarkNet-53 network is a lightweight backbone network that makes the model training faster. Since armored vehicles are in complex environments, there will be local occlusions and similarity confusion. The introduction of the DA attention mechanism resists confusing information and focuses on useful target information. By increasing the feature matrix and reshaping it in space and channels, the backbone network extracts more comprehensive features. Figure 4 As shown in the figure, the specific contents are as follows: In the DA attention mechanism module, the input passes through the channel attention module and the position attention module in sequence. The position attention module selectively aggregates the features of each position by weighted summing the features of all positions. Similar features will be related to each other regardless of distance. The channel attention module selectively emphasizes interdependent channel maps by integrating the relevant features between all channel maps. Adding the outputs of the two attention modules further improves the feature representation, which contributes to more accurate segmentation results. When the armored vehicle image is input to the backbone network, the input size of SPPF is [680, 680, 3, 2], which can be viewed in the output onnx file. Therefore, a set of random numbers of size [680, 680, 3, 2] is randomly generated and allowed to pass through each node, passing through the original Resnet-101 and DA-CSPDarknet-53 respectively, for 500 rounds. Experiments show that DA-CSPDarknet-53 has faster inference speed. At the same time, the mAP value verified by DA-CSPDarknet-53 reaches 94.5%, which is 2.8% higher than the original YOLACT model, which shows the advantages of DA-CSPDarknet-53 in speed and accuracy.

[0054] Step 4: Feature fusion stage: The original PANet model is replaced with the FPT model to enhance the utilization of key features, prevent false detection, and improve computing speed.

[0055] The FPT efficient feature interaction module is a feature fusion network introduced to target the misdetection of local areas of armored vehicles. Figure 5 As shown in Figure 1, an image is input and a feature pyramid can be constructed through the CNN network. The fine-grained and coarse-grained feature maps are at the low and high levels respectively. The low-level fine-grained feature map is represented as Xf , the high-level coarse-grained feature map is represented as X c . The feature pyramid is converted into a feature pyramid with richer contextual information through the same-level interaction module, the top-down interaction module and the bottom-up interaction module. FPT enables features to interact across space and scales. It is divided into ST (Self-Transformer), GT (Grounding-Transformer) and RT (Rendering-Transformer). First, ST, GT and RT feature enhancements are performed on each layer of features to obtain multiple enhanced features. Then, the enhanced features are sorted by size, and features of the same size are grouped together. The enhanced feature dimensions are restored to the same dimensions as the input through convolution.

[0056] The input of the same-level interaction operation is a single feature map X, and the matrices are 、 and ,in, , , , 、 、 For the corresponding linear transformation, first transform and Divide into N parts, and then calculate each group q in each part i,n and k j,n The similarity score s ij n ,Similarity calculation uses the Softmax function as the normalization function, which has lower memory and time costs compared to other normalization function methods.

[0057] GT is a top-down non-local operation that uses high-level coarse-grained features X c To enhance the low-level fine-grained features X f During calculation, the similarity calculation is replaced by the dot product with the more efficient Euclidean distance F eud , the formula is as follows:

[0058] ;

[0059] In contrast to GT, RT is a bottom-up, non-local operation that leverages lower-level, fine-grained features to enhance higher-level, coarse-grained features. RT is also locally constrained and computed channel-wise, defining high-level features as Q and low-level features as K and V.

[0060] Step 5: Construct BAP bilinear attention pooling to multiply the feature map with the attention map generated by the DA attention mechanism element by element, and then further extract more local features through convolution and pooling operations to improve the model detection accuracy.

[0061] The BAP bilinear attention pooling module is designed for the fine-grained classification of armored vehicles. It uses BAP bilinear attention pooling instead of global average pooling. Its structure is as follows: Figure 6 As shown. The Bayesian probability method is used to collect texture feature matrices from the shallow layer and retain semantic features from the deep layer. The network of the multi-attention structure lacks regional level labels and can only be trained using unsupervised or weak supervision, which will cause network degradation problems. To solve this problem, the regional independence loss is proposed. In order to extract shallow texture features, the attention map is first adjusted to the same scale as the feature map using bilinear interpolation, and then each attention map A is used separately. K Multiply the texture feature map F by elements to get the partial texture feature map F k Considering the differences between different regions, if traditional global average fusion is used, the fused feature vector will be affected by the strength of the attention map, which goes against the purpose of focusing on texture information. To solve this problem, this embodiment designs a standardized average pooling. The overall process is shown in formula (1):

[0062] (1);

[0063] Among them, F is the feature map extracted by CNN from the image. It refers to the feature map corresponding to the kth attention area, m is the height value of the spatial dimension, and n is the width value of the spatial dimension; It refers to the feature corresponding to the k-th attention area with a width of n and a height of m in the spatial dimension; It refers to the feature vector aggregated by the kth attention region, where k is the index of the attention region; and Represent the height and width of the overall feature space respectively; Represents the Euclidean norm, which calculates the distance between two points.

[0064] By superimposing V k Get the texture feature matrix Finally, the texture feature matrix is ​​sent to the classifier to obtain the category information. Among them, R represents the real number field, M is the number of rows of the texture feature matrix, and each row corresponds to the feature vector of an attention area. ; Refers to the number of columns of the matrix, representing the characteristics of each attention area Dimensions, It depends on the number of channels in the last layer of the ResNet101 network used. Since the number of channels in the last layer in this embodiment is 2048, The value is 2048.

[0065] The information from each feature map is dot-multiplied with the information from the attention map, that is, the corresponding matrices are multiplied. This results in 32 feature maps, generating a total of 32 × 1600 = 51,200 images. Global average pooling is then performed on these 51,200 images to generate 51,200 values. A batch size is N samples, resulting in a feature matrix of dimension N × 51,200. After the dot-multiplication, the features are directly added together and then normalized. Because the generated features after normalization are too small, the pooled normalized features are amplified 100 times for subsequent processing. The pooled normalized features are used to calculate the center loss function.

[0066] Step 6: Loss function. Replace the original loss function with the FIoU loss function, focusing on bounding box feature learning, while improving the model convergence speed and robustness.

[0067] The Intersection over Union (IoU) measures the degree of overlap between the predicted box and the ground-truth box. It is expressed as the ratio of the intersection area of ​​the two boxes to the union area. IoU values ​​range from 0 to 1, with values ​​closer to 1 indicating a higher degree of overlap between the two boxes, and values ​​closer to 0 indicating a lower degree of overlap. Confidence is a probability value derived from the model's judgment of whether an object exists within each box. Generally speaking, confidence values ​​range from 0 to 1, with values ​​closer to 1 indicating a greater probability of an object existing within the box, and values ​​closer to 0 indicating uncertainty about the existence of an object. Given the same IoU (Intersection over Union) and Conf (Confidence), the original YOLACT model misses detections, while the improved YOLACT segmentation is more accurate, with a higher confidence score for each detected instance. A higher confidence score means a more accurate model generates results, indicating that FIoU can help improve model accuracy.

[0068] In step 6, the FIoU loss function, in order to make the anchor box cover the target object more accurately, uses FIoU to regress the anchor box. It not only considers the spatial overlap between the anchor box and the ground-truth box, but also introduces feature similarity as an adjustment basis. When adjusting the size, shape, and position of the anchor box, the algorithm also considers how to maximize the similarity between the features inside the anchor box and the features of the target object. This comprehensive consideration allows the anchor box to more accurately locate and cover the target, especially for objects with complex shapes or specific texture features. The FIoU loss function is calculated as follows:

[0069] (2);

[0070] (3);

[0071] in, , by adjusting the lower limit of the scaling factor d and the upper limit of the scaling factor u, FIoU is focused on the multi-scale regression samples.

[0072] FIoU-optimized anchor box selection and regression adjustments improve model training efficiency and final detection performance. More accurate anchor boxes mean faster model convergence, reducing training time and resource consumption. Furthermore, improved detection accuracy directly impacts the model's usability and reliability in practical applications. By incorporating feature similarity evaluation into the anchor box selection and adjustment process, FIoU provides a more refined and efficient optimization method for object segmentation algorithms. This approach not only improves detection accuracy but also enhances the model's capabilities in complex scene processing and small object detection.

[0073] Combine Figure 2 and Figure 3 The specific process is as follows: In the Ubuntu 20.04, CUDA 12.2, Cudnn 9.0.0 and 3050 GPU server environment, epochs are set to 500, batch size = 2, image input size is 680 × 680, and a multi-category armored vehicle local segmentation model based on the improved YOLACT model is trained. The validation set is used to adjust the model parameters during the training process to finally obtain the optimal weights. Figure 2 This is the network structure diagram of the original YOLACT. Figure 3 This is a 22-layer architecture diagram for a multi-class armored vehicle local segmentation detection model based on the improved YOLACT model. It employs four strategies, including a new C2f-DA module, the introduction of the FPT feature pyramid module, the BAP bilinear attention pooling module, and the replacement of the loss function with the Fibonacci over Union (FIoU) loss function. These strategies address the low accuracy of multi-class armored vehicle local segmentation and improve the overall performance of the model.

[0074] Step 7: Training phase: Use the training set to train the multi-category armored vehicle local area segmentation model based on the improved YOLACT, and use the validation set to adjust the model parameters and obtain the optimal weights.

[0075] Evaluation indicators include the precision P (Precision), recall R (Recall), precision AP, average accuracy mAP of multiple categories, and detection frame rate FPS (FramesPerSecond) for each category. Precision Precision refers to the proportion of samples predicted by the model as positive that are actually positive, and recall Recall refers to the proportion of samples correctly predicted by the model as positive to all samples that are actually positive. AP (AveragePrecision) represents the average precision, which refers to the degree of overlap between the objects detected by the model and the real objects. It can be used to measure the detection performance of the model for a single category. mAP (MeanAveragePrecision) is a comprehensive indicator that represents the average precision. It is the average of the average precision of all categories, that is, the area under the precision-recall curve of each category. It is used to measure accuracy and evaluate the overall effect of the model in complex tasks. The higher the mAP, the better the performance of the model on multiple categories. FPS stands for frames per second. It is one of the important indicators for measuring system performance. FPS represents the number of image frames processed in one second. The formula is as follows:

[0076] (4);

[0077] (5);

[0078] (6);

[0079] (7);

[0080] (8);

[0081] There are four possible local feature segmentation methods for armored vehicles: TP (True Positive), FP (False Positive), TN (True Negative), and FN (False Negative). TP (True Positive) indicates the number of correctly detected targets; FP (False Positive) indicates the number of non-targets that are incorrectly detected as targets; and FN (False Negative) indicates the number of undetected targets. P(R) represents the Precision-Recall curve, where R represents the recall rate; N represents the total number of categories; and AP represents the number of undetected targets. i It represents the precision of the i-th target, and its value is equal to the area covered by the Precision-Recall function and the coordinate axis. is the preprocessing time for each image, is the inference time for each image, is the non-maximum suppression processing time for each image. The sum of these three times is the time required for each image detection, which is used to evaluate the processing speed of the model.

[0082] Step 8: Testing phase: Use the test set to evaluate the accuracy and speed of the multi-category armored vehicle local area segmentation model based on the improved YOLACT with optimal weights.

[0083] The experiments on the test set are mainly divided into two parts: ablation experiments and improvement comparison experiments.

[0084] Ablation experiment is a scientific research method, which is usually used to determine the contribution of a specific component in a system to the overall function or performance. In this experiment, by gradually removing or isolating certain elements or variables in the system, the impact on the overall system is observed in order to understand the role and importance of these elements or variables. This embodiment is based on the four strategies adopted in the armored vehicle local feature segmentation model of improving YOLACT, adding them individually and in combination to the YOLACT network model to observe the impact of each component on the overall network. The results of the ablation experiment show that the mAP values ​​of the four strategies adopted are improved when acting alone on the YOLACT model. The most obvious improvement in accuracy is achieved when combining two or three strategies. When the four strategies are mixed together, the mAP value is the best, which is 96.2%, an increase of 4.5% over the original YOLACT model.

[0085] An improved comparative experiment compared the armored vehicle local feature segmentation model based on the improved YOLACT with other advanced models on the armored vehicle local feature segmentation test set. The armored vehicle local feature segmentation model based on the improved YOLACT achieved the best mean average performance (mAP) of 96.2%. The model is lightweight and has fast inference speed. In summary, the armored vehicle local feature segmentation model based on the improved YOLACT performs best overall in solving the problem of extracting local features of armored vehicles and performing fine-grained recognition.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the present invention.

Claims

1. A multi-category armored vehicle local segmentation method based on improved YOLACT, characterized by: The following steps are involved: Step 1: Data construction phase: Use GoPro sports cameras to collect armored vehicle images, build an armored vehicle dataset, and use Labelme software to annotate them; Step 2: Data processing: The dataset is annotated with categories and masks. The armored vehicle images in the dataset are batch augmented using random flipping, random scaling, random cropping, and random rotation. The preprocessed dataset is then divided into training, validation, and test sets in a ratio of 8:1:

1. Step 3: Feature extraction stage: Establish the backbone network of the improved YOLACT multi-category armored vehicle local area segmentation model, replace the Resnet-101 of the backbone network with the lightweight CSPDarknet-53, and introduce the DA attention mechanism module into the feature extraction network of YOLACT, and integrate the DA attention mechanism with the cross-stage partial connection mechanism of the fusion focus technology characteristics in the last layer of the backbone network. By increasing the feature matrix in space and channel reshaping, capturing context dependencies, the model can extract more features. The optimized model strengthens the local and global feature extraction capabilities of key features. Step 4: Feature fusion stage: Replace the original PANet model with a feature pyramid model with efficient feature interaction to enhance the utilization of key features, prevent false detection, and improve computing speed. Step 5: Construct bilinear attention pooling to multiply the feature map by the attention map generated by the DA attention mechanism element by element, and then further extract more local features through convolution and pooling operations; Step 6: Loss function: Replace the original loss function with the FIoU loss function, focusing on the feature learning of the bounding box, while improving the model convergence speed and robustness; Step 7: Training phase: Use the training set to train the multi-category armored vehicle local area segmentation model based on the improved YOLACT, and use the validation set to adjust the model parameters to obtain the optimal weights; Step 8: Testing phase: Use the test set to evaluate the accuracy and speed of the multi-category armored vehicle local area segmentation model based on the improved YOLACT that obtains the optimal weight.

2. The multi-category armored vehicle local segmentation method based on improved YOLACT according to claim 1 is characterized by: In step 1, the armored vehicle dataset was shot using a GoPro sports camera at an actual weapons square at a distance of approximately 5 to 50 meters. It also includes some images captured from a military parade video. The images were annotated using Labelme software in JSON format, which was then converted to txt format.

3. The multi-category armored vehicle local segmentation method based on improved YOLACT according to claim 1 is characterized by: In step 3, the DA attention mechanism is integrated with the cross-stage partial connection mechanism of the last layer of the backbone network that integrates the focusing technology characteristics. The specific steps of reshaping in space and channels by adding feature matrices are as follows: Step 3.1: Generate a self-attention matrix that models the spatial relationship between any two pixels in its own semantic features; Step 3.2: Perform matrix multiplication of the self-attention matrix and the original features; Step 3.3: Perform an element-wise sum operation on the result matrix of the multiplication above and the original features to obtain the final representation reflecting the long-distance context.

4. The multi-category armored vehicle local segmentation method based on improved YOLACT according to claim 3 is characterized by: In step 4, the FPT efficient feature interaction model is a feature fusion network introduced to address the false detection of local areas of armored vehicles. The output features of different layers in the backbone network are extracted to obtain a corresponding feature pyramid network. Feature maps corresponding to contextual information of different scales are respectively generated according to the rules of the self-transformer, the top-down non-local interactor, and the bottom-up non-local interactor. First, a self-attention map is generated using the self-transformer, and then a channel self-attention operation is applied to the deep feature map to obtain a feature map. Finally, a feature map is obtained by applying the bottom-up non-local interactor to the shallow feature. The feature map is reordered according to size and concatenated with the original pyramid feature. The dimension is reduced through the convolution layer to finally obtain the FPT feature.

5. The multi-category armored vehicle local segmentation method based on improved YOLACT according to claim 4 is characterized by: The step 5 specifically includes: First, bilinear interpolation is used to adjust the attention map to the same scale as the feature map; then each attention map A is used K Multiply the texture feature map F by elements to get the partial texture feature map F k ; Design a standardized average pooling, the formula is as follows: （1）； Among them, F is the feature map extracted by CNN from the image. It refers to the feature map corresponding to the kth attention area, m is the height value of the spatial dimension, and n is the width value of the spatial dimension; It refers to the feature corresponding to the k-th attention area with a width of n and a height of m in the spatial dimension; It refers to the feature vector aggregated by the kth attention region, where k is the index of the attention region; and Represent the height and width of the overall feature space respectively; Represents the Euclidean norm, which calculates the distance between two points; By superimposing V k Get the texture feature matrix , and then the texture feature matrix is ​​sent to the classifier to obtain the category information, where R represents the real number field, M is the number of rows of the texture feature matrix, and each row corresponds to the feature vector of an attention area ; Refers to the number of columns of the texture feature matrix, representing the characteristics of each attention area Dimensions, Depends on the number of channels in the last layer of the ResNet101 network used.

6. The multi-category armored vehicle local segmentation method based on improved YOLACT according to claim 5 is characterized by: In step 6, the FIoU loss function is as follows: （2）； （3）； in, It represents the intersection over union (IoU), which is used to measure the degree of overlap between the predicted box and the true labeled box. It is expressed by calculating the ratio of the intersection area of ​​the two boxes to the union area. The value range of IoU is between 0 and 1. The closer it is to 1, the higher the degree of overlap between the two boxes, and the closer it is to 0, the lower the degree of overlap between the two boxes. , by adjusting the lower limit of the scaling factor d and the upper limit of the scaling factor u, FIoU is focused on the multi-scale regression samples.

7. The multi-category armored vehicle local segmentation method based on improved YOLACT according to claim 6 is characterized by: In the training of step 7, the evaluation indicators include the precision rate of each category, the recall rate, the precision rate AP, the average precision rate mAP of multiple categories, and the detection frame rate FPS; the precision rate AP represents the average precision, which refers to the degree of overlap between the objects detected by the model and the real objects, and is used to measure the detection performance of the model for a single category; the average precision rate mAP of multiple categories represents the mean average precision, which is used to measure accuracy and evaluate the overall effect of the model in complex tasks. The higher the mAP, the better the performance of the model on multiple categories; the detection frame rate FPS stands for frames per second, which is one of the important indicators for measuring system performance. FPS represents the number of image frames processed in one second; The formulas for each evaluation index are as follows: （4）； （5）； （6）； （7）； （8）； Among them, TP is true positive, which means the number of correctly detected targets; FP is false positive, which means the number of non-targets that are mistakenly detected as targets; FN is false negative, which means the number of undetected targets; P(R) represents the Precision-Recall curve, where R represents the recall rate; N represents the total number of categories; AP i It represents the precision of the i-th target, and its value is equal to the area covered by the Precision-Recall function and the coordinate axis; is the preprocessing time for each image, is the inference time for each image, is the non-maximum suppression processing time for each image. The sum of these three times is the time required for each image detection, which is used to evaluate the processing speed of the model.

Images