A method, system, medium and electronic device for semantic segmentation of camouflaged objects based on decision-level feature fusion modeling

Abstract

The present invention provides a camouflaged object semantic segmentation method based on decision-level feature fusion modeling, characterized in that the method comprises: S1 inputting an image to be segmented, using a camouflaged object image with pixel-level labels to supervise the model training, and obtaining the multi-scale feature extraction result of the image to be segmented through the supervised training; S2 based on the multi-scale feature extraction result, the camouflaged object in the image to be segmented is processed by global feature extraction and edge feature guidance to obtain high-resolution mask features; S3 based on the high-resolution mask feature result, the decision-level feature fusion method is used to focus on the high-resolution mask features, and the prediction result of the model is output. The present invention can comprehensively and accurately predict camouflaged objects.

Description

Technical Field

[0001] The present invention belongs to the technical field of image segmentation, and in particular relates to a method, system, medium and electronic equipment for semantic segmentation of camouflaged objects based on decision-level feature fusion modeling. Background Art

[0002] According to relevant research by biologists, camouflaged organisms have a high degree of similarity with their surroundings, which makes the related visual tasks of camouflaged objects more challenging than those of ordinary objects. In recent years, with the development of camouflaged object detection, the semantic segmentation task of camouflaged objects has also gradually developed.

[0003] Currently, the task of semantic segmentation of camouflaged objects requires the algorithm model to be able to understand the camouflaged semantic information in the image and segment the area to which the camouflaged object belongs from the image.

[0004] At the same time, the study found that with the application of decision-level feature fusion modeling technology, extracting detailed information of camouflaged objects in images at different scales, including texture, shape, and edges, can further enhance the algorithm model's ability to understand information about camouflaged objects, thereby improving the model's segmentation performance. Summary of the invention

[0005] In order to address the deficiencies of the prior art, the present invention provides a method, system, medium and electronic device for semantic segmentation of camouflaged objects based on decision-level feature fusion modeling.

[0006] The technical solution of the present invention is as follows:

[0007] A method for semantic segmentation of camouflaged objects based on decision-level feature fusion modeling, the method comprising:

[0008] S1 inputs an image to be segmented, uses a camouflaged object image with pixel-level labels to perform supervised training on the model, and obtains a multi-scale feature extraction result of the image to be segmented through the supervised training;

[0009] S2, based on the extraction result of the multi-scale features, the camouflaged object in the segmented image is processed by using global feature extraction and edge feature guidance to obtain a high-resolution mask feature;

[0010] S3 focuses on the high-resolution mask features based on the high-resolution mask features using a decision-level feature fusion method and outputs a prediction result of the model.

[0011] Furthermore, the S1 is specifically:

[0012] Use pyramid-based Transformer backbone network and convolutional neural backbone network to extract multi-scale features of the segmented image;

[0013] Specifically, the multi-scale feature extraction of the segmented image using the pyramid-based Transformer backbone network includes:

[0014] Perform a convolution layer with a kernel size of 1*1 on the image to be segmented and output the first feature map;

[0015] Performing two-fold downsampling and two PVT-V2 basic blocks on the first feature map, and outputting a second feature map;

[0016] Performing two-fold downsampling and two PVT-V2 basic blocks on the second feature map, outputting a third feature map;

[0017] Performing two-fold downsampling and 6 PVT-V2 basic blocks on the third feature map, and outputting a fourth feature map;

[0018] Performing two-fold downsampling and four PVT-V2 basis blocks on the fourth feature map, and outputting a fifth feature map;

[0019] The PVT-V2 basic block includes: a first multi-head self-attention module based on pooling, a first adder, a first normalization layer, a feedforward network, a second adder, and a second normalization layer connected in sequence;

[0020] Using a convolutional neural network backbone to extract multi-scale features from the segmented image includes:

[0021] Perform a convolution layer with a kernel size of 1*1 and a Res2Net basic block on the segmented image, and output the sixth feature map;

[0022] Performing two-fold downsampling and three Res2Net basic blocks on the sixth feature map, and outputting a seventh feature map;

[0023] Performing two-fold downsampling and four Res2Net basic blocks on the seventh feature map, and outputting an eighth feature map;

[0024] Performing two-fold downsampling and 6 Res2Net basic blocks on the eighth feature map, and outputting a ninth feature map;

[0025] Performing two-fold downsampling and three Res2Net basic blocks on the ninth feature map, and outputting a tenth feature map;

[0026] Among them, the Res2Net basic block includes: a convolutional layer with a convolution kernel size of 1*1 based on residual connection, a convolutional layer with a convolution kernel size of 3*3 based on residual connection, and a convolutional layer with a convolution kernel size of 1*1 based on residual connection.

[0027] Furthermore, the S2 is specifically:

[0028] Based on the second feature map, the third feature map, the fourth feature map, the fifth feature map, the seventh feature map, the eighth feature map, the ninth feature map and the tenth feature map, input into the global feature extraction GCAM module through a splicing operation, and obtain a second high-quality mask map, a third high-quality mask map, a fourth high-quality mask map, a fifth high-quality mask map, a sixth high-quality mask map and a seventh high-quality mask map of corresponding sizes;

[0029] Based on the second feature map, the third feature map, the fourth feature map, the fifth feature map, the seventh feature map, the eighth feature map, the ninth feature map and the tenth feature map, the concatenation operation is input into the boundary-guided BGM module, and detailed feature supervision training is performed to guide the feature aggregation of the GCAM module through back propagation.

[0030] Furthermore, the S3 is specifically:

[0031] Inputting the high-quality mask image into the MFM decision-level feature fusion module for multi-channel and multi-scale fusion to obtain fused features;

[0032] The fused features are then connected in sequence to form a dual attention layer, a convolution layer with a convolution kernel size of 3*3 and a void coefficient of 1, and then concatenated with the residual branch to obtain the final semantic segmentation prediction result of the camouflaged object.

[0033] The present invention also provides a camouflaged object semantic segmentation system based on decision-level feature fusion modeling, comprising:

[0034] A feature extraction module is used to input an image to be segmented, and use a camouflaged object image with pixel-level labels to perform supervised training on the model, thereby obtaining a multi-scale feature extraction result of the image to be segmented through the supervised training;

[0035] A multi-scale feature processing module, based on the multi-scale feature extraction result, processes the camouflaged object in the segmented image by using global feature extraction and edge feature guidance to obtain a high-resolution mask feature;

[0036] The model output prediction module is used to focus on the high-resolution mask feature based on the high-resolution mask feature result by using a decision-level feature fusion method to output the prediction result of the model.

[0037] Furthermore, it is characterized in that the multi-scale feature acquisition module is composed of a pyramid Transformer backbone network and a backbone network based on a convolutional neural network;

[0038] The pyramid Transformer backbone network includes: a first embedding block, a second embedding block, a third embedding block, a fourth embedding block and a fifth embedding block connected in sequence;

[0039] The backbone of the convolutional neural network includes: a first convolution block, a second convolution block, a third convolution block, a fourth convolution block and a fifth convolution block which are connected in sequence.

[0040] Further, the multi-scale feature processing module includes a global feature extraction GCAM module and a boundary guidance BGM module;

[0041] The global feature extraction GCAM module includes:

[0042] One residual branch and four side branches;

[0043] Among them, the four side branches are in parallel relationship with each other;

[0044] Wherein, the four side branches include: a first side branch, a second side branch, a third side branch and a fourth side branch;

[0045] The first side branch includes: a convolution layer with a convolution kernel size of 3*3 and a hole coefficient of 1, and a residual convolution layer with a convolution kernel size of 3*3, and then obtaining a first global feature through an upsampling operation;

[0046] The second side branch includes: a convolution layer with a convolution kernel size of 3*3 and a dilation coefficient of 2, and a residual convolution layer with a convolution kernel size of 3*3, and then obtaining a second global feature through an upsampling operation;

[0047] The third side branch includes: a convolution layer with a convolution kernel size of 3*3 and a dilation coefficient of 3, and a residual convolution layer with a convolution kernel size of 3*3, and then obtaining a third global feature through an upsampling operation;

[0048] The fourth side branch includes: sequentially connected to a convolution layer with a convolution kernel size of 3*3 and a dilation coefficient of 4, a residual convolution layer with a convolution kernel size of 3*3 and an element addition operation, and then obtaining a fourth global feature through an upsampling operation;

[0049] The residual branch includes: a convolution layer with a convolution kernel size of 3*3 and a splicing operation;

[0050] The BGM module comprises: a first input terminal, a second input terminal, a third input terminal and a fourth input terminal and a plurality of up-sampling operations;

[0051] Wherein, the first input end is used to input the first characteristic map and the seventh characteristic map;

[0052] The second input terminal is used to input the second characteristic map and the eighth characteristic map;

[0053] The third input terminal is used to input the third characteristic map and the ninth characteristic map;

[0054] The fourth input terminal is used to input the fourth characteristic map and the tenth characteristic map.

[0055] Furthermore, the model output prediction module includes an MFM module and a dual attention layer:

[0056] The MFM module comprises: a residual branch and three side branches;

[0057] Among them, the three side branches are in parallel relationship with each other;

[0058] Wherein, the three side branches include: a first side branch, a second side branch and a third side branch;

[0059] The first side branch includes: sequentially connected to a convolution layer with a convolution kernel size of 3*3 and a stacking coefficient of 1, a split operation, and then a splicing operation to obtain a first fusion feature;

[0060] The second side branch includes: sequentially connected to convolution layers with a convolution kernel size of 3*3 and a dilation coefficient of 2, a splitting operation, and then a splicing operation to obtain a second fusion feature;

[0061] The third side branch includes: sequentially connected to convolution layers with a convolution kernel size of 3*3 and a dilation coefficient of 3, a splitting operation, and then a splicing operation to obtain a third fusion feature;

[0062] The residual branch includes: three convolution layers with a convolution kernel size of 3*3 and a splicing operation;

[0063] The first fusion feature, the second fusion feature and the third fusion feature are sequentially connected to form a dual attention layer, a convolution layer with a convolution kernel size of 3*3 and a void coefficient of 1, an upsampling operation, and then a splicing operation is performed with the residual branch to obtain the final camouflaged object semantic segmentation prediction result;

[0064] The dual attention layer includes: a convolution layer with a convolution kernel size of 3*3 and a void coefficient of 1, a group normalization operation, an average pooling operation, a Softmax activation layer, a matrix multiplication operation, a Sigmoid activation layer and an element multiplication operation.

[0065] The present invention also provides a camouflaged object semantic segmentation device based on decision-level feature fusion modeling, characterized in that it includes a processor and a storage medium;

[0066] The storage medium is used to store instructions;

[0067] The processor may operate according to the instructions to execute the steps of the above method.

[0068] The present invention also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented.

[0069] Technical effects of the present invention:

[0070] The above-mentioned camouflaged object semantic segmentation method based on decision-level feature fusion modeling includes the following operations: inputting an image to be segmented, using a camouflaged object image with pixel-level labels to supervise the model training, and obtaining multi-scale feature extraction results based on convolutional neural networks and Transformer for the image to be segmented through supervised training; based on the multi-scale feature extraction results, processing the camouflaged object in the image to be segmented using a multi-channel spanning aggregation method to obtain a high-resolution mask feature result; based on the mask feature result, combining local and global features using a decision-level feature fusion method, and finally outputting the predicted segmentation result of the model.

[0071] In the above operation process, when mining camouflaged objects in images, the user first provides an image, and the algorithm will segment the pixel area containing the camouflaged objects in the image. Specifically, the pyramid Transformer and convolutional neural network are used because multi-scale feature extraction helps the model better explore the information of camouflaged objects in the image; the multi-channel cross-aggregation method is used because it helps the model retain more global information on different channels;

[0072] Specifically, the global feature extraction module is adopted because fusing adjacent features in a dilated convolution manner is conducive to the full fusion of multi-scale information; the boundary feature guidance module is then adopted because fully exploring the spatial information of the edge detail features of the camouflaged object is conducive to the model to more accurately predict the camouflaged object; the decision-level feature fusion module is adopted because the pyramid Transformer and convolutional neural network can make up for each other's shortcomings, which is conducive to the comprehensive and accurate prediction of the camouflaged object. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] The accompanying drawings generally illustrate various embodiments by way of example and not limitation, and together with the description and claims, serve to illustrate the embodiments of the invention. Where appropriate, the same reference numerals are used throughout the drawings to refer to the same or similar parts. Such embodiments are illustrative and are not intended to be exhaustive or exclusive embodiments of the present apparatus or method.

[0074] Figure 1 is a method flow chart of the first embodiment;

[0075] Figure 2 It is a model structure diagram of the first embodiment;

[0076] Figure 3 It is a global feature extraction module of the first embodiment;

[0077] Figure 4 It is a boundary feature guiding module of the first embodiment;

[0078] Figure 5 It is the decision-level feature fusion module of the first embodiment;

[0079] FIG6( a ) is an input image of the first embodiment;

[0080] FIG6( b ) is a pixel-level label of the first embodiment;

[0081] FIG6( c ) is a rendering of the first embodiment of the present invention;

[0082] FIG6( d ) is a rendering of the HitNet of the first embodiment;

[0083] FIG6( e ) is a rendering of the DGNet of the first embodiment;

[0084] FIG6( f ) is a diagram showing the effect of FEDER of the first embodiment. DETAILED DESCRIPTION

[0085] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0086] Embodiment 1

[0087] like Figure 1 As shown, this embodiment 1 provides a camouflaged object semantic segmentation method based on decision-level feature fusion modeling, including:

[0088] The image to be segmented is input, and the model is supervised and trained using the image of the camouflaged object with pixel-level labels. The multi-scale feature extraction results based on convolutional neural network and Transformer are obtained through supervised training. Based on the multi-scale feature extraction results, the camouflaged objects in the image to be segmented are processed using a multi-channel spanning aggregation method to obtain high-resolution mask feature results. Based on the mask feature results, the decision-level feature fusion method is used to combine local and global features, and finally the predicted segmentation result of the model is output.

[0089] S101: Input an image to be segmented (the image to be segmented refers to a disguised image to be segmented).

[0090] The model (specifically, the image segmentation processing model) is supervised and trained using camouflaged object images with pixel-level labels, and multi-scale features based on convolutional neural networks and Transformers of the image to be segmented are obtained through supervised training;

[0091] S102: Based on the multi-scale feature extraction result, the camouflaged object in the image to be segmented is processed by using global feature extraction and boundary feature guidance to obtain a high-resolution mask feature;

[0092] S103: Based on the mask feature result, the local and global features are combined by using the decision-level feature fusion method, and finally a predicted segmentation result map of the output model (the model specifically refers to the image segmentation processing model) is output.

[0093] like Figure 2 As shown, the technical solution of the embodiment of the present invention specifically divides the camouflaged object semantic segmentation task into the following three processing stages, namely, feature extraction stage, multi-scale feature processing stage and model output prediction stage.

[0094] In the feature extraction stage, a pyramid-based Transformer backbone network (PVT-V2Backbone) and a convolutional neural backbone network (Res2Net Backbone) are used to perform multi-scale feature extraction on the segmented image;

[0095] In the multi-scale feature processing stage, a global feature extraction module is first used to process the camouflaged objects in the segmented image using a multi-scale pooling method to obtain a high-resolution global mask feature; then a boundary feature guidance module is used to perform back propagation guidance on the camouflaged objects in the segmented image using a boundary guidance method to aggregate the features of the global feature processing module;

[0096] For the model output prediction stage, this embodiment uses a decision-level feature fusion module, utilizes a dual attention fusion method to focus on the high-resolution mask features, and finally outputs the predicted segmentation result of the model.

[0097] As one or more embodiments, during the execution of step S101, supervised training is performed on the model using a disguised object image with pixel-level labels, and the multi-scale feature extraction result of the image to be segmented is obtained through the supervised training, which specifically includes the following operation steps:

[0098] S10101: Extract features from the segmented image based on the Pyramid Transformer Backbone Network (PVT-V2 Backbone) and the Convolutional Neural Backbone Network (Res2Net Backbone);

[0099] The pyramid Transformer backbone network includes: a first embedding block, a second embedding block, a third embedding block, a fourth embedding block and a fifth embedding block connected in sequence;

[0100] The convolutional neural backbone network comprises: a first convolution block, a second convolution block, a third convolution block, a fourth convolution block and a fifth convolution block connected in sequence;

[0101] S10102: The first embedding block performs a convolution layer with a convolution kernel size of 1*1 on the image to be segmented, and outputs a first feature map;

[0102] S10103: the second embedding block performs two-fold downsampling and two PVT-V2 basic blocks on the first feature map, and outputs a second feature map;

[0103] S10104: the third embedding block performs two-fold downsampling and two PVT-V2 basic blocks on the second feature map, and outputs a third feature map;

[0104] S10105: the fourth embedding block performs two-fold downsampling and 6 PVT-V2 basic blocks on the third feature map, and outputs a fourth feature map;

[0105] S10106: The fifth embedding block performs two-fold downsampling and 4 PVT-V2 basic blocks on the fourth feature map, and outputs a fifth feature map.

[0106] S10107: the first convolution block performs a convolution layer with a convolution kernel size of 1*1 and a Res2Net basic block on the image to be segmented, and outputs a sixth feature map;

[0107] S10108: the second convolution block performs two-fold downsampling and three Res2Net basic blocks on the sixth feature map, and outputs a seventh feature map;

[0108] S10109: the third convolution block performs two-fold downsampling and four Res2Net basic blocks on the seventh feature map, and outputs an eighth feature map;

[0109] S10110: The fourth convolution block performs two-fold downsampling and 6 Res2Net basic blocks on the eighth feature map, and outputs a ninth feature map.

[0110] S10111: The fifth convolution block performs two-fold downsampling and three Res2Net basic blocks on the ninth feature map, and outputs the tenth feature map.

[0111] Among them, the PVT-V2 basic block includes: a first multi-head self-attention module based on pooling, a first adder, a first normalization layer, a feedforward network, a second adder, and a second normalization layer connected in sequence; the Res2Net basic block includes: a convolution layer with a convolution kernel size of 1*1 based on residual connection, a convolution layer with a convolution kernel size of 3*3 based on residual connection, and a convolution layer with a convolution kernel size of 1*1 based on residual connection. Therefore, the specific operations of the PVT-V2 basic block and Res2Net (each module included) are not repeated in the embodiments of this application.

[0112] As one or more embodiments, during the execution of the above step S102, based on the extraction result of the multi-scale features, the camouflaged object in the image to be segmented is processed by a multi-scale downsampling method to obtain a high-resolution global mask feature, which specifically includes the following operation steps:

[0113] S10201: Based on the second feature map, the third feature map, the fourth feature map, the fifth feature map, the seventh feature map, the eighth feature map, the ninth feature map and the tenth feature map, the second high-quality mask map, the third high-quality mask map, the fourth high-quality mask map, the fifth high-quality mask map, the sixth high-quality mask map and the seventh high-quality mask map of corresponding sizes are input into the global feature extraction GCAM module through a splicing operation;

[0114] S10202: Based on the second feature map, the third feature map, the fourth feature map, the fifth feature map, the seventh feature map, the eighth feature map, the ninth feature map and the tenth feature map, the second feature map is input into the boundary guided BGM module through a splicing operation, and detailed feature supervision training is performed to guide the feature aggregation of the GCAM module through back propagation;

[0115] The GCAM module comprises: a residual branch and four side branches;

[0116] Among them, the four side branches are in parallel relationship with each other;

[0117] Wherein, the four side branches include: a first side branch, a second side branch, a third side branch and a fourth side branch;

[0118] The first side branch includes: a convolution layer with a convolution kernel size of 3*3 and a hole coefficient of 1, and a residual convolution layer with a convolution kernel size of 3*3, and then obtaining a first global feature through an upsampling operation;

[0119] The second side branch includes: a convolution layer with a convolution kernel size of 3*3 and a dilation coefficient of 2, and a residual convolution layer with a convolution kernel size of 3*3, and then obtaining a second global feature through an upsampling operation;

[0120] The third side branch includes: a convolution layer with a convolution kernel size of 3*3 and a dilation coefficient of 3, and a residual convolution layer with a convolution kernel size of 3*3, and then obtaining a third global feature through an upsampling operation;

[0121] The fourth side branch includes: sequentially connected to a convolution layer with a convolution kernel size of 3*3 and a dilation coefficient of 4, a residual convolution layer with a convolution kernel size of 3*3 and an element addition operation, and then obtaining a fourth global feature through an upsampling operation;

[0122] The residual branch includes: a convolution layer with a convolution kernel size of 3*3 and a splicing operation;

[0123] The first global feature, the second global feature, the third global feature and the fourth global feature are concatenated with a convolution layer whose convolution kernel size is 1*1 and whose hole coefficient is 1, and then element-wise added with the residual branch to obtain the final global feature;

[0124] The BGM module comprises: a first input terminal, a second input terminal, a third input terminal and a fourth input terminal and a plurality of up-sampling operations;

[0125] Wherein, the first input end is used to input the first characteristic map and the seventh characteristic map;

[0126] The second input terminal is used to input the second characteristic map and the eighth characteristic map;

[0127] The third input terminal is used to input the third characteristic map and the ninth characteristic map;

[0128] The fourth input terminal is used to input the fourth characteristic map and the tenth characteristic map;

[0129] The first feature map and the seventh feature map are subjected to a convolution layer with a convolution kernel size of 1*1 and a hole coefficient of 1 to obtain a first detail feature;

[0130] The second feature map and the eighth feature map are subjected to an upsampling operation after passing through a convolution layer with a convolution kernel size of 1*1 and a hole coefficient of 1 to obtain a second detail feature;

[0131] The third feature map and the ninth feature map are subjected to an upsampling operation after passing through a convolution layer with a convolution kernel size of 1*1 and a hole coefficient of 1 to obtain a third detail feature;

[0132] The fourth feature map and the tenth feature map are subjected to an upsampling operation after passing through a convolution layer with a convolution kernel size of 1*1 and a hole coefficient of 1 to obtain a fourth detail feature;

[0133] The first detail feature, the second detail feature, the third detail feature and the fourth detail feature are subjected to an upsampling operation and a convolution layer with a convolution kernel size of 1*1 and a dilation coefficient of 1, and then respectively subjected to four convolution layers with a dilation coefficient of 1 and a residual convolution layer with a convolution kernel size of 3*3 to obtain a fifth detail feature, a sixth detail feature, a seventh detail feature and an eighth detail feature;

[0134] The fifth detail feature, the sixth detail feature, the seventh detail feature and the eighth detail feature are subjected to element-wise addition operation and a convolution layer with a dilation coefficient of 1 and a residual convolution layer with a convolution kernel size of 3*3 to obtain the final detail feature;

[0135] As one or more embodiments, during the execution of step S103, based on the GCAM global feature extraction result, the high-resolution mask feature is fused by the MFM decision-level feature fusion module, and the predicted segmentation result of the output model (the model specifically refers to the image segmentation processing model) is specifically included:

[0136] S10301: inputting the high-resolution global mask image into the MFM decision-level feature fusion module for multi-channel and multi-scale fusion to obtain fusion features;

[0137] The MFM module comprises: a residual branch and three side branches;

[0138] Among them, the three side branches are in parallel relationship with each other;

[0139] Wherein, the three side branches include: a first side branch, a second side branch and a third side branch;

[0140] The first side branch includes: sequentially connected to a convolution layer with a convolution kernel size of 3*3 and a stacking coefficient of 1, a split operation, and then a splicing operation to obtain a first fusion feature;

[0141] The second side branch includes: sequentially connected to convolution layers with a convolution kernel size of 3*3 and a dilation coefficient of 2, a splitting operation, and then a splicing operation to obtain a second fusion feature;

[0142] The third side branch includes: sequentially connected to convolution layers with a convolution kernel size of 3*3 and a dilation coefficient of 3, a splitting operation, and then a splicing operation to obtain a third fusion feature;

[0143] The residual branch includes: three convolution layers with a convolution kernel size of 3*3 and a splicing operation;

[0144] The first fusion feature, the second fusion feature and the third fusion feature are sequentially connected to form a dual attention layer, a convolution layer with a convolution kernel size of 3*3 and a void coefficient of 1, an upsampling operation, and then a splicing operation is performed with the residual branch to obtain the final camouflaged object semantic segmentation prediction result;

[0145] The dual attention layer includes: a convolution layer with a convolution kernel size of 3*3 and a void coefficient of 1, a group normalization operation, an average pooling operation, a Softmax activation layer, a matrix multiplication operation, a Sigmoid activation layer and an element multiplication operation;

[0146] S10302: In combination with the dual attention perception result, the dual attention weighted mask is normalized in the channel dimension to obtain an output model prediction result graph of the camouflaged object.

[0147] Deep neural network training, parameter initialization: For the first to fifth embedding blocks, the PVTV2 weight parameters pre-trained on the ImageNet1K dataset are used to initialize the network parameters; for the first to fifth convolution blocks, the Res2Net weight parameters pre-trained on the ImageNet1K dataset are used to initialize the network parameters.

[0148] Training optimization details: The technical solution of the embodiment of the present invention adopts an AdamW optimizer, with 55 training rounds, an initial learning rate of 1e-4, a batch size of 16, and a weight decay rate of 1e-4.

[0149] Table 1 shows the quantitative comparison results with 18 cutting-edge camouflaged object semantic segmentation models.

[0150] Table 1 shows a quantitative comparison with 18 cutting-edge camouflaged object semantic segmentation models.

[0151]

[0152] Figure 6(a) to Figure 6(f) The qualitative effect comparison diagram of the technical solution of the embodiment of the present invention and the current three cutting-edge models is shown in FIG.

[0153] Embodiment 2

[0154] The second embodiment provides a camouflaged object semantic segmentation system based on decision-level feature fusion modeling, including:

[0155] The multi-scale feature acquisition module is configured to: input an image to be segmented, use a camouflaged object image with pixel-level labels to perform supervised training on the model, and obtain a multi-scale feature extraction result of the image to be segmented through the supervised training;

[0156] A global feature extraction module is configured to: based on the extraction result of the multi-scale features, process the camouflaged objects in the image to be segmented by using a multi-scale pooling method to obtain a high-resolution global mask feature;

[0157] A boundary feature guiding module is configured to: based on the extraction result of the multi-scale features, perform back propagation to guide the feature aggregation of the global feature processing module on the camouflaged object in the image to be segmented by using a boundary guiding method;

[0158] The decision-level feature fusion module is configured to: focus on the high-resolution mask features based on the output result of the global feature extraction module by using the fusion dual attention method, and output the predicted segmentation result of the model.

[0159] It should be noted that the multi-scale feature acquisition module, global feature extraction module, boundary feature guidance module, and model output prediction module correspond to steps S101 to S103 in Embodiment 1, and the examples and application scenarios implemented by the modules and the corresponding steps are the same, but are not limited to the contents disclosed in Embodiment 1. It should be noted that the modules as part of the system can be executed in a computer system such as a set of computer executable instructions.

[0160] The proposed system can be implemented in other ways. For example, the system embodiment described above is only illustrative, and the division of the modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0161] Embodiment 3

[0162] This embodiment also provides a camouflaged object semantic segmentation device based on decision-level feature fusion modeling, including a processor and a storage medium; wherein the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the method described in the above embodiment one.

[0163] It should be understood that in this embodiment, the processor may be a central processing unit CPU or other general-purpose processors. The general-purpose processor may be a microprocessor or any conventional processor.

[0164] Embodiment 4

[0165] This fourth embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in the first embodiment is implemented.

[0166] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A camouflaged object semantic segmentation method based on decision-level feature fusion modeling, characterized in that: The method comprises: S1 inputs an image to be segmented, uses a camouflaged object image with pixel-level labels to perform supervised training on the model, and obtains a multi-scale feature extraction result of the image to be segmented through the supervised training; The S1 is specifically: Use pyramid-based Transformer backbone network and convolutional neural backbone network to extract multi-scale features of the segmented image; Specifically, the multi-scale feature extraction of the segmented image using the pyramid-based Transformer backbone network includes: Perform a convolution layer with a kernel size of 1*1 on the image to be segmented and output the first feature map; Performing two-fold downsampling and two PVT-V2 basic blocks on the first feature map, and outputting a second feature map; Performing two-fold downsampling and two PVT-V2 basic blocks on the second feature map, outputting a third feature map; Performing two-fold downsampling and 6 PVT-V2 basic blocks on the third feature map, and outputting a fourth feature map; Performing two-fold downsampling and four PVT-V2 basis blocks on the fourth feature map, and outputting a fifth feature map; The PVT-V2 basic block includes: a first multi-head self-attention module based on pooling, a first adder, a first normalization layer, a feedforward network, a second adder, and a second normalization layer connected in sequence; Using a convolutional neural network backbone to extract multi-scale features from the segmented image includes: Perform a convolution layer with a kernel size of 1*1 and a Res2Net basic block on the segmented image, and output the sixth feature map; Performing two-fold downsampling and three Res2Net basic blocks on the sixth feature map, and outputting a seventh feature map; Performing two-fold downsampling and four Res2Net basic blocks on the seventh feature map, and outputting an eighth feature map; Performing two-fold downsampling and 6 Res2Net basic blocks on the eighth feature map, and outputting a ninth feature map; Performing two-fold downsampling and three Res2Net basic blocks on the ninth feature map, and outputting a tenth feature map; The Res2Net basic block includes: a convolutional layer with a convolution kernel size of 1*1 based on residual connection, a convolutional layer with a convolution kernel size of 3*3 based on residual connection, and a convolutional layer with a convolution kernel size of 1*1 based on residual connection; S2, based on the multi-scale feature extraction result, the camouflaged object in the segmented image is processed by using global feature extraction and edge feature guidance to obtain a high-resolution mask feature; The S2 is specifically: Based on the second feature map, the third feature map, the fourth feature map, the fifth feature map, the seventh feature map, the eighth feature map, the ninth feature map and the tenth feature map, input into the global feature extraction GCAM module through a splicing operation, and obtain a second high-quality mask map, a third high-quality mask map, a fourth high-quality mask map, a fifth high-quality mask map, a sixth high-quality mask map and a seventh high-quality mask map of corresponding sizes; Based on the second feature map, the third feature map, the fourth feature map, the fifth feature map, the seventh feature map, the eighth feature map, the ninth feature map and the tenth feature map, the edge-guided BGM module is input through a splicing operation to perform detailed feature supervision training to guide the feature aggregation of the GCAM module through back propagation; S3 focuses on the high-resolution mask features based on the high-resolution mask features by using a decision-level feature fusion method, and outputs a prediction result of the model; The S3 is specifically: Inputting the second high-quality mask map, the third high-quality mask map, the fourth high-quality mask map, the fifth high-quality mask map, the sixth high-quality mask map and the seventh high-quality mask map into the MFM decision-level feature fusion module for multi-channel multi-scale fusion to obtain fused features; The fused features are then connected in sequence to form a dual attention layer, a convolution layer with a convolution kernel size of 3*3 and a void coefficient of 1, and then concatenated with the residual branch to obtain the final semantic segmentation prediction result of the camouflaged object.

2. A camouflaged object semantic segmentation system based on decision-level feature fusion modeling, characterized in that: include: A feature extraction module is used to input an image to be segmented, and use a camouflaged object image with pixel-level labels to perform supervised training on the model, thereby obtaining a multi-scale feature extraction result of the image to be segmented through the supervised training; Specifically: Use pyramid-based Transformer backbone network and convolutional neural backbone network to extract multi-scale features of the segmented image; Specifically, the multi-scale feature extraction of the segmented image using the pyramid-based Transformer backbone network includes: Perform a convolution layer with a kernel size of 1*1 on the image to be segmented and output the first feature map; Performing two-fold downsampling and two PVT-V2 basic blocks on the first feature map, and outputting a second feature map; Performing two-fold downsampling and two PVT-V2 basic blocks on the second feature map, outputting a third feature map; Performing two-fold downsampling and 6 PVT-V2 basic blocks on the third feature map, and outputting a fourth feature map; Performing two-fold downsampling and four PVT-V2 basis blocks on the fourth feature map, and outputting a fifth feature map; The PVT-V2 basic block includes: a first multi-head self-attention module based on pooling, a first adder, a first normalization layer, a feedforward network, a second adder, and a second normalization layer connected in sequence; Using a convolutional neural network backbone to extract multi-scale features from the segmented image includes: Perform a convolution layer with a kernel size of 1*1 and a Res2Net basic block on the segmented image, and output the sixth feature map; Performing two-fold downsampling and three Res2Net basic blocks on the sixth feature map, and outputting a seventh feature map; Performing two-fold downsampling and four Res2Net basic blocks on the seventh feature map, and outputting an eighth feature map; Performing two-fold downsampling and 6 Res2Net basic blocks on the eighth feature map, and outputting a ninth feature map; Performing two-fold downsampling and three Res2Net basic blocks on the ninth feature map, and outputting a tenth feature map; The Res2Net basic block includes: a convolutional layer with a convolution kernel size of 1*1 based on residual connection, a convolutional layer with a convolution kernel size of 3*3 based on residual connection, and a convolutional layer with a convolution kernel size of 1*1 based on residual connection; A multi-scale feature processing module, based on the multi-scale feature extraction result, processes the camouflaged object in the segmented image by using global feature extraction and edge feature guidance to obtain a high-resolution mask feature; Specifically: Based on the second feature map, the third feature map, the fourth feature map, the fifth feature map, the seventh feature map, the eighth feature map, the ninth feature map and the tenth feature map, input into the global feature extraction GCAM module through a splicing operation, and obtain a second high-quality mask map, a third high-quality mask map, a fourth high-quality mask map, a fifth high-quality mask map, a sixth high-quality mask map and a seventh high-quality mask map of corresponding sizes; Based on the second feature map, the third feature map, the fourth feature map, the fifth feature map, the seventh feature map, the eighth feature map, the ninth feature map and the tenth feature map, the edge-guided BGM module is input through a splicing operation to perform detailed feature supervision training to guide the feature aggregation of the GCAM module through back propagation; A model output prediction module, used to focus on the high-resolution mask feature based on the high-resolution mask feature result by using a decision-level feature fusion method, and output the prediction result of the model; Specifically: Inputting the second high-quality mask map, the third high-quality mask map, the fourth high-quality mask map, the fifth high-quality mask map, the sixth high-quality mask map and the seventh high-quality mask map into the MFM decision-level feature fusion module for multi-channel multi-scale fusion to obtain fused features; The fused features are then connected in sequence to form a dual attention layer, a convolution layer with a convolution kernel size of 3*3 and a void coefficient of 1, and then concatenated with the residual branch to obtain the final semantic segmentation prediction result of the camouflaged object.

3. The camouflaged object semantic segmentation system based on decision-level feature fusion modeling according to claim 2 is characterized in that: The multi-scale feature acquisition module is composed of a pyramid Transformer backbone network and a backbone network based on a convolutional neural network; The pyramid Transformer backbone network includes: a first embedding block, a second embedding block, a third embedding block, a fourth embedding block and a fifth embedding block connected in sequence; The backbone of the convolutional neural network includes: a first convolution block, a second convolution block, a third convolution block, a fourth convolution block and a fifth convolution block which are connected in sequence.

4. The camouflaged object semantic segmentation system based on decision-level feature fusion modeling according to claim 2 is characterized in that: The multi-scale feature processing module includes a global feature extraction GCAM module and an edge guidance BGM module; The global feature extraction GCAM module includes: One residual branch and four side branches; Among them, the four side branches are in parallel relationship with each other; Wherein, the four side branches include: a first side branch, a second side branch, a third side branch and a fourth side branch; The first side branch includes: a convolution layer with a convolution kernel size of 3*3 and a hole coefficient of 1, and a residual convolution layer with a convolution kernel size of 3*3, and then obtaining a first global feature through an upsampling operation; The second side branch includes: a convolution layer with a convolution kernel size of 3*3 and a dilation coefficient of 2, and a residual convolution layer with a convolution kernel size of 3*3, and then obtaining a second global feature through an upsampling operation; The third side branch includes: a convolution layer with a convolution kernel size of 3*3 and a dilation coefficient of 3, and a residual convolution layer with a convolution kernel size of 3*3, and then obtaining a third global feature through an upsampling operation; The fourth side branch includes: sequentially connected to a convolution layer with a convolution kernel size of 3*3 and a dilation coefficient of 4, a residual convolution layer with a convolution kernel size of 3*3 and an element addition operation, and then obtaining a fourth global feature through an upsampling operation; The residual branch includes: a convolution layer with a convolution kernel size of 3*3 and a splicing operation; The BGM module comprises: a first input terminal, a second input terminal, a third input terminal and a fourth input terminal and a plurality of up-sampling operations; Wherein, the first input end is used to input the first characteristic map and the seventh characteristic map; The second input terminal is used to input the second characteristic map and the eighth characteristic map; The third input terminal is used to input the third characteristic map and the ninth characteristic map; The fourth input terminal is used to input the fourth characteristic map and the tenth characteristic map.

5. The camouflaged object semantic segmentation system based on decision-level feature fusion modeling according to claim 2 is characterized in that: The model output prediction module includes an MFM module and a dual attention layer: The MFM module comprises: a residual branch and three side branches; Among them, the three side branches are in parallel relationship with each other; Wherein, the three side branches include: a first side branch, a second side branch and a third side branch; The first side branch includes: sequentially connected to a convolution layer with a convolution kernel size of 3*3 and a stacking coefficient of 1, a split operation, and then a splicing operation to obtain a first fusion feature; The second side branch includes: sequentially connected to convolution layers with a convolution kernel size of 3*3 and a dilation coefficient of 2, a splitting operation, and then a splicing operation to obtain a second fusion feature; The third side branch includes: sequentially connected to convolution layers with a convolution kernel size of 3*3 and a dilation coefficient of 3, a splitting operation, and then a splicing operation to obtain a third fusion feature; The residual branch includes: three convolution layers with a convolution kernel size of 3*3 and a splicing operation; The first fusion feature, the second fusion feature and the third fusion feature are sequentially connected to form a dual attention layer, a convolution layer with a convolution kernel size of 3*3 and a void coefficient of 1, an upsampling operation, and then a splicing operation is performed with the residual branch to obtain the final camouflaged object semantic segmentation prediction result; The dual attention layer includes: a convolution layer with a convolution kernel size of 3*3 and a void coefficient of 1, a group normalization operation, an average pooling operation, a Softmax activation layer, a matrix multiplication operation, a Sigmoid activation layer and an element multiplication operation.

6. A camouflaged object semantic segmentation device based on decision-level feature fusion modeling, characterized in that: including processor and storage medium; The storage medium is used to store instructions; The processor can be operated according to the instructions to execute the steps of the method of claim 1.

7. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, the steps of the method according to claim 1 are implemented.

Images