Instance segmentation method and device for small road target in remote sensing image
Through the DMH-YOLO model and the multi-domain attention feature fusion module, the problem of difficult identification of small targets in remote sensing images is solved, and high-precision road target instance segmentation is achieved.
Patent Information
- Application Number
- CN202411217307.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-02
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-09-02
AI Technical Summary
In remote sensing images, small targets such as intersections and pedestrian crossings are difficult to be accurately identified and segmented, and are greatly disturbed by external environmental factors, resulting in poor segmentation effect of instance segmentation model.
The DMH-YOLO model is used to combine the Darknet-53 backbone network and the dual-stream feature pyramid to extract features through spatial domain and frequency domain branches, and use the multi-domain attention feature fusion module to perform small-target instance segmentation.
It improves the information capture ability and segmentation accuracy of small targets, and enhances the ability to identify small targets on the road.
Smart Images

Figure CN119027667A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a method and device for segmenting road target instances in remote sensing images. Background Art
[0002] High-resolution remote sensing objectively records all kinds of ground information. Through artificial intelligence, it can obtain various types of information such as the location, shape, size, distribution, spatial relationship, etc. of various traffic and non-traffic elements, which is convenient for traffic command and decision-making, and is the main data for comprehensive traffic surveys. In urban traffic, crosswalks and intersections are important components of roads, and are also considered to be two key conflict points in urban traffic networks. The traffic composition and traffic characteristics of these locations are very complex, and mixed vehicles, traffic congestion, and irregular flow inevitably lead to traffic accidents.
[0003] In remote sensing images, intersections and crosswalks are usually classified as small objects. Small objects have less visual information, making it difficult to extract discriminative features, and are easily disturbed by external environmental factors, making it difficult for instance segmentation models to accurately identify and segment small objects. Summary of the invention
[0004] The technical problem to be solved by this application is the inability to accurately identify and segment small target objects.
[0005] In order to solve the above technical problems, this application proposes the following technical solutions:
[0006] A method for segmenting road targets in remote sensing images comprises the following steps:
[0007] The S1 remote sensing satellite acquires the remote sensing image of the target to be detected and pre-processes the remote sensing image through the input module;
[0008] The S2 backbone network extracts features from the preprocessed image and fuses the extracted multi-resolution features using a two-stream feature pyramid at the neck;
[0009] The S3 head network is divided into two branches: the detection head and the prototype. The image enters the head network to complete the refinement tasks of category prediction and instance segmentation;
[0010] S4 outputs the high-resolution remote sensing image road target instance segmentation results;
[0011] Wherein, the DMH-YOL0 model is used to perform instance segmentation on the target remote sensing image to determine the target object in the target remote sensing image;
[0012] The backbone network adopts the Darknet-53 structure framework and integrates the C2f module to realize the residual learning mechanism; mix-pooling is used, combining maximum pooling and soft-pooling; the neck adopts a two-stream feature pyramid to fuse the feature information extracted by the backbone network to obtain features P3, P4, and P5;
[0013] According to an embodiment of the present application, the head network takes the feature map P3 as input and adopts a dual-mask branch, which is divided into a spatial domain branch and a frequency domain branch;
[0014] The spatial domain branch is a fully convolutional structure. For input P3, mask features are first extracted through three 3×3 convolutional layers. Then the feature map is upsampled to a resolution twice that of the original feature map.
[0015] Perform 3×3 convolution on the feature map to obtain the mask feature F1 containing rich spatial domain information;
[0016] The F1 calculation formula is:
[0017] F1 = Conv(upSample(Conv(P3)))
[0018] The frequency domain branch uses the wavelet transform module to perform frequency domain feature mapping through the following steps:
[0019] First, the wavelet transform module is used to convert the input P3 into four frequency domain components, one of which is a low-frequency component, and the other three are horizontal, vertical and diagonal high-frequency components. The low-frequency information is convolved to obtain low-frequency features, and the three high-frequency information are concatenated to form a high-frequency signal through Concat.
[0020] Secondly, nonlinear operations are performed on it to obtain high-frequency features;
[0021] Again, the low-frequency features are combined with the high-frequency features through Concat to obtain the frequency domain features;
[0022] Then, the feature is upsampled to obtain the frequency domain feature F2. The calculation formula of F2 is:
[0023] F2 = Conv(upSample(WTB(P3)))
[0024] Finally, the spatial domain features are fused with the frequency domain features through the multi-domain attention feature fusion module, and then the number of channels of the feature map is adjusted to k through 1×1 convolution to obtain F mask :
[0025] Fmask=Conv(MDAFF(F1,F2)).
[0026] According to an embodiment of the present application, the fusion method of the multi-domain attention feature fusion module comprises the following steps:
[0027] S21 performs initial feature fusion on the spatial domain feature F1 and the frequency domain feature F2. The fused features focus on the scale of the channel through point-by-point convolution, including point-by-point convolution and ReLU modules;
[0028] S22 is activated by the sigmoid function to obtain a weight value with an output value of (0, 1). The weight value is obtained through training, and the weighted average of F1 and F2 is used to obtain the feature map F'.
[0029] Use the unsqueeze method to increase the dimension of the feature map F', expanding it from a three-dimensional tensor [height, width, channel] to a four-dimensional tensor [depth, height, width, channel], forming a 3D feature map for subsequent convolution;
[0030] S23 uses 3D convolution to extract sequence features from feature maps and restore the three-dimensional tensor. The calculation formula is:
[0031]
[0032]
[0033] F=squeeze(3DConv(unsqueeze(F′)))
[0034] The feature F is obtained after the fusion of spatial domain and frequency domain.
[0035] According to an embodiment of the present application, the loss function of the DMH-YOLO model includes three parts: classification loss, regression loss and mask classification loss; the classification loss uses a sigmoid function to calculate the probability of each category, and uses a binary cross entropy loss to calculate the global category loss;
[0036] The regression loss is NWD loss, and CIou_Loss is used to calculate the IoU between the predicted box and the target box;
[0037] The mask classification loss is calculated using binary cross entropy loss (BCE Loss).
[0038]
[0039] L box =L nwd =1-NWD(N p , N g )
[0040] Lmask=BCE(M,M G ).
[0041] According to an embodiment of the present application, the image preprocessing includes enhancing image brightness, adjusting image size and grayscale image properties, and the image size is adjusted to 1280*1280.
[0042] Another object of the present invention is to provide a remote sensing image road target instance segmentation device, comprising: an acquisition unit, used to acquire a target remote sensing image to be instance segmented;
[0043] The determination unit is used to perform instance segmentation on the remote sensing image using the DMH-YOLO model to determine the target object in the target remote sensing image, including: obtaining target multi-scale features, and then inputting the obtained multi-scale features into the DMH-YOLO model to determine the target object in the target remote sensing image.
[0044] The determination unit adopts the following steps to obtain features: mix-pooling, which combines maximum pooling and soft-pooling, respectively using maximum pooling to extract significant features F1, and using soft pooling to retain more detail information to obtain feature F2, and the feature is obtained by combining features F1 and F2.
[0045] The determination unit uses the following steps to obtain mask features: first, the mask features are extracted through three 3×3 convolutional layers. Then the feature map is upsampled to twice the resolution of the original feature map.
[0046] Furthermore, a 3×3 convolution is performed on the feature map to obtain a mask feature F1 containing rich spatial domain information.
[0047] The F1 calculation formula is:
[0048] F1 = Conv(upSample(Conv(P3)))
[0049] The frequency domain branch uses the wavelet transform module to perform frequency domain feature mapping through the following steps:
[0050] First, the input P3 is converted into four frequency domain components using the wavelet transform module, one of which is a low-frequency component and the other three are horizontal, vertical and diagonal high-frequency components. The low-frequency information is convolved to obtain low-frequency features, and the three high-frequency information are concatenated to form a high-frequency signal through Concat. Frequency domain information can provide complementary and useful information for spatial domain features. Wavelet transform plays an important role in image analysis and helps to better understand their structure and characteristics.
[0051] Secondly, nonlinear operations are performed on it to obtain high-frequency features;
[0052] Again, the low-frequency features are combined with the high-frequency features through Concat to obtain the frequency domain features;
[0053] Then, the feature is upsampled to obtain the frequency domain feature F2, and the calculation formula of F2 is:
[0054] F2 = Conv(upSample(WTB(P3)))
[0055] Finally, the spatial domain features are fused with the frequency domain features through the multi-domain attention feature fusion module (MDAFF), and then the number of channels of the feature map is adjusted to k through 1×1 convolution to obtain Fmask:
[0056] Fmask=Conv(MDAFF(F1,F2)).
[0057] In one embodiment, the multi-domain attention feature fusion module includes spatial domain features F1 and frequency domain features F2, where:
[0058] S21 performs initial feature fusion on the spatial domain features F1 and the frequency domain features F2. The fused features focus on the scale of the channel through point-by-point convolution. It includes point-by-point convolution and ReLU modules, which is equivalent to a lightweight channel attention module.
[0059] S22 is activated by the sigmoid function to obtain a weight value with an output value of (0, 1). The weight value is obtained through training, and the weighted average of F1 and F2 is used to obtain the feature map F'.
[0060] Use the unsqueeze method to increase the dimension of the feature map F', expanding it from a three-dimensional tensor [height, width, channel] to a four-dimensional tensor [depth, height, width, channel], forming a 3D feature map for subsequent convolution;
[0061] S23 uses 3D convolution to extract sequence features from feature maps and restore the three-dimensional tensor. The calculation formula is:
[0062]
[0063]
[0064] F=squeeze(3DConv(unsqueeze(F')))
[0065] The feature F is obtained after the fusion of spatial domain and frequency domain.
[0066] Compared with the prior art, this application has at least the following beneficial effects:
[0067] The DMH-YOLO model is used to perform instance segmentation on the target remote sensing image and determine the target object in the target remote sensing image. Specifically, the DMH-YOLO model uses mask branches to extract spatial and frequency domain features for small target instance segmentation, which improves the network's ability to capture information about small targets. The multi-domain attention feature fusion module (MDAFF) effectively bridges the spatial and frequency domain features, improving the segmentation accuracy of the DMH-YOLO model for small road targets. BRIEF DESCRIPTION OF THE DRAWINGS
[0068] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present application, rather than limiting the present application.
[0069] Figure 1 It is a flow chart of an embodiment of the present application.
[0070] Figure 2 2 is a schematic diagram of the DMH-YOLO model structure of an embodiment of the present application.
[0071] Figure 3 It is a schematic diagram of the structure of a double mask branch in an example of the present application. DETAILED DESCRIPTION
[0072] In order to make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme of the embodiment of the present application will be clearly and completely described in conjunction with the drawings of the embodiment of the present application. Obviously, the described embodiment is a part of the embodiment of the present application, not all of the embodiments. Based on the described embodiment of the present application, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of this application.
[0073] Unless otherwise defined, the technical or scientific terms used herein shall have the common meanings understood by persons with ordinary skills in the field to which this application belongs. The words "first", "second" and similar words used in the patent application specification and claims of this application do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one" or "an" do not indicate a quantity limitation, but indicate the existence of at least one.
[0074] The embodiments of the present application are described below with reference to the accompanying drawings.
[0075] See also Figures 1 to 3 According to an embodiment of the present application, a method for segmenting road target instances in remote sensing images is provided. In the embodiment of the present invention, the target objects may include smaller objects such as intersections and crosswalks. Figure 1As shown, the segmentation method includes the following steps:
[0076] The S1 remote sensing satellite acquires the remote sensing image of the target to be detected and pre-processes the remote sensing image through the input module;
[0077] The S2 backbone network extracts features from the preprocessed images and fuses the extracted multi-resolution features at the neck using a dual-stream feature pyramid network (DS-FPN).
[0078] The backbone network adopts the Darknet-53 structure framework and integrates the C2f module to implement the residual learning mechanism, so that the network can obtain richer gradient flow. Mix-pooling is used, which combines maximum pooling and soft-pooling. Maximum pooling extracts significant features, while soft pooling retains more detailed information. Especially in instance segmentation tasks, this method can more comprehensively capture the semantics and edge information of the target, thereby achieving better segmentation results.
[0079] The S3 head is divided into two branches: the detection head and the prototype (Proto). The image enters the head to complete the refinement tasks of category prediction and instance segmentation.
[0080] S4 outputs the high-resolution remote sensing image road target instance segmentation results.
[0081] Among them, Figure 2 As shown, the DMH-YOLO model includes a backbone network, a neck network and a head network. The DMH-YOLO model is used to perform instance segmentation on the target remote sensing image to determine the target object in the target remote sensing image.
[0082] The image is input into the DMH-YOLO model. The backbone network is designed to extract low-level texture features and high-level semantic features of the input image, and then pass the low-level texture features and high-level semantic features to the head network to build an enhanced feature pyramid network, which transmits robust semantic features from top to bottom and strong responses of local texture and pattern features from bottom to top.
[0083] The backbone network of the DMH-YOLO model adopts the Darknet-53 structure framework and incorporates the C2f (CSPLayer_2Conv) module to implement the residual learning mechanism. This design draws on the concepts of CSP and ELAN, adds multi-level skip layer connections and additional Split operations, allowing the network to effectively integrate gradient information into the feature map from beginning to end.
[0084] In DMH-YOLO, the Conv convolution module and the C2f module are stacked four times in series, and each stacking constitutes an independent stage. In addition, DMH-YOLO introduces the SPPF (Spatial Pyramid Pooling Fusion) module in the final stage to ensure the uniformity of the vector dimensions of feature maps of different scales.
[0085] In specific applications, it can be seen that in the DMH-YOLO model, by stacking the CBS module and the C2f module, 5 groups of feature maps of different scales can be extracted, namely feature map P1, feature map P2, feature map P3, feature map P4 and feature map P5. Among them, Pi represents the resolution of 1 / 2i of the target remote sensing image.
[0086] Specifically, P1 represents a resolution of 1 / 2 of the target remote sensing image, and correspondingly, P5 represents a resolution of 1 / 32 of the target remote sensing image.
[0087] In some embodiments, the feature map scale of feature map P5 is 20×20, the feature map scale of feature map P4 is 40×40, the feature map scale of feature map P3 is 80×80, and the feature map scale of feature map P2 is 160×160.
[0088] In the feature map Pi, the smaller the value of i is, the fewer times the downsampling is performed, so the more underlying feature information is contained therein.
[0089] The multi-scale features extracted by the backbone network enter the neck for feature fusion. The neck of the DMH-YOLO model also replaces the C3 module with the C2f module, and combines the concepts of PAN and FPN to establish a top-down and bottom-up feature pyramid structure.
[0090] The neck network uses a two-stream feature pyramid to fuse the feature information extracted by the backbone network to obtain features P3, P4, and P5.
[0091] In the head network, DMH-YOLO has made significant improvements, transforming the Anchor-Based coupled prediction head in YOLOv5 into Anchor r -Fre e Decoupled prediction head.
[0092] In one embodiment, Figure 3 As shown in the figure, the head network takes the feature map P3 as input and adopts a dual mask branch. The dual mask branch is divided into a spatial domain branch and a frequency domain branch. Since the dual mask branch is used to replace the existing Proto branch, a more refined mask can be generated.
[0093] The spatial domain branch is a fully convolutional structure. For input P3, it first passes through three 3×3 convolutional layers to extract mask features. Then the feature map is upsampled to twice the resolution of the original feature map.
[0094] Furthermore, a 3×3 convolution is performed on the feature map to obtain a mask feature F1 containing rich spatial domain information.
[0095] The F1 calculation formula is:
[0096] F1 = Conv(upSample(Conv(P3)))
[0097] Due to the complex background of remote sensing images and the large grayscale difference at the edge, the segmentation branch only uses spatial features, which makes it difficult to accurately segment small target instances, especially at the edge. The frequency domain branch uses the wavelet transform module to perform frequency domain feature mapping. Frequency domain information can provide supplementary and useful information for spatial domain features. The frequency domain branch uses the wavelet transform module to perform frequency domain feature mapping through the following steps:
[0098] First, the input P3 is converted into four frequency domain components using the wavelet transform module, one of which is a low-frequency component and the other three are horizontal, vertical and diagonal high-frequency components. The low-frequency information is convolved to obtain low-frequency features, and the three high-frequency information are concatenated to form a high-frequency signal through Concat. Frequency domain information can provide complementary and useful information for spatial domain features. Wavelet transform plays an important role in image analysis and helps to better understand their structure and characteristics.
[0099] Secondly, nonlinear operations are performed on it to obtain high-frequency features;
[0100] Again, the low-frequency features are combined with the high-frequency features through Concat to obtain the frequency domain features;
[0101] Then, the feature is upsampled to obtain the frequency domain feature F2, and the calculation formula of F2 is:
[0102] F2 = Conv(upSample(WTB(P3)))
[0103] Finally, the spatial domain features are fused with the frequency domain features through the multi-domain attention feature fusion module (MDAFF), and then the number of channels of the feature map is adjusted to k through 1×1 convolution to obtain F mask :
[0104] Fmask=Conv(MDAFF(F1,F2)).
[0105] In one embodiment, the fusion method of the multi-domain attention feature fusion module comprises the following steps:
[0106] S21 performs initial feature fusion on the spatial domain features F1 and the frequency domain features F2. The fused features focus on the scale of the channel through point-by-point convolution. It includes point-by-point convolution and ReLU modules, which is equivalent to a lightweight channel attention module.
[0107] S22 is activated by the sigmoid function to obtain a weight value with an output value of (0, 1). The weight value is obtained through training, and the weighted average of F1 and F2 is used to obtain the feature map F'.
[0108] Use the unsqueeze method to increase the dimension of the feature map F', expanding it from a three-dimensional tensor [height, width, channel] to a four-dimensional tensor [depth, height, width, channel], forming a 3D feature map for subsequent convolution;
[0109] S23 uses 3D convolution to extract sequence features from feature maps and restore the three-dimensional tensor. The calculation formula is:
[0110]
[0111]
[0112] F=squeeze(3DConv(unsqueeze(F')))
[0113] The feature F is obtained after the fusion of spatial domain and frequency domain.
[0114] In one embodiment, the loss function of the DMH-YOL0 model includes three parts: classification loss, regression loss, and mask classification loss; the classification loss uses a sigmoid function to calculate the probability of each category, and uses a binary cross entropy loss to calculate the global category loss;
[0115] The regression loss is NWD loss, and CIou_Loss is used to calculate the IoU between the predicted box and the target box;
[0116] The mask classification loss is calculated using binary cross entropy loss (BCE Loss);
[0117]
[0118] L box =L nwd =1-NWD(N p , N g )
[0119] Lmask=BCE(M,M G ).
[0120] In one embodiment, the image preprocessing includes enhancing image brightness, adjusting image size and grayscale image attributes, and the image size is adjusted to 1280*1280.
[0121] According to an embodiment of the present application, a remote sensing image road target instance segmentation device is provided, comprising: an acquisition unit, configured to acquire a target remote sensing image to be instance segmented;
[0122] The determination unit is used to perform instance segmentation on the remote sensing image using the DMH-YOLO model to determine the target object in the target remote sensing image, including: obtaining target multi-scale features, and then inputting the obtained multi-scale features into the DMH-YOLO model to determine the target object in the target remote sensing image.
[0123] The determination unit adopts the following steps to obtain features: mix-pooling, which combines maximum pooling and soft-pooling, respectively using maximum pooling to extract significant features F1, and using soft pooling to retain more detail information to obtain feature F2, and the feature is obtained by combining features F1 and F2.
[0124] The determination unit uses the following steps to obtain mask features: first, the mask features are extracted through three 3×3 convolutional layers. Then the feature map is upsampled to twice the resolution of the original feature map.
[0125] Furthermore, a 3×3 convolution is performed on the feature map to obtain a mask feature F1 containing rich spatial domain information.
[0126] The F1 calculation formula is:
[0127] F1 = Conv(upSample(Conv(P3)))
[0128] The frequency domain branch uses the wavelet transform module to perform frequency domain feature mapping through the following steps:
[0129] First, the input P3 is converted into four frequency domain components using the wavelet transform module, one of which is a low-frequency component and the other three are horizontal, vertical and diagonal high-frequency components. The low-frequency information is convolved to obtain low-frequency features, and the three high-frequency information are concatenated to form a high-frequency signal through Concat. Frequency domain information can provide complementary and useful information for spatial domain features. Wavelet transform plays an important role in image analysis and helps to better understand their structure and characteristics.
[0130] Secondly, nonlinear operations are performed on it to obtain high-frequency features;
[0131] Again, the low-frequency features are combined with the high-frequency features through Concat to obtain the frequency domain features;
[0132] Then, the feature is upsampled to obtain the frequency domain feature F2, and the calculation formula of F2 is:
[0133] F2 = Conv(upSample(WTB(P3)))
[0134] Finally, the spatial domain features are fused with the frequency domain features through the multi-domain attention feature fusion module (MDAFF), and then the number of channels of the feature map is adjusted to k through 1×1 convolution to obtain Fmask:
[0135] Fmask=Conv(MDAFF(F1,F2)).
[0136] In one embodiment, the multi-domain attention feature fusion module includes spatial domain features F1 and frequency domain features F2, where:
[0137] S21 performs initial feature fusion on the spatial domain features F1 and the frequency domain features F2. The fused features focus on the scale of the channel through point-by-point convolution. It includes point-by-point convolution and ReLU modules, which is equivalent to a lightweight channel attention module.
[0138] S22 is activated by the sigmoid function to obtain a weight value with an output value of (0, 1). The weight value is obtained through training, and the weighted average of F1 and F2 is used to obtain the feature map F'.
[0139] Use the unsqueeze method to increase the dimension of the feature map F', expanding it from a three-dimensional tensor [height, width, channel] to a four-dimensional tensor [depth, height, width, channel], forming a 3D feature map for subsequent convolution;
[0140] S23 uses 3D convolution to extract sequence features from feature maps and restore the three-dimensional tensor. The calculation formula is:
[0141]
[0142]
[0143] F=squeeze(3DConv(unsqueeze(F')))
[0144] The feature F is obtained after the fusion of spatial domain and frequency domain.
[0145] The method for segmenting small road targets in remote sensing images according to the embodiment of the present application can provide fast and accurate road target instance segmentation results.
[0146] The above description is merely an exemplary embodiment of the present application and is not intended to limit the protection scope of the present application. The protection scope of the present application is determined by the appended claims.
Claims
1. A method for segmenting road targets in remote sensing images, characterized in that: The steps include: The S1 remote sensing satellite acquires the remote sensing image of the target to be detected and pre-processes the remote sensing image through the input module; The S2 backbone network extracts features from the preprocessed image and fuses the extracted multi-resolution features using a two-stream feature pyramid at the neck; The S3 head is divided into two branches: the detection head and the prototype. The image enters the head to complete the category prediction and instance segmentation refinement tasks; S4 outputs the high-resolution remote sensing image road target instance segmentation results; Wherein, the DMH-YOLO model is used to perform instance segmentation on the target remote sensing image to determine the target object in the target remote sensing image; The backbone network adopts the Darknet-53 structural framework and integrates the C2f module to realize the residual learning mechanism; mix-pooling is used to combine maximum pooling and soft-pooling; the neck adopts a two-stream feature pyramid to fuse the feature information extracted by the backbone network to obtain features P3, P4, and P5.
2. The method for segmenting road target instances in remote sensing images according to claim 1, wherein the head network takes the feature map P3 as input and adopts a double mask branch, wherein the double mask branch is divided into a spatial domain branch and a frequency domain branch; in, The spatial domain branch is a fully convolutional structure. For input P3, it first passes through three 3×3 convolutional layers to extract mask features. The feature map is then upsampled to twice the resolution of the original feature map; Perform 3×3 convolution on the feature map to obtain the mask feature F1 containing rich spatial domain information; The F1 calculation formula is: F1 = Conv(upSample(Con(P3))) The frequency domain branch uses the wavelet transform module to perform frequency domain feature mapping through the following steps: First, the wavelet transform module is used to convert the input P3 into four frequency domain components, one of which is a low-frequency component, and the other three are horizontal, vertical and diagonal high-frequency components. The low-frequency information is convolved to obtain low-frequency features, and the three high-frequency information are concatenated to form a high-frequency signal through Concat. Secondly, nonlinear operations are performed on it to obtain high-frequency features; Again, the low-frequency features are combined with the high-frequency features through Concat to obtain the frequency domain features; Then, the feature is upsampled to obtain the frequency domain feature F2, and the calculation formula of F2 is: F2 = Conv(upSample(WTB(P3))) Finally, the spatial domain features are fused with the frequency domain features through the multi-domain attention feature fusion module, and then the number of channels of the feature map is adjusted to k through 1×1 convolution to obtain Fmask: Fmask=Con1v(MDAFF(F1,F2)).
3. The method for segmenting road target instances in remote sensing images according to claim 1, wherein the fusion method of the multi-domain attention feature fusion module comprises the following steps: S21 performs initial feature fusion on the spatial domain feature F1 and the frequency domain feature F2. The fused features focus on the scale of the channel through point-by-point convolution, including point-by-point convolution and ReLU modules; S22 is activated by the sigmoid function to obtain a weight value with an output value of (0, 1). The weight value is obtained through training, and the weighted average of F1 and F2 is used to obtain the feature map F'. Use the unsqueeze method to increase the dimension of the feature map F', expanding it from a three-dimensional tensor [height, width, channel] to a four-dimensional tensor [depth, height, width, channel], forming a 3D feature map for subsequent convolution; S23 uses 3D convolution to extract sequence features from feature maps and restore the three-dimensional tensor. The calculation formula is: F=squeeze(3DConv(unsqueeze(F'))) The feature F is obtained after the fusion of spatial domain and frequency domain.
4. The method for segmenting road target instances in remote sensing images as claimed in claim 1, wherein the loss function of the DMH-YOLO model comprises three parts: classification loss, regression loss and mask classification loss; the classification loss uses a sigmoid function to calculate the probability of each category, and a binary cross entropy loss is used to calculate the global category loss; The regression loss is NWD loss, and CIou_Loss is used to calculate the IoU between the predicted box and the target box; The mask classification loss is calculated using binary cross entropy loss (BCE Loss); Lbox=Lnwd=1-NWD(Np,Ng) Lmask=BCE(M,MG).
5. The method for segmenting road target instances in remote sensing images as described in claim 1, wherein the image preprocessing includes enhancing image brightness, adjusting image size and grayscale image attributes, and the image size is adjusted to 1280*1280.
6. A remote sensing image road target instance segmentation device, comprising: An acquisition unit, used for acquiring a target remote sensing image to be instance segmented; A determination unit, used to perform instance segmentation on the remote sensing image using the DMH-YOLO model to determine the target object in the target remote sensing image; wherein the determination unit uses the segmentation method according to claim 2 to perform instance segmentation; The multi-domain attention feature fusion module performs initial feature fusion on the spatial domain feature F1 and the frequency domain feature F2. The fused features focus on the scale of the channel through point-by-point convolution, and include modules of point-by-point convolution and ReLU. The multi-domain attention feature fusion module adopts the fusion method described in claim 3.
Citation Information
Patent Citations
Food image segmentation method based on discrete wavelet attention network
CN116630964A
Multi-scale feature optimized remote sensing image segmentation model and method
CN118154868A
Ground feature identification method and system for power transmission line engineering survey task
CN118379655A
Panoptic segmentation-based optical remote-sensing image raft mariculture area classification method
WO2024087574A1
Cited By
Lightweight lightning image instance segmentation method
CN119672348A
Selective state space model road segmentation method based on frequency domain feature compensation
CN119723078A
Remote sensing image segmentation method based on global feature enhancement and Fourier detail adjustment
CN120765933A
Small target defect detection method with periodic texture material, equipment and medium
CN122550600A
