Special material semantic segmentation method and system for capturing Fourier boundary features in wide field

Through the method of capturing Fourier boundary features through wide domain, the coarse-grained features of the image are extracted and fused, which solves the problem of inaccurate boundary feature capture in transparent special material segmentation, improves segmentation accuracy and stability, and avoids vertically deepening of the network Over-abstract influence.

CN119048760APending Publication Date: 2024-11-29CAS OF CHENGDU INFORMATION TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411272871.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-12
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

When handling semantic segmentation of transparent special materials, it is difficult to accurately capture robust boundary features and stable boundary constraints, resulting in reduced segmentation accuracy, especially in vertically deepening the negative impact of excessive abstract content capture in the network is difficult to avoid. .

Method used

The method of wide-domain capture of Fourier boundary features is used to extract target downsampled coarse-grained features and target upsampled coarse-grained features through pre-trained image segmentation model, and the Fourier transform and cross-transpose attention features are combined. Convolution and Batch normalization process enhances boundary features to resist the effects of reflected noise of transparent materials.

Benefits of technology

The segmentation accuracy of semantic segmentation of special materials is improved, and the negative impact of excessive abstract content capture caused by vertical deepening of the network is avoided, which enhances the robustness of boundary features and the stability of segmentation results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119048760A_ABST
    Figure CN119048760A_ABST
Patent Text Reader

Abstract

The invention discloses a special material semantic segmentation method and system for capturing Fourier boundary features in a wide range, and the method comprises the following steps: obtaining a to-be-segmented image which comprises an image of a special material, and the special material is a material with the transparency greater than a preset threshold value; inputting the to-be-segmented image into a pre-trained image segmentation model to enable the image segmentation model to extract a target down-sampling coarseness feature, a target up-sampling coarseness feature and a cross transposition attention feature of the to-be-segmented image, the target up-sampling coarseness features are obtained by up-sampling corresponding target down-sampling coarseness features; fourier transform is carried out on the target down-sampling coarse-grained features to obtain boundary enhancement features; and according to the cross transpose attention feature, fusing the boundary enhancement feature and the target up-sampling coarseness feature to obtain a fused feature.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and system for semantic segmentation of special materials by capturing Fourier boundary features over a wide range. Background Art

[0002] Common solid transparent special materials such as glass, transparent plastic plates and optical lenses are widely used in various scenarios, including but not limited to streets, building exteriors and industrial production workshops, due to their unique light transmission and reflection properties. However, while this characteristic gives these special materials a wide range of application value, it also poses a great challenge to intelligent systems based on machine vision. Through prior knowledge and a complex visual nervous system, the human cognitive system can recognize whether such special materials exist and distinguish between virtual objects and real objects reflected on their surfaces. For machine vision systems such as scene depth estimation, target detection and driving obstacle avoidance that rely on visual algorithms, their accuracy will be greatly affected by the introduction of erroneous depth information and visual noise due to the characteristics of special materials.

[0003] In actual application scenarios, transparent special materials can only provide blurred or incomplete boundaries, and the contextual content will be highly confused due to its transmission and reflection, resulting in reduced algorithm segmentation accuracy. Although current methods have made progress in combating blurred special material boundaries and incoherent contextual information, the core problem of segmentation tasks has not been fully explored. Existing methods often use deep and complex module models to extract fine-grained information to enhance the details of segmented objects. However, the excessive capture of large-area semantic information features is not compatible with the special material segmentation task. In addition, existing methods cannot provide robust boundary features and stable boundary constraints, and are easily interfered by other features in the subsequent network abstraction process, affecting the final segmentation results. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for semantic segmentation of special materials with wide-domain capture of Fourier boundary features, so as to avoid the negative impact of excessively abstract content capture caused by vertically deepening the network, thereby improving segmentation accuracy.

[0005] A first aspect of an embodiment of the present invention provides a method for semantic segmentation of special materials by capturing Fourier boundary features over a wide range, comprising the following steps:

[0006] Acquire an image to be segmented, wherein the image to be segmented includes an image of a special material, and the special material is a material whose transparency is greater than a preset threshold;

[0007] Inputting the image to be segmented into a pre-trained image segmentation model, so that the image segmentation model can extract the target down-sampling coarse-grained features, target up-sampling coarse-grained features and cross-transposed attention features of the image to be segmented, wherein the target up-sampling coarse-grained features are obtained by upsampling the corresponding target down-sampling coarse-grained features; and,

[0008] Performing Fourier transform on the target downsampled coarse-grained features to obtain boundary enhancement features;

[0009] According to the cross-transposed attention feature, the boundary enhancement feature and the target up-sampled coarse-grained feature are fused to obtain a fused feature; and,

[0010] A segmentation result output by the image segmentation model according to the fusion feature is obtained, wherein the segmentation result represents a boundary position of the imaging of the special material in the image to be segmented.

[0011] Furthermore, the image segmentation model is also used for:

[0012] Using convolution to extract features from the image to be segmented, and performing Batch normalization processing to obtain preprocessed features;

[0013] The preprocessed features are subjected to a ReLU activation function and a downsampling operation to obtain initial features, so as to extract target downsampling coarse-grained features, target upsampling coarse-grained features and cross-transposed attention features of the image to be segmented based on the initial features.

[0014] Further, the image segmentation model includes a wide-area feature capture module, the wide-area feature capture module includes a first feature capture unit, a second feature capture unit, a third feature capture unit and a fourth feature capture unit, the first feature capture unit, the second feature capture unit, the third feature capture unit and the fourth feature capture unit are connected in sequence, and accordingly, the image segmentation model extracts the target down-sampling coarse-grained features and the target up-sampling coarse-grained features of the image to be segmented, including:

[0015] Downsampling the initial features of the image to be segmented twice by the first feature capture unit to obtain twice downsampled coarse-grained features, and upsampling the second downsampled coarse-grained features twice to obtain twice upsampled coarse-grained features;

[0016] The coarse-grained features obtained by the second upsampling of the previous feature capture unit are downsampled twice by each of the second feature capture unit, the third feature capture unit, and the fourth feature capture unit in sequence to obtain twice downsampled coarse-grained features, and the second downsampled coarse-grained features are upsampled twice to obtain twice upsampled coarse-grained features;

[0017] The target down-sampled coarse-grained features are obtained by concatenating the two down-sampled coarse-grained features of each feature capture unit, and the target up-sampled coarse-grained features are obtained by concatenating the two up-sampled coarse-grained features of each feature capture unit.

[0018] Furthermore, the image segmentation model includes a cross-transposition attention module, the wide-domain feature capture module is connected to the cross-transposition attention module, and the cross-transposition attention features extracted by the image segmentation model include:

[0019] Downsampling the initial features of the image to be segmented to obtain features to be transposed;

[0020] Cross-multiplying the first down-sampled coarse-grained features of the first feature capture unit and the features to be transposed to obtain an attention matrix;

[0021] The first down-sampled coarse-grained features of the first feature capture unit and the features to be transposed are matrix multiplied with the attention matrix to obtain the cross-transposed attention features.

[0022] Furthermore, the boundary enhancement feature and the target up-sampled coarse-grained feature are fused according to the cross-transposed attention feature to obtain the fused feature including:

[0023] The cross-transposed attention feature is used as a modulation factor, the target up-sampled coarse-grained feature is used as a bias factor, and the boundary enhancement feature is convolutionally fused to obtain the fused feature.

[0024] Furthermore, the training of the image segmentation model includes:

[0025] Acquire a sample image to be segmented, wherein the sample image to be segmented includes a sample label, and the sample label is used to represent an ideal segmentation result of the sample image to be segmented;

[0026] Determining the real boundary features of the sample image to be segmented based on the Canny operator;

[0027] Inputting the sample image to be segmented into an initial image segmentation model to obtain predicted boundary features and segmentation results output by the image segmentation model;

[0028] For the real boundary features and the predicted boundary features, a pre-constructed boundary constraint loss Loss is used. B Calculate and obtain the first loss value;

[0029] For the segmentation result and the sample label, a second loss value is obtained by calculating a pre-constructed cross entropy loss;

[0030] The parameters of the image segmentation model are updated according to the first loss value and the second loss value until the image segmentation model converges.

[0031] A second aspect of an embodiment of the present invention provides a system for semantic segmentation of special materials by capturing Fourier boundary features in a wide area, using any of the above methods for semantic segmentation of special materials by capturing Fourier boundary features in a wide area, specifically comprising:

[0032] An image acquisition module, used for acquiring an image to be segmented, wherein the image to be segmented includes an image of a special material, and the special material is a material whose transparency is greater than a preset threshold;

[0033] The image preprocessing module is used for preprocessing the segmented image and includes:

[0034] Using convolution to extract features from the image to be segmented, and performing Batch normalization processing to obtain preprocessed features;

[0035] The preprocessed features are subjected to a ReLU activation function and a downsampling operation to obtain the initial features.

[0036] An image segmentation model running module is used to input the preprocessed image to be segmented into a pre-trained image segmentation model, so that the image segmentation model can extract the target down-sampling coarse-grained features, target up-sampling coarse-grained features and cross-transposed attention features of the image to be segmented; and,

[0037] Performing Fourier transform on the target downsampled coarse-grained features to obtain boundary enhancement features;

[0038] According to the cross-transposed attention feature, the boundary enhancement feature and the target up-sampled coarse-grained feature are fused to obtain a fused feature; and,

[0039] Obtaining a segmentation result output by the image segmentation model according to the fusion feature, wherein the segmentation result represents a boundary position of the imaging of the special material in the image to be segmented;

[0040] The image segmentation model training module is used to train the image segmentation model.

[0041] The present invention has at least the following beneficial effects:

[0042] The present invention provides a method and system for semantic segmentation of special materials by capturing Fourier boundary features over a wide range. The target down-sampling coarse-grained features of the image to be segmented and the corresponding target up-sampling coarse-grained features (i.e., the coarse-grained semantic information of the shallow layer of the image) extracted by this method include the undispersed and uneroded boundary features of the special material imaging in the image to be segmented. In addition, in view of the reflection characteristics of special materials, the present invention resists the influence of the reflection noise of transparent materials on the consistency destruction of the shallow semantic information area by cross-transposing attention. Finally, the boundary features are enhanced by Fourier convolution, which effectively balances the heterogeneous coarse-grained features, attention semantic areas, and Fourier boundary information. Compared with the existing methods, the present invention avoids the negative impact of the capture of excessively abstract content caused by vertically deepening the network, and improves the segmentation accuracy.

[0043] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flowchart of a method for semantic segmentation of special materials by capturing Fourier boundary features in a wide range provided by an embodiment of the present invention;

[0045] Figure 2 Schematic diagram of a device for a semantic segmentation system for special materials that captures Fourier boundary features over a wide range provided by an embodiment of the present invention;

[0046] Figure 3 It is a schematic diagram of the structure of a feature capture unit provided in an embodiment of the present invention.

[0047] Figure 4 It is a schematic diagram of the segmentation effect provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0049] In the description of the present invention, it is necessary to understand that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0050] Example 1: Combination Figure 1-Figure 4 To illustrate this embodiment,

[0051] This embodiment is a method for semantic segmentation of special materials by capturing Fourier boundary features in a wide range, including the following steps:

[0052] Acquire an image to be segmented, wherein the image to be segmented includes an image of a special material, and the special material is a material whose transparency is greater than a preset threshold;

[0053] Inputting the image to be segmented into a pre-trained image segmentation model, so that the image segmentation model can extract the target down-sampling coarse-grained features, target up-sampling coarse-grained features and cross-transposed attention features of the image to be segmented, wherein the target up-sampling coarse-grained features are obtained by upsampling the corresponding target down-sampling coarse-grained features; and,

[0054] Performing Fourier transform on the target downsampled coarse-grained features to obtain boundary enhancement features;

[0055] According to the cross-transposed attention feature, the boundary enhancement feature and the target up-sampled coarse-grained feature are fused to obtain a fused feature; and,

[0056] A segmentation result output by the image segmentation model according to the fusion feature is obtained, wherein the segmentation result represents a boundary position of the imaging of the special material in the image to be segmented.

[0057] The present invention provides a method and system for semantic segmentation of special materials by capturing Fourier boundary features over a wide range. The target down-sampling coarse-grained features of the image to be segmented and the corresponding target up-sampling coarse-grained features (i.e., the coarse-grained semantic information of the shallow layer of the image) extracted by this method include the undispersed and uneroded boundary features of the special material imaging in the image to be segmented. In addition, in view of the reflection characteristics of special materials, the present invention resists the influence of the reflection noise of transparent materials on the consistency destruction of the shallow semantic information area by cross-transposing attention. Finally, the boundary features are enhanced by Fourier convolution, which effectively balances the heterogeneous coarse-grained features, attention semantic areas, and Fourier boundary information. Compared with the existing methods, the present invention avoids the negative impact of the capture of excessively abstract content caused by vertically deepening the network, and improves the segmentation accuracy.

[0058] Furthermore, the image segmentation model is also used for:

[0059] Using convolution to extract features from the image to be segmented, and performing Batch normalization processing to obtain preprocessed features;

[0060] The preprocessed features are subjected to a ReLU activation function and a downsampling operation to obtain initial features, so as to extract target downsampling coarse-grained features, target upsampling coarse-grained features and cross-transposed attention features of the image to be segmented based on the initial features.

[0061] The specific implementation method is: use convolution to extract preliminary features of the segmented image x, and perform Batch normalization to obtain preprocessing features F pre , the specific formula is: in, Indicates that 3×3 convolution is used to extract features of the target image x to be segmented, and BN means Batch normalization processing;

[0062] For the preprocessing feature F pre Perform ReLU activation function and downsampling operation to obtain the initial feature A. The specific formula is: A = μ(ReLU(F pre )).

[0063] Further, the image segmentation model includes a wide-area feature capture module, the wide-area feature capture module includes a first feature capture unit, a second feature capture unit, a third feature capture unit and a fourth feature capture unit, the first feature capture unit, the second feature capture unit, the third feature capture unit and the fourth feature capture unit are connected in sequence, and accordingly, the image segmentation model extracts the target down-sampling coarse-grained features and the target up-sampling coarse-grained features of the image to be segmented, including:

[0064] Downsampling the initial features of the image to be segmented twice by the first feature capture unit to obtain twice downsampled coarse-grained features, and upsampling the second downsampled coarse-grained features twice to obtain twice upsampled coarse-grained features;

[0065] The coarse-grained features obtained by the second upsampling of the previous feature capture unit are downsampled twice by each of the second feature capture unit, the third feature capture unit, and the fourth feature capture unit in sequence to obtain twice downsampled coarse-grained features, and the second downsampled coarse-grained features are upsampled twice to obtain twice upsampled coarse-grained features;

[0066] The target down-sampled coarse-grained features are obtained by concatenating the two down-sampled coarse-grained features of each feature capture unit, and the target up-sampled coarse-grained features are obtained by concatenating the two up-sampled coarse-grained features of each feature capture unit.

[0067] The wide-area feature capture module is composed of four feature capture units connected horizontally. The feature capture unit structure is as follows: Figure 3As shown, the output of the previous feature capture unit serves as the input of the current feature capture unit. This module is responsible for capturing shallow low-level semantic information and matching the shape features of special materials with their inherent properties. The working steps of the wide-domain feature capture module are: (1) The input feature enters the constructed feature capture unit, and the feature capture unit specifically includes two consecutive downsamplings and two consecutive upsamplings. (2) The second downsampling result and the second upsampling result of this unit are recorded at the same time. (3) All the second downsampling results of the four feature capture units are output as the target downsampling coarse-grained features and the second upsampling results are output as the target upsampling coarse-grained features, and are named the first downsampling coarse-grained features, the first upsampling coarse-grained features, the second downsampling coarse-grained features, and the second upsampling coarse-grained features according to the feature capture unit serial number.

[0068] Among them, the initial feature A is downsampled twice and upsampled twice continuously through four horizontally spliced ​​feature capture units, and the second downsampling result and the second upsampling result of each feature capture unit are recorded. The specific formula is as follows:

[0069] A1=μ(A)

[0070] A2=μ(A1)

[0071] B1=η(A2)

[0072] B2=η(A1+B1)

[0073] Wherein, A is the initial feature, A1 represents the first down-sampled coarse-grained feature in the feature capture unit, A2 represents the second down-sampled coarse-grained feature in the feature capture unit, μ represents the down-sampling operation, B1 represents the first up-sampled coarse-grained feature in the feature capture unit, B2 represents the second up-sampled coarse-grained feature in the feature capture unit, η represents the up-sampling operation, and B2 is the output of the feature capture unit.

[0074] The output of the previous feature capture unit is used as the input of the current feature capture unit, and concatenated to obtain the target second down-sampled coarse-grained feature A out And the target second up-sampled coarse-grained feature B out , the specific formula is:

[0075]

[0076]

[0077] in, They represent the second down-sampling coarse-grained features and the second up-sampling coarse-grained features of the first feature capture unit of the module, respectively. They respectively represent the second down-sampled coarse-grained features and the second up-sampled coarse-grained features of the second feature capture unit of the module, and so on.

[0078] Furthermore, the image segmentation model includes a cross-transposition attention module, the wide-domain feature capture module is connected to the cross-transposition attention module, and the cross-transposition attention features extracted by the image segmentation model include:

[0079] Downsampling the initial features of the image to be segmented to obtain features to be transposed;

[0080] Cross-multiplying the first down-sampled coarse-grained features of the first feature capture unit and the features to be transposed to obtain an attention matrix;

[0081] The first down-sampled coarse-grained features of the first feature capture unit and the features to be transposed are matrix multiplied with the attention matrix to obtain the cross-transposed attention features.

[0082] The cross-transposition attention module is responsible for focusing on the large-area coarse-grained features of the target, while supplementing the fine-grained features to maintain regional consistency. First, the cross-transposition attention module obtains the preprocessed features F pre , and then the preprocessing feature F pre Downsampling is performed to obtain the feature to be transposed C as the input feature of the cross-transposed attention module. The specific formula is: C = μ(F pre ). Subsequently, the cross-transposed attention module combines the first down-sampled coarse-grained feature A1 of the first feature capture unit and the feature to be transposed C to perform transposed cross multiplication, and uses the normalized multiplication result as the attention matrix [c]. Specifically:

[0083] [c] = σ(A1·C T ),σ(A1 T C)

[0084] Among them, σ is a normalization operation. Finally, the attention matrix [c] is matrix-multiplied with the first down-sampled coarse-grained feature A1 of the first feature capture unit and the feature to be transposed C to obtain the cross-transposed attention feature D. The specific formula is: D = [c] · [A, C].

[0085] Furthermore, the boundary enhancement feature and the target up-sampled coarse-grained feature are fused according to the cross-transposed attention feature to obtain the fused feature including:

[0086] The cross-transposed attention feature is used as a modulation factor, the target up-sampled coarse-grained feature is used as a bias factor, and the boundary enhancement feature is convolutionally fused to obtain the fused feature.

[0087] By using the convolution theory, the extracted coarse-grained and fine-grained heterogeneous features are effectively fused, and a learnable solution for feature self-adjustment and balanced fusion is provided while using Fourier transform to enhance the robustness of the generated boundary features, effectively improving the anchoring of boundary information and optimizing the segmentation results. out , target upsampling coarse-grained feature B out and the cross-transposed attention feature D as three heterogeneous features. First, the coarse-grained feature A of the target is downsampled out Perform Fourier transform to obtain the boundary enhancement feature F A The specific formula is F A =FFT(A out ), where FFT(A out ) indicates that A out Perform a fast Fourier transform operation. Then use the cross-transposed attention feature D as the modulation factor and target upsample the coarse-grained feature B out As a paranoid factor, fusion is performed through convolution theory. The specific formula is:

[0088] Y=D·F A +B out

[0089] Among them, Y represents the segmentation result after Fourier convolution feature fusion, such as Figure 4 shown.

[0090] Furthermore, the training of the image segmentation model includes:

[0091] Acquire a sample image to be segmented, wherein the sample image to be segmented includes a sample label, and the sample label is used to represent an ideal segmentation result of the sample image to be segmented;

[0092] Based on the Canny operator, the real boundary features of the sample image to be segmented are determined. The specific formula is: B =Canny(x), where Canny(x) represents the actual boundary feature x of the sample image x to be segmented extracted using the Canny operator. B ;

[0093] Inputting the sample image to be segmented into an initial image segmentation model to obtain predicted boundary features and segmentation results output by the image segmentation model;

[0094] For the real boundary features and the predicted boundary features, a pre-constructed boundary constraint loss Loss is used. B Calculate the first loss value, the specific formula is: Loss B =CE(x B ,B out), where CE(x B ,B out ) indicates that the cross entropy is used to calculate the true boundary feature x B and the target upsampled coarse-grained feature B out loss;

[0095] For the segmentation result and the sample label, a second loss value is obtained by calculating the pre-constructed cross entropy loss, and the specific formula is: Loss = CE (Y, y);

[0096] Among them, Y represents the segmentation result after Fourier convolution feature fusion, and y represents the true value corresponding to the input image x;

[0097] The parameters of the image segmentation model are updated according to the first loss value and the second loss value until the image segmentation model converges.

[0098] Example 2

[0099] The embodiment of the present invention further provides a system for semantic segmentation of special materials by capturing Fourier boundary features in a wide area, which adopts the above-mentioned method for semantic segmentation of special materials by capturing Fourier boundary features in a wide area, and specifically includes:

[0100] An image acquisition module, used for acquiring an image to be segmented, wherein the image to be segmented includes an image of a special material, and the special material is a material whose transparency is greater than a preset threshold;

[0101] The image preprocessing module is used for preprocessing the segmented image and includes:

[0102] Using convolution to extract features from the image to be segmented, and performing Batch normalization processing to obtain preprocessed features;

[0103] Performing ReLU activation function and downsampling operation on the preprocessed features to obtain the initial features.

[0104] An image segmentation model running module is used to input the preprocessed image to be segmented into a pre-trained image segmentation model, so that the image segmentation model can extract the target down-sampling coarse-grained features, target up-sampling coarse-grained features and cross-transposed attention features of the image to be segmented; and,

[0105] Performing Fourier transform on the target downsampled coarse-grained features to obtain boundary enhancement features;

[0106] According to the cross-transposed attention feature, the boundary enhancement feature and the target up-sampled coarse-grained feature are fused to obtain a fused feature; and,

[0107] Obtaining a segmentation result output by the image segmentation model according to the fusion feature, wherein the segmentation result represents a boundary position of the imaging of the special material in the image to be segmented;

[0108] The image segmentation model training module is used to train the image segmentation model.

[0109] It should also be noted that the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, the elements defined by the sentence "includes one..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements. The words "first", "second" and the like are used to indicate names, but do not indicate any specific order. The above schematically describes the invention and its implementation methods, which is not restrictive. The invention can be implemented in other specific forms without departing from the spirit or basic features of the invention. What is shown in the accompanying drawings is only one of the implementation methods of the invention, and the actual structure is not limited thereto. Any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it, and does not deviate from the purpose of the invention, and designs a structural mode and an embodiment similar to the technical solution without creativity, they should all belong to the scope of protection of this patent.

Claims

1. A semantic segmentation method for special materials with wide-domain Fourier boundary features, characterized in that: The following steps are involved: Acquire an image to be segmented, wherein the image to be segmented includes an image of a special material, and the special material is a material whose transparency is greater than a preset threshold; Inputting the image to be segmented into a pre-trained image segmentation model, so that the image segmentation model can extract the target down-sampling coarse-grained features, target up-sampling coarse-grained features and cross-transposed attention features of the image to be segmented, wherein the target up-sampling coarse-grained features are obtained by upsampling the corresponding target down-sampling coarse-grained features; and, Performing Fourier transform on the target downsampled coarse-grained features to obtain boundary enhancement features; According to the cross-transposed attention feature, the boundary enhancement feature and the target up-sampled coarse-grained feature are fused to obtain a fused feature; and, A segmentation result output by the image segmentation model according to the fusion feature is obtained, wherein the segmentation result represents a boundary position of the imaging of the special material in the image to be segmented.

2. According to the method for semantic segmentation of special materials with wide-area capture of Fourier boundary features according to claim 1, the image segmentation model is also used for: Using convolution to extract features from the image to be segmented, and performing Batch normalization processing to obtain preprocessed features; The preprocessed features are subjected to a ReLU activation function and a downsampling operation to obtain initial features, so as to extract target downsampling coarse-grained features, target upsampling coarse-grained features and cross-transposed attention features of the image to be segmented based on the initial features.

3. According to a wide-area capture Fourier boundary feature special material semantic segmentation method according to claim 2, the image segmentation model includes a wide-area feature capture module, the wide-area feature capture module includes a first feature capture unit, a second feature capture unit, a third feature capture unit and a fourth feature capture unit, the first feature capture unit, the second feature capture unit, the third feature capture unit and the fourth feature capture unit are connected in sequence, and accordingly, the image segmentation model extracts the target down-sampling coarse-grained features and the target up-sampling coarse-grained features of the image to be segmented, including: Downsampling the initial features of the image to be segmented twice by the first feature capture unit to obtain twice downsampled coarse-grained features, and upsampling the second downsampled coarse-grained features twice to obtain twice upsampled coarse-grained features; The coarse-grained features obtained by the second upsampling of the previous feature capture unit are downsampled twice by each of the second feature capture unit, the third feature capture unit, and the fourth feature capture unit in sequence to obtain twice downsampled coarse-grained features, and the second downsampled coarse-grained features are upsampled twice to obtain twice upsampled coarse-grained features; The target down-sampled coarse-grained features are obtained by concatenating the two down-sampled coarse-grained features of each feature capture unit, and the target up-sampled coarse-grained features are obtained by concatenating the two up-sampled coarse-grained features of each feature capture unit.

4. According to claim 2, a method for semantic segmentation of special materials by capturing Fourier boundary features in a wide range, characterized in that: The image segmentation model includes a cross-transposition attention module, the wide-domain feature capture module is connected to the cross-transposition attention module, and the cross-transposition attention features extracted by the image segmentation model include: Downsampling the initial features of the image to be segmented to obtain features to be transposed; Cross-multiplying the first down-sampled coarse-grained features of the first feature capture unit and the features to be transposed to obtain an attention matrix; The first down-sampled coarse-grained features of the first feature capture unit and the features to be transposed are matrix multiplied with the attention matrix to obtain the cross-transposed attention features.

5. According to the method for semantic segmentation of special materials with wide-area Fourier boundary feature capture in claim 1, the boundary enhancement feature and the target up-sampled coarse-grained feature are fused according to the cross-transposed attention feature to obtain the fused feature, which includes: The cross-transposed attention feature is used as a modulation factor, the target up-sampled coarse-grained feature is used as a bias factor, and the boundary enhancement feature is convolutionally fused to obtain the fused feature.

6. According to the method for semantic segmentation of special materials with wide-area Fourier boundary feature capture as described in claim 1, the training of the image segmentation model comprises: Acquire a sample image to be segmented, wherein the sample image to be segmented includes a sample label, and the sample label is used to represent an ideal segmentation result of the sample image to be segmented; Determining the real boundary features of the sample image to be segmented based on the Canny operator; Inputting the sample image to be segmented into an initial image segmentation model to obtain predicted boundary features and segmentation results output by the image segmentation model; For the real boundary features and the predicted boundary features, a pre-constructed boundary constraint loss Loss is used. B Calculate and obtain the first loss value; For the segmentation result and the sample label, a second loss value is obtained by calculating a pre-constructed cross entropy loss; The parameters of the image segmentation model are updated according to the first loss value and the second loss value until the image segmentation model converges.

7. A special material semantic segmentation system that captures Fourier boundary features over a wide range, characterized in that: A method for semantic segmentation of special materials with wide-area Fourier boundary features as described in any one of claims 1 to 6 is used, specifically comprising: An image acquisition module, used for acquiring an image to be segmented, wherein the image to be segmented includes an image of a special material, and the special material is a material whose transparency is greater than a preset threshold; The image preprocessing module is used for preprocessing the segmented image and includes: Using convolution to extract features from the image to be segmented, and performing Batch normalization processing to obtain preprocessed features; The preprocessed features are subjected to a ReLU activation function and a downsampling operation to obtain the initial features. An image segmentation model running module is used to input the preprocessed image to be segmented into a pre-trained image segmentation model, so that the image segmentation model can extract the target down-sampling coarse-grained features, target up-sampling coarse-grained features and cross-transposed attention features of the image to be segmented; and, Performing Fourier transform on the target downsampled coarse-grained features to obtain boundary enhancement features; According to the cross-transposed attention feature, the boundary enhancement feature and the target up-sampled coarse-grained feature are fused to obtain a fused feature; and, Obtaining a segmentation result output by the image segmentation model according to the fusion feature, wherein the segmentation result represents a boundary position of the imaging of the special material in the image to be segmented; The image segmentation model training module is used to train the image segmentation model.

Citation Information

Patent Citations

  • Semantic segmentation method and device for transparent object in image and electronic equipment

    CN116993973A

  • Stationary service appliance for a poly functional roaming device

    US20240142994A1