Image segmentation dataset protection method based on contour and texture-aware perturbations

Abstract

The present invention relates to the technical field of data protection and medical image processing, and in particular to a method for protecting an image segmentation dataset based on contour and texture perception perturbations, comprising: preprocessing to obtain a standard image; performing multi-scale convolution and channel attention mechanism processing to complete encoding, and performing cross-layer feature aggregation to complete decoding; cropping the decoding result based on a segmentation mask image to obtain a contour perturbation; obtaining a texture feature map of the standard image, cropping the initial perturbation based on the texture feature map to obtain a texture perturbation; optimizing the loss functions of the contour perturbation and the texture perturbation; obtaining an optimal contour perturbation and an optimal texture perturbation based on an optimized contour perturbation generator and a texture generator; and fusing the standard image, the optimal contour perturbation, and the optimal texture perturbation to obtain a protected medical image. The present invention can protect a medical image segmentation dataset.

Description

Technical Field

[0001] The present invention relates to the technical field of data protection and medical image processing, and in particular to an image segmentation dataset protection method based on contour and texture perception perturbations. Background Art

[0002] With the widespread application of deep learning in medical image analysis, medical image segmentation (MIS) technology has played a vital role in disease diagnosis and treatment planning. To improve the accuracy of medical image analysis models, an increasing number of medical imaging datasets are being publicly released for researchers to use. However, with the widespread sharing and open use of these datasets, the risk of unauthorized data misuse has gradually increased. In commercial environments, unauthorized data may be used to train commercial models, posing the risk of data privacy breaches and unfair competition. Therefore, how to effectively protect data security while sharing medical image data has become a pressing issue.

[0003] To prevent the unauthorized use of image data, some methods have proposed the concept of "Unlearnable Examples" (UE). These methods add protective perturbations to images to disrupt the model's learning process, preventing unauthorized deep learning models from extracting useful features. These methods have achieved some success in natural image classification tasks, protecting image intellectual property rights. However, these UE methods have had limited success in medical image segmentation. Dynamic feature regions in medical images often contain low contrast and blurred boundaries, making it difficult for traditional feature extraction methods to accurately capture the characteristic information in these regions. Furthermore, how to adaptively adjust processing strategies based on feature importance and how to enhance the model's ability to identify key regions while preserving detail remain key challenges. In medical image processing, shallow features contain rich detail information such as edges and textures, while deep features contain more semantic information. These two types of features have their own unique but complementary characteristics. Effectively integrating these different levels of features to reduce information redundancy during feature fusion while maintaining anatomical integrity and boundary accuracy is also a key issue that needs to be addressed.

[0004] Therefore, MIS datasets urgently need a new protection method that can prevent unauthorized models from effectively learning the dataset by adding non-learnable perturbations without affecting the key features of medical images, while showing strong robustness on different model architectures to meet the growing security needs in medical image data sharing. Summary of the Invention

[0005] In view of the above problems, the present invention provides an image segmentation dataset protection method based on contour and texture perception perturbations, which solves the problems in the existing technology of how to accurately identify and process dynamic feature areas and how to fuse different levels of features in medical images.

[0006] The present invention provides an image segmentation dataset protection method based on contour and texture perception perturbation, comprising the following steps:

[0007] Step S1: preprocessing the medical images in the samples in the medical image segmentation dataset to obtain standard images; the samples in the medical image segmentation dataset include medical images and segmentation mask images;

[0008] Step S2: using a contour perturbation generator to perform multi-scale convolution and channel attention mechanism processing on the standard image to complete encoding, and performing cross-layer feature aggregation to complete decoding; cropping the decoding result based on the segmentation mask image corresponding to the standard image to obtain a contour perturbation;

[0009] Step S3: Processing the standard image using a texture generator to obtain an initial disturbance; obtaining a texture feature map of the standard image, and cropping the initial disturbance based on the texture feature map to obtain a texture disturbance;

[0010] Step S4: establishing a loss function for evaluating the contour perturbation and texture perturbation, optimizing the contour perturbation generator and the texture generator based on the loss function; and obtaining the optimal contour perturbation and the optimal texture perturbation based on the optimized contour perturbation generator and the texture generator;

[0011] Step S5: Fusing the standard image, the optimal contour perturbation, and the optimal texture perturbation to obtain a protected medical image.

[0012] Preferably, the step S2 specifically includes:

[0013] Step S2-1: using the standard image as an input feature map, performing processing by the dynamic feature enhancement module twice in sequence, and using the dynamic enhanced feature maps output by the first and second dynamic feature enhancement modules as the shallow feature map and the deep feature map, respectively;

[0014] Step S2-2: performing a convolution operation on the shallow feature map, performing an upsampling operation on the deep feature map, and concatenating the results of the convolution operation and the upsampling operation to obtain a cross-layer aggregated feature map;

[0015] Step S2-3: obtaining a binary image of the target area contour from the segmentation mask image corresponding to the standard image, and cropping the cross-layer aggregated feature map based on the binary image of the target area contour to obtain contour disturbance.

[0016] Preferably, in step S2-1, the processing expression of the dynamic feature enhancement module is:

[0017] F DFEM =σ(W c GlobalPool(F)+W s *F)·F

[0018] Among them, F DFEM is the dynamic enhancement feature map, F is the input feature map, GlobalPool(·) represents global pooling, W c and W s are channel weight and spatial attention weight respectively, σ(·) is the Sigmoid activation function, and * represents the convolution operation.

[0019] Preferably, in step S2-2, the calculation expression of the cross-layer aggregated feature map is:

[0020] F CLFAM =Concat(Conv1×1(F shallow ),Upsample(F deep ))

[0021] Among them, F CLFAM represents the cross-layer aggregated feature map, F shallow and F deep are shallow feature maps and deep feature maps respectively, Conv1×1(·) is a 1×1 convolution layer; Upsample(·) represents an upsampling operation, and Concat(·) represents a concatenation operation.

[0022] Preferably, in step S2-3, the calculation expression of the contour disturbance is:

[0023] δ c =Clip [-∈,∈] [F CLFAM ⊙y c ]

[0024] Among them, δ c Represents contour disturbance, Clip [-∈,∈] [·] indicates that the absolute value of the amplitude is limited to no more than ∈, ⊙ indicates the Hadamard product operation, y c A binary image representing the outline of the target area.

[0025] Preferably, step S3 specifically includes:

[0026] Step S3-1, extracting a texture feature map from the standard image using a local binary pattern algorithm;

[0027] Step S3-2, obtaining a non-contour target area based on a binary image of the target area contour using the segmentation mask image corresponding to the standard image;

[0028] Step S3-3: Process the standard image using a texture generator to obtain an initial disturbance, and crop the initial disturbance based on the texture feature map and the non-contour target area to obtain a texture disturbance.

[0029] Preferably, in step S3-2, the non-contour target area y t The calculation expression is:

[0030] y t =(1-y c )⊙y

[0031] Among them, y is the segmentation mask image corresponding to the standard image;

[0032] In step S3-3, the calculation expression of the texture disturbance is:

[0033]

[0034] Among them, δ t is the texture perturbation, x t is the texture feature map, y t is the non-contour target area, x represents the input standard image, F t (·) represents the texture generator, Indicates that the absolute value of the limit amplitude does not exceed ∈·x t ⊙y t .

[0035] Preferably, step S5 specifically includes:

[0036] The input standard image, optimal contour perturbation, and optimal texture perturbation are weighted averaged and cropped to generate a protected medical image. The expression is:

[0037] x p =Clip [0,1] [δ c +δ t +x]

[0038] Among them, x p Represents protected medical images, Clip [0,1] Indicates that the amplitude is limited to the interval [0,1].

[0039] Compared with the prior art, the present invention has at least the following beneficial effects:

[0040] (1) The present invention uses a contour perturbation generator to perform multi-scale convolution and channel attention processing, which can effectively extract image features and enhance decoding effects, thereby generating high-quality contour perturbations. At the same time, the application of a texture generator ensures that the delicate texture of the image is preserved and optimized, thereby generating texture perturbations. The standard image is fused with the optimal perturbation to form a protected medical image, improving the security of the medical image dataset.

[0041] (2) The present invention can achieve adaptive enhancement of features in different regions through the combination of multi-scale convolution and channel attention mechanism, enhance the feature detail extraction effect of low-contrast areas and fuzzy boundaries in medical images, improve the segmentation performance of the model when processing complex medical images, and especially enhance the ability to recognize the boundaries of different anatomical structures.

[0042] (3) This invention integrates features from different levels by adopting a multi-branch structure and introducing a feature compression mechanism to reduce information redundancy, effectively fusing shallow boundary features with deep semantic information. This improves the boundary accuracy and consistency of the segmentation results, and can better maintain the integrity of the anatomical structure when processing medical images with complex anatomical information. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings are only for purposes of illustrating particular embodiments and are not to be considered limiting of the invention.

[0044] Figure 1 This is a flowchart of the image segmentation dataset protection method based on contour and texture perception perturbations provided by the present invention.

[0045] Figure 2 This is a processing structure diagram of the image segmentation dataset protection method based on contour and texture perception perturbation provided by the present invention.

[0046] Figure 3 This is a flow chart of the dynamic feature enhancement processing provided by the present invention.

[0047] Figure 4 This is a flowchart of the cross-layer feature aggregation processing provided by the present invention. DETAILED DESCRIPTION

[0048] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. In addition, the present invention can also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited by the specific embodiments disclosed below.

[0049] The present invention provides a method for generating non-learnable samples based on contour and texture perception to protect medical image segmentation datasets from being used for training by unauthorized deep learning models.

[0050] In order to illustrate the effectiveness of the method proposed by the present invention, the above technical solution of the present invention is described in detail below through a specific embodiment. Figure 1 、 Figure 2 As shown in FIG, a method for protecting an image segmentation dataset based on contour and texture perception perturbations is disclosed. The specific implementation steps are as follows:

[0051] Step S1: pre-processing the medical images in the samples in the medical image segmentation dataset to obtain standard images; the samples in the medical image segmentation dataset include medical images and segmentation mask images.

[0052] For the input medical image, normalization, denoising and size adjustment are first performed to form a standard image.

[0053] In some embodiments, the input medical image may be a medical image in a sample in a medical image segmentation dataset, wherein each sample in the medical image segmentation dataset includes a paired medical image and a segmentation mask image, and the segmentation mask image is a binary image used to mark the true value of the segmentation result.

[0054] Step S2: Use a contour perturbation generator to perform multi-scale convolution and channel attention mechanism processing on the standard image to complete encoding, and perform cross-layer feature aggregation to complete decoding; based on the segmentation mask image corresponding to the standard image, the decoding result is cropped to obtain the contour perturbation.

[0055] In this step, the present invention uses an encoder-decoder-based contour perturbation generator to enhance the difference between the contour and background areas in the image, forming a perturbation targeting the contour in the image. This perturbation reduces the ability of unauthorized segmentation models to learn the original features while maintaining the perceptual quality of the image.

[0056] For the encoder of the contour perturbation generator, the present invention uses a dynamic feature enhancement module (DFEM) to extract features from the standardized image. The dynamic feature enhancement module combines multi-scale convolution and channel attention mechanism to achieve adaptive enhancement of features in different regions and obtain a dynamic enhanced feature map F D2EM .

[0057] Dynamic Enhanced Feature Map F DFEM The calculation expression is as follows:

[0058] F DFEM =σ(Wc GlobalPool(F)+W s *F)·F

[0059] Among them, F is the input feature map, GlobalPool(·) represents global pooling, W c and W s are channel weight and spatial attention weight respectively, σ(·) is the Sigmoid activation function, and * represents the convolution operation.

[0060] like Figure 3 As shown in the figure, the processing process of the dynamic feature enhancement module of the present invention is: perform a global pooling operation on the input feature map F to obtain a global feature vector. The global feature vector represents the overall information of the feature map. The global feature vector is pooled by the channel weight W c Perform linear transformation and pass spatial attention weight W to the input feature map F s Perform linear transformation, input the results of channel and spatial attention linear transformation into the Sigmoid activation function, multiply the activated features with the input feature map F to achieve adaptive enhancement of the features, and finally obtain a dynamically enhanced feature map.

[0061] In the encoder of the contour perturbation generator of the present invention, multi-scale convolution is performed, and the steps include: taking the standard image as the input feature map, inputting it into the dynamic feature enhancement module, performing the processing of the dynamic feature enhancement module twice in sequence, and taking the dynamic enhancement feature maps output by the two dynamic feature enhancement modules as the shallow feature maps F shallow and deep feature map F deep .

[0062] The dynamic enhancement feature map obtained by the dynamic feature enhancement module in the present invention can achieve dynamic enhancement of the features of low-contrast areas (such as fuzzy boundaries), improve the segmentation performance of the model on complex medical images, and dynamically adjust the weights according to the input features to improve the model's attention to feature details, especially the ability to recognize the boundaries of different anatomical structures.

[0063] For the decoder of the contour perturbation generator, the present invention adopts a cross-layer feature aggregation module (CLFAM) to decode the input feature map to achieve efficient integration of features at different levels and obtain a decoded image.

[0064] The cross-layer feature aggregation module integrates shallow and deep features through a multi-branch structure, and introduces a feature compression mechanism to reduce redundancy. The cross-layer aggregation feature map F CLFAM The calculation expression is as follows:

[0065] F CLFAM=Concat(Conv1×1(F shallow ),Upsample(F deep ))

[0066] Among them, F shallow and F deep are shallow feature maps and deep feature maps respectively. Conv1×1(·) is a 1×1 convolution layer for feature compression. Upsample(·) is an upsampling operation for adjusting feature resolution. Concat(·) represents a concatenation operation.

[0067] like Figure 4 As shown in the figure, the processing process of the cross-layer feature aggregation module of the present invention is as follows: a 1×1 convolution operation is performed on the shallow feature map to reduce redundant information while retaining important detail features; a 1×1 convolution operation is performed on the deep feature map F deep Upsampling is performed to match the size of the shallow feature map. The shallow features processed by 1×1 convolution and the upsampled deep features are concatenated to integrate the shallow detail information and the deep semantic information; ultimately, a cross-layer aggregated feature map is obtained.

[0068] The decoder of the contour perturbation generator of this invention can effectively fuse shallow boundary features with deep semantic information, improving the boundary accuracy and consistency of the segmentation results. It also reduces the inconsistency between shallow and deep features during the decoding process and enhances the complete representation of anatomical structures in complex images.

[0069] Process the segmentation mask image in the sample of the medical image segmentation dataset to obtain the binary image y of the target area contour c , based on y c , for the cross-layer aggregated feature map F CLFAM Perform cropping to obtain the contour perturbation δ c , the expression is:

[0070] δ c =Clip [-∈,∈] [F CLFAM ⊙y c ]

[0071] Among them, Clip [-∈,∈] [·] indicates that the amplitude is restricted to not exceed ∈, and ⊙ indicates the Hadamard product operation.

[0072] Step S3: Process the standard image using a texture generator to obtain an initial disturbance; obtain a texture feature map of the standard image, and crop the initial disturbance based on the texture feature map to obtain a texture disturbance.

[0073] In this step, the present invention generates texture perturbations based on the texture features in the medical image to generate adaptive texture perturbations to interfere with the learning of texture features by the unauthorized model. The specific steps are as follows:

[0074] The computer uses the Local Binary Pattern (LBP) algorithm to extract the texture feature map x in the standard image. t LBP converts the texture pattern in the image into a binary feature code, which represents the texture intensity of the local area and provides texture information for subsequent perturbation generation.

[0075] The binary image y of the target area contour c Processing is performed to obtain the non-contour target area y t , the expression is y t =(1-y c )⊙y, where y is the segmentation mask image corresponding to the standard image.

[0076] The texture generation module is equipped with a generator based on the input standard image and texture feature map x t Generate texture perturbation δ t , the perturbation generation formula is as follows:

[0077]

[0078] Among them, x t is the texture feature map, y t is the non-contour target area, x represents the input standard image, F t (·) represents the texture generator, Indicates that the limit does not exceed ∈·x t ⊙y t .

[0079] Specifically, the input standard image is input into the texture generator to obtain the initial perturbation, and then the initial perturbation is clipped with a limited amplitude based on the Hadamard product calculation result of the texture feature map and the non-contour target area, and finally the adaptive texture perturbation δ is obtained. t .

[0080] Through the above method, the present invention generates stronger disturbances in high texture intensity areas and weaker disturbances in low texture areas, which can ensure the natural fusion of disturbances and enhance the concealment of the generated texture disturbances.

[0081] Step S4: establishing a loss function for evaluating the contour perturbation and texture perturbation, optimizing the contour perturbation generator and the texture generator based on the loss function; and obtaining the optimal contour perturbation and the optimal texture perturbation based on the optimized contour perturbation generator and the texture generator.

[0082] To ensure that the perturbation is visually invisible and improve the perturbation effect on the segmentation model, this paper proposes an alternating optimization strategy of contour perturbation and texture perturbation. The specific process is as follows:

[0083] The computer first generates contour perturbations, then texture perturbations. After each generation, the computer evaluates the perturbations' concealment and interference effectiveness, and based on the results, it optimizes the two perturbations alternately, gradually enhancing the protection effect.

[0084] In some embodiments, the evaluation and optimization method of contour perturbation and texture perturbation may include: establishing a cross-entropy loss function for a contour perturbation generator and a texture generator, and optimizing both perturbations by performing error back propagation on the cross-entropy loss function.

[0085] After multiple rounds of alternating optimization, the optimal contour perturbation and optimal texture perturbation are obtained based on the optimized contour perturbation generator and texture generator.

[0086] Step S5: Fusing the standard image, the optimal contour perturbation, and the optimal texture perturbation to obtain a protected medical image.

[0087] The input standard image, optimal contour perturbation and optimal texture perturbation are fused to generate a protected medical image. The fusion operation is completed in the image processing unit through weighted averaging to ensure that the perturbation is naturally integrated with the original image. The final protected image x p Expressed as:

[0088] x p =Clip [0,1] [δ c +δ t +x]

[0089] Among them, Clo [0,1] Indicates that the amplitude is limited to the interval [0,1].

[0090] After optimization is completed, the computer stores the generated perturbation image in a standard medical image format for easy database storage and subsequent forensic verification.

[0091] In summary, the present invention provides an effective medical image data protection solution through contour perturbation generation, texture perturbation generation and alternating optimization strategy, ensuring efficient data protection without significantly changing the image visual quality.

[0092] Although the specific embodiments of the present invention depict various actions or steps in a specific order, this should be understood as requiring such actions or steps to be performed in the specific order shown or in a sequential order, or requiring that all illustrated actions or steps should be performed to obtain the desired results. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although a number of specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of this disclosure. Certain features described in the context of separate embodiments can also be implemented in combination in a single implementation. Conversely, the various features described in the context of a single implementation can also be implemented in multiple implementations individually or in any suitable sub-combination. The above is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or replacements that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered within the scope of protection of the present invention.

[0093] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. A method for protecting image segmentation datasets based on contour and texture-aware perturbations, characterized in that: The following steps are involved: Step S1, preprocessing the medical images in the samples in the medical image segmentation dataset to obtain standard images; The samples in the medical image segmentation dataset include medical images and segmentation mask images; Step S2: using a contour perturbation generator to perform multi-scale convolution and channel attention mechanism processing on the standard image to complete encoding, and performing cross-layer feature aggregation to complete decoding; Cropping the decoding result based on the segmentation mask image corresponding to the standard image to obtain a contour disturbance; Step S3: using a texture generator to process the standard image to obtain an initial disturbance; Acquire a texture feature map of the standard image, and clip the initial disturbance based on the texture feature map to obtain a texture disturbance; Step S4: establishing a loss function for evaluating the contour perturbation and texture perturbation, optimizing the contour perturbation generator and the texture generator based on the loss function; and obtaining the optimal contour perturbation and the optimal texture perturbation based on the optimized contour perturbation generator and the texture generator; Step S5: fusing the standard image, the optimal contour perturbation, and the optimal texture perturbation to obtain a protected medical image; The step S2 specifically includes: Step S2-1: using the standard image as an input feature map, performing processing by the dynamic feature enhancement module twice in sequence, and using the dynamic enhanced feature maps output by the first and second dynamic feature enhancement modules as the shallow feature map and the deep feature map, respectively; Step S2-2: performing a convolution operation on the shallow feature map, performing an upsampling operation on the deep feature map, and concatenating the results of the convolution operation and the upsampling operation to obtain a cross-layer aggregated feature map; Step S2-3: obtaining a binary image of the target area contour from the segmentation mask image corresponding to the standard image, and cropping the cross-layer aggregated feature map based on the binary image of the target area contour to obtain contour disturbance.

2. The image segmentation dataset protection method based on contour and texture perception perturbation according to claim 1 is characterized in that: In step S2-1, the processing expression of the dynamic feature enhancement module is: in, To dynamically enhance the feature map, is the input feature map, represents global pooling, and are channel weights and spatial attention weights respectively, is the Sigmoid activation function, Represents a convolution operation.

3. The image segmentation dataset protection method based on contour and texture perception perturbation according to claim 2 is characterized in that: In step S2-2, the calculation expression of the cross-layer aggregated feature map is: in, represents the cross-layer aggregated feature map, and They are shallow feature maps and deep feature maps respectively. It is a 1×1 convolutional layer; represents the upsampling operation, Represents a splicing operation.

4. The image segmentation dataset protection method based on contour and texture perception perturbation according to claim 3 is characterized in that: In step S2-3, the calculation expression of the contour disturbance is: in, represents the contour perturbation, Indicates that the absolute value of the limit range does not exceed , represents the Hadamard product operation, A binary image representing the outline of the target area.

5. The image segmentation dataset protection method based on contour and texture perception perturbation according to claim 4 is characterized in that: The step S3 specifically includes: Step S3-1, extracting a texture feature map from the standard image using a local binary pattern algorithm; Step S3-2, obtaining a non-contour target area based on a binary image of the target area contour using the segmentation mask image corresponding to the standard image; Step S3-3: Process the standard image using a texture generator to obtain an initial disturbance, and crop the initial disturbance based on the texture feature map and the non-contour target area to obtain a texture disturbance.

6. The image segmentation dataset protection method based on contour and texture perception perturbation according to claim 5, characterized in that: In step S3-2, the non-contour target area The calculation expression is: in, is the segmentation mask image corresponding to the standard image; In step S3-3, the calculation expression of the texture disturbance is: in, is the texture perturbation, is the texture feature map, is the non-contour target area, represents the input standard image, represents a texture generator, Indicates that the absolute value of the limit range does not exceed .

7. The image segmentation dataset protection method based on contour and texture perception perturbation according to claim 6 is characterized in that: The step S5 specifically includes: The input standard image, optimal contour perturbation, and optimal texture perturbation are weighted averaged and cropped to generate a protected medical image. The expression is: in, Represents a protected medical image, Indicates that the limit range is interval.

Images