A self-supervised liver lesion detection method based on frequency-aware image restoration

Abstract

This invention relates to a self-supervised liver lesion detection method based on frequency-aware image restoration, belonging to the field of image processing technology. First, this invention designs a self-supervised task based on synthetic anomalies to synthesize a wider range of suitable pseudo-anomaly images, thereby alleviating the problem of insufficient anomaly data during algorithm model training. Second, to suppress the sensitivity of the reconstruction network to synthetic liver anomalies, a module for extracting high-frequency information from the image is designed. By recovering the image from its high-frequency components, the unfavorable generalization of anomalies by the reconstruction network is mitigated. Finally, a weight decay training strategy is employed to train the segmentation sub-network, reducing the trivial solution problem in the early stages of training and achieving the detection of local, subtle lesions. Extensive experiments on real-world public datasets demonstrate that the method of this invention achieves leading performance in liver lesion detection.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a self-supervised liver lesion detection method based on frequency-aware image restoration. Background Technology

[0002] Clinically, computed tomography (CT) is a crucial tool for assessing the shape, texture, and local lesions of the liver. In clinical medicine, doctors must manually annotate liver lesions and shapes, a laborious and time-consuming process. Therefore, automated liver lesion detection using computer-aided diagnostic systems on CT images can help reduce doctors' workload and improve diagnostic efficiency. Supervised deep learning methods have been widely used for liver lesion detection, with CNN models achieving convincingly high success rates; however, they are limited by large datasets and computational resources. In most cases, liver datasets have a limited number of images, low resolution, and a much smaller number of lesion samples compared to normal samples. Therefore, unsupervised one-class classification (OCC) methods have achieved a breakthrough due to their label-free nature and ability to train only on normal samples.

[0003] These methods typically train a single classifier that learns from normal samples, and then, during the inference phase, determine whether a sample is a lesion or a normal sample based on its deviation from a normal distribution (e.g., the distance between the sample and a class center) to achieve classification. Traditional single classification methods usually include encoder-decoder networks and discriminative models (such as one type of support vector machine OCSVM and support vector data description SVDD). Self-supervised single classification learns better feature representations by setting an auxiliary agent task, but because the training set only contains normal samples, the model can only learn from the features of the normal class, thus easily leading to overfitting problems. Another type of single classification method based on pre-trained models usually utilizes large-scale datasets (such as ImageNet, COCO, etc.) to provide fixed features to fit the single classifier. However, the redundant feature information brought by pre-trained models is not suitable for semantically simple medical images. Currently, there are corresponding single classification methods explored in medical fields such as brain MR images, malignant polyp images, and lung X-rays, but methods for liver lesion detection remain to be explored.

[0004] Therefore, the challenges of liver lesion detection are: (1) In clinical medicine, there are few liver datasets available for training, and overfitting is likely to occur when only specific classes are used for training; (2) There is a large difference in the number of lesion images and normal images in the dataset; (3) Using a single-classification method with fine-tuned pre-trained features for liver lesion detection will result in redundant features in the pre-trained network, which may cause the model to fail to learn effective classification features; (4) Currently, considering the small size and low resolution of liver lesions (such as...), the detection of liver lesions is still a challenge. Figure 1 As shown, the technology for detecting liver lesions has not been thoroughly studied, and a single classifier is needed that has the sensitivity to identify both subtle differences and significant features. Summary of the Invention

[0005] The purpose of this invention is to overcome the problem that existing liver lesion region detection algorithms are not effective in detecting subtle lesions, and to provide a self-supervised liver lesion detection method based on frequency-aware image restoration.

[0006] To achieve the above objectives, the technical solution of the present invention is: a self-supervised method for detecting liver lesions based on frequency-aware image restoration, comprising:

[0007] Design an anomaly data synthesis module based on a self-supervised task of synthetic anomalies to synthesize pseudo-anomaly images;

[0008] A module for extracting high-frequency information from images is designed to mitigate the adverse generalization of anomalies by the reconstruction subnetwork by recovering the image from its high-frequency components.

[0009] We designed a weight decay segmentation subnetwork and used a weight decay training strategy to train the segmentation subnetwork to achieve the detection of local subtle lesions.

[0010] In one embodiment of the present invention, the specific process of the abnormal data synthesis module synthesizing the pseudo-abnormal image is as follows:

[0011] Noise mask image generation: First, a noise image P is randomly generated using a 2D Perlin noise generator to capture various anomalous shapes. Then, the noise image is binarized using a random threshold to obtain the anomalous mask image M. P Before synthesizing the pseudo-anomaly image, it is necessary to locate the liver region, that is, the area with a pixel value greater than 0 is the location of the liver. Therefore, focus on the corresponding region and return the liver mask image M. I Finally, the anomaly mask image M P Liver mask image M I The final mask image M is generated by element-wise multiplication;

[0012] Pseudo-abnormal image synthesis: First, the obtained mask image M is multiplied element-wise with the external image A and the original image to simulate abnormal regions; a transparency factor β is introduced to make the simulated abnormal regions more natural. Next, the original image is multiplied pixel-by-pixel with the inverted (1-M) of the mask image to obtain the normal regions of the image. Finally, the normal regions and abnormal regions are added together to obtain the synthesized pseudo-abnormal image. The overall process is defined as follows:

[0013] I a =(1-M)⊙I+(1-β)(M⊙I)+β(M⊙A)

[0014] Where ⊙ denotes element-wise multiplication, and β is a random, opaque parameter.

[0015] In one embodiment of the present invention, the external image in the pseudo-anomaly image synthesis is obtained from the external dataset BrainTumor.

[0016] In one embodiment of the present invention, the module for extracting high-frequency information from the image is specifically implemented as follows:

[0017] First, the synthesized pseudo-anomaly image I a The specific definition of the two-dimensional discrete Fourier transform (DFT) is as follows:

[0018]

[0019] Where m and n are the spatial dimensions of the image, F(μ, v) is the image function in the frequency domain, where μ and v correspond to its coordinates; and f(x, y) is the image function in the spatial domain, where x and y correspond to its coordinates.

[0020] Then, the DFT-processed image is subjected to a second-order Butterworth high-pass filter (BHPF) to obtain a high-pass image in the frequency domain. The second-order Butterworth high-pass filter is expressed as:

[0021]

[0022] Where D(μ, v) is the Euclidean distance from any point μ, v to the origin of the frequency plane, and D0 is the distance corresponding to H B (μ, v) = 1 and H B The cutoff frequency of the transition point between (μ, ν) = 0;

[0023] Finally, the final high-frequency component image I' is obtained by inverse Fourier transform (IFT). h :

[0024]

[0025] Where, F(μ, v) h This represents the filtering process, i.e., F(μ, v)·HB (μ, v).

[0026] In one embodiment of the present invention, the input of the reconstructed subnetwork is a high-frequency image I' generated by the abnormal data synthesis module and the high-frequency information extraction module. h The model is trained so that the reconstructed subnetwork can learn features that help distinguish between normal and abnormal by using agent tasks.

[0027] In one embodiment of the present invention, the reconstruction subnetwork adopts an encoder and decoder architecture, which implicitly eliminates high-frequency image I'. h The synthesis anomalies are reduced to make the image more closely resemble a real, normal image; the loss function of the reconstruction subnetwork is defined as follows:

[0028] L rec (I,I r )=λL SSIM (I,I r )+L2(I,I r )

[0029] Where λ is the hyperparameter of the balanced loss function, I represents the original image, I r Image L represents the image recovered after reconstruction by the network. SSIM (I,I r The structural similarity loss based on patches is defined as follows:

[0030]

[0031] Where H and W are the height and width of image I, respectively, and N... p It is the number of pixels in image I, SSIM(I,I) r ) (i,j) Let I and I be centered at image coordinates (i,j). r L2(I,I) r ) is defined as

[0032]

[0033] In one embodiment of the present invention, the weight decay segmentation subnetwork is specifically implemented as follows:

[0034] First, the reconstructed image I output by the reconstruction subnetwork r The features of the original image I are merged and input into a weight decay segmentation subnetwork to predict the mask image, i.e., to identify and locate anomalies; the feature map corresponding to each layer is defined as follows:

[0035]

[0036] Where α∈[0,1] is a weighting factor; Ei and D i This represents the i-th layer feature map of the encoder and decoder; Indicates the addition of features; Represents scalar multiplication;

[0037] Secondly, regarding D i The feature map is copied and compared with the obtained M. i By splicing, we obtain D' i The specific operation is defined as follows:

[0038]

[0039] in, This represents the copied features of the corresponding i-th layer feature map of the decoder, and Concat represents the concatenation operation;

[0040] Finally, use focus loss L a The L1 loss is used as the training objective for the weight decay segmentation subnetwork; among them, the focus loss can control the general segmentation region, and the L1 loss can refine the abnormal edges more finely, as defined below:

[0041] L1 = ||MM O ||1

[0042] L a =-α t (1-ρ t ) γ log(ρ t )

[0043] Where M represents the real label, that is, the anomalous label generated during anomalous synthesis. O Represents the mask label predicted by the network; in the focus loss, when the true pixel value in M ​​is 1, ρ t The probability ρ is equal to the predicted probability of the pixel category; conversely, when the true pixel value in M ​​is 0, ρ is equal to the predicted probability of the pixel category. t Equal to 1-ρ, α t γ and γ are two hyperparameters; the final total segmentation loss is defined as follows:

[0044] L seg =L1+L a .

[0045] In one embodiment of the present invention, in the weight decay segmentation subnetwork, during the training phase, the weight factor α is initialized to 1 and then gradually decays to 0 to reduce the impact of early skip links.

[0046] In one embodiment of the present invention, a testing phase is also included, which is specifically implemented as follows:

[0047] After extracting high-frequency component information from the test image, it is input into the reconstruction subnetwork to obtain a reconstructed normal image. Finally, the weight decay segmentation subnetwork predicts a mask to locate abnormal regions. First, the predicted mask label M is... O Smoothing is performed by aggregating local anomaly response information using an average filtering convolutional layer. The final image-level anomaly score is calculated by taking the maximum value from the smoothed anomaly score map, as defined below:

[0048] φ=max(M O *f sf×sf )

[0049] Among them, f sf×sf This represents an average filter of size sf×sf, and * represents the convolution operator.

[0050] The present invention also provides a computer-readable storage medium having stored thereon computer program instructions that can be executed by a processor, wherein when the processor executes the computer program instructions, it can implement the steps of the method described above.

[0051] Compared to existing technologies, this invention offers the following advantages: It proposes a self-supervised liver lesion detection method based on frequency-aware image restoration. First, this invention designs a self-supervised task based on synthetic anomalies to synthesize a wider range of suitable pseudo-anomaly images, thus alleviating the problem of insufficient anomaly data during algorithm model training. Second, to suppress the sensitivity of the reconstruction network to synthetic liver anomalies, a module for extracting high-frequency information from the image is designed. By recovering the image from its high-frequency components, the unfavorable generalization of anomalies by the reconstruction network is mitigated. Finally, a weight decay training strategy is employed to train the segmentation sub-network, reducing the trivial solution problem in the early stages of training and enabling the detection of local, subtle lesions. Extensive experiments on real-world public datasets demonstrate that the method of this invention achieves leading performance in liver lesion detection. Attached Figure Description

[0052] Figure 1 These are some examples from the dataset.

[0053] Figure 2 This is a structural diagram of the method of the present invention.

[0054] Figure 3 This refers to the anomalous image and its mask.

[0055] Figure 4 This is a diagram of the anomaly synthesis method.

[0056] Figure 5 This is a diagram of the sub-network architecture for weight decay.

[0057] Figure 6 This is a diagram of the testing phase.

[0058] Figure 7 The graph shows the area under the ROC curve of the algorithm on the LiTS and BTCV datasets.

[0059] Figure 8 This is a diagram illustrating a reasoning example of the present invention. Detailed Implementation

[0060] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0061] This invention proposes a self-supervised method for detecting liver lesions based on frequency-aware image restoration. First, a self-supervised task based on synthetic anomalies is designed to synthesize a wider range of suitable pseudo-anomaly images, thus alleviating the problem of insufficient anomaly data during algorithm model training. Second, to suppress the sensitivity of the reconstruction network to synthetic liver anomalies, a module for extracting high-frequency information from the image is designed. By recovering the image from its high-frequency components, the unfavorable generalization of anomalies by the reconstruction network is mitigated. Finally, a weight decay training strategy is employed to train the segmentation sub-network, reducing the fragmented solution problem in the early stages of training and enabling the detection of local, subtle lesions.

[0062] The following is a detailed implementation process of the present invention.

[0063] 1. Method Structure

[0064] The method proposed in this invention is as follows: Figure 2 As shown, the model consists of three parts: a module for synthesizing anomalies and extracting high-frequency images, a reconstruction subnetwork, and a weight-decaying segmentation subnetwork. First, a pseudo-abnormal liver image is obtained by enhancing a normal image to address the difficulty in obtaining liver lesion data. Specifically, the synthesis method involves two steps: locating the liver region and synthesizing the anomaly. Next, to suppress the sensitivity of the reconstruction network to anomaly information, a module for extracting high-frequency information is introduced before feeding the synthesized image into the reconstruction network. This involves using a high-pass filter to destroy local image information and reconstructing (restoring) the image from a high-frequency perspective. The reconstruction network is trained to restore the destroyed image to its original normal form. Finally, the reconstructed image and the original image are merged and input into the weight-decaying segmentation network for anomaly identification and localization, increasing the training difficulty of the model and thus mitigating overfitting.

[0065] 1.1 Abnormal Data Synthesis Module

[0066] This invention designs an effective data augmentation proxy task to simulate liver tumor abnormalities. Specifically, by synthesizing an anomaly proxy task, the reconstruction subnetwork learns features that help distinguish between abnormal and normal features. This is combined with a segmentation subnetwork to identify and locate anomalies, achieving self-supervised learning training. As shown in Figure 4, the proposed anomaly synthesis method mainly consists of two steps, as follows:

[0067] Noise mask generation: First, a noise image P is randomly generated using a 2D Perlin noise generator to capture various anomalous shapes. Then, the noise image is binarized using a random threshold to obtain the anomaly mask M. P Considering the direct use of M P Creating anomalies in the background region is impractical, as liver tumors do not exist in the background. Furthermore, simple manipulation increases the distributional discrepancy between synthetic and real anomaly data, hindering the model from learning effective discriminative information. Therefore, the liver region is first located before synthesizing anomaly data. Specifically, considering the unique characteristics of the liver dataset—that is, regions with pixel values ​​greater than 0 represent the liver—this region is targeted, and a liver mask image M is returned. I Finally, the two mask images are multiplied element-wise to generate the final mask image M.

[0068] Pseudo-abnormal image synthesis: Because pixels in the liver tumor area are usually darker (e.g., Figure 3 (As shown) This stage uses the external dataset BrainTumor. First, the obtained mask image M is multiplied element-wise with the external image A and the original image I to simulate anomalous regions. Here, a transparency factor β is introduced to make the simulated anomalous regions more natural. Next, the normal regions of the image are obtained by multiplying the original image I and the mask image pixel-by-pixel by the inverted (1-M) result. Finally, the normal and anomalous regions are added together to obtain the synthesized pseudo-anomalous image. The specific definitions of the above process are as follows:

[0069] I a =(1-M)⊙I+(1-β)(M⊙I)+β(M⊙A)

[0070] Where ⊙ denotes element-wise multiplication, and β is a random, opaque parameter.

[0071] 1.2 Image High-Frequency Information Extraction Module

[0072] Secondly, mitigating the sensitivity of the reconstruction network to anomalous information is a major challenge. To address this challenge, a module for extracting high-frequency image information is introduced to reduce the impact of anomalous regions on the reconstruction network while preserving image information. First, the method uses the synthesized anomalous image I... aThe specific definition of the two-dimensional Discrete Fourier Transform (DFT) is as follows:

[0073]

[0074] Where m and n are the spatial dimensions of the image. F(μ, v) is the image function in the frequency domain, where μ and v correspond to their coordinates. f(x, y) is the image function in the spatial domain, where x and y correspond to their coordinates. Then, the DFT-processed image is subjected to a second-order Butterworth High Pass Filter (BHPF) to obtain a high-pass image in the frequency domain. The second-order Butterworth High Pass Filter can be expressed as:

[0075]

[0076] Where D(μ, ν) is the Euclidean distance from any point μ, ν to the origin of the frequency plane, and D0 is the distance corresponding to H B (μ, v) = 1 and H B The cutoff frequency is the transition point between (μ, v) = 0. Finally, the final high-frequency component image I' is obtained by inverse Fourier transform (IFT). h The extraction results are as follows: Figure 2 As shown. Therefore, the above steps are specifically defined as follows:

[0077]

[0078] Where, F(μ, ν) h This represents the filtering process, i.e., F(μ, v)·H B (μ, v).

[0079] 1.3 Reconstructing Subnetworks

[0080] The high-frequency image I' generated by the abnormal data synthesis module and the high-frequency information extraction module. h The input is used to train the model in the reconstruction subnetwork. This allows the reconstruction subnetwork to learn features that help distinguish between normal and abnormal data using a proxy task. The reconstruction subnetwork employs an encoder-decoder architecture, and its function is to implicitly remove image I'. h The network synthesizes anomalies to more closely resemble real, normal images. It follows the pixel-level loss and patch-based structural similarity loss methods commonly used in reconstruction tasks as its training objectives. Therefore, the specific loss function is defined as follows:

[0081] L rec (I,I r )=λL SSIM (I,Ir )+L2(I,I r )

[0082] Where λ is the hyperparameter of the balanced loss function, I represents the original image, I r This represents the image after the network has been reconstructed.

[0083] 1.4 Weight decay segmentation subnetwork

[0084] Recently, the U-net architecture has proven highly effective in recovering fine-grained details of the target object, particularly in segmentation tasks, by introducing skip links between the encoder and decoder to bridge the semantic gap between them. However, skip links can lead the model to learn identity mappings by bypassing low-level features through "shortcuts," which can negatively impact its performance in liver tumor detection. To address this challenge, this paper introduces a weight-decay-based segmentation subnetwork after the reconstructed subnetwork. During network training, the skip links in U-net are gradually weakened to reduce their impact on early-stage skip links, thus improving the identification and localization of abnormal images. The specific structure is as follows: Figure 5 As shown.

[0085] First, the reconstructed image I output by the reconstruction subnetwork r The features of the original image I are merged and input into a weight decay segmentation sub-network to predict the mask image, i.e., to identify and locate anomalies. Unlike the original U-net model, the features corresponding to different levels of the encoder in this network are not directly added together and connected to the decoder. Specifically, the method first defines the feature map corresponding to each level as follows:

[0086]

[0087] Where α∈[0,1] is a weighting factor; E i and D i This represents the i-th layer feature map of the encoder and decoder; Indicates the addition of features; This indicates scalar multiplication.

[0088] Secondly, regarding Di 特 The image is copied and compared with the obtained M. i By splicing, we obtain D' i The specific operation is defined as follows:

[0089]

[0090] in, This represents the copied features of the corresponding i-th layer feature map of the decoder, and Concat represents the concatenation operation.

[0091] During the training phase of this model, the weight factor α is initialized to 1 and then gradually decays to 0 to reduce the impact of early skip links. When α is 1, the network has the same structure as the traditional U-Net paradigm. At this point, due to D... i Because of the existence of α, gradient calculation cannot be propagated from upper layers to lower layers. This means that in the first stage of training, the network focuses more on horizontal optimization. As α decreases, the network optimization focus gradually shifts from the horizontal direction to the vertical direction. This means that in the early stages of training, features from lower layers can gradually be aggregated into features from higher layers. It is worth noting that when α is 0, M... i Actually, it's D. i A copy of the network would degenerate into an encoder-decoder network without skip links.

[0092] Finally, focus loss L is used in this task. a The L1 loss is used as the training objective for this network. The focus loss controls the general segmentation of the region, while the L1 loss refines the edges of anomalies. The specific definitions are as follows:

[0093] L1 = ||MM O ||1

[0094] L a =-α t (1-ρ t ) γ log(ρ t )

[0095] Where M represents the real label, that is, the anomalous label generated during anomalous synthesis. O This represents the mask label predicted by the network. In the focus loss, when the true pixel value in M ​​is 1, ρ t The probability ρ is equal to the predicted probability of the pixel category; conversely, when the true pixel value in M ​​is 0, ρ is equal to the predicted probability of the pixel category. t It equals 1 - ρ. α t γ and γ are two hyperparameters. The final total segmentation loss is defined as follows:

[0096] L seg =L1+L a

[0097] 1.5 Testing Phase

[0098] like Figure 6 As shown, unlike the training phase, data augmentation methods are no longer used to generate anomalous images during testing. Instead, the test image is directly used as the model's input image. Specifically, high-frequency component information is extracted from the test image and input into the reconstruction network to obtain a reconstructed normal image. Finally, a weight decay segmentation subnetwork predicts a mask to locate anomalous regions. In the anomaly score calculation, the output predicted mask image M...O This method can be directly used for pixel-level anomaly scoring and can also determine whether an image is abnormal. The method first smooths the predicted mask image and then aggregates local anomaly response information using an average filtering convolutional layer. The final image-level anomaly score is calculated by taking the maximum value from the smoothed anomaly score map, as defined below:

[0099] φ=max(M O *f sf×sf )

[0100] Among them, f sf×sf This represents an average filter of size sf×sf, and * represents the convolution operator.

[0101] 2. Implementation Examples

[0102] We tested our proposed method using real-world liver datasets consisting of the LiTs dataset (a dataset of anomalous liver tumor segmentation) and the BTCV dataset (a dataset of anomalous abdominal CT multi-organ segmentation). LiTs, from the MICCAI 2017 challenge, contains abdominal CT scans from 131 patients. BTCV, from the MICCAI 2015 challenge, is a benchmark dataset for abdominal CT multi-organ segmentation. Specifically, for both datasets, the Hounsfield units (HUs) of the 3D scans were converted to grayscale images based on abdominal windows. The scans were then sliced ​​into 2D axial sections, and the regions of interest (ROIs) of the liver were extracted based on provided organ annotations. Finally, the slide intensity of the sections was normalized using histogram equalization. The lesion sections extracted from the LiTs dataset are shown below. Figure 3 As shown, LiTs are used to build the test and validation datasets, while BTCV is used to build the normal training dataset.

[0103] To evaluate the performance of the proposed method, we calculate AUROC (Area Under the Receiver Operating Characteristic curve, i.e., the area under the ROC curve). Image AUROC is used to determine whether an image is abnormal, while Pixel AUROC is used to evaluate anomaly localization performance. AUROC is a commonly used metric for anomaly detection and classification, defining the false positive rate (FPR) on the X-axis and the true positive rate (TPR) on the Y-axis, as shown in the following formula:

[0104]

[0105] In this system, TP stands for True Positive, meaning a positive sample predicted as positive by the model; FP stands for False Positive, meaning a negative sample predicted as positive by the model; TN stands for True Negative, meaning a positive sample predicted as positive by the model; and FN stands for False Negative, meaning a positive sample predicted as negative by the model. The closer the AUROC value is to 1, the better the classifier can distinguish between positive and negative samples.

[0106] The experimental environment for this invention was as follows: the method was implemented using Python 3.8 and PyTorch 1.9.0, and all experiments were conducted on an Ubuntu 18.04LTS operating system with an NVIDIA Tesla V100 GPU. Except for a very few networks that converged early on and therefore had separate training batches, the other comparative methods all had a training batch size of 230 epochs. The learning rate of the proposed method was set to 0.0001. During training, the BrainTumor dataset was used as the anomaly source dataset.

[0107] 2.1 Quantitative comparison

[0108] To quantitatively compare the detection and localization accuracy of several state-of-the-art unsupervised and self-supervised learning algorithms for tumors, we conducted experiments on the combined BTCV and LiTS datasets. RD++, RD, SQUID, DRAEM, and WDMT are unsupervised learning methods, while CutPaste, FPI, PII, NSA, and FARI are self-supervised learning methods. AUROC was used to quantitatively evaluate the results, and Table 1 shows the detection results of our invention on the LiTS and BTCV datasets. The best metric value in each column is bolded. It is clear that our proposed algorithm achieves optimal detection and localization performance on this public dataset. Figure 7 The figure shows the area under the ROC curve of our invented algorithm and other advanced algorithms, and the results quantitatively demonstrate the superiority of the invented method. Figure 8 It also shows an example diagram of our inventive method reasoning.

[0109] Table 1. Quantitative comparison between the present invention and state-of-the-art algorithms.

[0110] method Image AUROC Pixel AUROC CutPaste 54.85 97.75 WDMT 72.89 87.07 FPI 66.12 99.00 PII 74.21 98.68 NSA 77.74 98.23 SQUID 67.64 NA DRAEM 72.44 94.61 FAIR 73.18 99.10 RD++ 64.39 97.50 RD 66.40 97.80 Ours 89.60 99.10

[0111] The present invention also provides a computer-readable storage medium having stored thereon computer program instructions that can be executed by a processor, wherein when the processor executes the computer program instructions, it can implement the steps of the method described above.

[0112] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0113] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0114] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0115] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0116] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A self-supervised method for detecting liver lesions based on frequency-aware image restoration, characterized in that, include: Design an anomaly data synthesis module based on a self-supervised task of synthetic anomalies to synthesize pseudo-anomaly images; A module for extracting high-frequency information from images is designed to mitigate the adverse generalization of anomalies by the reconstruction subnetwork by recovering the image from its high-frequency components. We designed a weight decay segmentation subnetwork and used a weight decay training strategy to train the segmentation subnetwork to achieve the detection of local subtle lesions. The specific process of the abnormal data synthesis module synthesizing pseudo-abnormal images is as follows: Noise mask image generation: First, a noise image P is randomly generated using a 2D Perlin noise generator to capture various anomalous shapes. Then, the noise image is binarized using a random threshold to obtain the anomalous mask image M. P Before synthesizing the pseudo-anomaly image, it is necessary to locate the liver region, that is, the area with a pixel value greater than 0 is the location of the liver. Therefore, focus on the corresponding region and return the liver mask image M. I Finally, the anomaly mask image M P Liver mask image M I The final mask image M is generated by element-wise multiplication; Pseudo-abnormal image synthesis: First, the obtained mask image M is multiplied element-wise by the external image A and the original image I to simulate abnormal regions; a transparency factor β is introduced to make the simulated abnormal regions more natural. Next, the normal regions of the image are obtained by multiplying the original image I and the mask image pixel-by-pixel by taking the inversion of the original image I. Finally, the normal regions and the abnormal regions are added together to obtain the synthesized pseudo-abnormal image. The overall process is defined as follows: I a =(1-M)⊙I+(1-β)(M⊙I)+β(M⊙A) Where ⊙ denotes element-wise multiplication, and β is a random opaque parameter; The module for extracting high-frequency information from the image is implemented as follows: First, the synthesized pseudo-anomaly image I a The specific definition of the two-dimensional discrete Fourier transform (DFT) is as follows: Where m and n are the spatial dimensions of the image, F(μ, v) is the image function in the frequency domain, where μ and v correspond to its coordinates; and f(x, y) is the image function in the spatial domain, where x and y correspond to its coordinates. Then, the DFT-processed image is subjected to a second-order Butterworth high-pass filter (BHPF) to obtain a high-pass image in the frequency domain. The second-order Butterworth high-pass filter is expressed as: Where D(μ, v) is the Euclidean distance from any point (μ, v) to the origin of the frequency plane, and D0 is the distance corresponding to H B (μ, v) = 1 and H B The cutoff frequency of the transition point between (μ, v) = 0; Finally, the final high-frequency component image I' is obtained by inverse Fourier transform (IFT). h : Where, F(μ, v) h This represents the filtering process, i.e., F(μ, v)·H B (μ, v).

2. The self-supervised liver lesion detection method based on frequency-aware image restoration according to claim 1, characterized in that, In the synthesis of pseudo-anomaly images, the external images are obtained from the external dataset BrainTumor.

3. The self-supervised liver lesion detection method based on frequency-aware image restoration according to claim 1, characterized in that, The input to the reconstructed subnetwork is a high-frequency image I' generated by the abnormal data synthesis module and the high-frequency information extraction module. h The model is trained so that the reconstructed subnetwork can learn features that help distinguish between normal and abnormal by using agent tasks.

4. The self-supervised liver lesion detection method based on frequency-aware image restoration according to claim 1, characterized in that, The reconstruction subnetwork employs an encoder and decoder architecture, and its function is to implicitly eliminate high-frequency image I'. h The synthesis anomaly; the loss function for reconstructing the subnetwork is defined as follows: L rec (I,I r )=λL SSIM (I,I r )+L2(I,I r ) Where λ is the hyperparameter of the balanced loss function, I represents the original image, I r L represents the image recovered after the network reconstruction. SSIM (I,I r The structural similarity loss based on patches is defined as follows: Where H and W are the height and width of the original image I, respectively, and N... p It is the number of pixels in the original image I, SSIM(I,I) r ) (i,j) Let I and I be centered at image coordinates (i,j). r L2(I,I) r ) is defined as 5. The self-supervised liver lesion detection method based on frequency-aware image restoration according to claim 1, characterized in that, The specific implementation of the weight decay segmentation subnetwork is as follows: First, the reconstructed image I output by the reconstruction subnetwork r The features of the original image I are merged and input into a weight decay segmentation subnetwork to predict the mask image, i.e., to identify and locate anomalies; the feature map corresponding to each layer is defined as follows: Where α∈[0,1] is a weighting factor; E i and D i This represents the i-th layer feature map of the encoder and decoder; Indicates the addition of features; Represents scalar multiplication; Secondly, regarding D i The feature map is copied and compared with the obtained M. i By splicing, we obtain D' i The specific operation is defined as follows: in, This represents the copied features of the corresponding i-th layer feature map of the decoder, and Concat represents the concatenation operation; Finally, use focus loss L a The L1 loss is used as the training objective for the weight decay segmentation subnetwork, and is specifically defined as follows: L1=||M-M O ||1 L a =-a t (1-p t ) γ log(ρ t ) Where M represents the real label, that is, the anomalous label generated during anomalous synthesis. O Represents the mask label predicted by the network; in the focus loss, when the true pixel value in M ​​is 1, ρ t The probability ρ is equal to the predicted probability of the pixel category; conversely, when the true pixel value in M ​​is 0, ρ is equal to the predicted probability of the pixel category. t Equal to 1-ρ, α t γ and γ are two hyperparameters; the final total segmentation loss is defined as follows: L seg =L1+L a 。 6. The self-supervised liver lesion detection method based on frequency-aware image restoration according to claim 5, characterized in that, In the weight decay segmentation subnetwork, during the training phase, the weight factor α is initialized to 1 and then gradually decays to 0 to reduce the impact of early skip links.

7. The self-supervised liver lesion detection method based on frequency-aware image restoration according to claim 1, characterized in that, It also includes a testing phase, which is implemented as follows: After extracting high-frequency component information from the test image, it is input into the reconstruction subnetwork to obtain a reconstructed normal image. Finally, the weight decay segmentation subnetwork predicts a mask to locate abnormal regions. First, the predicted mask label M is... O Smoothing is performed by aggregating local anomaly response information using an average filtering convolutional layer. The final image-level anomaly score is calculated by taking the maximum value from the smoothed anomaly score map, as defined below: φ=max(M O *f sf×sf ) Among them, f sf×sf This represents an average filter of size sf×sf, and * represents the convolution operator.

8. A computer-readable storage medium having stored thereon computer program instructions executable by a processor, wherein when the processor executes the computer program instructions, it is able to implement the steps of the method as described in any one of claims 1-7.

Images