Fourier-based semantic data augmentation method using uncertainty guide, and apparatus therefor

Abstract

A Fourier-based semantic data augmentation method using an uncertainty guide, and an apparatus therefor are disclosed. The Fourier-based semantic data augmentation method using an uncertainty guide comprises the steps of: applying an input image to an augmentation module so as to convert the input image into a frequency domain, and then generating a context-augmented image through amplitude and phase conversion; applying a Bezier transformation result on the basis of the input image and an input mask to the augmentation module so as to convert same into the frequency domain, and then generating a position-augmented image through amplitude and phase conversion; applying the context-augmented image and the position-augmented image to segmentation models so as to generate each of a context uncertainty map and a position uncertainty map; generating an uncertainty guide map by fusing the context uncertainty map and the position uncertainty map; and generating a mutual augmentation image by using the context-augmented image, the position-augmented image and the uncertainty guide map.

Description

Fourier-based semantic data augmentation method and device using uncertainty guide

[0001] The present invention relates to a Fourier-based semantic data augmentation method and device using an uncertainty guide to improve the generalization ability of a medical image segmentation model.

[0002]

[0003] Medical image segmentation (MIS) plays a crucial role in medical image analysis, accurately segmenting specific anatomical structures to assist clinicians in making diagnoses and establishing treatment plans.

[0004] Recent deep learning-based MIS approaches have shown excellent performance in various image formats such as MRI, CT, and X-ray, but they suffer from performance degradation due to domain shift caused by mismatch between the distributions of training data (source domain) and evaluation data (target domain).

[0005] These domain migration issues can arise due to differences in equipment manufacturers, imaging protocols, image formats, and detailed data characteristics held by different hospitals. More specifically, there are cases where a deep learning model is trained on data captured by MRI to perform image segmentation and then evaluated on data captured by CT (differences in image formats), cases where a deep learning model is trained on data acquired by imaging equipment A but of the same format (e.g., MRI) and then evaluated on data acquired by imaging equipment B (differences in equipment manufacturers and imaging protocols), and cases where a deep learning model is trained on data acquired by hospital A and then evaluated on data acquired by hospital B (differences in data by institution).

[0006] To solve this domain shift problem, existing methods such as unsupervised domain adaptation (UDA) and multi-source domain generalization (MDG) have been proposed. However, UDA requires access to target domain data (i.e., evaluation data), and MDG has clear technical limitations for deep learning model learning when the diversity of source domains from the training data is insufficient. Therefore, existing methods have difficulty being utilized in actual clinical environments.

[0007] In this regard, single-source domain generalization (SDG) technology is attracting attention as it can achieve excellent performance in various target domains without domain shifting problems, even though training data is provided from only one source domain.

[0008] Typically, SDG relies on data augmentation techniques as a means to expand the distribution of training data, enhancing the diversity of data by using image intensity or texture augmentation or arbitrary region-to-region transformation techniques (cutout, cutmix, mixup, etc.).

[0009] However, these methods are only data augmentation methods specialized for SDGs, and they may have difficulty improving performance in MIS because they are not augmentation techniques that consider the structural characteristics of organ tissues for image segmentation and the image intensity within the segmented tissue area.

[0010]

[0011] The present invention provides a Fourier-based semantic data augmentation method and device utilizing an uncertainty guide.

[0012] In addition, the present invention provides a Fourier-based semantic data augmentation method and apparatus utilizing an uncertainty guide that can simultaneously improve domain generalization ability and image segmentation ability by utilizing a guide for an uncertainty region affecting segmentation performance, together with a fast Fourier transform-based data augmentation technique that can consider both image intensity changes for increasing data diversity and structural features for image segmentation.

[0013] In addition, the present invention provides a Fourier-based semantic data augmentation method and device therefor, which utilize an uncertainty guide capable of encompassing various domain variations through a semantic augmentation method for each amplitude and phase component through a Fourier transform, as a general-purpose data augmentation technique that is not limited to a specific data format and medical image segmentation model.

[0014]

[0015] According to one aspect of the present invention, a Fourier-based semantic data augmentation method utilizing uncertainty guides is provided.

[0016] According to one embodiment of the present invention, a Fourier-based semantic data augmentation method utilizing an uncertainty guide may be provided, including: applying an input image to an augmentation module to transform the input image into a frequency domain and then generating a context augmented image through amplitude and phase transformation; applying a result of performing a Bezier transform based on the input image and an input mask to the augmentation module to transform the result into a frequency domain and then generating a location augmented image through amplitude and phase transformation; applying the context augmented image and the location augmented image to a segmentation model, respectively, to generate a context uncertainty map and a location uncertainty map, respectively; fusing the context uncertainty map and the location uncertainty map to generate an uncertainty guide map; and generating a mutually augmented image using the context augmented image, the location augmented image, and the uncertainty guide map.

[0017] The augmentation module may perform the steps of: generating an amplitude component map and a phase component map by transforming amplitude and phase components after transforming input data into a frequency domain, wherein the input data is the input image or the result of the Bezier transform; calculating an angular density distribution based on the amplitude component map, extracting a maximum angle and a minimum angle from the angular density distribution, respectively, and masking a maximum angle amplitude area and a minimum angle amplitude area to generate a maximum angle mask and a minimum angle mask; generating an amplitude-masked image by multiplying an amplitude component by the maximum angle mask and then applying an inverse fast Fourier transform together with the phase component, and generating an internal modulation image by exchanging the maximum angle mask and the minimum angle mask areas; generating a phase image using the phase component map and refining it by applying a bilateral filter; and self-mixing the phase image to which the bilateral filter is applied, the amplitude-masked image, and the internal modulation image to generate a Fourier-based context-augmented image or the position-augmented image, respectively.

[0018]

[0019] The above uncertainty guide map is generated using the following mathematical formula:

[0020]

[0021] Here, GaussianBlur represents a smoothing function that applies a Gaussian filter to the uncertainty map, represents the context uncertainty map, represents the position uncertainty map.

[0022] The above mutually augmented image can be generated by multiplying the uncertainty guide map and the inverse of the uncertainty guide map by the context augmented image and the location augmented image.

[0023] The above mutually enhanced image is generated using the following mathematical formula:

[0024]

[0025] Here, represents a context-augmented image, represents a location augmented image, represents the uncertainty guide map, represents the constraint probability.

[0026] The step of applying the mutually augmented image to the segmentation model to generate a predicted segmentation mask and training the same may be further included.

[0027]

[0028] According to another aspect of the present invention, a device is provided for performing a Fourier-based semantic data augmentation method utilizing an uncertainty guide.

[0029] According to one embodiment of the present invention, a computing device may be provided, including a first augmentation module that transforms an input image into a frequency domain, generates a context augmented image through amplitude and phase transformation, and then applies the transformed image to a segmentation model to generate a context uncertainty map; a second augmentation module that transforms a result of performing a Bezier transform based on the input image and an input mask into a frequency domain, and then generates a position augmented image through amplitude and phase transformation, and applies the position augmented image to the segmentation model to generate a position uncertainty map; and a mutual augmentation module that fuses the context uncertainty map and the position uncertainty map to generate an uncertainty guide map, and generates a mutually augmented image using the context augmented image, the position augmented image, and the uncertainty guide map.

[0030]

[0031] By providing a Fourier-based semantic data augmentation method and device utilizing an uncertainty guide according to one embodiment of the present invention, domain generalization ability and image segmentation ability can be improved simultaneously by utilizing a guide for an uncertainty region affecting segmentation performance together with a fast Fourier transform-based data augmentation technology that can consider both image intensity changes for increasing data diversity and structural features for image segmentation.

[0032] In addition, the present invention is a general-purpose data augmentation technique that is not limited to a specific data format and medical image segmentation model, and has the advantage of being able to encompass various domain variations through a semantic augmentation method for each amplitude and phase component through Fourier transform.

[0033]

[0034] FIG. 1 is a flowchart illustrating a Fourier-based semantic data augmentation method utilizing an uncertainty guide according to one embodiment of the present invention.

[0035] FIG. 2 is a flowchart illustrating an augmentation method based on a fast Fourier transform according to one embodiment of the present invention.

[0036] FIG. 3 is a drawing illustrating the overall framework of an augmentation module according to one embodiment of the present invention.

[0037] FIG. 4 is a diagram illustrating pseudo code for an augmentation method based on a fast Fourier transform according to one embodiment of the present invention.

[0038] FIG. 5 is a diagram illustrating a Bezier transformation according to one embodiment of the present invention.

[0039] FIG. 6 is a diagram illustrating an overall framework for a data augmentation method according to one embodiment of the present invention.

[0040] FIG. 7 is a diagram illustrating pseudo code for a Fourier-based semantic data augmentation method utilizing an uncertainty guide according to one embodiment of the present invention.

[0041] FIG. 8 is a block diagram schematically illustrating the internal configuration of a computing device that performs a Fourier-based semantic data augmentation method utilizing an uncertainty guide according to one embodiment of the present invention.

[0042]

[0043] As used herein, singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "consist of" or "include" should not be construed to necessarily include all components or steps described in the specification, and should be construed to mean that some of the components or steps may not be included, or that additional components or steps may be included. In addition, terms such as "part" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware or software, or by a combination of hardware and software.

[0044] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0045]

[0046] FIG. 1 is a flowchart illustrating a Fourier-based semantic data augmentation method using an uncertainty guide according to an embodiment of the present invention, FIG. 2 is a flowchart illustrating an augmentation method based on a fast Fourier transform according to an embodiment of the present invention, FIG. 3 is a diagram illustrating an overall framework of an augmentation module according to an embodiment of the present invention, FIG. 4 is a diagram illustrating pseudo code for an augmentation method based on a fast Fourier transform according to an embodiment of the present invention, FIG. 5 is a diagram illustrating a Bezier transform according to an embodiment of the present invention, FIG. 6 is a diagram illustrating an overall framework for a data augmentation method according to an embodiment of the present invention, and FIG. 7 is a diagram illustrating pseudo code for a Fourier-based semantic data augmentation method using an uncertainty guide according to an embodiment of the present invention.

[0047] In step 110, the computing device (100) applies the input image to a frequency-based image manipulator to apply the input image to a first augmentation module to generate a Fourier-based augmented image.

[0048] Figure 2 illustrates a detailed process for generating a Fourier-based augmented image using the first augmentation module. The overall framework of the first augmentation module is illustrated in Figure 3.

[0049] A method for generating a Fourier-based augmented image through the first augmentation module will be described in more detail with reference to FIGS. 2 and 3.

[0050] At step 210, the computing device (100) uses a fast Fourier transform to change the amplitude and phase spectrum while maintaining the essential characteristics of the original data (input image).

[0051] For example, given a grayscale two-dimensional input image (X) with a resolution of H x W, the Fourier transform can be formulated as in Equation 1.

[0052]

[0053] Here, represents the 2D fast Fourier transform (FFT) that converts an image in the spatial domain into the frequency domain, and represents the coordinates in frequency space.

[0054] In step 215, the computing device (100) changes the amplitude component based on the fast Fourier transform result to generate an amplitude component map.

[0055] For example, amplitude component and phase components can be calculated from the input image as in Equations 2 and 3.

[0056]

[0057]

[0058] Here, R(X) and I(X) represent the real and imaginary parts of the Fourier transform, respectively. Without loss of generality, the amplitude and phase silver and In addition, the inverse FFT (iFFT) for converting the spectrum signal including amplitude and phase in the frequency domain back to the spatial domain is For convenience, let's express it as and is applied to the amplitude and phase spectra to center the low-frequency components. A shift operator that multiplies can be applied.

[0059] Source domain and target domain respectively and Let's write it as . is the source domain In N SRepresents a set of trained images and their corresponding correct labels, is the target domain In N T Represents a set of unused test images and their corresponding correct labels. am.

[0060] Also, W, H, and C represent the width, height, and number of classes, respectively. For the medical image segmentation (MIS) task. A segmentation model parameterized by is typically trained through fully supervised learning in the source domain, as in Equation 4.

[0061]

[0062] Here, represents the type of segmentation loss, is a split model represents the optimized weights.

[0063] In the SDG (domain generalization) scenario, the target domain is trained during the training phase. The goal is to improve segmentation performance in this unseen state. Therefore, by effectively augmenting the source image, we can induce learning so that the segmentation model can operate robustly and reliably even in other unused domains. This process is shown in Equation 5.

[0064]

[0065] This will be explained in more detail below.

[0066] Improving the ability to adapt to changes in texture and contrast in an image is crucial in medical imaging, especially when differentiating tissue types.

[0067] To this end, the amplitude component can be manipulated to change the global contrast that varies depending on the domain of the image within the spatial domain, and texture differences can be induced. Accordingly, the computing device (100) can be used to transform the amplitude can be performed. The amplitude transformation is a randomly generated probability as in Equation 6. This can be done by inverting the scale of the amplitude component.

[0068]

[0069] Here, and are the absolute value operation and amplitude operation respectively. represents the median. Because amplitude transformation affects the variation of domain-varying features in the spatial domain, it can generate image content with various characteristics. Amplitude transformation can make the model less sensitive to unknown textures in unseen domains.

[0070] For the convenience of understanding and explanation, the result of performing amplitude transformation on the amplitude component is called an amplitude component map.

[0071] In step 220, the computing device (100) can calculate the angular density distribution using the amplitude component map.

[0072] In one embodiment of the present invention, we assume that regions with high amplitude at specific angular points are important because they contain rich domain feature information. Each angular point represents the point where features that vary across domains are most prominent, and can provide valuable insights into the structural and textural elements of the image, including surface patterns and fine details distributed across the image.

[0073] Angular density distribution, which represents the amount of information about an angle to estimate meaningful angular points. can calculate the angular density distribution is the transformed amplitude At the center coordinate d center Based on at It shows information for each angle.

[0074] The angular density distribution can be calculated using mathematical formula 7.

[0075]

[0076] Here, R and are the converted amplitudes respectively It represents the width or height and the floor function.

[0077] In step 225, the computing device (100) calculates the maximum and minimum distributions from the calculated angular density distribution. Each angle can be derived. indicates the direction in which the distinct features or edges of the image are contained, whereas indicates a relatively less prominent direction.

[0078] In step 230, the computing device (100) extracts the maximum and minimum angles of the amplitude components and then uses them to determine the center coordinates d of the amplitude spectrum. center Radius from Angle of length Partial sector mask that masks the amplitude region within can be created.

[0079] Amplitude component masking can be expressed as in mathematical equation 8.

[0080]

[0081] Here, and represents a randomly selected angle. Therefore, in one embodiment of the present invention, a maximum angle mask is created using the derived maximum and minimum angles. and minimum angle mask can be generated respectively. The generated partial sector masks represent the indication functions performed in the polar coordinate system.

[0082] For amplitude masking, the maximum angle mask that can reduce the most dominant directional attribute. You can make other textural features more prominent by applying a maximum angle mask. Amplitude Multiply by the phase component Inverse FFT with Amplitude masked image by applying can be generated (step 235). If this is expressed as a mathematical formula, it is as follows:

[0083]

[0084] Here, represents the Hadamard product.

[0085] At step 240, the computing device (100) performs amplitude internal modulation using a maximum angle mask. and minimum angle mask Amplitude using You can change the values ​​within these angular masks. By changing the key directional information between these angular masks, you can emphasize or weaken the inherent properties of specific directions while maintaining the overall pattern and structure.

[0086] Maximum angle mask The value of the minimum angle mask It can be exchanged with the corresponding position (u, v) and vice versa. If this is expressed as a mathematical formula, it is as shown in Equation 10.

[0087]

[0088] Next, in step 245, the computing device (100) modulates the amplitude component internally. Using inverse FFT Internally modified images based on can be reconstructed. If this is expressed as a mathematical formula, it is as follows: Mathematical Formula 11.

[0089]

[0090] At step 250, the computing device (100) can perform a phase transformation on the phase component to generate a phase component map.

[0091] Utilizing topological information can reflect the shape and structural integrity of an object (i.e., domain-invariant properties), making the model more sensitive to objects of different shapes. Reconstructed images using amplitude manipulation and To enhance high-level semantics, we utilize a phase-based attention mechanism for data augmentation.

[0092] At step 255, the computing device (100) uses the phase component map to generate a spatial domain image. can be reconstructed. Here, is a constant representing the amplitude component for performing iFFT. For convenience, the spatial domain image reconstructed using the phase component map is called a phase image.

[0093] At step 260, the computing device (100) reconstructs the phase image The image can be refined using a bilateral filter to smooth out noise while preserving anatomical appearance (e.g., tissue boundaries).

[0094] Bilateral filtering uses spatial proximity between pixels (spatial kernel ) and intensity difference (range kernel (usage) can be considered. Each image pixel can be replaced by a weighted average of its neighboring pixels. Phase attention with bilateral filtering can be defined as in Equation 12.

[0095]

[0096] Here (i,j) and are each reconstructed phase images Indicate the center pixel position and pixel intensity processed at the location (ij), respectively. In addition, represents a window (i.e., a filter mask) around the pixel (ij), Is -Using norm in (ij) The Euclidean distance is calculated. The weights are determined based on the spatial distance and intensity difference, and the filter can be set to emphasize edges with large intensity changes.

[0097] In step 265, the computing device (100) finally reconstructs the image through amplitude manipulation. and , an image with bilateral filtering applied to the reconstructed phase image. can be used to generate Fourier-based augmented images.

[0098] If this is expressed as a mathematical formula, it is as follows: Mathematical Formula 13.

[0099]

[0100] Here, is a constant that controls the strength of image self-mixup adjustment, and can recognize structural integrity and perform accurate segmentation by appropriately applying domain-invariant representation through phase attention.

[0101] Figure 4 illustrates pseudocode for a method of generating a Fourier-based augmented image using an input image.

[0102] Referring back to FIG. 1, at step 115, the computing device (100) applies a Fourier-based augmented image (i.e., a context-augmented image) to the learned segmentation model to generate a context uncertainty map.

[0103] As described above, the input image is applied to the first augmentation module. A Fourier-based augmented image (context-augmented image) can be obtained as follows. Such context-augmentation can generate a transformation that affects the entire image context. The first augmentation module introduces disturbances to important context features through amplitude masking and internal modulation, and phase attention can maintain semantic information (Mathematical Formula 11).

[0104] A fine balance is achieved through self-mixing, ensuring that the augmented image reflects the essential characteristics of the original image. This allows the segmentation model to adapt to multiple contextual variations with diverse contextual properties, enabling it to effectively address unseen images.

[0105] A segmentation model using Fourier-based augmented images (i.e., context-augmented images) can be optimized with the objective function of Equation 14.

[0106]

[0107] Here, , and Y represents the actual value. Split model Silver cross entropy loss and Dice loss can be optimized by a combination of .

[0108] In step 120, the computing device (100) performs a Bezier transformation based on the input image and the input mask and applies the result to the augmentation module to generate a position augmented image.

[0109] The goal of augmented location awareness is to simulate segment-by-segment contrast mismatches that may arise due to unseen domain modalities.

[0110] The computing device (100) can adjust the contrast of each segment area by applying a Bezier curve to the input image and input mask pair. As illustrated in FIG. 5, the Bezier transform function starts at a starting position to limit the value range through the intensity scale of the input. and can be generated using . In addition, the two control positions P1 and P2 have a limited range can be generated randomly from.

[0111] The Bezier transformation can be performed using equation (15).

[0112]

[0113] Here, t represents a fractional value of the line length. For simplicity, we use the Bezier curve-based transformation function It is expressed as, is each class label Indicates the probability that the inverse transformation will be performed on the domain.

[0114] When activating the reverse transformation in , the values ​​of the start and end positions may be changed. For example, And, That is, it is the same as the right side of Fig. 5.

[0115] The Bezier-transformed result can be applied to an augmentation module after linear scaling and shift terms are additionally applied to expand data diversity, thereby generating a position-augmented image.

[0116] If this is expressed as a mathematical formula, it is as follows: Mathematical Formula 16.

[0117]

[0118] Here, and are two truncated Gaussian distributions represents the standard deviation, and C represents the number of classes.

[0119] In step 125, the computing device (100) applies the position augmented image to the learned segmentation model to generate a position uncertainty map.

[0120] Image segmentation model similar to context-aware augmented optimization silver Using and It can be trained based on the loss. This can be expressed mathematically as Equation 17.

[0121]

[0122] To handle uncertainty harmoniously during the augmentation process, the segmentation module can be trained to focus on regions where segmentation fails due to long-term or intra-long-term contrast similarity.

[0123] For this purpose, epistemic uncertainty can be induced by the learned segmentation model.

[0124] To estimate uncertainty U, Shannon entropy We decide to use . Here, represents the softmax activation logit (i.e., posterior probability) of the partition module for each partition c.

[0125] That is, the trained segmentation model From and can generate an uncertainty map, i.e., And, am.

[0126] In step 130, the computing device (100) aggregates and fuses the context uncertainty map and the location uncertainty map to generate an uncertainty guide map.

[0127] Context Uncertainty Map and location uncertainty map By taking into account the heterogeneity between them and appropriately fusing them through aggregate fusion, an uncertainty guide map can be generated by using the uncertainty map as a guide to induce augmentation in the confusion region.

[0128] An uncertainty guide map can be generated as in mathematical equation 18.

[0129]

[0130] Here, GaussianBlur represents a smoothing function that applies a Gaussian filter to the uncertainty map. The first term of Equation 18 highlights areas where one of the augmentation processes identified significant ambiguity, ensuring that these areas are prioritized in subsequent augmentation guides. Conversely, the second term of Equation 18 provides a holistic view of the uncertainty of both methods and smooths out extreme values ​​to avoid overemphasizing inconsistencies.

[0131] In step 135, the computing device (100) generates a mutually augmented image using the uncertainty guide map, the context augmented image, and the location augmented image.

[0132] Mutually augmented images is a context-augmented image C-FAT and location augmented image L-FAT can be generated based on linear interpolation fusion using .

[0133] More specifically, the computing device (100) has an uncertainty guide map And this role and By multiplying, a mutually enhanced image can be generated. This can be expressed mathematically as Equation 19.

[0134]

[0135] Here, represents the constraint probability, which is an uncertainty guide map It represents the average uncertainty probability for the area corresponding to the foreground. By using , the mutual reinforcement process can be controlled to prevent excessive reinforcement in areas where the model has not yet learned sufficiently (i.e., low model confidence).

[0136] In step 140, the computing device (100) applies the mutually augmented image to the segmentation model to generate a predicted segmentation mask.

[0137] Split model is the predicted mask Using and It can be optimized for loss.

[0138] If this is expressed as a mathematical formula, it is as follows: Mathematical Formula 20.

[0139]

[0140] The overall framework for this is illustrated in Fig. 6, and the overall pseudocode is illustrated in Fig. 7.

[0141] FIG. 8 is a block diagram schematically illustrating the internal configuration of a computing device that performs a Fourier-based semantic data augmentation method utilizing an uncertainty guide according to one embodiment of the present invention.

[0142] Referring to FIG. 8, a computing device (100) according to one embodiment of the present invention is configured to include a first augmentation module (810), a second augmentation module (820), a mutual augmentation module (830), a memory (840), and a processor (850).

[0143] The first augmentation module (810) can generate a context augmentation image by converting the input image into a frequency domain, then applying the converted image to a segmentation model to generate a context uncertainty map.

[0144] The second augmentation module (820) can generate a position augmentation image by performing a Bezier transformation based on an input image and an input mask, converting the result into a frequency domain, and then applying the position augmentation image to a segmentation model to generate a position uncertainty map.

[0145] The mutual augmentation module (830) can generate an uncertainty guide map by fusing the context uncertainty map and the position uncertainty map, and can generate a mutual augmentation image using the context augmentation image, the position augmentation image, and the uncertainty guide map. In addition, the mutual augmentation module (830) can apply the mutual augmentation image to a segmentation model to generate a predicted segmentation mask and train it.

[0146] The memory (840) stores at least one instruction for performing a Fourier-based semantic data augmentation method utilizing an uncertainty guide according to one embodiment of the present invention.

[0147] The processor (850) is a means for controlling internal components of a computing device (100) according to one embodiment of the present invention (e.g., a first augmentation module (810), a second augmentation module (820), a mutual augmentation module (830), and a memory (840), etc.).

[0148] Detailed descriptions of the operations of the first augmentation module (810), the second augmentation module (820), and the mutual augmentation module (830) are the same as those described with reference to FIGS. 1 to 7, so redundant descriptions will be omitted.

[0149]

[0150] The device and method according to the embodiment of the present invention may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands recorded on the computer-readable medium may be those specially designed and configured for the present invention or may be those known and usable by those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc.

[0151] The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

[0152] The present invention has been described above, focusing on specific embodiments thereof. Those skilled in the art will appreciate that the present invention can be implemented in modified forms without departing from its essential characteristics. Therefore, the disclosed embodiments should be considered illustrative rather than limiting. The scope of the present invention is set forth in the claims, not the foregoing description, and all differences within the scope equivalent thereto should be construed as being encompassed by the present invention.

Claims

A step of applying an input image to an augmentation module to transform it into a frequency domain and then generating a context augmented image through amplitude and phase transformation; A step of applying the result of performing a Bezier transformation based on the input image and input mask to the augmentation module to transform it into a frequency domain and then generating a position augmentation image through amplitude and phase transformation; A step of applying the context augmented image and the position augmented image to each segmentation model to generate a context uncertainty map and a position uncertainty map, respectively; A step of generating an uncertainty guide map by fusing the context uncertainty map and the location uncertainty map; and A Fourier-based semantic data augmentation method utilizing uncertainty guides, comprising the step of generating a mutual augmented image using the context augmented image, the location augmented image, and the uncertainty guide map. In the first paragraph, The above augmentation module, A step of generating an amplitude component map and a phase component map, respectively, by transforming input data into a frequency domain and then transforming amplitude and phase components, wherein the input data is the input image or the result of the Bezier transform; A step of calculating an angular density distribution based on the amplitude component map, extracting a maximum angle and a minimum angle from the angular density distribution, respectively, and masking the maximum angle amplitude area and the minimum angle amplitude area to generate a maximum angle mask and a minimum angle mask; A step of generating an amplitude masked image by multiplying the amplitude component by the maximum angle mask and then applying an inverse fast Fourier transform together with the phase component, and generating an internal modulation image by exchanging the areas of the maximum angle mask and the minimum angle mask; A step of generating a phase image using the phase component map and refining it by applying a bilateral filter; and A Fourier-based semantic data augmentation method utilizing an uncertainty guide, characterized in that the step of generating a Fourier-based context augmented image or a position augmented image by self-mixing the phase image to which the bilateral filter is applied, the amplitude masked image, and the internal modulation image, respectively. In the first paragraph, A Fourier-based semantic data augmentation method utilizing an uncertainty guide, characterized in that the uncertainty guide map is generated using the following mathematical formula. Here, GaussianBlur represents a smoothing function that applies a Gaussian filter to the uncertainty map, represents the context uncertainty map, represents the position uncertainty map. In the first paragraph, A Fourier-based semantic data augmentation method utilizing an uncertainty guide, characterized in that the mutual augmentation image is generated by multiplying the uncertainty guide map and the inverse of the uncertainty guide map by the context augmentation image and the location augmentation image. In paragraph 4, A Fourier-based semantic data augmentation method utilizing an uncertainty guide, characterized in that the mutual augmentation image is generated using the following mathematical formula. Here, represents a context-augmented image, represents a location augmented image, represents the uncertainty guide map, represents the constraint probability. In the first paragraph, A Fourier-based semantic data augmentation method utilizing uncertainty guides, further comprising a step of applying the mutual augmentation image to the segmentation model to generate a predicted segmentation mask and training the same. A first augmentation module that generates a context-augmented image by transforming an input image into a frequency domain, then applying the transformed image to a segmentation model to generate a context uncertainty map; A second augmentation module that converts the result of performing a Bezier transform based on the input image and input mask into a frequency domain, then generates a position augmentation image through amplitude and phase transformation, and applies the position augmentation image to the segmentation model to generate a position uncertainty map; and A computing device comprising a mutual augmentation module that generates an uncertainty guide map by fusing the context uncertainty map and the position uncertainty map, and generates a mutual augmentation image using the context augmented image, the position augmented image, and the uncertainty guide map.

Images