Device and Method providing learning model regarding denoise
By segmenting low-dose images into continuous and random segments to extract global and local information, the learning model is optimized for efficient denoising, addressing the inefficiencies of existing techniques and improving image quality.
Patent Information
- Application Number
- KR1020240080910
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2024-06-21
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2044-06-21
AI Technical Summary
Existing denoising techniques for low-dose images struggle to preserve object edges and are inefficient in resource management, particularly for images with non-linear noise characteristics.
A learning model is updated by dividing low-dose images into continuous and random segments, applying each segment to a learning model to acquire global and local information, and using this information to optimize the model for improved denoising performance.
The method enhances the learning model's ability to preserve detailed information and textures in low-dose images while optimizing resource usage, resulting in improved image quality and efficiency.
Smart Images

Figure 112024067217711-PAT00003_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to a method and device for providing a learning model for denoising that is updated according to information obtained by applying an image included in a low-dose image of an object to a learning model. More specifically, the invention relates to a technical field in which noise in a low-dose image can be efficiently removed and the performance of the learning model can be improved by dividing the low-dose image into continuous and random segments, applying each to a learning model, and providing an updated learning model that reflects each of the information obtained therefrom. Background Technology
[0003] Conventionally, filtering-based techniques such as Gaussian, Anisotropic Diffusion, and Bilateral have been used to remove noise from low-dose images. However, existing denoising techniques have faced many difficulties in preserving the edges of objects, and particularly, problems have arisen where the denoising effect is reduced for noise with non-linear characteristics. For example, since ultra-low-dose CBCT images are already faint due to severe scatter noise, it was inconvenient to perform efficient denoising while maintaining the shape of the object even if existing denoising techniques were used.
[0004] Consequently, various learning models and algorithms have recently been proposed to improve low-dose image quality. However, since efficient resource management for denoising low-dose images has not been considered, unnecessary resources are being used, and problems in the process of optimizing learning models still persist.
[0005] Therefore, there is a need for a technology that can improve the performance of a learning model for denoising using efficient resources by segmenting images included in low-dose images into continuous and random segments and feeding them to a learning model, and improving the learning model for denoising using the information obtained therefrom. Prior art literature
[0007] Korean Registered Patent 10-2448069 B1 (2022.09.22) Image processing method and apparatus for reducing noise included in an image acquired by radiography The problem to be solved
[0008] The present disclosure provides a method and device for providing a learning model for updated denoising by applying an image included in a low-dose image to a learning model and reflecting the acquired information to solve the problem described above.
[0009] Specifically, the present invention aims to provide a learning model for updated denoising using global information representing image characteristics of the entire object acquired by dividing low-dose images into continuous segments and applying them to a learning model, and local information representing image characteristics of each region of the object acquired by dividing them randomly and applying them to a learning model.
[0010] The problems to be solved in this disclosure are not limited to those mentioned above, and other unmentioned problems will be clearly understood by a person skilled in the art from the description below. means of solving the problem
[0012] As a technical means for achieving the technical problem described above, a method for a device according to the first aspect of the present disclosure to provide a learning model for denoising may include: a receiving unit acquiring a low-dose image and a high-dose image of a target; a processor acquiring a low-dose continuous segmented image by dividing the low-dose image so as to be continuous; the processor acquiring global information representing image characteristics for the entire target by applying the low-dose continuous segmented image to a learning model; the processor acquiring a low-dose random segmented image by randomly dividing the low-dose image; the processor acquiring local information representing image characteristics by region of the target by applying the low-dose random segmented image to the learning model; and the method of updating the learning model using the global information and the local information to acquire an updated learning model.
[0013] Additionally, the step of acquiring the global information may include: the step of the processor acquiring a denoising continuous segmented image for the target by applying the low-dose continuous segmented image to the learning model; the step of the processor acquiring a denoising whole image using the denoising continuous segmented image; and the step of the processor acquiring the global information based on the comparison result between the denoising whole image and the high-dose image.
[0014] Additionally, the step of acquiring the local information may include: the step of the processor applying the low-dose random segmented image to the learning model to acquire a denoising random segmented image for the subject; the step of the processor randomly segmenting the high-dose image to acquire a high-dose random segmented image; and the step of the processor acquiring the local information based on the comparison result between the denoising random segmented image and the high-dose random segmented image.
[0015] Additionally, the method may further include the step of acquiring a new denoising image of the new target using a new low-dose image of the new target and the updated learning model.
[0016] In addition, the above global information may include brightness information for the entire object.
[0017] In addition, the local information may include information about the shape of a structure included in the object.
[0018] Additionally, the step of acquiring the global information may include: a step in which the processor applies the low-dose continuous segmented image to the learning model to acquire a predicted high-dose image for the object; and a step in which the processor acquires the global information based on a comparison result between the predicted high-dose image and the high-dose image.
[0019] Additionally, the step of acquiring the local information may include: a step in which the processor applies the low-dose random segmented image to the learning model to acquire a predicted high-dose random segmented image for the subject; a step in which the processor randomly segments the high-dose image to acquire a high-dose random segmented image; and a step in which the processor acquires the local information based on a comparison result between the predicted high-dose random segmented image and the high-dose random segmented image.
[0020] In addition, the step of acquiring the low-dose continuous segmented image may determine the number of the low-dose continuous segmented images based on the size of the low-dose image and the predicted noise level for the low-dose image.
[0021] Additionally, the step of obtaining global information based on the comparison result between the entire denoising image and the high-dose image may include: a step of obtaining a first brightness value indicating the degree of brightness or darkness for the entire denoising image; a step of obtaining a second brightness value indicating the degree of brightness or darkness for the high-dose image; and a step of obtaining global information based on an error value output by applying the first brightness value and the second brightness value to a loss function.
[0022] Additionally, the step of obtaining local information based on the comparison result between the denoising random segmented image and the high-dose random segmented image may include: a step of obtaining a first sharpness value representing the sharpness of the edge of the shape of the structure included in the object included in the denoising random segmented image; a step of obtaining a second sharpness value representing the sharpness of the edge of the shape of the structure included in the object included in the high-dose random segmented image corresponding to the denoising random segmented image; and a step of obtaining local information based on an error value output by applying the first sharpness value and the second sharpness value to a loss function.
[0023] A device for providing a learning model for denoising according to a second aspect of the present disclosure may include: a receiver that acquires a low-dose image and a high-dose image of an object; and a processor that divides the low-dose image so as to be continuous to acquire a low-dose continuous divided image, applies the low-dose continuous divided image to a learning model to acquire global information representing image characteristics of the entire object, divides the low-dose image randomly to acquire a low-dose random divided image, applies the low-dose random divided image to the learning model to acquire local information representing image characteristics of each region of the object, and updates the learning model using the global information and the local information to acquire an updated learning model.
[0024] Additionally, a third aspect of the present disclosure may provide a computer-readable recording medium having a program for executing the method of the first aspect on a computer. Effects of the invention
[0026] According to one embodiment of the present disclosure, learning model optimization can be performed by updating the learning model by reflecting both global information representing image characteristics of the entire object acquired by dividing the image included in the low-dose image so as to be continuous and inputting it to the learning model, and local information representing image characteristics of each region of the object acquired by dividing it randomly and inputting it to the learning model, thereby resolving the problem of difficulty in preserving detailed information about the low-dose image and textures about the object.
[0027] In addition, as a training model optimized for low-dose images is executed, the quality of low-dose images can be improved by utilizing efficient resources that facilitate performance enhancement of the training model.
[0028] The effects of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below. Brief explanation of the drawing
[0030] FIG. 1 is a block diagram schematically illustrating the configuration of a device providing a learning model for denoising according to one embodiment. FIG. 2 is a flowchart illustrating a method for a device according to one embodiment to provide a learning model for denoising. FIG. 3 is a diagram illustrating the process of applying a low-dose continuous segmented image, obtained by dividing a device according to one embodiment to a learning model so that the low-dose image can be continuous. FIG. 4 is a diagram illustrating an example in which a device according to one embodiment applies a denoising whole image and a high-dose image obtained from a denoising continuous segmented image to a loss function to obtain global information. FIG. 5 is a diagram illustrating the process of a device according to one embodiment randomly dividing a low-dose image and applying the acquired low-dose randomly divided image to a learning model. FIG. 6 is a diagram illustrating an example in which a device according to one embodiment applies a denoising random segmented image and a high-dose random segmented image to a loss function to obtain local information. FIG. 7 is a diagram illustrating an example in which a device according to one embodiment acquires a new denoising image through an updated learning model using global information and local information. FIG. 8 is a diagram schematically illustrating the flow of interaction between a device, a shooting device, a database, and a user device according to one embodiment. Specific details for implementing the invention
[0031] The advantages and features of the present disclosure and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below but can be implemented in various different forms, and the embodiments provided are merely to make the disclosure complete and to fully inform those skilled in the art of the scope of the present disclosure.
[0032] The terms used in this specification are for describing embodiments and are not intended to limit the disclosure. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. The terms “comprises” and / or “comprising” as used in this specification do not exclude the presence or addition of one or more other components in addition to the components mentioned. Throughout the specification, the same reference numerals refer to the same components, and “and / or” includes each of the mentioned components and all combinations of one or more. Although terms such as “first,” “second,” etc., are used to describe various components, these components are not limited by these terms. These terms are used merely to distinguish one component from another. Accordingly, the first component mentioned below may be the second component within the technical scope of this disclosure.
[0033] Unless otherwise defined, all terms used herein (including technical and scientific terms) may be used in a sense that is commonly understood by a person skilled in the art. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise. Spatially relative terms such as "below," "beneath," "lower," "above," and "upper" may be used to easily describe the relationship between one component and other components, as illustrated in the drawings.
[0034] Spatially relative terms should be understood as terms that include different orientations of components during use or operation, in addition to the orientations depicted in the drawings. For example, if a component depicted in a drawing is inverted, a component described as "below" or "beneath" another component may be placed "above" of that other component. Therefore, the exemplary term "below" may include both the downward and upward directions. Components may also be oriented in other directions, and accordingly, spatially relative terms may be interpreted according to the orientation.
[0035] As used in this disclosure, the term "model" may be understood as a system implemented using mathematical concepts and language to solve a specific problem, a set of software units to solve a specific problem, or an abstract model regarding a processing process to solve a specific problem. For example, a neural network "model" may refer to an overall system implemented as a neural network that possesses problem-solving capabilities through learning. In this case, the neural network may possess problem-solving capabilities by optimizing parameters connecting nodes or neurons through learning. A neural network "model" may include a single neural network or a set of neural networks composed of multiple neural networks.
[0036] The term "image" as used in this disclosure may refer to multidimensional data composed of discrete image elements. In other words, "image" may be understood as a term referring to a digital representation of an object visible to the human eye. For example, "image" may refer to multidimensional data composed of elements corresponding to pixels in a two-dimensional image. "Image" may refer to multidimensional data composed of elements corresponding to voxels in a three-dimensional image.
[0037] Various embodiments are described in detail below with reference to the drawings.
[0038] FIG. 1 is a block diagram schematically illustrating the configuration of a device (100) that provides a learning model for denoising according to one embodiment.
[0039] Referring to the drawing, the device (100) may include a receiving unit (110), a processor (120), and a transmitting unit (130).
[0040] A person skilled in the art will understand that, in addition to the components shown in FIG. 1, other general-purpose components may be further included in the device (100). For example, the device (100) may further include a memory (not shown). Or, according to other embodiments, a person skilled in the art will understand that some of the components shown in FIG. 1 may be omitted.
[0041] A receiver (110) according to one embodiment can acquire low-dose images and high-dose images of an object from a shooting device or a database. The low-dose and high-dose images may be images included in a medical image in which a patient or a part of a patient is photographed, or a simulation model corresponding to the shape of a part of a patient is photographed.
[0042] Additionally, the receiver (110) can obtain a base storage continuous division count from the database. The base storage continuous division count may refer to an optimized division count for obtaining global information representing the image characteristics of the entire object by dividing low-dose images so as to be continuous, and may be updated according to the learning results obtained from the learning model.
[0043] Additionally, the receiver (110) can obtain a base number of random divisions from a database. The base number of random divisions may refer to an optimized number of divisions for obtaining local information representing the image characteristics of a target area by randomly dividing a low-dose image, and may be updated according to the learning results obtained from a learning model.
[0045] A processor (120) according to one embodiment can obtain a low-dose continuous segmented image by dividing a low-dose image of an object obtained from a receiver (110) so that it can be continuous. Additionally, the processor (120) can obtain global information representing the image characteristics of the entire object by applying the low-dose continuous segmented image to a learning model. Additionally, the processor (120) can obtain a low-dose random segmented image by randomly dividing a low-dose image of an object obtained from a receiver (110). Additionally, the processor can obtain local information representing the image characteristics of each region of the object by applying the low-dose random segmented image to a learning model. Additionally, the processor (120) can obtain an updated learning model by updating the learning model using the global information and local information.
[0046] A processor (120) according to one embodiment can process computational processes such as processing input data for updating a learning model, extracting features, and calculating error values based on backpropagation. A processor (120) for performing such data processing may include a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). Since the above-described types of processors (120) are merely examples, the types of processors (120) may be configured in various ways within a range understandable to those skilled in the art based on the contents of this disclosure.
[0047] A processor (120) according to one embodiment can generate training data for a deep learning model to denoise images included in low-quality medical images to high quality. The processor (120) can generate input data and label data for training the deep learning model by using metadata associated with medical data. The processor (120) can generate input data and label data for the deep learning model from medical data by considering the characteristics of the metadata of the medical data. At this time, the medical data may include medical images that have been accelerated or accelerated simulated. And, the deep learning model may include a neural network model that restores the medical images that have been accelerated or accelerated simulated to a normal shooting state. In other words, the deep learning model may include a neural network model that generates an output that denoises the medical images that have been accelerated or accelerated simulated or improves the resolution. For example, the neural network model may include a U-NET based on a convolutional neural network. The types of neural networks described above are merely examples, and can be configured in various ways within the scope understandable to those skilled in the art based on the contents of this disclosure.
[0049] FIG. 2 is a flowchart illustrating a method for a device (100) according to one embodiment to provide a learning model for denoising.
[0050] Referring to step S210, the device (100) can acquire low-dose images and high-dose images of the target.
[0051] For example, low-dose images and high-dose images may be obtained from one or more frames included in a medical image. The medical image is not limited to a specific medical image and has the same meaning as generally understood by those skilled in the art.
[0052] For example, the device (100) can acquire low-dose and high-dose images of an object in pairs from a shooting device. A low-dose image may refer to an image where the dose can be reduced as the shooting time is reduced, but severe blurring occurs, requiring image quality improvement. Additionally, if the object corresponds to a general patient, there is a high probability that the general patient will be overexposed to radiation during the process of capturing high-dose images of the object, and it may be difficult to acquire information about an object with the same phase due to the general patient's movement. Accordingly, the low-dose (e.g., 20 mAs) and high-dose (e.g., 200 mAs) images acquired by the device (100) in pairs can be acquired by shooting from a simulation model including a skull phantom or human phantom with a fixed position. Not limited thereto, the device (100) can acquire low-dose and high-dose images of a specific object stored in a database, rather than from a shooting device, in pairs.
[0054] Referring to step S220, the device (100) can obtain a low-dose continuous segmented image by dividing the low-dose image so that it can be continuous.
[0055] For example, a low-dose continuous segmented image may include one or more low-dose continuous segmented images (hereinafter also referred to as 'forward patches') that are segmented from a low-dose image to be continuous and applied to a learning model. For instance, the device (100) may obtain N number of low-dose continuous segmented images by segmenting the low-dose image into 0 forward patches, 1 forward patch, ..., N forward patches. Additionally, depending on the selection input regarding the size of the low-dose image or the number of N user accounts, the device (100) may determine the size and number of N low-dose continuous segmented images differently.
[0056] FIG. 3 is a diagram illustrating the process of applying a low-dose continuous segmented image (310), obtained by dividing a low-dose image (300) so that it can be continuous, to a learning model (320) by a device (100) according to one embodiment.
[0058] Referring to FIG. 3, a device (100) according to one embodiment can obtain N low-dose continuous segmented images (310) by dividing the acquired low-dose image (300) into N numbers (e.g., N=9) corresponding to the size (e.g., 1000x1000x600). The acquired low-dose continuous segmented images (310) are applied to a learning model (320), and the device (100) can obtain global information representing image characteristics of the entire object. Additionally, the device (100) can determine N low-dose continuous segmented images (310) based on the number of stored continuous segments obtained from a database.
[0059] As a result of this, the number of base storage consecutive segments is the number of optimized forward patches updated by applying one or more existing low-dose images (300) to the learning model (320), and it is possible to obtain precise global information representing the image characteristics of the entire object included in the low-dose image (300).
[0061] A device (100) according to one embodiment can determine the number of low-dose continuous segmented images (310) based on the size of the low-dose image (300) and the predicted noise level for the low-dose image (300).
[0062] The predicted noise level may be included in the image preprocessing process performed by the device (100) as the low-dose image (300) is acquired. That is, each pixel included in the low-dose image (300) is scanned, and the device (100) can determine a predicted noise level (e.g., high) indicating the degree of blurring of the low-dose image (300).
[0063] Specifically, when the size of the low-dose image (300) is greater than or equal to a preset image size and the predicted noise level (e.g., high) for the low-dose image (300) is greater than or equal to a preset noise level (e.g., medium), the device (100) can determine the number of low-dose continuous segmented images (310) obtained by dividing the low-dose image (300) so that it can be continuous to be greater than or equal to a threshold number (e.g., N = 18).
[0064] As a result of this, when the noise of the low-dose image (300) is severe before being applied to the learning model (320), the number of low-dose continuous segmented images (310) is increased beyond a threshold number, thereby increasing the probability of reducing errors in global information representing the image characteristics of the entire object, and thus the learning model (320) can be updated efficiently.
[0066] Referring to step S230, the device (100) can obtain global information representing image characteristics of the entire object by applying a low-dose continuous segmented image (310) to a learning model (320).
[0067] For example, the learning model (320) to which the low-dose continuous segmented image (310) is applied is not limited to a specific learning model, and a deep learning learning model capable of extracting features or characteristics of the entire object from each low-dose continuous segmented image (310) may be used. Additionally, the learning model (320) may be updated and added according to the selection input of the user account. The global information obtained by applying to the learning model (320) is a global feature extracted to improve the denoising performance of the low-dose image (300), and since the low-dose image (300) contains a mixture of bright and dark parts, it may include brightness information of the entire object to which uniformization of brightness of the entire part can be applied.
[0069] A device (100) according to one embodiment can obtain a denoising continuous segmented image of an object by applying a low-dose continuous segmented image (310) to a learning model (320), obtain a denoising whole image using the denoising continuous segmented image, and obtain global information based on the comparison result between the denoising whole image and the high-dose image.
[0070] Specifically, the denoising continuous segmented image may represent the result of denoising for each forward patch when a low-dose continuous segmented image (310) is applied to a learning model (320). When the denoising continuous segmented image is acquired, the device (100) can combine it to acquire a denoising whole image for performing a comparison with a high-dose image. Accordingly, by performing a comparison between the individual pixel information included in the denoising whole image and the individual pixel information of the high-dose image, global information containing brightness information for the entire object can be acquired.
[0071] As a result of this, there is an effect of preventing the problem of the learning time becoming relatively long when processing without considering the amount of computation according to the size of the low-dose image (300). That is, by comparing the entire denoising image, which is a combination of each denoising sequential segmented image, with the high-dose image, brightness information for the entire object is obtained, thereby efficiently improving the performance of the learning model (320).
[0073] A device (100) according to one embodiment can obtain a predicted high-dose image of an object by applying a low-dose continuous segmented image (310) to a learning model (320) and can obtain global information based on the comparison result between the predicted high-dose image and the high-dose image.
[0074] Specifically, the predicted high-dose image may mean a denoised image in which a low-dose continuous segmented image (310) is applied to a learning model (320) and combined, and is an image similar to the high-dose image. That is, the device (100) can obtain global information representing the image characteristics of the entire object from the comparison result obtained by performing a comparison between the brightness level of the predicted high-dose image and the brightness level of the high-dose image.
[0075] As a result of this, the process of obtaining additional denoising full images can be reduced by obtaining a predicted high-dose image that is predicted to be a high-dose image rather than a denoising continuous segment image from a low-dose continuous segment image (310) applied to the learning model (320), thereby shortening the execution time of the learning model (320).
[0077] FIG. 4 is a diagram illustrating an example in which a device (100) according to one embodiment applies a denoising whole image (410) and a high-dose image (420) obtained from a denoising continuous segmented image (400) to a loss function (430) to obtain global information (440).
[0078] For example, the device (100) can acquire a first brightness value indicating a degree of brightness or darkness for a denoising whole image (410), acquire a second brightness value indicating a degree of brightness or darkness for a high-dose image (420), and acquire global information (440) based on an error value output by applying the first brightness value and the second brightness value to a loss function (430).
[0079] Referring to FIG. 4, the loss function (430) may represent a function for calculating an error value (e.g., 8%) by comparing a first brightness value (e.g., 20%) for the denoising whole image (410) and a second brightness value (e.g., 28%) for the high-dose image (420), with the denoising whole image (410) and the high-dose image (420) being input values. The first brightness value and the second brightness value may be obtained from the average brightness value for all pixels included in each of the denoising whole image (410) and the high-dose image (420), and the device (100) may update the existing learning model (320) by reflecting the error value output using the loss function (430) in the global information (440).
[0080] As a result of this, when a new low-dose image is applied to the learning model (320), the error value for the brightness value obtained from the loss function (430) is reflected and updated in the error value used in the existing learning model (320), so that the low-dose continuous segmented image (310) is obtained as an improved denoising continuous segmented image (400), and the effect of gradually increasing the improvement in image quality for the entire object is achieved.
[0082] Referring to step S240, the device (100) can obtain a low-dose randomly divided image by randomly dividing the low-dose image (300).
[0083] In the above-described step S230, the device (100) divides the low-dose image (300) so that it can be continuous, and the acquired N number of low-dose continuous segmented images (310) are applied to the learning model (320) to obtain global information (440) from which brightness information or global features for the object are extracted. In step S240, as a process of obtaining local information by extracting information about the shape of a structure included in the object or local features, the device (100) may obtain low-dose random segmented images (hereinafter also referred to as 'random patches') by randomly dividing a part of the object included in the low-dose image (300). For example, the device (100) may obtain M number of low-dose random segmented images by dividing the low-dose image (300) into 0 random patches, 1 random patch, ..., M random patches. Additionally, depending on the size of the low-dose image (300) or the shape of the structure included in the object, the device (100) may determine the size and number of M of the low-dose random segmented images differently. For example, each low-dose random segmented image may include a structure that is part of the object, and may also include soft tissue that is part of the object, but is not limited thereto. That is, if the object corresponds to a tooth, a person skilled in the art can intuitively infer the name related to the object appearing in the image, so it is obvious that the low-dose random segmented image is not limited to any part of the object.
[0085] FIG. 5 is a diagram illustrating the process of a device (100) according to one embodiment randomly dividing a low-dose image (300) and applying the acquired low-dose randomly divided image (500) to a learning model (320).
[0087] Referring to FIG. 5, a device (100) according to one embodiment can obtain M low-dose random segmented images (500) by dividing an acquired low-dose image (300) into M random patches (e.g., M=6) corresponding to the shape (e.g., tooth length) of a structure included in an object within the low-dose image (300). The acquired low-dose random segmented images (500) are applied to a learning model (320), and the device (100) can obtain local information representing the image characteristics of each region of the object. Additionally, the device (100) can determine M low-dose random segmented images (500) based on a pre-stored number of random segments obtained from a database.
[0088] As a result of this, the number of base-stored random segments is the number of optimized random patches updated by applying one or more existing low-dose images (300) to the learning model (320), and it is possible to obtain precise local information representing the image characteristics of the target area.
[0089] A device (100) according to one embodiment may determine the number of low-dose random segmented images (500) differently depending on the size of the low-dose image (300). That is, if the size of the low-dose image (300) (e.g., size 6) is greater than or equal to a preset standard size (e.g., size 5), the device (100) may obtain an additional number of segments representing the number of additional segments from the existing number of low-dose random segmented images (500) and apply the low-dose random segmented images (500) to the learning model (320). The unit for the size of the low-dose image (300) is not limited to a specific unit and can be added and updated according to user input.
[0090] As a result of this, in the case of a low-dose image (300) larger than a preset standard size, a larger number of low-dose random segmented images (500) are acquired and applied to a learning model (320), thereby allowing for the extraction of more precise regional features or characteristics of the object corresponding to the structure or soft tissue of the object.
[0091] A device (100) according to one embodiment may determine the number of low-dose random segmented images (500) differently depending on the area of a structure included in an object within a low-dose image (300). That is, in the case of a structure area (e.g., molars) for an object within a low-dose image (300), the shape of the molars is more likely to appear as complex curvatures than the shape of the incisors, and accordingly, noise may appear severely at the boundaries between teeth. Accordingly, the device (100) may acquire a structure area corresponding to molars within the low-dose image (300) with a number of low-dose random segmented images (500) (e.g., M=5), and a structure area corresponding to incisors with a number of low-dose random segmented images (500) (e.g., M=3).
[0092] As a result of this, since noise is expected to be severe in some areas with complex curvature regarding the shape of the structure in the low-dose image (300), the number of low-dose random segmented images (500) is divided into more than some areas without complex curvature, thereby reducing the probability of errors in local information representing the image characteristics of the target area.
[0094] Referring to step S250, the device (100) can obtain local information representing the region-specific image characteristics of an object by applying a low-dose random segmented image (500) to a learning model (320).
[0095] For example, the learning model (320) to which the low-dose random segmented image (500) is applied is not limited to a specific learning model, and a deep learning learning model capable of extracting region-specific features or features of an object from each low-dose random segmented image (500) may be used. The local information obtained by applying to the learning model (320) is a local feature that represents region-specific features of an object to improve denoising performance for the low-dose image (300), and may include information about the shape of a structure included in an object that allows for the preservation of edges appearing in each region of the object.
[0097] A device (100) according to one embodiment can obtain a denoising random segmented image of a target by applying a low-dose random segmented image (500) to a learning model (320), obtain a high-dose random segmented image by randomly segmenting a high-dose image (420), and obtain local information based on the comparison result between the denoising random segmented image and the high-dose random segmented image.
[0098] Specifically, the denoising random segmented image may represent a result value that is denoised for each random patch when a low-dose random segmented image (500) is applied to a learning model (320). When the denoising random segmented image is acquired, the device (100) can acquire a high-dose random segmented image corresponding to each region within the denoising random segmented image. Accordingly, by performing a comparison between the pixel information contained in each denoising random segmented image and the pixel information of each high-dose random segmented image, local information containing information about the shape of the structure included in the object can be acquired.
[0099] As a result of this, the amount of computation varies depending on the size of the low-dose image (300) during the process of acquiring image characteristics of each region of the target, so there is a high probability of unexpected overload, which has the effect of preventing the problem of difficult resource management. That is, as each denoising random segmented image is compared with the corresponding high-dose image, local information about each region of the target (e.g., structural region or soft tissue) is acquired, thereby efficiently improving the performance of the learning model (320).
[0101] A device (100) according to one embodiment can obtain a predicted high-dose random segmented image for a target by applying a low-dose random segmented image (500) to a learning model (320), obtain a high-dose random segmented image by randomly segmenting a high-dose image (420), and obtain local information based on the comparison result between the predicted high-dose random segmented image and the high-dose random segmented image.
[0102] Specifically, the predicted high-dose random segmented image may refer to a denoised random segmented image obtained by applying a low-dose random segmented image (500) to a learning model (320). That is, the device (100) can obtain local information representing the image characteristics of a region of an object from the comparison result obtained by performing a comparison between the edge sharpness of a part of a structure included in an object for each predicted high-dose random segmented image (e.g., 40%) and the edge sharpness of a part of a structure included in an object for each corresponding high-dose random segmented image (e.g., 50%).
[0103] As a result of this, local information can be obtained quickly as the time to perform comparison with the high-dose image (420) is shortened by obtaining a predicted high-dose random segment image rather than a denoising random segment image from the low-dose random segment image (500) applied to the learning model (320).
[0105] FIG. 6 is a diagram illustrating an example in which a device (100) according to one embodiment applies a denoising random segmented image (600) and a high-dose random segmented image (610) to a loss function (430) to obtain local information (620).
[0106] For example, the device (100) can obtain a first sharpness value representing the sharpness of the edge of the shape of the structure included in the object included in the denoising random segmented image (600), and can obtain a second sharpness value representing the sharpness of the edge of the shape of the structure included in the object included in the high-dose random segmented image (610) corresponding to the denoising random segmented image (600), and can obtain local information (620) based on the error value output by applying the first sharpness value and the second sharpness value to the loss function (430).
[0107] Referring to FIG. 6, the loss function (430) may represent a function for calculating an error value (e.g., 10%) by comparing a first sharpness value (e.g., 60%) representing the sharpness of the edge of the shape of the structure included in the object of each denoising random segmented image (600) and a second sharpness value (e.g., 70%) representing the sharpness of the edge of the shape of the structure included in the object of the corresponding high-dose random segmented image (610), when the denoising random segmented image (600) and the corresponding high-dose random segmented image (610) are input values. The first sharpness value and the second sharpness value can be obtained from the average sharpness value of pixels adjacent to the edge of the shape of the structure included in the object corresponding to each of the denoising random segmented image (600) and the high-dose random segmented image (610), and the device (100) can update the existing learning model (320) by reflecting the output error value in the local information (620) using the loss function (430).
[0108] As a result of this, when a new low-dose image is applied to the learning model (320) by reflecting and updating the error value for the sharpness value obtained from the loss function (430) in the error value used in the existing learning model (320), the low-dose random segmented image (500) is acquired as an improved denoising random segmented image (600), and the improvement in image quality of the shape of the structure included in the object is gradually increased.
[0110] A device (100) according to one embodiment can acquire a target random segmented image that is one of a plurality of denoising random segmented images (600), acquire a target random segmented image that is one of a plurality of high-dose random segmented images (610), and acquire local information (620) representing image characteristics of a target area according to a similarity level indicating similarity between the shape information of a structure included in the target random segmented image and the shape information of a structure included in the target random segmented image. That is, when the shape of a structure included in the target of the target random segmented image and the shape of a structure included in the target of the target random segmented image are similar (e.g., the similarity level is higher than a preset similarity level), the device (100) can acquire local information (620) including information about the shape of a structure included in the target according to the comparison result between the target random segmented image and the target random segmented image. Additionally, the device (100) can update the existing learning model (320) by inputting the target random segmented image and the target random segmented image into the loss function (430) and reflecting the output error value in the local information (620).
[0111] As a result of this, the target random segment image among the multiple high-dose random segment images (610) that are most similar to the target random segment image among the multiple denoising random segment images (600) is compared, and as a precise error value is determined, the performance improvement of the existing learning model (320) can be achieved more quickly.
[0113] A device (100) according to one embodiment can update a learning model (320) based on an error value output by applying a denoising random whole image, in which a plurality of denoising random segmented images (600) are combined into one image, and a high-dose image to a loss function (430). Accordingly, based on the comparison result between the output error value (e.g., 1) and the reference error value (e.g., 1.2), the device (100) can update the reference error value to a value closer to 0 and obtain an updated learning model that reflects this.
[0114] As a result of this, as the reference error value approaches 0, the learning model (320) is updated, and the new low-dose image of the new object can be gradually improved to an image close to the high-dose image by obtaining a new denoising image with minimized noise due to the updated learning model.
[0116] In addition, the error value between the part area of the object appearing in each random patch of the high-dose random segmented image (610) corresponding to the part area of the object appearing in each random patch of the denoising random segmented image (600) can be obtained, thereby reducing the comparison execution time and resources used.
[0118] Referring to step S260, the device (100) can obtain an updated learning model by updating the learning model (320) using global information (440) and local information (620).
[0119] Referring to steps S230 and S250 described above, the device (100) has obtained global information (440) representing image characteristics of the entire object and local information (620) representing image characteristics of each region of the object. Accordingly, by reflecting the obtained global information (440) and local information (620) into the existing learning model (320), the device (100) can obtain an updated learning model.
[0121] FIG. 7 is a diagram illustrating an example in which a device (100) according to one embodiment acquires a new denoising image (710) through an updated learning model (700) using global information (440) and local information (620).
[0122] Referring to FIG. 7, the device (100) can obtain a new denoising image (710) of a new object using a new low-dose image of a new object and an updated learning model (700). By applying a low-dose continuous segmented image (310) to an existing learning model (320) to obtain global information (440) representing the image characteristics of the entire object and applying a low-dose random segmented image (500) to the learning model (320) to obtain local information (620) representing the image characteristics of each region of the object, the device (100) can obtain an updated learning model (700) by updating the existing learning model (320) by reflecting these in the model.
[0123] A device (100) according to one embodiment can obtain an updated learning model (700) by updating a learning model (320) based on weights that are differently assigned to global information (440) and local information (620), respectively. That is, among the update value (e.g., 0.5) corresponding to global information (440) containing brightness information for the entire object and the update value (e.g., 0.3) corresponding to local information (620) containing information about the shape of a structure included in the object, a higher weight is assigned to the local information (620) that is closer to 0, and the device (100) can obtain an updated learning model (700) by reflecting this in the existing learning model (320).
[0125]
[0126] A device (100) according to one embodiment can obtain an updated learning model (700) by assigning preset weights to global information (440) and local information (620), respectively. Referring to Equation 1, the device (100) can calculate an error value of a loss function (430) by assigning preset weights to global information (440) obtained based on the comparison result between the acquired denoising whole image (410) and the high-dose image (420), and to local information (620) obtained based on the comparison result between the denoising random segmented image (600) and the high-dose random segmented image (610), respectively. Accordingly, the device (100) can obtain an updated learning model (700) by reflecting the calculated error value.
[0127] As a result of this, different weights are assigned to each of the global information (440) obtained by extracting wide-area features of the low-dose image (300) and the local information (620) obtained by extracting local features of the low-dose image (300), thereby enabling the acquisition of a balanced updated learning model (700) whenever it is updated, and enabling the acquisition of a new denoising image (710) with improved quality from a new low-dose image of a new target.
[0128] FIG. 8 is a diagram for schematically illustrating the flow of interaction between a device (100) according to one embodiment, a shooting device (800), a database (820), and a user device (810).
[0129] Referring to FIG. 8, the device (100) can acquire a low-dose image (300) and a high-dose image (420) of an object from a shooting device (800). Additionally, if there is a history of low-dose images (300) and high-dose images (420) corresponding to the object being acquired in the past, the device (100) can acquire the low-dose image (300) and the high-dose image (420) of the object from a database (820). The device (100) can acquire a low-dose continuous segmented image (310) that divides the low-dose image (300) so that it can be continuous, and a low-dose random segmented image (500) that divides it randomly, and by applying these to a learning model (320), it can acquire global information (440) representing the image characteristics of the entire object and local information (620) representing the image characteristics of each region of the object, respectively. Accordingly, the device (100) can obtain an updated learning model (700) by updating the learning model (320) using the acquired global information (440) and local information (620), and when a denoising low-dose image of an object exhibiting quality most similar to the high-dose image (420) is requested from the user device (810), the device (100) can provide the user device (810) with a denoising low-dose image that has been trained with the updated learning model (700) and improved to high quality.
[0131] Various embodiments of the present disclosure may be implemented as software comprising one or more instructions stored in a storage medium (e.g., memory) readable by a machine (e.g., a display device or a computer). For example, a processor of the machine (e.g., processor (120)) may call at least one of the one or more instructions stored from the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' merely means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.
[0132] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created in a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.
[0133] Those skilled in the art related to the embodiments described above will understand that they may be implemented in modified forms without departing from the essential characteristics of the description. Therefore, the disclosed methods should be considered in an illustrative rather than a restrictive sense. The scope of the invention is defined by the claims, not by the foregoing description, and all variations within the scope of equivalence should be interpreted as being included in the invention. Explanation of the symbols
[0135] 100: Device 110: Receiver 120: Processor 130: Transmitter 300: Low-dose image 310: Low-dose serial segmentation image 320: Learning Model 400: Denoised serial segmented image 410: Denoised full image 420: Image of amount of loss 430: Loss function 440: Global Information 500: Low-dose random segmented image 600: Denoising random segmented image 610: High-dose random segmentation image 620: Local information 700: Updated training model 710: New denoised image 800: Shooting device 810: User device 820: Database
Claims
Claim 1 A method for a device to provide a learning model for denoising comprises: a receiving unit acquiring a low-dose image and a high-dose image of an object; a processor dividing the low-dose image so as to be continuous to acquire a low-dose continuous segmented image; the processor applying the low-dose continuous segmented image to the learning model to acquire global information representing image characteristics for the entire object; the processor randomly dividing the low-dose image to acquire a low-dose random segmented image; the processor applying the low-dose random segmented image to the learning model to acquire local information representing image characteristics for each region of the object; and the processor updating the learning model using the global information and the local information to acquire an updated learning model; wherein the step of acquiring the low-dose continuous segmented image comprises: the processor determining the number of low-dose continuous segmented images based on at least one of the size of the low-dose image and the predicted noise level for the low-dose image. A method comprising the step of the processor dividing the low-dose image into continuous segments according to the determined number to obtain the low-dose continuous segmented image. Claim 2 The method according to claim 1, wherein the step of acquiring global information comprises: the step of the processor acquiring a denoising continuous segmented image for the subject by applying the low-dose continuous segmented image to the learning model; the step of the processor acquiring a denoising whole image using the denoising continuous segmented image; and the step of the processor acquiring global information based on a comparison result between the denoising whole image and the high-dose image. Claim 3 The method according to claim 1, wherein the step of acquiring local information comprises: the step of the processor applying the low-dose random segmented image to the learning model to acquire a denoising random segmented image for the subject; the step of the processor randomly segmenting the high-dose image to acquire a high-dose random segmented image; and the step of the processor acquiring local information based on a comparison result between the denoising random segmented image and the high-dose random segmented image. Claim 4 A method according to claim 1, further comprising the step of acquiring a new denoising image of the new object using a new low-dose image of the new object and the updated learning model. Claim 5 A method according to claim 1, wherein the global information includes brightness information for the entire object. Claim 6 A method according to claim 1, wherein the local information includes information about the shape of a structure included in the object. Claim 7 A method according to claim 1, wherein the step of acquiring global information comprises: a step in which the processor applies the low-dose continuous segmented image to the learning model to acquire a predicted high-dose image for the subject; and a step in which the processor acquires the global information based on a comparison result between the predicted high-dose image and the high-dose image. Claim 8 A method according to claim 1, wherein the step of acquiring local information comprises: a step in which the processor applies the low-dose random segmented image to the learning model to acquire a predicted high-dose random segmented image for the subject; a step in which the processor randomly segments the high-dose image to acquire a high-dose random segmented image; and a step in which the processor acquires local information based on a comparison result between the predicted high-dose random segmented image and the high-dose random segmented image. Claim 9 A method according to claim 1, wherein the step of acquiring the low-dose random segmented image comprises: the step of the processor calculating the curvature complexity of the structure of the object included in the low-dose image; and the step of the processor determining the number of the low-dose random segmented images based on the calculated curvature complexity. Claim 10 In claim 2, the step of obtaining global information based on a comparison result between the denoising whole image and the high-dose image comprises: a step of obtaining a first brightness value indicating a degree of brightness or darkness for the denoising whole image; a step of obtaining a second brightness value indicating a degree of brightness or darkness for the high-dose image; and a step of obtaining global information based on an error value output by applying the first brightness value and the second brightness value to a loss function. Claim 11 In claim 3, the step of obtaining local information based on a comparison result between the denoising random segmented image and the high-dose random segmented image comprises: obtaining a first sharpness value representing the sharpness of an edge for a shape of a structure included in the object included in the denoising random segmented image; obtaining a second sharpness value representing the sharpness of an edge for a shape of a structure included in the object included in the high-dose random segmented image corresponding to the denoising random segmented image; and obtaining local information based on an error value output by applying the first sharpness value and the second sharpness value to a loss function. Claim 12 A device for providing a learning model for denoising, comprising: a receiver for acquiring a low-dose image and a high-dose image of an object; and a processor for acquiring a low-dose continuous segmented image by dividing the low-dose image so as to be continuous, applying the low-dose continuous segmented image to a learning model to acquire global information representing image characteristics of the entire object, randomly dividing the low-dose image to acquire a low-dose random segmented image, applying the low-dose random segmented image to the learning model to acquire local information representing image characteristics of each region of the object, and updating the learning model using the global information and the local information to acquire an updated learning model; wherein the processor determines the number of low-dose continuous segmented images based on at least one of the size of the low-dose image and the predicted noise level for the low-dose image, and acquires the low-dose continuous segmented images by dividing the low-dose image so as to be continuous according to the determined number. Claim 13 A computer-readable recording medium having a program stored on it for executing the method of any one of claims 1 to 11 on a computer.
Citation Information
Patent Citations
Multi-center federated learning CT imaging method and system based on global information sharing
CN116416332A
Method for learning contouring model based on deep learning using denoising, method for contouring using denoising based on the contouring model, apparatus and computer program for performing the methods
KR1020230102253A
Method and apparatus for reducing noise of image obtained by radiography
KR102448069B1
Transforming projection data in tomography by means of machine learning
US20180018757A1
Generative adversarial low-dose CT denoising method based on double-domain U-net discriminator
CN113538257A
