ETE detection method and ETE detection device

Abstract

An ETE detection method is disclosed. The ETE detection method according to the present invention comprises the steps of: providing an ultrasound image to a first learning model pre-trained to extract a thyroid region, thereby acquiring a first boundary of the thyroid, and providing the ultrasound image to a second learning model pre-trained to extract a nodule region, thereby acquiring a second boundary of a nodule; and outputting a superimposed image in which the first boundary and the second boundary overlap, thereby enabling ETE detection.

Description

ETE detection method and ETE detection device

[0001] This application claims priority to Korean Patent Application No. 10-2024-0059280, filed on May 3, 2024, and all contents disclosed in the specification and drawings of the said application are incorporated herein by reference.

[0002] The present invention relates to an ETE detection method and an ETE detection device that can improve the accuracy of ETE detection by detecting the boundary of a thyroid gland and the boundary of a nodule from an ultrasound image and then providing them in an overlapping manner.

[0003] Extrathyroidal extension (ETE) is one of several diagnostic indicators for thyroid cancer. Specifically, ETE refers to the expansion of thyroid cancer beyond the thyroid gland into the surrounding tissues. This phenomenon, which occurs depending on the stage of thyroid cancer (i.e., as cancer cells grow), plays a crucial role in the diagnosis and treatment of thyroid cancer. Therefore, ETE is a key indicator for assessing the risk of thyroid cancer and is also an important indicator for assessing the presence of metastasis.

[0004] Ultrasound examinations are commonly performed to detect ETEs, and physicians typically detect ETEs by viewing ultrasound images. While ultrasound offers the advantage of enabling noninvasive ETE detection, it has the limitation that it displays internal organs in black and white images, requiring distinction between the thyroid gland and nodules based solely on differences in brightness. Therefore, conventional ETE detection using ultrasound images has relied on the physician's extensive experience and intuition, and misdiagnosis has frequently occurred even among inexperienced physicians and sometimes even experienced physicians. In particular, although accurate detection of ETE requires precise detection of the borders of the thyroid gland and nodules, the limitations of ultrasound imaging have made accurate ETE detection even more difficult.

[0005] The present invention is intended to solve the above-described problems, and the purpose of the present invention is to provide an ETE detection method and an ETE detection device capable of improving the accuracy of ETE detection by detecting the boundary of the thyroid gland and the boundary of a nodule from an ultrasound image and then providing them in an overlapping manner.

[0006] The ETE detection method according to the present invention comprises the steps of: providing an ultrasound image to a first learning model pre-trained to extract a thyroid region to obtain a first boundary of the thyroid gland; providing the ultrasound image to a second learning model pre-trained to extract a nodule region to obtain a second boundary of the nodule; and outputting an overlapping image in which the first boundary and the second boundary overlap so as to enable ETE detection.

[0007] In this case, the first learning model can be pre-trained using ultrasound images in which part or all of the nodule is located on the thyroid gland as training data and boundary images of the thyroid gland as labeling data.

[0008] In this case, the first learning model can be pre-trained using an ultrasound image in which a nodule obscures the boundary of the thyroid gland as training data and a boundary image of the thyroid gland as labeling data.

[0009] In this case, the boundary image of the thyroid gland can be generated by estimating the boundary of the thyroid gland obscured by the nodule based on prior knowledge of the shape of the thyroid gland.

[0010] Meanwhile, the step of obtaining the first boundary of the thyroid by providing an ultrasound image to a first learning model pre-trained to extract the thyroid region may include a step of obtaining the first boundary in which a part of the boundary is restored through the first learning model, even though a nodule in the ultrasound image covers a part of the boundary of the thyroid in the ultrasound image.

[0011] Meanwhile, the step of outputting an overlapping image in which the first boundary and the second boundary overlap may include a step of outputting first state information when the second boundary exists inside the first boundary, outputting second state information when the second boundary and the first boundary touch, and outputting third state information when the second boundary protrudes outside the first boundary.

[0012] In this case, the step of outputting the overlapping image may further include the step of outputting the distance between the first boundary and the second boundary.

[0013] Meanwhile, the ETE detection device according to the present invention includes an output unit, a communication unit that acquires an ultrasound image, and a control unit that provides the ultrasound image to a first learning model pre-trained to extract a thyroid region to acquire a first boundary of the thyroid gland, provides the ultrasound image to a second learning model pre-trained to extract a nodule region to acquire a second boundary of the nodule, and outputs an overlapping image in which the first boundary and the second boundary overlap through the output unit so as to enable ETE detection.

[0014] In this case, the first learning model can be pre-trained using ultrasound images in which part or all of the nodule is located on the thyroid gland as training data and boundary images of the thyroid gland as labeling data.

[0015] In this case, the first learning model can be pre-trained using an ultrasound image in which a nodule obscures the boundary of the thyroid gland as training data and a boundary image of the thyroid gland as labeling data.

[0016] In this case, the boundary image of the thyroid gland can be generated by estimating the boundary of the thyroid gland obscured by the nodule based on prior knowledge of the shape of the thyroid gland.

[0017] Meanwhile, the control unit can obtain the first boundary, in which a portion of the boundary is restored, through the first learning model, even though the nodule in the ultrasound image covers a portion of the boundary of the thyroid gland in the ultrasound image.

[0018] Meanwhile, the control unit may output first state information when the second boundary exists inside the first boundary, output second state information when the second boundary and the first boundary touch, and output third state information when the second boundary protrudes outside the first boundary.

[0019] In this case, the control unit can output the distance between the first boundary and the second boundary.

[0020] Meanwhile, a computer program stored in a non-transitory computer-readable medium according to the present invention executes an ETE detection method including the steps of: providing an ultrasound image to a first learning model pre-trained to extract a thyroid region to obtain a first boundary of the thyroid gland, providing the ultrasound image to a second learning model pre-trained to extract a nodule region to obtain a second boundary of the nodule, and outputting an overlapping image in which the first boundary and the second boundary overlap so as to enable ETE detection.

[0021] Whether a nodule protrudes beyond the thyroid gland's border, whether the nodule's border is in contact with the thyroid gland's border, or whether the nodule is located within the thyroid gland's border can be crucial in determining the degree (or type) of ETE. Furthermore, according to the present invention, by accurately extracting the thyroid gland's border and the nodule's border and then providing them in an overlapping manner, physicians can be enabled to make an accurate diagnosis of ETE.

[0022] FIG. 1 is a block diagram illustrating components of an ETE detection device according to the present invention.

[0023] Figure 2 is a drawing showing an example of an ultrasound image used to detect ETE.

[0024] Figure 3 is a drawing for explaining a training method of an artificial intelligence model according to the present invention.

[0025] Figure 4 is a flowchart for explaining an ETE detection method according to the present invention.

[0026] FIG. 5 is a drawing illustrating a first boundary, a second boundary, and an overlapping image according to the present invention.

[0027] Figure 6 is a drawing showing various positional relationships between the thyroid gland and nodules according to the present invention.

[0028] Figure 7 is a diagram illustrating the effect of the first learning model restoring the boundary of the thyroid gland obscured by a nodule.

[0029] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components will be given the same reference numbers and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably only for the convenience of writing the specification, and do not in themselves have distinct meanings or roles. In addition, when describing the embodiments disclosed in this specification, if it is determined that a specific description of a related known technology may obscure the gist of the embodiments disclosed in this specification, a detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and technical scope of the present invention.

[0030] Terms that include ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by these terms. These terms are used solely to distinguish one component from another.

[0031] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0032] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprises" or "have" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0033] In implementing the present invention, components may be described in detail for convenience of explanation, but these components may be implemented in one device or module, or one component may be implemented by being divided into multiple devices or modules.

[0034] FIG. 1 is a block diagram illustrating components of an ETE detection device according to the present invention.

[0035] Hereinafter, detection of ETE in the thyroid gland using ultrasound images will be described. However, the present invention is not limited thereto, and the present invention can also be applied to detecting organs inside the human body and nodules on organs using medical images. Here, the organs inside the human body may include not only the thyroid gland, but also various organs located inside the body such as the lungs, stomach, and liver. In addition, the medical images may include ultrasound images, x-ray images, CT images, MRI images, etc. In addition, the medical images include organs and tumors, and the medical images can be used to determine the location of the tumor on the organ, whether the tumor has invaded the outside of the organ, and the distance between the border of the tumor and the border of the organ. In addition, in the present specification, a nodule may be a concept including a mass or tumor.

[0036] The ETE detection device (100) (hereinafter referred to as “device (100)”) may include a communication unit (110), a control unit (120), a memory (130), and an output unit (140). The device (100) may include only some of the components illustrated in FIG. 1, or may include additional components not illustrated in FIG. 1.

[0037] The communication unit (110) can communicate with external devices via one or more wired or wireless networks. To this end, the communication unit (110) may include a communication circuit or communication module for communicating with the external device.

[0038] The communication unit may be referred to as a communicator or a communication interface, and may include one or more communication modules (or communication circuits) to transmit or receive data with devices other than the ETE detection device (100).

[0039] The communication unit (110) can receive ultrasound images from an external device. For example, the communication unit (110) can receive ultrasound images generated by the ultrasound imaging equipment when the ultrasound imaging equipment photographs a patient. The ultrasound image may include the photographed thyroid gland. If a nodule develops in the patient's thyroid gland, the ultrasound image may further include the nodule photographed along with the thyroid gland.

[0040] As another example, the ETE detection device (100) may further include a photographing unit that captures ultrasonic images. In this case, the communication unit (110) may receive the ultrasonic images from the photographing unit.

[0041] The control unit (120) can control the overall operation of the device (100). The term “control unit” may be used interchangeably with terms such as “microprocessor,” “controller,” “microcontroller,” and “processor.”

[0042] The control unit (120) may be expressed as a controller and may be composed of one or more processors (or microcontrollers, or microprocessors). Furthermore, one or more processors (or microcontrollers, or microprocessors) may be coupled to other components of the ETE detection device (100). Specifically, the control unit (120) may be coupled to other components of the ETE detection device (100) to transmit and receive signals for processing data.

[0043] In addition, the control unit (120) can obtain the first boundary of the thyroid gland and the second boundary of the nodule using an ultrasound image, and generate an overlapping image in which the first boundary and the second boundary are overlapped.

[0044] Meanwhile, the control unit (120) can read and execute an artificial intelligence model from the memory (130). Accordingly, the operations of the artificial intelligence models (first learning model and second learning model) described below can be viewed as operations of the control unit (120).

[0045] The output unit (140) can output information processed by the control unit (120), and the information processed by the control unit (120) can include an overlapping image in which the first boundary and the second boundary overlap. If the overlapping image is implemented to be displayed on an external terminal (e.g., a terminal used by a doctor), the output unit (140) can include a communication module (or a communication circuit) and transmit the overlapping image to the external terminal. If the overlapping image is implemented to be displayed on the ETE detection device (100), the output unit (140) can include a display and display the overlapping image on the display.

[0046] The memory (130) can store programs, commands, and other data for operating the device (100). In addition, the memory (130) can include a database in which a plurality of medical images are stored.

[0047] Additionally, the memory (130) can store an artificial intelligence model. Specifically, the artificial intelligence model can be implemented as software or a combination of hardware and software, and one or more commands constituting the artificial intelligence model can be stored in the memory (130). The artificial intelligence model can include a first learning model used to extract the thyroid region and a second learning model used to extract the nodule region.

[0048] Figure 2 is a drawing showing an example of an ultrasound image used to detect ETE.

[0049] Examples of criteria indicating the degree (or type) of ETE include no ETE, minor ETE, and gross ETE. No ETE is shown in Fig. 2a, minor ETE in Fig. 2b, and gross ETE in Fig. 2c.

[0050] In more detailed classification, thyroid capsular abutment is the absence of normal thyroid tissue between the malignant nodule and the thyroid capsule. Capsular disruption, classified as Minor ETE to Anterolateral, is the disappearance of the echogenic line surrounding the thyroid gland adjacent to the malignant nodule. Thyroid capsular protrusion, classified as Minor ETE to Posterior, is the bulging of the malignant nodule into the thyroid structures regardless of the presence of capsular disruption. Replacement of strap muscles, classified as Gross ETE to strap muscle, is the protrusion of the nodule into the strap muscle and the border with the strap muscle is unclear. In addition, Protrusion into TEG, classified as Gross ETE to RLN, and Obtuse angle, classified as Gross ETE to trachea, are also used to determine the degree (or type) of ETE.

[0051] Considering these criteria, what is important in determining the degree (or type) of ETE is whether the nodule protrudes beyond the border of the thyroid gland, how far the nodule protrudes beyond the thyroid gland, whether the border of the nodule touches the border of the thyroid gland, whether the nodule is located within the border of the thyroid gland, and how close a nodule located within the border of the thyroid gland approaches the border of the thyroid gland. Therefore, the purpose of this specification is to accurately detect the border of the thyroid gland and the border of the nodule and the positional relationship between the two borders.

[0052] Figure 3 is a drawing for explaining a training method of an artificial intelligence model according to the present invention.

[0053] The artificial intelligence model may include a first learning model (310) and a second learning model (320). The first learning model (310) may be pre-trained to extract a thyroid region from an ultrasound image, and the second learning model (320) may be pre-trained to extract a nodule region from an ultrasound image.

[0054] Here, extracting the thyroid region may include extracting the region occupied by the thyroid gland within the ultrasound image, or extracting the boundary of the thyroid gland within the ultrasound image (i.e., the boundary between the region occupied by the thyroid gland and another region). If the learning model is implemented in a manner that extracts the region occupied by the thyroid gland, the control unit (120) may generate the boundary of the thyroid gland based on the predicted results of the learning model (the region occupied by the thyroid gland and the region not occupied by the thyroid gland).

[0055] In addition, extracting a nodule region may include extracting the entire region occupied by the nodule within the ultrasound image, or extracting the boundary of the nodule within the ultrasound image (i.e., the boundary between the region occupied by the nodule and another region). If the learning model is implemented in a manner that extracts the region occupied by the nodule, the control unit (120) may generate the boundary of the nodule based on the prediction result of the learning model (the region occupied by the nodule and the region not occupied by the nodule).

[0056] Each of the first learning model (310) and the second learning model (320) can be implemented as a neural network including one or more hidden layers and trained using a supervised learning algorithm. In this specification, "training" may refer to the process of updating the parameters (weights, biases, etc.) of the corresponding model.

[0057] Referring to FIG. 3a, the ETE detection device (100) (hereinafter referred to as “device (100)”) or other learning device can train the first learning model (310). Specifically, the device (100) or other learning device can train the first learning model (310) using training data (ultrasound images) and labeling data matching the ultrasound images based on a supervised learning algorithm.

[0058] The training data may be ultrasound images generated by taking an ultrasound scan of the thyroid gland. Therefore, the ultrasound images may include the thyroid gland.

[0059] Additionally, the ultrasound image may further include a nodule along with the thyroid gland, wherein part or all of the nodule may be located on the thyroid gland. Here, the fact that part or all of the nodule is located on the thyroid gland may mean that part or all of the nodule obscures the thyroid gland at the imaging angle from which the ultrasound image was taken. For example, at the imaging angle from which the ultrasound image was taken, part of the area occupied by the nodule may be located above the area occupied by the thyroid gland, and another part of the area occupied by the nodule may be located outside the area occupied by the thyroid gland. In another example, at the imaging angle from which the ultrasound image was taken, the entire area occupied by the nodule may be located above the area occupied by the thyroid gland.

[0060] The present invention aims to accurately detect the boundary of the thyroid gland from an ultrasound image containing both thyroid nodules and nodules. Therefore, by providing ultrasound images containing some or all of the nodules located on the thyroid gland as training data, the first learning model (310) can accurately distinguish between the thyroid gland and nodules and then detect the boundary of the thyroid gland.

[0061] Additionally, nodules may obscure the borders of the thyroid gland on ultrasound images.

[0062] Specifically, within the imaging angle from which an ultrasound image is taken, there are areas occupied by the thyroid gland and areas not occupied by the thyroid gland. The area actually occupied by the thyroid gland but obscured by a nodule can be defined as the thyroid gland-occupied area. Conversely, the area not actually occupied by the thyroid gland (i.e., the background outside the thyroid gland) can be defined as the non-thyroid area.

[0063] Additionally, the area occupied by the thyroid gland may include the thyroid margin (the area occupied by the thyroid gland but where it meets the background outside the thyroid gland) and the thyroid non-margin (the area occupied by the thyroid gland but where it does not meet the background outside the thyroid gland).

[0064] Additionally, within the imaging angle at which an ultrasound image is taken, there are areas occupied by a nodule and areas not occupied by a nodule. The area not occupied by a nodule may include at least one of the thyroid area outside the nodule or the background outside the thyroid. Furthermore, the area occupied by a nodule may include the nodule's border (the area occupied by the nodule but intersecting with the area not occupied by the nodule) and the nodule's non-border (the area occupied by the nodule but not intersecting with the area not occupied by the nodule).

[0065] Furthermore, the fact that a nodule obscures the border of the thyroid gland in an ultrasound image may mean that the area occupied by the nodule obscures the border of the thyroid gland. For example, at the imaging angle at which the ultrasound image was taken, only the border of the nodule overlaps with the border of the thyroid gland, so the nodule may obscure the border of the thyroid gland (e.g., Fig. 6b). In another example, at the imaging angle at which the ultrasound image was taken, the area occupied by the nodule (the border of the nodule and the non-border of the nodule) overlaps with the border of the thyroid gland, so the nodule may obscure the border of the thyroid gland (e.g., Fig. 6c).

[0066] As previously explained, when determining the degree (or type) of ETE, the boundaries of the thyroid gland, the boundaries of the nodules, and the positional relationship between the boundaries of the thyroid gland and the nodules are crucial. Therefore, the present invention utilizes ultrasound images in which the boundaries of the thyroid gland are obscured by nodules as training data, thereby enabling the first learning model (310) to accurately extract the boundaries of the thyroid gland despite the nodules obscuring the boundaries.

[0067] The device (100) or other learning device can obtain labeling data matching the training data (ultrasound image). The labeling data refers to the correct answer that the first learning model (310) that has received the training data (ultrasound image) should output, and may be a boundary image of the thyroid gland. In addition, the boundary image of the thyroid gland may be a portion that indicates the boundary of the thyroid gland (the area occupied by the thyroid gland and the area that intersects the area not occupied by the thyroid gland) within the ultrasound image provided as training data.

[0068] If the border of the thyroid gland is not obscured by a nodule, it is relatively easy to obtain labeling data (a thyroid gland boundary image), and in some lucky cases, this may be possible using only an ultrasound image (for example, if the thyroid gland boundary is not obscured by a nodule and the ultrasound image is taken so that the nodule and the thyroid gland are clearly distinguished). On the other hand, if the border of the thyroid gland is obscured by a nodule, the labeling data must be obtained by estimating the border of the thyroid gland, which can be a very difficult task. In addition, in the present invention, the labeling data (a thyroid gland boundary image) can be generated by estimating the border of the thyroid gland obscured by a nodule based on prior knowledge of the thyroid gland shape.

[0069] Specifically, organs within the human body have a standardized shape. Therefore, a boundary image of the thyroid gland can be generated by estimating the occluded boundary based on prior knowledge of the thyroid gland's shape. For example, a skilled physician can estimate the boundary of the thyroid gland occluded by a nodule in a specific ultrasound image based on medical knowledge and clinical experience regarding the thyroid gland's shape. The labeling data (thyroid gland boundary image) according to the present invention can be generated by reflecting the physician's estimation.

[0070] Referring to FIG. 3a, the control unit (120) of the device (100) or other learning device can train the first learning model (310) by configuring training data and labeling data matching the training data into one training set.

[0071] As an example of a training method, the control unit (120) or other learning device may input training data into the first learning model (310). In this case, the first learning model (310) may process the training data based on its parameters (weights, biases, etc.) and then output an output image. The output image may include a boundary image of the thyroid gland predicted by the first learning model (310). In this case, the control unit (120) or other learning device may calculate the difference (LOSS) between the output image (more specifically, the boundary image of the thyroid gland predicted by the first learning model (310)) and labeling data (the boundary image provided as the correct answer), and update the parameters of the first learning model (310) based on the calculated difference (LOSS).

[0072] The control unit (120) or other learning device can repeatedly train the first learning model (310) using various training sets.

[0073] As described above, an ultrasound image in which part or all of a nodule is located on the thyroid gland can be used as training data, and the control unit (120) or other learning device can train the first learning model (310) using the ultrasound image in which part or all of a nodule is located on the thyroid gland as training data and the boundary image of the thyroid gland as labeling data.

[0074] In addition, as described above, an ultrasound image in which a nodule obscures the boundary of the thyroid gland can be used as training data, and the control unit (120) or other learning device can train the first learning model (310) using the ultrasound image in which a nodule obscures the boundary of the thyroid gland as training data and the boundary image of the thyroid gland as labeling data.

[0075] Additionally, the control unit (120) can train the first learning model (310) using ultrasound images in which nodules exist as training data and ultrasound images in which nodules exist but do not obscure the border of the thyroid gland.

[0076] By training using various training sets, the parameters of the first learning model (310) can be optimized. Accordingly, despite the difficulty in distinguishing between the thyroid and nodules due to the fact that ultrasound images represent the thyroid and nodules in black and white, despite the fact that the thyroid and nodules may have various positional relationships, and despite the fact that nodules may obscure the boundaries of the thyroid, the first learning model (310) can be pre-trained to accurately extract the boundaries of the thyroid. The first learning model (310) upon completion of training is stored in the memory (130) and can be used in steps S410 to S430.

[0077] Referring to FIG. 3b, the device (100) or other learning device can train the second learning model (320). Specifically, the device (100) or other learning device can train the second learning model (320) using training data (ultrasound images) and labeling data matching the ultrasound images based on a supervised learning algorithm.

[0078] The training data may be ultrasound images generated by imaging the thyroid gland and a nodule partially or completely located on the thyroid gland. Therefore, the ultrasound image may contain both the thyroid gland and the nodule. As previously explained, the fact that a nodule is partially or completely located on the thyroid gland may mean that the nodule partially or completely obscures the thyroid gland from the angle at which the ultrasound image was taken.

[0079] The present invention aims to accurately detect the boundaries of nodules from ultrasound images containing a mixture of thyroid glands and nodules. Therefore, by providing ultrasound images containing some or all of the nodules located on the thyroid gland as training data, the second learning model (320) can accurately distinguish between the thyroid gland and nodules and then detect the boundaries of the nodules.

[0080] Also, as in the first learning model (310), an ultrasound image in which a nodule obscures the boundary of the thyroid gland can be provided as training data to the first learning model (320). As previously explained, what is important when determining the degree (or type) of ETE is the boundary of the thyroid gland, the boundary of the nodule, and the positional relationship between the boundary of the thyroid gland and the boundary of the nodule. Therefore, in the present invention, by using an ultrasound image in which a nodule obscures the boundary of the thyroid gland as training data, the second learning model (320) can accurately extract the boundary of the nodule in a situation in which the boundary of the thyroid gland is obscured by a nodule.

[0081] The device (100) or other learning device can obtain labeling data matching the training data (ultrasound image). The labeling data refers to the correct answer that the second learning model (320) that has received the training data (ultrasound image) should output, and may be a boundary image of a nodule. In addition, the boundary image of the nodule may be a boundary of the nodule (a region occupied by the nodule and a portion where the region does not occupy the nodule intersects) within the ultrasound image provided as training data.

[0082] Similar to the boundary image of the thyroid gland, the boundary image of a nodule can also be generated based on a physician's medical knowledge and clinical experience. However, since the boundary of a nodule is not obscured by the thyroid gland and the shape of the nodule is not standardized, the boundary image of the nodule can be generated based on prior knowledge of distinguishing the nodule area from the thyroid area in an ultrasound image that provides limited information (e.g., intensity), rather than prior knowledge of the shape of the nodule. For example, a skilled physician can distinguish the nodule area from the thyroid area in an ultrasound image based on medical knowledge and clinical experience and then estimate the boundary of the nodule area. The labeling data (the boundary image of the thyroid gland) according to the present invention can be generated by reflecting the physician's estimate.

[0083] Referring to FIG. 3b, the control unit (120) of the device (100) or other learning device can train the second learning model (320) by configuring training data and labeling data matching the training data into one training set.

[0084] As an example of a training method, the control unit (120) or other learning device may input training data into the second learning model (320). In this case, the second learning model (320) may process the training data based on its own parameters (weights, biases, etc.) and then output an output image. The output image may include a boundary image of a nodule predicted by the second learning model (320). In this case, the control unit (120) or other learning device may calculate the difference (LOSS) between the output image (more specifically, the boundary image of the nodule predicted by the second learning model (320)) and labeling data (the boundary image of the nodule provided as the correct answer), and update the parameters of the second learning model (320) based on the calculated difference (LOSS).

[0085] The control unit (120) or other learning devices can repeatedly train the second learning model (320) using various training sets. Ultrasound images in which some or all of a nodule is located on the thyroid gland, ultrasound images in which no nodules exist, ultrasound images in which nodules obscure the border of the thyroid gland, ultrasound images in which nodules do not obscure the border of the thyroid gland, etc. can be used as various training data.

[0086] By training using various training sets, the parameters of the second learning model (320) can be optimized. Accordingly, despite the difficulty in distinguishing between the thyroid and nodules due to the fact that ultrasound images represent the thyroid and nodules in black and white, and despite the fact that the thyroid and nodules may have various positional relationships, the second learning model (320) can be pre-trained to accurately extract the boundaries of nodules. The second learning model (320) upon completion of training is stored in the memory (130) and can be used in steps S410 to S430.

[0087] Figure 4 is a flowchart for explaining an ETE detection method according to the present invention.

[0088] The ETE detection method according to the present invention may include a step (S410) of obtaining a first boundary of the thyroid gland by providing an ultrasound image to a first learning model pre-trained to extract a thyroid region, a step (S420) of obtaining a second boundary of the nodule by providing the ultrasound image to a second learning model pre-trained to extract a nodule region, and a step (S430) of outputting an overlapping image in which the first boundary and the second boundary overlap so as to enable ETE detection. For convenience of explanation, S410 is described as a previous step and S420 is described as a next step, but the order of S410 and S420 may be changed.

[0089] FIG. 5 is a drawing illustrating a first boundary, a second boundary, and an overlapping image according to the present invention.

[0090] The control unit (120) can obtain an ultrasound image through the communication unit (110). The ultrasound image may be an image of the thyroid gland and nodules captured through ultrasound photography.

[0091] Referring to FIG. 5A, the control unit (120) can obtain the first boundary (511) of the thyroid gland by providing an ultrasound image to a first learning model (310) that has been pre-trained to extract a thyroid region (S410). Specifically, the control unit (120) can input an ultrasound image to the first learning model (310) that has been pre-trained to extract a thyroid region. The first learning model (310) that has received the ultrasound image can process the ultrasound image based on its own parameters (weights, biases, etc.) to distinguish between a thyroid region and a nodule region, and output a first image (510) that includes the first boundary (511) of the thyroid gland.

[0092] Referring to FIG. 5b, the control unit (120) can obtain the second boundary (521) of the nodule by providing an ultrasound image to a second learning model (320) that has been pre-trained to extract a nodule region (S420). Specifically, the control unit (120) can input an ultrasound image to the second learning model (320) that has been pre-trained to extract a nodule region. The second learning model (320) that has received the ultrasound image can process the ultrasound image based on its own parameters (weights, biases, etc.) to distinguish between a thyroid region and a nodule region, and output a second image (520) that includes the second boundary (521) of the nodule.

[0093] The ultrasound image input to the first learning model (310) and the ultrasound image input to the second learning model (320) may be the same. That is, from the same ultrasound image, the first learning model (310) may extract and output only the first boundary of the thyroid gland, and the second learning model (320) may extract and output only the second boundary of the nodule.

[0094] Referring to FIG. 5c, the control unit (120) can output an overlapping image (530) in which the first boundary (511) and the second boundary (521) overlap to enable ETE detection. For example, the control unit (120) can output the second image (520) by overlaying it on the first image (510), but is not limited thereto. What is important is that the first boundary (511) and the second boundary (521) extracted from the same ultrasound image can be displayed together in a single overlapping image (530).

[0095] The control unit (120) can output the superimposed image (530) through the output unit (140). For example, the control unit (120) can transmit the superimposed image (530) to the doctor's terminal or directly display the superimposed image (530) so that the doctor can check it.

[0096] Figure 6 is a drawing showing various positional relationships between the thyroid gland and nodules according to the present invention.

[0097] Figure 6a shows a superimposed image in which the entire nodule is located on the thyroid gland and the nodule does not obscure the border of the thyroid gland.

[0098] Referring to FIG. 6a, the first learning model (310) can output the first boundary of the thyroid gland by distinguishing between the area occupied by the thyroid gland and the area not occupied by the thyroid gland. In other words, the first learning model (310) can distinguish between the area occupied by the thyroid gland (including the area obscured by the nodule) and the background outside the thyroid gland. Accordingly, the control unit (120) can obtain the first boundary that distinguishes the thyroid gland from the background.

[0099] Additionally, the second learning model (320) can output a second boundary by distinguishing between an area occupied by a nodule and an area not occupied by a nodule. Accordingly, the control unit (120) can obtain a second boundary that distinguishes between a nodule and a thyroid area outside the nodule.

[0100] Figure 6b shows a superimposed image in which the entire nodule is located on the thyroid gland and only the border of the nodule obscures the border of the thyroid gland.

[0101] Referring to Fig. 6b, the first learning model (310) can output the first boundary of the thyroid by distinguishing between the area occupied by the thyroid gland and the area not occupied by the thyroid gland. Even though the boundary of the nodule is adjacent to the boundary of the thyroid gland in the ultrasound image and the boundary of the thyroid gland is obscured by the boundary of the nodule, the first learning model (310) can distinguish between the area occupied by the thyroid gland (including the area obscured by the nodule) and the background outside the thyroid gland. Accordingly, the control unit (120) can obtain the first boundary distinguishing between the thyroid gland and the background.

[0102] Additionally, the second learning model (320) can output a second boundary by distinguishing between an area occupied by a nodule and an area not occupied by a nodule. Accordingly, the control unit (120) can obtain a second boundary that distinguishes between a nodule and a thyroid area outside the nodule.

[0103] Figure 6c shows a superimposed image in which part of a nodule is located on the thyroid gland and the area occupied by the nodule (the border of the nodule and the non-border of the nodule) obscures the border of the thyroid gland.

[0104] Referring to FIG. 6c, the first learning model (310) can output the first boundary of the thyroid gland by distinguishing between an area occupied by the thyroid gland and an area not occupied by the thyroid gland. Even though a part of the boundary of the thyroid gland is obscured by a nodule in the ultrasound image, the first learning model (310) can distinguish between the area occupied by the thyroid gland (including the area obscured by the nodule) and the background outside the thyroid gland. In other words, even though a part of the boundary of the thyroid gland in the ultrasound image is obscured by a nodule in the ultrasound image, the control unit (120) can obtain the first boundary in which a part of the boundary is restored through the first learning model (310).

[0105] Additionally, the second learning model (320) can output a second boundary by distinguishing between an area occupied by a nodule and an area not occupied by a nodule. Accordingly, the control unit (120) can obtain a second boundary that distinguishes between a nodule and a thyroid area outside the nodule.

[0106] Meanwhile, the control unit (120) may output one of the state information based on the position of the first boundary and the position of the second boundary in the overlapping image. Specifically, the control unit (120) may output the first state information when the second boundary exists inside the first boundary in the overlapping image, output the second state information when the second boundary and the first boundary in the overlapping image touch, and output the third state information when the second boundary in the overlapping image protrudes outside the first boundary.

[0107] In the overlapping image, the presence of the second boundary within the first boundary may indicate that the entire nodule is located on the thyroid gland and does not obscure the thyroid gland's border. Accordingly, the control unit (120) may output first status information (e.g., "NO" in FIG. 6a) indicating that the entire nodule is located on the thyroid gland and does not obscure the thyroid gland's border. The first status information may be used by a physician to diagnose that ETE has not occurred.

[0108] In the overlapping image, the fact that the second boundary touches the first boundary may mean that the entire nodule is located on the thyroid gland and only the boundary of the nodule obscures the boundary of the thyroid gland. Accordingly, the control unit (120) may output second status information (e.g., “pending” in FIG. 6b) indicating that the entire nodule is located on the thyroid gland and only the boundary of the nodule obscures the boundary of the thyroid gland.

[0109] In the overlapping image, the second boundary protruding outside the first boundary means that part of the nodule is located outside the thyroid gland, another part of the nodule is located on the thyroid gland, and the area occupied by the nodule (the boundary of the nodule and the non-boundary of the nodule) overlaps the boundary of the thyroid gland. Accordingly, the control unit (120) can output third status information (e.g., “Yes” in FIG. 6c) indicating that part of the nodule is located outside the thyroid gland.

[0110] Additionally, the control unit (120) can output the distance between the first boundary and the second boundary. For example, referring to FIG. 6a, the control unit (120) can select a first point on the first boundary of the thyroid gland and a second point on the second boundary of the nodule, which have the closest distance, and output the distance between the first point and the second point. In this case, it can be confirmed how close the nodule is to the boundary of the thyroid gland. For another example, referring to FIG. 6c, the control unit (120) can select a point on the second boundary that protrudes the farthest from the first boundary and output the distance between the selected point and the first boundary. Accordingly, it can be confirmed how far the nodule protrudes from the boundary of the thyroid gland.

[0111] Additionally, the control unit (120) can output the overlap ratio between the thyroid region and the nodule region. For example, if the second boundary exists inside the first boundary, the control unit (120) can output information that the overlap ratio is 100%. For another example, if 30% of the nodule region is located outside the thyroid gland and 70% of the nodule region is located on the thyroid gland, the control unit (120) can output information that the overlap ratio is 70%.

[0112] Figure 7 is a diagram illustrating the effect of the first learning model restoring the boundary of the thyroid gland obscured by a nodule.

[0113] Figure 7a is a diagram assuming that the first learning model does not restore the thyroid boundary. Referring to Figure 7a, since the thyroid boundary obscured by the nodule is not visible, it is impossible to distinguish whether the nodule boundary is contiguous with the thyroid boundary or whether the thyroid boundary protrudes beyond the thyroid boundary.

[0114] Figure 7b is a diagram illustrating a situation in which the first learning model (310) restores the boundary of the thyroid gland, as in the present invention. Referring to Figure 7b, the first learning model (310) restores and provides the boundary even obscured by the nodule. Accordingly, it can be clearly seen that the boundary of the nodule protrudes beyond the boundary of the thyroid gland.

[0115] As previously explained, whether a nodule protrudes beyond the thyroid border, whether the nodule border is adjacent to the thyroid border, or whether the nodule is located within the thyroid border can be crucial in determining the degree (or type) of ETE. Furthermore, according to the present invention, by accurately extracting the thyroid border and the nodule border and then providing them in an overlapping manner, it is possible to enable physicians to make an accurate diagnosis of ETE.

[0116] Furthermore, despite the difficulty in distinguishing between the thyroid and nodules due to the black-and-white nature of ultrasound images, the variable locational relationship between the thyroid and nodules, and the potential for nodules to obscure the thyroid's borders, the method accurately extracts and provides the boundaries of the thyroid and nodules, thereby minimizing misdiagnosis by inexperienced physicians and even experienced physicians. This allows for improved diagnostic accuracy while leveraging the non-invasive nature of ultrasound imaging.

[0117] The present invention described above can be implemented as computer-readable code on a medium having a program recorded thereon. Computer-readable media include all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include hard disk drives (HDDs), solid-state disks (SSDs), silicon disk drives (SDDs), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. In addition, the computer may include a control unit. Accordingly, the above detailed description should not be construed as limiting in any respect, but rather as illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are intended to be included within the scope of the present invention.

Claims

1. A step of obtaining a first boundary of the thyroid by providing an ultrasound image to a first learning model pre-trained to extract a thyroid region, and obtaining a second boundary of the nodule by providing the ultrasound image to a second learning model pre-trained to extract a nodule region; and Including a step of outputting an overlapping image in which the first boundary and the second boundary overlap to enable ETE (Extrathyroidal extension) detection; The above first learning model is, Pre-trained using ultrasound images in which part or all of the nodules are located on the thyroid gland and the nodules cover the border of the thyroid gland as training data and the border images of the thyroid gland as labeling data. ETE detection method.

2. In paragraph 1, The above thyroid boundary image is, It is generated by estimating the border of the thyroid gland obscured by nodules based on prior knowledge of the thyroid gland shape. ETE detection method.

3. In paragraph 1, The step of obtaining the first boundary of the thyroid gland by providing an ultrasound image to a first learning model that has been pre-trained to extract the thyroid region is as follows: A step of obtaining the first boundary, in which a part of the boundary is restored through the first learning model, even though the nodule in the ultrasound image covers a part of the boundary of the thyroid gland in the ultrasound image; ETE detection method.

4. In paragraph 1, The step of outputting an overlapping image in which the first boundary and the second boundary overlap, A step of outputting first state information when the second boundary exists inside the first boundary, outputting second state information when the second boundary and the first boundary touch, and outputting third state information when the second boundary protrudes outside the first boundary; ETE detection method.

5. In paragraph 4, The step of outputting the above overlapping image is: further comprising a step of outputting the distance between the first boundary and the second boundary; ETE detection method.

6. Output section; a communication unit for acquiring ultrasound images; and A control unit that provides an ultrasound image to a first learning model pre-trained to extract a thyroid region to obtain a first boundary of the thyroid, provides the ultrasound image to a second learning model pre-trained to extract a nodule region to obtain a second boundary of the nodule, and outputs an overlapping image in which the first boundary and the second boundary overlap through the output unit so that ETE (Extrathyroidal Extension) detection is possible; The above first learning model is, Pre-trained using ultrasound images in which part or all of the nodules are located on the thyroid gland and the nodules cover the border of the thyroid gland as training data and the border images of the thyroid gland as labeling data. ETE detection device.

7. In paragraph 6, The above thyroid boundary image is, It is generated by estimating the border of the thyroid gland obscured by nodules based on prior knowledge of the thyroid gland shape. ETE detection device.

8. In paragraph 6, The above control unit, Even though the nodule in the ultrasound image covers part of the boundary of the thyroid gland in the ultrasound image, the first boundary is obtained by restoring part of the boundary through the first learning model. ETE detection device.

9. In paragraph 6, The above control unit, If the second boundary exists inside the first boundary, first state information is output, if the second boundary and the first boundary are in contact, second state information is output, and if the second boundary protrudes outside the first boundary, third state information is output. ETE detection device.

10. In paragraph 9, The above control unit, Outputting the distance between the first boundary and the second boundary ETE detection device.

11. A step of obtaining a first boundary of the thyroid by providing an ultrasound image to a first learning model pre-trained to extract a thyroid region, and obtaining a second boundary of the nodule by providing the ultrasound image to a second learning model pre-trained to extract a nodule region; and Including a step of outputting an overlapping image in which the first boundary and the second boundary overlap to enable ETE (Extrathyroidal extension) detection; The above first learning model is, A computer program stored in a non-transitory computer-readable medium for executing an ETE detection method, which is pre-trained using ultrasound images in which a part or all of a nodule is located on the thyroid gland and the nodule obscures the border of the thyroid gland as training data, and a border image of the thyroid gland as labeling data.

Images