Lesion matching method and lesion matching device

Abstract

A lesion matching method is disclosed. The lesion matching method according to the present invention comprises the steps of: acquiring mammography craniocaudal and mediolateral oblique images; using first distance information between a common identifier in the craniocaudal image and a first lesion, and second distance information between the common identifier in the mediolateral oblique image and a second lesion, thereby determining whether the first lesion matches the second lesion; and providing current characteristics of unmatched lesions and past characteristics from previous mammograms.

Description

Lesion matching method and lesion matching device

[0001] This application claims priority to Korean Patent Application No. 10-2024-0003077, filed on January 8, 2024, and all contents disclosed in the specification and drawings of the said application are incorporated herein by reference.

[0002] The present invention relates to a lesion matching method and a lesion matching device that allow matching of lesions in upper-lower images and medial-lateral oblique images generated by mammography and enable follow-up observation of non-matched lesions.

[0003] Mammography is a standard imaging method used to precisely diagnose the size, shape, thickness of the epidermis, and fibrosis of lesions in order to make an early differential diagnosis of malignant tumors or common diseases occurring in the breast area.

[0004] In mammography, each breast is photographed twice, for a total of four breast images. Specifically, cranio-caudal (CC) and mediolateral oblique (MLO) images are taken for the left breast, while cranio-caudal (CC) and mediolateral oblique (MLO) images are taken for the right breast. Hereinafter, the images taken in the cranio-caudal (CC) direction are referred to as superior-inferior images, and the images taken in the mediolateral oblique (MLO) direction are referred to as mediolateral oblique images.

[0005] During mammography, sufficient pressure is applied to the breast to ensure accurate interpretation. This pressure not only makes it difficult to identify the true location of lesions, but also makes it difficult to match lesions seen in superior and inferior images with those seen in mediolateral oblique images.

[0006] For example, referring to Figure 1, superior-inferior (CC) and mediolateral oblique (MLO) images of the left breast are shown. Although the lesions within the rectangular boxes are actually the same lesion, the shooting direction and the shape of the breast at the time of shooting differ between the two images, making it difficult to determine whether the two lesions appearing in the two images (CC, MLO) are the same.

[0007] Additionally, if asymmetry is present, with a lesion appearing only on one of the two images (CC, MLO), the lesion is generally unlikely to be significant (e.g., malignant). However, even in lesions with asymmetry, if the lesion is new or has grown in size compared to the previous examination, such as developing asymmetry or focal asymmetry, continuous follow-up observation is necessary as it may develop into a serious disease.

[0008] The present invention is intended to solve the above-described problems, and the purpose of the present invention is to provide a lesion matching method and a lesion matching device that allow lesions in upper-lower images and internal-external oblique images generated by mammography to be matched with each other and enable follow-up observation of non-matched lesions.

[0009] A lesion matching method according to the present invention comprises the steps of acquiring upper-lower images and mediolateral oblique images of a mammogram, determining whether the first lesion and the second lesion are matched using first distance information between a common identifier and the first lesion in the upper-lower images and second distance information between the common identifier and the second lesion in the mediolateral oblique images, and providing current characteristics of non-matched lesions and past characteristics from past mammograms.

[0010] In this case, the step of providing the upper-lower images and the medial-lateral oblique images to a learned artificial intelligence model and obtaining the area and features of the lesion output by the artificial intelligence model may be further included.

[0011] In this case, the common identifier can be commonly present in the upper-lower image and the inner-outer oblique image and maintain its shape despite breast compression due to mammography.

[0012] In this case, the step of determining whether there is a match includes a step of determining that the first lesion and the second lesion are matched if the difference between the first distance information and the second distance information is less than a threshold value, and the first distance information may indicate the degree to which the first lesion approaches the common identifier within the breast, and the second distance information may indicate the degree to which the second lesion approaches the common identifier within the breast.

[0013] In this case, if the difference between the first distance information and the second distance information is less than a threshold value, the step of determining that the first lesion and the second lesion are matched may include the step of calculating the first distance information by dividing the first distance between the common identifier and the first lesion by the length of a first straight line that starts from the common identifier, passes through the first lesion, and ends at the end of the breast, and the step of calculating the second distance information by dividing the second distance between the common identifier and the second lesion by the length of a second straight line that starts from the common identifier, passes through the second lesion, and ends at the end of the breast.

[0014] Meanwhile, the step of determining whether the matching is made includes a step of determining whether the first lesion and the second lesion surface are matched by further utilizing the similarity between the matching characteristics of the first lesion and the matching characteristics of the second lesion, and the matching characteristics of the first lesion and the matching characteristics of the second lesion may include at least one of mass / calcification information of the lesion, benign / malignant information of the lesion, shape / distribution information of the lesion, characteristic information of an image in which the lesion appears, or grade information of the lesion.

[0015] Meanwhile, the step of determining whether the first lesion and the second lesion are matched may include the step of obtaining the first distance information between the common identifier in the upper-lower image and the first lesion and the third distance information between the common identifier in the upper-lower image and the third lesion, the step of obtaining the second distance information between the common identifier in the mediolateral image and the second lesion and the fourth distance information between the common identifier in the mediolateral image and the fourth lesion, and the step of determining that the first lesion and the fourth lesion are not matched if the difference between the first distance information and the fourth distance information is greater than a threshold value, and the step of determining that the first lesion and the second lesion are matched if the difference between the first distance information and the second distance information is less than a threshold value.

[0016] Meanwhile, in a case where the first lesion and the second lesion are matched, a step of calculating the final diagnosis result value of the lesion by averaging the diagnosis result value of the first lesion and the diagnosis result value of the second lesion may be further included.

[0017] Meanwhile, the current characteristics and past characteristics may include at least one of asymmetry, location of the lesion, type of lesion, shape / distribution information, or size of the lesion.

[0018] Meanwhile, a lesion matching device according to the present invention includes a communication unit that acquires upper and lower mammographic images and medial and lateral oblique images, and a control unit that determines whether the first lesion and the second lesion are matched using first distance information between a common identifier in the upper and lower images and second distance information between the common identifier in the medial and lateral oblique images, and provides current characteristics of non-matched lesions and past characteristics from past mammography.

[0019] In this case, the control unit can provide the upper and lower images and the internal and external oblique images to the learned artificial intelligence model, and obtain the area and features of the lesion output by the artificial intelligence model.

[0020] In this case, the common identifier can be commonly present in the upper-lower images and the inner-outer oblique images and maintain its shape despite breast compression due to mammography.

[0021] In this case, if the difference between the first distance information and the second distance information is less than a threshold value, the control unit determines that the first lesion and the second lesion are matched, and the first distance information may indicate the degree to which the first lesion approached the common identifier within the breast, and the second distance information may indicate the degree to which the second lesion approached the common identifier within the breast.

[0022] In this case, the control unit may divide the first distance between the common identifier and the first lesion by the length of a first straight line that starts from the common identifier, passes through the first lesion, and ends at the end of the breast, thereby calculating the first distance information, and may divide the second distance between the common identifier and the second lesion by the length of a second straight line that starts from the common identifier, passes through the second lesion, and ends at the end of the breast, thereby calculating the second distance information. Meanwhile, the control unit may further use the similarity between the matching characteristics of the first lesion and the matching characteristics of the second lesion to determine whether the first lesion and the second lesion surface are matched, and the matching characteristics of the first lesion and the matching characteristics of the second lesion may include at least one of mass / calcification information of the lesion, benign / malignant information of the lesion, shape / distribution information of the lesion, characteristic information of an image in which the lesion appears, or grade information of the lesion.

[0023] Meanwhile, the control unit may obtain the first distance information between the common identifier in the upper-lower image and the first lesion and the third distance information between the common identifier in the upper-lower image and the third lesion, obtain the second distance information between the common identifier in the mediolateral image and the second lesion and obtain the fourth distance information between the common identifier in the mediolateral image and the fourth lesion, and determine that the first lesion and the fourth lesion do not match if the difference between the first distance information and the fourth distance information is greater than a threshold value, and determine that the first lesion and the second lesion are matched if the difference between the first distance information and the second distance information is less than a threshold value.

[0024] Meanwhile, when the first lesion and the second lesion are matched, the control unit can calculate the final diagnosis result value of the corresponding lesion by averaging the diagnosis result value of the first lesion and the diagnosis result value of the second lesion.

[0025] Meanwhile, the current characteristics and past characteristics may include at least one of asymmetry, location of the lesion, type of lesion, shape / distribution information, or size of the lesion.

[0026] A computer program according to the present invention is stored in a non-transitory computer-readable medium to execute a lesion matching method, including the steps of acquiring upper-lower images and mediolateral oblique images of a mammogram, determining whether the first lesion and the second lesion are matched using first distance information between a common identifier and a first lesion in the upper-lower images and second distance information between the common identifier and a second lesion in the mediolateral oblique images, and providing current characteristics of non-matching lesions and past characteristics from past mammograms.

[0027] Figure 1 is a diagram showing superior-inferior images and mediolateral oblique images.

[0028] Figure 2 is a block diagram illustrating a lesion matching device according to the present invention.

[0029] Figure 3 is a flowchart for explaining a lesion matching method according to the present invention.

[0030] FIG. 4 is a drawing for explaining a method of matching a first lesion and a second lesion according to the present invention.

[0031] FIG. 5 is a diagram illustrating a method for comparing matching characteristics of a first lesion and matching characteristics of a second lesion according to the present invention.

[0032] Figure 6 is a drawing for explaining a method for recording the location of a lesion according to the present invention.

[0033] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components will be given the same reference numbers and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably only for the convenience of writing the specification, and do not in themselves have distinct meanings or roles. In addition, when describing the embodiments disclosed in this specification, if it is determined that a specific description of a related known technology may obscure the gist of the embodiments disclosed in this specification, a detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and technical scope of the present invention.

[0034] Terms that include ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by these terms. These terms are used solely to distinguish one component from another.

[0035] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0036] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprises" or "have" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0037] In implementing the present invention, components may be described in detail for convenience of explanation, but these components may be implemented in one device or module, or one component may be implemented by being divided into multiple devices or modules.

[0038] Figure 2 is a block diagram illustrating a lesion matching device according to the present invention.

[0039] The lesion matching device (100) according to the present invention (hereinafter referred to as “device (100)”) may include a communication unit (110), a control unit (120), and a memory (130). The device (100) may include only some of the components illustrated in FIG. 1, or may include additional components not illustrated in FIG. 1.

[0040] The communication unit (110) can communicate with external devices via one or more wired or wireless networks. To this end, the communication unit (110) may include a communication circuit or communication module for communicating with the external device.

[0041] The communication unit (110) can receive images taken by mammography of the same patient from an external device. Here, the images can include upper-lower view (CC) images and mediolateral oblique (MLO) images of the left breast, and upper-lower view (CC) images and mediolateral oblique (MLO) images of the right breast. In addition, the communication unit (110) can receive images taken by mammography of various patients.

[0042] Additionally, the communication unit (110) can transmit the results of processing the upper-lower image (CC) and the mediolateral oblique image (MLO) by the device (100) to the doctor's terminal. The results processed here are the detection results of lesions appearing in the superior-inferior image (CC) and mediolateral oblique image (MLO), the location of the lesion in the superior-inferior image (CC) and mediolateral oblique image (MLO), the standard location that synthesizes the location of the lesion in the superior-inferior image (CC) and the location of the lesion in the mediolateral oblique image (MLO), the matching result of the lesion in the superior-inferior image (CC) and the lesion in the mediolateral oblique image (MLO) (whether the lesion in the superior-inferior image (CC) matches the lesion in the mediolateral oblique image (MLO), which lesion among several lesions in the superior-inferior image (CC) matches with which lesion among several lesions in the mediolateral oblique image (MLO), which lesion is not matched, etc.), the final result that synthesizes the diagnostic result value for the lesion in the superior-inferior image (CC), the diagnostic result value for the lesion in the mediolateral oblique image (MLO), the diagnostic result value for the lesion in the superior-inferior image (CC), and the diagnostic result value for the lesion in the mediolateral oblique image (MLO). It may include diagnostic results, lesion matching characteristics (lesion mass / calcification information, lesion benign / malignant information, lesion shape / distribution information, image characteristic information in which lesion appears, lesion grade information, etc.), current and past characteristics of lesions (current lesion shape / distribution information, past lesion shape / distribution information, etc.).

[0043] The control unit (120) can control the overall operation of the device (100). The term “control unit” may be used interchangeably with terms such as “microprocessor,” “controller,” “microcontroller,” and “processor.”

[0044] The memory (130) can store programs, commands, and other data for driving the device (100). In addition, the memory (130) can store the results of the device (100) processing the upper-lower image (CC) and the mediolateral oblique image (MLO).

[0045] Meanwhile, the device (100) may further include an output unit (not shown). The output unit (not shown) may, under the control of the control unit (120), provide the user with information stored in or processed by the device (100). To this end, the output unit (not shown) may include a display module that displays information or a communication module that transmits information to a user terminal.

[0046] Additionally, the device (100) may further include an input unit (not shown). The input unit (not shown) may receive various types of information from the user.

[0047] Figure 3 is a flowchart for explaining a lesion matching method according to the present invention.

[0048] The lesion matching method according to the present invention may include a step of acquiring upper-lower images and medial-lateral oblique images of mammography (S310), a step of determining whether the first lesion and the second lesion are matched using first distance information between a common identifier in the upper-lower images and the first lesion and second distance information between a common identifier in the medial-lateral oblique images and the second lesion (S320), a step of calculating a final diagnosis result value of the lesion using the diagnosis result value of the first lesion and the diagnosis result value of the second lesion when the first lesion and the second lesion are matched (S330), and a step of providing the current characteristics of the lesion that is not matched and the past characteristics of the past mammography (S340).

[0049] In relation to S310, the control unit (120) can acquire upper-lower images and mediolateral oblique images of mammography. Specifically, the control unit (120) can acquire four images taken by mammography for the same patient. Hereinafter, the upper-lower images and mediolateral oblique images of the right breast will be described as an example, but the present invention is not limited thereto, and the following description can also be applied to upper-lower images and mediolateral oblique images of the left breast.

[0050] Next, the control unit (120) can determine whether the first lesion and the second lesion are matched using the first distance information between the common identifier and the first lesion in the upper-lower image and the second distance information between the common identifier and the second lesion in the medial-lateral oblique image (S320). This will be described with reference to FIG. 4.

[0051] FIG. 4 is a drawing for explaining a method of matching a first lesion and a second lesion according to the present invention.

[0052] Referring to FIG. 4a, the control unit (120) can extract a first lesion from the upper-lower image (CC). In this case, the control unit (120) can extract an area estimated to be a lesion from the upper-lower image (CC) using a known algorithm. For example, the control unit (120) can extract an area estimated to be a lesion using an object detection model trained using an image of the lesion.

[0053] Referring to FIG. 4a, it can be seen that the control unit (120) extracts the 1-1 lesion (R1) and the 1-2 lesion (R2) from the upper-lower image (CC).

[0054] Referring to FIG. 4b, the control unit (120) can extract a second lesion from the mediolateral oblique (MLO) image. In this case, the control unit (120) can extract an area estimated to be a lesion from the mediolateral oblique (MLO) image using a known algorithm. For example, the control unit (120) can extract an area estimated to be a lesion using an object detection model trained using the lesion image.

[0055] Referring to FIG. 4b, it can be seen that the control unit (120) extracts the 2-1 lesion (R1) and the 2-2 lesion (R2) from the medial and lateral oblique (MLO) image.

[0056] Then, the control unit (120) can determine whether the first lesion in the superior-inferior image (CC) and the second lesion in the medial-lateral oblique image (MLO) are matched. Here, matching may mean that the first lesion in the superior-inferior image (CC) and the second lesion in the medial-lateral oblique image (MLO) are actually the same lesion.

[0057] As explained above, mammography is difficult to accurately match two lesions because it requires sufficient compression of the breast to capture the image.

[0058] Therefore, in the present invention, a point that exists in both the superior-inferior (CC) image and the medial-lateral oblique (MLO) image and that maintains its shape despite breast compression during mammography is used as a common identifier. An example of a common identifier is the nipple.

[0059] In other words, when the breast is compressed, the shape of the breast itself changes, making it difficult to accurately identify the actual location of the lesion. Conversely, the nipple, located at the end of the breast, remains unchanged despite breast compression, making identification and location easy. Therefore, the present invention utilizes a common identifier and the relative distance between the lesions to determine whether two lesions are identical.

[0060] Specifically, the control unit (120) can calculate first distance information between a common identifier and a first lesion in the upper-lower image, and can calculate second distance information between a common identifier and a second lesion in the medial-lateral oblique image.

[0061] Here, the first distance information indicates the degree to which the first lesion approaches the common identifier within the breast, and may represent a relative distance. Furthermore, the second distance information indicates the degree to which the second lesion approaches the common identifier within the breast, and may represent a relative distance.

[0062] Specifically, referring to the upper-lower image (CC) of FIG. 4a, the control unit (120) can calculate the 1-2 distance information by dividing the 1-2 distance (the length of the shorter straight line among the two straight lines in the upper part of FIG. 4a) between the common identifier and the 1-2 lesion (R2) by the length of the 1-2 straight line that starts from the common identifier, passes through the 1-2 lesion (R2), and ends at the end of the breast (the length of the longer straight line among the two straight lines in the upper part of FIG. 4a). Here, the end of the breast may be the point where the breast ends (for example, the point where the pectoral muscle begins).

[0063] Also, referring to the mediolateral oblique image (MLO) of FIG. 4b, the control unit (120) can calculate the 2-2 distance information by dividing the 2-2 distance (the length of the shorter straight line among the two straight lines in the upper part of FIG. 4b) between the common identifier and the 2-2 lesion (R2) by the length of the 2-1 straight line (the length of the longer straight line among the two straight lines in the upper part of FIG. 4b) that starts from the common identifier and passes through the 2-2 lesion (R2) and ends at the end of the breast.

[0064] And if the difference between the first distance information and the second distance information is less than the threshold value, the control unit (120) can determine that the first lesion and the second lesion are matched. In other words, the control unit (120) can determine that the first lesion and the second lesion are actually the same lesion.

[0065] On the other hand, if the difference between the first distance information and the second distance information is greater than the threshold value, the control unit (120) can determine that the first lesion and the second lesion do not match. In other words, the control unit (120) can determine that the first lesion and the second lesion are actually different lesions.

[0066] Even when there are multiple lesions in one image, the control unit (120) can determine whether the lesions in the superior-inferior image and the lesions in the mediolateral oblique image match. For example, referring to FIG. 4, the superior-inferior image (CC) has a 1-1 lesion (R1) and a 1-2 lesion (R2), and the mediolateral oblique image (MLO) has a 2-1 lesion (R1) and a 2-2 lesion (R2). In this case, the control unit (120) can obtain the 1-1 distance information between the common identifier in the superior-inferior image (CC) and the 1-1 lesion, and the 1-2 distance information between the common identifier in the superior-inferior image (CC) and the 1-2 lesion. In addition, the control unit (120) can obtain 2-1 distance information between a common identifier and a 2-1 lesion in the mediolateral oblique image (MLO) and 2-2 distance information between a common identifier and a 2-2 lesion in the mediolateral oblique image.

[0067] In this case, the control unit (120) can determine whether the 1-1 lesion and the 2-1 lesion are matched by using the 1-1 distance information between the common identifier and the 1-1 lesion in the upper-lower image and the 2-1 distance information between the common identifier and the 2-1 lesion in the mediolateral image, and can determine whether the 1-1 lesion and the 2-2 lesion are matched by using the 1-1 distance information between the common identifier and the 1-1 lesion in the upper-lower image and the 2-2 distance information between the common identifier and the 2-2 lesion in the mediolateral image.

[0068] If the difference between the 1-1 distance information and the 2-2 distance information is greater than the threshold value, the control unit (120) can determine that the 1-1 lesion and the 2-2 lesion do not match. If the difference between the 1-1 distance information and the 2-1 distance information is less than the threshold value, the control unit (120) can determine that the 1-1 lesion and the 2-1 lesion match.

[0069] In addition, the control unit (120) can determine whether the 1-2 lesion and the 2-1 lesion are matched using the 1-2 distance information between the common identifier and the 1-2 lesion in the upper-lower images and the 2-1 distance information between the common identifier and the 2-1 lesion in the mediolateral oblique images, and can determine whether the 1-2 lesion and the 2-2 lesion are matched using the 1-2 distance information between the common identifier and the 1-2 lesion in the upper-lower images and the 2-2 distance information between the common identifier and the 2-2 lesion in the mediolateral oblique images.

[0070] If the difference between the 1-2 distance information and the 2-1 distance information is greater than the threshold value, the control unit (120) can determine that the 1-2 lesion and the 2-1 lesion do not match. If the difference between the 1-2 distance information and the 2-2 distance information is less than the threshold value, the control unit (120) can determine that the 1-2 lesion and the 2-2 lesion match.

[0071] For example, assume that lesions A (distance information 0.3), B (distance information 0.4), and C (distance information 0.6) are found in the upper-lower images, and lesions D (distance information 0.32) and E (distance information 0.42) are found in the mediolateral oblique images. The threshold value can be changed through breast density, compression ratio, and user adjustments, but the current threshold value is assumed to be 0.05.

[0072] In this case, the control unit (120) can match lesions A and D as being the same, and lesions B and E as being the same. On the other hand, since there is no lesion matching lesion C, the control unit (120) can designate lesion C as a lesion showing asymmetry.

[0073] Most breast cancers are associated with hormonal imbalances in women, and women with hormonal imbalances may have multiple lesions. Furthermore, the present invention offers the advantage of accurately matching identical lesions even in situations where multiple lesions are present.

[0074] Meanwhile, although distance information has been explained above as meaning relative distance, it is not limited to this, and distance information can also be implemented as absolute distance (distance between common identifier and lesion).

[0075] Meanwhile, the control unit (120) can determine whether the first lesion and the second lesion are matched by further utilizing the similarity between the matching characteristics of the first lesion and the matching characteristics of the second lesion, as well as the distance information between the common identifier and the lesion. Here, the matching characteristics may include at least one of the following: mass / calcification information of the lesion, benign / malignant information of the lesion, shape / distribution information of the lesion, characteristic information of the image in which the lesion appears, or grade information of the lesion.

[0076] The mass / calcification information of a lesion may include information on whether the lesion in the image is a mass or a calcification. Furthermore, the benign / malignant information of a lesion may include information on whether the lesion in the image is benign or malignant.

[0077] The morphology / distribution information of a lesion refers to detailed characteristics that are displayed according to the type of lesion (mass or calcification), and may include margin, shape, distribution, morphology, etc.

[0078] The characteristic information of the image in which the lesion appears refers to the characteristics in which the lesion is displayed within the image, and may include, for example, the average brightness and standard deviation of the area in which the lesion appears in the image (superior-inferior image, mediolateral oblique image).

[0079] Lesion grading information may include lesion grading systems used in clinical practice. An example of such lesion grading system is BIRADS, a breast imaging reporting and data system established by the American College of Radiology (ACR).

[0080] The control unit (120) can obtain the first matching characteristic of the first lesion in the upper-lower image and the second matching characteristic of the second lesion in the mediolateral oblique image using various known algorithms such as a deep learning algorithm.

[0081] FIG. 5 is a diagram illustrating a method for comparing matching characteristics of a first lesion and matching characteristics of a second lesion according to the present invention.

[0082] The control unit (120) can calculate a correlation coefficient between the matching characteristics of the first lesion and the matching characteristics of the second lesion, and can calculate the similarity between the first lesion and the second lesion using the correlation coefficient. In addition, the control unit (120) can determine whether the first lesion and the second lesion are matched using the difference between the first distance information of the first lesion and the second distance information of the second lesion, as described above, and the similarity between the first lesion and the second lesion.

[0083] In the preceding FIG. 4, the 1-1 lesion (R1) in the superior-inferior image (CC) and the 2-1 lesion (R1) in the mediolateral oblique image (MLO) are actually the same lesion, and the 1-1 lesion (R1) in the superior-inferior image (CC) and the 2-2 lesion (R2) in the mediolateral oblique image (CC) are actually different lesions. And referring to FIG. 5a, it can be seen that the similarity between the matching characteristic (RCC_R1) of the 1-1 lesion in the superior-inferior image and the matching characteristic (RMLO_R1) of the 2-1 lesion in the mediolateral oblique image is high, whereas the similarity between the matching characteristic (RCC_R1) of the 1-1 lesion in the superior-inferior image and the matching characteristic (RMLO_R2) of the 2-2 lesion in the mediolateral oblique image is low.

[0084] Referring to Fig. 5b, the 1-1 lesion (R1) in the superior-inferior image (CC) corresponds to calcification and BIRADS 4A grade and shows a regional distribution. In addition, the 2-1 lesion (R1) in the mediolateral oblique image (MLO) also corresponds to calcification and BIRADS 4A grade and shows a regional distribution, whereas the 2-2 lesion (R2) in the mediolateral oblique image (MLO) has a mass, BIRADS grade 2, and an oval shape. In other words, it can be seen that the matching characteristics of the 1-1 lesion (R1) in the superior-inferior image (CC) and the 2-1 lesion (R1) in the mediolateral oblique image (MLO), which are actually the same lesion, are similar to each other.

[0085] Next, when the first lesion and the second lesion are matched, the control unit (120) can calculate the final diagnosis result value of the lesion using the diagnosis result value of the first lesion and the diagnosis result value of the second lesion (S330).

[0086] Specifically, the diagnostic result value of the first lesion and the diagnostic result value of the second lesion may include whether the lesion is a mass, the probability that the lesion is a mass, whether the lesion is calcified, the probability that the lesion is calcified, whether the lesion is benign, the probability that the lesion is benign, whether the lesion is malignant, the probability that the lesion is malignant, the grade of the lesion, etc. In this case, the control unit (120) may synthesize the diagnostic result value of the first lesion and the diagnostic result value of the second lesion to produce the final diagnostic result value of the corresponding lesion.

[0087] In this case, the control unit (120) can calculate the final diagnosis result value of the lesion by averaging the diagnosis result value of the first lesion and the diagnosis result value of the second lesion. For example, if the diagnosis result value of the first lesion is a mass (MASS), benign (Benign) and a benign probability of 30%, and the diagnosis result value of the second lesion is a mass (MASS), benign (Benign) and a benign probability of 54%, the control unit (120) can calculate the final diagnosis result value including a mass (MASS), benign (Benign) and a benign probability of 42%.

[0088] When the final diagnosis result value is calculated, the control unit (120) can transmit the lesion information generated by processing the upper-lower (CC) image and the mediolateral oblique (MLO) image to the doctor's terminal. Here, the lesion information includes the detection results of the lesion appearing in the superior-inferior image (CC) and the mediolateral oblique image (MLO), the location of the lesion in the superior-inferior image (CC) and the mediolateral oblique image (MLO), the standard location that synthesizes the location of the lesion in the superior-inferior image (CC) and the location of the lesion in the mediolateral oblique image (MLO), the matching result of the lesion in the superior-inferior image (CC) and the lesion in the mediolateral oblique image (MLO) (whether the lesion in the superior-inferior image (CC) matches the lesion in the mediolateral oblique image (MLO), which lesion among several lesions in the superior-inferior image (CC) matches with which lesion among several lesions in the mediolateral oblique image (MLO), and which lesion is not matched, etc.), the final result that synthesizes the diagnostic result value for the lesion in the superior-inferior image (CC), the diagnostic result value for the lesion in the mediolateral oblique image (MLO), the diagnostic result value for the lesion in the superior-inferior image (CC), and the diagnostic result value for the lesion in the mediolateral oblique image (MLO). It may include diagnostic results, lesion matching characteristics (lesion mass / calcification information, lesion benign / malignant information, lesion shape / distribution information, lesion characteristic information of the image in which the lesion appears, lesion grade information, etc.).

[0089] In the past, there was a problem in that it was difficult to match lesions appearing in superior-inferior images with lesions appearing in mediolateral oblique images due to different shooting directions and the pressure applied to the breast during shooting. Accordingly, superior-inferior images and mediolateral oblique images were each processed individually to indicate the location or risk of the lesion. Therefore, in the past, it was difficult for doctors to determine whether two lesions were the same, and there was a problem in that the diagnostic results of the two lesions were displayed as different values, which caused confusion in the diagnosis. However, according to the present invention, by matching lesions appearing in superior-inferior images with lesions appearing in mediolateral oblique images, the actual number of lesions can be accurately presented and a guide for accurate diagnosis can be provided. In addition, according to the present invention, there is an advantage in that the stability of the diagnosis can be secured by presenting an integrated diagnostic result value by synthesizing superior-inferior images and mediolateral oblique images.

[0090] Figure 6 is a drawing for explaining a method for recording the location of a lesion according to the present invention.

[0091] The control unit (120) can store lesion information generated by processing the upper-lower (CC) image and the mediolateral oblique (MLO) image.

[0092] In addition, when the first lesion and the second lesion are matched, the control unit (120) can calculate the actual location of the lesion in the breast by synthesizing the location of the first lesion in the upper-lower mammographic image and the location of the second lesion in the medial-lateral oblique image, and store the actual location of the lesion in the breast. In this case, as illustrated in FIG. 6, the control unit (120) can store the area (superior outer, superior inner, inferior outer, inferior inner, subareolar, inverse outer, etc.) to which the location of the lesion belongs.

[0093] Here, the zone may refer to the breast divided into sections on the breast cancer test results sheet, as illustrated in FIG. 6. Furthermore, the control unit (120) may output a breast cancer test results sheet that records the location and information of the lesion. That is, the device (100) of the present invention automatically enters the location and information of the lesion on the breast cancer test results sheet, thereby preventing confusion in writing the location of the lesion and enabling the radiologist to perform the work quickly and conveniently.

[0094] Meanwhile, if a non-matching lesion exists, the control unit (120) can store the location of the non-matching lesion and the lesion information of the lesion. In this case, the control unit (120) can also store the area (superior outer, superior inner, inferior outer, inferior inner, subareolar, reverse outer, etc.) to which the location of the non-matching lesion belongs.

[0095] A mismatched lesion may mean that a lesion matching a lesion in a first image does not exist in a second image. Specifically, a lesion matching a first lesion in a superior-inferior image may not exist in a mediolateral oblique image, or a lesion matching a second lesion in a mediolateral oblique image may not exist in a superior-inferior image.

[0096] For example, a lesion may be detected in the first image, but the detected lesion may not exist in the second image. In another example, a first lesion may be detected in the first image and a second lesion may be detected in the second image, but the difference between the first distance information (distance information between the common identifier in the first image and the first lesion) and the second distance information (distance information between the common identifier in the second image and the second lesion) may be greater than a threshold. In another example, a first lesion may be detected in the first image and a second lesion may be detected in the second image, but the similarity between the matching characteristics of the first lesion and the matching characteristics of the second lesion may be less than a threshold.

[0097] When mismatched lesions exist, asymmetry can be described. This asymmetry is due to the nature of mammography, specifically the fact that imaging is performed from different directions (superiorly and medially and laterally) and under pressure to the breast.

[0098] When asymmetry is present, the lesion is generally unlikely to be significant (e.g., malignant). However, even in lesions with asymmetry, ongoing follow-up is necessary for lesions that are new or have increased in size compared to previous examinations, such as developing asymmetry or focal asymmetry, as these may develop into serious diseases.

[0099] When a mismatched lesion appears like this, the control unit (120) can provide the current characteristics of the mismatched lesion and the past characteristics from past mammography (S340).

[0100] Specifically, assume that four images were acquired in a previous mammography of the same patient, and that a lesion appeared in the upper and lower images of the left breast, and that asymmetry occurred in which the lesion did not appear in the mediolateral oblique images of the left breast. In this case, the control unit (120) can store information that the lesion asymmetry appeared in the previous upper and lower images of the left breast, and the characteristics of the lesion.

[0101] And when upper-lower images of the left breast are acquired through a current mammography for the same patient, the control unit (120) can determine whether the lesion in the current upper-lower image and the lesion in the past upper-lower image are the same. For example, the control unit (120) can determine that the two lesions are the same if the location (or area) of the lesion in the past upper-lower image and the location (or area) of the lesion in the current upper-lower image match.

[0102] Then, the control unit (120) can transmit the current characteristics of the lesion and the past characteristics of the past mammography to the doctor's terminal. Here, the current characteristics and the past characteristics can include at least one of whether the lesion is asymmetry, the location of the lesion, the type of the lesion (mass / calcification information of the lesion, benign / malignant information of the lesion, etc.), shape / distribution information (margin, shape, distribution, morphology, etc.), or the size of the lesion.

[0103] In other words, the physician can receive the past and current characteristics of the lesion in which asymmetry has occurred, and use these characteristics to determine the location of the lesion in which asymmetry has occurred, whether the type of lesion has changed, how the shape / distribution of the lesion has changed, and how the size of the lesion has changed. Accordingly, the physician can reduce the burden of repeatedly reviewing multiple high-resolution images while reviewing past diagnostic history. This has the advantage of reducing the physician's workload, shortening the interpretation time, and preventing physician errors.

[0104] Meanwhile, if a new lesion corresponding to asymmetry appears in the current mammography, the control unit (120) can transmit its current characteristics to the doctor's terminal. Furthermore, for lesions that do not exhibit asymmetry, the control unit (120) can transmit the current and past characteristics of the lesion to the doctor's terminal.

[0105] Meanwhile, the lesion matching method may further include providing superior-inferior and mediolateral oblique images to a trained artificial intelligence model and acquiring the lesion region and features output by the AI ​​model. The AI ​​model may include a lesion region detection model and a lesion feature extraction model. In this regard, the training method for the AI ​​model will first be described.

[0106] A lesion matching device (100) or other learning device can train a lesion area detection model based on image segmentation technology.

[0107] Specifically, the lesion matching device (100) or other learning device can configure a training data set including a mammography image (upper-lower image or medial-lateral oblique image) and a correct mask image. Here, the correct mask image is a binary image in which each pixel is composed of 0 or 1, and pixels within the lesion area can be represented as 1, and pixels within an area other than the lesion area can be represented as 0.

[0108] Then, the lesion matching device (100) or other learning device can provide mammography images (upper-lower images or medial-lateral oblique images) as input data to the lesion region detection model. In this case, the lesion region detection model can process the input data based on its own parameters to output a prediction mask image in which pixels within the lesion region are marked as 1 and pixels within areas other than the lesion region are marked as 0.

[0109] Next, the lesion matching device (100) or other learning device can calculate the difference (Loss) between the predicted mask image and the correct mask image based on a supervised learning algorithm, and update the parameters (weights, bias, etc.) of the lesion area detection model based on the difference (Loss).

[0110] A lesion matching device (100) or other learning device can train a lesion detection model using multiple training data sets. Accordingly, training is performed in a direction that reduces the difference between the predicted mask image output by the lesion detection model and the correct mask image, and the parameters of the lesion detection model can be gradually optimized. Furthermore, the trained lesion detection model can be loaded into the memory (130) of the device (100).

[0111] Meanwhile, a lesion matching device (100) or other learning device can train a lesion feature extraction model.

[0112] Specifically, the lesion matching device (100) or other learning device can configure a training data set including mammography images (upper-lower images or medial-lateral oblique images) and correct answer data.

[0113] To extract various features from mammography images, the correct data can be composed of multiple labels representing multiple lesion characteristics. These multiple features can include the relative brightness of the lesion, the roundness of the lesion border, and calcification components within the lesion that exhibit a certain brightness or higher.

[0114] Then, the lesion matching device (100) or other learning device can provide mammography images (upper-lower images or medial-lateral oblique images) as input data to the lesion feature extraction model. In this case, the lesion feature extraction model can output multiple features of the lesion by processing the input data based on its own parameters.

[0115] Next, the lesion matching device (100) or other learning device can calculate the difference (Loss) between the multiple features output by the lesion feature extraction model and the multiple features of the correct data based on a supervised learning algorithm, and update the parameters (weights, biases, etc.) of the lesion feature extraction model based on the difference (Loss).

[0116] A lesion matching device (100) or other learning device can train a lesion feature extraction model using multiple training data sets. Accordingly, training is performed in a way that reduces the difference between the multiple features output by the lesion feature extraction model and the multiple features of the correct data, and the parameters of the lesion feature extraction model can be gradually optimized. Furthermore, the trained lesion feature extraction model can be loaded into the memory (130) of the device (100).

[0117] When the upper-lower images and the medial-lateral oblique images of the mammography are acquired (S310), the control unit (120) provides the upper-lower images and the medial-lateral oblique images to the learned artificial intelligence model, and can acquire the area and features of the lesion output by the artificial intelligence model.

[0118] Specifically, the control unit (120) can provide the mammography image (at least one of the upper-lower image or the medial-lateral oblique image) acquired in S310 to the lesion region detection model. In this case, the lesion region detection model can output a prediction mask image that marks an area estimated to be a lesion as 1 and an area other than the lesion as 0 based on its own parameters (weight, bias, etc.). In this case, the control unit (120) can extract the area marked as 1 as the lesion area within the mammography image.

[0119] In addition, the control unit (120) can provide the mammography image (at least one of the upper-lower image or the medial-lateral oblique image) acquired in S310 to the lesion feature extraction model. In this case, the lesion feature extraction model can extract multiple features based on its own parameters (weights, biases, etc.). In this case, the control unit (120) can obtain multiple features output by the lesion feature extraction model as multiple features indicated by the lesion in the mammography image.

[0120] The present invention described above can be implemented as computer-readable code on a medium having a program recorded thereon. Computer-readable media include all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include hard disk drives (HDDs), solid-state disks (SSDs), silicon disk drives (SDDs), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. In addition, the computer may include a control unit. Accordingly, the above detailed description should not be construed as limiting in any respect, but rather as illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are intended to be included within the scope of the present invention.

Claims

1. Step of acquiring superior-inferior and medial-lateral oblique images of mammography; A step of determining whether the first lesion and the second lesion are matched by using the first distance information between the common identifier and the first lesion in the upper and lower images and the second distance information between the common identifier and the second lesion in the medial and extra-lateral oblique images; and A step of providing current characteristics of the mismatched lesion and past characteristics from past mammography; Lesion matching method.

2. In paragraph 1, A step of providing the upper and lower images and the medial and lateral oblique images to a learned artificial intelligence model and obtaining the area and features of the lesion output by the artificial intelligence model; Lesion matching method.

3. In paragraph 1, The above common identifier is, Commonly present in the above upper-lower images and the above inner-outer oblique images, and the shape is maintained despite breast compression due to mammography Lesion matching method.

4. In paragraph 3, The step of determining whether the above matches is: A step of determining that the first lesion and the second lesion are matched if the difference between the first distance information and the second distance information is less than a threshold value; The above first distance information indicates the degree to which the first lesion approaches the common identifier within the breast, The above second distance information indicates the degree to which the second lesion approached the common identifier within the breast. Lesion matching method.

5. In paragraph 4, If the difference between the first distance information and the second distance information is less than a threshold value, the step of determining that the first lesion and the second lesion are matched is: A step of calculating the first distance information by dividing the first distance between the common identifier and the first lesion by the length of a first straight line starting from the common identifier, passing through the first lesion, and ending at the end of the breast; and A step of calculating the second distance information by dividing the second distance between the common identifier and the second lesion by the length of a second straight line starting from the common identifier, passing through the second lesion, and ending at the end of the breast; Lesion matching method.

6. In paragraph 1, The step of determining whether the above matches is: A step of determining whether the first lesion and the second lesion surface are matched by further utilizing the similarity between the matching characteristics of the first lesion and the matching characteristics of the second lesion; including; The matching characteristics of the first lesion and the matching characteristics of the second lesion are, Contains at least one of the following: mass / calcification information of the lesion, benign / malignant information of the lesion, shape / distribution information of the lesion, characteristic information of the image in which the lesion appears, or grade information of the lesion. Lesion matching method.

7. In paragraph 1, The step of determining whether the first lesion and the second lesion match is: A step of obtaining first distance information between the common identifier in the upper and lower images and the first lesion and third distance information between the common identifier in the upper and lower images and the third lesion; A step of obtaining second distance information between a common identifier in the above-mentioned intra- and extra-ocular images and fourth distance information between a common identifier in the above-mentioned extra-ocular images and a fourth lesion; A step of determining that the first lesion and the fourth lesion are not matched if the difference between the first distance information and the fourth distance information is greater than a threshold value, and determining that the first lesion and the second lesion are matched if the difference between the first distance information and the second distance information is less than a threshold value; including Lesion matching method.

8. In paragraph 1, In case the first lesion and the second lesion are matched, a step of calculating the final diagnosis result value of the corresponding lesion by averaging the diagnosis result value of the first lesion and the diagnosis result value of the second lesion is further included. Lesion matching method.

9. In paragraph 1, The above present and past characteristics are, Whether it corresponds to asymmetry, at least one of the following: location of the lesion, type of lesion, shape / distribution information, or size of the lesion. Lesion matching method.

10. A communication unit for obtaining superior-inferior and medial-lateral oblique images of mammography; and A control unit that determines whether the first lesion and the second lesion are matched by using the first distance information between the common identifier and the first lesion in the upper and lower images and the second distance information between the common identifier and the second lesion in the medial and lateral oblique images, and provides the current characteristics of the lesion that is not matched and the past characteristics in the past mammography; Lesion matching device.

11. In paragraph 10, The above control unit, The above upper-lower images and medial-lateral oblique images are provided to a learned artificial intelligence model, and the area and features of the lesion output by the artificial intelligence model are obtained. Lesion matching device.

12. In paragraph 10, The above common identifier is, Commonly present in the above upper-lower images and the above inner-outer oblique images, and the shape is maintained despite breast compression due to mammography Lesion matching device.

13. In paragraph 12, The above control unit, If the difference between the first distance information and the second distance information is less than the threshold value, it is determined that the first lesion and the second lesion are matched. The above first distance information indicates the degree to which the first lesion approaches the common identifier within the breast, The above second distance information indicates the degree to which the second lesion approached the common identifier within the breast. Lesion matching device.

14. In paragraph 13, The above control unit, The first distance between the common identifier and the first lesion is divided by the length of the first straight line starting from the common identifier, passing through the first lesion, and ending at the end of the breast to derive the first distance information. The second distance between the common identifier and the second lesion is divided by the length of a second straight line starting from the common identifier, passing through the second lesion, and ending at the end of the breast to derive the second distance information. Lesion matching device.

15. In paragraph 10, The above control unit, Further using the similarity between the matching characteristics of the first lesion and the matching characteristics of the second lesion, it is determined whether the first lesion and the second lesion surface are matched, The matching characteristics of the first lesion and the matching characteristics of the second lesion are, Contains at least one of the following: mass / calcification information of the lesion, benign / malignant information of the lesion, shape / distribution information of the lesion, characteristic information of the image in which the lesion appears, or grade information of the lesion. Lesion matching device.

16. In paragraph 10, The above control unit, Obtain the first distance information between the common identifier in the upper and lower images and the first lesion and the third distance information between the common identifier in the upper and lower images and the third lesion, Obtain the second distance information between the common identifier in the above-mentioned internal and external oblique images and the second lesion, and the fourth distance information between the common identifier in the above-mentioned internal and external oblique images and the fourth lesion, If the difference between the first distance information and the fourth distance information is greater than the threshold value, it is determined that the first lesion and the fourth lesion do not match, and if the difference between the first distance information and the second distance information is less than the threshold value, it is determined that the first lesion and the second lesion match. Lesion matching device.

17. In paragraph 10, The above control unit, When the first lesion and the second lesion are matched, the diagnostic result value of the first lesion and the diagnostic result value of the second lesion are averaged to produce the final diagnostic result value of the corresponding lesion. Lesion matching device.

18. In paragraph 10, The above present and past characteristics are, Whether it corresponds to asymmetry, at least one of the following: location of the lesion, type of lesion, shape / distribution information, or size of the lesion. Lesion matching device.

19. Step of acquiring superior-inferior and medial-lateral oblique images of mammography; A step of determining whether the first lesion and the second lesion are matched by using the first distance information between the common identifier and the first lesion in the upper and lower images and the second distance information between the common identifier and the second lesion in the medial and lateral oblique images; and A computer program stored on a non-transitory computer-readable medium for executing a lesion matching method, comprising the steps of: providing current characteristics of unmatched lesions and historical characteristics from past mammograms;

Images