Artificial Intelligence-based Method and Device for Detecting Colorectal Lesions

Abstract

To provide large-intestine lesion detection method and apparatus that allow even less skilled large intestine scopists to accurately detect thin, flat planar polyps on large intestine mucous membranes.SOLUTION: An artificial intelligence-based large-intestine lesion detection method disclosed in the present invention is executed by an apparatus, and includes: receiving an image captured by an endoscope in real time; recognizing an image of each section including large intestine mucous membranes and blood vessels in the received image through a deep learning model; and displaying a first visual effect indicating a blood vessel image in which a large intestine blood vessel is disconnected in an image of each recognized section; wherein the deep learning model is a model that is machine-learned based on blood vessel data in a plurality of large intestine images acquired from an external annotator, a degree to which the blood vessel image is disconnected, and a blood vessel pattern.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to a method and apparatus for detecting colorectal polyps. More specifically, the present invention relates to a method and apparatus for detecting colorectal polyps by vascular learning of an artificial intelligence board.

Background Art

[0002] In recent years, due to Western-style eating habits and lack of exercise, the incidence of colorectal cancer in Korea has increased rapidly ing.

[0003] 80-90% of colorectal cancers start from polyps (adenomas), which are small masses formed in the large intestine .

[0004] According to a certain statistic, there is also a report that if this polyp is detected and removed early by colonoscopy, the mortality rate due to colorectal cancer can be reduced to 66%.

[0005] Medical professionals diagnose the diseases of the examinees through various examinations and present treatment methods.

[0006] In some cases, for more accurate diagnosis, pathologists read the radiographic images and tissue slides scanned pathological images.

[0007] However, it belongs to a very difficult task to find abnormal sites of 100×100 pixel size from an image of 100,000×100,000 pixel size by human eyes. Even an experienced doctor who has undergone a professional training process, it is not easy to distinguish tumor tissue from normal tissue with the naked eye, and it may take several tens of minutes to several hours for analysis depending on the image

[0008] Moreover, situations where abnormal cases are missed often occur. .

[0009] In addition, situations where abnormal cases are missed often occur.

[0010] The large intestine is long and highly convoluted, with many wrinkles, so it is very difficult for a skilled doctor to perfectly observe the inner wall surface of the large intestine. Especially, thin and flat film planar polyps existing on the large intestine mucosa are not easily discovered at an ordinary skill level, and minute changes in the large intestine mucosa and cancer cells having a size of around 1 mm are difficult to distinguish with the naked eye.

[0011] Moreover, due to the operation of confirmation with the human eye, there is a possibility of human error, and often different diagnoses are made for the same examinee. Considering a lot of information that a specialist has to consider within a limited time, the possibility of misdiagnosis cannot be completely eliminated. Typically, as the cause of cancer misdiagnosis damage, it is most well-known that the response ratio of overlooking additional examinations or having a reading error is the highest.

[0012] In addition, the smaller or thinner the polyp is, the higher the overlooking rate tends to be, and cases where colorectal cancer is diagnosed within several years after receiving a normal diagnosis by a colonoscope often occur. Also, depending on elements such as the resolution, contrast ratio, and brightness for support on the image reading monitor of the colonoscope, the possibility that the image cannot be seen properly cannot be completely eliminated.

[0013] The image reading monitor of the colonoscope requires elements such as a high resolution under at least 8-bit conditions.

[0014]

[0015]

[0016]

[0017] ​​​​​​​​​Although it is essential to support, it is an expensive piece of equipment in reality, so there are limitations in its purchase and use. There was such a limit.

[0018] For the purpose of overcoming such limitations and more efficiently interpreting medical images, in recent years, artificial intelligence (AI) technologies such as deep learning have been introduced into the field of diagnosis using medical images.

[0019] Artificial intelligence (AI) technologies based on machine learning such as deep learning are the basis for making a leap in accurately diagnosing the diseases of the examinee using medical images. They are the foundation for bringing about a revolutionary development.

[0020] Therefore, the inventor uses artificial intelligence to recognize the images of each section containing the large intestine mucosa and large intestine blood vessels in the images of the colon endoscope, and generates a deep learning model for the presence or absence of interruption of the large intestine blood vessel image, so that even an examiner of a colon endoscope with low proficiency can accurately detect a thin and flat film planar polyp existing on the large intestine mucosa. As a result, the inventor has invented a method and device for detecting colon polyps based on vascular learning of an artificial intelligence foundation.

Summary of the Invention

Problems to be Solved by the Invention

[0021] The problem to be solved by the present disclosure is to use artificial intelligence that has learned the shape of the colon vascular structure to detect the appearance that the blood vessel image visible between the large intestine mucosae is interrupted centered on the displayed line, and to be able to discriminate it as a suspected lesion. It is to provide a method for detecting colon polyps based on vascular learning of an artificial intelligence foundation.

[0022] Another object of the present invention is to provide a colorectal polyp detection device based on vascular learning of an artificial intelligence platform for achieving the above object.

Means for Solving the Problems

[0023] A method for detecting colorectal polyps by vascular learning of an artificial intelligence platform according to one aspect of the present invention for solving the above-described problems is a method executed by a device, and includes: (a) receiving, in real time, an image captured from an endoscope inserted into the large intestine of a subject; (b) recognizing, in the image, an image of each section including the large intestine mucosa and large intestine blood vessels; (c) determining, for each image of each section, whether there is a break in the large intestine blood vessel image; (d) displaying, in the image of each section, a first visual effect indicating a blood vessel image in which the large intestine blood vessels are interrupted; and (e) displaying, in the image of each section, a second visual effect indicating a blood vessel image in which the large intestine blood vessels are continuous. The step (b) recognizes the image of each section by a deep learning model, and the deep learning model is a model machine-learned based on blood vessel data in a plurality of large intestine images of the subject obtained from an external annotator and the degree of interruption of the blood vessel image and the pattern of the blood vessels due to light irradiated inside the large intestine. (c)

[0024] The first visual effect in the method for detecting colorectal polyps by vascular learning of an artificial intelligence platform according to one aspect of the present invention for solving the above-described problems includes a visual effect in which markers are respectively displayed on the blood vessel images in which the large intestine blood vessels are interrupted in the images of the respective sections. ​​​​​​​​​​​​​​

[0025] According to one aspect of the present invention for solving the above problems, by vascular learning on an artificial intelligence basis In the method for detecting a colorectal polyp, the size of each of the markers is determined based on the degree to which the corresponding blood vessel image is interrupted and is characterized in that.

[0026] According to one aspect of the present invention for solving the above problems, by vascular learning on an artificial intelligence basis The method for detecting a colorectal polyp is such that the control unit determines the presence or absence and size of a thin-film planar polyp on the colorectal mucosa by the first visual effect, and determines the fact that there is no thin-film planar polyp on the colorectal mucosa by the second visual effect, and is characterized in that.

[0027] According to one aspect of the present invention for solving the above problems, by vascular learning on an artificial intelligence basis In the step (c) of the method for detecting a colorectal polyp, the presence or absence of interruption of the colorectal blood vessel image is determined by whether the degree of interruption of the corresponding blood vessel image changes to be equal to or higher than a preset percentage threshold, and when the degree of interruption of the blood vessel image is equal to or higher than the preset percentage threshold, the control unit determines that there is a thin-film planar polyp in the corresponding region on the colorectal mucosa and is characterized in that.

[0028] According to one aspect of the present invention for solving the above problems, by vascular learning on an artificial intelligence basis The method for detecting a colorectal polyp is performed in combination with a computer which is hardware, and is characterized in that it is stored as a computer program in a computer-readable recording medium.

[0029] According to another aspect of the present invention for solving the other problems, by vascular learning on an artificial intelligence basis ​​The colorectal polyp detection device includes a display unit and a colonoscope inserted into the large intestine of the examinee. A communication unit that receives the captured images in real time from the colonoscope, and a storage unit that stores a deep learning model for recognizing colorectal blood vessels in the received images and the previously received images. The control unit recognizes the images of the respective sections including the colorectal mucosa and colorectal blood vessels in the received images by the deep learning model, and displays a first visual effect showing a vascular image with interrupted colorectal blood vessels on the display unit. At the same time, a control unit that displays a second visual effect showing a continuous vascular image of colorectal blood vessels on the display unit in the images of the respective sections is included. The deep learning model is a model that is machine-learned based on the blood vessel data in a plurality of colorectal images of the examinee obtained from an external annotator, the degree to which the vascular image due to the light irradiated inside the large intestine is interrupted, and the pattern of the blood vessels. The first visual effect of the colorectal polyp detection device according to another aspect of the present invention for solving the other problems, based on the vascular learning of the artificial intelligence platform, includes a visual effect in which respective markers are displayed on the corresponding vascular images where the colorectal blood vessels in the images of the respective sections are interrupted. The size of each of the markers is determined based on the degree to which the corresponding vascular image is interrupted. The control unit according to another aspect of the present invention for solving the other problems, based on the vascular learning of the artificial intelligence platform, determines the presence and size of thin film planar polyps on the colorectal mucosa by the first visual effect, and determines the presence and size of

[0030] thick film polyps on the colorectal mucosa by the second visual effect.

[0031] The control unit according to another aspect of the present invention for solving the other problems, based on the vascular learning of the artificial intelligence platform, determines the presence and size of thin film planar polyps on the colorectal mucosa by the first visual effect, and at the same time, determines the presence and size of thick film polyps on the colorectal mucosa by the second visual effect. ​​​​​​​It is characterized by determining the fact that there is no thin-film planar polyp on the intestinal mucosa.

[0032] Regarding another aspect of the present invention for solving the above other problems, by artificial intelligence-based vascular learning The control unit of the colorectal polyp detection device determines whether there is a break in the colorectal blood vessel image by whether the degree of interruption of the corresponding blood vessel image changes to be equal to or higher than a preset percentage threshold. If it is equal to or higher than the preset percentage threshold, it is determined that there is a thin-film planar polyp in the corresponding area on the colorectal mucosa.

[0033] Other specific matters of the present invention are included in the detailed description and drawings.

Advantages of the Invention

[0034] According to the present invention, when interpreting colorectal diseases based on the images of a colonoscope, even an examiner of a colonoscope with low proficiency can detect and excise polyps with high accuracy for abnormal colorectal lesions in which thin and flat-shaped thin-film planar polyps existing on the colorectal mucosa are hidden.

[0035] The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by ordinary technicians from the following description.

Brief Description of the Drawings

[0036]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Mode for Carrying Out the Invention

[0037] The advantages, features, and methods for achieving them of the present invention will be described in detail below with reference to the accompanying drawings. It will become clear by referring to the embodiments described below. However, the present invention is not limited to the embodiments disclosed below, and can be embodied in various different forms. However, these embodiments are provided to make the disclosure of the present invention complete and to enable those of ordinary skill in the technical field to which the present invention pertains to fully understand the scope of the present invention. The present invention is only defined by the scope of the claims. The terms used in this specification are for the purpose of explaining the embodiments and are not intended to limit the present invention. In this specification, the singular form includes the plural form unless otherwise specifically mentioned. As used in the specification, "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components in addition to the recited components. Throughout the specification, the same reference numerals indicate the same components, and "and" is used as follows.

[0038] The terms used in this specification are for the purpose of explaining the embodiments and are not intended to limit the present invention. In this specification, the singular form includes the plural form unless otherwise specifically mentioned. In this specification, the singular form includes the plural form unless otherwise specifically mentioned. In the specification, "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components in addition to the recited components. Throughout the specification, the same reference numerals indicate the same components, and "and" is used as follows. Throughout the specification, the same reference numerals indicate the same components, and "and" " / or" includes each and all combinations of the recited components. For example, even if terms such as "first", "second", etc. are used to describe various components, these components are of course not limited by these terms. These terms are merely used to distinguish one component from another. Thus, it goes without saying that the first component mentioned below can also be the second component within the technical concept of the present invention.

[0039] Unless otherwise defined, all terms (including technical and scientific terms) used in this specification are used as meanings commonly understood by those skilled in the art to which the present invention pertains. Also, terms defined in commonly used dictionaries are not ideally or overly interpreted unless

[0040] spatially relative terms such as "below", "beneath", "lower", "above", "upper", etc. can be used to easily describe the correlation between one component and another as shown in the drawings. Spatially relative terms should be understood as terms including different directions of components during use or operation in addition to the illustrated directions. For example, when the illustrated component is turned over, a component described as "below" or "beneath" another component can be placed "above" another component. Thus, the exemplary term "below" can include both the directions of below and above. Since the component can be oriented in other directions Opposite terms can be interpreted according to the orientation.

[0041] Throughout the specification of the present invention, the same reference numerals denote the same components. This specification does not describe all elements of the embodiments, and general contents in the technical field to which the present invention pertains, or overlapping contents among the embodiments, are omitted. The terms "section, module, member, block" used in the specification can be embodied by software or hardware, and depending on the embodiments, a plurality of "sections, modules, members, blocks" may be embodied as one constituent element, or one "section, module, member, block" may include a plurality of constituent elements.

[0042] Also, when a certain part "includes" a certain constituent element, this does not exclude other constituent elements unless otherwise stated, and means that other constituent elements can be further included.

[0043] At each stage, the identification codes are used for convenience of explanation, and the identification codes do not explain the order of each stage. Unless each stage is clearly described in a specific order in the context, it can be implemented in an order different from the specified order.

[0044] In the present invention, the control unit 240 is a processor, a controller, a microcontroller, a microcomputer, etc., and overall controls the organic operations of the communication unit 210, the display unit 220, and the storage unit 230, and various Means a component that performs judgment and calculation, and can be implemented by hardware or firmware Firmware or software, or a combination thereof.

[0045] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0046] FIG. 1 is a block diagram of a system including a large intestine polyp detection device based on artificial intelligence-based vascular learning according to an embodiment of the present invention, including a large intestine endoscope 100 and a large intestine polyp detection device 2 00. 00.

[0047] The large intestine endoscope 100 includes a camera 110 and illumination 120, and the large intestine polyp detection device 200 includes a communication unit 210, a display unit 220, a storage unit 230, and a control unit 240.

[0048] FIG. 2 is a block diagram schematically showing that a deep learning model used for detecting a large intestine polyp based on artificial intelligence-based vascular learning according to an embodiment of the present invention is learned is learned. diagram.

[0049] FIG. 3 is an exemplary diagram showing that the large intestine mucosa with a thin film plate-shaped polyp hidden in the captured endoscope image is displayed for each of four cases according to an embodiment of the present invention. is displayed.

[0050] FIG. 4 is an exemplary diagram showing that display lines are additionally displayed in the exemplary diagrams for each of the four cases shown in FIG. 3 is shown.

[0051] FIG. 5 is a cross-sectional view of the large intestine wall explaining the difference in the degree to which blood vessels can be seen according to an embodiment of the present invention is shown.

[0052] FIG. 6 is related to another embodiment of the present invention, and shows the detection of large intestine polyps by artificial intelligence-based vascular learning It is a sequence diagram explaining the overall operation of the method.

[0053] Referring to FIGS. 1 to 6, the organic operation of a method for detecting colorectal polyps based on vascular learning of an artificial intelligence infrastructure according to an embodiment of the present invention will be described in detail as follows. When the organic operation of the method for detecting colorectal polyps is described in detail, it is as follows.

[0054] Deep learning refers to a machine learning method based on an artificial neural network that mimics human biological neurons so that a machine can learn. It means a machine learning method based on an artificial neural network that mimics human biological neurons so that a machine can learn. etwork) that mimics human biological neurons so that a machine can learn.

[0055] In deep learning technology, learning data is repeatedly learned to form a diagnostic model for diagnosing diseases. However, since the types of diseases used as learning data are diverse, it is important to develop a diagnostic model specialized for each disease. In deep learning technology, learning data is repeatedly learned to form a diagnostic model for diagnosing diseases. However, since the types of diseases used as learning data are diverse, it is important to develop a diagnostic model specialized for each disease. Therefore, in the present disclosure, an image of the large intestine is analyzed, and a deep learning algorithm that recognizes polyps is used by displaying a location where the blood vessel image suddenly breaks off as a location where there may be a thin and flat-shaped thin film planar polyp.

[0056] Therefore, in the present disclosure, an image of the large intestine is analyzed, and a location where the blood vessel image suddenly breaks off is displayed as a location where there may be a thin and flat-shaped thin film planar polyp, and a deep learning algorithm that recognizes polyps is used. Therefore, in the present disclosure, an image of the large intestine is analyzed, and a location where the blood vessel image suddenly breaks off is displayed as a location where there may be a thin and flat-shaped thin film planar polyp, and a deep learning algorithm that recognizes polyps is used. Therefore, in the present disclosure, an image of the large intestine is analyzed, and a location where the blood vessel image suddenly breaks off is displayed as a location where there may be a thin and flat-shaped thin film planar polyp, and a deep learning algorithm that recognizes polyps is used.

[0057] In the deep learning algorithm of the present invention, artificial intelligence (AI) that has learned the shape of the colon vascular structure detects a state in which the image of the blood vessels located in the lower layer of the large intestine mucosa breaks off centered on a display line that is a kind of marker, and discriminates it as a suspected lesion. In the deep learning algorithm of the present invention, artificial intelligence (AI) that has learned the shape of the colon vascular structure detects a state in which the image of the blood vessels located in the lower layer of the large intestine mucosa breaks off centered on a display line that is a kind of marker, and discriminates it as a suspected lesion. In the deep learning algorithm of the present invention, artificial intelligence (AI) that has learned the shape of the colon vascular structure detects a state in which the image of the blood vessels located in the lower layer of the large intestine mucosa breaks off centered on a display line that is a kind of marker, and discriminates it as a suspected lesion. That is, when performing a colonoscopy, the colorectal polyp detection device 200 uses the image of the colonoscope to

[0058] That is, when performing a colonoscopy, the colorectal polyp detection device 200 uses the image of the colonoscope to Recognize an image of a severed blood vessel centered on the indication line and provide the corresponding image to the examiner of the colonoscope. to provide.

[0059] This enables easy detection of a thin and flat film plane type polyp existing on the colon mucosa. to be easily discovered.

[0060] The colon polyp detection device 200 varies the visual effect according to the presence or absence of discontinuity in the image of the blood vessel centered on the indication line recognized in the image of the colonoscope, and displays it. When the visibility of the blood vessel is high, instead of confirming only a part of the blood vessel located in the lower layer of the colon mucosa and proceeding to the next step, it guides so that the entire blood vessel can be confirmed. By varying the visual effect according to the presence or absence of discontinuity in the image of the blood vessel centered on the indication line recognized in the image of the colonoscope and displaying it, when the visibility of the blood vessel is high, instead of confirming only a part of the blood vessel located in the lower layer of the colon mucosa and proceeding to the next step, it guides so that the entire blood vessel can be confirmed. When the visibility of the blood vessel is high, instead of confirming only a part of the blood vessel located in the lower layer of the colon mucosa and proceeding to the next step, it guides so that the entire blood vessel can be confirmed. to proceed, it guides so that the entire blood vessel can be confirmed.

[0061] This enables clear determination of the presence or absence of a polyp existing in a thin and flat shape on the colon mucosa. to be able to do so.

[0062] Referring to FIG. 1, the colonoscope 100 is a device inserted into the colon to observe the living tissue in the colon, and includes a camera 110, a lighting 120, etc. which includes a camera 110, a lighting 120, etc.

[0063] In addition, the colon polyp detection device 200 includes a communication unit 210, a display unit 220, a storage unit 230 and a control unit 240.

[0064] The communication unit 210 can include one or more modules that enable wireless communication between the colon polyp detection device 200 and a wireless communication system, between the colon polyp detection device 200 and the endoscope 100, or between the colon polyp detection device 200 and an external device (not shown).

[0065] The communication unit 210 receives the taken endoscope An endoscope that can receive mirror images in real time and is inserted into the large intestines of multiple examinees Receive images taken from the endoscope 100, or receive images taken from the endoscope 100 that has been inserted into the large intestine of one examinee multiple times is possible.

[0066] The communication unit 210, for the training of the deep learning model 231, receives, for images of the large intestine of an examinee taken by the endoscope 100, an annotator r, as an example, vascular data located at the lower part of the large intestine mucosa obtained from medical staff can be received.

[0067] The display unit 220 can be embodied as a touch screen by forming a layer structure with the touch sensor or being integrally formed with it.

[0068] Such a touch screen provides an input interface between the large intestine polyp detection device 200 and the user, and at the same time provides an output interface between the large intestine polyp detection device 200 and the user.

[0069] The display unit 220 displays various information generated by the control unit 240 and provides it to the user and at the same time can receive input of various information from the user.

[0070] More specifically, the display unit 220 can display a first visual effect showing a vascular image in which the large intestine blood vessels located at the lower part of the large intestine mucosa are interrupted, for each section of the endoscope image received from the communication unit 210 is possible.

[0071] Also, the display unit 220 can display a second visual effect showing a continuous vascular image in which the large intestine blood vessels are not interrupted, in the image of each section. ​​​​

[0072] The storage unit 230 can store information that supports various functions of the large intestine polyp detection device 200. It can.

[0073] The storage unit 230 can store a number of application programs (application program or application (appl ication)), data for the operation of the large intestine polyp detection device 200, and instruction words. It can store. It can.

[0074] At least some of these application programs can be downloaded from an external server (not shown) via wireless communication. Also, at least some of these application programs rams can exist for the basic functions of the large intestine polyp detection device 200. rams can exist for the basic functions of the large intestine polyp detection device 200. It can exist.

[0075] On the other hand, the application program is stored in the storage unit 230, installed on the large intestine polyp detection device 200, and can be driven by the control unit 240 to perform the operation (or function) of the large intestine polyp detection device 200. operation (or function) of the large intestine polyp detection device 200.

[0076] The storage unit 230 can store a deep learning model 231 for recognizing blood vessels located in the lower part of the large intestine mucosa in the images taken from the colonoscope 100 inserted into the large intestine of the subject. It can store a deep learning model 231 for recognizing blood vessels located in the lower part of the large intestine mucosa in the images taken from the colonoscope 100 inserted into the large intestine of the subject. It can store.

[0077] Here, the deep learning model 231 can include a convolutional neural network (CN N, convolutional neural network, hereinafter referred to as CNN), but is not necessarily limited to this, and can be formed by neural networks of various structures. It can include, but is not necessarily limited to this, and can be formed by neural networks of various structures.

[0078] The CNN can be formed in a structure that repeatedly alternates a convolution layer that applies multiple filters to each region of an image to create a feature map and a pooling layer that can extract features invariant to changes in position and rotation by spatially integrating the feature maps a plurality of times. e Map) and a pooling layer (Pooling Layer) that spatially integrates the feature maps so as to be able to extract features invariant to changes in position and rotation. By doing so, features at various levels can be extracted, from low-level features such as points, lines, and planes to complex and meaningful high-level features. The convolution layer can obtain a feature map by taking the inner product of a filter and a local receptive field for each patch of the input image and applying a non-linear activation function. Compared with other network structures, the CNN can have the feature of using filters with sparse connectivity and shared weights.

[0079] Such a connection structure reduces the number of parameters to be learned, makes the learning by the backpropagation algorithm efficient, and as a result, can improve the prediction performance. Thus, the features finally extracted by repeating the convolution layer and the pooling layer can be used by a multi-layer perceptron (MLP: Multi-Layer Perceptron) or a s

[0080] For each patch of the input image, the convolution layer can obtain a feature map by taking the inner product of a filter and a local receptive field and applying a non-linear activation function. eceptive Field) and applying a non-linear activation function (Activation F unction). be done.

[0081] Compared with other network structures, the CNN can have the feature of using filters with sparse connectivity (Sparse Conne ctivity) and shared weights. This feature can reduce the number of parameters to be learned, make the learning by the backpropagation algorithm efficient, and as a result, improve the prediction performance.

[0082] Such a connection structure reduces the number of parameters to be learned, makes the learning by the backpropagation algorithm efficient, and as a result, can improve the prediction performance. This can improve the prediction performance.

[0083] Thus, the features finally extracted by repeating the convolution layer and the pooling layer can be used by a multi-layer perceptron (MLP: Multi-Layer Perceptron) or a s MLP:Multi-Layer Perceptron) or a s A classification model such as a port vector machine (SVM: Support Vector Machine) is coupled to the shape of a fully-connected layer and can be used for learning and prediction of the classification model. A classification model such as a port vector machine (SVM: Support Vector Machine) is coupled to the shape of a fully-connected layer and can be used for learning and prediction of the classification model. A classification model such as a port vector machine (SVM: Support Vector Machine) is coupled to the shape of a fully-connected layer and can be used for learning and prediction of the classification model.

[0084] The storage unit 230 can store the images of the colonoscope received by the communication unit 210.

[0085] In addition, the storage unit 230 can store images taken from the endoscope 100 inserted into the large intestine of a plurality of examinees, or images taken from the endoscope 100 inserted into the large intestine of one examinee multiple times. In addition, the storage unit 230 can store images taken from the endoscope 100 inserted into the large intestine of a plurality of examinees, or images taken from the endoscope 100 inserted into the large intestine of one examinee multiple times. In addition, the storage unit 230 can store images taken from the endoscope 100 inserted into the large intestine of a plurality of examinees, or images taken from the endoscope 100 inserted into the large intestine of one examinee multiple times.

[0086] Alternatively, the storage unit 230 can store, as an example of an annotator, vascular data obtained from medical staff for the images of the large intestine of the examinee taken by the endoscope 100 for the training of the deep learning model 231. Alternatively, the storage unit 230 can store, as an example of an annotator, vascular data obtained from medical staff for the images of the large intestine of the examinee taken by the endoscope 100 for the training of the deep learning model 231. Alternatively, the storage unit 230 can store, as an example of an annotator, vascular data obtained from medical staff for the images of the large intestine of the examinee taken by the endoscope 100 for the training of the deep learning model 231.

[0087] The control unit 240 can generally control the overall operation of the colon polyp detection device 200 in addition to operations related to the application program. The control unit 240 can generally control the overall operation of the colon polyp detection device 200 in addition to operations related to the application program.

[0088] The control unit 240 processes signals, data, information, etc. input or output by the above-described components, or drives the application program stored in the storage unit 230, thereby providing or processing appropriate information or functions to the user. The control unit 240 processes signals, data, information, etc. input or output by the above-described components, or drives the application program stored in the storage unit 230, thereby providing or processing appropriate information or functions to the user. The control unit 240 processes signals, data, information, etc. input or output by the above-described components, or drives the application program stored in the storage unit 230, thereby providing or processing appropriate information or functions to the user.

[0089] The control unit 240 controls the operations of the components shown in FIG. 1, that is, the communication unit 210, the display unit 220, and the storage unit 230, in order to drive the application program stored in the storage unit 230. The control unit 240 controls the operations of the components shown in FIG. 1, that is, the communication unit 210, the display unit 220, and the storage unit 230, in order to drive the application program stored in the storage unit 230. The control unit 240 controls the operations of the components shown in FIG. 1, that is, the communication unit 210, the display unit 220, and the storage unit 230, in order to drive the application program stored in the storage unit 230.

[0090] The operation of the control unit 240 will be described in detail as follows.

[0091] The control unit 240 can recognize images of each section of the inner wall of the large intestine from the images captured by the endoscope 100 inserted into the large intestine of the subject, based on the deep learning model 231.

[0092] That is, the control unit 240 recognizes blood vessels located at the lower part of the large intestine mucosa in the images of each section, and can identify the difference in the degree to which the blood vessels are visible.

[0093] As shown in FIGS. 1 and 2, the control unit 240 recognizes the images of each section by the deep learning model 231, and the deep learning model 231 is a model learned based on the blood vessel data in the large intestine images of a plurality of subjects obtained from an external annotator and the degree to which the blood vessel images are interrupted and the pattern of the blood vessels by the light irradiated inside the large intestine. It may be.

[0094] Specifically, the control unit 240 acquires at least one image captured at least once from the endoscope 100 inserted into the large intestine of a plurality of subjects, and for each of the plurality of images, blood vessel data can be obtained from the annotator.

[0095] Here, the annotator is an expert who can well identify the blood vessels of the large intestine, and the plurality of images may be images of the endoscope performed multiple times on one subject or images of the endoscope performed on a plurality of subjects.

[0096] Thereafter, the control unit 240 can perform machine learning based on the degree to which the blood vessel images inside the large intestine are interrupted and the blood vessel data. ​​​​​​

[0097] Here, the degree to which the blood vessel image is interrupted can be determined by the control unit 240 based on whether the blood vessel image visible by the light irradiated from the illumination 120 of the endoscope 100 onto the inner wall of the large intestine has changed to be equal to or greater than a preset percentage threshold. Specifically, when the degree to which the blood vessel image is interrupted is equal to or greater than the preset percentage threshold, the control unit 240 can determine that there is a polyp at that site. In addition, the control unit 240 can recognize the pattern of blood vessels formed in the endoscope image of the large intestine and recognize the region with polyps in the endoscope image of the large intestine based on the recognized blood vessel pattern.

[0098] Specifically, based on the recognized blood vessel pattern, the control unit 240 can recognize that at least one blood vessel pattern in the endoscope image of the large intestine is the blood vessel pattern generated by the light irradiated from the illumination 120 of the endoscope 100 onto the inner wall of the large intestine, and recognize the region where polyps exist in the endoscope image of the large intestine. The control unit 240 can define, as one section, the distance from the point where the endoscope 100 is inserted to the point identified by the light irradiated into the large intestine by the illumination 120 provided in the endoscope 100.

[0099] That is, the control unit 240 can divide the large intestine into n sections and recognize the blood vessels in each section. Here, the large intestine of an adult is about 150 cm to 170 cm, and the endoscope 100 is provided with

[0100] Specifically, based on the recognized blood vessel pattern, the control unit 240 can recognize that at least one blood vessel pattern in the endoscope image of the large intestine is the blood vessel pattern generated by the light irradiated from the illumination 120 of the endoscope 100 onto the inner wall of the large intestine, and recognize the region where polyps exist in the endoscope image of the large intestine. The control unit 240 can define, as one section, the distance from the point where the endoscope 100 is inserted to the point identified by the light irradiated into the large intestine by the illumination 120 provided in the endoscope 100. That is, the control unit 240 can divide the large intestine into n sections and recognize the blood vessels in each section.

[0101] The control unit 240 can define, as one section, the distance from the point where the endoscope 100 is inserted to the point identified by the light irradiated into the large intestine by the illumination 120 provided in the endoscope 100. That is, the control unit 240 can divide the large intestine into n sections and recognize the blood vessels in each section.

[0102] That is, the control unit 240 can divide the large intestine into n sections and recognize the blood vessels in each section.

[0103] Here, the large intestine of an adult is about 150 cm to 170 cm, and the endoscope 100 is provided with The distance to the point identified by the light irradiated from the illuminating unit 120 can be approximately 10 cm to 15 cm. Specifically, the control unit 240 can divide the large intestine into approximately 10 to 15 sections, and recognize the presence of polyps for each section.

[0104] Specifically, the control unit 240 can divide the large intestine into approximately 10 to 15 sections, and recognize the presence of polyps for each section. Specifically, the control unit 240 can divide the large intestine into approximately 10 to 15 sections, and recognize the presence of polyps for each section.

[0105] Specifically, the control unit 240 can generate, based on the deep learning model 231, the point where the endoscope 100 is inserted and the point identified by the light irradiated into the large intestine by the illuminating unit 120 provided in the endoscope 100. Specifically, the control unit 240 can generate, based on the deep learning model 231, the point where the endoscope 100 is inserted and the point identified by the light irradiated into the large intestine by the illuminating unit 120 provided in the endoscope 100. Specifically, the control unit 240 can generate, based on the deep learning model 231, the point where the endoscope 100 is inserted and the point identified by the light irradiated into the large intestine by the illuminating unit 120 provided in the endoscope 100.

[0106] The control unit 240 can display a first visual effect showing a vascular image where the large intestine blood vessels are interrupted in the image of each section. The control unit 240 can display a first visual effect showing a vascular image where the large intestine blood vessels are interrupted in the image of each section.

[0107] On the other hand, the control unit 240 can display a second visual effect showing a continuous vascular image where the large intestine blood vessels are not interrupted in the image of each section. On the other hand, the control unit 240 can display a second visual effect showing a continuous vascular image where the large intestine blood vessels are not interrupted in the image of each section.

[0108] The first visual effect can include a visual effect where markers are respectively displayed on the inner wall of the large intestine in the image of each section. The first visual effect can include a visual effect where markers are respectively displayed on the inner wall of the large intestine in the image of each section.

[0109] The length of each marker can be determined based on the degree to which the vascular image is interrupted.

[0110] Thereafter, the control unit 240 displays the first visual effect for the presence and length of the markers, enabling the specialist to easily confirm the presence and size of flat, thin film - type polyps on the large intestine mucosa. Thereafter, the control unit 240 displays the first visual effect for the presence and length of the markers, enabling the specialist to easily confirm the presence and size of flat, thin film - type polyps on the large intestine mucosa. Thereafter, the control unit 240 displays the first visual effect for the presence and length of the markers, enabling the specialist to easily confirm the presence and size of flat, thin film - type polyps on the large intestine mucosa.

[0111] The large intestine endoscope 100 can, in multiple sections, display the blood vessels in the large intestine according to the first and second visual effects.​ While checking the presence or absence of image discontinuity one by one, it is possible to move into the interior of the large intestine.

[0112] Specifically, after the endoscope 100 recognizes the degree to which the blood vessel image is interrupted for each region, for each region, after returning to the starting point where the region begins, it is possible to check each region without omission. .

[0113] For example, referring to Fig. 4(a), in the first section, the control unit 240 can display the first visual effect with an image in which the blood vessel image is interrupted centered on the first marker (M1) shown by a curve (black) extending horizontally. (black), with an image in which the blood vessel image is interrupted centered on the first marker (M1), the first visual effect can be displayed.

[0114] Referring to Fig. 4(b), in the second section, the control unit 240 can display the first visual effect with an image in which the blood vessel image is interrupted centered on the second marker (M2) shown by a straight line (black) extending in the diagonal direction. (black), with an image in which the blood vessel image is interrupted centered on the second marker (M2), the first visual effect can be displayed.

[0115] Referring to Fig. 4(c), in the second section, the control unit 240 can display the first visual effect with an image in which the blood vessel image is interrupted centered on the third marker (M3) shown by a straight line (black) extending in the diagonal direction and having a length longer than that of the second marker (M2) in the second section. centered on the third marker (M3) shown by a straight line (black) extending in the diagonal direction and having a length longer than that of the second marker (M2) in the second section, in the third section, with an image in which the blood vessel image is interrupted, the first visual effect can be displayed.

[0116] Referring to Fig. 4(d), in the second section, the control unit 240 can display the first visual effect with an image in which the blood vessel image is interrupted centered on the fourth marker (M4) shown by a curve (black) extending in the vertical direction. shown by a curve (black) extending in the vertical direction, with an image in which the blood vessel image is interrupted centered on the fourth marker (M4), the first visual effect can be displayed.

[0117] In addition, the control unit 240 can display the second visual effect showing a continuous blood vessel image without interruption of the large intestine blood vessels in the image of each section. effect can be displayed.

[0118] Thus, when a specialist interprets a colorectal disease through the endoscope 100, the control unit 240 can surely provide the first visual effect for an abnormal colorectal lesion in which a thin and flat-shaped thin-film planar polyp is hidden, or the second visual effect for the inner wall of the normal large intestine, so that all thin and flat polyps can be confirmed without overlooking polyps that cannot be confirmed, thereby improving the accuracy of colorectal examinations.

[0119] FIG. 6 is a sequence diagram for explaining the general operation of a method for detecting colorectal polyps by artificial intelligence-based vascular learning according to an embodiment of the present invention.

[0120] First, the control unit 240 recognizes images of respective sections including the colorectal mucosa and colorectal blood vessels from the images captured by the endoscope 100 inserted into the large intestine of the subject and received in real time (S100) through the communication unit 210.

[0121] Here, the control unit 240 recognizes images of respective sections including the colorectal mucosa and colorectal blood vessels based on the light irradiated inside the large intestine by the endoscope 100.

[0122] In addition, the control unit 240 recognizes images of respective sections by the deep learning model 231.

[0123] Here, the deep learning model 231 can be a model machine-learned based on vascular data in a plurality of large intestine images of the subject obtained from an external annotator and the degree to which the blood vessel images are interrupted and the pattern of blood vessels by the light irradiated inside the large intestine.

[0124] ​​​​​​​The control unit 240 determines whether there is a break in the large intestine blood vessel image for each image of each section (S 300).

[0125] If the result of the determination by the control unit 240 is that there is an image of a blood vessel in which the large intestine blood vessel is interrupted in the image of each section, a first visual effect representing the interrupted blood vessel image is displayed on the display unit 220 (S400).

[0126] The first visual effect includes a visual effect in which respective markers are displayed on the blood vessel image in which the large intestine blood vessel is interrupted in the image of each section, and the size of each marker can be determined based on the degree to which the blood vessel image is interrupted.

[0127] If the result of the determination by the control unit 240 is that there is an image of a continuous blood vessel in which the large intestine blood vessel is not interrupted in the image of each section, a second visual effect representing the continuous blood vessel image is displayed on the display unit 220 (S500).

[0128] The control unit 240 provides, through the first visual effect displayed on the display unit 220, the presence and size of a thin and flat film planar polyp on the large intestine mucosa, and also provides, through the second visual effect, the fact that there is no polyp on the large intestine mucosa in a manner that can be easily understood by a medical professional.

[0129] FIG. 6 describes that the steps (S100) to (S500) are sequentially executed, but this is only an exemplary explanation of the technical idea of this embodiment, and those having ordinary knowledge in the technical field to which this embodiment belongs can change the order described in FIG. 6 or execute one or more steps in parallel without departing from the essential characteristics of this embodiment, and thus can be variously modified and applied. Therefore, FIG. 6 is not limited to a chronological order. ​​​​​​

[0130] Thus, when diagnosing colorectal diseases, the probability of missing an abnormal colorectal lesion in which a thin and flat film planar polyp is hidden can be significantly reduced.

[0131] In addition, even an examiner of a colonoscope with low proficiency in examinations can detect and excise a thin and flat film planar polyp with high accuracy by utilizing the colorectal polyp detection method of the present invention.

[0132] As described above, the present invention uses an artificial intelligence that has learned the morphology of the colonic vascular structure to detect a state in which the blood vessel image visible between the colorectal mucosae is interrupted around the displayed line, and discriminates it as a suspected lesion. The present invention provides a colorectal polyp detection method and apparatus based on artificial intelligence vascular learning that can perform such discrimination.

[0133] Accordingly, the present invention enables even an examiner of a colonoscope with low proficiency in examinations to detect and excise a polyp with high accuracy for an abnormal colorectal lesion in which a thin and flat film planar polyp existing on the colorectal mucosa is hidden when diagnosing colorectal diseases based on an image of a colonoscope.

[0134] The method according to the present invention described above can be embodied in a program (or application) and stored in a medium in order to be executed in combination with a server that is hardware.

[0135] For the aforementioned program to cause a computer to load the program and execute the method embodied by the program, a control unit (CPU) of the computer can read it via a device interface of the computer in C, C++, JAVA (registered trademark), machine language, etc. It can include code encoded in any computer language. Such code can include functional code related to functions that define the functions necessary to execute the method, etc., and can include control code related to the execution procedure necessary for the control unit of the computer to execute the functions according to a predetermined procedure. In addition, such code can further include code related to storage reference, regarding at which position (address) in the internal or external storage unit of the computer the additional information and media necessary for the control unit of the computer to execute the functions should be referenced. Furthermore, when communication with any other computer or server, etc. located remotely is necessary for the control unit of the computer to execute the functions, the code can further include code related to communication, regarding how to communicate with any other computer or server, etc. located remotely using the communication module of the computer, and what information and media should be transmitted and received during communication. The stored medium does not mean a medium that stores data for a short time, such as a register, cache, storage unit, etc., but means a medium that stores data semi-permanently and can be read by the device. Specifically, examples of the stored medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. That is, the program can be stored on various recording media on various servers connected to the computer, or on various recording media on the user's computer. In addition, the medium can be distributed in a computer system connected by a network, and the distributed way is not limited to this.

[0136] The stored medium does not refer to a medium that stores data for a short time, such as a register, cache, storage unit, etc., but refers to a medium that stores data semi-permanently and can be read (reading) by the device. Specifically, examples of the stored medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. That is, the program can be stored on various recording media on various servers connected to the computer, or on various recording media on the user's computer. In addition, the medium can be distributed in a computer system connected by a network, and the distributed way is not limited to this. That is, the program can be stored on various recording media on various servers connected to the computer, or on various recording media on the user's computer. Also, the medium can be distributed in a computer system connected by a network, and the distributed​ In a formula, computer-readable code can be stored.

[0137] The steps of the methods or algorithms described in connection with the embodiments of the present invention may be implemented directly in hardware, implemented by software modules executed by the hardware, or implemented by a combination thereof. The software modules can always be present in RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, hard disk, removable disk, CD-ROM, or any form of computer-readable recording medium well-known in the technical field to which the present invention pertains.

[0138] As described above, the embodiments of the present invention have been described with reference to the accompanying drawings. However, those of ordinary skill in the technical field to which the present invention pertains will understand that the present invention can be implemented in other specific forms without changing its technical idea and essential features. Therefore, the embodiments described above should be understood as illustrative in all aspects and not restrictive. ​

Claims

1. In a method executed by a device, receiving in real time an image captured by an endoscope inserted into the large intestine of a subject; recognizing, through a deep learning model, an image of each section in the image that contains large intestine blood vessels; displaying a first visual effect showing a severed blood vessel image of the large intestine blood vessels in each recognized section image; comprising The deep learning model is a machine learning-based model learned based on blood vessel data in a plurality of large intestine images of a subject obtained from an external annotator, and the degree of interruption of the blood vessel image and the pattern of the blood vessels due to light irradiated into the large intestine. An artificial intelligence-based method for detecting large intestine lesions.

2. The first visual effect includes a visual effect in which respective markers are displayed on the blood vessel image where the large intestine blood vessels are severed in the image of each section, and is characterized in that, the artificial intelligence-based method for detecting large intestine lesions according to claim 1.

3. The size of each of the markers is determined based on the degree to which the blood vessel image is severed, and is characterized in that, the artificial intelligence-based method for detecting large intestine lesions according to claim 2.

4. By the first visual effect showing the blood vessel image where the large intestine blood vessels located below the lower part of the large intestine mucosa recognized in the image of each section, determine the presence and size of a thin film planar polyp on the large intestine mucosa, Characterized in that, by the second visual effect showing a continuous blood vessel image of the large intestine blood vessels recognized in the image of each section, it is determined that there is no thin film planar polyp on the large intestine mucosa, and the artificial intelligence-based method for detecting large intestine lesions according to claim 1.

5. Whether there is a break in the blood vessel image is determined by whether the degree of interruption of the corresponding blood vessel image changes to be equal to or higher than a preset percentage threshold, When the degree of interruption of the blood vessel image is equal to or higher than the preset percentage threshold, the control unit determines that there is a thin film planar polyp in the area of the blood vessel image on the large intestine mucosa, and is characterized in that, the artificial intelligence-based method for detecting large intestine lesions according to claim 3.

6. A computer program stored in a computer-readable recording medium for performing the artificial intelligence-based method for detecting large intestine lesions according to any one of claims 1 to 5, combined with a computer which is hardware.

7. A display unit, A communication unit that receives, in real time, images captured by a colonoscope inserted into the colon of a subject; A storage unit that stores the received images and a deep learning model for recognizing colon blood vessels in the received images; A control unit that, through the deep learning model, recognizes images of respective sections in the received images that contain colon blood vessels, and displays, on the display unit, a first visual effect indicating a broken blood vessel image of the colon blood vessels in each of the recognized images of the respective sections; comprising; The deep learning model is a model learned based on blood vessel data in a plurality of colon images of a subject acquired from an external annotator and the degree of interruption of the blood vessel image and the pattern of the blood vessels due to light irradiated into the interior of the colon, an artificial intelligence-based colon lesion detection device.

8. The first visual effect is including a visual effect in which respective markers are displayed on the blood vessel image in which the colon blood vessels are interrupted in the image of each of the respective sections; The size of each of the markers is determined based on the degree to which the blood vessel image is interrupted, the artificial intelligence-based colon lesion detection device according to claim 7.

9. The control unit is judging the presence and size of a thin film planar polyp on the colon mucosa by the first visual effect showing the blood vessel image in which the colon blood vessels located at the lower part of the recognized colon mucosa in the image of the section are interrupted, judging the fact that there is no thin film planar polyp on the colon mucosa by a second visual effect showing a continuous blood vessel image of the colon blood vessels recognized in the image of the section, the artificial intelligence-based colon lesion detection device according to claim 7.

10. The control unit is judging whether or not there is an interruption in the blood vessel image based on whether or not the degree of interruption of the blood vessel image changes to be equal to or higher than a preset percentage threshold, and when it is equal to or higher than the preset percentage threshold, judging that there is a thin film planar polyp in the region of the blood vessel image on the colon mucosa, the artificial intelligence-based colon lesion detection device according to claim 8.

Images