Systems and methods for diagnosing small intestinal cleanliness

Abstract

The present invention relates to a system for diagnosing small intestine cleanliness. The system may include: a similarity analysis unit for analyzing to select a representative image of a similar small intestine image from a plurality of small intestine images; an image classification unit for predicting the small intestine cleanliness by applying the representative image to the learning result when a series of a plurality of small intestine images whose cleanliness is to be diagnosed are received in a state where a plurality of small intestine images have been learned, thereby classifying the small intestine cleanliness according to a score; and a cleanliness diagnosis unit for calculating a final small intestine cleanliness of a series of a plurality of small intestine images based on the score of the small intestine cleanliness of the representative image and the number of small intestine images similar to the representative image.

Description

Technical Field

[0001] The present invention relates to a system and method for diagnosing small intestine cleanliness, and more particularly, to a system and method for displaying small intestine cleanliness on a user interface (UI) through pre-learning and automatic diagnosis. Background Art

[0002] The small intestine refers to a digestive tract located between the stomach and the large intestine and is 6 to 7 meters long, and is an important organ for digesting and absorbing nutrients in digestion. The small intestine is divided into three regions from above, namely the duodenum, jejunum, and ileum.

[0003] The cleanliness of the small intestine is information that needs to be filled in when making a diagnosis report after diagnosing the condition of the small intestine. Currently, medical experts learn tens of thousands of images of the small intestine, and diagnose the cleanliness through the judgment of the experts based on the learned images.

[0004] However, this conventional diagnosis of small intestinal cleanliness has an inconsistency problem because experts make subjective decisions by diagnosing small intestinal cleanliness themselves, so different experts who diagnose small intestinal cleanliness are likely to make different decisions. Therefore, a system that automatically diagnoses and objectifies small intestinal cleanliness is needed. Summary of the invention

[0005] Technical issues

[0006] The present invention is intended to solve the above-mentioned conventional problems, and one aspect of the present invention is to objectively and consistently diagnose the cleanliness of the small intestine.

[0007] Technical Solution

[0008] In order to achieve the above-mentioned aspects of the present invention, a system for diagnosing small intestine cleanliness is provided, comprising: a similarity analysis unit, which is configured to perform analysis to select a representative image of a similar small intestine image from a plurality of small intestine images; an image classification unit, which is configured to predict the small intestine cleanliness by applying the representative image to the learning result when a series of multiple small intestine images to be diagnosed for cleanliness are received in a state where multiple small intestine images have been learned, thereby classifying the multiple small intestine images according to scores for small intestine cleanliness; and a cleanliness diagnosis unit, which is configured to calculate a final small intestine cleanliness for a series of multiple small intestine images based on the score of the representative image for which the small intestine cleanliness has been scored and the number of small intestine images similar to the representative image.

[0009] Here, the similarity analysis unit may be configured to select the representative image by measuring the similarity based on comparison of color histograms between two small intestine images by a Bhattacharyya distance algorithm and selecting the representative image from a plurality of small intestine images selected based on a given similarity reference.

[0010] Furthermore, as the similarity between the two small intestine images decreases, the similarity reference converges to "0", and as the similarity increases, the similarity reference converges to "1".

[0011] In addition, the image classification unit can be configured to learn color and texture features of multiple small intestine images based on a convolutional neural network (CNN) model, and predict small intestine cleanliness by applying the small intestine images to the learning results based on landmark information.

[0012] Furthermore, the image classification unit may be configured to perform learning after color reference normalization of the plurality of small intestine images output from the plurality of image sensors.

[0013] In addition, the color reference may adopt at least one of RGB, hue saturation value (HSV), and color coordinates (Lab).

[0014] In addition, multiple small intestine images can be scored on a scale of '1' to '10' for small intestine cleanliness.

[0015] Further, the final small intestine cleanliness can be obtained by multiplying the number of similar images corresponding to the first representative image to the nth representative image by the scored cleanliness score, accumulating the multiplication results and dividing the accumulated sum by the total number of multiple small intestine images.

[0016] Furthermore, the image classification unit may be configured to learn the small intestine region based on color and texture in the plurality of small intestine images, and classify the plurality of small intestine images according to the small intestine region based on landmark information set in the small intestine region.

[0017] In order to achieve aspects of the present invention, a method for diagnosing small intestine cleanliness is provided, comprising: receiving a plurality of scored small intestine images; learning the plurality of scored small intestine images; when receiving a series of a plurality of small intestine images for which small intestine cleanliness is to be diagnosed, performing analysis to select a representative image of similar small intestine images from the plurality of small intestine images; predicting small intestine cleanliness by applying the representative image to the learning result, thereby classifying the plurality of small intestine images according to scores for small intestine cleanliness; and calculating a final small intestine cleanliness for the plurality of small intestine images based on the representative image.

[0018] Beneficial Effects

[0019] According to the present invention, the system and method for diagnosing small intestine cleanliness have the following effects.

[0020] First, a consistent and objective index for small intestinal cleanliness can be provided. In the system and method for diagnosing small intestinal cleanliness according to the present invention, the system learns a large number of pictures or images about the small intestine in advance and automatically diagnoses small intestinal cleanliness, thereby providing small intestinal cleanliness more consistently, unlike the small intestinal cleanliness that was inconsistently diagnosed based on the subjective judgment of experts in the past.

[0021] Secondly, time can be saved. In the system and method for diagnosing small intestine cleanliness according to the present invention, when a picture of the small intestine that the user wants to diagnose based on a previously learned picture or image of the small intestine is input, the small intestine cleanliness is immediately diagnosed based on the learned algorithm, thereby saving the time spent on diagnosing the small intestine cleanliness.

[0022] Third, convenience can be provided. As described above, it is easy for the user to diagnose the cleanliness of the small intestine because when a picture of the small intestine to be diagnosed by the user is input, the cleanliness of the small intestine is immediately diagnosed according to the learned algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a block diagram of a system for diagnosing small intestine cleanliness according to an embodiment of the present invention.

[0024] Figure 2 is a flow chart illustrating a method for diagnosing small intestine cleanliness according to an embodiment of the present invention.

[0025] Figure 3 An image of the small intestine that has been scored for small intestine cleanliness in the system for diagnosing small intestine cleanliness according to an embodiment of the present invention is shown.

[0026] Figure 4 It schematically shows that in a system for diagnosing small intestine cleanliness according to an embodiment of the present invention, multiple color references are normalized before learning a small intestine image based on a convolutional neural network (CNN) model.

[0027] Figure 5 The diagram schematically illustrates the learning of small intestine images based on a CNN model in a small intestine cleanliness diagnosis system according to an embodiment of the present invention.

[0028] Figure 6 Schematically illustrating comparison of two small intestine images in a color histogram by a Bhattacharyya distance algorithm used in a system for diagnosing small intestine cleanliness according to an embodiment of the present invention.

[0029] Figure 7 The diagram schematically shows the selection of representative images based on similarity in a system for diagnosing small intestine cleanliness according to an embodiment of the present invention.

[0030] Figure 8 The diagram schematically shows that the small intestine cleanliness is finally calculated in the system for diagnosing small intestine cleanliness according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, some features irrelevant to the subject matter of the present invention may be omitted or summarized, but the omitted features are not necessary for the present invention and may be used in combination by a person skilled in the art to which the present invention belongs.

[0032] <System for diagnosing small intestinal cleanliness>

[0033] Figure 1 is a block diagram of a system for diagnosing small intestine cleanliness according to an embodiment of the present invention.

[0034] like Figure 1 As shown, the system for diagnosing small intestine cleanliness according to an embodiment of the present invention may include a data receiver 100 , an image classification unit 110 , a similarity analysis unit 120 , a cleanliness diagnosis unit 130 , and a database 140 .

[0035] The data receiver 100 is configured to receive a plurality of small intestine images from an expert. The data receiver 100 may be connected to an expert's terminal via a wired / wireless network to receive information. The expert may rate the plurality of small intestine images with scores of "1" to "10" for small intestine cleanliness according to the difficulty of inspection and diagnosis. Here, the plurality of small intestine images may include tens of thousands of small intestine images.

[0036] The image classification unit 110 is configured to score and classify representative images in small intestine cleanliness when receiving a series of multiple small intestine images in real time after learning small intestine cleanliness based on multiple small intestine images received in the data receiver 100. The image classification unit 110 can learn multiple small intestine images based on an artificial neural network. In this case, multiple small intestine images can be learned based on a convolutional neural network (CNN) model. The small intestine images are scored on a scale of "1" to "10" for small intestine cleanliness, and the image classification unit 110 learns the color and texture features of the small intestine images based on the scores associated with the small intestine images scored on a scale of "1" to "10" for small intestine cleanliness, and predicts small intestine cleanliness by applying an image of the small intestine region to the learning result based on landmark information, so that when a series of multiple small intestine images are input later, the small intestine images are classified according to the scores for small intestine cleanliness.

[0037] The similarity analysis unit 120 is configured to perform analysis to select representative images of similar small intestine images from a series of multiple small intestine images received by the data receiver 100 in real time. The similarity analysis unit 120 selects representative images by analyzing whether two small intestine images among the multiple small intestine images are similar small intestine images, so that the image classification unit 110 can classify the multiple small intestine images relative to the small intestine, so that the cleanliness diagnosis unit 130 can quickly diagnose the final small intestine cleanliness. The similarity analysis unit 120 adopts a similarity algorithm to select only representative images of similar small intestine images, so that the image classification unit 110 classifies the multiple small intestine images relative to the small intestine cleanliness without analyzing all of the tens of thousands of small intestine images. In this way, the similarity analysis unit 120 speeds up the classification of small intestine images when the small intestine cleanliness is diagnosed, thereby helping to optimize the system 1 for diagnosing the cleanliness of the small intestine.

[0038] The cleanliness diagnosis unit 130 is configured to calculate the final small intestine cleanliness of the small intestine image in a state of receiving a series of multiple small intestine images in real time. The final small intestine cleanliness needs to be calculated by considering how many similar images the representative image represents. In other words, the small intestine cleanliness of the small intestine image is different depending on the area of ​​the small intestine. Therefore, by accumulating the scores according to the area of ​​the small intestine, the small intestine cleanliness is finally diagnosed based on the average of the small intestine cleanliness of the series of multiple received small intestine images.

[0039] The database 140 is configured to store and retain a plurality of small intestine images received from the expert according to the scores for the small intestine cleanliness. Further, the database 140 may also store and retain the history of the small intestine cleanliness that has been diagnosed by the cleanliness diagnosis unit 130.

[0040] Next, a method for diagnosing small intestine cleanliness according to the present invention will be described.

[0041] Figure 2 is a flowchart illustrating a method of diagnosing small intestine cleanliness according to an embodiment of the present invention.

[0042] like Figure 2 As shown, according to the method for diagnosing small intestine cleanliness according to an embodiment of the present invention, a plurality of small intestine images that have been scored are first received. <s20>

[0043] Here, a plurality of small intestine images for which scoring has been completed will be described in further detail. Figure 3 Give a description.

[0044] Figure 3 An image of the small intestine that has been scored for small intestine cleanliness in the system for diagnosing small intestine cleanliness according to an embodiment of the present invention is shown.

[0045] like Figure 3 As shown, for small intestine cleanliness, the small intestine image can be scored on a scale of "1" to "10". Specifically, the score for small intestine cleanliness can be scored by subjective judgment made by experts when viewing small intestine images based on various experiences. In this case, the expert scores the small intestine cleanliness based on the image clarity of the small intestine with respect to the small intestine image, the proportion of the portion corresponding to the small intestine in the entire image, and floating objects on the image.

[0046] refer to Figure 3 , when the small intestine cleanliness score is "1", the shape of the small intestine is barely visible and unclear on the small intestine image. Therefore, the expert can determine the small intestine cleanliness of the corresponding small intestine image as "1".

[0047] In addition, when the small intestine cleanliness score is "2", the outline, inner wall and shape of the small intestine are clearer than when the score is "1", but there are a lot of floating objects. Therefore, the expert can determine the small intestine cleanliness of the corresponding small intestine image as "2".

[0048] In addition, when the small intestine cleanliness score is "10", the small intestine is seen very clearly on the small intestine image, not only the inner wall of the small intestine but also the villi can be observed, and there are few floating objects. Therefore, the expert can determine the small intestine cleanliness of the corresponding small intestine image as "10".

[0049] Information on the small intestine images scored by the experts as above for the plurality of small intestine images may be received in the data receiver 100 and then stored in the database 140 according to the scores for the cleanliness of the small intestine.

[0050] Next, the image classification unit 110 learns a plurality of small intestine images for which scoring has been completed. <s21>

[0051] The image classification unit 110 may use an artificial neural network to learn the small intestine images scored by experts with scores for small intestine cleanliness. Figure 4 and Figure 5 Describe the process in detail.

[0052] Figure 4 The figure schematically shows that in a system for diagnosing small intestine cleanliness according to an embodiment of the present invention, multiple color references are standardized before learning a small intestine image based on a CNN model. Figure 5 The diagram schematically illustrates the learning of small intestine images based on a CNN model in a small intestine cleanliness diagnosis system according to an embodiment of the present invention.

[0053] First, if Figure 4 As shown, the image classification unit 110 can apply preprocessing to the small intestine images before learning the multiple small intestine images. Specifically, the multiple small intestine images are received from various image sensors, so the color references are different. In other words, even if the small intestine images are scored with the same score for small intestine cleanliness, the small intestine images may be different in red, blue, green (RGB), hue saturation value (HSV), and color coordinates (Lab), so the color reference needs to be standardized to explicitly learn the multiple small intestine images. The color reference is used as a learning input.

[0054] For example, Figure 4 As shown, when learning a small intestine image with a small intestine cleanliness score of "6", the small intestine image with a small intestine cleanliness score of "6" may be different in color reference because the image is received from various image sensors. Therefore, the image classification unit 110 can apply preprocessing to the plurality of small intestine images in the following manner: input grayscale converted from RGB, S selected from HSV, and b selected from Lab for an image with a small intestine cleanliness score of "6".

[0055] Therefore, the image classification unit 110 can learn based on a color reference that is standardized by subjecting a plurality of small intestine images stored according to scores for small intestine cleanliness to preprocessing according to scores for small intestine cleanliness. In addition, the image classification unit 110 can apply preprocessing to a plurality of small intestine images based on a standardized color reference for all scores from "1" to "10" for small intestine cleanliness.

[0056] In addition, if Figure 5 As shown, the image classification unit 110 according to the present invention can use an artificial neural network to learn multiple small intestine images. Specifically, the image classification unit 110 can learn multiple small intestine images stored according to the scores for small intestine cleanliness based on the matching between the state of the small intestine image and the score for small intestine cleanliness. In this case, the artificial neural network can use a CNN model when learning multiple small intestine images.

[0057] Generally, CNN refers to an artificial neural network, which consists of an input layer and an output layer and multiple hidden layers and uses the mathematical operation of convolution. The hidden layer of CNN generally includes a series of convolution layers associated with multiplication or other inner products. The activation function is usually a rectified linear unit (ReLU) layer, followed by additional convolutions, such as pooling layers, fully connected layers, and normalization layers, which are called hidden layers because the input and output are masked by the activation function and the final convolution. This CNN has a structure suitable for learning two-dimensional data and can be trained by a back-propagation algorithm. The CNN model is one of the representative models widely used in various application fields such as object classification and object detection in images, and is therefore used to classify images. According to the present invention, the image classification unit 110 can learn based on the CNN model and classify multiple small intestine images. CNN is well known, so its detailed description will be omitted.

[0058] For example, suppose Figure 5 As shown, five small intestine images are learned. The five images include: a first small intestine image, where stride = 1, filter = 3×3, and width and height are 320×320; a second small intestine image, where stride = 1, filter = 1×1, and width and height are 160×160; a third small intestine image, where stride = 1, filter = 3×3, and width and height are 80×80; a fourth small intestine image, where stride = 1, filter = 1×1, and width and height are 40×40; and a fifth small intestine image, where stride = 1, filter = 3×3, and width and height are 20×20.

[0059] Therefore, the image classification unit 110 uses CNN to apply convolution 1 to the first small intestine image, thereby performing pooling or average pooling through the ReLU layer. Similar to the first small intestine image, the second small intestine image, the third small intestine image, the fourth small intestine image, and the fifth small intestine image are respectively subjected to convolution 2, convolution 3, convolution 4, and convolution 5, thereby undergoing the ReLU layer and maximum pooling or average pooling.

[0060] Here, when RGB, HSV, and Lab are set as learning inputs, average pooling is used at the S level of HSV and the b level of Lab, and maximum pooling is used at the gray level of RGB.

[0061] In this case, when learning the small intestine cleanliness, the color and texture features of the small intestine image are learned. In other words, the small intestine image is different in features such as clarity, color, and texture according to the score. Therefore, the image classification unit 110 performs learning by classifying the color and texture of the small intestine image with a score of "1", the color and texture of the small intestine image with a score of "2", and the color and texture of the small intestine image with a score of "10", thereby learning the features about the score for the small intestine cleanliness.

[0062] In addition, in order to distinguish between regions of the small intestine, the small intestine region can be learned based on the landmark information. In other words, the landmark is set based on the characteristics of the villi or inner wall of the small intestine and is used to distinguish between regions of the small intestine. By distinguishing between regions of the small intestine, the image classification unit 110 can classify the scores of small intestine cleanliness according to the regions of the small intestine. For such landmark information, a specific mark set by a system user can be used.

[0063] Using such a process, when gray, S, and b are used as color references, the image classification unit 110 can learn an image with a score of "6" for the cleanliness of the small intestine.

[0064] When a series of a plurality of small intestine images for which small intestine cleanliness diagnosis is desired is received, analysis is performed to select a representative image of similar small intestine images from among the plurality of small intestine images. <s22>

[0065] Reference Figure 6 Describe the similarities in multiple small intestine images to analyze similarity.

[0066] Figure 6 Schematically illustrating comparison of two small intestine images in a color histogram by a Bhattacharyya distance algorithm used in a system for diagnosing small intestine cleanliness according to an embodiment of the present invention.

[0067] like Figure 6 As shown, the similarity analysis unit 120 can select a representative image by analyzing the small intestine images similar in small intestine cleanliness among the multiple small intestine images. The selection of the representative image among the multiple small intestine images speeds up the process of diagnosing the small intestine cleanliness, so that the small intestine cleanliness can be diagnosed in a short time.

[0068] According to the present invention, the similarity analysis unit 120 can use the Bhattacharyya distance algorithm to analyze the similarity between the small intestine images. The Bhattacharyya distance algorithm is known to be a digital image processing technique that measures the statistical separability of spectral amplitudes when a provability value of correct classification is given. The Bhattacharyya distance algorithm is the strongest algorithm among the algorithms for obtaining the distance between two distributions. In other words, the similarity analysis unit 120 can measure the similarity based on the comparison of the color histograms between the two small intestine images. Therefore, for the similarity based on the comparison of the color histograms between the two images, as the similarity increases, the similarity converges to "0", but as the similarity decreases, the similarity converges to "1".

[0069] Figure 6 Such a color histogram is illustrated. The color histogram may use grayscale as a comparison type, or may use a local binary pattern (LBP: local binary pattern) or a multi-block local binary pattern (MB-LBP: multi-block local binary pattern) image pattern. Here, LBP refers to a similarity of visual descriptors used for classification of computer vision. LBP is well known, so its detailed description will be omitted. Therefore, the similarity analysis unit 120 may convert a plurality of small intestine images into comparison type images for comparison in a color histogram.

[0070] The color histogram will be described in detail. Figure 6 In FIG. 1 , the two upper small intestine images are converted into color histograms H1 and H2 based on grayscale by the similarity analysis unit 120. The color histograms of the two small intestine images show a pattern that increases on the left but decreases on the right. In addition, the color histograms of the two small intestine images are converted into color histograms H1 and H2 based on grayscale by the similarity analysis unit 120. Figure 6 The two lower small intestine images in are converted into color histograms H1 and H2. The color histograms of these two small intestine images can be approximately expressed as Patterned form.

[0071] The similarity analysis unit 120 may measure the similarity between the two small intestine images by comparing color histograms based on a grayscale color histogram or a color histogram of an LBP image pattern.

[0072] Specifically, the similarity analysis unit 120 can calculate the Bhattacharyya distance d(H1, H2) with respect to the color histograms of the two small intestine images. Here, H1 and H2 refer to the color histograms of the two small intestine images mentioned above. and is the average color histogram of the two small intestine images, I is the pixel value, and N is the total number of bins in the color histogram.

[0073] Therefore, the similarity analysis unit 120 may calculate the Bhattacharyya distance d(H1, H2) based on the following elements.

[0074]

[0075] The distance between the two distributions is calculated based on the calculated Shore distance, so the color histograms of the two small intestine images can be compared and the similarity can be measured. The similarity is calculated as a value between '0' and '1', and when comparing the two images in the color histogram, the similarity converges to '0' as the similarity decreases, but converges to '1' as the similarity increases.

[0076] Furthermore, selection of a representative image from among small intestine images between which similarities are analyzed will be described.

[0077] Figure 7 The diagram schematically shows the selection of representative images based on similarity in a system for diagnosing small intestine cleanliness according to an embodiment of the present invention.

[0078] like Figure 7 As shown, similar small intestine images are selected by comparing a plurality of small intestine images after the similarity analyzing unit 120 measures the similarity.

[0079] Specifically, the representative image can be selected by repeatedly comparing the similarity between two small intestine images with the reference similarity on consecutive frames of the small intestine region. For example, it will be assumed that the reference similarity is '0.850'. In this case, Figure 7 As shown, the similarity analysis unit 120 selects one of the first compared small intestine images as a representative image, and performs analysis based on a comparison between the small intestine image and subsequent small intestine images, thereby selecting the representative image and similar small intestine images.

[0080] In other words, reference Figure 7 , the leftmost first small intestine image is compared with the second small intestine image, the third small intestine image, and the nth small intestine image on the right side of the first small intestine image in turn. As a result, the similarity between the second small intestine image and the first small intestine image is '0.923', which is higher than the benchmark similarity '0.850', thereby analyzing that the first small intestine image and the second small intestine image are similar. Between them, the similarity analysis unit 120 can select the first small intestine image as the representative image A, and then compare and analyze the subsequent small intestine images. Similarly, the Bhattacharyya distance algorithm is performed on the first small intestine image and the third small intestine image in the same manner, and then the similarity analysis unit 120 compares and analyzes the histograms of the two small intestine images. As a result, a similarity of '0.921' is obtained, which is higher than the benchmark similarity '0.850', thereby analyzing that the first small intestine image and the third small intestine image are similar to each other. In addition, the fourth small intestine image is compared with the first small intestine image, and a similarity of '0.918' is obtained, which is higher than the reference similarity '0.850', thereby analyzing that the first small intestine image and the fourth small intestine image are similar to each other.

[0081] In this way, the similarity analysis unit 120 can compare and analyze the representative image with other images, thereby calculating the similarity of the nth small intestine image, and similar small intestine images can fall into the small intestine image group to which the representative image belongs. In this case, if there is a small intestine image whose similarity is lower than the reference similarity '0.850', a new representative image is selected.

[0082] Specifically, it is analyzed that the representative images A and B of the small intestine image are not similar because the similarity '0.754' between the two small intestine images is lower than the reference similarity '0.850'. Therefore, the similarity analysis unit 120 can select a similar image by treating the small intestine image B as a new representative image. When the small intestine image after the representative image B is compared with the representative image B that is the same as the image similar to the representative image A, similarities of '0.874' and '0.856' are obtained. Therefore, it is analyzed that the two small intestine images are similar to the representative image B. Therefore, the small intestine image group to which the representative image B belongs can be selected.

[0083] Next, the small intestine cleanliness is predicted by applying the representative image to the learning result, thereby classifying the small intestine images according to the scores for the small intestine cleanliness. <s23>

[0084] The image classification unit 110 classifies the plurality of small intestine images received in real time for small intestine cleanliness using the plurality of learned small intestine images based on the representative image selected in advance from the plurality of small intestine images by the similarity analysis unit 120. In other words, the score of small intestine cleanliness may be given on a scale of '1' to '10', and the image classification unit 110 may classify the representative images according to the scores of '1' to '10' based on the learning result, and may also classify the small intestine images selected as being similar to the representative images together with the representative images according to the scores.

[0085] Finally, the final small intestine cleanliness with respect to the plurality of small intestine images is calculated based on the representative image. <s24>

[0086] Will refer to Figure 8 The description is that the cleanliness diagnosis unit 130 finally calculates the cleanliness of the small intestine.

[0087] Figure 8 The diagram schematically shows the finally calculated small intestine cleanliness in the system for diagnosing small intestine cleanliness according to an embodiment of the present invention.

[0088] like Figure 8 As shown, the final small intestine cleanliness can be diagnosed by considering how many small intestine images the representative image represents among similar small intestine images.

[0089] Specifically, the final small intestine cleanliness is obtained by multiplying the number of similar small intestine images represented by the representative image by the small intestine cleanliness of the representative image, accumulating the multiplication results of all representative images, and dividing the accumulated sum by the total number of small intestine images. In other words, the final small intestine cleanliness is diagnosed based on the average small intestine cleanliness of all small intestine images. This is expressed as follows.

[0090] Final small intestine cleanliness = {(the number of similar images represented by representative image A × the score of small intestine cleanliness '1') + ... + (the number of similar images represented by representative image n × the score of small intestine cleanliness 'n')} / the total number of small intestine images

[0091] For example, Figure 8 As shown, it is assumed that a plurality of small intestine images received in real time by the data receiver 100 are scored for small intestine cleanliness and classified by the image classification unit 110. In this case, the small intestine cleanliness score of representative image A is '9' points, and the total number of small intestine images similar to representative image A and representative image A is '7'. In addition, the small intestine cleanliness score of representative image B is '8' points, and the total number of small intestine images similar to representative image B and representative image B is '3'. Therefore, the cleanliness diagnosis unit 130 diagnoses the final small intestine cleanliness as follows: the value obtained by multiplying the 7 similar small intestine images of representative image A by the '9' point of small intestine cleanliness of representative image A and the value obtained by multiplying the three similar small intestine images of representative image B by the '8' point of small intestine cleanliness of representative image B, and the accumulated sum is divided by the total number of small intestine images 10. This is expressed as follows.

[0092]

[0093] Thus, when the cleanliness diagnosis unit 130 makes a final diagnosis on the small intestine image according to the score for the small intestine cleanliness, it is diagnosed that the score for the small intestine cleanliness is "8.7" for a series of multiple small intestine images received in real time.

[0094] The embodiments of the present invention have been disclosed for illustrative purposes, and those skilled in the art will appreciate that various changes, modifications and additions may be made within the scope of the present invention, and such changes, modifications and additions are all within the scope of the appended claims of the present invention.

Claims

1. A system for diagnosing small intestinal cleanliness, comprising: a similarity analysis unit configured to perform analysis to select a representative image of similar small intestine images from a plurality of small intestine images; an image classification unit configured to, when receiving a series of a plurality of small intestine images of desired diagnosis cleanliness in a state where a plurality of small intestine images are learned, predict the small intestine cleanliness by applying the representative image to the learning result, thereby classifying the plurality of small intestine images according to the scores of the small intestine cleanliness; as well as a cleanliness diagnosis unit configured to calculate a final small intestine cleanliness with respect to the series of a plurality of small intestine images based on the score of the representative image scored for the small intestine cleanliness and the number of small intestine images similar to the representative image, and The image classification unit is configured to learn the color and texture features of the plurality of small intestine images based on a convolutional neural network (CNN) model, and predict the cleanliness of the small intestine by applying the small intestine images to the learning results based on landmark information. wherein the image classification unit is configured to perform learning after color reference normalization of the plurality of small intestine images output from the plurality of image sensors, The color reference uses RGB, HSV hue and saturation values ​​and Lab color coordinates. wherein the color reference standardization of the plurality of small intestine images comprises converting the RGB of the plurality of small intestine images into grayscale, selecting an S component from the HSV hue saturation values ​​of the plurality of small intestine images, and selecting a b component from the Lab color coordinates of the plurality of small intestine images, The CNN model uses average pooling when the HSV hue saturation value is at the S level and the Lab color coordinate is at the b level, and uses maximum pooling when RGB is gray. The final small intestine cleanliness is obtained by multiplying the number of similar images corresponding to the first representative image to the nth representative image by the scored cleanliness score, accumulating the multiplication results and dividing the accumulated sum by the total number of the multiple small intestine images.

2. The system according to claim 1, wherein: The similarity analysis unit is configured to measure the similarity by comparing the color histograms between the two small intestine images using a Bhattacharyya distance algorithm and select the representative image based on a given similarity criterion.

3. The system according to claim 2, wherein: As the similarity between the two small intestine images decreases, the similarity reference converges to 0, and as the similarity increases, the similarity reference converges to 1.

4. The system according to claim 1, wherein: The plurality of small intestine images were scored on a scale of 1 to 10 for small intestine cleanliness.

Images