Patch level severity determination method, slice level severity determination method, and computing system executing same

Abstract

The invention discloses a patch level severity judgment method, a slice level severity judgment method and a computing system executing the patch level severity judgment method and the slice level severity judgment method. According to one aspect of the present invention, a slice-level severity determination method is provided, the slice-level severity determination method is executed in a computing system including a first deep learning model, the first deep learning model is pre-trained to segment a pathological slice image into respective local images of a predetermined unit size if an input divides the pathological slice image into respective local images of a predetermined unit size; the method comprises the following steps of: segmenting an established pathological section image to be judged into each of a plurality of local images to be judged with the unit size, a step of determining an effective level of the to-be-judged local image based on a judgment result of the to-be-judged local image output by the first deep learning model after the to-be-judged local image is input; and determining the overall slice level severity level of the to-be-judged pathological slice image based on the effective level of each of the plurality of to-be-judged local images forming the to-be-judged pathological slice image.

Description

Technical Field

[0001] The present invention relates to a patch-level severity judgment method, a slice-level severity judgment method and a computing system for executing the same. More specifically, it relates to a method for judging an effective level using the output of a deep learning model (the deep learning model is trained to obtain a pathological image input and output histological severity level information), and a method for judging the slice-level severity of the entire tissue included in a pathological slice image using the output of a deep learning model and a computing system for executing the same (the deep learning model is trained to analyze a given partial image including tissue to judge the histological severity of the disease). Background Art

[0002] After a portion or all of a tissue is removed to treat a serious disease such as cancer, a histopathological examination is performed on the removed tissue to diagnose the severity of the disease, predict the patient's prognosis, and determine the treatment method.

[0003] Different diseases have different histological grading systems, which grade and report the severity of the disease based on the morphological characteristics of the tissue. For example, for breast cancer, there is a grading system called the Nottingham system, which measures the degree of maintenance of tissue architecture (tubule formation), the degree of uniformity of the cell nucleus (nuclear pleomorphism), and the degree of mitosis (nuclear mitotic activity), and grades the severity of the tissue by adding the grades from 1 to 3. For ovarian cancer, there is a grading system called the Silverberg system, which is similar to the Nottingham system, and also grades the severity based on the grades of 1 to 3 for each category of tissue architecture, cell nucleus, and mitosis. For prostate cancer, there is a grading system called the Gleason grading system, which distinguishes tissue patterns corresponding to grades 1 to 5 based on the morphological characteristics of the glandular tissue, and grades the overall severity based on the proportion of each grade area in the entire tissue.

[0004] The biggest problem with this grading system is that it is extremely difficult to accurately determine the grade of a lesion based on the system's criteria. The morphology of biological tissues gradually changes as the disease progresses, and within a lesion there may be tissues that can be classified into multiple grades, so it is often difficult to determine a specific grade.

[0005] Recently, due to the development of deep learning technology, the field of image recognition has been able to develop deep learning models with accuracy that exceeds human cognitive ability. Therefore, in the medical field, technologies that use deep learning to diagnose diseases are also being developed and used. Whole slide images generated by scanning biological tissue slices with digital slide scanners are very large in size, so the following method is used, that is, when analyzing them to diagnose diseases or diagnose severity, the whole slide image is roughly divided into small local images (patches or puzzles), each of which is analyzed using a deep learning model, and the results are summarized to generate a diagnosis result for the whole slide.

[0006] When using a deep learning model to analyze tissue images to identify the severity level of the tissue, as described above, due to the fuzzy distinction between levels, there is a problem that the model's recognition results for tissues corresponding to the boundaries between levels are not accurate. By applying a dual-category labeling method that allows for ambiguity in judgment to tissues at the boundaries (Korean Patent No. 10-2162895; Disease diagnosis system and method supporting dual categories), etc., the analysis results of the deep learning model for graded fuzzy tissues can be at the boundaries between levels, but it is extremely difficult to train it to strongly support a certain level.

[0007] On the contrary, when the analysis results of the deep learning model do not strongly support a certain level, the histological severity of the corresponding tissue will be judged to be at the boundary of two levels. By taking into account this boundary information to calculate the histological severity of the whole film, it is expected that a more refined severity grading can be achieved than the original histological severity reporting system. In the research on the development of histological severity judgment technology for prostate cancer (Nagpal, K., Foote, D., Liu, Y. et al. Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer. npj Digit. Med. 2, 48, 2019 https: / / doi.org / 10.1038 / s41746-019-0112-2), Nagpal et al. mentioned a method of assigning values ​​corresponding to the boundary to tissues with ambiguous grade judgments. However, this study only mentioned the feasibility of this method and the example of assigning values ​​corresponding to the boundaries based on the degree of consensus of grading among multiple pathologists. It did not mention the method of using a deep learning model to assign values ​​corresponding to the grade boundaries to lesions, or the method of integrating the severity grades finely assigned to each lesion to grade the severity of the entire tissue.

[0008] On the other hand, in the existing histological severity reporting system for each disease, the severity of the entire tissue is mainly determined in the following manner. For example, in the Gleason grading system for prostate cancer, the overall severity is determined by the sum of the first two severity levels based on the area of ​​the region. Specifically, the average severity of the entire region can be expressed as follows.

[0009] [Table 1]

[0010]

[0011] In addition, for breast cancer, the observed area can also be assigned an average grade of the respective ranges of tissue structure, cell nucleus, and cell division, and these can be added up to give an overall grade, so that the severity of the entire tissue can be considered to be determined in the form of the average severity of the entire area. However, depending on the situation, when there is a lesion with a higher level of severity, it will also need to be assigned a higher severity regardless of its size. Therefore, when outputting the histological severity information of each local image, methods for integrating it include using the maximum value or the average value. However, as shown in the "rare case" of the prostate cancer severity level mentioned above, the general maximum / minimum / average value alone is not enough to determine the overall severity, and a more accurate slice-level severity judgment method can be developed using machine learning (deep learning) models. Summary of the invention

[0012] Technical issues

[0013] The technical problem to be achieved by the present invention is to provide a slice-level severity judgment method that can more finely distinguish the severity of a full-slice image of a biological tissue through deep learning technology.

[0014] In addition, a method for effectively judging the effectiveness level of a biological tissue image is provided.

[0015] Technical Solution

[0016] According to one aspect of the present invention, a slice-level severity judgment method is provided. The slice-level severity judgment method is executed in a computing system including a first deep learning model. The first deep learning model is an artificial neural network that is pre-trained to output a judgment result of the input local image if a pathological slice image is divided into local images of a predetermined unit size. The method includes: for each of a plurality of local images to be judged that are divided into the unit size, a step of determining the effective level of the local image to be judged based on the judgment result of the local image to be judged output by the first deep learning model after the local image to be judged is input; and a step of determining the effective level of the local image to be judged based on the effective level of each of the plurality of local images to be judged that constitute the pathological slice image to be judged. The invention further comprises the steps of determining the slice-level severity level of the entire pathological slice image to be judged; wherein the judgment result of the local image output by the first deep learning model is the likelihood value of each level of a given disease in a given histological severity grading system, and each level in the severity grading system is pre-assigned with a given grade score; and, based on the judgment result of the local image to be judged output by the first deep learning model after the local image to be judged is input, the step of determining the effective grade of the local image to be judged comprises the step of determining the effective grade of the local image to be judged based on the likelihood value of each level of the local image to be judged in the severity grading system and the grade score of each level in the severity grading system.

[0017] In one embodiment, the step of determining the effective level of the local image to be judged based on the possibility values ​​of each level on the severity grading system and the level scores of each level on the severity grading system output by the first deep learning model after the local image to be judged is input may include: taking the possibility values ​​of each level on the severity grading system as weighted values, calculating the weighted average of the level scores of each level on the severity grading system to determine the effective level of the pathological image to be judged; taking the possibility values ​​of each of the multiple upper levels with higher possibility values ​​among the levels on the severity grading system as weighted values, calculating the weighted average of the level scores of each of the multiple upper levels to determine the effective level of the pathological image to be judged; or taking the possibility values ​​of each of the multiple upper levels with higher possibility values ​​on the severity grading system as weighted values, calculating the weighted average of the level scores of each of the multiple upper levels to determine the effective level of the pathological image to be judged. The highest level with the highest probability value The grade score is determined as the pathological image to be judged. Steps of effective hierarchy.

[0018] According to another aspect of the present invention, a slice-level severity judgment method is provided, which is executed in a computing system including a first deep learning model, wherein the first deep learning model is an artificial neural network pre-trained to output a judgment result of the input partial image if a pathological slice image is divided into individual partial images of a predetermined unit size as an input, wherein the method comprises: for each of a plurality of partial images to be judged of a predetermined pathological slice image to be judged divided into the unit size, a step of determining the effective level of the partial image to be judged based on the judgment result of the partial image to be judged output by the first deep learning model after the partial image to be judged is input; and a step of determining the slice-level severity level of the entire pathological slice image to be judged based on the effective level of each of the plurality of partial images to be judged constituting the pathological slice image to be judged; wherein the partial images output by the first deep learning model The judgment result of the partial image includes the judgment result of each pixel constituting the partial image, and the judgment result of each pixel is the likelihood value of each level of a given disease in a given histological severity grading system, and each level in the severity grading system is pre-assigned with a given grade score; and the step of determining the effective grade of the partial image to be judged based on the judgment result of the partial image to be judged output by the first deep learning model after the partial image to be judged is input includes: for each pixel to be judged constituting the partial image to be judged, the step of determining the effective grade of the pixel to be judged based on the likelihood value of each level of the pixel to be judged in the severity grading system and the grade score of each level in the severity grading system; and the step of determining the effective grade of the partial image to be judged based on the effective grade of each pixel to be judged constituting the partial image to be judged.

[0019] In one embodiment, the step of determining the effective level of the pixel to be determined based on the likelihood values ​​of each level of the pixel to be determined in the severity grading system and the level scores of each level in the severity grading system output by the first deep learning model may include: taking the likelihood values ​​of each level of the pixel to be determined in the severity grading system as weighted values, calculating a weighted average of the level scores of each level in the severity grading system, The step of determining the effective level of the pixel to be judged; Describe the severity of The steps of taking the probability value of each of the multiple upper levels with higher probability values ​​among the levels on the severity grading system as a weighted value, calculating the weighted average of the grade scores of each of the multiple upper levels to determine the effective grade of the pixel to be determined; or determining the grade score of the highest level with the highest probability value among the levels of the pixel to be determined on the severity grading system as the effective grade of the pixel to be determined.

[0020] In one embodiment, the step of determining the effective level of the local image to be judged based on the effective level of each pixel to be judged constituting the local image to be judged may include: calculating a representative value of the effective level of each pixel to be judged constituting the local image to be judged to determine the effective level of the local image to be judged.

[0021] In one embodiment, the step of calculating a representative value of the effective level of each pixel to be judged constituting the local image to be judged to determine the effective level of the local image to be judged may include: the step of calculating any one of the maximum value, minimum value, average value, median value, and average value within the interquartile range (InterQuartileRange: IQR) of the effective level of each pixel to be judged constituting the local image to be judged as a representative value.

[0022] In one embodiment, the step of determining the slice-level severity level of the pathological slice image to be judged as a whole based on the effective level of each of the multiple partial images to be judged that constitute the pathological slice image to be judged may include: calculating a representative value of the effective level of each partial image to be judged that constitutes the pathological slice image to be judged, so as to determine the slice-level severity level of the pathological slice image to be judged as a whole.

[0023] In one embodiment, the step of calculating a representative value of the effective levels of each local image to be judged constituting the pathological slice image to be judged to determine the slice-level severity level of the entire pathological slice image to be judged may include: the step of calculating any one of the maximum value, minimum value, average value, median value, and average value within the interquartile range (InterQuartile Range: IQR) of the effective levels of each local image to be judged constituting the pathological slice image to be judged as a representative value.

[0024] In one embodiment, the computing system may further include a second deep learning model as an artificial neural network, which is pre-trained to output a method for determining the pathological slice image if a data set is input. The severity level of the image The data set includes segmenting the pathological slice image into the units The step of determining the severity level of the pathological slice image to be judged based on the effective level of each of the multiple local images to be judged that constitute the pathological slice image to be judged may include: the step of determining the severity level of the pathological slice image to be judged based on the output value output by the second deep learning model after the data set is input; wherein the data set includes the effective level of each of the multiple local images to be judged that constitute the pathological slice image to be judged.

[0025] In one embodiment, the second deep learning model is an artificial neural network pre-trained according to a second deep learning model training method, and the second deep learning model training method may include: the step of obtaining a plurality of training pathological slice images; the step of generating a data set corresponding to the training pathological slice image for each of the plurality of training pathological slices; and the step of inputting each of the data sets corresponding to the plurality of training pathological slice images into the second deep learning model to train the second deep learning model; wherein the step of generating a data set corresponding to the training pathological slice image may include: the step of determining the effective level of the local image for data set generation based on the judgment result output by the first deep learning model after the local image for data set generation is input after the training pathological slice image is divided into each of the plurality of local images for data set generation of the unit size; and the step of generating a data set including the effective level of each of the plurality of local images for data set generation.

[0026] According to another aspect of the present invention, a patch-level severity determination method is provided, and the patch-level severity determination method is executed in a computing system including a first deep learning model, wherein the first deep learning model is an artificial neural network that is pre-trained to output a determination result of the input local image if a pathological slice image is input and divided into local images of a predetermined unit size. The method comprises: inputting a predetermined local image to be determined into the first deep learning model, and obtaining a determination result of the local image to be determined output by the first deep learning model, wherein the determination result of the local image to be determined is a likelihood value of each level of a predetermined disease in a predetermined severity grading system, and each level in the severity grading system is pre-assigned a predetermined grade score; and based on the first deep learning model after inputting the local image to be determined The model outputs the following severity scores: The probability values ​​of each level on the level system and the The step of determining the effective level of the partial image to be judged by using the level scores of each level on the severity grading system.

[0027] According to another aspect of the present invention, a patch-level severity determination method is provided, and the patch-level severity determination method is executed in a computing system including a first deep learning model, and the first deep learning model is an artificial neural network that is pre-trained to output a determination result of the input local image if a pathological slice image is input and divided into local images of a predetermined unit size. The method comprises: inputting a predetermined local image to be determined into the first deep learning model, and obtaining a determination result of the local image to be determined output by the first deep learning model, wherein the determination result of the local image to be determined output by the first deep learning model includes the local image to be determined. The judgment result of each pixel of the image, wherein the judgment result of each pixel is the likelihood value of each level of a given disease in a given histological severity grading system, and each level in the severity grading system is pre-assigned with a given grade score; for each pixel to be judged constituting the local image to be judged, based on the likelihood value of each level of the pixel to be judged in the severity grading system and the grade score of each level in the severity grading system, the step of determining the effective level of the pixel to be judged; and the step of determining the effective level of the pathological image to be judged based on the effective level of each pixel to be judged constituting the local image to be judged.

[0028] According to another aspect of the present invention, there is provided a computer program recorded in a medium, which is installed in a data processing device and is used to execute the above method.

[0029] According to another aspect of the present invention, there is provided a computer-readable recording medium having recorded thereon a computer program for executing the above method.

[0030] According to another aspect of the present invention, there is provided a computing system comprising a processor and a memory, wherein the memory stores a computer program and a first deep learning model, the first deep learning model being an artificial neural network pre-trained to output a judgment result of the input local image if a pathological section image is input and divided into local images of a predetermined unit size, and wherein when the computer program is executed by the processor, it controls the computing system to execute the above method.

[0031] In one embodiment, the memory may further store a second deep learning algorithm as an artificial neural network. A model is pre-trained to output an output value for judging the severity level of the pathological slice image if a data set is input, the data set including the effective level of each of the plurality of partial images obtained by dividing the pathological slice image into the unit size.

[0032] Effects of the Invention

[0033] According to the technical concept of the present invention, a slice-level severity judgment method can be provided, which can use the output of a deep learning model to more finely distinguish the severity of the entire tissue (in other words, the whole slice image), wherein the deep learning model is trained to divide the whole slice image into a part of a given unit size, that is, a local image (for example, a patch or a tile) to analyze to judge the histological severity of a given disease.

[0034] In addition, a method can be provided for effectively judging the effectiveness level of a local image itself using the output of a deep learning model, wherein the deep learning model is trained to output histological severity level information of a biological tissue image. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to provide a more complete understanding of the drawings cited in this summary, a brief description of the drawings is provided.

[0036] Figure 1 The figure briefly shows an example of the structure of a computing system according to an embodiment of the present invention.

[0037] Figure 2 A diagram for describing a first deep learning model according to an embodiment of the present invention.

[0038] Figure 3 The flowchart is a flowchart showing an example of a patch level severity determination method executed by a computing system according to an embodiment of the present invention.

[0039] Figure 4 The present invention is a flowchart showing a process of training a first deep learning model by a computing system according to an embodiment of the present invention.

[0040] Figure 5a and Figure 5b A diagram of a first deep learning model for describing another embodiment of the present invention.

[0041] Figure 6 The flowchart is a flowchart showing an example of a patch level severity determination method executed by a computing system according to another embodiment of the present invention.

[0042] Figure 7 The flowchart is a flowchart showing an example of a method for determining the severity of a slice level executed by a computing system according to an embodiment of the present invention.

[0043] Figure 8 The figure shows a specific example of a slice-level severity determination method according to an embodiment of the present invention.

[0044] Fig. 9 A diagram illustrating a process of training a second deep learning model by a computing system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The present invention may have various variations and may have multiple embodiments, and specific embodiments are listed in the drawings and described in detail in the content of the invention. However, this is not intended to limit the present invention to a specific embodiment, but should be understood to include all variations, equivalents and even substitutes included in the concept and technical scope of the present invention. In describing the present invention, when it is judged that the specific description of the relevant known technology may confuse the gist of the present invention, its detailed description is omitted.

[0046] The terms "first", "second" etc. can be used to describe various components, but the components are not limited by the terms. The terms are only used to distinguish one component from other components.

[0047] The terms used in this application are only used to describe specific embodiments and are not used to limit the present invention. As long as there is no clear difference in the context, a singular expression includes a plural expression.

[0048] It should be understood that, in this specification, terms such as "including" or "having" are only intended to specify the existence of features, numbers, steps, actions, components, accessories or their combinations recorded in the specification, and do not preclude the existence or additional possibilities of one or more other features or numbers, steps, actions, components, accessories or their combinations.

[0049] In addition, in this specification, when it is mentioned that a certain component "transmits" data to another component, it means that the component can directly transmit the data to the other component, and can also transmit the data to the other component through at least one other component. On the contrary, when it is mentioned that a certain component "directly transmits" data to another component, it means that the data is transmitted from the component to the other component without passing through other components.

[0050] The present invention will be described in detail below with reference to the accompanying drawings, focusing on the embodiments of the present invention. The same reference numerals in the drawings represent the same components.

[0051] The patch-level severity determination method and the slice-level severity determination method according to the technical concept of the present invention can be executed by a computing system. The computing system can be a data processing device with computing power, including not only a data processing device that a client can access through a network, that is, a server, but also a computing device such as a personal computer or a portable terminal.

[0052] Figure 1 FIG. 1 is a diagram briefly showing an example of the structure of a computing system according to an embodiment of the present invention. The computing system 100 may have the following physical structure: Figure 1The computing system 100 may include a memory 120 and a processor 110. The memory 120 stores a computer program 123 and a first deep learning model 121 for implementing the technical concept of the present invention. The processor 110 is used to run the computer program 123 stored in the memory 120. According to an embodiment, the memory 120 may further store a second deep learning model 122.

[0053] A person skilled in the art of the present invention can easily deduce that, according to the implementation example of the computing system 100, the processor 110 can be named various names such as CPU, APU, microprocessor, ASIC, etc. In addition, a person skilled in the art of the present invention can easily deduce that the computing system 100 can be implemented by organically combining multiple physical devices. In this case, the processor 160 can be equipped with at least one for each physical device to implement the computing system 100 of the present invention. The processor 160 may also include a GPU for training the first deep learning model 121 and / or the second deep learning model 122.

[0054] The memory 120 may be implemented by any form of storage device that stores the program 123 and is accessible to the processor 110 to drive the program 123. In addition, according to a hardware implementation example, the memory 120 may be implemented by multiple storage devices instead of a certain storage device. In addition, the memory 120 includes not only a main memory device but also a temporary memory device. In addition, it may also be implemented by a volatile memory or a non-volatile memory. The memory 120 may include, for example, a flash memory, a ROM, a RAM, an EEROM, an EPROM, an EEPROM, a hard disk, and a register. Alternatively, the memory 120 may be defined as including all forms of information storage devices that are implemented in a manner that stores the program 123 and can be driven by the processor 110.

[0055] According to an embodiment, the computing system 100 may further include various peripheral devices (peripheral devices 1 to M, 130 - 1 to 130 -M). A person skilled in the art of the present invention may easily deduce that, for example, a keyboard, a display device, a graphics card, a network device, a memory device, etc. may be included in the computing system 100 as peripheral devices.

[0056] A person skilled in the art can easily deduce that the computing system 100 can be implemented by any physical device, but can also be implemented by organically combining multiple physical devices according to the technical concept of the present invention.

[0057] A person skilled in the art can easily deduce that, when a predetermined module is mentioned in the specification below to execute a certain function, it means that the processor 110 drives the program 123 configured in the memory 120 to execute the function.

[0058] In this specification, a deep learning model may mean an information set that includes a multilayer perceptron model and expresses a series of design matters that define an artificial neural network.

[0059] In one embodiment, the deep learning model may be a convolutional neural network. As is well known, a convolutional neural network may include an input layer, a plurality of hidden layers, and an output layer. Each of the plurality of hidden layers may include a convolutional layer and a pooling layer (or a subsampling layer). The convolutional neural network may be defined by functions, filters, strides, weight factors, etc., for defining such various layers. In addition, the output layer may be defined as a fully connected feedforward layer.

[0060] The design matters of each layer constituting the convolutional neural network are well known. For example, for the number of layers to be included in the multiple layers, the convolution function, the pooling function, and the activation function used to define the multiple layers, either a known function or a function defined separately for realizing the technical concept of the present invention can be used.

[0061] As an example of a convolution function, there are discrete convolution and the like. As an example of a pooling function, max pooling, average pooling, etc. can be used. As an example of an activation function, there are sigmoid, Tanh hyperbolic (tanh), ReLU (rectified linear unit), etc. After the design matters of such a convolutional neural network are defined, the convolutional neural network with the defined design matters can be stored in a storage device. Moreover, after the convolutional neural network is trained, the weight factors corresponding to each layer will be specified. That is, the training of the convolutional neural network may mean the process of determining the weight factors of each layer. Moreover, after the convolutional neural network is trained, the trained convolutional neural network can input input data through the input layer and output output data through the predefined output layer.

[0062] The neural network according to the embodiment of the present invention may be defined by selecting any one or more of the well-known design items as described above, or unique design items may be defined for the neural network.

[0063] The neural network may be a classification neural network that can be used for classification of input data. The classification neural network may be a neural network that is used to classify the judgment result of the input data into any one of a plurality of predefined results.

[0064] According to an embodiment, the neural network may also be a segmentation neural network. A segmentation neural network is a neural network used to specify an area that meets a specific condition (e.g., an onset area of ​​a disease) in an input image, and is also called a pixel-level classification neural network that performs classification in pixel units.

[0065] On the other hand, the first deep learning model 121 may be an artificial neural network pre-trained to output a judgment result of the input partial image if the pathological slice image is divided into partial images of a predetermined unit size. The pathological slice image may be a full-slice image of a pathological specimen slice scanned using a digital scanner.

[0066] The digital slice image of the pathological sample can be generated by slicing the pathological sample to make a glass slice, then staining it with a predetermined dye and digitizing it. The pathological sample can be a biopsy collected from various organs of the human body or a biological tissue removed by surgery.

[0067] The local image input into the first deep learning model 121 is a portion of the pathological slice image divided into a predetermined unit size, which can be called a patch or a mosaic.

[0068] In one embodiment, the first deep learning model 121 may be a patch-level classification neural network. In this case, the first deep learning model 121 may be an artificial neural network that is pre-trained to output a judgment result of the input pathological image if a local image is input. The judgment result of the pathological image output by the first deep learning model may be a likelihood value of each level of a given disease in a given histological severity grading system, and each level in the severity grading system may be pre-assigned with a given grade score. For example, the severity grading system may be the Nottingham system, the Silverberg system, the Gleason grading system, etc.

[0069] Figure 2 is a diagram for describing a first deep learning model 121 according to an embodiment of the present invention.

[0070] Reference Figure 2 The first deep learning model 121 may input a partial image 1 through an input layer, and output a judgment result 2 of the input partial image 1 through an output layer. The judgment result 2 may be, for example, a probability value of each level of prostate cancer severity based on the Gleason grading system. At this time, the sum of the probability values ​​of each level may be 1.

[0071] On the other hand, a grade score of 3 may be assigned in advance to each grade on the Gleason grading system.

[0072] Figure 3 The flowchart of an example of a patch level severity determination method performed by the computing system 100 is shown. The computing system 100 may utilize Figure 2 The first deep learning model 121 shown is used to determine the severity of the local image (to be precise, the effective level of a given disease on a given histological severity grading system), in other words, to determine the severity at the patch level.

[0073] Reference Figure 3 , the computing system 100 may input a predetermined partial image to be determined into the first deep learning model 121, and obtain a determination result S100 of the partial image to be determined output by the first deep learning model 121. At this time, the determination result of the partial image to be determined may be the probability value of each level of a predetermined disease in a predetermined severity grading system, as described above, and each level in the severity grading system may be pre-assigned with a predetermined grade score.

[0074] Then, the computing system 100 may determine the validity level of the partial image to be determined based on the determination result of the partial image to be determined S110 .

[0075] In one embodiment, the computing system 100 can use the probability value of each level in the severity grading system as a weighted value to calculate the weighted average of the grade scores of each level in the severity grading system to determine the effective grade of the partial image to be determined. Figure 2 Taking the case of Figure 2 The local image 1 is input into the first deep learning model 121 to obtain the probability value 2 of each Gleason grade, and the probability value of each grade in the Gleason grading system is used as a weighted value to calculate the weighted average of the grade scores of each grade in the Gleason grading system. At this time, benign can be excluded when calculating the weighted average. Figure 2 The effective level calculated in the case of is as follows (rounded to the fourth decimal place).

[0076] (3×0.1+4×0.5+5×0.3) / (0.1+0.5+0.3)=3.8 / 0.9=4.222

[0077] In another embodiment, the computing system 100 can use the probability value of each of the multiple upper levels with higher probability values ​​in each level on the severity grading system as a weighted value to calculate the weighted average of the grade scores of each of the multiple upper levels to determine the effective grade of the partial image to be determined. Assuming that the computing system 100 uses the probability values ​​of the first two levels with higher probability values ​​as weighted values ​​to calculate the weighted average effective grade of the corresponding levels, the computing system 100 can calculate the effective grade of the weighted average of the corresponding levels. Figure 2 In the case of , the effective level is calculated as follows.

[0078] (4×0.5+5×0.3) / (0.5+0.3)=3.5 / 0.8=4.375

[0079] In another embodiment, the computing system 100 may determine the grade score of the highest grade with the highest probability value among the grades in the severity grading system as the effective grade of the partial image to be determined. Figure 2 In the case of , the computing system 100 may determine the grade score 4 of the maximum grade G4 with a probability of 0.5 as the valid grade.

[0080] On the other hand, the first deep learning model 121 is a pre-trained artificial neural network. Figure 4 Flowchart showing the process of training the first deep learning model 121.

[0081] Reference Figure 4 The computing system 100 may obtain a plurality of partial images for training (S200), and each of the plurality of partial images for training may be marked with any of the grades in the histological severity grading system. The plurality of partial images for training may be generated by dividing one or more full-film images for training into unit sizes.

[0082] The computing system 100 may input each of the plurality of training partial images into the first deep learning model 121 to train the first deep learning model 121 .

[0083] The deep learning process based on training data marked with correct answer labels is briefly described. When the deep learning model outputs the results after inputting the training data, the deep learning model reflects the error between the output result and the correct answer label through the back propagation process. In the back propagation process, methods such as gradient descent can be applied. Through this deep learning process, the specific process of training the model from the training data is already well known, so it will not be repeated.

[0084] Previous reference Figures 2 to 4The embodiment of the present invention being described relates to a situation where the first deep learning model 121 is a classification neural network, that is, the first deep learning model 121 is a neural network for determining whether an input image corresponds to a certain level (grade).

[0085] However, according to another embodiment of the present invention, the first deep learning model 121 may be a segmentation neural network or a pixel-level classification neural network. Figure 6 This is described in more detail.

[0086] Figure 5a and Figure 5b A diagram for describing the segmentation neural network of the present invention, namely, the first deep learning model 121-1.

[0087] First refer to Figure 5a The first deep learning model 121-1 may input a local image 10 through an input layer, and the local image 10 may be composed of a plurality of pixels (for example, 11 pixels). Figure 5a In the example of , a situation where the local image 10 is composed of 4×4 pixels is shown. On the other hand, the first deep learning model 121-1 can output a judgment result 20 for the input local image 10, and the judgment result 20 for the local image 10 can include a judgment result (for example, 21) of each pixel constituting the local image 10. Figure 5a In the example of FIG. 1 , the determination result 20 for the partial image 10 may include a determination result for each of the 4×4 pixels.

[0088] Figure 5b It shows the composition Figure 5a The judgment result of any one 11 of the pixels of the local image 10 is shown in FIG. Figure 5b , the judgment result 21 of the pixel 11 may be, for example, the probability values ​​of each level of severity of prostate cancer based on the Gleason grading system. In this case, the sum of the probability values ​​of each level may be 1. On the other hand, the level score 3 of each level on the Gleason grading system may be assigned in advance.

[0089] Figure 6 FIG. 1 is a flowchart showing another example of a patch level severity determination method performed by the computing system 100. The computing system 100 may utilize Figure 5a The first deep learning model 121-1 determines the severity of the local image (to be precise, the effective level of a given disease on a given histological severity grading system).

[0090] Reference Figure 6, the computing system 100 may input a predetermined partial image to be judged into the first deep learning model 121-1, and obtain the judgment result of the partial image to be judged output by the first deep learning model 121-1 (S150). At this time, the judgment result of the partial image to be judged may include the judgment result of each pixel to be judged constituting the partial image to be judged, as described above, and the judgment result of each pixel to be judged may be the probability value of each level of a predetermined disease in a predetermined severity grading system, and each level in the severity grading system may be pre-assigned with a predetermined grade score.

[0091] Then, the computing system 100 may determine the validity level S160 of the pixel to be judged p for each pixel to be judged constituting the local image to be judged based on the judgment result of the pixel to be judged p by the first deep learning model 121 - 1 .

[0092] There are many methods for determining the effective level of the pixel p to be determined.

[0093] In one embodiment, the computing system 100 can calculate the probability of each level in the severity grading system by taking the probability value of each level in the severity grading system as a weighted value. Weighted average of grade scores value to determine the effective level of the pixel p to be judged. Figure 5b of The computing system 100 can obtain Figure 5b The probability values ​​21 of each Gleason grade of the pixel 11 are used as weighted values ​​to calculate the weighted average of the grade scores of each grade in the Gleason grading system. At this time, benign (benign) can be excluded when calculating the weighted average. Figure 5b The effective levels calculated in the case of are as follows.

[0094] (3×0.1+4×0.3+5×0.4) / (0.1+0.3+0.4)=3.5 / 0.8=4.375

[0095] In another embodiment, the computing system 100 can use the probability value of each of the multiple upper levels with higher probability values ​​in each level on the severity grading system as a weighted value to calculate the weighted average of the grade scores of each of the multiple upper levels to determine the effective grade of the pixel to be determined. Assuming that the computing system 100 uses the probability values ​​of the first two levels with higher probability values ​​as weighted values ​​to calculate the weighted average effective grade of the corresponding level, the computing system 100 can be Figure 5b In the case of , the effective level of pixel 11 is calculated as follows (rounded to the 4th decimal place).

[0096] (4×0.3+5×0.4) / (0.3+0.4)=3.2 / 0.7=4.571

[0097] In another embodiment, the computing system 100 may determine the grade score of the highest grade with the highest probability value among the grades in the severity grading system as the effective grade of the pixel to be determined. Figure 5b In the case of , the computing system 100 may determine the grade score 5 of the maximum grade G5 with a probability of 0.4 as a valid grade.

[0098] Then, the computing system 100 may determine the validity level of the partial image to be determined based on the validity level of each pixel constituting the partial image to be determined S170.

[0099] In one embodiment, the computing system 100 may calculate an average value of the effective levels of the pixels constituting the partial image to be determined, and determine the average value as the effective level of the partial image to be determined.

[0100] However, it is not necessary to determine the average value of the effective level of the pixels as the effective level of the local image to be judged. This is just an example for easy understanding. According to an embodiment, the computing system 100 can calculate the representative value of the effective level of each pixel constituting the local image to be judged by various statistical methods, and determine it as the effective level of the local image to be judged. The representative value of more than two values ​​may be the average value of these values, but is not limited to this, and may be a value calculated based on any one of these values ​​or at least a part of these values. For example, the representative value of more than two values, in addition to the average value, may be the maximum value, the minimum value, the median value, the n-quantile value, the average value within the interquartile range (InterQuartile Range: IQR), or the average value of only the values ​​that meet the established conditions (for example, the average value of the remaining values ​​except the values ​​outside the established range), the most frequent value of the histogram, etc.

[0101] Figure 5a The first deep learning model 121-1 is also a pre-trained artificial neural network. Figure 5a The process of the first deep learning model 121-1 shown in FIG. 1 is similar to that of the first deep learning model 121-1, except that the information marked on the training image is the information of the lesion area or the severity level of each pixel. Figure 4 The process shown is almost the same and will not be described again.

[0102] On the other hand, the computing system 100 may integrate the effective levels of the partial images determined by the patch-level severity determination method described above to determine the slice-level severity of the pathological slice image composed of the partial images. Figure 7The flowchart is a flowchart showing an example of a slice-level severity determination method executed by the computing system 100 .

[0103] Reference Figure 7 , the computing system 100 may divide the pathological slice image S to be determined into predetermined unit sizes S300. Assuming that the pathological slice image S to be determined can be divided into N partial images, the divided partial images may be P1 to P N The unit size may be the same as the size of the input data input into the first deep learning model 121 , and the size may be a pixels×b pixels (a and b are integers greater than 1, respectively).

[0104] The computing system 100 may be configured to determine the plurality of partial images P1 to P N In each, the local image P to be judged i (integer i is 1 to N) is input into the first deep learning model 121 S310, S320, and based on the input of the local image P to be judged i (integer i is 1 to N) and then the first deep learning model 121 outputs the local image P to be judged i The judgment result is used to determine the partial image P to be judged. i Effective level D i S330.

[0105] In step S330, the computing system 100 determines the partial image P to be determined. i Effective level D i , you can use the previous reference Figure 3 or Figure 6 Describe the patch-level severity determination method.

[0106] Then, the computing system 100 may calculate the effective level (D1 to D2) of each of the plurality of partial images to be determined based on the effective level (D1 to D2) of each of the plurality of partial images to be determined. N ) to determine the slice-level severity level of the pathological slice image S to be judged S340.

[0107] Figure 8 The figure shows a specific example of a slice-level severity determination method according to an embodiment of the present invention.

[0108] Reference Figure 8 The computing system 100 may obtain a predetermined pathological slice image 10 to be determined, and divide it into a plurality of local images 20 having a predetermined unit size S300. Figure 6 An example of segmentation into 15 partial images (20-1 to 20-15) is shown.

[0109] On the other hand, the computing system 100 may input each of the segmented multiple local images (20-1 to 20-15) into the first deep learning model 121S310, S320, and determine the effective level 30S330 of each of the multiple local images (20-1 to 20-15) based on the judgment result of each of the multiple local images (20-1 to 20-15) output by the first deep learning model 121.

[0110] Then, the computing system 100 may determine the slice-level severity level of the to-be-judged pathological slice image 10 based on the effectiveness level 30 of each of the plurality of partial images ( 20 - 1 to 20 - 15 ) S340 .

[0111] exist Figure 7 In step S340, there are various methods for determining the slice-level severity level.

[0112] In one embodiment, the computing system 100 may calculate a representative value of the effective level of each of the plurality of partial images to be judged constituting the pathological slice image to be judged, and determine the calculated representative value as the slice-level severity level of the entire pathological slice image to be judged.

[0113] For example, the computing system 100 may calculate the maximum value, minimum value, mean value, median value, mean value within the interquartile range (IQR), or the mean value of only values ​​that meet predetermined conditions (for example, the mean value of values ​​other than values ​​outside a predetermined range), the most frequent value of a histogram, etc. of the effective levels of each local image to be judged constituting the pathological section image to be judged as representative values.

[0114] pass Figure 8 In a more specific example, when the computing system 100 uses the average value as the representative value of the effectiveness level of each of the plurality of partial images to be judged, Figure 8 In the example, the computing system 100 may determine the average value of the effective levels of all 15 partial images (20-1 to 20-15), that is, 3.46, as the slice-level severity level of the pathological slice image 10 to be judged.

[0115] Alternatively, when the computing system 100 uses the maximum value as the representative value of the effectiveness level of each of the plurality of partial images to be determined, Figure 8 In the example of , the computing system 100 may determine the maximum value of the valid levels of the 15 local images (20-1 to 20-15), that is, 4.2, as the slice-level severity level of the pathological slice image 10 to be judged.

[0116] Alternatively, when the computing system 100 uses the minimum value as the representative value of the effectiveness level of each of the plurality of partial images to be determined, Figure 8 In the example of , the computing system 100 may determine the minimum value of the valid levels of the 15 local images (20-1 to 20-15), that is, 2.9, as the slice-level severity level of the pathological slice image 10 to be judged.

[0117] Alternatively, when the computing system 100 uses the average of the first few effective levels with larger values ​​among the effective levels of each of the plurality of partial images to be determined as the representative value of the effective level of each of the plurality of partial images to be determined, Figure 8 In the example, the computing system 100 may determine the average of the first three valid levels of the 15 local images (20-1 to 20-15), i.e., 4.1 (= (4.2+4.1+4) / 3) as the slice-level severity level of the pathological slice image 10 to be judged.

[0118] In another embodiment, the computing system 100 may also determine the average value of the effective levels above the predetermined critical value in the effective levels of each of the multiple partial images to be judged as the slice-level severity level of the entire pathological slice image to be judged. Alternatively, when the computing system 100 uses the average value of the effective levels above the predetermined critical value in the effective levels of each of the multiple partial images to be judged as the representative value of the effective level of each of the multiple partial images to be judged, Figure 8 In the example, the computing system 100 may determine the average value 3.9 of the effective levels above the critical value 3.5 among the effective levels of the 15 local images (20-1 to 20-15) as the slice-level severity level of the pathological slice image 10 to be judged.

[0119] In addition, Figure 7 In step S340 of , there may of course be various methods for determining the slice-level severity level.

[0120] In one embodiment, the computing system 100 may also utilize deep learning to determine the slice-level severity level. In this case, the second deep learning model 122 may be utilized.

[0121] The second deep learning model 122 may be an artificial neural network that is pre-trained to output an output value for determining the severity level of the pathological slice image if a data set is input, the data set including the effective level of each of the plurality of partial images of the unit size that divide the pathological slice image. For example, the second deep learning model 122 may be a neural network that is inputted by Figure 8After the data set 30 consisting of the effective level of each of the plurality of partial images shown is formed, an output value for judging the slice-level severity 40 of the pathological slice image 10 is output.

[0122] According to this embodiment, in step S340, the computing system 100 may determine the slice-level severity level of the pathological slice image S to be determined based on the output value output by the second deep learning model 122 after inputting the data set, wherein the data set includes the effective level (D1 to D2) of each of the plurality of partial images to be determined constituting the pathological slice image S to be determined. N ).

[0123] The specific process of determining the slice-level severity level of the pathological slice image S to be determined based on the output value of the second deep learning model 122 can be Figure 3 or Figure 6 The effective level determination process is similar to that shown in Figure 3 Similar to the process shown, the second deep learning model 122 can output the probability value of each level on the severity grading system, and the computing system 100 can determine the slice-level severity level by the weighted average of the grade scores of each level on the severity grading system calculated with the probability value of each level on the severity grading system as the weighted value, the weighted average of the grade scores of each of the multiple upper levels calculated with the probability value of each of the multiple upper levels with higher probability values ​​among the levels on the severity grading system as the weighted value, the grade score of the highest level with the highest probability value among the levels on the severity grading system, etc.

[0124] The second deep learning model 122 may be a neural network that is pre-trained before being used to determine the slice-level severity level. Fig. 9 A diagram showing the training process of the second deep learning model 122.

[0125] Reference Fig. 9 The computing system 100 may obtain a training pathological slice image R and divide the training pathological slice image R into the unit sizes S400. Assuming that the training pathological slice image R is divided into N partial images, the partial images for generating each data set are Q1 to Q N .

[0126] Then, the computing system 100 may generate local images (Q1 to Q N ), the data set is generated using a local image Q i(integer i is 1 to N) is input to the first deep learning model 121 S410, S420, and a local image Q is generated based on the input data set i The first deep learning model 121 outputs a judgment result to determine the data set generation using the local image Q i Effective level E i S430.

[0127] Then, the computing system 100 may input the dataset including the effective level (E1 to E2) of each of the local images used for generating the plurality of datasets into the second deep learning model 122 to train the second deep learning model 122. N ).

[0128] On the other hand, the method of the embodiment of the present invention can be implemented in the form of computer-readable program instructions and stored in a computer-readable recording medium, and the control program or target program of the embodiment of the present invention can also be stored in a computer-readable recording medium. Computer-readable recording media include all kinds of recording devices that store data that can be read by a computer system.

[0129] The program instructions recorded in the recording medium may be specially designed for the present invention, or may be well known and usable by software practitioners.

[0130] Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floppy disks, and hardware devices specifically configured to store and execute program instructions such as read-only memories (ROMs), random access memories (RAMs), and flash memories. In addition, computer-readable recording media can be distributed in computer systems connected via a network to store and run computer-readable codes in a distributed manner.

[0131] Examples of the program instructions include not only machine codes such as codes generated by a compiler but also high-level language codes executable by a device (eg, a computer) that electronically processes information using an interpreter or the like.

[0132] In order to execute the actions of the present invention, the aforementioned hardware devices may be configured to operate as one or more software modules, and vice versa.

[0133] The above description is for illustrative purposes only, and those skilled in the art will appreciate that the present invention can be easily transformed into other specific forms without changing the technical concept or essential features of the present invention. Therefore, the above described embodiments are to be interpreted as exemplary and non-restrictive in all respects. For example, the components described as a single type may also be implemented in a dispersed manner, and similarly, the components described as a dispersed type may also be implemented in a combined form.

[0134] The scope of the present invention is represented by the following claims compared to the above-described invention content, and all modifications or changes derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.

[0135] [Industrial Applicability]

[0136] The present invention can be used for a patch-level severity determination method, a slice-level severity determination method and a computing system for executing the same.

Claims

1. A method for determining the severity of a slice level, the method for determining the severity of a slice level being executed in a computing system including a first deep learning model, the first deep learning model being an artificial neural network pre-trained to output a determination result of the input partial image if a pathological slice image is divided into partial images of a predetermined unit size, the method comprising: For each of the plurality of partial images to be judged that are divided into the unit sizes of the predetermined pathological slice image to be judged, a step of determining the effective level of the partial image to be judged based on a judgment result of the partial image to be judged output by the first deep learning model after the partial image to be judged is input; and A step of determining a slice-level severity level of the entire pathological slice image to be determined based on the effective level of each of the plurality of partial images to be determined constituting the pathological slice image to be determined; The judgment result of the local image output by the first deep learning model is the probability value of each level of a given disease in a given histological severity grading system, and each level in the severity grading system is pre-assigned with a given grade score. Moreover, the step of determining the validity level of the partial image to be determined based on the determination result of the partial image to be determined output by the first deep learning model after the partial image to be determined is input includes: The step of determining the effective level of the partial image to be judged based on the possibility value of each level of the partial image to be judged in the severity grading system and the level score of each level in the severity grading system.

2. The method for determining the severity of a slice according to claim 1, wherein: The step of determining the effective level of the partial image to be determined based on the possibility values ​​of each level in the severity grading system and the level scores of each level in the severity grading system output by the first deep learning model after the partial image to be determined is input includes: The step of calculating a weighted average of the grade scores of each grade in the severity grading system by taking the probability value of each grade in the severity grading system as a weighted value to determine the effective grade of the pathological image to be determined; The step of calculating a weighted average of the grade scores of each of the plurality of upper grades with higher likelihood values ​​among the grades on the severity grading system, using the likelihood value of each of the plurality of upper grades as a weighted value, so as to determine the effective grade of the pathological image to be determined; or The step of determining the grade score of the highest grade with the highest probability value among the grades on the severity grading system as the effective grade of the pathological image to be determined.

3. A slice-level severity determination method, the slice-level severity determination method is executed in a computing system including a first deep learning model, the first deep learning model is an artificial neural network pre-trained to output a determination result of the input partial image if a pathological slice image is divided into partial images of a predetermined unit size, the method comprising: For each of the plurality of partial images to be judged that are divided into the unit sizes of the predetermined pathological slice image to be judged, a step of determining the effective level of the partial image to be judged based on a judgment result of the partial image to be judged output by the first deep learning model after the partial image to be judged is input; and A step of determining a slice-level severity level of the entire pathological slice image to be determined based on the effective level of each of the plurality of partial images to be determined constituting the pathological slice image to be determined; The judgment result of the local image output by the first deep learning model includes the judgment result of each pixel constituting the local image, and the judgment result of each pixel is the likelihood value of each level of a given disease in a given histological severity grading system, and each level in the severity grading system is pre-assigned with a given grade score. Moreover, the step of determining the validity level of the partial image to be determined based on the determination result of the partial image to be determined output by the first deep learning model after the partial image to be determined is input includes: For each pixel to be determined constituting the partial image to be determined, the effective level of the pixel to be determined is determined based on the possibility value of each level of the pixel to be determined in the severity grading system and the level score of each level in the severity grading system; and The step of determining the effectiveness level of the partial image to be determined based on the effectiveness levels of the pixels to be determined that constitute the partial image to be determined.

4. The method for determining the severity of a slice according to claim 3, wherein: The step of determining the effective level of the pixel to be determined based on the possibility values ​​of each level of the pixel to be determined in the severity grading system and the level scores of each level in the severity grading system output by the first deep learning model comprises: The step of calculating a weighted average of the grade scores of each grade in the severity grading system using the probability values ​​of each grade of the pixel to be determined in the severity grading system as weighted values ​​to determine the effective grade of the pixel to be determined; The step of calculating a weighted average of the grade scores of each of the plurality of upper grades with higher likelihood values ​​among the grades of the pixel to be determined on the severity grading system using the likelihood value of each of the plurality of upper grades as a weighted value to determine the effective grade of the pixel to be determined; or The step of determining the grade score of the highest grade with the highest probability value among the grades of the pixel to be determined in the severity grading system as the effective grade of the pixel to be determined.

5. The method for determining the severity of a slice according to claim 3, wherein: The step of determining the effective level of the partial image to be determined based on the effective levels of the pixels to be determined constituting the partial image to be determined comprises: The step of calculating a representative value of the effective level of each pixel to be judged constituting the partial image to be judged to determine the effective level of the pathological image to be judged.

6. The method for determining the severity of a slice according to claim 1 or 3, wherein: The step of determining the slice-level severity level of the entire pathological slice image to be determined based on the effective level of each of the plurality of partial images to be determined constituting the pathological slice image to be determined comprises: The step of calculating representative values ​​of the effective levels of the partial images to be judged constituting the pathological slice image to be judged, so as to determine the slice-level severity level of the entire pathological slice image to be judged.

7. The method for determining the severity of a slice according to claim 6, wherein: The step of calculating the representative value of the effective level of each partial image to be judged constituting the pathological slice image to be judged to determine the slice-level severity level of the entire pathological slice image to be judged comprises: The step of calculating any one of the maximum value, minimum value, average value, median value, and average value within the interquartile range (IQR) of the effective level of each partial image to be judged constituting the pathological section image to be judged as a representative value.

8. The method for determining the severity of a slice according to claim 1 or 3, wherein: The computing system further includes a second deep learning model as an artificial neural network, which is pre-trained to output an output value for determining the severity level of the pathological slice image if a data set is input, the data set including the effective level of each of the plurality of partial images segmented into the unit size for the pathological slice image; Furthermore, the step of determining the severity level of the pathological section image to be determined based on the effective level of each of the plurality of partial images to be determined constituting the pathological section image to be determined comprises: A step of determining the severity level of the pathological section image to be judged based on the output value output by the second deep learning model after inputting a data set; wherein the data set includes the effective level of each of the multiple local images to be judged that constitute the pathological section image to be judged.

9. The method for determining the severity of a slice according to claim 8, wherein: The second deep learning model is an artificial neural network pre-trained according to a second deep learning model training method, Wherein, the second deep learning model training method comprises: The step of obtaining a plurality of pathological slice images for training; For each of the plurality of training pathological sections, a step of generating a data set corresponding to the training pathological section image; and A step of inputting each of the data sets corresponding to the plurality of training pathological section images into the second deep learning model to train the second deep learning model; Furthermore, the step of generating a data set corresponding to the training pathological slice image includes: For each of the plurality of partial images for data set generation that are obtained by dividing the training pathological slice image into the unit sizes, a step of determining a validity level of the partial image for data set generation based on a judgment result output by the first deep learning model after the partial image for data set generation is input; and The step of generating includes generating a dataset of the effectiveness level of each of the partial images using the plurality of datasets.

10. A patch-level severity determination method, the patch-level severity determination method being executed in a computing system including a first deep learning model, the first deep learning model being an artificial neural network pre-trained to output a determination result of the input partial image if a pathological slice image is divided into partial images of a predetermined unit size, the method comprising: The step of inputting a predetermined partial image to be determined into the first deep learning model to obtain a determination result of the partial image to be determined output by the first deep learning model, wherein the determination result of the partial image to be determined is a probability value of each level of a predetermined disease in a predetermined severity grading system, and each level in the severity grading system is pre-assigned with a predetermined grade score; and The step of determining the effective level of the local image to be judged based on the possibility value of each level in the severity grading system and the level score of each level in the severity grading system output by the first deep learning model after the local image to be judged is input.

11. A patch-level severity determination method, the patch-level severity determination method being executed in a computing system including a first deep learning model, the first deep learning model being an artificial neural network pre-trained to output a determination result of the input partial image if a pathological slice image is divided into partial images of a predetermined unit size, the method comprising: The step of inputting a predetermined partial image to be judged into the first deep learning model, and obtaining a judgment result of the partial image to be judged output by the first deep learning model, wherein the judgment result of the partial image to be judged output by the first deep learning model includes a judgment result of each pixel constituting the partial image to be judged, and the judgment result of each pixel is a likelihood value of each level of a predetermined disease in a predetermined histological severity grading system, and each level in the severity grading system is pre-assigned with a predetermined grade score; For each pixel to be determined constituting the partial image to be determined, the effective level of the pixel to be determined is determined based on the possibility value of each level of the pixel to be determined in the severity grading system and the level score of each level in the severity grading system; and The step of determining the effective level of the pathological image to be determined based on the effective levels of each pixel to be determined constituting the partial image to be determined.

12. A computer program recorded in a medium, installed in a data processing device, for executing the method described in any one of claims 1, 3, 10 and 11.

13. A computer-readable recording medium having recorded thereon a computer program for executing the method described in any one of claims 1, 3, 10 and 11.

14. A computing system comprising: Processor and memory; The memory stores a computer program and a first deep learning model, which is an artificial neural network pre-trained to output a judgment result of the input local image if the pathological slice image is divided into local images of a predetermined unit size. Moreover, when the computer program is executed by the processor, it controls the computing system to execute a slice-level severity determination method. The slice-level severity determination method includes: For each of the plurality of partial images to be judged that are divided into the unit sizes of the predetermined pathological slice image to be judged, a step of determining the effective level of the partial image to be judged based on a judgment result of the partial image to be judged output by the first deep learning model after the partial image to be judged is input; and A step of determining a slice-level severity level of the entire pathological slice image to be determined based on the effective level of each of the plurality of partial images to be determined constituting the pathological slice image to be determined; The judgment result of the local image output by the first deep learning model is the likelihood value of each level of a given disease in a given histological severity grading system, and each level in the severity grading system is pre-assigned with a given grade score. Moreover, the step of determining the validity level of the partial image to be determined based on the determination result of the partial image to be determined output by the first deep learning model after the partial image to be determined is input includes: The step of determining the effective level of the pathological image to be judged based on the possibility value of each level of the partial image to be judged in the severity grading system and the level score of each level in the severity grading system.

15. The computing system of claim 14, wherein: The step of determining the effective level of the pathological image to be determined based on the possibility value of each level in the severity grading system and the level score of each level in the severity grading system output by the first deep learning model after the local image to be determined is input includes: The step of calculating a weighted average of the grade scores of each grade in the severity grading system by taking the probability value of each grade in the severity grading system as a weighted value to determine the effective grade of the pathological image to be determined; The step of calculating a weighted average of the grade scores of each of the plurality of upper grades with higher likelihood values ​​among the grades on the severity grading system, using the likelihood value of each of the plurality of upper grades as a weighted value, so as to determine the effective grade of the pathological image to be determined; or The step of determining the grade score of the highest grade with the highest probability value among the grades on the severity grading system as the effective grade of the pathological image to be determined.

16. A computing system comprising: Processor and memory; The memory stores a computer program and a first deep learning model, which is an artificial neural network pre-trained to output a judgment result of the input local image if the pathological slice image is divided into local images of a predetermined unit size. Moreover, when the computer program is executed by the processor, it controls the computing system to execute a slice-level severity determination method. The slice-level severity determination method includes: For each of the plurality of partial images to be judged that are divided into the unit sizes of the predetermined pathological slice image to be judged, a step of determining the effective level of the partial image to be judged based on a judgment result of the partial image to be judged output by the first deep learning model after the partial image to be judged is input; and A step of determining a slice-level severity level of the entire pathological slice image to be determined based on the effective level of each of the plurality of partial images to be determined constituting the pathological slice image to be determined; The judgment result of the local image output by the first deep learning model includes the judgment result of each pixel constituting the local image, and the judgment result of each pixel is the likelihood value of each level of a given disease in a given histological severity grading system, and each level in the severity grading system is pre-assigned with a given grade score. Moreover, the step of determining the validity level of the partial image to be determined based on the determination result of the partial image to be determined output by the first deep learning model after the partial image to be determined is input includes: For each pixel to be determined constituting the partial image to be determined, based on the possibility value of each level of the pixel to be determined in the severity grading system and the level score of each level in the severity grading system, the effective level of the pixel to be determined is determined; and The step of determining the effectiveness level of the partial image to be determined based on the effectiveness levels of the pixels to be determined that constitute the partial image to be determined.

17. The computing system of claim 16, wherein: The step of determining the effective level of the pixel to be determined based on the likelihood values ​​of each level of the pixel to be determined in the severity grading system and the level scores of each level in the severity grading system comprises: The step of calculating a weighted average of the grade scores of each grade in the severity grading system using the probability values ​​of each grade of the pixel to be determined in the severity grading system as weighted values ​​to determine the effective grade of the pixel to be determined; The step of calculating a weighted average of the grade scores of each of the plurality of upper grades with higher likelihood values ​​among the grades of the pixel to be determined on the severity grading system using the likelihood value of each of the plurality of upper grades as a weighted value to determine the effective grade of the pixel to be determined; or The step of determining the grade score of the highest grade with the highest probability value among the grades of the pixel to be determined in the severity grading system as the effective grade of the pixel to be determined.

18. The computing system of claim 16, wherein: The step of determining the effective level of the partial image to be determined based on the effective levels of the pixels to be determined constituting the partial image to be determined comprises: The step of calculating a representative value of the effective level of each pixel to be judged constituting the partial image to be judged to determine the effective level of the pathological image to be judged.

19. The computing system of claim 14 or 16, wherein: The step of determining the slice-level severity level of the entire pathological slice image to be determined based on the effective level of each of the plurality of partial images to be determined constituting the pathological slice image to be determined comprises: The step of calculating representative values ​​of the effective levels of the partial images to be judged constituting the pathological slice image to be judged, so as to determine the slice-level severity level of the entire pathological slice image to be judged.

20. The computing system of claim 19, wherein: The step of calculating the representative value of the effective level of each partial image to be judged constituting the pathological slice image to be judged to determine the slice-level severity level of the entire pathological slice image to be judged comprises: The step of calculating any one of the maximum value, minimum value, average value, median value, and average value within the interquartile range of the effective level of each partial image to be judged constituting the pathological section image to be judged as a representative value.

21. The computing system of claim 14 or 16, wherein: The memory also stores a second deep learning model as an artificial neural network, which is pre-trained to output an output value for judging the severity level of the pathological slice image if a data set is input, the data set including the effective level of each of the plurality of partial images of the unit size into which the pathological slice image is segmented; Furthermore, the step of determining the severity level of the pathological section image to be determined based on the effective level of each of the plurality of partial images to be determined constituting the pathological section image to be determined comprises: A step of determining the severity level of the pathological section image to be judged based on the output value output by the second deep learning model after inputting a data set; wherein the data set includes the effective level of each of the multiple local images to be judged that constitute the pathological section image to be judged.

22. The computing system of claim 21, wherein: The second deep learning model is an artificial neural network pre-trained according to a second deep learning model training method, Wherein, the second deep learning model training method comprises: The step of obtaining a plurality of pathological slice images for training; For each of the plurality of training pathological sections, a step of generating a data set corresponding to the training pathological section image; and A step of inputting each of the data sets corresponding to the plurality of training pathological section images into the second deep learning model to train the second deep learning model; Furthermore, the step of generating a data set corresponding to the training pathological slice image includes: For each of the plurality of partial images for data set generation that are obtained by dividing the training pathological slice image into the unit sizes, based on a judgment result output by the first deep learning model after the partial image for data set generation is input, a step of determining a validity level of the partial image for data set generation; and The step of generating includes generating a dataset of the effectiveness level of each of the partial images using the plurality of datasets.

23. A computing system comprising: Processor and memory; The memory stores a computer program and a first deep learning model, which is an artificial neural network pre-trained to output a judgment result of the input local image if the pathological slice image is divided into local images of a predetermined unit size. Furthermore, when the computer program is executed by the processor, the computer system is controlled to execute a patch level severity determination method. Wherein, the patch level severity determination method includes: A step of inputting a predetermined partial image to be determined into the first deep learning model to obtain a determination result of the partial image to be determined output by the first deep learning model, wherein the determination result of the partial image to be determined is a likelihood value of each level of a predetermined disease in a predetermined severity grading system, and each level in the severity grading system is pre-assigned with a predetermined grade score; and The step of determining the effective level of the local image to be judged based on the possibility value of each level in the severity grading system and the level score of each level in the severity grading system output by the first deep learning model after the local image to be judged is input.

24. A computing system comprising: Processor and memory; The memory stores a computer program and a first deep learning model, which is an artificial neural network pre-trained to output a judgment result of the input local image if the pathological slice image is divided into local images of a predetermined unit size. Furthermore, when the computer program is executed by the processor, the computer system is controlled to execute a patch level severity determination method. Wherein, the patch level severity determination method includes: The step of inputting a predetermined partial image to be judged into the first deep learning model, and obtaining a judgment result of the partial image to be judged output by the first deep learning model, wherein the judgment result of the partial image to be judged output by the first deep learning model includes a judgment result of each pixel constituting the partial image to be judged, and the judgment result of each pixel is a likelihood value of each level of a predetermined disease in a predetermined histological severity grading system, and each level in the severity grading system is pre-assigned with a predetermined grade score; For each pixel to be determined constituting the partial image to be determined, the effective level of the pixel to be determined is determined based on the possibility value of each level of the pixel to be determined in the severity grading system and the level score of each level in the severity grading system; and The step of determining the effective level of the pathological image to be determined based on the effective levels of each pixel to be determined constituting the partial image to be determined.

Images