Nasal cavity tumor identification model training method and training device and nasal cavity tumor identification method and device

Abstract

The invention discloses a training method and a training device for a nasal cavity tumor identification model. The training method of the nasal cavity tumor identification model comprises the following steps: extracting a plurality of block images from a nasal endoscope image comprising a nasal cavity region; training a lesion detection model based on a neural network by using the nasal endoscope image and the extracted plurality of block images; detecting whether a lesion exists in the nasal endoscope image or not by using the trained lesion detection model; generating a combined image by combining the nasal endoscopic image and a computed tomography (CT) image of the nasal cavity region when the presence of the lesion in the nasal endoscopic image is detected; and training an identification model based on a neural network, which is used for identifying nasal cavity masses aiming at the lesion, on the basis of the generated combined image.

Description

Technical Field

[0001] The embodiments of the present invention relate to a nasal tumor identification model training technology and a nasal tumor identification technology. Background Art

[0002] If the best treatment period for inverted papilloma is missed, it may damage surrounding tissues and induce malignant transformation, so timely diagnosis is needed to ensure timely and appropriate treatment. However, nasal polyps and inverted papilloma are similar in appearance and difficult to distinguish during clinical examinations. The current conventional technology for detecting nasal lesions has the limitation of low accuracy. Therefore, research is urgently needed to overcome the above limitations. Summary of the invention

[0003] Technical methods to solve problems

[0004] According to one embodiment, a training method for a nasal cavity mass identification model includes the following steps: extracting a plurality of block images from a nasal endoscopic image including a nasal cavity area; using the nasal endoscopic image and the extracted plurality of block images to train a neural network-based lesion detection model; using the trained lesion detection model to detect whether there is a lesion in the nasal endoscopic image; when the lesion is detected in the nasal endoscopic image, generating a combined image by combining the nasal endoscopic image and a computerized tomography (CT) image of the nasal cavity area; and training a neural network-based identification model for identifying nasal cavity masses for the lesions based on the generated combined image.

[0005] The nasal endoscope image corresponds to an overall image including the nasal cavity area, and the block images correspond to partial images of the overall image.

[0006] The extracting step includes the following steps: randomly cutting out a partial area from the entire area of ​​the nasal endoscopic image, and extracting the plurality of block images corresponding to the partial area from the nasal endoscopic image.

[0007] According to one embodiment, the training method further includes the following steps: when the center of the lesion area is set in the nasal endoscope image, the multiple block images corresponding to the partial area are extracted from the nasal endoscope image by cropping the partial area based on the center of the lesion area.

[0008] According to one embodiment, the training method further includes the following steps: performing image processing including at least one of enlarging the size, reducing the size, rotating and shifting on at least one of the extracted block images and the nasal endoscope image, and the step of training the lesion detection model includes the following steps: training the lesion detection model based on at least one of the extracted block images and the nasal endoscope image that have undergone the image processing.

[0009] The lesion detection model distinguishes whether the nasal endoscope image is a lesion image including a lesion area or a regular image not including a lesion area.

[0010] The step of training the lesion detection model comprises the following steps: assigning weights to the texture and shape of the nasal area included in the nasal endoscope image and at least one of the extracted multiple block images; and subjecting the lesion detection model to curriculum learning based on the weighted texture and shape of the nasal area.

[0011] The discrimination model distinguishes the lesion into any one of nasal polyps, inverted papilloma, and normal tissue.

[0012] According to one embodiment, a method for identifying nasal tumors includes the following steps: extracting multiple block images from a nasal endoscopic image including a nasal area; using a lesion detection model to detect whether a lesion exists based on the nasal endoscopic image and at least one of the multiple block images; when the lesion is detected in the nasal endoscopic image, generating a combined image by combining the nasal endoscopic image and a computed tomography (CT) image of the nasal area; and using a neural network-based identification model to identify the nasal tumor of the lesion based on the generated combined image.

[0013] The nasal endoscope image corresponds to an overall image including the nasal cavity area, and the block images correspond to partial images of the overall image.

[0014] The extracting step includes the following steps: randomly cutting out a partial area from the entire area of ​​the nasal endoscopic image, and extracting the plurality of block images corresponding to the partial area from the nasal endoscopic image.

[0015] The lesion detection model is trained using a nasal endoscope image and a plurality of block images, and after training, it is distinguished whether the nasal endoscope image is a lesion image including a lesion area or a regular image not including a lesion area.

[0016] The lesion detection model assigns weights to texture and shape of a nasal cavity region included in at least one of a nasal endoscope image and a plurality of block images, and is cyclically learned based on the weighted texture and shape of the nasal cavity region.

[0017] The identification model is trained based on a combined image generated by combining a nasal endoscope image and a CT image of the nasal region, and after training, the lesion is distinguished as any one of nasal polyps, inverted papilloma and normal tissue.

[0018] According to one embodiment, a training device for a nasal cavity mass identification model comprises: an image preprocessing unit, which extracts a plurality of block images from a nasal endoscopic image including a nasal cavity area; a lesion detection model training unit, which uses the nasal endoscopic image and the extracted plurality of block images to train a lesion detection model based on a neural network; a detection unit, which uses the trained lesion detection model to detect whether there is a lesion in the nasal endoscopic image; a combined image generating unit, which generates a combined image by combining the nasal endoscopic image and a computerized tomograph (CT) image of the nasal cavity area when the lesion is detected in the nasal endoscopic image; and an identification model training unit, which trains a neural network-based identification model for identifying nasal cavity masses for the lesions based on the generated combined image.

[0019] The image preprocessing unit randomly cuts out a partial area from the entire area of ​​the nasal endoscopic image, thereby extracting the plurality of block images corresponding to the partial area from the nasal endoscopic image.

[0020] When the center of the lesion area is set in the nasal endoscopic image, the image preprocessing unit extracts the multiple block images corresponding to the partial area from the nasal endoscopic image by cropping the partial area based on the center of the lesion area.

[0021] The lesion detection model training unit assigns weights to the texture and shape of the nasal area included in the nasal endoscope image and at least one of the multiple extracted block images, and causes the lesion detection model to undergo curriculum learning based on the weighted texture and shape of the nasal area.

[0022] According to one embodiment, a nasal tumor identification device includes: an image preprocessing unit, which extracts multiple block images from a nasal endoscope image including a nasal area; a detection unit, which uses a lesion detection model to detect whether there is a lesion based on the nasal endoscope image and at least one of the multiple block images; a combined image generation unit, which, when the lesion is detected in the nasal endoscope image, generates a combined image by combining the nasal endoscope image and a computed tomography (CT) image of the nasal area; and a nasal tumor identification unit, which uses a neural network-based identification model to identify the nasal tumor of the lesion based on the generated combined image.

[0023] The lesion detection model assigns weights to texture and shape of a nasal cavity region included in at least one of a nasal endoscope image and a plurality of block images, and obtains curriculum learning based on the weighted texture and shape of the nasal cavity region.

[0024] Effects of the Invention

[0025] According to one embodiment, firstly, whether there is a lesion is detected based on the nasal endoscope image, and then the lesion is identified as a nasal tumor based on both the CT image and the nasal endoscope image, thereby improving the accuracy of nasal tumor identification.

[0026] According to one embodiment, nasal tumors that are usually difficult to identify are identified by using a neural network-based model, thereby achieving automation and convenience in the identification of nasal tumors and improving the accuracy of identification.

[0027] According to one embodiment, the diagnosis of nasal tumors can be assisted, helping clinicians to diagnose nasal endoscopy images, thereby avoiding unnecessary treatments through accurate diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a schematic diagram of a nasal tumor identification system according to one embodiment.

[0029] Figure 2 is a flowchart illustrating a method for training a nasal tumor identification model according to one embodiment.

[0030] Figure 3 is a flow chart illustrating a method for identifying nasal tumors according to one embodiment.

[0031] Figure 4 FIG. 4 is a diagram illustrating a process of extracting a plurality of patch images from a nasal endoscope image according to an embodiment.

[0032] Figure 5 is a diagram illustrating a lesion detection model according to another embodiment.

[0033] Figure 6 is a structural diagram of a training device according to an embodiment.

[0034] Figure 7 It is a structural diagram of a nasal tumor identification device according to one embodiment. DETAILED DESCRIPTION

[0035] The description of the specific structure or function of the embodiment is only for example, and the embodiment can be implemented in different forms. The actual implementation is not limited to the specific embodiment disclosed, and the scope of this specification includes all changes within the technical ideas described by the embodiment, their equivalents and even their substitutes.

[0036] The terms "first" or "second" may be used to describe different components, but are only used to distinguish one component from other components. For example, the first component may be named the second component, and similarly, the second component may also be named the first component.

[0037] When it is stated that one component is “connected to” another component, it may be directly connected to or in contact with the other component, or there may be another component between them.

[0038] In the absence of special instructions in the content, the singular forms "a", "an", and "the" are intended to include the plural forms as well. In this specification, the terms "include / comprise" and / or "include / comprises" are used to express the existence of the stated features, numbers, steps, operations, elements, and / or parts, but do not exclude the possibility of the existence or additional existence of one or more other features, numbers, steps, operations, elements, parts, and / or their combinations.

[0039] In the absence of other definitions, all terms, including technical or scientific terms used herein, have the common meanings understood by ordinary technicians in the field to which the examples belong. Commonly used terms as defined in dictionaries should be understood as the meanings in the relevant technical content, and should not be interpreted as idealized or overly formalized meanings if they are not clearly defined in this specification.

[0040] In the process of describing the examples with reference to the drawings, the same components are marked with the same reference numerals and repeated descriptions are omitted. In the process of describing the example embodiments, when it is determined that the detailed description of the relevant well-known structure or function will cause unnecessary confusion to the present disclosure, its detailed description is omitted.

[0041] Figure 1 is a schematic diagram of a nasal tumor identification system according to one embodiment.

[0042] The nasal tumor identification system described in this specification detects whether there is a lesion based on the nasal endoscope image. When the lesion is detected, it can identify the type of lesion based on the nasal endoscope image and the CT image. The nasal tumor identification system described in this specification can use the nasal endoscope image and the CT image to identify the nasal tumor at the same time, which can improve the accuracy of nasal tumor identification.

[0043] Reference Figure 1 The nasal tumor identification system 120 may include a lesion detection model 140 and an identification model 150. The lesion detection model 140 and the identification model 150 may be neural network models based on deep learning, and may be trained using at least one of recurrent learning, transfer learning, and supervised learning.

[0044] The lesion detection model 140 may receive an input of a nasal endoscope image 110. The nasal endoscope image 110 may be image preprocessed. The lesion detection model 140 may detect whether a lesion exists based on the nasal endoscope image 110. When the lesion detection model 140 detects the presence of a lesion based on the nasal endoscope image 110, the identification model 150 may receive an input nasal endoscope image and a nasal region CT image. Here, the nasal endoscope image and the nasal region CT image may be combined into a combined image. The identification model 150 may output an identification result 160 based on the nasal endoscope image and the nasal region CT image. The identification result 160 may distinguish the lesion into any one of nasal polyps, visceral papilloma, and normal tissue. In addition, the identification result 160 may include a probability value that the lesion is any one of nasal polyps, visceral papilloma, and normal tissue.

[0045] Figure 2 is a flowchart illustrating a method for training a nasal tumor identification model according to one embodiment.

[0046] Reference Figure 2 In step 210, the training device extracts a plurality of block images from a nasal endoscopic image including a nasal region. Here, the nasal endoscopic image may correspond to an overall image including a nasal region, and as learning data may be classified into a learning set, a tuning set, and a test set. The ratio of classifying the nasal endoscopic images into a training set, a tuning set, and a test set may be a training set 7, a tuning set 1, and a test set 2, and may be randomly divided. The nasal endoscopic images classified as the learning set may be used to train at least one of the lesion detection model and the discrimination model to be described below. The nasal endoscopic images classified as the tuning set may be used to tune parameters to improve the performance of at least one of the lesion detection model and the discrimination model trained based on the nasal endoscopic images classified as the learning set. In addition, the nasal endoscopic images classified as the test set may be used to test the performance of at least one of the lesion detection model and the discrimination model. The classification of nasal endoscopic images is not limited to the content of the examples in this article.

[0047] The block images may correspond to partial images of the whole image or partial images of the nasal endoscope image. The process by which the training device extracts a plurality of block images from the nasal endoscope image may be referred to as an image preprocessing process. In one embodiment, the training device extracts a plurality of block images corresponding to the partial regions from the nasal endoscope image by randomly cropping the partial regions from the entire region of the nasal endoscope image.

[0048] In addition, in another embodiment, when the center of the lesion area is set in the nasal endoscopic image, the training device extracts a plurality of block images corresponding to the partial area from the nasal endoscopic image by cropping the partial area based on the center of the lesion area. For example, when the center of the lesion area is set in the nasal endoscopic image based on a user input input by a medical staff or a user selection, the training device may extract a plurality of block images corresponding to the partial area from the nasal endoscopic image by cropping the partial area including the center set as the center of the lesion area.

[0049] In one embodiment, the training device may perform image processing including at least one of enlargement, reduction, rotation and shifting on at least one of the extracted block images and the nasal endoscope image, thereby achieving image preprocessing.

[0050] In step 220, the training device may train a neural network-based lesion detection model using the nasal endoscope image and the extracted multiple block images. The training device may train the lesion detection model based on at least one extracted block image and the nasal endoscope image that have undergone image processing. The training device may assign weights to the texture and shape of the nasal cavity area included in the nasal endoscope image and at least one of the extracted multiple block images. For example, the lesion detection model of the training device may be designed to learn the texture and shape of the nasal endoscope image, thereby detecting whether there is a lesion by fusing local information and global information of the nasal endoscope image. Here, the local information may include specific information obtained through local information, and the global information may include the overall shape and texture of the lesion.

[0051] The training device may assign weights to the texture and shape of the nasal region determined to be characteristic based on a predetermined criterion. The training device may enable a lesion detection model to be curriculum learned based on the weighted texture and shape of the nasal region. The training device may reflect the texture and shape of various lesions in the process of training the lesion detection model by assigning weights to the texture and shape of the nasal region.

[0052] In another embodiment, the training device can train the lesion detection model based on the block image extracted by cropping a part of the area based on the center of the lesion area, and at this time, a cyclic learning based on the lesion center can be performed. The cyclic learning based on the lesion center can train the deep learning model based on a limited medical data set (nasal endoscopy images or others) and explain the location and characteristics of the lesion.

[0053] In step 230, the training device may detect whether there is a lesion in the nasal endoscopic image using the trained lesion detection model. In one embodiment, the training device distinguishes whether the nasal endoscopic image is a lesion image including a lesion area or a regular image not including a lesion area. When it is determined that the nasal endoscopic image is a lesion image including a lesion area, the training device may detect the presence of a lesion in the nasal endoscopic image.

[0054] When a lesion is detected in the nasal endoscopic image, the training device can obtain and divide the region including the lesion from the nasal endoscopic image. In step 240, the training device generates a combined image by combining the nasal endoscopic image with a computerized tomography (CT) image of the nasal region.

[0055] In step 250, the training device trains a neural network-based discrimination model for discriminating nasal cavity masses from lesions based on the generated combined image. The discrimination model can distinguish the lesions as any one of nasal polyps, inverted papilloma, and normal tissue. In one embodiment, the training device can train the discrimination model based on at least one of transfer learning and recurrent learning using the combined image.

[0056] The training device described in this specification can train a nasal tumor identification model, which can use both three-dimensional CT images and two-dimensional nasal endoscope images to more accurately identify nasal tumors.

[0057] The attention mechanism of at least one of the trained lesion detection model and the discrimination model can be consistent with the local lesion-related areas that experienced otolaryngologists focus on during nasal endoscopy.

[0058] Figure 3 is a flow chart illustrating a method for identifying nasal tumors according to one embodiment.

[0059] Reference Figure 3In step 310, the nasal tumor identification device may extract a plurality of block images from the nasal endoscope image including the nasal region. Step 310 may be similar to Figure 2 The nasal endoscope image corresponds to the whole image including the nasal cavity area, and the block image corresponds to a partial image of the whole image.

[0060] The nasal tumor identification device can extract a plurality of block images corresponding to the partial area from the nasal endoscopic image by randomly cutting out the partial area from the entire area of ​​the nasal endoscopic image. In addition, in another embodiment, when the center of the lesion area is set in the nasal endoscopic image based on user input or user selection input by a medical staff, a plurality of block images corresponding to the partial area can be extracted from the nasal endoscopic image by cutting out the partial area including the center set as the center of the lesion area.

[0061] In step 320, the nasal tumor identification device uses a lesion detection model to detect whether a lesion exists based on the nasal endoscope image and at least one of the multiple block images. In one embodiment, the nasal endoscope image and the multiple block images can be used to train the lesion detection model. The lesion detection model assigns weights to the texture and shape of the nasal region included in the nasal endoscope image and at least one of the multiple block images, and is cyclically learned based on the weighted texture and shape of the nasal region. After training, the lesion detection model can distinguish whether the nasal endoscope image is a lesion image including a lesion area or a regular image not including a lesion area. Step 320 can be similar to Figure 2 Step 230 is described in detail in FIG.

[0062] In step 330, when a lesion is detected in the nasal endoscope image, the nasal tumor identification device generates a combined image by combining the nasal endoscope image with a nasal region computed tomography (CT) image. Step 330 may be similar to Figure 2 Step 240 is described in detail in FIG.

[0063] In step 340, the nasal tumor identification device may use a neural network-based identification model to identify pathological nasal tumors based on the generated combined image.

[0064] The identification model can be trained by combining the nasal endoscope image and the nasal area CT image in the learning data. After training, the identification model distinguishes the lesion into any one of nasal polyps, inverted papilloma and normal tissue to identify the lesion nasal tumor.

[0065] Figure 4FIG. 4 is a diagram illustrating a process of extracting a plurality of patch images from a nasal endoscope image according to an embodiment.

[0066] The training device and the nasal tumor identification device described in this specification can extract multiple block images 420 from the nasal endoscope image 410. The nasal endoscope image 410 corresponds to the whole image including the nasal region, and the block images 420 correspond to partial images of the whole image or partial images of the nasal endoscope image 410.

[0067] The training device or the nasal tumor identification device may extract a plurality of block images 420 corresponding to the partial area in the nasal endoscopic image 410 by randomly cutting out the partial area from the entire area of ​​the nasal endoscopic image 410. In addition, in another embodiment, when the center of the lesion area is set in the nasal endoscopic image 410 based on a user input or user selection input by a medical staff, the nasal tumor identification device or the training device may extract a plurality of block images 420 corresponding to the partial area from the nasal endoscopic image 410 by cutting out the partial area including the center set as the center of the lesion area.

[0068] Figure 5 is a diagram illustrating a lesion detection model according to another embodiment.

[0069] Reference Figure 5 , the lesion detection model may include a first lesion detection model 515 receiving a block image 510 extracted from a nasal endoscope image and a second lesion detection model 525 receiving a nasal endoscope image 520. Here, the block image 510 may have a resolution of, for example, 256*256, and the nasal endoscope image 520 may have a resolution of, for example, 512*512. The first lesion detection model 515 may include a convolution model and average pooling (average pooling and mean pooling), and the second lesion detection model 525 may include a convolution model and average pooling (Global average pooling).

[0070] At least one of the first lesion detection model 515 and the second lesion detection model 525 may include InceptionResNetV2 to detect lesions including any one of nasal polyps, visceral papilloma, and normal tissue from nasal endoscopic images. InceptionResNetV2 may be composed of an initial layer and residual connections, wherein the residual connections include various convolutional blocks (or convolutional filters) connected to the residual blocks. InceptionResNetV2 may prevent a gradient descent problem that may occur due to the depth of a deep learning network, and may also shorten training time.

[0071] The first lesion detection model 515 and the second lesion detection model 525 can be cyclically learned or transfer learned while sharing weights. The InceptionResNetV2 of the first lesion detection model 515 and the second lesion detection model 525 can first be trained based on ILSVRC 25 (ImageNet Large Scale Visual Recognition Competition 25). These convolution-based neural networks are not learned from scratch, but can learn features of conventional or other medical images based on other domain data sets. The first lesion detection model 515 and the second lesion detection model 525 can be trained by fine-tuning only the last layer in a state where all images included in the learning data are pre-learned.

[0072] According to the prior art, the full-resolution nasal endoscopic image includes complex patterns with lesions, organs, and tissues, which may make training too complex due to limited data sets. The lesion detection model described herein can receive a block image 510 extracted from a nasal endoscopic image 520 as input to improve the training and adjustment of the prior art. The block image 510 can correspond to half of the existing resolution of the nasal endoscopic image 520. The lesion detection model can use the full-resolution nasal endoscopic image 520 to fine-tune the parameters included in the network of the lesion detection model to compensate for the image difference between the block image 510 and the nasal endoscopic image 520.

[0073] The trained first lesion detection model 515 can receive the block image 510 and output an output value by inputting the received block image 510 into a convolution model and average pooling. In addition, the trained second lesion detection model 525 can obtain an output value by receiving a nasal endoscope image 520 and inputting the received nasal endoscope image 520 into a convolution model and average pooling.

[0074] The output value output from the first lesion detection model 515 and the output value output from the second lesion detection model 525 may be calculated as a classification result 540 through a fully connected layer (FC) 530 and a weighted Softmax function 535. For example, the classification result 540 may include a result of distinguishing the lesion as any one of nasal polyps, visceral papilloma, and normal tissue, and may include a probability value that the lesion is any one of nasal polyps, visceral papilloma, and normal tissue. The cross entropy cost function of binary classification associated with the probability value may be expressed as the following formula.

[0075] Formula 1

[0076] Loss(y,f)=-y log f-(1-y)log(1-f)

[0077] Here, f and y may represent the expected output corresponding to the inferred probability value and output value, respectively.

[0078] Figure 6 is a structural diagram of a training device according to an embodiment.

[0079] Reference Figure 6 The training device 600 may include an image preprocessing unit 610, a lesion detection model training unit 620, a detection unit 630, a combined image generation unit 640, and a discrimination model training unit 650. Here, the training device 600 may correspond to the training device described in this specification.

[0080] The image preprocessing unit 610 may extract a plurality of block images from the nasal endoscopic image including the nasal cavity area. The image preprocessing unit 610 may extract a plurality of block images corresponding to the partial area from the nasal endoscopic image by randomly cutting out the partial area from the entire area of ​​the nasal endoscopic image. In another embodiment, when the center of the lesion area is set in the nasal endoscopic image, the image preprocessing unit 610 extracts a plurality of block images corresponding to the partial area from the nasal endoscopic image by cutting out the partial area based on the center of the lesion area.

[0081] The lesion detection model training unit 620 can use the nasal endoscope image and the extracted multiple block images to train the lesion detection model based on the neural network. The lesion detection model training unit 620 assigns weights to the texture and shape of the nasal cavity area included in at least one of the nasal endoscope image and the extracted multiple block images, and causes the lesion detection model to be cyclically learned based on the weighted texture and shape of the nasal cavity area.

[0082] The detection unit 630 may detect whether a lesion exists in the nasal endoscope image using a trained lesion detection model.

[0083] When the presence of a lesion in the nasal endoscope image is detected, the combined image generating unit 640 generates a combined image by combining the nasal endoscope image with a computed tomography (CT) image of the nasal cavity region.

[0084] The discrimination model training section 650 trains a neural network-based discrimination model for discriminating nasal cavity masses from lesions based on the generated combined image.

[0085] The operations performed by the above-mentioned image preprocessing unit 610, lesion detection model training unit 620, detection unit 630, combined image generation unit 640 and identification model training unit 650 can be performed by a device including one or more processors and a memory. For example, the operations of the preprocessing unit 610, lesion detection model training unit 620, detection unit 630, combined image generation unit 640 and identification model training unit 650 can be performed by a processor executing instructions stored in a memory.

[0086] Figure 7 It is a structural diagram of a nasal tumor identification device according to one embodiment.

[0087] Reference Figure 7 The nasal tumor identification device 700 may include an image preprocessing unit 710, a detection unit 720, a combined image generating unit 730, and a nasal tumor identification unit 740. Here, the nasal tumor identification device 700 may correspond to the nasal tumor identification device described in this specification.

[0088] The image preprocessing unit 710 may extract a plurality of block images from the nasal endoscopic image including the nasal cavity area. In addition, the detection unit 720 may detect whether there is a lesion based on at least one of the nasal endoscopic image and the plurality of block images using a lesion detection model.

[0089] When a lesion is detected in the nasal endoscope image, the combined image generation unit 730 generates a combined image by combining the nasal endoscope image with a computed tomography (CT) image of the nasal area, and the nasal tumor identification unit 740 can use a neural network-based identification model to identify the diseased nasal tumor based on the generated combined image.

[0090] The operations performed by the above-mentioned image preprocessing unit 710, detection unit 720, combined image generation unit 730 and nasal tumor identification unit 740 can be performed by a device including one or more processors and a memory. For example, the operations of the image preprocessing unit 710, detection unit 720, combined image generation unit 730 and nasal tumor identification unit 740 can be performed by a processor that executes instructions stored in a memory.

[0091] The above-described device can be implemented by hardware components, software components, and / or a combination of hardware components and software components. For example, the device and components described in the embodiment can be embodied using one or more general-purpose computers or special-purpose computers such as processors, controllers, arithmetic logic units (arithmetic logic units, ALU), digital signal processors (digital signal processors), microcomputers, field programmable arrays (field programmable arrays, FPA), programmable logic units (programmable logic units, PLU), microprocessors, or any other device capable of executing and responding to instructions (instructions). The processing device can execute an operating system (OS) and one or more application software executed in the operating system. In addition, the processing device responds to the execution of the software to access, store, operate, process and generate data. For ease of understanding, a method with only one processing device is described, but a person of ordinary skill in the art should understand that the processing device can include multiple processing elements (processing element) and / or multiple types of processing elements. For example, the processing device can include multiple processors or a processor and a controller. Furthermore, other processing configurations such as parallel processors can also be included.

[0092] Software can include a computer program, code, instructions, or a combination of more than one of these, that causes a processing device to operate in a desired manner, or that, alone or collectively, commands a processing device. Software and / or data can be permanently or temporarily embody any type of device, component, physical device, virtual device, computer storage medium or device, or transmitted signal wave for interpretation by a processing device or for providing commands or data to a processing device. Software is distributed on computer systems connected by a network and can be stored or executed in a distributed manner. Software and data can be stored in one or more computer-readable storage media.

[0093] The method according to the embodiment is embodied in the form of a program command that can be executed by a variety of computer means and recorded in a computer-readable medium. The computer-readable medium can include program commands, data files, data structures, etc. in a single or combined form. The program instructions recorded in the medium can be instructions specially designed and constructed to implement the embodiment, or instructions that ordinary technicians in the field of computer software can use based on known ones. The computer-readable recording medium can include magnetic media such as hard disks, floppy disks, and tapes; optical media similar to CD-ROMs, DVDs, etc.; magneto-optical media similar to floptical disks, and hardware devices specially constructed for storing and executing program commands similar to read-only memories (ROMs), random access memories (RAMs), flash memories, etc. Examples of program instructions include not only machine language codes generated by compilers, but also high-level language codes that can be executed by computers by using interpreters, etc.

[0094] To perform the operations of the embodiments, the hardware device may be configured in such a manner that the operations are implemented with one or more software modules, and vice versa.

[0095] In summary, the embodiments are described through limited drawings, and a person skilled in the art can make various changes and modifications based on the description. For example, the described techniques are performed in a different order from the described method, and / or the described systems, structures, devices, circuits and other components are combined or combined in a different form from the described method, or replaced or substituted by other components or equivalents, and appropriate results can also be obtained.

[0096] Accordingly, other embodiments, other examples, and their equivalents are within the scope of the appended claims.

Claims

1. A training method for a nasal tumor identification model, characterized in that: The following steps are involved: extracting a plurality of segmented images from a nasal endoscopic image including a nasal cavity region; Using the nasal endoscope image and the extracted multiple block images to train a neural network-based lesion detection model; Detecting whether a lesion exists in the nasal endoscope image using the trained lesion detection model; When the lesion is detected in the nasal endoscopic image, generating a combined image by combining the nasal endoscopic image with a computed tomography (CT) image of the nasal cavity region; and A neural network-based identification model for identifying nasal tumors with respect to the lesions is trained based on the generated combined image.

2. The training method according to claim 1, characterized in that: The nasal endoscope image corresponds to an overall image including the nasal cavity area, The block images correspond to partial images of the overall image.

3. The training method according to claim 1, characterized in that: The extraction step comprises the following steps: By randomly cutting out a partial area from the entire area of ​​the nasal endoscopic image, the plurality of block images corresponding to the partial area are extracted from the nasal endoscopic image.

4. The training method according to claim 1, characterized in that: The following steps are also included: When the center of the lesion area is set in the nasal endoscopic image, the plurality of block images corresponding to the partial area are extracted from the nasal endoscopic image by cutting out the partial area based on the center of the lesion area.

5. The training method according to claim 1, characterized in that: The following steps are also included: performing image processing including at least one of enlargement, reduction, rotation and shift on at least one of the extracted block images and the nasal endoscope image, The step of training the lesion detection model comprises the following steps: The lesion detection model is trained based on at least one of the extracted block images and the nasal endoscope image that has undergone the image processing.

6. The training method according to claim 1, characterized in that: The lesion detection model distinguishes whether the nasal endoscope image is a lesion image including a lesion area or a regular image not including a lesion area.

7. The training method according to claim 1, characterized in that: The step of training the lesion detection model comprises the following steps: assigning weights to textures and shapes of nasal regions included in at least one of the nasal endoscopic image and the extracted plurality of block images; and The lesion detection model is recurrently learned based on the texture and shape of the weighted nasal region.

8. The training method according to claim 1, characterized in that: The discrimination model distinguishes the lesion as any one of nasal polyps, inverted papilloma, and normal tissue.

9. A method for identifying nasal tumors, characterized in that: The following steps are involved: extracting a plurality of segmented images from a nasal endoscopic image including a nasal cavity region; detecting whether a lesion exists based on the nasal endoscopic image and at least one of the plurality of block images using a lesion detection model; When the lesion is detected in the nasal endoscopic image, generating a combined image by combining the nasal endoscopic image with a computed tomography (CT) image of the nasal cavity region; and The pathological nasal tumor is identified based on the generated combined image using a neural network-based identification model.

10. The method for identifying nasal tumors according to claim 9, characterized in that: The nasal endoscope image corresponds to an overall image including the nasal cavity area, The block images correspond to partial images of the overall image.

11. The method for identifying nasal tumors according to claim 9, characterized in that: The extraction step comprises the following steps: By randomly cutting out a partial area from the entire area of ​​the nasal endoscopic image, the plurality of block images corresponding to the partial area are extracted from the nasal endoscopic image.

12. The method for identifying nasal tumors according to claim 9, characterized in that: The lesion detection model is trained using a nasal endoscope image and a plurality of block images, and after training, it is distinguished whether the nasal endoscope image is a lesion image including a lesion area or a regular image not including a lesion area.

13. The method for identifying nasal tumors according to claim 9, characterized in that: The lesion detection model assigns weights to textures and shapes of a nasal cavity region included in at least one of a nasal endoscope image and a plurality of block images, and is cyclically learned based on the weighted textures and shapes of the nasal cavity region.

14. The method for identifying nasal tumors according to claim 9, characterized in that: The identification model is trained based on a combined image generated by combining a nasal endoscope image and a CT image of the nasal cavity area, and after training, the lesion is distinguished as any one of nasal polyps, inverted papilloma and normal tissue.

15. A training device for a nasal tumor identification model, characterized in that: include: An image preprocessing unit extracts a plurality of block images from a nasal endoscope image including a nasal cavity area; A lesion detection model training unit, which uses the nasal endoscope image and the extracted multiple block images to train a lesion detection model based on a neural network; a detection unit, which detects whether there is a lesion in the nasal endoscope image by using the trained lesion detection model; a combined image generating unit that generates a combined image by combining the nasal endoscopic image and a computed tomography (CT) image of the nasal cavity region when the presence of the lesion is detected in the nasal endoscopic image; and A discrimination model training unit is configured to train a neural network-based discrimination model for discriminating nasal tumors based on the generated combined image.

16. The training device according to claim 15, characterized in that The image preprocessing unit randomly cuts out a partial area from the entire area of ​​the nasal endoscopic image, thereby extracting the plurality of block images corresponding to the partial area from the nasal endoscopic image.

17. The training device according to claim 15, characterized in that When the center of the lesion area is set in the nasal endoscopic image, the image preprocessing unit extracts the multiple block images corresponding to the partial area from the nasal endoscopic image by cropping the partial area based on the center of the lesion area.

18. The training device according to claim 15, characterized in that The lesion detection model training unit assigns weights to textures and shapes of nasal regions included in at least one of the nasal endoscopic image and the extracted plurality of block images, The lesion detection model is cyclically learned based on the weighted texture and shape of the nasal cavity area.

19. A nasal tumor identification device, characterized in that: include: An image preprocessing unit extracts a plurality of block images from a nasal endoscope image including a nasal cavity area; a detection unit that detects whether a lesion exists based on the nasal endoscope image and at least one of the plurality of block images using a lesion detection model; a combined image generating unit that generates a combined image by combining the nasal endoscopic image and a computed tomography (CT) image of the nasal cavity region when the presence of the lesion is detected in the nasal endoscopic image; and A nasal tumor identification unit identifies the pathological nasal tumor based on the generated combined image using an identification model based on a neural network.

20. The nasal tumor identification device according to claim 19, characterized in that: The lesion detection model assigns weights to textures and shapes of a nasal cavity region included in at least one of a nasal endoscope image and a plurality of block images, and is cyclically learned based on the weighted textures and shapes of the nasal cavity region.

Images