Method, information processing apparatus, and storage medium

Abstract

To improve a technique related to annotation of image information of a thin linear structure.SOLUTION: The method includes a step of receiving an input in which a partial structure of image information of a thin line-like structure is set as a positive region, a step of determining a peripheral region of the positive region as a negative region, and a step of training a learning model using annotation information including the positive region and the negative region.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a method, an information processing device, and a program. [Background technology]

[0002] Conventionally, a trained model that estimates a region of a thin line structure, such as a blood vessel region, is known for image information of the thin line structure. For example, Patent Document 1 describes a trained model that receives a fundus image, which is training data for a machine learning model, as input data, and outputs a blood vessel image that estimates a blood vessel region in the fundus image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2023 / 032887 Summary of the Invention [Problem to be solved by the invention]

[0004] When an image has a thin linear structure such as a blood vessel region, it is difficult to annotate the entire blood vessel region, and there is room for improvement in annotation technology for image information of such thin linear structures.

[0005] In view of the above circumstances, an object of the present disclosure is to improve annotation techniques for image information of thin line structures. [Means for solving the problem]

[0006] (1) A method according to one embodiment of the present disclosure includes: A method executed by an information processing device, receiving an input indicating that a part of the image information of the thin line structure is a positive region; determining a region surrounding the positive region as a negative region; training a learning model using annotation information including the positive regions and the negative regions; Includes.

[0007] (2) A method according to one embodiment of the present disclosure is the method described in (1), In the training step, the learning model is trained without using regions that are not defined as either positive regions or negative regions by the annotation information for learning.

[0008] (3) A method according to one embodiment of the present disclosure is a method according to (1) or (2), The peripheral region is a region within a predetermined distance from the positive region.

[0009] (4) A method according to an embodiment of the present disclosure is a method according to any one of (1) to (3), The area where the peripheral area and the positive area overlap is defined as a positive area.

[0010] (5) A method according to one embodiment of the present disclosure is the method according to (3), The predetermined distance is determined based on the area size ratio between the area size of the peripheral area and the area size of the positive area.

[0011] (6) A method according to one embodiment of the present disclosure is the method according to (5), The predetermined distance is further determined based on a label miss rate.

[0012] (7) A method according to one embodiment of the present disclosure is the method according to (6), The predetermined distance is determined so that the region size ratio is equal to or greater than a first predetermined value and the mislabeling rate is less than a second predetermined value.

[0013] (8) An information processing device according to an embodiment of the present disclosure includes: An information processing device including a control unit, The control unit Accepting an input of a part of the image information of the thin line structure as a positive region; A region surrounding the positive region is determined as a negative region; A learning model is trained using annotation information including the positive regions and the negative regions.

[0014] (9) A program according to an embodiment of the present disclosure includes: On the computer, receiving an input indicating that a part of the image information of the thin line structure is a positive region; determining a region surrounding the positive region as a negative region; training a learning model using annotation information including the positive regions and the negative regions; Execute the following. [Effects of the Invention]

[0015] According to one embodiment of the present disclosure, an annotation technique for image information of thin line structures is improved. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a block diagram illustrating a schematic configuration of an information processing device according to an embodiment of the present disclosure. [Figure 2] 10 is a flowchart illustrating an operation of an information processing device according to an embodiment of the present disclosure. [Figure 3] 10 is an example of image information of a thin line structure. [Figure 4] FIG. 1 is a conceptual diagram showing the relationship between a positive region and a negative region. [Figure 5] FIG. 10 is a conceptual diagram of an example of overwriting a negative area with a positive area. [Figure 6] FIG. 10 is a conceptual diagram relating to the size of the surrounding area and the label error rate. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, an embodiment of the present disclosure will be described. Note that in the present embodiment, the image information is assumed to be two-dimensional image information, but is not limited to this. The image information may be three-dimensional image information.

[0018] (Outline of the embodiment) First, an overview of this embodiment will be described, and details will be provided later. The information processing device 10 receives input in which a portion of image information of a thin-line structure is designated as a positive region. The information processing device 10 also determines a region surrounding the positive region (hereinafter also referred to as a surrounding region) as a negative region. The information processing device 10 then trains a learning model using information including the positive region and the negative region (hereinafter also referred to as annotation information).

[0019] As described above, according to this embodiment, based on an input in which a portion of image information of a thin line structure is designated as a positive region, the surrounding region of the positive region is determined as a negative region. Furthermore, a learning model is trained using annotation information including the positive and negative regions thus determined. In other words, according to this embodiment, it is only necessary to annotate a portion of the image information of the thin line structure, thereby streamlining the annotation work for the entire complex line structure. Therefore, annotation technology for image information of thin line structures is improved.

[0020] (Configuration of information processing device) As shown in FIG. 1, the information processing device 10 includes a control unit 11, a storage unit 12, an input unit 13, an output unit 14, and a communication unit 15.

[0021] The control unit 11 includes at least one processor, at least one dedicated circuit, or a combination thereof. The processor is a general-purpose processor such as a central processing unit (CPU) or a graphics processing unit (GPU), or a dedicated processor specialized for a specific process. The dedicated circuit is, for example, a field-programmable gate array (FPGA) or an application specific integrated circuit (ASIC). The control unit 11 executes processes related to the operation of the information processing device 10 while controlling each unit of the information processing device 10.

[0022] The storage unit 12 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or a combination of at least two of these. The semiconductor memory is, for example, a random access memory (RAM) or a read only memory (ROM). The RAM is, for example, a static random access memory (SRAM) or a dynamic random access memory (DRAM). The ROM is, for example, an electrically erasable programmable read only memory (EEPROM). The storage unit 12 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 12 stores data used in the operation of the information processing device 10 and data obtained by the operation of the information processing device 10.

[0023] The input unit 13 includes at least one input interface. The input interface is, for example, a physical key, a capacitance key, a pointing device, or a touch screen integrated with a display. The input interface may also be, for example, a sound sensor that accepts voice input, or a camera that accepts gesture input. The input unit 13 accepts an operation to input data used for the operation of the information processing device 10. The input unit 13 may be connected to the information processing device 10 as an external input device instead of being provided in the information processing device 10. Any connection method may be used, for example, a Universal Serial Bus (USB), a High-Definition Multimedia Interface (HDMI) (registered trademark), or Bluetooth (registered trademark).

[0024] The output unit 14 includes at least one output interface. The output interface is, for example, a display that outputs information as a video, or a speaker that outputs information as a sound. The display is, for example, an LCD (liquid crystal display) or an organic EL (electro luminescence) display. The output unit 14 displays and outputs data obtained by the operation of the information processing device 10. The output unit 14 may be connected to the information processing device 10 as an external output device instead of being provided in the information processing device 10. Any connection method can be used, for example, USB, HDMI (registered trademark), or Bluetooth (registered trademark).

[0025] The communication unit 15 includes at least one external communication interface. The communication interface may be either a wired communication interface or a wireless communication interface. In the case of wired communication, the communication interface is, for example, a LAN (Local Area Network) interface or a USB (Universal Serial Bus). In the case of wireless communication, the communication interface is, for example, an interface compatible with mobile communication standards such as LTE (Long Term Evolution), 4G (4th generation), or 5G (5th generation), or an interface compatible with short-range wireless communication such as Bluetooth (registered trademark). The communication unit 15 receives data used in the operation of the information processing device 10 and transmits data obtained by the operation of the information processing device 10.

[0026] The functions of the information processing device 10 are realized by executing a program according to this embodiment on a processor corresponding to the control unit 11. That is, the functions of the information processing device 10 are realized by software. The program causes a computer to execute the operations of the information processing device 10, thereby causing the computer to function as the information processing device 10. That is, the computer functions as the information processing device 10 by executing the operations of the information processing device 10 in accordance with the program.

[0027] In this embodiment, the program can be recorded on a computer-readable recording medium. The computer-readable recording medium includes non-transitory computer-readable media, such as a magnetic recording device, an optical disc, a magneto-optical recording medium, or a semiconductor memory. The program can be distributed, for example, by selling, transferring, or lending a portable recording medium, such as a DVD (digital versatile disc) or a CD-ROM (compact disc read only memory), on which the program is recorded. The program can also be distributed by storing the program in the storage of an external server and transmitting the program from the external server to another computer. The program can also be provided as a program product.

[0028] Some or all of the functions of the information processing device 10 may be implemented by a dedicated circuit equivalent to the control unit 11. In other words, some or all of the functions of the information processing device 10 may be implemented by hardware.

[0029] (Operation of information processing device) The operation of the information processing device 10 according to this embodiment will be described with reference to FIG.

[0030] Step S100: The control unit 11 of the information processing device 10 receives an input indicating that a part of the image information of the thin line structure is a positive region.

[0031] Any method can be used for the process of accepting input for designating a portion of the image information of the thin line structure as a positive region. For example, the control unit 11 may accept a portion of the image information of the thin line structure as a positive region via the input unit 13 based on a user's input operation on a pointing device or the like. Here, the user designates only a portion of the image information of the thin line structure as a positive region. In other words, the user only needs to annotate a portion of the image information of the thin line structure as a positive region.

[0032] Step S200: The control unit 11 determines the surrounding area of ​​the positive area as a negative area.

[0033] The peripheral region may be, for example, a region within a predetermined distance from the positive region. For example, the control unit 11 may perform an expansion process on the positive region at a predetermined expansion rate, and the expanded region may be the peripheral region. The expansion rate may be, for example, 5.43 times. Note that the expansion rate is not limited to this, and any value may be set.

[0034] In this embodiment, the image information of a thin line structure that is the subject of the present invention is, for example, an image of a blood vessel region in a fundus image, but is not limited to this. FIG. 3 shows an example of image information 20. Image information 20 includes thin line structures 21 and 22. Structure 22 is a thin line structure located at the back of the paper in three-dimensional space and is indicated by a dashed line. FIG. 4 is a conceptual diagram showing the relationship between positive and negative regions. Image information 100 includes regions 101, 102, and 103. Region 101 is an area annotated as a positive area. Region 102 is a peripheral area. Therefore, region 102 is determined to be a negative area. Region 103 is neither a positive area nor a negative area. In other words, region 103 is an undefined area.

[0035] Step S300: The control unit 11 trains a learning model using annotation information including positive and negative regions.

[0036] A learning model is a model created by machine learning using a machine learning algorithm. The learning model may be, for example, a machine learning model built based on a decision tree. Examples of machine learning models built based on a decision tree include, but are not limited to, Light GBM and XGBoost. Alternatively, the learning model may be a model generated based on a machine learning algorithm such as a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), or other deep learning. A learned model is generated by training the learning model with annotation information. When image information of a thin line structure is input to the learned model, information identifying the thin line structure in the image information is output. In other words, the learned model can estimate the location of the thin line structure in the image information.

[0037] As described above, the information processing device 10 according to this embodiment accepts input in which a portion of image information of a thin-line structure is designated as a positive region. The information processing device 10 also determines the surrounding region of the positive region as a negative region. The information processing device 10 then trains a learning model using annotation information including the positive region and the negative region.

[0038] According to this configuration, based on an input that designates a portion of image information of a thin-line structure as a positive region, the surrounding region of the positive region is determined as a negative region. Furthermore, a learning model is trained using annotation information including the positive and negative regions thus determined. In other words, according to this embodiment, it is only necessary to annotate a portion of the image information of the thin-line structure, thereby streamlining the annotation work for the entire complex linear structure. Therefore, annotation technology for image information of thin-line structures is improved.

[0039] (Example) The recall results (hereinafter referred to as "Recall results") obtained by training with the method according to this embodiment and the method according to the comparative example are shown below, using multiple test data (Data 1, Data 2, and Data 3) for verification related to images of neurons as image information of thin-line structures. The method according to the comparative example is a method in which all areas not annotated as positive regions are defined as negative regions and a learning model is trained. Data 1, Recall result learned by the method according to this embodiment: 0.752 Data 2, Recall result learned by the method according to this embodiment: 0.824 Data 3, Recall result learned by the method according to this embodiment: 0.774 Data 1, Recall result of training using the method related to the comparison example: 0.027 Data 2, Recall result of training using the method related to the comparison example: 0.683 Data 3, Recall result of training using the method related to the comparison example: 0.558 As shown in these results, for all test data, the method according to this embodiment had higher Recall results than the method according to the comparative example, demonstrating that the annotation technology according to this embodiment is superior in terms of accuracy.

[0040] Although the present disclosure has been described based on the drawings and examples, it should be noted that those skilled in the art may make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each component or step can be rearranged so as not to be logically inconsistent, and multiple components or steps can be combined or divided into one.

[0041] For example, the control unit 11 may train the learning model without using regions that are not defined as either positive or negative regions by the annotation information for learning. In other words, the control unit 11 may train the learning model without taking into account regions that are not defined as either positive or negative regions by the annotation information in the loss calculation during learning. In other words, for example, when the image information 100 shown in FIG. 3 is training data, only regions 101 and 102 may be used for learning. In this case, region 103 is not used for learning. In this way, it is possible to suppress the adverse effects of undefined regions on learning of the learning model.

[0042] For example, an area where a peripheral area and a positive area overlap may be defined as a positive area. In other words, an area where a peripheral area and a positive area overlap may be overwritten by the positive area. FIG. 5 shows a conceptual diagram of an example of overwriting a negative area with a positive area. In image information 110 in FIG. 5, area 111 is a positive area, and area 112 is a peripheral area, i.e., a negative area. Here, area 113 is a negative area sandwiched between adjacent positive areas. In this case, if the peripheral area is defined as being within a predetermined distance from the positive area, the peripheral area and the positive area overlap. In this case, the overlapping portion of the peripheral area and the positive area may be defined as a positive area. In other words, if the peripheral area encroaches on an adjacent positive area, a positive area annotation may be superimposed on the encroached portion. By defining the overlapping area as a positive area in this way, input accepted as a positive area can be processed with priority.

[0043] For example, in this embodiment, the size of the peripheral region may be adjusted as appropriate. For example, if the peripheral region is defined as a region within a predetermined distance from the positive region, the predetermined distance may be determined as appropriate. Similarly, if the peripheral region is defined as a region enlarged by expanding the positive region at a predetermined expansion rate, the expansion rate may be determined as appropriate. Such a predetermined distance or expansion rate may be determined based on the ratio of the region sizes of the positive region and the peripheral region (negative region) (hereinafter also referred to as the region size ratio). The region size ratio may be determined, for example, by the following mathematical formula (1): (Area size ratio) = (Area size of negative area) / (Area size of positive area) (1)

[0044] If the image information is a two-dimensional image, the region size ratio may be determined by the ratio of pixels in a positive region to pixels in a negative region in the image information. In other words, if the image information is a two-dimensional image, the region size ratio may be determined by the area ratio of the positive region to the negative region in the image information. If the image information is a three-dimensional image, the region size ratio may be determined by the ratio of voxels in a positive region to voxels in a negative region in the image information. In other words, if the image information is a three-dimensional image, the region size ratio may be determined by the volume ratio of the positive region to the negative region in the image information. For example, the predetermined distance may be determined so that the region size ratio is equal to or greater than a first predetermined value.

[0045] The predetermined distance or expansion rate may be determined based on the mislabeling rate. The mislabeling rate is an index determined by the ratio of the number of mislabeled pixels to the number of pixels in the negative region of the image information. The mislabeling rate may be determined, for example, by the following formula (2): (Mislabeling rate) = (Number of mislabeled pixels) / (Number of pixels in negative regions) (2) Here, mislabeled pixels are pixels in a negative region in a certain image that are estimated to be in a positive region as a result of inference based on a trained model. For example, the predetermined distance or expansion rate may be determined so that the mislabeling rate is less than a second predetermined value. FIG. 6 shows a conceptual diagram relating to the size of a surrounding region and the mislabeling rate. In image information 120 in FIG. 6, region 121 is a positive region, and region 122 is a surrounding region, i.e., a negative region. Region 123 is an unannotated region that should be considered a positive region. If the size of the surrounding region is too large, region 123 will be annotated as a negative region, which will increase the mislabeling rate. Therefore, as shown in FIG. 6, the predetermined distance or expansion rate may be determined so that the mislabeling rate is less than a second predetermined value. [Explanation of symbols]

[0046] 10. Information processing equipment 11 Control section 12 Storage section 13 Input section 14 Output section 15 Communications Department 100 Image Information 101 areas 102 areas 103 areas 110 Image Information 111 areas 112 areas 113 areas 120 Image Information 121 areas 122 areas 123 areas

Claims

1. A method executed by an information processing device, receiving an input indicating that a part of the image information of the thin line structure is a positive region; determining a region surrounding the positive region as a negative region; training a learning model using annotation information including the positive regions and the negative regions; A method comprising:

2. 10. The method of claim 1, In the training step, the learning model is trained without using regions that are not defined as positive regions or negative regions by the annotation information for learning.

3. 10. The method of claim 1, The method, wherein the peripheral region is a region within a predetermined distance from the positive region.

4. 10. The method of claim 1, The method of claim 1, wherein the overlapping area of ​​the surrounding area and the positive area is determined as a positive area.

5. 4. The method of claim 3, The method of claim 1, wherein the predetermined distance is determined based on an area size ratio between an area size of the peripheral area and an area size of the positive area.

6. 6. The method of claim 5, The method, wherein the predetermined distance is further determined based on a mislabeling rate.

7. 7. The method of claim 6, The method, wherein the predetermined distance is determined such that the region size ratio is greater than or equal to a first predetermined value and the mislabeling rate is less than a second predetermined value.

8. An information processing device including a control unit, The control unit Accepting an input of a part of the image information of the thin line structure as a positive region; A region surrounding the positive region is determined as a negative region; Training a learning model using annotation information including the positive region and the negative region. Information processing device.

9. On the computer, receiving an input indicating that a part of the image information of the thin line structure is a positive region; determining a region surrounding the positive region as a negative region; training a learning model using annotation information including the positive regions and the negative regions; A program that executes the following.

Images