Production device, system, production method, image processing method, and program

Abstract

To solve the problem in which conventionally, it has been difficult to obtain information for accurately detecting an area of a purpose category from a target image including two or more categories.SOLUTION: An image processing device includes: a useful key area information storage unit that stores one or more pieces of useful key area information; an image division unit that divides a target image and acquires two or more small-image fragments; a key area determination unit that determines one or more key areas for each of the two or more small-image fragments acquired by the image division unit; a small-image fragment information amount acquisition unit that inspects whether key area information about each of the one or more key areas is similar to each piece of useful key area information stored in the useful key area information storage unit, and acquires a small-image fragment information amount according to the inspection result for each of the two or more small-image fragments; an image construction unit that constructs an output image that reflects the two or more small-image fragment information amounts; and an image output unit that outputs the output image. The image processing device allows an image for accurately detecting an area of a purpose category to be obtained from the target image including two or more categories.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an image processing device and the like that detects an area of ​​a desired category from a target image. [Background technology]

[0002] Conventionally, there has been a technique related to a classifier that classifies an area into two values ​​indicating whether or not it is a cancer area (see, for example, Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] IEEE TRANSACTION MEDICAL IMAGING VOL.38, NO.2 FEBRUARY 2019, From Detection of Individual Metastases to Classification of Lymph Node Status at the Patient Level: The CAMELYON17 Challenge Summary of the Invention [Problem to be solved by the invention]

[0004] However, in conventional techniques, it has been difficult to obtain information for detecting an area of ​​a target category with high accuracy from a target image that includes two or more categories.

[0005] More specifically, conventional technology could only identify the binary region as cancerous or not, forcing doctors to look at detected regions; however, false positives and non-detections were inevitable, so doctors were often forced to diagnose regions that could be ignored based on medical experience. [Means for solving the problem]

[0006] The useful key region information production device of the first invention is a useful key region information production device comprising an image storage unit in which images are stored, a key region determination unit which determines one or more key regions which satisfy a key condition from the image, an information amount acquisition unit which acquires an amount of information which is information on the ability to determine a region of a target category and is information on each of the one or more key regions determined by the key region determination unit, a useful key region determination unit which determines one or more useful key regions which satisfy a selection condition for selecting a useful key region using the amount of information acquired by the information amount acquisition unit, and a storage unit which accumulates useful key region information on the one or more useful key regions determined by the useful key region determination unit.

[0007] With this configuration, it is possible to acquire useful information for obtaining information for detecting an area of ​​a target category with high accuracy from a target image that includes two or more categories.

[0008] In addition, the production apparatus of the second invention, compared to the first invention, further includes a reference image storage unit in which one or more target reference images which are images including an area of ​​the target category and one or more other target reference images which are images not including an area of ​​the target category are stored, and the information amount acquisition unit performs a first inspection which is an inspection as to whether or not an area corresponding to each of the one or more key areas determined by the key area determination unit is present in each of the one or more target reference images, and a second inspection which is an inspection as to whether or not an area corresponding to each of the one or more key areas determined by the key area determination unit is present in each of the one or more other reference images, and acquires the information amount of each of the one or more key areas using the results of the first inspection and the second inspection.

[0009] With this configuration, it is possible to acquire useful information for obtaining information for detecting an area of ​​a target category with high accuracy from a target image that includes two or more categories.

[0010] Furthermore, the production apparatus of the third invention is a production apparatus in which, compared to the first or second invention, the useful key area determination unit uses the amount of information acquired by the information amount acquisition unit to acquire a discrimination value of each of one or more key areas, and determines one or more useful areas whose discrimination value satisfies the selection condition.

[0011] With this configuration, it is possible to acquire useful information for obtaining information for detecting an area of ​​a target category with high accuracy from a target image that includes two or more categories.

[0012] In addition, the image processing device of the fourth invention is an image processing device comprising a useful key region information storage unit in which one or more useful key region information is stored, a target image storage unit in which a target image that is the image to be processed is stored, an image division unit in which the target image is divided and two or more small image pieces are acquired, a key region determination unit in which one or more key regions are determined for each of the two or more small image pieces acquired by the image division unit, a small image piece information amount acquisition unit in which the key region information for each of the one or more key regions determined by the key region determination unit is similar to each of the useful key region information stored in the useful key region information storage unit and an output unit in which information reflecting the two or more small image piece information amounts acquired by the small image piece information amount acquisition unit is output.

[0013] With this configuration, it is possible to obtain information for detecting an area of ​​a target category with high accuracy from a target image that includes two or more categories.

[0014] In addition, compared to the fourth invention, the image processing device of the fifth invention is an image processing device that further includes an image construction unit that constructs an output image reflecting two or more small image piece information amounts acquired by the small image piece information amount acquisition unit, and the information reflecting the two or more small image piece information amounts acquired by the small image piece information amount acquisition unit is the output image.

[0015] With this configuration, an image for detecting an area of ​​a target category with high accuracy from a target image including two or more categories can be obtained.

[0016] In addition, the image processing device of the sixth invention, compared to the fourth or fifth invention, further includes a small image piece identification value acquisition unit that acquires a small image piece identification value, which is the identification value of the small image piece, using two or more small image piece information amounts acquired by the small image piece information amount acquisition unit, and the output unit outputs information reflecting the two or more small image piece identification values ​​acquired by the small image piece identification value acquisition unit for the target image, in addition to or instead of the two or more small image piece information amounts.

[0017] With this configuration, it is possible to obtain information for detecting an area of ​​a target category with higher accuracy from a target image that includes two or more categories.

[0018] In addition, the image processing device of the seventh invention is an image processing device in which, compared to any one of the fourth to sixth inventions, the small image piece information amount acquisition unit, when a change in the similarity conditions is accepted, checks whether the key region information for each of one or more key regions determined by the key region determination unit is similar to each useful key region information stored in the useful key region information storage unit based on the changed similarity conditions, acquires a new small image piece information amount for each of two or more small image pieces according to the inspection results, and the output unit outputs information reflecting the new small image piece information amount.

[0019] With this configuration, the output image can be dynamically changed in response to changes in the accepted similarity conditions.

[0020] Furthermore, the image processing device of the eighth invention is an image processing device in which, compared to any of the fourth to seventh inventions, the useful key region information storage unit stores one or more target useful key region information and one or more other useful key region information, the small image piece information amount acquisition unit checks whether the key region information for each of the one or more key regions determined by the key region determination unit is similar to each target useful key region information stored in the useful key region information storage unit, and acquires a target small image piece information amount according to the inspection result for each of two or more small image pieces, and checks whether the key region information for each of the one or more key regions determined by the key region determination unit is similar to each other useful key region information stored in the useful key region information storage unit, and acquires a other small image piece information amount according to the inspection result for each of the two or more small image pieces, and the output unit outputs information reflecting the target small image piece information amount and the other small image piece information amount acquired by the small image piece information amount acquisition unit for each small image piece for the target image.

[0021] With this configuration, it is possible to obtain information for detecting an area of ​​a target category with higher accuracy from a medical image including two or more categories.

[0022] In addition, the image processing device of the ninth invention is the image processing device of any one of the fourth to eighth inventions, further comprising a detection signal acquisition unit that compares the small image piece information amount acquired by the small image piece information amount acquisition unit with a threshold, and acquires a detection signal when the small image piece information amount is greater than the threshold; The output unit is an image processing device that outputs a detection signal.

[0023] With this configuration, it is possible to obtain information for detecting an area of ​​a target category with higher accuracy from a medical image including two or more categories.

[0024] The image processing device of the tenth aspect of the present invention is the image processing device according to any one of the fourth to ninth aspects of the present invention, in which the target image is a medical image.

[0025] With this configuration, an image for detecting an area of ​​a target category (for example, cancer) with high accuracy from a medical image including two or more categories can be obtained. Effect of the Invention

[0026] According to the image processing device of the present invention, an image for detecting an area of ​​a target category with high accuracy from a target image including two or more categories can be obtained. [Brief description of the drawings]

[0027] [Figure 1] Block diagram of production device 1 according to embodiment 1. [Diagram 2] Block diagram of the image processing device 2 [Diagram 3] A flowchart illustrating an example of the operation of the production device 1. [Figure 4] A flowchart for explaining an example of the information amount acquisition process [Diagram 5] A flowchart illustrating an example of the discrimination value acquisition process [Figure 6] A flowchart for explaining an example of the operation of the image processing device 2. [Figure 7] FIG. 11 is a diagram for explaining an example of processing performed by the key region determination unit 131. [Figure 8] FIG. 11 is a diagram for explaining an example of processing performed by the key region determination unit 131. [Figure 9] FIG. 11 is a diagram for explaining an example of processing performed by the key region determination unit 131. [Figure 10] FIG. 11 is a diagram for explaining an example of processing performed by the key region determination unit 131. [Figure 11] FIG. 11 is a diagram for explaining an example of processing performed by the key region determination unit 131. [Figure 12] FIG. 13 is a diagram for explaining the concept of correspondence between the key region information and the purpose useful key region information. [Figure 13] FIG. 13 is a diagram for explaining an example of processing by the small image piece information amount acquisition unit 233. [Figure 14] FIG. 13 is a diagram for explaining an example of processing by the small image piece information amount acquisition unit 233. [Figure 15] FIG. 13 is a diagram for explaining an example of processing by the small image piece information amount acquisition unit 233. [Figure 16] FIG. 13 is a diagram for explaining an example of processing by the small image piece discrimination value acquisition unit 234. [Figure 17] Overview of the computer system according to the above embodiment. [Figure 18] Block diagram of the computer system DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] Hereinafter, an embodiment of an image processing device and the like will be described with reference to the drawings. Note that, in the embodiments, components with the same reference numerals perform similar operations, so that repeated description may be omitted.

[0029] (Embodiment 1) In this embodiment, a description will be given of a useful key region information producing device that determines a key region from an image, obtains the amount of information of the key region, determines a useful key region using the amount of information, and accumulates the useful key region information. Note that in this embodiment, it is preferable to obtain a discrimination value using the amount of information, and determine a useful key region using the discrimination value based on the amount of information.

[0030] In addition, in this embodiment, we will describe an image processing device that divides an image to be processed, determines one or more key regions for each of two or more small image pieces obtained, determines whether the key region information of each of the one or more key regions is similar to useful key region information, and constructs and outputs an output image including images of each small image piece according to the determination result.

[0031] The output image including the image of each small image piece according to the judgment result is, for example, an output image reflecting the amount of information for each small image piece according to the number of similar useful key region information. Also, the output image including the image of each small image piece according to the judgment result is, for example, an output image reflecting the discrimination value obtained using the amount of information for each small image piece. Also, the output image including the image of each small image piece according to the judgment result is, for example, an output image reflecting the amount of information and the discrimination value.

[0032] 1 is a block diagram of a production device 1 according to the present embodiment. The production device 1 is a device that acquires and stores useful key region information, which will be described later. The useful key region information is usually information that is used by an image processing device, which will be described in the second embodiment.

[0033] The production apparatus 1 includes a first storage unit 11, a first reception unit 12, a first processing unit 13, and a first output unit 14. The first storage unit 11 includes an image storage unit 111, a reference image storage unit 112, a condition storage unit 113, and a useful key region information storage unit 114. The first processing unit 13 includes a key region determination unit 131, an information amount acquisition unit 132, a useful key region determination unit 133, and an accumulation unit 134.

[0034] 2 is a block diagram of the image processing device 2 in this embodiment. The image processing device 2 includes a second storage unit 21, a second reception unit 22, a second processing unit 23, and a second output unit 24. The second storage unit 21 includes a useful key region information storage unit 114, and a target image storage unit 211. The second processing unit 23 includes an image division unit 231, a key region determination unit 232, a small image piece information amount acquisition unit 233, a small image piece discrimination value acquisition unit 234, and an image construction unit 235. The second output unit 24 includes an image output unit 241.

[0035] The production device 1 and the image processing device 2 may be integrated into one device. Furthermore, the processes realized by the production device 1 and the image processing device 2 may be shared and performed by three or more devices.

[0036] Various types of information are stored in the first storage unit 11 constituting the production device 1. The various types of information include, for example, images (to be described later), judgment conditions (to be described later), and useful key region information (to be described later).

[0037] The image storage unit 111 stores one or more images. The images here are images that are the source of acquiring useful key region information, which will be described later. The images are, for example, medical images. The images are, for example, fMRI images, PET images, ultrasound images, etc., and the types and data structures thereof are not important.

[0038] The reference image storage unit 112 stores two or more reference images. A reference image is an image used when acquiring the amount of information of a key region. Each of the two or more reference images corresponds to one of two or more categories. Each of the two or more reference images corresponds to a category identifier that identifies the category. For example, when the category is "cancer" or "normal", each of the two or more reference images is an image of "cancer" or an image that is "normal".

[0039] A key region is a region in an image that has a large amount of information. For example, a key region is a key point obtained using a SIFT feature amount obtained from an image. Note that a region is one pixel or a set of two or more pixels. The key region may be a small region that is useful for distinguishing the categories of a target image, and may be, for example, a mosaic feature image of the Viola-Jones method used for pedestrian and vehicle recognition.

[0040] A category is a type of a partial area in an image. For example, a category is a "cancer" area or an area of ​​"normal" cells. The number of categories may be two or more. When there are three or more categories, for example, "cancer," "liver cirrhosis," and "normal." Here, the category to be detected (for example, a "cancer" area) is appropriately referred to as the target category. Also, categories other than the detection target (for example, a "normal" area) are appropriately referred to as other categories.

[0041] The reference image is, for example, a target reference image or an other reference image. The target reference image is an image of a target category (e.g., an image consisting of a "cancer" region) and is a reference image corresponding to the target category. The other reference image is an image of a different category (e.g., an image consisting of a "normal" region) and is a reference image corresponding to the different category.

[0042] One or more conditions are stored in the condition storage unit 113. The conditions are, for example, key conditions, selection conditions, and similarity conditions, which will be described later.

[0043] The key condition is a condition for determining a key region from an image, for example, a condition for determining that a key point is obtained using SIFT features obtained from an image.

[0044] The selection condition is a condition for selecting a useful key region, which will be described later, from the key region. The selection condition is, for example, that the discrimination value, which will be described later, is equal to or greater than a threshold value, or that the discrimination value, which will be described later, is greater than a threshold value.

[0045] One or more pieces of useful key region information are stored in the useful key region information storage unit 114. The useful key region information is information stored in the storage unit .

[0046] The useful key region information is information related to a useful key region. The useful key region information is, for example, a feature vector that is a collection of two or more features of a useful key region. The two or more features constituting the feature vector include, for example, one or more SIFT features obtained from an image. The two or more features constituting the feature vector include, for example, one or more HOG features obtained from an image. The useful key region is a region within the key region that is useful for identifying a region of a target category (for example, a cancer region).

[0047] It is preferable that one or more pieces of purpose useful key region information and one or more pieces of other useful key region information are stored in the useful key region information storage unit 114. Note that useful key region information of a useful key region having a high ability to distinguish a purpose category is referred to as purpose useful key region information. Also, useful key region information of a useful key region having a high ability to distinguish other categories is referred to as other useful key region information.

[0048] The first reception unit 12 receives various instructions and information. The various instructions and information are, for example, a start instruction, an image, a reference image, and a similarity condition. The start instruction is an instruction to start obtaining useful key region information.

[0049] The acceptance of the similarity condition may be the acceptance of a threshold value for determining whether or not two vectors are similar. In addition, the acceptance of the similarity condition may be, for example, the acceptance of a threshold value that is a numerical value, the acceptance of a threshold value using a slider bar, or the like, and the acceptance method is not limited.

[0050] It should be noted that reception here is a concept that includes the reception of information input from input devices such as a keyboard, mouse, or touch panel, the reception of information transmitted via a wired or wireless communication line, and the reception of information read from recording media such as an optical disk, a magnetic disk, or a semiconductor memory.

[0051] Furthermore, the means for inputting various instructions and information may be any means, such as a touch panel, a keyboard, a mouse, or a menu screen.

[0052] The first processing unit 13 performs various types of processing. The various types of processing are, for example, processing performed by a key region determination unit 131, an information amount acquisition unit 132, a useful key region determination unit 133, and a storage unit .

[0053] The key region determination unit 131 determines one or more key regions that satisfy the key condition of the condition storage unit 113 from an image. The image is the image of the image storage unit 111. Determining a key region means acquiring key region information and acquiring information that identifies the key region (for example, one or more coordinate values ​​in the image). The key region information is information related to the key region. The key region information is, for example, a feature vector that is a collection of two or more feature amounts of the key region. The two or more feature amounts that constitute the feature vector include, for example, one or more SIFT feature amounts acquired from the image. The two or more feature amounts that constitute the feature vector include, for example, one or more HOG feature amounts acquired from the image.

[0054] The key region determination unit 131 determines one or more key points from an image. Since the process of determining key points from an image is a known technique, a detailed description thereof will be omitted here. In addition, the key region determination unit 131 acquires key region information of the key points from the target image, for example.

[0055] The information amount acquisition unit 132 acquires the information amount of each of the one or more key regions determined by the key region determination unit 131 .

[0056] The information content is information about the key domain. The information content is information about the ability to determine the domain of the target category.

[0057] The information amount acquiring unit 132, for example, performs a first test and a second test, and acquires an information amount of each of one or more key regions using a result of the first test and a result of the second test.

[0058] The first check is a check as to whether or not a region corresponding to each of the one or more key regions determined by the key region determination unit 131 exists in each of the one or more target reference images.

[0059] The second check is a check as to whether or not a region corresponding to each of the one or more key regions determined by the key region determination unit 131 exists in each of the one or more other reference images.

[0060] In addition, a region corresponding to a key region is a region for region information similar to the key region information. A region corresponding to a key region is, for example, a region for region information that satisfies a similarity condition with the key region information. The similarity condition is that the similarity between two pieces of region information (for example, vectors) is equal to or greater than a threshold value. The similarity condition is stored in the condition storage unit 113.

[0061] For example, as a result of the first inspection, the more target reference images have an area corresponding to a key region, the larger the amount of information acquiring unit 132 acquires for that key region.

[0062] For example, the more other reference images have an area corresponding to the key region as a result of the second inspection, the larger the amount of information acquired by the information amount acquisition unit 132 for that key region. For example, the more other reference images have an area corresponding to the key region as a result of the second inspection, the smaller the amount of information acquired by the information amount acquisition unit 132 for that key region may be.

[0063] The information amount acquisition unit 132 acquires, for example, for each of one or more target reference images, region information that is most similar to the key region information, and acquires the information amount by an increasing function (for example, addition) using the similarity as a parameter.

[0064] The information amount acquisition unit 132 acquires, for example, for each of one or more other reference images, region information that is most similar to the key region information, and acquires the information amount by an increasing function with the similarity as a parameter. The information amount acquisition unit 132 acquires, for example, for each of one or more other reference images, region information that is most similar to the key region information, and acquires the information amount by a decreasing function (e.g., subtraction) with the similarity as a parameter.

[0065] The information amount acquisition unit 132 calculates the information amount corresponding to the key region using, for example, the following formulas 1, 2, and 3.

[0066]

number

[0067]

number

[0068]

number

[0069] Here, in the above formula, "K i " "K i’ " is the i-th key region (e.g., the i-th key point). "h(K i ) is the amount of information in the i-th key region. "C" and "C'" are categories. For example, "cancer" is "C=1" and "normal" is "C=2". Also, "p(K i ,C)" is the key area ?? ?? "q(C)" is the probability distribution for category C of the entire key region. In Equation 3, "q(C)" is the probability that when a reference image is selected, it is an image of category C. "N c " is the number of categories of interest. For example, in the case of distinguishing between cancer and normal, "N c =2". N(K i ,C)” is the number of images in category C that have a key region similar to the key region.

[0070] The information amount acquiring unit 132 may calculate “q(C)” using Equation 4 or Equation 5 instead of Equation 3.

[0071]

number

[0072]

number

[0073] The method for calculating "q(C)" using formulas 4 and 5 is to calculate the key area "K i In formulas 4 and 5, instead of "n(C)", we use the number of images with key regions similar to the key region among the reference images of the category, which is the sum of the number of images with key regions similar to the key region for all key regions, "N k Using "(C)" (see Equation 4), the information amount acquiring unit 132 calculates "q(C)" (see Equation 5).

[0074] The useful key region determination unit 133 uses the amount of information acquired by the amount of information acquisition unit 132 to determine one or more useful key regions that satisfy a selection condition for selecting a useful key region.

[0075] The useful key region determination unit 133, for example, uses the amount of information acquired by the amount of information acquisition unit 132 to acquire the discrimination value of one or more key regions, and determines one or more useful regions whose discrimination value satisfies the selection condition. Note that the discrimination value is information regarding the ability to distinguish the target category. Note that the amount of information used when acquiring the discrimination value of one key region is the amount of information of that one key region.

[0076] The selection condition may be, for example, that the discrimination value is equal to or greater than a threshold, that the discrimination value is greater than a threshold, that the discrimination value is one of the top N key regions, or the like.

[0077] The useful key region determination unit 133 calculates the discrimination value “v(K i ,C) is calculated.

[0078]

number

[0079] Here, if the possible categories C are "cancer" and "normal," the useful key region "K i The discrimination value v(K i , Gun) can be calculated using the following formula 7.

[0080]

number

[0081] For example, the larger the target reference information, the larger the discrimination value that the useful key region determination unit 133 acquires. For example, the useful key region determination unit 133 acquires the discrimination value by an increasing function with the target reference information as a parameter. Note that the target reference information is information on a target reference image having an area corresponding to a key region. The target reference information is, for example, a number calculated by an increasing function with the number of target reference images having an area corresponding to a key region as a parameter. The target reference information is, for example, the above-mentioned "p(K i , objective category) (Equation 2).

[0082] For example, the smaller the other reference information is, the larger the discrimination value that the useful key region determination unit 133 acquires. For example, the useful key region determination unit 133 acquires the discrimination value by a decreasing function with the other reference information as a parameter. Note that the other reference information is information on other reference images having an area corresponding to the key region. The other reference information is, for example, a number calculated by an increasing function with the number of other reference images having an area corresponding to the key region as a parameter. The other reference information is, for example, the above-mentioned "p(K i , other categories) (Equation 2).

[0083] The useful key region determination unit 133 may determine one or more useful regions whose amount of information acquired by the information amount acquisition unit 132 satisfies a selection condition. In this case, the selection condition is, for example, that the amount of information is equal to or greater than a threshold value, or that the amount of information is greater than the threshold value. Note that the amount of information in this case is larger as the number of target reference images in which an area corresponding to the key region exists increases, and is smaller as the number of other reference images in which an area corresponding to the key region exists increases.

[0084] In the above, the useful key region determination unit 133 determines a useful key region having a high ability to distinguish the target category. However, the useful key region determination unit 133 may determine a useful key region having a high ability to distinguish other categories. If the useful key region determination unit 133 exchanges the target category with the other categories and performs the above-mentioned process, the useful key region determination unit 133 can determine a useful key region having a high ability to distinguish other categories. When there are three or more categories, it is preferable that the useful key region determination unit 133 determines, for each category, a useful key region having a high ability to distinguish the category.

[0085] The accumulation unit 134 accumulates useful key region information relating to one or more useful key regions determined by the useful key region determination unit 133. The accumulation unit 134 accumulates the useful key region information in the useful key region information storage unit 114, for example.

[0086] It is preferable that the storage unit 134 stores the useful key region information in association with the amount of information acquired by the information amount acquisition unit 132. It is also preferable that the storage unit 134 stores the useful key region information in association with the discrimination value acquired by the useful key region determination unit 133.

[0087] It is preferable that the storage unit 134 stores one or more pieces of purpose useful key region information and one or more pieces of other useful key region information. In this case, it is preferable that the storage unit 134 stores the useful key region information in association with a category identifier. The number of categories may be three or more.

[0088] The first output unit 14 outputs various information, such as useful key region information and information indicating that the processing has ended.

[0089] Here, output is a concept that includes display on a display, projection using a projector, printing on a printer, sound output, transmission to an external device, storage on a recording medium, and handing over the processing results to other processing devices or other programs, etc.

[0090] Various types of information are stored in the second storage unit 21 constituting the image processing device 2. The various types of information are, for example, a target image and useful key region information, which will be described later. Note that it is preferable that the one or more pieces of useful key region information in the useful key region information storage unit 114 are useful key region information acquired by the production device 1.

[0091] The target image storage unit 211 stores one or more target images. The target image is an image to be processed. The target image is an image that is the source of an output image described later. The target image is, for example, a medical image. The target image is, for example, a medical image including a cancer region. The target image is, for example, an fMRI image, a PET image, an ultrasound image, etc., and the type does not matter. The target image is, for example, an image such as TIFF, PNG, JPEG, etc., but the data structure, etc. does not matter.

[0092] The second reception unit 22 receives various instructions and information. The various instructions and information are, for example, a start instruction, a target image, useful key region information, and a similarity condition to be changed. The start instruction is an instruction to start output processing of an output image.

[0093] The means for inputting various instructions and information may be any means, such as a touch panel, a keyboard, a mouse, or a menu screen.

[0094] The second processing unit 23 performs various types of processing, such as processing performed by an image dividing unit 231, a key region determining unit 232, a small image piece information amount acquiring unit 233, a small image piece discrimination value acquiring unit 234, and an image composing unit 235.

[0095] The second processing unit 23, for example, compares the amount of small image piece information acquired by a small image piece information amount acquisition unit 233 described below with a threshold value, and acquires a detection signal when the amount of small image piece information is greater than the threshold value. Such detection signal acquisition processing may be performed by a detection signal acquisition unit (not shown). The detection signal is, for example, information indicating that a small image piece falls into a target category. The threshold value is stored in the second storage unit 21.

[0096] The image division unit 231 divides the target image and obtains two or more small image pieces. The method of dividing the target image is not important. A small image piece is one part of the result of dividing the target image. It is preferable that the small image piece is a fixed size (for example, a size of x dots by y dots (x and y are natural numbers)), but it does not have to be a fixed size. The size of the small image piece is also not important. A small image piece has two or more pixels. Also, it is preferable that the small image piece is rectangular, but the shape is not important.

[0097] The key region determination unit 232 determines one or more key regions for each of the two or more small image pieces acquired by the image division unit 231. The process by which the key region determination unit 232 determines a key region from an image may be the same as that of the key region determination unit 131.

[0098] The small image piece information amount acquisition section 233 checks whether or not key region information for each of the one or more key regions determined by the key region determination section 232 is similar to each useful key region information stored in the useful key region information storage section 114, and acquires a small image piece information amount according to the inspection result for each of the two or more small image pieces. The small image piece information amount is the amount of information of a small image piece.

[0099] Note that being similar usually means satisfying a similarity condition, for example, "similarity is equal to or greater than a threshold" or "similarity is greater than a threshold."

[0100] In addition, the small image piece information amount acquisition unit 233 typically calculates the similarity between the key region information and one or more pieces of useful key region information stored in the useful key region information storage unit 114, and determines whether they are similar based on whether the similarity satisfies the similarity condition.

[0101] For example, as a result of the above inspection, the small image piece information amount acquiring section 233 acquires a larger amount of small image piece information as the amount of similar useful key region information increases.

[0102] For example, the more key region information similar to any useful key region information among the key region information of one or more key regions in a small image piece, the larger the small image piece information amount acquisition section 233 acquires.

[0103] The small image piece information amount acquisition unit 233 calculates the small image piece information amount by an increasing function with the information amount of one or more key area information of the one or more key areas in the small image piece that is determined to be similar to any useful key area information as a parameter. Note that such an increasing function is, for example, an addition.

[0104] The small image piece information amount acquisition unit 233, for example, acquires from the useful key region information storage unit 114 the amount of information that pairs with one or more useful key region information that is similar to the key region information of one or more key regions in the small image piece, and calculates the small image piece information amount using an increasing function (e.g., addition) with the amount of information as a parameter.

[0105] It is preferable that the small image piece information amount acquisition unit 233 acquires the target small image piece information amount and the other small image piece information amount for each of two or more small image pieces. That is, it is preferable that the small image piece information amount acquisition unit 233 checks whether the key region information for each of one or more key regions determined by the key region determination unit 232 is similar to each of the target useful key region information stored in the useful key region information storage unit 114, acquires the target small image piece information amount according to the inspection result for each of two or more small image pieces, and checks whether the key region information for each of one or more key regions determined by the key region determination unit 232 is similar to each of the other useful key region information stored in the useful key region information storage unit 114, and acquires the other small image piece information amount according to the inspection result for each of two or more small image pieces.

[0106] The small image piece discrimination value acquisition section 234 acquires, for each small image piece, a small image piece discrimination value, which is the discrimination value of the small image piece, using the small image piece information amount acquired by the small image piece information amount acquisition section 233 .

[0107] The small image piece discrimination value acquisition unit 234 calculates the small image piece information amount by an increasing function with the discrimination value of one or more key area information of one or more key areas in the small image piece as a parameter, the discrimination value of one or more key area information that is determined to be similar to any useful key area information. Note that this increasing function is, for example, an addition.

[0108] In addition, the small image piece discriminatory value acquisition unit 234, for example, acquires from the useful key region information storage unit 114 a discriminatory value that pairs with one or more useful key region information that is similar to the key region information of one or more key regions in the small image piece, and calculates the small image piece discriminatory value using an increasing function (e.g., addition) with the discriminatory value as a parameter.

[0109] Moreover, the small image piece discriminative value acquisition section 234 calculates the small image piece discriminative value, for example, by an increasing function with the small image piece information amount as a parameter.

[0110] The small image piece discriminative value acquisition section 234, for example, calculates information (for example, difference, ratio) relating to the difference between the amount of target small image piece information and the amount of other small image piece information for each small image piece, and acquires the small image piece discriminative value using the information relating to the difference. Note that the small image piece discriminative value may also be information relating to the difference. The small image piece discriminative value acquisition section 234 calculates the small image piece discriminative value for each small image piece, for example, using an increasing function with the amount of target small image piece information as a parameter, and a decreasing function with the amount of other small image piece information as a parameter.

[0111] The image construction section 235 constructs an output image that reflects two or more small image piece information amounts acquired by the small image piece information amount acquisition section 233. The image construction section 235 constructs an output image that reflects the small image piece information amount for each small image piece, for example.

[0112] In addition, the output image reflecting the small image piece information amount may be an output image reflecting the small image piece identification value. The image composition unit 235, for example, composes an output image reflecting two or more small image piece identification values ​​acquired by the small image piece identification value acquisition unit 234.

[0113] The image construction section 235 constructs an output image that reflects, for example, two or more small image piece information amounts acquired by the small image piece information amount acquisition section 233 and two or more small image piece identification values ​​acquired by the small image piece identification value acquisition section 234. An output image that reflects the small image piece information amount and / or small image piece identification value of a small image piece is an output image in which small image pieces having image parameters corresponding to the small image piece information amount of the small image piece and / or image parameters corresponding to the small image piece identification value of the small image piece are arranged. The image parameters are parameters of an image, such as brightness, luminance, saturation, and color.

[0114] The image construction section 235 may construct an output image that reflects the small image piece information amount or small image piece discrimination value of each small image piece for the target image.

[0115] The second output unit 24 outputs various information. The various information is, for example, information reflecting two or more small image piece information amounts acquired by the small image piece information amount acquisition unit 233. The information reflecting two or more small image piece information amounts acquired by the small image piece information amount acquisition unit 233 may be information reflecting the above-mentioned small image piece identification value. Moreover, the information reflecting two or more small image piece information amounts acquired by the small image piece information amount acquisition unit 233 is, for example, an output image. Moreover, the information reflecting two or more small image piece information amounts acquired by the small image piece information amount acquisition unit 233 is, for example, a detection signal. Note that, if the second output unit 24 does not output an output image, the image construction unit 235 and the image output unit 241 are not necessary.

[0116] The image output unit 241 outputs the output image constructed by the image construction unit 235. Here, output is a concept including display on a display, projection using a projector, printing on a printer, transmission to an external device, storage on a recording medium, delivery of the processing result to another processing device, another program, etc.

[0117] The first storage unit 11, image storage unit 111, reference image storage unit 112, condition storage unit 113, useful key region information storage unit 114, second storage unit 21, and target image storage unit 211 are preferably non-volatile recording media, but can also be realized using volatile recording media.

[0118] There is no restriction on the process by which information is stored in the first storage unit 11 etc. For example, information may be stored in the first storage unit 11 etc. via a recording medium, information transmitted via a communication line etc. may be stored in the first storage unit 11 etc., or information inputted via an input device may be stored in the first storage unit 11 etc.

[0119] The first reception unit 12 and the second reception unit 22 can be realized by a device driver for an input means such as a touch panel or a keyboard, or control software for a menu screen.

[0120] The first processing unit 13, the key region determination unit 131, the information amount acquisition unit 132, the useful key region determination unit 133, the storage unit 134, the second processing unit 23, the image division unit 231, the key region determination unit 232, the small image piece information amount acquisition unit 233, the small image piece identification value acquisition unit 234, and the image construction unit 235 can usually be realized by a processor, a memory, etc. The processing procedure of the first processing unit 13, etc. is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit). The processor is, for example, a CPU, an MPU, a GPU, etc., and the type is not important.

[0121] The first output unit 14, the second output unit 24, and the image output unit 241 may or may not include output devices such as a display, a speaker, etc. The first output unit 14 may be realized by driver software for an output device, or a combination of driver software for an output device and an output device, etc.

[0122] Next, an example of the operation of the production device 1 will be described with reference to the flowchart of Fig. 3. Note that, for example, when the first reception unit 12 receives a start instruction, the processes from step S301 onwards are performed.

[0123] (Step S301) The second processing unit 23 assigns 1 to a counter i.

[0124] (Step S302) Second processing unit 23 judges whether or not the i-th image exists in image storage unit 111. If the i-th image exists, the process proceeds to step S303, and if the i-th image does not exist, the process ends.

[0125] (Step S303) The key region determination unit 131 acquires the i-th image from the image storage unit 111.

[0126] (Step S304) The key region determination unit 131 acquires one or more pieces of key region information from the i-th image acquired in step S303, and temporarily stores the information in a buffer (not shown). Note that the process of acquiring key region information from an image can be performed by a known technique.

[0127] (Step S305) The second processing unit 23 assigns 1 to a counter j.

[0128] (Step S306) The second processing unit 23 judges whether the j-th key region information exists in a buffer (not shown). If the j-th key region information exists, the process proceeds to step S307. If not, the process proceeds to step S312.

[0129] (Step S307) The information amount acquisition unit 132 acquires the information amount of the j-th key region information. An example of such information amount acquisition processing will be described with reference to the flowchart of FIG.

[0130] (Step S308) The useful key region determination unit 133 acquires the discrimination value of the j-th key region information. An example of such discrimination value acquisition processing will be described with reference to the flowchart of FIG.

[0131] (Step S309) The useful key region determination unit 133 acquires the selection conditions from the condition storage unit 113. The useful key region determination unit 133 judges whether the discrimination value acquired in step S308 satisfies the selection conditions. If the selection conditions are satisfied, the process proceeds to step S310, and if the selection conditions are not satisfied, the process proceeds to step S311.

[0132] (Step S310) The accumulation unit 134 accumulates the j-th key region information, the information amount of the j-th key region information, and the discrimination value of the j-th key region information in the useful key region information storage unit 114 in association with each other.

[0133] (Step S311) The second processing unit 23 increments the counter j by 1. The process returns to step S306.

[0134] (Step S312) The second processing unit 23 increments the counter i by 1. The process returns to step S302.

[0135] Next, an example of the information amount acquisition process in step S307 will be described with reference to the flowchart in FIG.

[0136] (Step S401) The information amount acquiring unit 132 performs an initialization process. That is, the information amount acquiring unit 132 assigns 0 to the variable "target reference information" and the variable "other reference information".

[0137] The variable "target reference information" is information obtained from the target reference image, and is the original information for obtaining the amount of information of the target key region. The variable "target reference information" is, for example, the number of target reference images having key region information similar to the target key region information (satisfying the similarity condition). The variable "target reference information" is, for example, a value obtained by adding up the similarity of similar key region information in target reference images having key region information similar to the target key region information.

[0138] The variable "other reference information" is information obtained from other reference images, and is the original information for obtaining the amount of information of the target key region. The variable "other reference information" is, for example, the number of other reference images having key region information similar to the target key region information (satisfying the similarity condition). The variable "other reference information" is, for example, a value obtained by adding up the similarity of similar key region information in other reference images having key region information similar to the target key region information.

[0139] (Step S402) The information amount obtaining unit 132 assigns 1 to a counter i.

[0140] (Step S403) The information amount acquisition unit 132 judges whether or not the i-th target reference image exists in the reference image storage unit 112. If the i-th target reference image exists, the process proceeds to step S404, and if not, the process proceeds to step S413.

[0141] (Step S404 ) The information amount acquiring unit 132 acquires the i-th target reference image from the reference image storage unit 112 .

[0142] (Step S405) The key region determination unit 131 acquires one or more pieces of reference key region information, which are key region information, from the i-th target reference image acquired in step S404.

[0143] (Step S406) The information amount obtaining unit 132 assigns 1 to a counter j.

[0144] (Step S407) The information amount acquisition unit 132 judges whether or not the j-th reference key region information exists among the reference key region information acquired in step S405. If the j-th reference key region information exists, the process proceeds to step S408, and if not, the process proceeds to step S411.

[0145] (Step S408) The information amount obtaining unit 132 calculates the similarity between the target key region information and the j-th reference key region information.

[0146] (Step S409) The information amount acquisition unit 132 judges whether or not the similarity calculated in step S408 satisfies the similarity condition. If the similarity condition is satisfied (if similar), the process proceeds to step S410, and if the similarity condition is not satisfied (if not similar), the process proceeds to step S412.

[0147] (Step S410) The information amount acquisition unit 132 updates the variable "purpose reference information". Note that the information amount acquisition unit 132 normally increases the variable "purpose reference information". For example, the information amount acquisition unit 132 adds 1 to the variable "purpose reference information". For example, the information amount acquisition unit 132 adds the similarity calculated in step S408 to the variable "purpose reference information".

[0148] Furthermore, next, without proceeding to step S411, the similarity between the key region information and all reference key region information of the i-th target reference image may be obtained, and the target reference information of the key region information may be obtained using the similarity.

[0149] (Step S411) The information amount obtaining unit 132 increments the counter i by 1. The process returns to step S402.

[0150] (Step S412) The information amount obtaining unit 132 increments the counter j by 1. The process returns to step S407.

[0151] (Step S413) The information amount obtaining unit 132 assigns 1 to a counter k.

[0152] (Step S414) The information amount acquisition unit 132 judges whether or not the kth other reference image exists in the reference image storage unit 112. If the kth other reference image exists, the process proceeds to step S415, and if not, the process proceeds to step S424.

[0153] (Step S415 ) The information amount obtaining unit 132 obtains the k-th other reference image from the reference image storage unit 112 .

[0154] (Step S416) The key region determination unit 131 acquires one or more pieces of reference key region information, which are key region information, from the k-th other reference image acquired in step S415.

[0155] (Step S417) The information amount obtaining unit 132 assigns 1 to a counter l.

[0156] (Step S418) The information amount acquisition unit 132 judges whether or not the l-th reference key region information exists among the reference key region information acquired in step S416. If the l-th reference key region information exists, the process proceeds to step S419, and if not, the process proceeds to step S422.

[0157] (Step S419) The information amount obtaining unit 132 calculates the similarity between the target key region information and the l-th reference key region information.

[0158] (Step S420) The information amount obtaining unit 132 judges whether or not the similarity calculated in step S419 satisfies the similarity condition. If the similarity condition is satisfied, the process proceeds to step S421, and if the similarity condition is not satisfied, the process proceeds to step S423.

[0159] (Step S421) The information amount acquisition unit 132 updates the variable "other reference information". Note that the information amount acquisition unit 132 normally increases the variable "other reference information". For example, the information amount acquisition unit 132 adds 1 to the variable "other reference information". For example, the information amount acquisition unit 132 adds the similarity calculated in step S408 to the variable "other reference information".

[0160] Furthermore, next, without proceeding to step S422, the similarity between the key region information and all reference key region information of the i-th other reference image may be obtained, and other reference information of the key region information may be obtained using the similarity.

[0161] (Step S422) The information amount obtaining unit 132 increments the counter k by 1. The process returns to step S414.

[0162] (Step S423) The information amount obtaining unit 132 increments the counter l by 1. The process returns to step S418.

[0163] (Step S424) The information amount acquisition unit 132 calculates the amount of information using the variables "target reference information" and "other reference information". It then returns to the upper level processing. The information amount acquisition unit 132 normally calculates the amount of information using an increasing function with the value of the variable "target reference information" as a parameter. The information amount acquisition unit 132 normally calculates the amount of information using a decreasing function with the value of the variable "other reference information" as a parameter. Since the "target reference information" and "other reference information" have the property that as one of them increases, the other decreases, in order to obtain the effect of distinguishing between categories, the information amount acquisition unit 132 may calculate the amount of information using an increasing function with the value of the variable "other reference information" as a parameter.

[0164] Next, an example of the discrimination value acquisition process in step S408 will be described with reference to the flowchart in FIG.

[0165] (Step S501) The useful key region determination unit 133 acquires the value of the variable "target reference information."

[0166] (Step S502) The useful key region determination unit 133 acquires the value of the variable "other reference information".

[0167] (Step S503) The useful key region determination unit 133 calculates an identification value using the value of the variable "target reference information" and the value of the variable "other reference information", associates the identification value with the target key region information, and temporarily stores it in a buffer (not shown). Then, the process returns to the upper level process.

[0168] The useful key region determination unit 133 normally calculates the discrimination value by an increasing function with the value of the variable "target reference information" as a parameter. Also, the useful key region determination unit 133 calculates the discrimination value by, for example, a decreasing function with the value of the variable "other reference information" as a parameter.

[0169] Next, an example of the operation of the image processing device 2 will be described with reference to the flowchart of Fig. 6. Note that, here, the process of acquiring an output image from one target image will be described.

[0170] (Step S601) The image dividing unit 231 acquires a target image from the target image storage unit 211.

[0171] (Step S602) The image dividing unit 231 divides the target image acquired in step S601 to obtain two or more small image pieces.

[0172] (Step S603) The second processing unit 23 assigns 1 to a counter i.

[0173] (Step S604) Second processing section 23 performs an initialization process, which is a process of substituting 0 into a variable "small image piece information amount" and a process of substituting 0 into a variable "small image piece discrimination value".

[0174] (Step S605) The key region determination unit 232 judges whether or not the i-th small image piece exists among the small image pieces acquired in step S602. If the i-th small image piece exists, the process proceeds to step S606, and if not, the process proceeds to step S620.

[0175] (Step S606) The key region determination unit 232 acquires information about one or more key regions in the i-th small image piece.

[0176] (Step S607) The second processing unit 23 assigns 1 to a counter j.

[0177] (Step S608) The second processing unit 23 judges whether or not the j-th key region information exists in the key region information acquired in step S606. If the j-th key region information exists, the process proceeds to step S609, and if not, the process proceeds to step S617.

[0178] (Step S609) The second processing unit 23 assigns 1 to a counter k.

[0179] (Step S610) The second processing unit 23 judges whether or not the kth useful key region information exists in the useful key region information storage unit 114. If the kth useful key region information exists, the process proceeds to step S611, and if not, the process proceeds to step S616.

[0180] (Step S611) The small image piece information amount acquisition section 233 calculates the similarity between the j-th key region information and the k-th useful key region information.

[0181] (Step S612) The small image piece information amount acquisition unit 233 judges whether or not the similarity calculated in step S611 satisfies the similarity condition. If the similarity condition is satisfied, the process proceeds to step S613, and if the similarity condition is not satisfied, the process proceeds to step S615.

[0182] (Step S613) The small image piece information amount acquisition section 233 updates the variable "small image piece information amount". For example, the small image piece information amount acquisition section 233 adds the amount of information paired with the k-th useful key region information to the variable "small image piece information amount". For example, the small image piece information amount acquisition section 233 adds 1 to the variable "small image piece information amount".

[0183] (Step S614) The small image piece discriminative value acquisition unit 234 updates the variable "small image piece discriminative value". The small image piece information amount acquisition unit 233 adds, for example, to the variable "small image piece discriminative value", the discriminative value that pairs with the k-th useful key region information. The small image piece discriminative value acquisition unit 234 adds 1 to the variable "small image piece discriminative value", for example.

[0184] (Step S615) The counter k is incremented by 1. The process returns to step S610.

[0185] (Step S616) The counter j is incremented by 1. The process returns to step S608.

[0186] (Step S617) The image construction unit 235 acquires an image of the i-th small image piece in the target image using the values ​​of the variable "small image piece information amount" and the variable "small image piece discrimination value" for the i-th small image piece. Note that the image construction unit 235 determines attribute values ​​such as luminance, brightness, color, etc. of the i-th small image piece in the target image using the value of the variable "small image piece information amount" and / or the variable "small image piece discrimination value", and constructs the image of the i-th small image piece.

[0187] (Step S618) The image construction section 235 arranges the image of the i-th small image piece acquired in step S617 at the position of the i-th small image piece.

[0188] (Step S619) The second processing unit 23 increments the counter i by 1. The process returns to step S605.

[0189] (Step S620) The image output unit 241 outputs the output image constructed by the process in step S618, and the process ends.

[0190] In the flow chart of FIG. 6, the small image piece information amount acquisition section 233 may acquire the target small image piece information amount and the other small image piece information amount for each small image piece.

[0191] In the flow chart of FIG. 6, the small image piece discriminant value acquisition section 234 may acquire a small image piece discriminant value for each small image piece by using the target small image piece information amount and the other small image piece information amount.

[0192] A specific example of the operation of the production device 1 in this embodiment will be described below.

[0193] It is assumed that one or more medical images are stored in the image storage unit 111 of the production device 1. It is assumed that the medical image includes a cancer region and a normal region.

[0194] Also, it is assumed that the reference image storage unit 112 stores two or more reference images associated with the category identifier "cancer". Such reference images are images of areas of cancer cells. It is also assumed that the reference image storage unit 112 stores two or more reference images associated with the category identifier "normal". Such reference images are images of areas of normal cells.

[0195] In this situation, it is assumed that the user inputs a start instruction to the production device 1. Next, the first reception unit 12 of the production device 1 receives the start instruction.

[0196] Then, the following processing is performed on each of one or more medical images in the image storage unit 111, and a large amount of useful key region information is accumulated in the useful key region information storage unit 114.

[0197] That is, first, the key region determination unit 131 acquires a medical image (hereinafter, appropriately referred to as an “image”) from the image storage unit 111.

[0198] Next, the key region determination unit 131 determines one or more key points (an example of a key region) from the acquired image. To explain this using SIFT as a preferred image feature, the key region determination unit 131 determines a set of key points (σ, k) for the acquired image 701 as shown in FIG. 1 σ, k 2 σ, k 3 σ, k 4σ (σ, k x is a positive number, and x is a natural number). Next, the key region determination unit 131 obtains a Doc image that is a difference image between two smoothed images with adjacent scales (see FIG. 7).

[0199] Next, the key region determination unit 131 detects extreme values ​​from each Doc image. The key region determination unit 131 compares the DoG value of the pixel of interest with 26 neighboring pixels in the image scale space. If the DoG value is greater than or less than the 26 neighboring pixels, the point is determined as a candidate for a key point (see FIG. 8). In FIG. 8, the key region determination unit 131 compares the DoG value of the pixel of interest (the center pixel of the colored pixels in the middle Doc image of the set of three Doc images) with the DoG values ​​of pixels adjacent to the pixel of interest, and with the Doc images of corresponding pixels (colored pixels in the upper Doc image and the lower Doc image) in the upper Doc image and the lower Doc image of the set of three Doc images, and if the DoG value of the pixel of interest is greater than or less than the DoG values ​​of other pixels to be compared, the pixel of interest is determined as a candidate for a key point.

[0200] Furthermore, the key region determination unit 131 narrows down the key point candidates by excluding points on edges and points with small DoG values ​​(for example, points with a DoG value of 0.03 or less) from the key point candidates. It is preferable that the key region determination unit 131 performs sub-pixel position estimation and contrast threshold processing.

[0201] Next, the key region determination unit 131 calculates the gradient direction from the region corresponding to the scale of each of the one or more remaining keypoints. This conceptual diagram is shown in FIG. 9. The key region determination unit 131 also adds a weight obtained by multiplying the Gaussian window by the gradient strength to the gradient direction histogram of 36 directions. The key region determination unit 131 also assigns a peak that is, for example, 80% or more of the maximum value of the histogram as the orientation of the keypoint (see FIG. 10). The assigned keypoint is the selected keypoint. Note that some keypoints may have two orientations.

[0202] Next, the key region determination unit 131 obtains a 128-dimensional feature vector as key region information based on the detected orientation. First, the key region determination unit 131 rotates the region describing the feature according to the orientation of the key point. The feature vector has gradient information around the key point as elements. This gradient information is information obtained from a circle with the key point as the center and the scale of the key point as the radius. This region is divided into 4×4 blocks, and each block consists of 8 directions, so that the region has a total of 128-dimensional feature. The key region determination unit 131 creates a gradient orientation histogram as shown in FIG. 11 using this feature. At that time, the key region determination unit 131 normalizes the length of the 128-dimensional feature vector by the sum of the vectors. As a result, the key region determination unit 131 obtains a SIFT feature vector that is resistant to illumination changes.

[0203] By performing the above process or other processes that can extract small image regions useful for evaluating categorical differences on multiple medical images, a large amount of key region information is obtained. The key region information is a 128-dimensional feature vector calculated as SIFT features.

[0204] Next, the information amount acquiring unit 132 calculates the information amount of each of the one or more pieces of key region information acquired by the key region determining unit 131 using the above-mentioned formulas 1, 2, and 3.

[0205] Next, the useful key region determination unit 133 substitutes the amount of information of each of the one or more pieces of key region information into the above-mentioned formula 7 (which can also be called formula 6), and executes formula 7 to obtain a discrimination value "v(K i , cancer) is calculated.

[0206] Next, the useful key region determination unit 133 judges whether or not the discrimination value of each of the one or more pieces of key region information matches a selection condition, and acquires each of the one or more pieces of key region information that matches the selection condition as useful key region information.

[0207] Next, the accumulation unit 134 accumulates in the useful key region information storage unit 114 one or more pieces of useful key region information, the information amount of each piece of useful key region information, and the discrimination value of each piece of useful key region information in association with each other.

[0208] Through the above process, the production device 1 can accumulate a large amount of useful key region information that has a high ability to identify "gun." Note that the useful key region information that has a high ability to identify "gun" is referred to as target useful key region information.

[0209] Also, assume that the production device 1 can accumulate a large amount of useful key region information with a high ability to identify "normal" by performing processing using the same algorithm as the above processing. If the "gun" image and the "normal" image are replaced and the production device 1 operates using the same algorithm as the above processing, a large amount of useful key region information with a high ability to identify "normal" can be accumulated. Here, useful key region information with a high ability to identify "normal" is referred to as other useful key region information.

[0210] Next, the image processing device 2 uses a large amount of useful key region information accumulated by the production device 1 to construct and output an output image from the medical image including the cancer region, by the process described below. It is assumed that a large amount of purpose useful key region information and a large amount of other useful key region information acquired by the production device 1 are accumulated in the useful key region information storage unit 114 of the image processing device 2. It is also assumed that a medical image including a cancer region (hereinafter referred to as a "target image" as appropriate) is stored in the target image storage unit 211.

[0211] In this situation, it is assumed that the user inputs a start instruction to the image processing device 2. Then, the second receiving unit 22 of the image processing device 2 receives the start instruction.

[0212] Next, the image dividing section 231 acquires the target image from the target image storage section 211. Next, the image dividing section 231 divides the acquired target image to acquire two or more small image pieces.

[0213] Next, the key region determination unit 232 acquires key region information of the key points for each of, for example, two or more small image pieces. Note that here, the key region information is assumed to be a vector of SIFT features.

[0214] Next, the small image piece information amount acquisition unit 233 determines, for each of the two or more small image pieces, purpose useful key region information corresponding to each of the one or more pieces of key region information acquired by the key region determination unit 232. Then, the small image piece information amount acquisition unit 233 calculates the number of pieces of key region information corresponding to the purpose useful key region information for each of the two or more small image pieces. Note that useful key region information corresponding to key region information is useful key region information that satisfies a similarity condition with respect to the key region information. Also, the concept of correspondence between key region information and purpose useful key region information is shown in FIG. 12. (a) in FIG. 12 is an image including a purpose useful key region that has a high ability to identify "cancer", and (b) is a target image. It can be determined that the area in the target image that corresponds well to the image (a) is an area containing "cancer" cells.

[0215] FIG. 13 shows an image of the processing performed by the small image piece information amount acquisition unit 233. 1301 in FIG. 13 is an image (image of the "cancer" area) corresponding to purpose useful key region information (vector of key points) with a high ability to identify "cancer". The circles in 1301 are purpose useful key region information. 1302 is a target image having many small image pieces. And 1303 is the number of key region information corresponding to the purpose useful key region information for each small image piece. This number is the small image piece information amount. In other words, the small image piece information amount acquisition unit 233 calculates the number of key region information corresponding to the purpose useful key region information for each small image piece.

[0216] FIG. 14 shows an example of an image constructed using the purpose useful key region information for each small image piece acquired by the small image piece information amount acquisition unit 233. 1401 in FIG. 14 is an image corresponding to the purpose useful key region information, and is a specimen image of a cancer region. 1402 is a target image. The target image includes a cancer region and a normal region. 1403 is an image in which small image pieces that become darker the larger the numerical value for each small image piece in 1303 in FIG. 13 (the number of key region information corresponding to the purpose useful key region information) are arranged. 1403 is an image in which small image pieces that are recognized to be cancer-like are clearly shown.

[0217] Next, the small image piece information amount acquisition section 233 determines, for each of the two or more small image pieces, other useful key region information corresponding to each of the one or more pieces of key region information acquired by the key region determination section 232. Then, the small image piece information amount acquisition section 233 calculates the number of pieces of key region information corresponding to the other useful key region information for each of the two or more small image pieces.

[0218] FIG. 15 shows an example of an image constructed using the purpose useful key region information for each small image piece acquired by the small image piece information amount acquisition unit 233. Reference numeral 1501 in FIG. 15 is an image corresponding to other useful key region information, and is a sample image of a normal region. Reference numeral 1502 is a target image. The target image includes a cancer region and a normal region. Reference numeral 1503 is an image in which small image pieces that become darker the larger the numerical value for each small image piece (the number of key region information corresponding to other useful key region information) are arranged. Reference numeral 1503 is an image in which small image pieces that are deemed to be normal are clearly shown.

[0219] Next, the small image piece identification value acquisition unit 234 calculates the difference between the number of key region information corresponding to the purpose useful key region information and the number of key region information corresponding to other useful key region information for each of the two or more small image pieces. This difference is the small image piece identification value for each small image piece.

[0220] Next, the image constructor 235 constructs an output image using the small image piece discrimination value for each small image piece.

[0221] Next, the image output unit 241 outputs the output image.

[0222] Fig. 16 is a diagram for explaining the operation of small image piece discrimination value acquisition section 234 and image output section 241. Reference numeral 1601 in Fig. 16 is an image in which small image pieces that are recognized to be cancer-like are clearly indicated, and is the same as 1403. Reference numeral 1602 is an image in which small image pieces that are recognized to be normal are clearly indicated, and is the same as 1503. Reference numeral 1603 is an image in which normality is removed from cancer-likeness, and the color of the small image pieces in the cancer-like areas is darker than in 1403, making it easier for people to recognize the cancer areas.

[0223] It is also possible to adopt a simple configuration in which the output of the image processing device 2 is not an image, but the positions of small image pieces in the target image that have an amount of small image piece information greater than a threshold. The threshold is, for example, information that is stored in advance, or information that is input and accepted by the second accepting unit 22. The positions of small image pieces may also be a single coordinate, the coordinates of two diagonal points of a rectangle that represents an area, the center and radius of a circle, etc.

[0224] As described above, according to the production device 1 of the present embodiment, it is possible to acquire useful information for obtaining an image for detecting an area of ​​a target category with high accuracy from a target image including two or more categories.

[0225] Moreover, according to the image processing device 2 of this embodiment, an image for detecting an area of ​​a target category with high accuracy from a target image including two or more categories is obtained.

[0226] Furthermore, according to the image processing device 2 of this embodiment, the output image can be dynamically changed in response to a change in the accepted similarity condition.

[0227] The process in this embodiment may be realized by software. This software may be distributed by software download or the like. This software may be recorded on a recording medium such as a CD-ROM and distributed. This also applies to other embodiments in this specification. The software for realizing the production device 1 in this embodiment is a program as follows. That is, this program is a program for making a computer function as a key region determination unit that determines one or more key regions that satisfy a key condition from an image, an information amount acquisition unit that acquires an information amount that is information on the ability to determine a region of a target category and is information on each of the one or more key regions determined by the key region determination unit, a useful key region determination unit that determines one or more useful key regions that satisfy a selection condition for selecting a useful key region using the information amount acquired by the information amount acquisition unit, and a storage unit that accumulates useful key region information on the one or more useful key regions determined by the useful key region determination unit.

[0228] Moreover, the software for realizing the image processing device 2 is the following program. In other words, this program is a program for causing a computer that can access a useful key region information storage unit in which one or more pieces of useful key region information are stored and a target image storage unit in which a target image that is an image to be processed is stored to function as an image division unit that divides the target image and acquires two or more small image pieces, a key region determination unit that determines one or more key regions for each of the two or more small image pieces acquired by the image division unit, a small image piece information amount acquisition unit that checks whether or not key region information on each of the one or more key regions determined by the key region determination unit is similar to each of the useful key region information stored in the useful key region information storage unit and acquires a small image piece information amount according to the check result for each of the two or more small image pieces, and an output unit that outputs information reflecting the two or more small image piece information amounts acquired by the small image piece information amount acquisition unit.

[0229] Fig. 17 shows the appearance of a computer that executes the programs described in this specification to realize the production device 1 and image processing device 2 of the various embodiments described above. The above-mentioned embodiments can be realized by computer hardware and a computer program executed thereon. Fig. 17 is an overview of this computer system 300, and Fig. 18 is a block diagram of the system 300.

[0230] In FIG. 17, a computer system 300 includes a computer 301 including a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

[0231] 18, computer 301 includes, in addition to CD-ROM drive 3012, MPU 3013, bus 3014 connected to CD-ROM drive 3012 etc., ROM 3015 for storing programs such as a boot-up program, RAM 3016 connected to MPU 3013 for temporarily storing instructions of application programs and providing temporary storage space, and hard disk 3017 for storing application programs, system programs, and data. Although not shown here, computer 301 may further include a network card for providing connection to a LAN.

[0232] A program for causing computer system 300 to execute the functions of image processing device 2 and the like of the above-mentioned embodiment may be stored on CD-ROM 3101, inserted into CD-ROM drive 3012, and further transferred to hard disk 3017. Alternatively, the program may be transmitted to computer 301 via a network (not shown) and stored on hard disk 3017. The program is loaded into RAM 3016 at the time of execution. The program may also be loaded directly from CD-ROM 3101 or the network.

[0233] The program does not necessarily include an operating system (OS) or a third party program that causes the computer 301 to execute the functions of the image processing device 2 of the above-mentioned embodiment. The program only needs to include an instruction portion that calls appropriate functions (modules) in a controlled manner to obtain a desired result. How the computer system 300 operates is well known, and a detailed description will be omitted.

[0234] In addition, in the above program, the steps of transmitting information and receiving information do not include processing performed by hardware, such as processing performed by a modem or interface card in the transmitting step (processing that is performed only by hardware).

[0235] The program may be executed by a single computer or a plurality of computers. That is, the program may be executed by a centralized processing or a distributed processing.

[0236] Furthermore, in each of the above embodiments, each process may be realized by centralized processing in a single device, or may be realized by distributed processing in a plurality of devices.

[0237] An example of distributed processing is as follows: In other words, when distributed processing is performed between an isolated island where it is difficult to secure a sufficient communication band and a base diagnostic center, a remote input unit (not shown) for inputting a target image and an abnormal image particle position receiving unit (not shown) are provided on the isolated island, and the output of the image processing device 2 is limited to the position of the abnormal image particle, so that the output result can be quickly transmitted from the base diagnostic center to the isolated island.

[0238] The present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these modifications are also included within the scope of the present invention. [Industrial Applicability]

[0239] INDUSTRIAL APPLICABILITY As described above, the image processing device according to the present invention has an effect of obtaining an image for detecting an area of ​​a target category with high accuracy from a target image including two or more categories, and is useful as an image processing device or the like. [Explanation of symbols]

[0240] 1 Production equipment 2. Image Processing Device 11 First storage section 12 First Reception Department 13 First Processing Section 14 First output section 21 Second storage section 22 Second Reception Department 23 Second Processing Section 24 Second output section 111 Image storage section 112 Reference image storage unit 113 Condition Storage Section 114 Useful key area information storage section 131, 232 Key area determination unit 132 Information acquisition unit 133 Useful Key Region Determination Unit 134 Storage Unit 211 Target image storage unit 231 Image division unit 233 Small image piece information acquisition unit 234 Small image piece identification value acquisition unit 235 Image Composition Department 241 Image output unit

Claims

1. an image storage unit in which images are stored; a key region determiner for determining one or more key regions from the image that satisfy a key condition; an information amount acquiring unit that acquires an information amount that is information regarding an ability to determine a region of a target category, the information amount being information regarding each of the one or more key regions determined by the key region determining unit; a useful key region determination unit that determines one or more useful key regions that satisfy a selection condition for selecting a useful key region, using the amount of information acquired by the information amount acquisition unit; a storage unit that stores useful key region information regarding the one or more useful key regions determined by the useful key region determination unit; a reference image storage unit for storing one or more target reference images which are images including an area of ​​a target category, and one or more other target reference images which are images not including an area of ​​the target category; The information amount acquisition unit A useful key region information producing device which performs a first inspection to check whether or not an area corresponding to each of the one or more key regions determined by the key region determination unit exists in each of the one or more target reference images, and a second inspection to check whether or not an area corresponding to each of the one or more key regions determined by the key region determination unit exists in each of the one or more other reference images, and obtains the amount of information of each of the one or more key regions using the results of the first inspection and the second inspection.

2. An image storage unit in which an image is stored; a key region determiner for determining one or more key regions from the image that satisfy a key condition; an information amount acquiring unit that acquires an information amount that is information regarding an ability to determine a region of a target category, the information amount being information regarding each of the one or more key regions determined by the key region determining unit; a useful key region determination unit that determines one or more useful key regions that satisfy a selection condition for selecting a useful key region, using the amount of information acquired by the information amount acquisition unit; a storage unit that stores useful key region information related to one or more useful key regions determined by the useful key region determination unit, The useful key region determination unit A useful key region information production device that uses the amount of information acquired by the information amount acquisition unit to acquire a discrimination value of each of the one or more key regions, and determines one or more useful key regions whose discrimination values ​​satisfy the selection condition.

3. A system having a useful key region information production device according to claim 1 or 2 and an image processing device, The image processing device includes: a useful key region information storage unit in which the one or more pieces of useful key region information accumulated by the production apparatus according to claim 1 or 2 are stored; a target image storage unit for storing a target image which is an image to be processed; an image division unit that divides the target image to obtain two or more small image pieces; a key region determination unit that determines one or more key regions for each of the two or more small image pieces acquired by the image division unit; a small image piece information amount acquiring section for inspecting whether or not key region information relating to each of the one or more key regions determined by the key region determining section is similar to each of the useful key region information stored in the useful key region information storage section, and acquiring a small image piece information amount according to the inspection result for each of two or more small image pieces; an output section that outputs information reflecting the two or more small image piece information amounts acquired by the small image piece information amount acquisition section.

4. The image processing device, an image construction unit that constructs an output image reflecting the two or more small image piece information amounts acquired by the small image piece information amount acquisition unit; 4. The system according to claim 3, wherein the information reflecting the two or more small image piece information amounts acquired by the small image piece information amount acquisition section is the output image.

5. The image processing device, a small image piece identification value acquisition unit that acquires a small image piece identification value, which is an identification value of the small image piece, using the two or more small image piece information amounts acquired by the small image piece information amount acquisition unit; The output unit is A system as described in claim 3 or claim 4, which outputs information reflecting the two or more small image piece identification values ​​acquired by the small image piece identification value acquisition unit in addition to or instead of the two or more small image piece information amounts for the target image.

6. The small image piece information amount acquisition unit when a change in the similarity condition is accepted, checking whether or not key region information relating to each of the one or more key regions determined by the key region determination unit is similar to each of the useful key region information stored in the useful key region information storage unit based on the changed similarity condition, and acquiring a new amount of small image piece information according to the checking result for each of the two or more small image pieces; The output unit is 6. The system according to claim 3, further comprising: outputting information reflecting the new amount of small image piece information.

7. The useful key region information storage unit includes: One or more pieces of purpose useful key area information and one or more pieces of other useful key area information are stored. The small image piece information amount acquisition unit Inspect whether or not key region information relating to each of the one or more key regions determined by the key region determination unit is similar to each of the target useful key region information stored in the useful key region information storage unit, and obtain a target small image piece information amount according to the inspection result for each of the two or more small image pieces; Inspect whether or not key region information relating to each of the one or more key regions determined by the key region determination unit is similar to each of other useful key region information stored in the useful key region information storage unit, and obtain an amount of other small image piece information according to the inspection result for each of two or more small image pieces; The output unit is A system as described in any one of claims 3 to 6, which outputs information reflecting the target small image piece information amount and the other small image piece information amount acquired by the small image piece information amount acquisition unit for each small image piece of the target image.

8. The image processing device, a detection signal acquisition unit that compares the small image piece information amount acquired by the small image piece information amount acquisition unit with a threshold, and acquires a detection signal when the small image piece information amount is greater than the threshold, The image processing device according to claim 3 , wherein the output unit outputs the detection signal.

9. The system of claim 3 , wherein the target image is a medical image.

10. A production method comprising all steps performed by the production apparatus described in claim 1 or claim 2.

11. An image processing method comprising all the steps performed by a system described in any one of claims 3 to 9.

12. A computer comprising: A program for causing the production device according to claim 1 or 2 to function.

13. Computer, A program for causing the system according to any one of claims 3 to 9 to function.

Images