Image processing device, image processing method, and program

Abstract

To solve such a problem that it is difficult to detect a region of a target category from an object image with high accuracy.SOLUTION: The above-mentioned problem can be solved by an image processing device comprising: an image division unit which acquires two or more small image pieces from an object image; a key region decision unit which decides two or more key regions from the object image; a local processing unit which acquires an identification value of one or more key regions in the small image piece by using one or more pieces of useful key region information for each small image piece and acquires an identification representative value of the one or more identification values; a reliability processing unit which executes the first inspection on whether or not a region corresponding to each of one or more key regions exists in each of one or more target reference images and the second inspection on whether or not a region corresponding to each of the one or more key regions decided by the key region decision unit exists in each of one or more other reference images to acquire the reliability of each of one or more key regions; a boarder line decision unit which decides a boarder line by using an identification representative value and reliability; an image constitution unit which constitutes an image in which the boarder line is clearly shown with respect to the object image; and an output unit which outputs the image.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an image processing device, etc., for detecting a region of a target category from a target image. [Background technology]

[0002] Conventionally, there has been technology for a classifier that identifies whether or not a region is cancerous (see, for example, Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] IEEE TRANSACTION MEDICAL IMAGING VOL.38, NO.2 FEBRUARY 2019, From Detection of Individual Metastases to Classification of Lymph Node Status at the Patient Level: The CAMELYON17 Challenge [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] However, with conventional techniques, it has been difficult to obtain information necessary to accurately detect the region of a target category from an image containing two or more categories.

[0005] More specifically, conventional technology could only distinguish between cancerous areas and non-cancerous areas, forcing doctors to examine the detected areas. However, false positives and false negatives were unavoidable, meaning doctors were often forced to diagnose areas that could be ignored based on their medical experience. [Means for solving the problem]

[0006] The production apparatus for useful key region information of the first invention comprises: an image storage unit for storing images; a key region determination unit for determining one or more key regions that satisfy key conditions from images for each of two or more size information relating to the size of key regions; an identification value acquisition unit for acquiring identification values ​​which are information relating to the ability to determine regions of a target category and relating to each of the one or more key regions determined by the key region determination unit; a useful key region determination unit for determining one or more useful key regions that satisfy selection conditions for selecting useful key regions using the identification values ​​acquired by the identification value acquisition unit; and a storage unit for accumulating useful key region information relating to the one or more useful key regions determined by the useful key region determination unit in association with each of the two or more size information.

[0007] This configuration allows for the acquisition of useful information for accurately detecting the region of a target category from a target image containing two or more categories.

[0008] Furthermore, the production apparatus of the second invention further comprises a reference image storage unit that stores one or more target reference images which are images containing the region of the target category, and one or more other target reference images which are images which do not contain the region of the target category, and the identification value acquisition unit performs a first inspection, which is an inspection to check whether or not the region corresponding to each of the one or more key regions determined by the key region determination unit exists in each of the one or more target reference images, and a second inspection, which is an inspection to check whether or not the region corresponding to each of the one or more key regions determined by the key region determination unit exists in each of the one or more other reference images, and acquires the identification value of each of the one or more key regions using the results of the first and second inspections.

[0009] This configuration allows for the acquisition of useful information for accurately detecting the region of a target category from a target image containing two or more categories.

[0010] Furthermore, the image processing apparatus of this third invention includes a reference image storage unit that stores one or more target reference images which are images containing the region of the target category and one or more other target reference images which are images which do not contain the region of the target category; a useful key region information storage unit that stores one or more useful key region information associated with two or more size information units; a target image storage unit that stores a target image which is the image to be processed; an image splitting unit that acquires two or more small image fragments of different regions from the target image; a key region determination unit that determines two or more key regions from the target image; a local processing unit that acquires the identification value of one or more key regions in each of the two or more small image fragments acquired by the image splitting unit and acquires an identification representative value which is a representative value of one or more identification values; and a local processing unit that acquires the identification value of one or more key regions which correspond to one or more key regions determined by the key region determination unit. The image processing apparatus comprises a reliability processing unit that performs a first inspection, which checks whether or not each of the above target reference images exists, and a second inspection, which checks whether or not each of the one or more key regions determined by the key region determination unit exists in one or more other reference images, and uses the results of the first and second inspections to obtain the reliability of each of the one or more key regions; a boundary determination unit that detects small image fragments or key regions within small image fragments that correspond to a value of less than or equal to a first threshold, and for which the reliability of one or more key regions of the small image fragment is large enough to satisfy the reliability condition, and determines the boundary line corresponding to the small image fragment or key region; an image constructing unit that constructs an image in which the boundary line determined by the boundary determination unit is clearly indicated for the target image; and an output unit that outputs the image constructed by the image constructing unit.

[0011] This configuration allows for the acquisition of an image in which the region of the target category is detected with high accuracy from a target image containing two or more categories.

[0012] Furthermore, the image processing apparatus of the fourth invention, compared to the third invention, is an image processing apparatus in which the reliability processing unit acquires a larger first check value for each of the one or more key regions determined by the key region determination unit, the more areas corresponding to each key region that exist in one or more target reference images, and acquires a larger second check value for each of the one or more other reference images that exist in one or more areas corresponding to each key region, calculates the reliability using an increasing function with the first check value and the second check value as parameters, and acquires different first check values ​​and different second check values ​​according to the size information corresponding to the key region.

[0013] This configuration allows for the acquisition of an image in which the region of the target category is detected with high accuracy from a target image containing two or more categories.

[0014] Furthermore, the image processing apparatus of the fifth invention, compared to the fourth or fourth invention, is an image processing apparatus in which the reliability processing unit obtains a large first check value for each of the one or more key regions determined by the key region determination unit, the larger the number of regions corresponding to each key region that exist in each of the one or more target reference images, obtains a large second check value for each of the one or more regions corresponding to each key region that exists in each of the other reference images, calculates the reliability using an increasing function with the first check value and the second check value as parameters, obtains a confidence representative value which is a representative value of the reliability of each of the one or more key regions belonging to each of the two or more small image fragments, the boundary determination unit determines two or more small image fragments for which the identification representative value is less than or equal to the first threshold or less than the first threshold, and two or more small image fragments for which the confidence representative value is greater than or equal to the second threshold or greater than the second threshold, and determines a boundary line connecting the representative points of each of the two or more small image fragments.

[0015] This configuration allows for the acquisition of an image in which the region of the target category is detected with high accuracy from a target image containing two or more categories.

[0016] Further, the image processing apparatus of the sixth invention includes a reference image storage unit that stores one or more target reference images that are images including regions of a target category, and one or more other target reference images that are images not including regions of the target category, a useful key region information storage unit that stores one or more pieces of useful key region information, a target image storage unit that stores a target image that is an image to be processed, a key region determination unit that determines two or more key regions from the target image, a local processing unit that obtains the discrimination values of the two or more key regions obtained by the key region determination unit using one or more pieces of useful key region information, a first inspection that is an inspection of whether regions corresponding to the one or more key regions determined by the key region determination unit exist in the one or more target reference images, and a second inspection that is an inspection of whether regions corresponding to the one or more key regions determined by the key region determination unit exist in the one or more other reference images, a reliability processing unit that obtains the reliability of the two or more key regions using the results of the first inspection and the second inspection, a boundary line that divides a region where the discrimination value of the key region is a positive value and a region where the discrimination value of the key region is a negative value, a boundary line determination unit that determines a boundary line where the reliability of one or more key regions on or near the boundary line satisfies a reliability condition, an image composition unit that composes an image in which the boundary line determined by the boundary line determination unit is明示された画像を構成する画像構成部と、画像構成部が構成した画像を出力する画像出力部とを具備する画像処理装置である。

[0017] With such a configuration, a boundary line for accurately detecting a region of a target category can be drawn for a target image including two or more categories.

[0018] Further, the image processing apparatus of the seventh invention is an image processing apparatus in which, for any one of the third to sixth inventions, the target image is a medical image.

[0019] With such a configuration, an image in which a region of a target category (for example, cancer) is accurately detected can be obtained from a target image medical image including two or more categories.

Effects of the Invention

[0020] It should be noted that there seems to be an unclear part "明示された画像を構成する画像構成部" in the translation of . You may need to check and clarify the original text for a more accurate translation.According to the image processing apparatus of the present invention, an image can be obtained in which the region of the target category is detected with high accuracy from a target image containing two or more categories. [Brief explanation of the drawing]

[0021] [Figure 1] Block diagram of production apparatus 1 in Embodiment 1 [Figure 2] Block diagram of the image processing device 2 [Figure 3] A flowchart illustrating an example of the operation of the production device 1. [Figure 4] A flowchart illustrating an example of the process for obtaining the same identification value. [Figure 5] A flowchart illustrating an example of the operation of the image processing device 2. [Figure 6] An example of boundary line determination processing is explained, along with a flowchart. [Figure 7] A flowchart illustrating an example of the same image composition processing. [Figure 8] A diagram illustrating an example of processing by the key region determination unit 131. [Figure 9] A diagram illustrating an example of processing by the key region determination unit 131. [Figure 10] A diagram illustrating an example of processing by the key region determination unit 131. [Figure 11] A diagram illustrating an example of processing by the key region determination unit 131. [Figure 12] A diagram illustrating an example of processing by the key region determination unit 131. [Figure 13] A diagram illustrating the characteristics of key regions based on the same keypoint size. [Figure 14] Figure showing an example of the output of the same image. [Figure 15] Block diagram of the same image processing device 2 [Figure 16] Overview of the computer system [Figure 17] Block diagram of the computer system [Modes for carrying out the invention]

[0022] The following describes embodiments of the image processing device and the like with reference to the drawings. Note that components denoted by the same reference numerals in these embodiments perform similar operations, and therefore, further explanation may be omitted.

[0023] (Embodiment 1) In this embodiment, a production apparatus for useful key region information is described, which determines key regions for each of two or more size information from an image, obtains the identification value of the key regions, determines useful key regions using the identification value, and stores the useful key region information in association with size information. In this embodiment, it is preferable to obtain the identification value using the identification value and to determine the useful key regions using the identification value based on the identification value.

[0024] Furthermore, this embodiment describes an image processing apparatus that acquires two or more small image fragments from an image to be processed, determines one or more key regions for each of the two or more small image fragments, acquires a representative identification value and confidence level which are representative values ​​of the identification value of each of the one or more key regions, determines the boundary between a region of the first category (e.g., cancer) and a region of the second category (e.g., normal) using the representative identification value and confidence level, constructs an image, and outputs it.

[0025] Figure 1 is a block diagram of the production apparatus 1 in this embodiment. The production apparatus 1 is a device that acquires and stores useful key area information, which will be described later, for each of the two or more size information of the key area. The useful key area information is typically the information used by the image processing apparatus described in Embodiment 2.

[0026] The production apparatus 1 comprises a first storage unit 11, a first receiving unit 12, a first processing unit 13, and a first output unit 14. The first storage unit 11 comprises a size information storage unit 110, an image storage unit 111, a reference image storage unit 112, a condition storage unit 113, and a useful key area information storage unit 114. The first processing unit 13 comprises a control unit 130, a key area determination unit 131, an identification value acquisition unit 132, a useful key area determination unit 133, and a storage unit 134.

[0027] Figure 2 is a block diagram of the image processing device 2 in this embodiment. The image processing device 2 comprises a second storage unit 21, a second receiving unit 22, a second processing unit 23, and a second output unit 24. The second storage unit 21 comprises a reference image storage unit 112, a useful key area information storage unit 114, and a target image storage unit 211. The second processing unit 23 comprises an image division unit 231, a key area determination unit 232, a local processing unit 233, a reliability processing unit 234, a boundary line determination unit 235, and an image configuration unit 236. The second output unit 24 comprises an image output unit 241.

[0028] Furthermore, the production device 1 and the image processing device 2 may be a single integrated device. Alternatively, the processing performed by the production device 1 and the image processing device 2 may be shared among three or more devices.

[0029] Various types of information are stored in the first storage unit 11, which constitutes the production apparatus 1. These types of information include, for example, size information, images, judgment conditions, and useful key area information, which will be described later.

[0030] The size information storage unit 110 stores two or more pieces of size information. Size information is information that specifies the range or size of the key area, which will be described later. For example, the size information is the range of the maximum length (for example, the diameter) of the key area. For example, the size information is "0 < size <= 2", "2 < size <= 4", "4 < size <= 6", "6 < size <= 8", "8 < size <= 10", "10 < size <= 12", "12 < size". The numerical values ​​such as "2" and "4" in the size information are, for example, the number of pixels.

[0031] For example, if the key region is a circle, the size of the key region is, for example, the diameter or radius of the circle. Also, depending on the size of the key region, the distribution of each category (e.g., "cancer" or "normal") differs, as shown in Figure 13 below. More specifically, "cancer" appears more frequently in smaller key regions, while "normal" appears more frequently in larger key regions.

[0032] The image storage unit 111 stores one or more images. These images are the source images from which useful key region information, described later, is obtained. The images are, for example, medical images. The images may be, for example, fMRI images, PET images, ultrasound images, etc., and their type and data structure are not restricted.

[0033] The reference image storage unit 112 stores two or more reference images. Reference images are images used when obtaining the identification value of a key region. Each of the two or more reference images is associated with one of two or more categories. Each of the two or more reference images is associated with a category identifier that identifies the category. For example, if the category is "cancer" or "normal", each of the two or more reference images is either an image of "cancer" or an image of "normal".

[0034] A key region is an area within an image that has high discriminatory value. A key region is, for example, a key point obtained using SIFT features acquired from an image. A region is a single pixel or a set of two or more pixels. A key region can be any small area useful for distinguishing the aforementioned category in the target image, and may be, for example, a mosaic-like feature image from the Viola-Jones method used for pedestrian and vehicle recognition.

[0035] A category is a type of region within an image. For example, a category could be a "cancer" region or a region of "normal" cells. A category can be two, three, or more. If there are three or more categories, for example, they could be "cancer," "cirrhosis," and "normal." Here, the category being detected (for example, the "cancer" region) is referred to as the "target category," and other categories (for example, the "normal" region) are referred to as "other categories."

[0036] Reference images are, for example, target reference images or other reference images. A target reference image is an image of the target category (for example, an image consisting of the "cancer" region) and is a reference image associated with the target category. Other reference images are images of other categories (for example, an image consisting of the "normal" region) and are reference images associated with other categories.

[0037] The condition storage unit 113 stores one or more conditions. These conditions include, for example, key conditions, selection conditions, and similarity conditions, which will be described later.

[0038] Key conditions are criteria used to determine key regions from an image. For example, key conditions are the conditions required for an area to be identified as a keypoint using SIFT features obtained from an image.

[0039] The selection criteria are the conditions for selecting useful key regions from the key regions, as described later. For example, the selection criteria include the fact that the identification value, as described later, is greater than or equal to a threshold, or that the identification value, as described later, is greater than a threshold.

[0040] The useful key area information storage unit 114 stores one or more useful key area information. The useful key area information is information stored by the storage unit 134. The useful key area information storage unit 114 stores one or more useful key area information for each of the two or more size information.

[0041] Useful key region information is information about useful key regions. Useful key region information is, for example, a feature vector which is a set of two or more features of a useful key region. The two or more features that make up the feature vector include, for example, one or more SIFT features obtained from an image. The two or more features that make up the feature vector include, for example, one or more HOG features obtained from an image. A useful key region is a region within a key region that is useful for identifying a region of the target category (for example, a cancer region).

[0042] It is preferable that the useful key area information storage unit 114 stores one or more purpose-specific useful key area information and one or more other useful key area information. The useful key area information of a useful key area with a high ability to identify a purpose category is referred to as purpose-specific useful key area information. Similarly, the useful key area information of a useful key area with a high ability to identify other categories is referred to as other useful key area information. It is preferable that the useful key area information storage unit 114 stores one or more purpose-specific useful key area information and one or more other useful key area information for each of the two or more size information items.

[0043] The first reception unit 12 receives various instructions and information. These instructions and information include, for example, start instructions, images, reference images, and similarity conditions. A start instruction is an instruction to start acquiring useful key region information.

[0044] Accepting similarity conditions can also mean accepting a threshold value used to determine whether two vectors are similar or not. Furthermore, the method of accepting similarity conditions is not limited; for example, it could be a numerical threshold value, a threshold value set using a slider bar, etc.

[0045] Here, "reception" refers to a concept that includes receiving information input from input devices such as keyboards, mice, and touch panels; receiving information transmitted via wired or wireless communication lines; and receiving information read from recording media such as optical discs, magnetic discs, and semiconductor memory.

[0046] Furthermore, the means of inputting various instructions and information can be anything, such as a touch panel, keyboard, mouse, or menu screen.

[0047] The first processing unit 13 performs various processes. These various processes include, for example, those performed by the control unit 130, the key area determination unit 131, the identification value acquisition unit 132, the useful key area determination unit 133, and the storage unit 134.

[0048] The control unit 130 acquires two or more size information stored in the size information storage unit 110 and passes each of these size information to the key area determination unit 131.

[0049] The key region determination unit 131, for each of the two or more size information provided by the control unit 130, determines from the image one or more key regions that match the size information and satisfy the key conditions of the condition storage unit 113. The image is the image stored in the image storage unit 111. Determining a key region involves obtaining key region information and information that identifies the key region (for example, one or more coordinate values ​​within the image).

[0050] The key region determination unit 131 acquires key region information, and / or information that identifies the key region, for example, by associating it with two or more pieces of size information.

[0051] Key region information refers to information about a key region. Key region information is, for example, a feature vector, which is a set of two or more features of the key region. The two or more features that make up the feature vector include, for example, one or more SIFT features obtained from the image. The two or more features that make up the feature vector include, for example, one or more HOG features obtained from the image.

[0052] The key region determination unit 131 determines one or more key points from the image. Since the process of determining key points from an image is a known technique, a detailed explanation is omitted here. The key region determination unit 131 also obtains key region information of key points from the target image, for example.

[0053] The identification value acquisition unit 132 acquires the identification value of each of the one or more key regions determined by the key region determination unit 131. It is preferable for the identification value acquisition unit 132 to acquire the identification value of each of the one or more key regions determined by the key region determination unit 131 for each size information.

[0054] Distinguishing value is information about key domains. Distinguishing value is information about the ability to determine the domain of the target category. Preferably, distinguishing value is information about the ability to determine the domain of a specific category other than the target category (e.g., "normal").

[0055] The identification value acquisition unit 132 performs, for example, a first inspection and a second inspection for each key area, and uses the results of the first inspection and the second inspection to acquire the identification value of one or more key areas. Alternatively, the identification value acquisition unit 132 performs, for example, a first inspection and a second inspection for each key area for each size information. It is preferable that the identification value acquisition unit 132 then uses the results of the first inspection and the second inspection for each key area for each size information to acquire the identification value of one or more key areas.

[0056] The first inspection is a check to determine whether the regions corresponding to each of the one or more key regions determined by the key region determination unit 131 exist in each of the one or more target reference images. Each of the one or more target reference images is stored in the reference image storage unit 112.

[0057] The second inspection is a check to determine whether each of the one or more key regions determined by the key region determination unit 131 exists in each of the one or more other reference images. The one or more other reference images are stored in the reference image storage unit 112.

[0058] The region corresponding to the key region is the region for region information similar to the key region information. For example, the region corresponding to the key region is the region for region information that satisfies the similarity condition with the key region information. The similarity condition is that the similarity between two pieces of region information (e.g., vectors) is equal to or greater than a threshold. The similarity condition is stored in the condition storage unit 113.

[0059] The identification value acquisition unit 132 acquires a higher identification value for a key region if, for example, the results of the first inspection show that there are many target reference images in which a region corresponding to the key region exists.

[0060] The identification value acquisition unit 132 preferably acquires a first test value, which is a value representing the likelihood of an abnormality (e.g., cancer) in the key region, using the result of the first test of the key region. The identification value acquisition unit 132 preferably acquires different first test values ​​depending on the size information corresponding to the key region.

[0061] The identification value acquisition unit 132 acquires a smaller identification value for a key region if, for example, the result of the second inspection shows that there are many other reference images in which a region corresponding to the key region exists. It is preferable for the identification value acquisition unit 132 to use the result of the second inspection of the key region to acquire a second inspection value, which is a value representing the normality of the key region.

[0062] The identification value acquisition unit 132 preferably acquires different second check values ​​according to the size information corresponding to the key area.

[0063] The identification value acquisition unit 132 acquires, for example, region information that is most similar to key region information for each of the one or more target reference images, and acquires the identification value by an increasing function (e.g., addition) that uses the similarity with such region information as a parameter.

[0064] The identification value acquisition unit 132, for example, acquires region information that is most similar to the key region information for one or more other reference images, and acquires the identification value by using a reduction function (e.g., subtraction) that uses the similarity with such region information as a parameter.

[0065] It is preferable for the identification value acquisition unit 132 to store the identification value in a buffer (not shown) in association with the key region information. It is also preferable for the identification value acquisition unit 132 to store the first test value and the second test value in a buffer (not shown) in association with the key region information.

[0066] The identification value acquisition unit 132 acquires the first test value, for example, using the following formula 1. Note that if the category is "cancer," the first test value is a value indicating cancerous characteristics, and can be said to be a cancer feature.

[0067] [Number]

[0068] The identification value acquisition unit 132 acquires the second inspection value, for example, according to the following mathematical formula 2. When the category is "cancer", the second inspection value is a value indicating normality, and it can also be said that there are normal characteristics.

[0069] [Number]

[0070] The identification value acquisition unit 132 acquires the identification value, for example, according to the following mathematical formula 3. When the category is "cancer" or "normal", the identification value is the value obtained by subtracting the normality from the cancerousness, and it can also be said that it is a neutral characteristic.

[0071] [Number]

[0072] Here, in the above mathematical formula, "P(fs)" is the probability of being "cancer" when there is a feature quantity fs within the range of size information s when the category is "cancer". "P(f s 0 )" is the probability of being "cancer" when there is a neutral feature quantity f within the range of size information s s 0 is present. "C(f s ,f s 0 )" is an identification value that indicates the case of being "cancer" with a positive value and the case of being "normal" with a negative value for the feature quantity f s present within the range of size information s.

[0073] The identification value acquisition unit 132, according to the size information corresponding to the key area, has different P(f s oIt is preferable to calculate the first and second test values ​​using ). The identification value acquisition unit 132 acquires the size corresponding to the key area and P(f s o The value of ) is obtained from the table in the first storage unit 11. The table contains information for each size of 2 or more and P(f s o This is a table for managing the correspondence between ) and ).

[0074] The useful key region determination unit 133 determines one or more useful key regions that satisfy the selection criteria for selecting a useful key region, using the identification value acquired by the identification value acquisition unit 132 for each of the two or more size information pieces. It is preferable that the selection criteria differ for each size information piece.

[0075] The useful key region determination unit 133 determines, for each of the two or more size information, one or more useful regions whose identification value, for example, acquired by the identification value acquisition unit 132, satisfies the selection criteria. Identification value refers to information about the ability to identify the target category.

[0076] Furthermore, selection criteria include, for example, that the identification value is greater than or equal to a threshold, that the identification value is greater than a threshold, or that the identification value is among the top N key regions.

[0077] The storage unit 134 stores useful key area information relating to one or more useful key areas determined by the useful key area determination unit 133 for each of the two or more size information. For example, the storage unit 134 stores useful key area information in the useful key area information storage unit 114 for each of the two or more size information. For example, the storage unit 134 stores one or more useful key area information paired with an identifier that identifies each of the two or more size information.

[0078] It is preferable for the storage unit 134 to store useful key region information in association with the identification value acquired by the identification value acquisition unit 132.

[0079] The storage unit 134 preferably stores one or more purpose-useful key region information and one or more other-useful key region information. In this case, the storage unit 134 preferably stores the useful key region information in association with a category identifier. There may be three or more categories. Furthermore, purpose-useful key region information is information that identifies the key region of the target category (for example, "cancer"). Other-useful key region information is information that identifies the key region of a category that is not the target (for example, "normal").

[0080] The first output unit 14 outputs various types of information. These types of information include, for example, useful key area information and information indicating that processing has been completed.

[0081] Here, "output" is a concept that includes display on a screen, projection using a projector, printing with a printer, sound output, transmission to an external device, storage on a recording medium, and transfer of processing results to other processing devices or other programs.

[0082] The second storage unit 21, which constitutes the image processing device 2, stores various types of information. These types of information include, for example, a reference image, a target image (described later), useful key region information, and the size of one or more small image fragments. Preferably, the one or more useful key region information in the useful key region information storage unit 114 is useful key region information acquired by the production device 1. The small image fragment size is information that identifies the size of a small image fragment.

[0083] The target image storage unit 211 stores one or more target images. A target image is an image to be processed. A target image is the image that will be the source of the output image described later. A target image is, for example, a medical image. A target image is, for example, a medical image that includes a cancerous area. A target image is, for example, an fMRI image, PET image, ultrasound image, etc., and the type is not limited. A target image is, for example, a TIFF, PNG, JPEG, etc., but the data structure is not limited.

[0084] The useful key area information storage unit 114 of the second storage unit 21 stores useful key area information for each of the two or more size information units. Preferably, the useful key area information storage unit 114 stores one or more target useful key area information and one or more other useful key area information for each of the two or more size information units. A category identifier is associated with such useful key area information.

[0085] The second reception unit 22 receives various instructions and information. These instructions and information include, for example, start instructions, target images, resizing instructions, useful key region information, and similar conditions to be changed. A start instruction is an instruction to start the output processing of the output image.

[0086] Resizing is an instruction to change the size of a small image fragment. Resizing involves size information for the small image fragment. This size information includes, for example, the diameter of the small image fragment and its length and width.

[0087] Any means of inputting instructions and information is acceptable, such as a touch panel, keyboard, mouse, or menu screen.

[0088] The second processing unit 23 performs various processes. These various processes include, for example, the processes performed by the image segmentation unit 231, the key region determination unit 232, the local processing unit 233, the reliability processing unit 234, the boundary determination unit 235, and the image composition unit 236.

[0089] The image segmentation unit 231 acquires two or more small image fragments from the target image, each from a different region. The shape of the small image fragments is not restricted. For example, the shape of the small image fragments can be circular, elliptical, or rectangular. There may be overlap in the regions of two or more small image fragments. It is preferable that the small image fragments have a fixed size (for example, x dots × y dots (where x and y are natural numbers)), but they may also have a variable size received by the second reception unit 22, or the image segmentation unit 231 may automatically acquire two or more sizes and acquire two or more small image fragments for each size. Small image fragments usually have two or more pixels.

[0090] The key region determination unit 232 determines two or more key regions from the target image. Typically, the key region determination unit 232 determines one or more key regions for each of the two or more small image fragments acquired by the image segmentation unit 231. It is preferable for the key region determination unit 232 to determine one or more key regions for each of the two or more size information units. The process by which the key region determination unit 232 determines key regions from an image may be the same as that of the key region determination unit 131.

[0091] The local processing unit 233 obtains the identification value of one or more key regions within each of the two or more small image fragments acquired by the image segmentation unit 231, and obtains an identification representative value which is a representative value of the one or more identification values.

[0092] The local processing unit 233 checks whether the key region information relating to each of the two or more small image fragments acquired by the image segmentation unit 231 is similar to the useful key region information stored in the useful key region information storage unit 114 for each of the one or more key regions within the small image fragment, and acquires an identification value for each key region according to the check result. Next, the local processing unit 233 acquires an identification representative value, which is the absolute value of the representative identification value of one or more key regions within the small image fragment, for each of the two or more small image fragments acquired by the image segmentation unit 231. The representative identification value of one or more key regions is preferably the average of the identification values, but the median or the like may also be used. Furthermore, the absolute value of the identification value of one or more key regions may also be used as the representative identification value of one or more key regions.

[0093] The local processing unit 233 checks whether the key area information and the useful key area information stored in the useful key area information storage unit 114 satisfy the similarity condition for each of the one or more key areas, and obtains a greater identification value the more useful key area information that satisfies the similarity condition.

[0094] The local processing unit 233, for example, checks whether the key area information and the purpose-useful key area information stored in the useful key area information storage unit 114 satisfy the similarity condition for each of the one or more key areas. The more purpose-useful key area information that satisfies the similarity condition, the larger the first check value obtained. The local processing unit 233 obtains the first check value using, for example, formula 1 described above. The purpose category is, for example, "cancer".

[0095] The local processing unit 233, for example, checks whether the key area information and other useful key area information stored in the useful key area information storage unit 114 satisfy the similarity condition for each of the one or more key areas. The more target useful key area information that satisfies the similarity condition, the larger the second check value obtained. The local processing unit 233 obtains the second check value, for example, using formula 2 described above. Note that other categories are, for example, "normal".

[0096] The reliability processing unit 234 performs a first check and a second check corresponding to each of the one or more key regions determined by the key region determination unit 232, and uses the results of the first and second checks to obtain the reliability of each of the one or more key regions. The reliability processing unit 234 may perform the first check by obtaining the result of a first check that has already been performed. The reliability processing unit 234 may perform the second check by obtaining the result of a second check that has already been performed. The result of the first check is, for example, the first check value. The result of the second check is, for example, the second check value. The first check value and second check value that have already been obtained are usually values ​​obtained by the local processing unit 233.

[0097] Confidence level refers to the degree of confidence in the identifying value. Confidence level is used when determining boundaries, as described later. Furthermore, key cancer regions show high confidence levels, while fatty regions show low confidence levels.

[0098] The reliability processing unit 234, for example, for each of the one or more key regions determined by the key region determination unit 232, obtains a first check value that is larger the more regions corresponding to each key region exist in each of the one or more target reference images, and obtains a second check value that is larger the more regions corresponding to each key region exist in each of the one or more other reference images. Then, the reliability processing unit 234 calculates the reliability for each key region, for example, using an increasing function with the first check value and the second check value as parameters. Next, for each of the two or more small image fragments, the reliability processing unit 234 obtains a confidence representative value, which is a representative value of the reliability of each of the one or more key regions belonging to each small image fragment. Here, the confidence representative value is preferably the average value of the reliability, but it may also be the median, etc. Note that one or more target reference images and one or more other reference images may be stored in the second storage unit 21.

[0099] The reliability processing unit 234 obtains the reliability for each of the one or more key regions determined by the key region determination unit 232 using the following formula 4.

[0100]

number

[0101] The boundary determination unit 235 determines a boundary line that separates the region where the identification value of one or more key regions is a positive value (for example, the region of category "cancer") from the region where the identification value of one or more key regions is a negative value (for example, the region of category "normal").

[0102] The boundary determination unit 235 determines a boundary line that separates, for example, an area where the identification value of one or more key areas is positive (e.g., the area of ​​category "cancer") from an area where the identification value of one or more key areas is negative (e.g., the area of ​​category "normal"), and determines a boundary line where the confidence level of one or more key areas on or near the boundary line (e.g., within a threshold distance, less than a threshold distance) satisfies the confidence condition.

[0103] The boundary determination unit 235 determines, for example, a boundary line corresponding to a key region within a small image fragment, where the representative identification value of the small image fragment satisfies the identification condition and the confidence level of one or more key regions within the small image fragment satisfies the confidence condition.

[0104] The identification condition is the condition under which any point or region within a small image fragment is suitable for a boundary. The identification condition is that the value of the representative identification value is small. Another identification condition is that the representative identification value is less than or equal to the first threshold. For example, the identification conditions are "representative identification value = 0" and "representative identification value <= 0.1".

[0105] A preferred representative value for discrimination is ΣC(f,f0), which is the sum of feature quantities f in a small image fragment (or over an appropriate image range). This preferred representative value has the property that when typical features indicating "cancer" and typical features indicating "normal" are mixed in the small image fragment, their values ​​cancel each other out, resulting in a value close to 0.

[0106] The confidence condition is the requirement that the confidence level of one or more key regions in a small image fragment is appropriate for the boundary.

[0107] A preferred confidence representative value is ΣD(f,f0), which is the sum of f values ​​in the small image fragment (or over an appropriate image range). Unlike the aforementioned discrimination representative value, when the small image fragment contains a mixture of typical features indicating "cancer" and typical features indicating "normality," the values ​​do not cancel each other out but rather complement each other, resulting in a large positive value. A preferred confidence representative value close to 0 occurs when the small image fragment (or over an appropriate image range) does not contain typical features indicating either "cancer" or "normality," or when there are hardly any features at all. The confidence condition is, for example, that the confidence representative value is at or above the second threshold or greater than the second threshold. The confidence representative value is a representative value of the confidence of one or more key regions of the small image fragment. The confidence representative value is, for example, the average value of the confidence of one or more key regions, but the median or the like may also be used. The confidence condition is, for example, that the proportion of confidence values ​​of one or more key regions of the small image fragment that are at or above the third threshold or greater than the third threshold is at or above the fourth threshold or greater than the fourth threshold. The confidence condition is, for example, that the proportion of confidence values ​​of one or more key regions of the small image fragment that are at or above the fifth threshold or less than the fifth threshold is at or below the sixth threshold or less than the sixth threshold. The confidence condition is, for example, that the number of confidence values ​​of one or more key regions of the small image fragment that are at or above the seventh threshold or greater than the seventh threshold is at or above the eighth threshold or greater than the eighth threshold. The confidence condition is, for example, that the number of confidence values ​​of one or more key regions of the small image fragment that are at or below the ninth threshold or less than the ninth threshold is at or below the tenth threshold or less than the tenth threshold.

[0108] The boundary determination unit 235 determines, for example, a small image fragment whose identification representative value satisfies the identification condition and whose reliability of one or more key regions satisfies the reliability condition, and determines a boundary line connecting the representative points of each of the two or more small image fragments. The representative points are preferably the centroid or center point of the small image fragment, but may also be points that constitute the outline of the small image fragment.

[0109] The image composition unit 236 constructs an image in which the boundary lines determined by the boundary line determination unit 235 are clearly indicated relative to the target image. The image relative to the target image may be an image using the target image as is, or it may be an image using the identification values ​​of two or more small image fragments acquired relative to the target image.

[0110] The image constructor 236 may, for example, construct an image with clearly defined boundaries on top of the target image, or it may construct an image with clearly defined boundaries on top of an image that reflects the identification representative values ​​of two or more small image fragments acquired by the local processing unit 233.

[0111] The second output unit 24 outputs various types of information. These types of information include, for example, images.

[0112] The image output unit 241 outputs the image constructed by the image composition unit 236. Such an image may also be called an output image. Here, output is a concept that includes display on a screen, projection using a projector, printing with a printer, transmission to an external device, storage on a recording medium, and transfer of processing results to other processing devices or other programs.

[0113] The first storage unit 11, the size information storage unit 110, the image storage unit 111, the reference image storage unit 112, the condition storage unit 113, the useful key area information storage unit 114, the second storage unit 21, and the target image storage unit 211 are preferably made of non-volatile recording media, but can also be made of volatile recording media.

[0114] The process by which information is stored in the first storage unit 11, etc. is not relevant. For example, information may be stored in the first storage unit 11, etc. via a recording medium, information transmitted via a communication line, etc. may be stored in the first storage unit 11, etc., or information input via an input device may be stored in the first storage unit 11, etc.

[0115] The first reception unit 12 and the second reception unit 22 can be implemented using device drivers for input means such as touch panels and keyboards, or control software for menu screens. The first reception unit 12 and the second reception unit 22 may also be implemented using communication means.

[0116] The first processing unit 13, control unit 130, key region determination unit 131, identification value acquisition unit 132, useful key region determination unit 133, storage unit 134, second processing unit 23, image segmentation unit 231, key region determination unit 232, local processing unit 233, reliability processing unit 234, boundary determination unit 235, and image composition unit 236 can typically be implemented using a processor, memory, etc. The processing procedures of the first processing unit 13, etc., are typically implemented in software, and this software is recorded on a recording medium such as ROM. However, it may also be implemented in hardware (dedicated circuitry). The processor can be, for example, a CPU, MPU, GPU, etc., and its type is not limited.

[0117] The first output unit 14, the second output unit 24, and the image output unit 241 may or may not be considered to include output devices such as displays and speakers. The first output unit 14 can be implemented using driver software for an output device, or driver software for an output device and an output device. The first output unit 14, the second output unit 24, and the image output unit 241 may also be implemented using communication means.

[0118] Next, an example of the operation of the production apparatus 1 will be explained using the flowchart in Figure 3. For example, when the first receiving unit 12 receives a start instruction, the processing from step S301 onwards will be performed.

[0119] (Step S301) The control unit 130 assigns 1 to counter i.

[0120] (Step S302) The control unit 130 determines whether or not the i-th image exists in the image storage unit 111. If the i-th image exists, the process proceeds to step S303; if the i-th image does not exist, the process ends.

[0121] (Step S303) The key region determination unit 131 obtains the i-th image from the image storage unit 111.

[0122] (Step S304) The control unit 130 assigns 1 to counter j.

[0123] (Step S305) The control unit 130 determines whether the j-th size information is stored in the size information storage unit 110. If the j-th size information is stored, the unit proceeds to step S306; otherwise, the unit proceeds to step S315.

[0124] (Step S306) The control unit 130 obtains the j-th size information from the size information storage unit 110.

[0125] (Step S307) The key region determination unit 131 obtains key region information for one or more key regions whose size matches the j-th size information obtained in step S306 from the i-th image obtained in step S303, and temporarily stores it in a buffer (not shown) in pairs with the j-th size information. Note that the process of obtaining key region information for a key region of a specific size from an image is possible by known technology.

[0126] (Step S308) The control unit 130 assigns 1 to counter k.

[0127] (Step S309) The control unit 130 determines whether the k-th key region information, which is paired with the j-th size information, exists in a buffer (not shown). If the k-th key region information exists, the process proceeds to step S310; otherwise, the process proceeds to step S314. The k-th key region information is the information stored in step S307.

[0128] (Step S310) The identification value acquisition unit 132 acquires the identification value of the k-th key region information that is paired with the j-th size information. An example of such identification value acquisition process will be explained using the flowchart in Figure 4.

[0129] (Step S311) The useful key area determination unit 133 obtains the selection conditions from the condition storage unit 113. The useful key area determination unit 133 determines whether the identification value obtained in step S310 satisfies the selection conditions. If the selection conditions are met, the unit proceeds to step S312; otherwise, the unit proceeds to step S313.

[0130] (Step S312) The storage unit 134 associates the j-th size information, the k-th key area information, and the identification value of the k-th key area information, and stores them in the useful key area information storage unit 114.

[0131] (Step S313) The control unit 130 increments the counter k by 1. Return to step S309.

[0132] (Step S314) The control unit 130 increments counter j by 1. Return to step S305.

[0133] (Step S315) The control unit 130 increments counter i by 1. Return to step S302.

[0134] Next, an example of the identification value acquisition process in step S310 will be explained using the flowchart in Figure 4.

[0135] (Step S401) The identification value acquisition unit 132 performs initialization processing. That is, the identification value acquisition unit 132 assigns 0 to the variables "target reference information" and "other reference information".

[0136] The variable "Target Reference Information" is information obtained from the target reference image and is the source information for obtaining the identification value of the target key region. The variable "Target Reference Information" is, for example, the number of target reference images that have key region information similar to (satisfying the similarity condition of) the target key region information. The variable "Target Reference Information" is, for example, the sum of the similarity scores of the similar key region information in target reference images that have key region information similar to the target key region information.

[0137] The variable "Other Reference Information" is information obtained from other reference images and is the source information for obtaining the identification value of the target key region. For example, the variable "Other Reference Information" is the number of other reference images that have key region information similar to (satisfying the similarity condition of) the target key region information. For example, the variable "Other Reference Information" is the sum of the similarity scores of the similar key region information in other reference images that have key region information similar to the target key region information.

[0138] (Step S402) The identification value acquisition unit 132 assigns 1 to counter i.

[0139] (Step S403) The identification value acquisition unit 132 determines whether the i-th target reference image exists in the reference image storage unit 112. If the i-th target reference image exists, the process proceeds to step S404; otherwise, the process proceeds to step S413.

[0140] (Step S404) The identification value acquisition unit 132 acquires the i-th target reference image from the reference image storage unit 112.

[0141] (Step S405) The key region determination unit 131 obtains one or more reference key region pieces of key region information from the i-th target reference image acquired in step S404.

[0142] (Step S406) The identification value acquisition unit 132 assigns 1 to counter j.

[0143] (Step S407) The identification value acquisition unit 132 determines whether or not the j-th reference key area information exists among the reference key area information acquired in step S405. If the j-th reference key area information exists, the unit proceeds to step S408; otherwise, the unit proceeds to step S411.

[0144] (Step S408) The identification value acquisition unit 132 calculates the similarity between the target key region information (the k-th key region information in S309) and the j-th reference key region information.

[0145] (Step S409) The identification value acquisition unit 132 determines whether the similarity calculated in step S408 satisfies the similarity conditions. If the similarity conditions are met (they are similar), the unit proceeds to step S410; otherwise, the unit proceeds to step S412.

[0146] (Step S410) The identification value acquisition unit 132 updates the variable "target reference information". Normally, the identification value acquisition unit 132 increases the variable "target reference information". For example, the identification value acquisition unit 132 adds 1 to the variable "target reference information". For example, the identification value acquisition unit 132 adds the similarity calculated in step S408 to the variable "target reference information".

[0147] Alternatively, instead of proceeding to step S411, the similarity between the key region information and all the reference key region information of the i-th target reference image may be obtained, and the target reference information of the key region information may be obtained using this similarity.

[0148] (Step S411) The identification value acquisition unit 132 increments counter i by 1. Return to step S402.

[0149] (Step S412) The identification value acquisition unit 132 increments counter j by 1. Return to step S407.

[0150] (Step S413) The identification value acquisition unit 132 assigns 1 to counter k.

[0151] (Step S414) The identification value acquisition unit 132 determines whether the k-th other reference image exists in the reference image storage unit 112. If the k-th other reference image exists, the process proceeds to step S415; otherwise, the process proceeds to step S424.

[0152] (Step S415) The identification value acquisition unit 132 acquires the k-th other reference image from the reference image storage unit 112.

[0153] (Step S416) The key region determination unit 131 obtains one or more reference key region pieces of key region information from the k-th other reference image obtained in step S415.

[0154] (Step S417) The identification value acquisition unit 132 assigns 1 to counter l.

[0155] (Step S418) The identification value acquisition unit 132 determines whether the l-th reference key area information exists among the reference key area information acquired in step S416. If the l-th reference key area information exists, the unit proceeds to step S419; otherwise, the unit proceeds to step S422.

[0156] (Step S419) The identification value acquisition unit 132 calculates the similarity between the target key region information and the l-th reference key region information.

[0157] (Step S420) The identification value acquisition unit 132 determines whether the similarity calculated in step S419 satisfies the similarity conditions. If the similarity conditions are met, the unit proceeds to step S421; otherwise, the unit proceeds to step S423.

[0158] (Step S421) The identification value acquisition unit 132 updates the variable "other reference information". Normally, the identification value acquisition unit 132 increases the variable "other reference information". For example, the identification value acquisition unit 132 adds 1 to the variable "other reference information". For example, the identification value acquisition unit 132 adds the similarity calculated in step S408 to the variable "other reference information".

[0159] Alternatively, instead of proceeding to step S422, the similarity between the key region information and all the reference key region information of the i-th other reference image may be obtained, and the other reference information of the key region information may be obtained using this similarity.

[0160] (Step S422) The identification value acquisition unit 132 increments the counter k by 1. Return to step S414.

[0161] (Step S423) The identification value acquisition unit 132 increments counter l by 1. Return to step S418.

[0162] (Step S424) The identification value acquisition unit 132 calculates the identification value using the variable "target reference information" and the variable "other reference information". It then returns to the higher-level processing.

[0163] The identification value acquisition unit 132 typically calculates the identification value using an increasing function that takes the value of the variable "target reference information" as a parameter. Alternatively, the identification value acquisition unit 132 typically calculates the identification value using a decreasing function that takes the value of the variable "other reference information" as a parameter. However, since "target reference information" and "other reference information" have the property that if one increases, the other decreases, the identification value acquisition unit 132 may also calculate the identification value using an increasing function that takes the value of the variable "other reference information" as a parameter in order to obtain the effect of distinguishing categories.

[0164] The identification value acquisition unit 132 calculates a first test value that identifies the likelihood of cancer, for example, using an increasing function (e.g., formula 1) that uses the value of the variable "target reference information" as a parameter. The identification value acquisition unit 132 also calculates a second test value that identifies the likelihood of normality, for example, using an increasing function (e.g., formula 2) that uses the value of the variable "other reference information" as a parameter. Next, the identification value acquisition unit 132 calculates the identification value, for example, using "first test value - second test value" (formula 3).

[0165] Next, an example of the operation of the image processing device 2 will be explained using the flowchart in Figure 5. Here, we will describe the process of obtaining an output image from a target image.

[0166] (Step S501) The second reception unit 22 determines whether or not it has received the target image. If it has received the target image, it proceeds to step S502; if it has not received the target image, it proceeds to step S521. Note that receiving a target image is, for example, receiving the identifier of a target image stored in the target image storage unit 211.

[0167] (Step S502) The key region determination unit 232 acquires the target image received in step S501. Next, the key region determination unit 232 determines one or more key regions from the target image and acquires key region information for each of the one or more key regions.

[0168] (Step S503) The image splitting unit 231 assigns 1 to counter i.

[0169] (Step S504) The image segmentation unit 231 determines whether the i-th small image fragment size exists in the second storage unit 21. If the i-th small image fragment size exists, the process proceeds to step S505; otherwise, the process returns to step S501. It is preferable that the first small image fragment size acquired is the largest size among the small image fragment sizes in the second storage unit 21.

[0170] (Step S505) The image splitting unit 231 obtains the size of the i-th small image piece from the second storage unit 21.

[0171] (Step S506) The image splitting unit 231 assigns 1 to counter j.

[0172] (Step S507) The image segmentation unit 231 determines whether or not a small image fragment of the size of the i-th small image fragment and the j-th small image fragment can be obtained (whether or not it exists) within the target image. If the j-th small image fragment can be obtained, the process proceeds to step S507; otherwise, the process proceeds to step S515.

[0173] (Step S508) The local processing unit 233 assigns 1 to counter k.

[0174] (Step S509) The local processing unit 233 determines whether or not the k-th key region information exists in the j-th small image fragment. If the k-th key region information exists, the process proceeds to step S510; otherwise, the process proceeds to step S513.

[0175] (Step S510) The local processing unit 233 obtains the identification value of the k-th key region information. It is preferable to use the useful key region information stored in the useful key region information storage unit 114 for the process of obtaining the identification value of the key region information. However, the process of obtaining the identification value of the key region information may be the same as the process described in Figure 4.

[0176] (Step S511) The confidence processing unit 234 obtains the confidence level of the k-th key area information. The confidence processing unit 234 obtains the confidence level by, for example, calculating the sum of the first and second test values ​​obtained in step S510.

[0177] (Step S512) The local processing unit 233 increments the counter k by 1. Return to step S509.

[0178] (Step S513) The local processing unit 233 obtains an identification representative value, which is the absolute value of the representative value of the identification value of one or more key regions in the j-th small image fragment, associates it with the j-th small image fragment, and stores the identification representative value in a buffer (not shown).

[0179] (Step S514) The image splitting unit 231 increments the counter j by 1. Return to step S507.

[0180] (Step S515) The boundary determination unit 235 performs boundary determination processing using the representative identification value and confidence level of each of the two or more small image fragments. An example of the boundary determination processing will be explained using the flowchart in Figure 6.

[0181] (Step S516) The image composition unit 236 performs image composition processing. An example of image composition processing will be explained using the flowchart in Figure 7.

[0182] (Step S517) The image composition unit 236 stores the images composed in step S516, associating them with the size of the target image and the small image fragments. The storage location of these images is, for example, the second storage unit 21, but is not limited to that location.

[0183] (Step S518) The image output unit 241 determines whether or not to output an image. If an image is to be output, the process proceeds to step S519; otherwise, it proceeds to step S520. Note that if an image is to be output, for example, if no image has been output, or if an image corresponding to the accepted resizing has been obtained.

[0184] (Step S519) The image output unit 241 outputs the image constructed in step S516.

[0185] (Step S520) The image splitting unit 231 increments the counter i by 1. Return to step S504.

[0186] (Step S521) The second reception unit 22 determines whether or not the size change has been accepted. If the size change has been accepted, the process proceeds to step S522; otherwise, the process returns to step S501.

[0187] (Step S522) The second processing unit 23 determines whether an image composed of small image fragments of the size corresponding to the size change received in step S521 has already been stored. If such an image exists, the process proceeds to step S524; otherwise, the process proceeds to step S506.

[0188] (Step S523) The image output unit 241 acquires an image corresponding to the sub-image fragment size corresponding to the resizing received in step S521.

[0189] (Step S524) The image output unit 241 outputs the image acquired in step S523. Return to step S501.

[0190] In the flowchart shown in Figure 5, processing is terminated by power-off or processing termination interrupts.

[0191] Next, an example of the boundary determination process in step S515 will be explained using the flowchart in Figure 6.

[0192] (Step S601) The boundary determination unit 235 assigns 1 to counter i.

[0193] (Step S602) The boundary determination unit 235 determines whether or not the i-th small image fragment exists. If the i-th small image fragment exists, the process proceeds to step S603; if the i-th small image fragment does not exist, the process proceeds to step S609.

[0194] (Step S603) The boundary determination unit 235 obtains the identification representative value of the i-th small image fragment.

[0195] (Step S604) The boundary determination unit 235 determines whether the representative identification value obtained in step S603 satisfies the identification conditions. If the identification conditions are met, the unit proceeds to step S605; otherwise, the unit proceeds to step S608.

[0196] (Step S605) The boundary determination unit 235 obtains the confidence level of one or more key regions in the i-th small image piece.

[0197] (Step S606) The boundary determination unit 235 determines whether the confidence score of 1 or more obtained in step S605 satisfies the confidence condition. If the confidence condition is met, the process proceeds to step S607; otherwise, the process proceeds to step S608.

[0198] (Step S607) The boundary determination unit 235 acquires a representative point of the region of the i-th small image piece and temporarily stores the representative point (e.g., coordinate value) in a buffer (not shown). This representative point constitutes boundary information.

[0199] (Step S608) The boundary determination unit 235 increments the counter i by 1. Return to step S602.

[0200] (Step S609) The boundary determination unit 235 acquires boundary information, which is information about the boundary line through which one or more representative points obtained in step S607 pass, and stores the boundary information in a buffer (not shown). It then returns to the higher-level processing.

[0201] Next, an example of the image composition processing in step S516 will be explained using the flowchart in Figure 7.

[0202] (Step S701) The image composition unit 236 acquires the target image.

[0203] (Step S702) The image composition unit 236 assigns 1 to counter i.

[0204] (Step S703) The image composition unit 236 determines whether or not the i-th sub-image fragment exists in the target image. If the i-th sub-image fragment exists, the unit proceeds to step S704; otherwise, the unit proceeds to step S707.

[0205] (Step S704) The image compiling unit 236 obtains the identification value corresponding to the i-th small image fragment.

[0206] (Step S705) The image composing unit 236 makes the target image an image in which the identification value obtained in step S704 is reflected in the region of the i-th small image piece. For example, the image composing unit 236 sets the color of the region of the i-th small image piece in the target image to the color corresponding to the identification value obtained in step S704.

[0207] (Step S706) The image composition unit 236 increments the counter i by 1. Return to step S703.

[0208] (Step S707) The image composition unit 236 uses boundary information stored in a buffer (not shown) to draw a boundary line on the image processed in step S705, and constructs an image with the boundary line clearly indicated. The process returns to the higher-level processing unit.

[0209] The boundary line, for example, is the boundary between a cancerous area and a normal area.

[0210] The following describes a specific example of the operation of the production apparatus 1 in this embodiment.

[0211] The size information storage unit 110 of the production device 1 stores seven size information values: "0 < size <= 2", "2 < size <= 4", "4 < size <= 6", "6 < size <= 8", "8 < size <= 10", "10 < size <= 12", and "12 < size". The size information is a condition for determining the size of the key area to be detected. In this case, the size information is the number of pixels.

[0212] The image storage unit 111 stores one or more medical images. The medical images include areas of cancer and normal areas.

[0213] The reference image storage unit 112 stores two or more reference images associated with the category identifier "cancer." These reference images are images of cancer cell regions. The reference image storage unit 112 also stores two or more reference images associated with the category identifier "normal." These reference images are images of normal cell regions.

[0214] In this situation, the user inputs a start command to production device 1. Next, the first receiving unit 12 of production device 1 receives the start command.

[0215] Then, for each of the one or more medical images in the image storage unit 111, the following processing is performed, and a large amount of useful key region information is accumulated in the useful key region information storage unit 114 for each size information.

[0216] In other words, first the control unit 130 obtains the size information "0 < size <= 2" from the size information storage unit 110.

[0217] Furthermore, the key region determination unit 131 acquires medical images (hereinafter referred to as "images" as appropriate) from the image storage unit 111.

[0218] Next, the key region determination unit 131 determines one or more key points (an example of a key region) from the acquired image that have a size that matches the size information "0 < size <= 2". To explain using SIFT features as preferred image features, as shown in Figure 8, the key region determination unit 131 applies different scales (σ, k) to the acquired image 801. 1 σ, k 2 σ, k 3 σ, k 4 σ···(σ,k x A smoothed image is obtained (where x is a positive number and x is a natural number). Next, the key region determination unit 131 obtains a Doc image, which is the difference image of two adjacent smoothed images (see Figure 8).

[0219] Next, the key region determination unit 131 detects extreme values ​​from each Doc image. The key region determination unit 131 compares the DoG value of the pixel of interest with 26 neighbors in the image scale space. If the value is greater than or less than the 26 neighbors, that point is designated as a candidate key point (see Figure 9). In Figure 9, the key region determination unit 131 compares the DoG value of the pixel of interest (the central pixel of the colored pixels in the middle Doc image of the set of three Doc images) with the DoG values ​​of pixels adjacent to the pixel of interest, and with the Doc images of corresponding pixels in the upper and lower Doc images of the set of three Doc images (the colored pixels in the upper and lower Doc images). If the DoG value of the pixel of interest is greater than or less than the DoG values ​​of the other pixels being compared, that pixel of interest is designated as a candidate key point.

[0220] Furthermore, the key region determination unit 131 narrows down the candidate key points by excluding points on edges and those with small DoG values ​​(for example, those with a DoG value of 0.03 or less) from the candidate key points. It is preferable for the key region determination unit 131 to perform subpixel position estimation and contrast thresholding.

[0221] Next, the key region determination unit 131 calculates the gradient direction from the region corresponding to the scale that matches the size information "0 < size <= 2" of each of the remaining one or more key points. A conceptual diagram of this is shown in Figure 10. The key region determination unit 131 also adds a weight obtained by multiplying the Gaussian window and the gradient intensity to the gradient direction histogram for 36 directions. Furthermore, the key region determination unit 131 assigns the peak with the maximum value of the histogram, for example, 80% or more, as the orientation of the key point (see Figure 11). The key point that is assigned in this way is the selected key point. Note that some key points may have two orientations.

[0222] Next, the key region determination unit 131 obtains a 128-dimensional feature vector as key region information based on the detected orientation. First, the key region determination unit 131 rotates the region describing the features to match the orientation of the keypoint. The feature vector has gradient information around the keypoint as an element. This gradient information is obtained from a circle with the keypoint as the center and the scale of that keypoint as the radius. For example, this region is divided into 4x4 blocks, and each block consists of 8 directions, so the region has a total of 128 dimensions of features. The key region determination unit 131 uses these features to create a gradient direction histogram as shown in Figure 12. At that time, the key region determination unit 131 normalizes the length of the 128-dimensional feature vector by the sum of the vectors. As a result, the key region determination unit 131 obtains a SIFT feature vector that is resistant to changes in illumination.

[0223] The key area determination unit 131 performs the above processing for each piece of size information in the size information storage unit 110, and determines the key area for each piece of size information.

[0224] Furthermore, the inventor's experiments revealed that, as shown in Figure 13, key regions with small size (keypoint size) are often cancerous regions, while key regions with large size are often normal regions. In other words, in Figure 13, when the keypoint size is small (a), it is often a cancerous region, and when the keypoint size is large (b), it is often a normal region.

[0225] By performing the above processing or other processing that can extract small image regions useful for evaluating differences in categories on multiple medical images, a large amount of key region information is obtained. In this case, the key region information is a 128-dimensional feature vector calculated as SIFT features.

[0226] Next, the identification value acquisition unit 132 calculates the identification value of each of the one or more key region information acquired by the key region determination unit 131 using the above-described formulas 1, 2, and 3.

[0227] Next, the useful key region determination unit 133 determines whether the identification value of one or more key region information matches the selection criteria, and acquires one or more key region information that matches the selection criteria as useful key region information.

[0228] Next, the storage unit 134 associates one or more useful key area information pieces with the identification value of each useful key area information piece for each size information piece in the size information storage unit 110, and stores them in the useful key area information storage unit 114. It is preferable for the storage unit 134 to also store the first check value and the second check value in the useful key area information storage unit 114, associating them with each useful key area information piece.

[0229] Through the above process, the production apparatus 1 was able to store a large amount of useful key region information with a high ability to identify "guns" for each size information in the size information storage unit 110. Here, the useful key region information with a high ability to identify "guns" is referred to as the target useful key region information.

[0230] Furthermore, by performing the same algorithm as described above, the production device 1 was able to accumulate a large number of useful key region information with a high ability to identify "normal" for each size information in the size information storage unit 110. Note that if the "cancer" image and the "normal" image were swapped and the production device 1 operated using the same algorithm as above, it would be possible to accumulate a large number of useful key region information with a high ability to identify "normal". Here, the useful key region information with a high ability to identify "normal" is referred to as other useful key region information.

[0231] Next, the image processing device 2 constructs and outputs an output image from a medical image including the cancerous region, using the numerous useful key region information for each size information in the size information storage unit 110, which has been accumulated by the production device 1, through the processing described below. It is assumed that the useful key region information storage unit 114 of the image processing device 2 stores numerous target useful key region information and numerous other useful key region information acquired by the production device 1 for each size information in the size information storage unit 110. It is also assumed that the target image storage unit 211 stores a medical image including the cancerous region (hereinafter referred to as the "target image" as appropriate).

[0232] In this situation, suppose the user inputs a start command to the image processing device 2. Then, the second receiving unit 22 of the image processing device 2 receives the start command.

[0233] Next, the key region determination unit 232 acquires the target image corresponding to the start instruction from the target image storage unit 211. Then, the key region determination unit 232 determines one or more key regions from the target image and acquires key region information for each key region.

[0234] Next, the image segmentation unit 231 obtains the small image fragment size included in the start instruction, or the default small image fragment size. Here, it is assumed that the small image fragments are circular, and the small image fragment size is the diameter of the circle.

[0235] The image segmentation unit 231 acquires two or more small image fragments of the acquired small image fragment size from the target image while shifting the center of the small image fragment size.

[0236] Next, the local processing unit 233 obtains the identification value of the key region information within each of the two or more small image fragments. The process of obtaining the identification value of the key region information is a process using the above equations 1, 2, and 3.

[0237] Next, the local processing unit 233 obtains a representative identification value for each of the two or more small image fragments, which is the absolute value of the average of the identification values ​​of one or more key regions within the small image fragment.

[0238] Furthermore, the reliability processing unit 234 obtains the reliability of key region information for each of the two or more small image fragments within the small image fragment. The process for obtaining the reliability of key region information is a process using the above-mentioned equations 1, 2, and 4.

[0239] Next, the confidence processing unit 234 obtains a confidence representative value for each of the two or more small image fragments, which is the average of the confidence values ​​of one or more key regions within the small image fragment.

[0240] Next, the boundary determination unit 235 determines two or more small image fragments from among the two or more small image fragments such that the identification representative value satisfies the identification condition (for example, "identification representative value <= first threshold") and the confidence representative value satisfies the confidence condition (for example, "identification representative value <= second threshold"). Next, the boundary determination unit 235 determines the coordinate values ​​of the center points of the two or more determined small image fragments (for example, "(x1,y1)(x2,y2)···(x n ,y n This constitutes boundary information including ")".

[0241] Next, the image composition unit 236 acquires the target image. Next, the image composition unit 236 acquires the identification value corresponding to each of the two or more sub-image fragments of the target image. Next, the image composition unit 236 reflects the identification value in the region of each of the two or more sub-image fragments of the target image. For example, the image composition unit 236 fills the region of the center circle of each of the two or more sub-image fragments of the target image in red if the identification value of the sub-image fragment is positive, and in blue if it is negative. Next, the image composition unit 236 uses the boundary line information constructed by the boundary line determination unit 235 to connect two or more points included in the boundary line information with lines in order of proximity, and draws lines on the target image. Through the above processing, the image composition unit 236 has constructed the output image.

[0242] Next, the image output unit 241 outputs the image constructed by the image composition unit 236. An example of such output is shown in Figure 14. In Figure 14, the cancerous region and the normal region are colored separately, clearly indicating the boundary between the two regions. In Figure 14, 1402 in the target image 1401 is the boundary line between the cancerous region 1403 and the normal region 1404. Regions 1406 and 1407 are enlarged versions of region 1405, where the cancerous region and the normal region exist within a narrow range.

[0243] It is also possible to adopt a simpler configuration in which the output of the image processing device 2 is boundary information instead of an image.

[0244] As described above, the production apparatus 1 of this embodiment can obtain useful information for obtaining an image that accurately detects the region of a target category from a target image containing two or more categories.

[0245] Furthermore, according to the image processing device 2 of this embodiment, an image is obtained for accurately detecting the region of a target category from a target image containing two or more categories.

[0246] Furthermore, the image processing device 2 of this embodiment can clearly show the shape of the cancerous area by displaying the boundary line between the cancerous area and the normal area. As a result, it becomes easier to determine whether the cancerous area is primary cancer or metastatic cancer. It should be noted that metastatic cancer areas tend to be similar to a circular shape, while primary cancers tend to have an irregular shape.

[0247] Furthermore, according to the image processing device 2 of this embodiment, the output image can be dynamically changed in response to changes in the accepted similar conditions.

[0248] In this embodiment, the image processing device 2 may acquire the boundary lines to the target image without acquiring small image fragments from the target image. A block diagram of the image processing device 2 in this case is shown in Figure 15. The image processing device 2 also includes a reference image storage unit 112 that stores one or more target reference images which are images containing the region of the target category and one or more other target reference images which are images which do not contain the region of the target category; a useful key region information storage unit 114 that stores one or more pieces of useful key region information; a target image storage unit 211 that stores the target image which is the image to be processed; a key region determination unit 131 that determines two or more key regions from the target image; a local processing unit 233 that uses the one or more pieces of useful key region information to acquire the identification value of each of the two or more key regions acquired by the key region determination unit; and a first inspection, which is an inspection of whether or not the region corresponding to each of the one or more key regions determined by the key region determination unit exists in each of the one or more target reference images. The image processing device 2 comprises: a reliability processing unit 234 that performs a second inspection, which is an inspection to determine whether or not a region corresponding to each of the one or more key regions determined by the key region determination unit exists in each of the one or more other reference images, and uses the results of the first and second inspections to obtain the reliability of each of the two or more key regions; a boundary line determination unit 235 that determines a boundary line that separates a region where the identification value of the key region is a positive value from a region where the identification value of the key region is a negative value, and determines a boundary line where the reliability of one or more key regions on or near the boundary line satisfies the reliability condition; an image configuration unit 236 that constructs an image in which the boundary line determined by the boundary line determination unit is clearly indicated for the target image; and an image output unit 241 that outputs the image configured by the image configuration unit.

[0249] Furthermore, the processing in this embodiment may be implemented by software. This software may be distributed by software download or the like. Alternatively, this software may be recorded on a recording medium such as a CD-ROM and distributed. This also applies to other embodiments in this specification. The software that implements the production apparatus 1 in this embodiment is the following program. In other words, this program is a program that causes a computer, a computer capable of accessing an image storage unit where images are stored, to function as a key area determination unit that determines one or more key areas from the image that satisfy key conditions for each of two or more size information relating to the size of the key area; an identification value acquisition unit that acquires identification value, which is information relating to the ability to determine the area of ​​the target category and relating to each of the one or more key areas determined by the key area determination unit; a useful key area determination unit that determines one or more useful key areas that satisfy selection conditions for selecting a useful key area using the identification value acquired by the identification value acquisition unit; and a storage unit that stores useful key area information relating to the one or more useful key areas determined by the useful key area determination unit, in association with each of the two or more size information.

[0250] Furthermore, the software that implements the image processing device 2 is a program as follows. In other words, this program is a computer that can access a reference image storage unit which stores one or more target reference images which are images that include the region of the target category and one or more other target reference images which are images that do not include the region of the target category, a useful key region information storage unit which stores one or more useful key region information associated with two or more size information items, and a target image storage unit which stores the target image which is the image to be processed, an image division unit which acquires two or more small image fragments of different regions from the target image, a key region determination unit which determines two or more key regions from the target image, a local processing unit which, for each of the two or more small image fragments acquired by the image division unit, acquires the identification value of one or more key regions in the small image fragment using the one or more useful key region information, and acquires an identification representative value which is a representative value of the one or more identification values, and the region corresponding to each of the one or more key regions determined by the key region determination unit A reliability processing unit that performs a first inspection, which is an inspection to check whether or not a key exists in each of the one or more target reference images, and a second inspection, which is an inspection to check whether or not a region corresponding to each of the one or more key regions determined by the key region determination unit exists in each of the one or more other reference images, and uses the results of the first and second inspections to obtain the reliability of each of the one or more key regions; a boundary determination unit that detects small image fragments or key regions within small image fragments where the identification representative value is less than or equal to a first threshold or a value smaller than the first threshold, and where the reliability of one or more key regions in the small image fragment is large enough to satisfy the reliability condition, and determines the boundary line corresponding to the small image fragment or the key region; an image constructing unit that constructs an image in which the boundary line determined by the boundary determination unit is clearly indicated for the target image; and an output unit that outputs the image constructed by the image constructing unit.

[0251] Figure 16 also shows the appearance of a computer that executes the program described herein to realize the production apparatus 1 and image processing apparatus 2 of the various embodiments described above. The embodiments described above can be realized with computer hardware and computer programs executed thereon. Figure 16 is an overview of this computer system 300, and Figure 17 is a block diagram of the system 300.

[0252] In Figure 16, the computer system 300 includes a computer 301 with a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

[0253] In Figure 17, the computer 301 includes, in addition to the CD-ROM drive 3012, an MPU 3013, a bus 3014 connected to the CD-ROM drive 3012, a ROM 3015 for storing programs such as boot-up programs, a RAM 3016 connected to the MPU 3013 for temporarily storing instructions for application programs and providing temporary storage space, and a hard disk 3017 for storing application programs, system programs, and data. Although not shown here, the computer 301 may further include a network card for providing connectivity to a LAN.

[0254] The program that causes the computer system 300 to execute the functions of the image processing device 2, etc., as described above, may be stored on the CD-ROM 3101, inserted into the CD-ROM drive 3012, and then transferred to the hard disk 3017. Alternatively, the program may be transmitted to the computer 301 via a network (not shown) and stored on the hard disk 3017. The program is loaded into the RAM 3016 during execution. The program may also be loaded directly from the CD-ROM 3101 or the network.

[0255] The program does not necessarily have to include an operating system (OS) or third-party program that causes the computer 301 to execute functions such as the image processing device 2 of the above embodiment. The program only needs to include the instruction portion that calls appropriate functions (modules) in a controlled manner and obtains the desired result. How the computer system 300 operates is well known, so a detailed explanation is omitted.

[0256] In the above program, steps such as sending information and receiving information do not include hardware-based processing, such as processing performed by a modem or interface card in the transmission step (processing that can only be performed by hardware).

[0257] Furthermore, the computer running the above program may be a single computer or multiple computers. In other words, it may perform centralized processing or distributed processing.

[0258] Furthermore, in each of the above embodiments, each process may be implemented by centralized processing by a single device, or by distributed processing by multiple devices.

[0259] As an example of distributed processing, the following can be given: In other words, when performing distributed processing between a remote island where it is difficult to secure sufficient communication bandwidth and a base diagnostic center, a remote input unit (not shown) for inputting target images and a boundary information receiving unit (not shown) can be placed on the remote island, and by setting the output of the image processing device 2 to only boundary information, the output results can be quickly transmitted from the base diagnostic center to the remote island.

[0260] It goes without saying that the present invention is not limited to the embodiments described above, and various modifications are possible, all of which are also included within the scope of the present invention. [Industrial applicability]

[0261] As described above, the image processing apparatus 2 according to the present invention has an effect of obtaining an image for accurately detecting a region of a target category from a target image including two or more categories, and is useful as an image processing apparatus or the like.

Explanation of Signs

[0262] 1 Production apparatus 2 Image processing apparatus 11 First storage unit 12 First reception unit 13 First processing unit 14 First output unit 21 Second storage unit 22 Second reception unit 23 Second processing unit 24 Second output unit 110 Size information storage unit 111 Image storage unit 112 Reference image storage unit 113 Condition storage unit 114 Useful key region information storage unit 130 Control unit 131, 232 Key region determination unit 132 Identification value acquisition unit 133 Useful key region determination unit 134 Accumulation unit 211 Target image storage unit 231 Image division unit 233 Local processing unit 234 Reliability processing unit 235 Boundary line determination unit 236 Image composition unit 241 Image output unit

Claims

1. A reference image storage unit that stores one or more target reference images which are images that include the region of the target category, and one or more other target reference images which do not include the region of the target category, A useful key area information storage unit stores one or more useful key area information items, each associated with two or more size information items. A target image storage unit where the target image, which is the image to be processed, is stored, An image segmentation unit that acquires two or more small image fragments from different regions from the aforementioned target image, A key region determination unit that determines two or more key regions from the aforementioned target image, A local processing unit obtains, for each of the two or more small image fragments acquired by the image segmentation unit, the identification value of one or more key regions within the small image fragment using the one or more useful key region information, and a representative identification value which is a representative value of the one or more identification values. A reliability processing unit performs a first inspection, which is an inspection to determine whether the region corresponding to each of the one or more key regions determined by the key region determination unit exists in each of the one or more target reference images, and a second inspection, which is an inspection to determine whether the region corresponding to each of the one or more key regions determined by the key region determination unit exists in each of the one or more other reference images, and uses the results of the first inspection and the second inspection to obtain the reliability of each of the one or more key regions, A boundary determination unit detects small image fragments or key regions within small image fragments where the aforementioned identification representative value corresponds to a value less than or equal to a first threshold, and where the reliability of one or more key regions of the small image fragment is large enough to satisfy the reliability condition, and determines the boundary line corresponding to the small image fragment or the key region. An image component comprising the target image and an image in which the boundary line determined by the boundary line determination unit is clearly indicated, An image processing apparatus comprising: an image composing unit and an output unit that outputs the image composed by the image composing unit.

2. The aforementioned reliability processing unit, For each of the one or more key regions determined by the key region determination unit, a larger first check value is obtained as the number of regions corresponding to each key region in the one or more target reference images increases, and a larger second check value is obtained as the number of regions corresponding to each key region in the one or more other reference images increases. The confidence level is then calculated using an increasing function with the first check value and the second check value as parameters. The image processing apparatus according to claim 1, wherein a different first inspection value and a different second inspection value are obtained according to the size information corresponding to the key region.

3. The aforementioned reliability processing unit, For each of the one or more key regions determined by the key region determination unit, a larger first check value is obtained as the number of regions corresponding to each key region in each of the one or more target reference images increases, and a larger second check value is obtained as the number of regions corresponding to each key region in each of the one or more other reference images increases. The confidence level is calculated using an increasing function with the first check value and the second check value as parameters, and for each of the two or more small image fragments, a confidence representative value is obtained, which is a representative value of the confidence level of each of the one or more key regions belonging to each small image fragment. The boundary determination unit, The image processing apparatus according to claim 2, wherein the identified representative value is two or more small image fragments corresponding to a value less than or equal to a first threshold, and the trusted representative value is determined to be two or more small image fragments corresponding to a value greater than or equal to a second threshold, and a boundary line connecting the representative points of each of the two or more small image fragments is determined.

4. The image processing apparatus according to any one of claims 1 to 3, wherein the target image is a medical image.

5. An image processing method implemented by a reference image storage unit that stores one or more target reference images which are images containing the region of the target category and one or more other target reference images which are images not containing the region of the target category; a useful key region information storage unit that stores one or more useful key region information associated with two or more size information units; a target image storage unit that stores a target image which is the image to be processed; an image segmentation unit; a key region determination unit; a local processing unit; a reliability processing unit; a boundary determination unit; an image composition unit; and an output unit. The aforementioned image segmentation unit performs an image segmentation step in which it obtains two or more small image fragments from the target image, each from a different region. The key region determination unit performs a key region determination step of determining two or more key regions from the target image, The local processing unit performs a local processing step in which, for each of the two or more small image fragments acquired in the image segmentation step, it obtains the identification value of one or more key regions in the small image fragment using the one or more useful key region information, and obtains an identification representative value which is a representative value of the one or more identification values, The reliability processing unit performs a reliability processing step in which it checks whether the region corresponding to each of the one or more key regions determined in the key region determination step exists in each of the one or more target reference images, and a reliability processing step in which it checks whether the region corresponding to each of the one or more key regions determined in the key region determination step exists in each of the one or more other reference images, and uses the results of the first and second checks to obtain the reliability of each of the one or more key regions, The boundary determination step involves the boundary determination unit detecting a small image fragment or a key region within a small image fragment whose identification representative value corresponds to a value less than or equal to a first threshold, and whose confidence level of one or more key regions within the small image fragment is large enough to satisfy the confidence condition, and determining the boundary line corresponding to the small image fragment or the key region. The image composing unit comprises an image composing step in which the boundary line determined in the boundary line determination step is clearly indicated for the target image, An image processing method comprising: an output unit and an output step of outputting the image configured in the image configuration step.

6. A computer capable of accessing a reference image storage unit that stores one or more target reference images which are images containing the region of the target category, and one or more other target reference images which are images that do not contain the region of the target category, a useful key region information storage unit that stores one or more useful key region information associated with two or more size information units, and a target image storage unit that stores the target image which is the image to be processed, An image segmentation unit that acquires two or more small image fragments from different regions from the aforementioned target image, A key region determination unit that determines two or more key regions from the aforementioned target image, A local processing unit obtains, for each of the two or more small image fragments acquired by the image segmentation unit, the identification value of one or more key regions within the small image fragment using the one or more useful key region information, and a representative identification value which is a representative value of the one or more identification values. A reliability processing unit performs a first inspection, which is an inspection to determine whether the region corresponding to each of the one or more key regions determined by the key region determination unit exists in each of the one or more target reference images, and a second inspection, which is an inspection to determine whether the region corresponding to each of the one or more key regions determined by the key region determination unit exists in each of the one or more other reference images, and uses the results of the first inspection and the second inspection to obtain the reliability of each of the one or more key regions, A boundary determination unit detects small image fragments or key regions within small image fragments where the aforementioned identification representative value corresponds to a value less than or equal to a first threshold a, and the reliability of one or more key regions of the small image fragment is large enough to satisfy the reliability condition, and determines the boundary line corresponding to the small image fragment or the key region. An image component comprising the target image and an image in which the boundary line determined by the boundary line determination unit is clearly indicated, A program for causing the image component to function as an output unit that outputs the image it has constructed.

Images