Detection system, detection device, detection method, and program

Abstract

To stabilize performance of detecting an image of a minute detection target in a captured image.SOLUTION: A detection system includes a candidate generation section, a pre-processing section, and a detection section. The candidate generation section generates candidates for a detection target image, in which a detection target is captured, from a captured image for detection that includes an image of an object having the detection target and a noise image. The pre-processing section determines whether each small region image obtained by dividing the captured image for detection into small regions has features of subjects different from an object having the detection target on the basis of an image feature amount of the small region image to obtain an exclusion region in the captured image on the basis of a determination result. The detection section detects the detection target image from among the candidates for the detection target image that are not included in the exclusion region by predetermined detection processing.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a detection system, a detection device, a detection method, and a program. [Background technology]

[0002] There are techniques for detecting an image of a minute detection target in an image (captured image) captured at a predetermined magnification, and image processing algorithms for particle analysis and the like are known.

[0003] However, image processing algorithms such as particle analysis have difficulty distinguishing between minute granular detection targets and noise in captured images. This can result in both minute detection targets and large amounts of noise being extracted from the captured image, reducing the accuracy of particle analysis. Furthermore, depending on the fluorescent dye, experimental conditions, or detection target, the image quality of the captured image can be degraded, making it difficult to distinguish between detection targets and noise in the captured image.

[0004] In the image processing algorithm for classifying cell types and tissues, classification is performed using a magnification and observation equipment that allows an observer to classify cell types, etc., and minute objects in captured images containing a large amount of noise are not treated as detection targets. Furthermore, because the shape of a minute detection target is unclear in a captured image, it may not be possible to distinguish between the image of the detection target and a noise image in the captured image simply by focusing on the characteristics of the shape.

[0005] Furthermore, especially when tumors or tissues are spatially distributed, there may be differences in density depending on the region. In high-density regions, the visibility of the minute shapes of the target may be reduced by subjects other than the target, such as tumors or tissues, which may impede detection and make the detection performance unstable. Summary of the Invention [Problem to be solved by the invention]

[0006] An object of the present invention is to provide a detection system, a detection device, a detection method, and a program that can stabilize the performance of detecting an image of a minute detection target in a captured image. [Means for solving the problem]

[0007] One aspect of the present invention is a detection system comprising: a candidate extraction unit that extracts candidate detection target images in which the detection target is captured from a captured image for detection, which includes an image of an object having the detection target and a noise image; a preprocessing unit that divides the captured image for detection into small regions and, for each small region image, determines whether the small region image has features of a subject different from the object having the detection target based on image features of the small region image, and obtains an exclusion region in the captured image based on the determination result; and a detection unit that detects the detection target image from among the candidate detection target images that are not included in the exclusion region by a predetermined detection process. [Effects of the Invention]

[0008] According to the present invention, it is possible to stabilize the performance of detecting an image of a minute detection target in a captured image. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a detection system in a first embodiment. [Figure 2] FIG. 4 is a diagram illustrating an example of first pre-processing for a captured image in the first embodiment. [Figure 3] FIG. 10 is a diagram illustrating an example of second pre-processing for a captured image in the first embodiment. [Figure 4] 5A to 5C are diagrams illustrating an example of pre-processing and detection processing for a captured image for detection in the first embodiment. [Figure 5] 4 is a flowchart showing an example of the operation of the detection device in the first embodiment. [Figure 6] FIG. 10 is a diagram illustrating an example of the configuration of a detection system in a second embodiment. [Figure 7]FIG. 10 is a diagram illustrating an example of the operation of a second pre-processing of the detection device in the second embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of the configuration of a detection system in a third embodiment. [Figure 9] FIG. 11 is a diagram showing an example of a captured image for learning in the third embodiment. [Figure 10] FIG. 11 is a diagram illustrating an example of training data in the third embodiment. [Figure 11] 11 is a flowchart showing an example of the operation of the learning device in the learning stage in the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail with reference to

[0011] (First embodiment) FIG. 1 is a diagram illustrating an example configuration of a detection system 1 according to a first embodiment. The detection system 1 is a system that detects an image of a minute detection target in a captured image. Here, the detection system 1 extracts candidate images of the detection target from the captured image using an image processing algorithm that focuses on the structure (shape, color) surrounding the minute detection target. The detection system 1 extracts areas in the captured image where detection of the minute detection target is hindered, for example, areas with a high density of subjects other than the detection target, such as tumors or tissue, and excludes the extracted areas from the area in which the minute detection target is to be detected. The detection system 1 detects the detection target image from among the candidate images of the detection target extracted from areas in the captured image that are different from the excluded areas, using a predetermined detection process such as a machine learning model.

[0012] The captured image is, for example, an image of a biological sample captured at a predetermined magnification using an optical microscope. The captured image may be a color image or a monochrome image (gray image). The color image may be an RGB (Red Green Blue) image or an image in another color space (for example, Lab color space).

[0013] Detection targets include, for example, organelles, cytoskeleton, and proteins. Organelles include, for example, lysosomes, autophagosomes, cell tissues, vesicles, and mitochondria. Cytoskeleton includes, for example, growth cones, dendritic spines of neurons, actin filaments, and microtubules. Proteins (protein aggregates) include synapsin, synaptophysin, vesicular glutamate transporter (vGLUT), vesicular GABA transporter (vGAT), PSD-95 (postsynaptic density-95), drebrin, Homer, cell nuclei, micronuclei, stress granules, prions, beta-amyloid, and alpha-synuclein, which are accumulated in neuronal synapses.

[0014] The size of the minute detection target is, for example, about 0.01 to several μm 3. Note that the size of the minute detection target is not limited to a specific size as long as it is minute (equal to or less than a predetermined threshold) relative to the captured image.

[0015] The detection system 1 includes a communication line 2, an image transmission device 3, and a detection device 5. The communication line 2 may be a wired communication line or a wireless communication line. Furthermore, the functional units of each device in the detection system 1 may be distributed over a network such as the Internet using cloud technology. Furthermore, each functional unit of each device in the detection system 1 may be handled by a single information processing device.

[0016] The image transmitting device 3 is, for example, a server device. The image transmitting device 3 stores a plurality of captured images in advance. The captured images include, for example, images of cells captured by an optical microscope, a fluorescent microscope, or the like. The image transmitting device 3 transmits the captured images for detection to the detecting device 5 in response to a request from the detecting device 5.

[0017] The detection device 5 detects a detection target image in a captured image for detection by a predetermined detection process using a pre-constructed machine learning model, etc. The detection device 5 includes an operation unit 51, a communication unit 52, a learning model storage device 53, a storage device 54, a memory 55, a detection execution unit 56, and a display unit 57. The storage device 54 may also serve as the learning model storage device 53.

[0018] The detection device 5 is realized as software by a processor such as a CPU (Central Processing Unit) executing a program stored in a storage device 54 having a non-volatile recording medium (non-transitory recording medium) and a memory 55. The program may be recorded on a computer-readable recording medium. The detection device 5 may also be realized using hardware including an electronic circuit (electronic circuit or circuitry) using, for example, an LSI (Large Scale Integrated circuit), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array).

[0019] The operation unit 51 is an operation device such as a mouse, a keyboard, or a touch panel. The operation unit 51 accepts operations by a user. The operations by the user include, for example, selecting a captured image from which a detection target image is to be detected, selecting which machine learning model to use when multiple machine learning models are registered, and inputting an instruction to the detection device 5 as to whether or not to terminate the execution of inference.

[0020] The communication unit 52 communicates with other devices via the communication line 2. The communication unit 52 transmits a request for a captured image for detection to the image transmission device 3, and receives the captured image transmitted by the image transmission device 3 in response to the request. The learning model storage device 53 stores one or more types of machine learning models. The storage device 54 stores in advance a computer program for inference processing using the machine learning models. The storage device 54 further stores the machine learning models acquired from the learning model storage device 53. The storage device 54 may store coordinates of candidates for each detection target image in the captured image. A computer program for machine learning such as deep learning is loaded from the storage device 54 into the memory 55. The memory 55 may include an auxiliary storage device such as a GPU (Graphics Processing Unit).

[0021] The detection execution unit 56 uses a machine learning model to detect a detection target image in the captured image for detection obtained from the image transmission device 3. Such a machine learning model uses, for example, image feature amounts obtained from pixel values ​​of candidates for the detection target image as input. The detection execution unit 56 includes a candidate generation unit 560, a pre-processing unit 561, a detection unit 564, and a post-processing unit 565.

[0022] The candidate generation unit 560 acquires the captured image for detection received by the communication unit 52 from the image transmission device 3. The candidate generation unit 560 generates candidates for detection target images to be subjected to detection processing using a machine learning model from the acquired captured image for detection. The captured image includes, for example, an image of an object (e.g., a cell) having the detection target (e.g., drebrin) and a noise image. The candidate for detection target image is an image of a predetermined shape and size that is cut out from the captured image for detection so as to include a portion of the candidate for detection target. The portion of the candidate for detection target is an image of granular shapes (spherical, convex) distributed in the captured image, or an image of a predetermined shape and size that includes an image of granular shapes distributed in the captured image. The shape of the candidate for detection target image is, for example, rectangular. The dimensions of the candidate for detection target image are arbitrary as long as they are smaller than the dimensions of the captured image (are very small compared to the captured image) and are equal to or larger than the dimensions of the detection target image.

[0023] The preprocessing unit 561 performs first and second preprocessing steps (described later) to narrow down candidates for detection target images to be subjected to the detection process before detecting the detection target image from candidates for detection target images using a machine learning model. The preprocessing unit 561 includes a first preprocessing unit 562 and a second preprocessing unit 563.

[0024] The first preprocessing unit 562 performs first preprocessing to narrow down candidates for the detection target image based on image feature amounts (e.g., brightness) based on the structure surrounding the detection target. That is, the first preprocessing unit 562 obtains one or both of an extraction area and an exclusion area in the captured image for detection based on the image feature amounts based on the structure surrounding the detection target. The extraction area is an area that is likely to include an image of the structure surrounding the detection target. The exclusion area is an area that is unlikely to include the structure surrounding the detection target. The first preprocessing unit 562 extracts, from the candidates for the detection target image generated by the candidate generation unit 560, candidates obtained from the extraction area as candidates for the detection target image to be used in the detection process. Furthermore, the first preprocessing unit 562 excludes, from the candidates for the detection target image extracted by the candidate generation unit 560, candidates obtained from the exclusion area from the candidates for the detection target image to be used in the detection process.

[0025] The first pre-processing unit 562 may determine an extraction region or an exclusion region from the captured image by performing a Hough transform on the captured image. Alternatively, the first pre-processing unit 562 may determine an extraction region or an exclusion region from the captured image based on the convex curvature (shape index value) of the image distributed in the captured image. The first pre-processing unit 562 may determine an extraction region or an exclusion region from the captured image based on a blob filter using a Hessian, histograms of oriented gradients using first-order differences, or a difference of Gaussian filter using a Gaussian.

[0026] For example, if the detection target is a microstructure surrounding a synapse, the first preprocessing unit 562 extracts only the image surrounding the dendrite from the candidate detection target image generated by the candidate generation unit 560 by setting only the image surrounding the dendrite as the extraction region. If the fluorescent staining of the cell body including the dendrite is different from the fluorescent staining of the synapse, the first preprocessing unit 562 may perform a distance transformation process on the captured image to detect peaks in the convex cell body. Furthermore, the first preprocessing unit 562 may identify coordinates near the cell nucleus by fluorescently staining the cell nucleus. The first preprocessing unit 562 may apply a circular mask to the region including the peak position based on distance information from the peak position. That is, the first preprocessing unit 562 excludes candidates obtained from the exclusion region represented as a circular mask from the candidate detection target image generated by the candidate generation unit 560. In this way, the first preprocessing unit 562 may use a circular mask in the captured image to remove cell body images including cell nucleus images from among the candidates for detection target images generated by the candidate generating unit 560.

[0027] In addition, after the first preprocessing unit 562 determines the extraction area or the exclusion area, the candidate generation unit 560 may generate candidates for the image to be detected from the extraction area in the captured image for detection, or from the area excluding the exclusion area in the captured image for detection.

[0028] The second preprocessing unit 563 performs a second preprocessing to narrow down candidates for the detection target image by eliminating regions in the captured image for detection in which structures similar to the detection target appear or in which the structure of the detection target is difficult to recognize as regions unsuitable for detection in the detection target image. Regions in which the structure of the detection target is difficult to recognize are, for example, regions with a high density of subjects other than the detection target, such as tumors or tissue. A specific example of such a region is a region where dendrites overlap.

[0029] The detection unit 564 executes a detection process on the captured image for detection using a machine learning model. The detection unit 564 inputs each candidate of the detection target image narrowed down by the first preprocessing unit 562 and the second preprocessing unit 563 to the machine learning model. The input may be performed by batch processing. The detection unit 564 inputs the candidate of the detection target image to the machine learning model, thereby obtaining output (probability distribution (probability map), probability score) of the machine learning model. As a result, the detection unit 564 determines whether the input candidate of the detection target image is the detection target image based on the output of the machine learning model.

[0030] The detection unit 564 may derive statistics regarding the size of the candidate detection target image and statistics regarding information surrounding the candidate detection target image in the captured image. The detection unit 564 may determine whether the candidate detection target image is the detection target image based on the statistics by using a support vector machine on the preprocessed captured image. The detection unit 564 may determine whether the candidate detection target image is the detection target image based on the results of a clustering method (e.g., K-Means) based on the statistics.

[0031] The post-processing unit 565 performs predetermined post-processing based on the detection result by the detection unit 564. As the predetermined post-processing, the post-processing unit 565 may generate a captured image or the like in which the position of the detection target is emphasized by at least one of marking and coloring. As the predetermined post-processing, the post-processing unit 565 may generate an image representing a predetermined index. Furthermore, the post-processing unit 565 may count the number of detection target images detected by the detection unit 564 as one of the indices. The post-processing unit 565 may derive the maturity of the cells as one of the indices based on the number of detection target images detected by the detection unit 564.

[0032] The display unit 57 is a display device such as a liquid crystal display, etc. The display unit 57 displays the image generated by the post-processing unit 565.

[0033] Next, the first preprocessing performed by the first preprocessing unit 562 of the detection device 5 will be described. FIG. 2 is a diagram showing an example of the first preprocessing performed on a captured image in the first embodiment. The captured image 100 is a captured image (original image) for detection. Note that here, an image portion of MAP2 (Microtubule-associated protein 2) in the original image is used as the captured image 100. The captured image 100 includes images of multiple cells stained with one or more types of dyes or immunostaining methods. The shape of each cell body resembles a granular shape (spherical, convex). The imaging magnification (angle of view) of the captured image 100 is, for example, that of an optical microscope, and is arbitrary.

[0034] The first pre-processing unit 562 performs binarization processing on the captured image 100 (gray image) acquired from the image transmitting device 3. As a result, the first pre-processing unit 562 generates a captured image 100a. If the captured image acquired from the image transmitting device 3 is a color image, the first pre-processing unit 562 converts the captured image 100 into a one-channel gray image by extracting a specific channel from the acquired captured image or by combining multiple channels, and then performs binarization processing. The first pre-processing unit 562 performs dust removal processing on the captured image 100a. As a result, the first pre-processing unit 562 generates a captured image 100b.

[0035] The first pre-processing unit 562 performs distance transformation and peak detection on the captured image 100b. As a result, the first pre-processing unit 562 generates the captured image 100c. Because the shape of each cell is similar to a granular shape (spherical, convex), the position of the peak detected by the peak detection process is near the center (near the cell nucleus) of each cell body image in the captured image 100c. The first pre-processing unit 562 generates a mask image 101 based on the position of the detected peak and the result of the distance transformation. The mask image 101 includes a circular mask image centered near the center of each cell body image. The size of the circular mask image is determined according to the result of the distance transformation.

[0036] The first preprocessing unit 562 performs mask processing using the mask image 101 on the captured image 100b. As a result, the first preprocessing unit 562 generates a captured image 100d. In the captured image 100d, images of each cell body are removed by the mask processing, leaving linear images such as axon images and dendrite images having synapse images. Drebrin, an example of a detection target, accumulates at the synapses of neurons. Therefore, for example, an amorphous image not located near a synapse image in the captured image 100 is likely to be a noise image. By obtaining an image (distribution) of synapses, which are structures surrounding drebrin, in the captured image 100d, the first preprocessing unit 562 makes it easier to distinguish between drebrin images and noise images in the captured image 100 (original image). The first preprocessing unit 562 narrows down the extraction region from which candidates for the detection target image are obtained from the captured image 100 to the region surrounding the linear image in the captured image 100d. That is, the first preprocessing unit 562 extracts, from among the candidates for the detection target image generated by the candidate generating unit 560, candidates obtained from the area surrounding the linear image in the captured image 100d as targets for the detection process.

[0037] Next, the second preprocessing executed by the second preprocessing unit 563 of the detection device 5 will be described. FIG. 3 is a diagram showing an example of the second preprocessing performed on a captured image in the first embodiment. The second preprocessing unit 563 evaluates the degree of an obstruction factor in the detection of a detection target image for each small region of the captured image to obtain an evaluation value, and if the evaluation value satisfies a predetermined criterion, sets the region as an exclusion region that excludes the region from the detection target. Examples of obstruction factors include subjects that are different from the detection target, such as tumors or tissue, and in particular, subjects that have a shape or brightness value similar to that of the detection target. The presence of a subject that has a shape or brightness value similar to that of the detection target may lead to missed or false detection of the detection target, which may result in an error in the number of detection target images counted by the postprocessing unit 565, for example.

[0038] One example of evaluating the degree of an obstruction factor is a method of dividing the captured image 151 (gray image) that captures the obstruction factor into small regions and obtaining the statistical value of the brightness value for each small region as an evaluation value. It is desirable to set the size of the small region to be the same as or larger than the size of the candidate image to be detected. The shape of the small region is not limited to rectangular, and may be other shapes such as circular. Examples of statistical values ​​include the average, median, and mode of brightness values ​​obtained from the pixel values ​​of each pixel included in the small region.

[0039] Another example of evaluating the degree of inhibiting factors is a method of detecting inhibiting factors such as cell tumors and tissues from the captured image 151 (gray image), and calculating the number of detected inhibiting factors contained in each small region or the area that they occupy in the small region, and using this as an evaluation value.

[0040] An example of a predetermined criterion is a method in which, if the evaluation value for each small region satisfies a predetermined condition, the region is determined to be unsuitable for detection of the detection target image. An example of a predetermined condition is a method in which, if the average brightness value of each pixel included in a small region exceeds a threshold, the small region is designated as an exclusion region in which detection is invalid. The threshold may be a predetermined one or may be determined dynamically. An example of dynamically determining the threshold is a method in which a threshold is calculated by discriminant analysis from a histogram of the median brightness values ​​calculated for each small region in the entire captured image, and small regions exceeding the threshold are designated as invalid (exclusion regions). The threshold may also be determined based on image quality, fluorescence intensity, etc., or a threshold input via the operation unit 51 may be used.

[0041] The second preprocessing unit 563 generates a mask image 154 for masking small regions determined to be exclusion regions based on the above criteria. The second preprocessing unit 563 may create the small regions adjacent to each other without overlapping, or may create the small regions with overlap in at least one of the horizontal and vertical directions. If the small regions are created with overlapping, it is desirable that the regions to be masked in the mask image also have overlapping portions.

[0042] The second preprocessing is used in combination with the first preprocessing described in FIG. 2, but the first preprocessing does not have to be performed. In this case, a small area obtained from the area excluding the exclusion area of ​​the second preprocessing may be used as a candidate for the detection target image. When the second preprocessing is used in combination with the first preprocessing, the second preprocessing unit 563 synthesizes a mask image by taking the logical sum of the mask image 154 created by the second preprocessing and the mask image 155 created by the first preprocessing. The second preprocessing unit 563 masks the image obtained by binarizing the captured image 151 using the synthesized mask image. An example of this masking process will be described below.

[0043] Captured image 152 is the result of masking the binarized image of captured image 151 with mask image 154 created by the second preprocessing. It can be seen that the upper part and lower right part where dendrites are densely packed have been removed from captured image 152 by this processing. Captured image 153 is the result of further masking captured image 152 with mask image 155 created by the first preprocessing. This corresponds to masking using a combined mask image. It can be seen that the upper part where cell bodies are present has been further removed from captured image 153 by this processing.

[0044] The second preprocessing unit 563 narrows down the candidates for the detection target image to be determined by excluding candidates for the detection target image obtained from the same region in the captured image for detection as the small region determined as the exclusion region from the candidates for the captured image extracted by the first preprocessing unit 562 through the first preprocessing. Note that if the shape and size of the candidate for the detection target image generated by the candidate generation unit 560 from the captured image match the shape and size of the small region used in the second processing by the second preprocessing unit 563, the candidate for the detection target image generated by the candidate generation unit 560 can be used as the image of the small region in the second preprocessing performed by the second preprocessing unit 563.

[0045] In this way, the pre-processing unit 561 excludes from the captured image 151 an area where an object having a shape or brightness value similar to that of the detection target exists, thereby suppressing missed detection or false detection of the detection target. This can reduce the risk of an error in the number of detection target images counted by the post-processing unit 565, for example.

[0046] Next, the pre-processing, detection processing (inference processing), and post-processing in the detection device 5 will be described. FIG. 4 is a diagram showing an example of preprocessing and detection processing for a captured image for detection in the first embodiment. A captured image 200 is a captured image for detection (original image). In the preprocessing stage, a candidate generation unit 560 generates candidates for a detection target image from the captured image 200 acquired from the image transmission device 3, and a preprocessing unit 561 performs first preprocessing and second preprocessing on the candidates for the detection target image generated by the candidate generation unit 560 to narrow down the candidates. In this way, the preprocessing unit 561 generates candidates for a detection target image (Drebrin image) in the captured image 200a. The preprocessing unit 561 associates coordinates in the captured image 200a with candidates for the detection target image in the captured image 200a.

[0047] In the captured image 200a shown in Fig. 4, a plurality of circles are drawn for the purpose of conveniently indicating the positions of candidate images of the detection target (drebrin) at the synapse of a nerve cell. The position of each circle indicates the position of the image of each candidate detection target. In the captured image 200a, in order to ensure ease of viewing, only the detection target image 201 and the noise image 202, which are representative of the candidate images of the detection target, are assigned symbols.

[0048] The candidate detection target image 210 is one of the candidate detection target images input to the machine learning model in the inference stage. Here, the machine learning model inputs the candidate detection target image and outputs a "Drebrin class" (detection target class) or a "noise class" (background class). The candidate detection target image 210 includes a detection target image 201 (a grainy image). The candidate detection target image 211 is another candidate detection target image input to the machine learning model in the inference stage. The candidate detection target image 211 includes a noise image 202 (an irregular image). In this way, after the first preprocessing and the second preprocessing by the preprocessing unit 561 have been performed, at a stage before the detection unit 564 performs the detection process using the machine learning model (at the inference stage before the detection process is performed), the candidate detection target images may include the noise image 202.

[0049] The detection unit 564 executes a detection process (inference process) on the captured image 200a using a machine learning model. Here, the detection unit 564 inputs each candidate of the detection target image into the machine learning model. In FIG. 4, the detection unit 564 inputs the candidate 210 into the machine learning model to obtain an output "Drebrin class" of the machine learning model (trained model). As a result, the detection unit 564 determines that the detection target image 201 of the candidate 210 is a detection target image (Drebrin image). Furthermore, the detection unit 564 inputs the candidate 211 into the machine learning model to obtain an output "noise class" of the machine learning model (trained model). As a result, the detection unit 564 determines that the noise image 202 of the candidate 211 is not a detection target image. In other words, the detection unit 564 determines that the noise image 202 of the candidate 211 is a noise image.

[0050] The post-processing unit 565 performs predetermined post-processing on each candidate determined to be a detection target image. For example, the post-processing unit 565 generates a captured image 200b by drawing a circle at each position of the detection target image 201 in the captured image 200. The post-processing unit 565 may count the number of detection target images 201 in the captured image 200b as one of the indices. The post-processing unit 565 may derive the maturity of cells as one of the indices based on the number of detection target images 201 (drebrin images) in the captured image 200.

[0051] The second preprocessing by the second preprocessing unit 563 may be performed after detection by the detection unit 564. That is, the detection unit 564 inputs each detection target candidate obtained by performing the first preprocessing by the first preprocessing unit 562 into a machine learning model and obtains a determination result of "Drebrin class" or "Noise class." The second preprocessing unit 563 performs the second preprocessing on each detection target candidate determined to be in the "Drebrin class," excludes candidates obtained from areas of the captured image 200a determined to be exclusion areas from the "Drebrin class," and obtains the remaining "Drebrin class" candidates as the detection target image.

[0052] Next, an example of the operation of the detection system 1 will be described. FIG. 5 is a flowchart showing an example of the operation of the inference stage of the detection system 1 in the first embodiment. The detection unit 564 of the detection device 5 acquires a machine learning model (trained model) from the learning model storage device 53 (step S101). The candidate generation unit 560 acquires a captured image 200 for detection from the image transmission device 3 (step S102). The candidate generation unit 560 generates candidates for the detection target image from the captured image 200. The preprocessing unit 561 extracts candidates for the detection target image 210, 211, etc. from the candidates for the detection target image generated by the candidate generation unit 560 based on the structure around the detection target (distribution of synapses) and the degree of factors inhibiting detection of the detection target image for each small region (step S103). The preprocessing unit 561 records the coordinates of the candidates for the detection target image 210, 211 in the storage device 54 (step S104).

[0053] The detection unit 564 inputs each candidate for the detection target image into the machine learning model (step S105). Based on the output "Drebrin class" of the machine learning model, the detection unit 564 selects the candidate 210, etc., including the detection target image 201, as an image including the detection target image 201, from among the candidates 210, 211, etc. for each detection target image (step S106).

[0054] The post-processing unit 565 derives a predetermined index (e.g., cellular maturity) based on the detection target images 201 and the like included in the image selected in step S106. For example, the post-processing unit 565 counts the detection target structure and proteins (drebrin in this example) accumulated in synapses based on the number of detection target images 201. The post-processing unit 565 derives the unit length and density per unit area of ​​dendrites using information about the dendrites. The post-processing unit 565 derives the cellular maturity based on the density per unit area (step S107). The post-processing unit 565 causes the display unit 57 to display the captured image for detection 200, the detection target images 201 and the like included in the candidate 210 selected in the captured image for detection 200, and the derived index (step S108).

[0055] The detection unit 564 determines whether to end the detection process based on, for example, an operation received by the operation unit 41. If the detection process is to be continued (step S109: NO), the detection unit 564 returns the process to step S102. If the detection process is to be ended (step S109: YES), the detection unit 564 ends the detection process of FIG.

[0056] As described above, the detection system 1 includes the detection device 5. The detection device 5 includes a candidate generation unit 560, a preprocessing unit 561, and a detection unit 564. The detection unit 564 acquires a machine learning model from the learning model storage device 53. The candidate generation unit 560 acquires candidates for detection target images in which the detection target is captured from the captured image 200 for detection, which includes an image of an object having the detection target and a noise image. The preprocessing unit 561 extracts candidates for detection target images 210 and 211 in the captured image 200a from among the candidates for detection target images 210 and 211, for example, based on the structure surrounding the detection target in the captured image 200a (e.g., the distribution of synapses) and the degree of an inhibitory factor for the detection target. The detection unit 564 uses the machine learning model to detect, for example, the detection target image 201 of the candidate 210.

[0057] This makes it possible to improve the performance of detecting an image of a minute detection target in a captured image.

[0058] (Second embodiment) The second embodiment illustrates an example of learning when the second preprocessing in the first embodiment is performed by machine learning. That is, in the second embodiment, the degree of inhibitory factors such as tumors and tissues is determined by machine learning. The second embodiment will be described, focusing on the differences from the first embodiment. The processing flow diagram in the inference stage of the second embodiment is the same as that of the first embodiment, so description thereof will be omitted.

[0059] FIG. 6 is a diagram showing an example of the configuration of a detection system 1a according to the second embodiment. In the figure, the same components as those in the detection system 1 according to the first embodiment shown in FIG. 1 are assigned the same reference numerals, and their description will be omitted. The detection system 1a shown in FIG. 6 differs from the detection system 1 shown in FIG. 1 in that it includes a detection device 5a instead of the detection device 5. The detection device 5a differs from the detection device 5 shown in FIG. 1 in that it further includes a learning unit 58. In addition, the learning model storage device 53 stores a trained preprocessing model, which is a trained machine learning model that performs second preprocessing.

[0060] The following describes the flow of learning the trained preprocessing model used to determine the exclusion region in the second preprocessing by the learning unit 58. Training images including subjects other than the detection target, such as tumors or tissues that fall into the exclusion region, are associated with correct labels indicating whether or not the subject falls into the exclusion region and stored as training data in the storage device 54 in advance.

[0061] In the learning stage of the trained preprocessing model, the learning unit 58 performs learning of a machine learning model based on the training image. The machine learning model includes, for example, a neural network such as a convolutional neural network (CNN). The convolutional neural network has a convolutional layer and a fully connected layer. In addition to a neural network, any machine learning model that performs supervised learning for classification, such as a support vector machine or a random forest, may be used. There may be one type of machine learning model or multiple types. Furthermore, the machine learning model in the embodiment may be generated by fine-tuning a pre-trained machine learning model using training data.

[0062] The learning unit 58 performs learning of the machine learning model using the training images and correct labels stored in the storage device 54. The learning unit 58 adjusts the parameters of the machine learning model to reduce the difference between the output of the machine learning model and the correct label, for example, by using backpropagation of the output (probability distribution (probability map), probability score) of the machine learning model. Here, one type of detection target is associated with each probability distribution. When multiple machine learning models are used, the probability distributions for each detection target may be integrated (added). The learning unit 58 writes the learned machine learning model to the learning model storage device 53 as a trained preprocessing model.

[0063] The detection system 1a may have a configuration including a learning device having a learning unit 58 and the detection device 5 of the first embodiment, instead of the detection device 5a.

[0064] FIG. 7 is a process flow diagram in which the detection device 5a uses a trained pre-processing model that has previously trained the features of the exclusion region when setting the exclusion region in the second pre-processing.

[0065] The second preprocessing unit 563 acquires a trained preprocessing model used for determining an exclusion region from the training model storage device 53 (step S201). Next, the second preprocessing unit 563 acquires the captured image 100 acquired by the preprocessing unit 561 in step S102 of FIG. 5 (step S202) and divides the captured image 100 into small regions of a size suitable for input to the trained preprocessing model (step S203). The second preprocessing unit 563 selects an unselected small region from the divided small regions (step S204).

[0066] Next, the second preprocessing unit 563 inputs the selected small region into the trained preprocessing model for evaluation and determines whether it corresponds to an exclusion region (step S205). If the second preprocessing unit 563 determines that the selected small region is an exclusion region (YES in step S206), it masks the corresponding region in the mask image so that the region is not used to detect the detection target image in subsequent processing (step S207). On the other hand, if the second preprocessing unit 563 does not determine that the region is an exclusion target region (NO in step S206), it skips step S207 and proceeds to the next step S208.

[0067] The second pre-processing unit 563 repeats steps S204 to S207 for the entire captured image (step S208). That is, if there are unselected small regions (NO in step S208), the second pre-processing unit 563 repeats the processes from step S204, and if all small regions have been selected (YES in step S208), the second pre-processing unit 563 ends the process of FIG. 7. The second pre-processing unit 563 excludes, from the candidates 210 and 211 of detection target images extracted by the first pre-processing unit 562, the candidates 210 and 211 of detection target images obtained from regions in the captured image for detection 200a that correspond to mask regions in the mask image generated in the process of FIG. 7. The detection device 5a performs the processes from step S104 in FIG. 5 using the candidates 210 and 211 of detection target images that have not been excluded and remain.

[0068] The second preprocessing unit 563 may create adjacent small regions without overlapping, or may create small regions with overlapping in at least one of the horizontal and vertical directions. When repeating step S207, the second preprocessing unit 563 may pre-extract and omit regions where it is clear that no detection target image exists, such as empty background regions.

[0069] By using a machine learning model to determine whether or not an area is excluded, it is expected that more accurate determinations can be made than with rule-based methods.

[0070] (Third embodiment) This embodiment includes a learning device that learns a machine learning model used by the detection device 5 of the first embodiment or the detection device 5a of the second embodiment to determine the detection target image. The third embodiment will be described focusing on the differences from the first embodiment, but the differences between the third embodiment and the first embodiment may also be applied to the detection system of the second embodiment.

[0071] Fig. 8 is a diagram showing an example of the configuration of a detection system 1b in the third embodiment. In the figure, the same components as those in the detection system 1 according to the first embodiment shown in Fig. 1 are given the same reference numerals, and their description will be omitted. The detection system 1b shown in Fig. 8 differs from the detection system 1 shown in Fig. 1 in that it further includes a learning device 4 and includes a detection device 5b instead of the detection device 5. The detection device 5b differs from the detection device 5 shown in Fig. 1 in that it does not include a learning model storage device 53.

[0072] The learning device 4 and the detection device 5b may be integrated or separate. That is, a single information processing device may function as both the learning device 4 and the detection device 5b depending on a computer program installed on the single information processing device. Furthermore, the functional units of the devices in the detection system 1b may be distributed over a network such as the Internet using cloud technology.

[0073] In the learning stage (learning phase), the learning device 4 executes learning of a machine learning model used by the detection unit 564 based on the training image. The machine learning model includes, for example, a neural network such as a convolutional neural network (CNN). The convolutional neural network has a convolutional layer and a fully connected layer. There may be one type of machine learning model or multiple types of machine learning models. Furthermore, the machine learning model in this embodiment may be generated by fine-tuning a previously trained machine learning model using training data.

[0074] The learning device 4 includes an operation unit 41, a communication unit 42, a storage device 43, a memory 44, a learning execution unit 45, and a display unit 46.

[0075] The learning device 4 is realized as software by a processor such as a CPU executing a program stored in a storage device 43 having a non-volatile recording medium (non-transitory recording medium) and a memory 44. The program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and non-transitory recording media such as hard disks or solid-state drives (SSDs) built into computer systems. The learning device 4 may also be realized using hardware including electronic circuits using, for example, LSIs, ASICs, PLDs, or FPGAs.

[0076] The operation unit 41 is an operation device such as a mouse, keyboard, or touch panel. The operation unit 41 accepts operations by the user. An example of the user operation is an operation to input an instruction to the learning device 4 as to whether or not to terminate the execution of learning. An example of the user operation may be an operation to input a correct answer label of training data used for machine learning into the learning device 4. This allows machine learning to be performed according to variations in the input training data, making it possible to generate a machine learning model that matches the user's sensibilities.

[0077] The communication unit 42 communicates with the image transmission device 3. The communication unit 42 also communicates with the communication unit 52. The storage device 43 stores in advance a computer program for a learning process using a machine learning model and training data. The training data is a combination of training images (explanatory variables) and correct labels (objective variables). A computer program for machine learning such as deep learning is loaded from the storage device 43 into the memory 44. The memory 44 may include an auxiliary storage device such as a graphic processing unit (GPU).

[0078] The learning execution unit 45 uses the training images and the correct labels to execute learning of the machine learning model. The learning execution unit 45 adjusts the parameters of the machine learning model, for example, by using backpropagation algorithm on the output (probability distribution (probability map), probability score) of the machine learning model. Here, one type of detection target is associated with each probability distribution. When multiple machine learning models are used, the probability distributions for each detection target may be integrated (added).

[0079] The learning execution unit 45 includes a learning image generation unit 450, a pre-processing unit 451, a learning unit 452, and a post-processing unit 453. The learning image generation unit 450 performs the same processing as the candidate generation unit 560 on captured images for learning acquired from the image transmission device 3 to generate learning images. For example, the learning image generation unit 450 sets granular (spherical, convex) images distributed in the captured images as candidates for the detection target. The learning image generation unit 450 sets an image of the candidate for the detection target, or an image of a predetermined shape and size cut out so as to include a portion of the candidate for the detection target, as the learning image.

[0080] The preprocessing unit 451 performs first preprocessing similar to that performed by the first preprocessing unit 562 on the training images acquired by the training image generation unit 450. That is, the preprocessing unit 451 obtains an extraction region or an exclusion region in the captured images for training based on image feature amounts based on the surrounding structure of the detection target. The preprocessing unit 451 extracts, from the training images generated by the training image generation unit 450, images obtained from the extraction region as training images to be used in the training process. Alternatively, the preprocessing unit 451 excludes, from the training images generated by the training image generation unit 450, images obtained from the exclusion region from the training images to be used in the training process. In this way, the preprocessing unit 451 extracts training images including an image of the detection target (e.g., drebrin) based on image feature amounts (e.g., brightness) based on the surrounding structure of the detection target.

[0081] For example, the pre-processing unit 451 may perform a Hough transform on the captured image, and extract from the captured image an extracted region that includes the image of the detection target, or an excluded region that is unlikely to include the image of the detection target, based on a blob filter using a Hessian, an oriented gradient histogram using a first-order difference, or a Gaussian difference filter using a Gaussian, based on the convex curvature of the images distributed in the captured image.

[0082] In such an extraction algorithm using pixel values, false positives (FP) may occur in the candidate determination results depending on the sensitivity to pixel values. Note that if the captured images contain few noise images, a segmentation network may be used for the captured images if it is acceptable for the user to take the extra effort of assigning correct labels to the training images.

[0083] When many false positives occur in the candidate determination results, in order to increase the specificity of the detection target in the candidate determination, the preprocessing unit 451 may extract training images including an image of the detection target from the captured images based on the biological structure of the detection target. For example, when a microstructure existing around a synapse is determined as the detection target, the preprocessing unit 451 may select only an image around a dendrite as a candidate for the detection target image.

[0084] When the fluorescent staining of the cell body including the dendrites is different from the fluorescent staining of the synapse, the preprocessing unit 451 may perform a distance transformation process on the captured image to detect peaks in the convex cell body. Furthermore, by fluorescently staining the cell nucleus, the preprocessing unit 451 may identify coordinates near the cell nucleus. The preprocessing unit 451 may apply a circular mask (exclusion region) to a region including the position of the peak based on distance information from the position of the peak. That is, the preprocessing unit 451 may use the circular mask on the captured image to remove cell body images including cell nucleus images from the training images generated by the training image generating unit 450.

[0085] The preprocessing unit 451 creates training data by assigning a correct answer label to each candidate for a detection target image in each training image based on a user operation. The preprocessing unit 451 may display an image in which the candidate for a detection target image is highlighted on the display unit 46 at a single or multiple magnifications. This eliminates the need for the user to perform a complex annotation method using a segmentation network. Note that the user does not need to forcefully assign a correct answer label to a candidate for which it is difficult for even the user to determine whether it is a detection target image. The user may assign a correct answer label to such candidate for a detection target image, indicating that it will not be used in machine learning.

[0086] The learning unit 452 performs learning of the machine learning model based on the training images. The learning unit 452 inputs each training image to the machine learning model. The input may be performed by batch processing. The learning unit 452 adjusts the parameters of the machine learning model so as to reduce the error between the output (class) of the machine learning model and the correct label for each training image.

[0087] The post-processing unit 453 executes predetermined post-processing based on the learning result by the learning unit 452. As the predetermined post-processing, the post-processing unit 453 may generate a captured image in which the position of the detection target is emphasized by at least one of marking and coloring. As the predetermined post-processing, the post-processing unit 453 may generate an image showing a predetermined indicator.

[0088] The display unit 46 is a display device such as a liquid crystal display, etc. The display unit 46 displays the image generated by the post-processing unit 453.

[0089] Next, the learning process and post-processing in the learning device 4 will be described. FIG. 9 is a diagram showing an example of a captured image 100 for training in the third embodiment. Each training image in the teacher data is a partial image (each small region, each small image) defined in the captured image 100 for training. The shape and dimensions of the training image are the same as those of the detection target image candidate cut out from the captured image for detection by the candidate generation unit 560 of the detection device 5b. The training image 110 includes a detection target image 111 (Drebrin image) at a neuronal synapse. The training image 120 includes a detection target image 121 (Drebrin image) at a neuronal synapse. The training image 130 includes a noise image 131 at a position other than a neuronal synapse.

[0090] 9, in order to ensure ease of viewing, only the training images 110, 120, and 130, which are representative of the training images, are assigned with reference numerals. More training images may be defined for the training image 100.

[0091] The user operates the operation unit 41 while viewing the learning captured image 100 displayed on the display unit 46. By operating the operation unit 41, the user defines a correct label for each learning image. Here, drebrin, which is an example of a detection target, accumulates in the synapses of nerve cells. Therefore, for example, a granular image located near an axon image and a dendrite image that include a synapse image is likely to be a drebrin image.

[0092] For example, since training image 110 includes a synapse image, the user associates the correct label "Drebrin class" (detection target class) with training image 110. Also, since training image 120 includes a synapse image, the user associates the correct label "Drebrin class" with training image 120.

[0093] For example, the training image 130 does not contain a synapse image, and the noise image 131 is an irregular image (non-grained image). Also, the contrast component of the contour of the noise image 131 is not high. Based on these facts, the user associates the correct label "noise class" or "background class" with the training image 120.

[0094] FIG. 10 is a diagram showing an example of training data in the third embodiment. The training data includes a combination of a training image and a correct label "detection target class." The training data may also include a combination of a training image and a correct label "noise class." Here, the correct label may be defined as two classes, a detection target class and a noise class (background class), or as a class for each grain (protrusion) size and a noise class.

[0095] Next, an example of the operation of the detection system 1b will be described. 11 is a flowchart showing an example of the operation of the learning device 4 in the learning stage in the third embodiment. The learning image generation unit 450 of the learning device 4 acquires the captured images 100 for learning from the image transmission device 3 (step S301). The learning image generation unit 450 generates learning images from the captured images 100. The pre-processing unit 451 extracts the learning images 110 and 120, etc. from the learning images generated by the learning image generation unit 450, based on whether they are within a predetermined distance from the structures (synapses) surrounding the detection target (drebrin). The pre-processing unit 451 may also extract the learning images 130, etc. from the captured images 100 for learning based on whether they are apart from the structures surrounding the detection target by a predetermined distance or more (step S302).

[0096] The preprocessing unit 451 associates the training images extracted in step S302 with the correct labels based on the operation received by the operation unit 41. For example, the preprocessing unit 451 associates the training images 110, which include a synapse image and a detection target image 111 (Drebrin image), with the correct label "Drebrin class." For example, the preprocessing unit 451 associates the training images 130, which do not include a synapse image, with the correct label "Noise class" (step S303).

[0097] The learning unit 452 inputs each training image into the machine learning model (step S304). The learning unit 452 adjusts the parameters of the machine learning model so as to reduce the error between the output (class) of the machine learning model for each training image and the correct label (step S305). The learning unit 452 determines whether to end the learning process based on, for example, an operation received by the operation unit 41 (step S306). If the learning process is to be continued (step S306: NO), the learning unit 452 returns the process to step S301. If the learning process is to be ended (step S306: YES), the learning unit 452 records the machine learning model (trained model) in the storage device 43 (step S307).

[0098] In the inference stage (inference phase), the detection device 5b of the detection system 1b performs pre-processing, detection processing, and post-processing similar to those of the detection device 5 of the first embodiment. However, in the processing of step S101 of the flowchart of the first embodiment shown in FIG. 5, the detection unit 564 of the detection device 5b acquires a machine learning model (trained model) from the learning device 4.

[0099] The pre-processing unit 451 of the learning device 4 may further perform the second pre-processing in the first embodiment on the learning captured image 100. The learning execution unit 45 performs the processes in step S303 and thereafter, excluding the learning images obtained from the region corresponding to the excluded region obtained by the second pre-processing in the learning captured image 100 from among the learning images extracted in step S302.

[0100] In the learning stage, the detection system generates training images including images of the detection target from training captured images including images of the detection target object and noise images, and trains a machine learning model. The detection system may perform training using training images extracted from the training captured images based on the structure surrounding the detection target and the degree of obstruction to the detection target. In the inference stage, the detection system performs first and second preprocessing to extract candidate images of the detection target from the detection captured images including images of the detection target object and noise images based on the structure surrounding the detection target and the degree of obstruction to the detection target. The detection system applies a machine learning model to the extracted candidate images of the detection target and detects the image of the detection target from among the candidates. In the inference stage, if the detection system does not perform the second preprocessing to narrow down the candidate images of the detection target based on the degree of obstruction to the detection target, the visibility of small shapes may be reduced in areas with a high density of subjects other than the detection target, such as tumors and tissue, which may impede detection and result in unstable detection performance. The detection system of the above-described embodiment performs a second preprocessing in addition to the first preprocessing to extract a high-density area of ​​subjects other than the detection target, such as tumors or tissues that inhibit the detection of the minute detection target, and excludes the extracted area from the area in which the minute detection target is to be detected. Therefore, it is possible to stabilize the performance of detecting the image of the minute detection target in the captured image.

[0101] The present invention includes the following aspects. [1] A detection system comprising: a candidate generation unit that generates candidate detection target images in which the detection target is captured from a captured image for detection, which includes an image of an object having the detection target and a noise image; a pre-processing unit that divides the captured image for detection into small regions and, for each small region image, determines whether the small region image has features of a subject different from the object having the detection target based on image features of the small region image, and obtains an exclusion region in the captured image based on the determination result; and a detection unit that detects the detection target image from among the candidate detection target images that are not included in the exclusion region by a predetermined detection process. [2] The pre-processing unit excludes small areas in which structures similar to the target appear, or in which the target structure is difficult to recognize, from the small area image. [1] The detection system of [1] [3] The detection system described in [1], wherein the preprocessing unit makes the judgment using a machine learning model trained using images of areas that are not suitable for detection of the detection target image. [4] The detection system described in [3] further comprises a learning unit that learns the machine learning model based on a small area image acquired from a learning captured image that includes an image of an object having the detection target and a noise image, and label information indicating whether or not the image includes a subject other than the object having the detection target. [5] A detection system described in any of [1] to [4], wherein the detection unit detects the detection target image from the candidates of the detection target image generated by the candidate generation unit using the specified detection process, and excludes the detection target image included in the exclusion area from the detected detection target images. [6] The detection system described in [1] further includes a learning unit that trains a machine learning model based on candidate detection target images obtained from learning captured images including images of objects having the detection target and noise images, and label information indicating whether the candidate is a detection target image, and the detection unit uses the machine learning model to detect the detection target image from among the candidate detection target images that are not included in the exclusion area in the detection captured images. [7] A detection system described in any of [1] to [6], wherein the pre-processing unit extracts an object to be subjected to the predetermined detection processing from among the candidate detection object images based on image features of the structure surrounding the detection object in the captured image for detection. [8] The detection system described in [7], wherein the pre-processing unit extracts an object to be subjected to the specified detection processing from among the candidate detection object images based on a linear image corresponding to the structure surrounding the detection object in the captured image for detection. [9] A detection device having a candidate generation unit that generates candidates for a detection target image in which the detection target is captured from a detection capture image that includes an image of an object having the detection target and a noise image; a pre-processing unit that determines, for each small area image obtained by dividing the detection capture image into small areas, whether the small area image has features of a subject different from the object having the detection target based on image features of the small area image, and obtains an exclusion area in the capture image based on the determination result; and a detection unit that detects the detection target image from among the candidates for the detection target image that are not included in the exclusion area by a predetermined detection process.

[10] A detection method comprising: a candidate generation step of generating candidates for a detection target image in which the detection target is captured from a captured image for detection, the candidate generating step including an image of an object having the detection target and a noise image; a preprocessing step of determining, for each small region image obtained by dividing the captured image for detection into small regions, whether or not the small region image has features of a subject different from the object having the detection target based on image features of the small region image, and obtaining an exclusion region in the captured image based on the determination result; and a detection step of detecting the detection target image from among the candidate detection target images that are not included in the exclusion region by a predetermined detection process.

[11] A program for causing a computer to function as the detection device described in [9].

[0102] Although an embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention. [Explanation of symbols]

[0103] 1, 1a, 1b...detection system, 2...communication line, 3...image transmission device, 4...learning device, 5, 5a, 5b...detection device, 41...operation unit, 42...communication unit, 43...storage device, 44...memory, 45...learning execution unit, 46...display unit, 51...operation unit, 52...communication unit, 53...learning model storage device, 54...storage device, 55...memory, 56...detection execution unit, 57...display unit, 58...learning unit, 100, 100a, 100b, 100c, 100d, 151...captured image, 101, 154, 155...mask image, 110... Training image, 111...detection target image, 120...training image, 121...detection target image, 152, 153...captured image, 130...training image, 131...noise image, 200, 200a, 200b...captured images, 201...detection target image, 202...noise image, 210...candidate, 211...candidate, 450...training image generation unit, 451...pre-processing unit, 452...training unit, 453...post-processing unit, 560...candidate generation unit, 561...pre-processing unit, 562...first pre-processing unit, 563...second pre-processing unit, 564...detection unit, 565...post-processing unit [Prior art documents] [Patent documents]

[0104] [Patent Document 1] Patent No. 6629762 [Patent Document 2] Special Publication No. 2021-517255

Claims

1. a candidate generation unit that generates candidates for a detection target image in which the detection target is captured from a captured image for detection that includes an image of an object having the detection target and a noise image; a pre-processing unit that divides the captured image for detection into small regions, determines whether each small region image has features of a subject different from the object to be detected based on image feature amounts of the small region image, and obtains an exclusion region in the captured image based on the determination result; a detection unit that detects the detection target image from among the candidates for the detection target image that are not included in the exclusion area by a predetermined detection process; A detection system comprising:

2. the preprocessing unit designates, as an exclusion region, the small region image in which a structure similar to the detection target appears or the small region image in which the structure of the detection target is difficult to recognize. The detection system of claim 1 .

3. the preprocessing unit performs the determination using a machine learning model trained using an image of an area of ​​the detection target image that is not suitable for detection; The detection system of claim 1 .

4. The machine learning system further includes a learning unit that learns the machine learning model based on small region images acquired from learning captured images including an image of an object having a detection target and a noise image, and label information indicating whether or not the small region images include a subject other than the object having the detection target. The detection system of claim 3 .

5. the detection unit detects the detection target image from among the candidates for the detection target image generated by the candidate generation unit by the predetermined detection process, and excludes the detection target image included in the exclusion area from the detected detection target images. The detection system of claim 1 .

6. The present invention further includes a learning unit that learns a machine learning model based on candidates for detection target images obtained from learning captured images including an image of an object having a detection target and a noise image, and label information indicating whether the candidates are detection target images, the detection unit detects the detection target image from among candidates of the detection target image that are not included in the exclusion area in the captured image for detection, using the machine learning model; The detection system of claim 1 .

7. the preprocessing unit extracts a target to be subjected to the predetermined detection process from among the candidates for the detection target image based on image feature amounts of a structure surrounding the detection target in the captured image for detection; The detection system of claim 1 .

8. the preprocessing unit extracts an object to be subjected to the predetermined detection process from among the candidates for the detection object image based on a linear image corresponding to a structure surrounding the detection object in the captured image for detection; The detection system of claim 7 .

9. a candidate generation unit that generates candidates for a detection target image in which the detection target is captured from a captured image for detection that includes an image of an object having the detection target and a noise image; a pre-processing unit that divides the captured image for detection into small regions, determines whether each small region image has features of a subject different from the object to be detected based on image feature amounts of the small region image, and obtains an exclusion region in the captured image based on the determination result; a detection unit that detects the detection target image from among the candidates for the detection target image that are not included in the exclusion area by a predetermined detection process; A detection device having:

10. a candidate generation step of generating candidates for a detection target image in which the detection target is captured from a captured image for detection including an image of an object having the detection target and a noise image; a pre-processing step of determining whether or not each small-region image obtained by dividing the captured image for detection into small regions has features of a subject different from the object to be detected based on image feature amounts of the small-region image, and obtaining an exclusion region in the captured image based on the determination result; a detection step of detecting the detection target image from among the candidates of the detection target image that are not included in the exclusion area by a predetermined detection process; A detection method comprising:

11. Computer, A program for causing the detection device according to claim 9 to function.

Images