Image processing device, image processing method, and program

Abstract

This image processing device includes at least one processor. The processor: performs division processing, which is processing for dividing an image into a plurality of partial images, the image being divided such that an edge portion of each partial image has an overlapping portion overlapping another adjacent partial image; for each of the plurality of partial images, performs calculation processing for calculating, for each pixel, a numerical value to be used for class classification, the numerical value corresponding to a probability that the pixel belongs to a specific class; and performs integration processing for integrating the calculated numerical values for each of the plurality of partial images to a size of the image prior to the division processing.

Description

Image processing device, image processing method and program

[0001] The disclosed technology relates to an image processing device, an image processing method, and a program.

[0002] The following techniques are known as techniques related to image processing: JP 2020-16928 A discloses an image processing device that includes a likelihood calculation unit that calculates a likelihood indicating the possibility that an area included in a target image is an area representing a structure, a separation calculation unit that calculates a separability indicating the ease of separating an area representing the structure from an area representing a non-structure based on statistics of the likelihood, a conversion unit that, if the separability is smaller than a first threshold, converts the target image to generate a new target image for which the likelihood is calculated, and a combination unit that generates a combined likelihood by combining the likelihoods calculated for each target image.

[0003] JP 2021-149473 A describes an image processing device having a division unit that divides high-quality images and low-quality images into multiple regions, a learning unit that trains multiple learning models on pairs of low-quality images and high-quality images for each divided region, and a restoration unit that divides the low-quality images to be restored using the same method as the division unit, inputs the divided regions into the corresponding learning models, and restores the low-quality images to be restored based on the high-quality images output from each learning model.

[0004] Japanese Patent Application Laid-Open No. 2020-160997 describes an image processing device that includes an acquisition unit that acquires training images, a patch image generation unit that generates a plurality of training patch images from the training images, a label assignment unit that assigns a label to each of the plurality of training patch images in accordance with preset label assignment conditions, and a pseudo image generation unit that trains a neural network that uses the training patch images and labels as inputs and generates pseudo images that restore the training patch images.

[0005] Segmentation, known as one of the image recognition techniques, involves classifying all the images that make up an image. Segmentation can be achieved using a convolutional neural network (CNN). Images input to a CNN can be excessively large, or the CNN may run out of computing resources, such as memory. To address this issue, a method can be considered in which an image is divided into multiple partial images, segmented for each partial image, and then the results of each are integrated.

[0006] On the other hand, in CNN-based segmentation, classification accuracy tends to be lower near the edges of an image than in the center. This is because class classification is performed taking into account the pixels surrounding the target pixel. To avoid this problem, an image can be divided so that each partial image has an overlapping portion with adjacent partial images at the edge (hereinafter referred to as an overlapping portion), and the overlapping portion of each partial image is classified using the classification label (e.g., "0" or "1") obtained for each of the multiple partial images forming the overlapping portion. In other words, the classification label obtained for each of the multiple partial images is integrated to the size of the original image. According to this aspect, the overlapping portion can only be processed by selectively using one of the classification labels obtained for each of the multiple partial images forming the overlapping portion, and the effect of improving the accuracy of class classification near the image edges is limited. As a result, in the label map obtained by integrating the segmentation results for each partial image, low-precision areas appear near the boundaries between partial images, resulting in inaccurate and unnatural results. In other words, when the classification labels obtained for each of multiple partial images are integrated into the size of the original image, it may be difficult to obtain accurate and natural segmentation results due to the lack of information contained in the classification labels.

[0007] The disclosed technology has been made in consideration of the above points, and aims to improve the accuracy of class classification in image processing for class classification, which includes processing of multiple partial images obtained by dividing an image.

[0008] The image processing device according to the disclosed technology includes at least one processor. The processor performs a division process to divide an image into a plurality of partial images so that each partial image has an overlapping portion at its edge where the partial images overlap with other adjacent partial images, a calculation process to calculate, for each of the plurality of partial images, a numerical value used for class classification, which corresponds to the probability that the pixel belongs to a specific class, and an integration process to integrate the calculated numerical value for each of the plurality of partial images to the size of the image before the division process.

[0009] The numerical value used for class classification may be the probability that the pixel belongs to a particular class or its logit.

[0010] In the integration process, the processor may perform a calculation process using numerical values ​​in each overlapping portion of a plurality of partial images that form a common overlapping portion as calculation targets. The processor may perform a process of calculating statistical values ​​for the numerical values ​​to be calculated. The processor may perform a calculation for the numerical values ​​to be calculated according to pixel positions. The processor may perform a process of calculating a weighted average for the numerical values ​​to be calculated using weighting coefficients according to pixel positions.

[0011] The processor may perform class classification for the image based on an integrated value obtained by integrating the numerical values ​​in the integration process. The processor may perform class classification based on the integrated value using an Argmax function or a threshold.

[0012] The image processing method according to the disclosed technology is a process of dividing an image into a plurality of partial images, in which a division process is performed to divide the image so that the edge portions of each partial image have overlapping portions that overlap with other adjacent partial images, a calculation process is performed for each pixel of each of the plurality of partial images to calculate a numerical value used in class classification that indicates the probability that the pixel belongs to a specific class, a calculation process is performed for each pixel of the plurality of partial images to calculate a numerical value used in class classification that corresponds to the probability that the pixel belongs to a specific class, and an integration process is performed to integrate the numerical values ​​calculated for each of the plurality of partial images to the size of the image before the division process, performed by at least one processor possessed by the image processing device.

[0013] The program relating to the disclosed technology is a program for causing at least one processor possessed by an image processing device to execute a process of dividing an image into multiple partial images, performing a division process that divides the image so that the edge portions of each partial image have overlapping portions that overlap with other adjacent partial images, performing a calculation process for each of the multiple partial images that calculates a numerical value used for class classification, which indicates the probability that the pixel belongs to a specific class, for each pixel, and performing an integration process that integrates the numerical value calculated for each of the multiple partial images to the size of the image before the division process.

[0014] According to the disclosed technology, it is possible to improve the accuracy of class classification in image processing for class classification, which includes processing of a plurality of partial images obtained by dividing an image.

[0015] FIG. 1 is a diagram illustrating an example of a hardware configuration of an image processing device according to an embodiment of the disclosed technology. FIG. 2 is a functional block diagram illustrating an example of a functional configuration of an image processing device according to an embodiment of the disclosed technology. FIG. 3 is a diagram illustrating an example of an aspect of division processing according to an embodiment of the disclosed technology. FIG. 4 is a diagram illustrating an example of an aspect of division processing according to an embodiment of the disclosed technology. FIG. 5 is a diagram illustrating a concept of integration processing according to an embodiment of the disclosed technology. FIG. 6 is a flowchart illustrating an example of a flow of image processing according to an embodiment of the disclosed technology.

[0016] Hereinafter, an example of an embodiment of the disclosed technology will be described with reference to the drawings. In each drawing, the same or equivalent components and parts are given the same reference numerals, and redundant description will be omitted.

[0017] FIG. 1 is a diagram illustrating an example of the hardware configuration of an image processing device 10 according to an embodiment of the disclosed technology. The image processing device 10 has a function of performing class classification using segmentation on objects included in an image to be processed (hereinafter referred to as the processing target image). The processing target image is not particularly limited, but may be, for example, a radiographic image acquired during non-destructive testing of industrial products. This radiographic image may include, for example, an image of a defect (flaw) occurring inside the object (e.g., a metal product). The image processing device 10 may, for example, detect or identify defects (flaws) included in this radiographic image by class classification.

[0018] The image processing device 10 includes a CPU (Central Processing Unit) 101, a RAM (Random Access Memory) 102, a non-volatile memory 103, an input device 104, a display 105, and a network interface 106. These hardware components are connected to a bus 107. The display 105 is, for example, a liquid crystal display or an LED (Light Emitting Diode) display. The input device 104 includes, for example, a keyboard and a mouse, and may also include a proximity input device such as a touch panel display, or a voice input device such as a microphone. The network interface 106 is an interface for connecting the image processing device 10 to a network.

[0019] The non-volatile memory 103 is a non-volatile storage medium such as a hard disk or flash memory. An image processing program 110 and a classification model 120 are stored in the non-volatile memory 103. The RAM 102 is a work memory for the CPU 101 to execute processing. The CPU 101 loads the image processing program 110 stored in the non-volatile memory 103 into the RAM 102 and executes processing in accordance with the image processing program 110. The CPU 101 is an example of a "processor" in the disclosed technology.

[0020] The classification model 120 is a mathematical model based on a CNN constructed to predict classification labels for each pixel of a target image. The classification model 120 may be, for example, a known encoder-decoder model (ED-CNN: Encoder-Decoder Convolutional Neural Network). The encoder-decoder model is a model consisting of an encoder that extracts features from an image using a convolutional layer and a decoder that outputs a probability map based on the extracted features. The probability map is the result of deriving the probability that a pixel in the target image belongs to a certain class for each pixel. The classification model 120 is constructed by machine learning using multiple images with correct labels as training data.

[0021] 2 is a functional block diagram showing an example of the functional configuration of the image processing device 10. The image processing device 10 functions as an image acquisition unit 11, a segmentation processing unit 12, a calculation processing unit 13, an integration processing unit 14, and a classification processing unit 15 by the CPU 101 executing an image processing program 110.

[0022] The image acquisition unit 11 acquires an image to be processed. The image to be processed may be downloaded from an image server (not shown) and stored in the non-volatile memory 103, for example.

[0023] The segmentation processing unit 12 performs segmentation processing to divide the processing target image acquired by the image acquisition unit 11 into multiple partial images. FIG. 3A is a diagram showing an example of the segmentation processing. As shown in FIG. 3A, the segmentation processing unit 12 segments the processing target image 200 into multiple partial images 210. The shape of each partial image 210 may be, for example, a square or a rectangle. The size of each partial image 210 is set so that computational resources, including the CPU 101 and RAM 102, are not insufficient when processing using the classification model 120 (a calculation process described below) is performed for each partial image 210. The size of each partial image 210 may be, for example, 256 pixels x 256 pixels. For example, if the processing target image 200 is a radiographic image containing images of defects (flaws) occurring inside an object (e.g., a metal product), the size of the defects (flaws) to be classified is, for example, approximately 1 to 10 pixels. That is, each partial image 210 may contain images of multiple defects (flaws).

[0024] The segmentation processing unit 12 segments the processing target image 200 so that each edge of the partial image 210 has an overlapping portion 211 overlapping with an adjacent partial image 210. FIG. 3B illustrates partial image 210E and adjacent partial images 210A, 210B, 210C, 210D, 210F, 210G, 210H, and 210I. For example, the top edge of partial image 210E has an overlapping portion 211T overlapping with partial image 210B. The right edge of partial image 210E has an overlapping portion 211R overlapping with partial image 210F. The bottom edge of partial image 210E has an overlapping portion 211B overlapping with partial image 210H. The left edge of partial image 210E has an overlapping portion 211L overlapping with partial image 210D. The upper right corner of partial image 210E has an overlapping portion 211TR with partial images 210B, 210C, and 210F. The lower right corner of partial image 210E has an overlapping portion 211BR with partial images 210F, 210I, and 210H. The lower left corner of partial image 210E has an overlapping portion 211BL with partial images 210H, 210G, and 210D. The upper left corner of partial image 210E has an overlapping portion 211TL with partial images 210D, 210A, and 210B.

[0025] 3A and 3B illustrate an example in which the division process is performed so that the plurality of partial images 210 are arranged in a matrix, but the division process may also be performed so that the plurality of partial images 210 are arranged in a honeycomb pattern, as shown in Fig. 4. In other words, the partial images 210 may be arranged so that the distance between the centers of the partial images 210 is constant and the partial images 210 are laid out.

[0026] The calculation processing unit 13 performs a calculation process for each of the plurality of partial images 210 using the classification model 120 to calculate, for each pixel, a numerical value used for classification, which is a numerical value corresponding to the probability that the pixel belongs to a specific class. The "numerical value" above is a numerical value used to estimate a classification label to be assigned to the pixel, and may be, for example, the probability φ that the pixel belongs to a specific class, or the logit f(φ) of the probability φ. The logit f(φ) of the probability φ is expressed by the following equation (1). In the following, an example will be described in which the numerical value calculated by the calculation processing unit 13 is the probability φ that the pixel belongs to a specific class. That is, the calculation processing unit 13 uses the classification model 120 to generate a probability map for each of the plurality of partial images 210.

[0027] The integration processing unit 14 performs integration processing to integrate the probability φ (probability map) calculated for each pixel for each of the multiple partial images 210 into the size of the processing target image 200 before the division processing. FIG. 5 is a diagram showing the concept of the integration processing performed by the integration processing unit 14. For parts other than the overlapping parts 211 of the partial images 210, the integration processing unit 14 outputs the probability φ calculated for each pixel for each of the multiple partial images 210 as an integrated value of the pixel. On the other hand, the integration processing unit 14 performs calculation processing on the probability φ in each overlapping part 211 of the multiple partial images 210 that form a common overlapping part 211. Specifically, the integration processing unit 14 calculates the probability φ in the overlapping part 211 of each of the two partial images 210 shown in FIG. 5 ６ and probability φ ８ The value f(φ) obtained by the calculation process ６ , φ ８ ) is output as the integrated value of one pixel in the overlapping portion 211, and the probability φ ７ and probability φ ９ The value f(φ) obtained by the calculation process ７ , φ ９) is output as the integrated value of the other pixel in the overlapping portion 211. For corner portions where there are four partial images 210 that form the common overlapping portion 211 (see FIG. 3B ), calculation processing is performed using the four probabilities φ calculated for each of the four partial images 210.

[0028] In the above-described calculation process performed on the overlapping portion 211, the integration processing unit 14 may perform a process of calculating a statistical value for the probability φ to be calculated. The statistical value may be, for example, an average value, a median value, a maximum value, or a minimum value. For example, when the statistical value is an average value, in the example shown in FIG. 5 , the integration processing unit 14 calculates the probability φ ６ and probability φ ８ is output as the integrated value of one pixel in the overlapping portion 211, and the probability φ ７ and probability φ ９ 5, the integration processing unit 14 outputs the average value of probability φ as the integrated value of the other pixel in the overlapping portion 211. On the other hand, when the statistical value is the maximum value, in the example shown in FIG. ６ and probability φ ８ The one showing the maximum value is output as the integrated value of one pixel in the overlapping portion 211, and the probability φ ７ and probability φ ９ The one showing the maximum value is output as the integrated value of the other pixel in the overlapping portion 211.

[0029] In the above-described calculation process performed on the overlapping portion 211, the integration processing unit 14 may perform a calculation of the probability φ to be calculated according to the pixel position. Specifically, the integration processing unit 14 may perform a process of calculating a weighted average of the probability φ to be calculated using a weighting coefficient according to the pixel position. In this case, it is preferable to use a weighting coefficient that decreases as the pixel position approaches the edge, so that the contribution rate of the probability φ calculated for pixels close to the edge of each partial image 210 becomes relatively low. That is, in the example shown in FIG. 5 , the probability φ in the overlapping portion 211 ６ , φ ７ , φ ８ , φ ９ The weighting coefficients added to each are w ６ , w ７ , w ８ , w９ In this case, the following equations (2) to (5) hold: Weighted average X(φ ６ ,φ ８ ) and X(φ ７, φ ９ ) is expressed by the following equations (6) and (7): ６ >w ７ ... (2) w ８ <w ９ ... (3) w ６ +w ８ = 1 ... (4) w ７ +w ９ =1...(5) X(φ ６ ,φ ８ ) = w ６ ・φ ６ +w ８ ・φ ８ ... (6) X (φ ７ ,φ ９ ) = w ７ ・φ ９ +w ７ ・φ ９ ... (7)

[0030] It is preferable to set the weighting coefficients so that the difference between adjacent pixels in the integrated value obtained as a result of the integration process falls within a predetermined range. In other words, it is preferable to set the weighting coefficients so that the integrated values ​​of adjacent pixels do not differ significantly. Such weighting coefficients can be expressed, for example, by the following equations (8) to (13).

[0031] In equation (8), f(P) is a processing function of the integration process for the set P of multiple partial images p that form the common overlapping portion 211. ｐis a weighting coefficient assigned to a pixel in partial image p. value(p) is a processing function for the calculation process that calculates the probability φ for partial image p. Equation (9) indicates that the sum of the weighting coefficients is 1. In equation (10), size is the size of partial image p. pos(p) is a function that outputs the pixel position of partial image p in a Cartesian coordinate system. Equation (10) indicates that a weighting coefficient of "0" is assigned to pixels located at the outermost periphery of partial image p. Equation (11) indicates that the weighting coefficient changes continuously depending on the distance from the edge of the pixel position.

[0032] Equation (12) indicates that the weighting factor assigned to a pixel varies linearly with the distance from the edge of the pixel, while equation (13) indicates that the weighting factor assigned to a pixel varies exponentially with the distance from the edge of the pixel.

[0033] The calculation according to the pixel position is not limited to the weighted average using the weighting coefficient according to the pixel position. The integration processing unit 14 may output the probability φ calculated for the pixel farthest from the edge among the probabilities φ to be calculated as the integrated value. For example, in the example shown in FIG. 5, the integration processing unit 14 may output the probability φ ６ and probability φ ８ Among these, the probability φ calculated for the pixel farther from the edge ６ is output as the integrated value of one pixel in the overlapping portion 211, and the probability φ ７ and probability φ ９ Among these, the probability φ calculated for the pixel farther from the edge ９ may be output as the integrated value of the other pixel in the overlapping portion 211.

[0034] The classification processing unit 15 performs class classification for each pixel of the processing target image 200 based on the integrated value of the probability φ obtained by the integration process. That is, the classification processing unit 15 estimates the class to which each pixel of the processing target image 200 belongs based on the integrated value of the probability φ, and assigns a classification label corresponding to the estimated class to the pixel. The classification processing unit 15 may perform class classification using, for example, an Argmax function. Alternatively, the classification processing unit 15 may perform class classification using, for example, a threshold value. Note that the classification processing unit 15 may be implemented in an image processing device separate from the image processing device 10. In this case, the image processing device 10 does not need to perform classification processing.

[0035] 6 is a flowchart showing an example of the flow of image processing performed by the CPU 101 executing the image processing program 110. The image processing program 110 is executed when, for example, the user operates the input device 104 to instruct the start of processing.

[0036] In step S1, the image acquisition unit 11 acquires the processing target image 200. In step S2, the division processing unit 12 performs division processing to divide the processing target image 200 acquired in step S1 into a plurality of partial images 210. The division processing unit 12 divides the processing target image 200 so that each partial image 210 has an overlapping portion 211 at its edge where it overlaps with another adjacent partial image 210.

[0037] In step S3, the calculation processing unit 13 performs a calculation process for each of the plurality of partial images 210, using the classification model 120, to calculate a numerical value used in classification, which is a numerical value corresponding to the probability that the pixel belongs to a specific class. The "numerical value" may be, for example, the probability φ that the pixel belongs to a specific class, or its logit f(φ).

[0038] In step S4, the integration processing unit 14 performs integration processing to integrate the probability φ (probability map) calculated for each of the multiple partial images 210 to the size of the processing target image 200 before the division processing. For parts other than the overlapping parts 211 of the partial images 210, the integration processing unit 14 outputs the probability φ calculated for each pixel for each of the multiple partial images 210 as an integrated value of that pixel. On the other hand, the integration processing unit 14 performs calculation processing on the probability φ in each overlapping part 211 of the multiple partial images 210 that form the common overlapping part 211.

[0039] In step S5, the classification processing unit 15 performs class classification for each pixel of the processing target image based on the integrated value obtained by the integration processing in step S4. That is, the classification processing unit 15 estimates the class to which each pixel of the processing target image 200 belongs based on the integrated value of the probability φ, and assigns a classification label corresponding to the estimated class to the pixel. The classification processing unit 15 may perform class classification using, for example, an Argmax function. Alternatively, the classification processing unit 15 may perform class classification using, for example, a threshold value.

[0040] As described above, the image processing device 10 according to the embodiment of the disclosed technology performs a process of dividing a processing target image 200 into a plurality of partial images 210, dividing the processing target image 200 so that each partial image 210 has an overlapping portion 211 at its edge where the partial image 210 overlaps with another adjacent partial image. The image processing device 10 performs a calculation process of calculating, for each of the plurality of partial images 210, a numerical value used for class classification, which corresponds to the probability that the pixel in question belongs to a specific class. The image processing device 10 performs an integration process of integrating the numerical value calculated for each of the plurality of partial images 210 to the size of the processing target image 200 before the division process. In the integration process, the image processing device 10 performs a calculation process that targets the numerical value in each overlapping portion of the plurality of partial images 210 that form a common overlapping portion. The image processing device 10 performs class classification for the processing target image 200 based on an integrated value obtained by integrating the numerical values ​​in the integration process.

[0041] Segmentation, a well-known image recognition technique, can be achieved using a CNN. However, if the size of the image input to a CNN is excessively large, it may run out of computational resources such as memory. To address this issue, a method can be considered in which the image is divided into multiple partial images, segmented for each partial image, and then the results of each are integrated.

[0042] On the other hand, in segmentation using a CNN, classification accuracy tends to be lower near the edges of an image than in the center. This is because class classification is performed taking into account the pixels surrounding the target pixel. To avoid this problem, an image can be divided so that each partial image has an overlapping portion with adjacent partial images at the edge, and the overlapping portion of each partial image is classified using the classification label (e.g., "0" or "1") obtained for each of the multiple partial images forming the overlapping portion. In other words, the classification label obtained for each of the multiple partial images is integrated to the size of the original image. According to this aspect, the overlapping portion can only be processed by selectively using one of the classification labels obtained for each of the multiple partial images forming the overlapping portion, and the effect of improving the accuracy of class classification near the image edges is limited. As a result, in the label map obtained by integrating the segmentation results for each partial image, low-precision areas appear near the boundaries between partial images, resulting in inaccurate and unnatural results. In other words, when the classification labels obtained for each of multiple partial images are integrated into the size of the original image, it may be difficult to obtain accurate and natural segmentation results due to the lack of information contained in the classification labels.

[0043] According to the image processing device 10 of the embodiment of the disclosed technology, numerical values ​​such as the probability φ calculated for each of the multiple partial images 210 are subjected to the integration process. This enables more advanced calculation processing to be performed in the integration process compared to integrating classification labels, thereby improving the accuracy of class classification in the overlapping portion 211. Furthermore, as described above, in segmentation using a CNN, classification accuracy tends to be lower near the edges of an image than in the center of the image. According to the image processing device 10, calculations are performed according to pixel position in the calculation processing in the integration process, making it possible to address the problem of reduced accuracy in class classification near the edges of an image.

[0044] In each of the above embodiments, the following various processors can be used as the hardware structure of processing units that perform various processes, such as the image acquisition unit 11, the segmentation processing unit 12, the calculation processing unit 13, the integration processing unit 14, and the classification processing unit 15. As described above, the various processors include CPUs and GPUs, which are general-purpose processors that execute software (programs) and function as various processing units, as well as dedicated electrical circuits, such as programmable logic devices (PLDs) that are processors whose circuit configuration can be changed after manufacture, such as FPGAs, and application-specific integrated circuits (ASICs), which are processors with a circuit configuration designed specifically for performing specific processes.

[0045] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0046] Examples of configuring multiple processing units with a single processor include: first, a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server computers, and this processor functions as multiple processing units; second, a form in which a processor is used to realize the functions of an entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs); in this way, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0047] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0048] Furthermore, in the above embodiment, the image processing program 110 is described as being pre-stored (installed) in the non-volatile memory 103, but this is not limiting. The image processing program 110 may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. The image processing program 110 may also be downloaded from an external device via a network. In other words, the program described in this embodiment (i.e., the program product) may be provided on a recording medium or may be distributed from an external computer.

[0049] The following supplementary note is further disclosed regarding the above embodiment: (Supplementary Note 1) An image processing device having at least one processor, wherein the processor performs a division process to divide an image into a plurality of partial images so that each partial image has an overlapping portion at an edge portion where the partial images overlap with other adjacent partial images, a calculation process to calculate, for each of the plurality of partial images, a numerical value used for class classification, the numerical value corresponding to the probability that the pixel belongs to a specific class, and an integration process to integrate the numerical value calculated for each of the plurality of partial images to the size of the image before the division process.

[0050] (Supplementary Note 2) The image processing device according to Supplementary Note 1, wherein the numerical value is a probability that the pixel belongs to a specific class or a logit thereof.

[0051] (Supplementary Note 3) The image processing device according to Supplementary Note 1 or Supplementary Note 2, wherein the processor performs a calculation process in the integration process, with the numerical values ​​in each overlapping portion of a plurality of partial images forming a common overlapping portion as calculation targets.

[0052] (Supplementary Note 4) The image processing device according to Supplementary Note 3, wherein the processor performs processing to calculate a statistical value for the numerical value of the object of calculation.

[0053] (Supplementary Note 5) The image processing device according to Supplementary Note 3, wherein the processor performs a calculation on the numerical value of the calculation target according to a pixel position.

[0054] (Supplementary Note 6) The image processing device according to Supplementary Note 5, wherein the processor performs processing to calculate a weighted average of the numerical values ​​of the objects of calculation using weighting coefficients according to pixel positions.

[0055] (Supplementary Note 7) The image processing device according to any one of Supplementary Note 1 to Supplementary Note 6, wherein the processor performs the class classification for the image based on an integrated value obtained by integrating the numerical values ​​in the integration process.

[0056] (Supplementary Note 8) The image processing device according to Supplementary Note 7, wherein the processor performs the class classification using an Argmax function or a threshold value based on the integrated value.

[0057] (Supplementary Note 9) An image processing method executed by at least one processor of an image processing device, comprising: a process for dividing an image into a plurality of partial images, wherein the process divides the image so that the edge of each of the partial images has an overlapping portion where the partial images overlap with other adjacent partial images; a calculation process for calculating, for each pixel of each of the plurality of partial images, a numerical value used for class classification that indicates the probability that the pixel belongs to a specific class; a calculation process for calculating, for each pixel of the plurality of partial images, a numerical value used for class classification that corresponds to the probability that the pixel belongs to a specific class; and an integration process for integrating the numerical value calculated for each of the plurality of partial images to the size of the image before the division process.

[0058] (Supplementary Note 10) A program for causing at least one processor of an image processing device to execute the following processes: a process of dividing an image into a plurality of partial images, wherein the process divides the image so that the edge of each of the partial images has an overlapping portion where the partial images overlap with other adjacent partial images; a calculation process for calculating, for each of the plurality of partial images, a numerical value used for class classification, which indicates the probability that the pixel belongs to a specific class; and an integration process for integrating the numerical value calculated for each of the plurality of partial images to the size of the image before the division process.

[0059] The disclosure of Japanese Patent Application No. 2022-153069, filed on September 26, 2022, is incorporated herein by reference in its entirety. In addition, all documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually indicated to be incorporated by reference.

Claims

1. 1. An image processing device having at least one processor, The processor, A process of dividing an image into a plurality of partial images, the process dividing the image into partial images such that each of the partial images has an overlapping portion at an edge portion thereof that overlaps with another adjacent partial image; performing a calculation process for calculating, for each of the plurality of partial images, a numerical value used for class classification, the numerical value corresponding to the probability that the pixel belongs to a particular class; A consolidation process is performed to consolidate the numerical values ​​calculated for each of the plurality of partial images to the size of the image before the division process. Image processing device.

2. The value is the probability that the pixel belongs to a particular class, or its logit. The image processing device according to claim 1 .

3. In the integration process, the processor performs a calculation process on the numerical values ​​in each overlapping portion of a plurality of partial images forming a common overlapping portion. The image processing device according to claim 2 .

4. The processor performs a process of calculating a statistical value for the numerical value of the object of calculation. The image processing device according to claim 3 .

5. The processor performs a calculation on the numerical value of the object of calculation according to a pixel position. The image processing device according to claim 3 .

6. The processor performs a process of calculating a weighted average of the numerical values ​​of the objects of calculation using weighting coefficients according to pixel positions. The image processing device according to claim 5 .

7. The processor performs the class classification for the image based on an integrated value obtained by integrating the numerical values ​​in the integration process. The image processing device according to claim 1 .

8. The processor performs the classification using an Argmax function or a threshold based on the integration value. The image processing device according to claim 7.

9. A process of dividing an image into a plurality of partial images, the process dividing the image into partial images such that each of the partial images has an overlapping portion at an edge portion thereof that overlaps with another adjacent partial image; performing a calculation process for calculating, for each of the plurality of partial images, a numerical value used for class classification, the numerical value corresponding to the probability that the pixel belongs to a particular class; A consolidation process is performed to consolidate the numerical values ​​calculated for each of the plurality of partial images to the size of the image before the division process. The image processing method is executed by at least one processor included in the image processing device.

10. A process for dividing an image into a number of partial images, the process being carried out by detecting the edges of each of the partial images. performing a division process for dividing the image so as to have an overlapping portion that overlaps with another adjacent partial image; performing a calculation process for calculating, for each of the plurality of partial images, a numerical value used for class classification, the numerical value corresponding to the probability that the pixel belongs to a particular class; A consolidation process is performed to consolidate the numerical values ​​calculated for each of the plurality of partial images to the size of the image before the division process. A program for causing at least one processor of an image processing device to execute the above steps.

11. In the integration process, the processor uses the numerical value calculated for each pixel of each of the plurality of partial images as an integrated value for the pixel other than the overlapping portion of the partial images. The image processing device according to claim 2 .

Images