Image processing device, image processing method, and image processing program

Abstract

Provided is an algorithm for identifying regions of objects included in an image of interest. This image processing device comprises: a binarization unit that generates a binary image by binarizing an image of interest, which includes a plurality of objects, into object regions and regions other than the object regions; a distance conversion unit that subjects the binary image to distance conversion so as to generate a distance map; a local maximum point extraction unit that extracts a plurality of local maximum points from the distance map; a boundary point extraction unit that extracts a plurality of boundary points positioned on the boundaries between the object regions and the regions other than the object regions in the image of interest or the binary image; and an identification unit that, for each of the plurality of boundary points, calculates the distance to each of the plurality of local maximum points so as to determine a plurality of the boundary points to be associated with each of the local maximum points, and approximates the plurality of boundary points associated with each local maximum point by using graphic information pertaining to a prescribed shape having the local maximum point as a central point, thereby identifying an object region including the local maximum point.

Description

Image processing apparatus, image processing method, and image processing program

[0001] The present disclosure relates to an image processing apparatus, an image processing method, and an image processing program.

[0002] As a method for specifying the area of each object in a target image including a plurality of dense objects, for example, a method using image analysis and a method using deep learning are known. Specifically, as a method using image analysis, algorithms such as the watershed method and Selective search are known. Also, as a method using deep learning, algorithms such as Mask R-CNN (Region-Based Convolutional Neural Networks) are known. According to these methods, even when the shape, size, color, etc. of each object are not uniform, the area of each object can be specified in the target image.

[0003] Japanese Patent Application Laid-Open No. 2022-74092, Patent No. 6267710, Patent No. 6832479, International Publication No. 2013-524361

[0004] However, all of the above methods have a problem that prior work by the user is required to operate each algorithm, and the work load of the user is high. For example, in the case of the method using image analysis, the user needs to adjust and set appropriate parameters for each target image. Also, in the case of the method using deep learning, the user needs to generate learning data and perform pre-learning of a model based on the generated learning data.

[0005] On the other hand, depending on the target image, domain knowledge such as the shape of each object being generally uniform may be obtained. In such a case, it is expected to reduce the work load of the user by applying an algorithm that utilizes the domain knowledge instead of the above algorithms.

[0006] The present disclosure provides an algorithm for specifying the area of each object included in a target image.

[0007] An image processing apparatus according to a first aspect of the present disclosure includes: a binarization unit that binarizes a target image containing multiple objects into regions of objects and regions other than the regions of objects to generate a binary image; a distance transformation unit that transforms the binary image by distance to generate a distance map; a maximal point extraction unit that extracts multiple maximal points from the distance map; a boundary point extraction unit that extracts multiple boundary points located at the boundary between the regions of objects and regions other than the regions of objects in the target image or the binary image; and a specification unit that determines a plurality of boundary points associated with each of the plurality of boundary points by calculating the distance from each of the plurality of maximal points to each of the plurality of boundary points, and identifies the region of an object containing the maximal point by approximating the plurality of boundary points associated with the maximal point using graphic information of a predetermined shape with the maximal point as the center point.

[0008] A second aspect of this disclosure is an image processing apparatus according to the first aspect, wherein the identifying unit identifies the region of an object with respect to the maximum points among the plurality of maximum points, excluding the maximum points for which the number of associated boundary points is less than or equal to a predetermined number.

[0009] A third aspect of the present disclosure is an image processing apparatus according to the first or second aspect, comprising a display unit that superimposes and displays information relating to the region of an object, which includes a maximum point, a plurality of boundary points associated with the maximum point, and graphic information of a predetermined shape that approximates the plurality of boundary points associated with the maximum point, onto a target image.

[0010] A fourth aspect of the present disclosure is an image processing apparatus according to the third aspect, wherein the display unit displays first information relating to the region of a first object and second information relating to the region of a second object for two adjacent objects, using different colors for each object, and, if the two objects overlap, displays both the graphic information included in the first information and the graphic information included in the second information for the overlapping portion.

[0011] A fifth aspect of this disclosure is an image processing apparatus according to any of the first to fourth aspects, wherein the graphic information of a predetermined shape includes the position coordinates of each point constituting the graphic of the predetermined shape, and is calculated by a function representing the graphic of the predetermined shape.

[0012] A sixth aspect of this disclosure is an image processing apparatus according to the fifth aspect, wherein the predetermined shape of the figure is a figure that is point-symmetric with respect to a center point.

[0013] A seventh aspect of this disclosure is an image processing apparatus according to the fifth or sixth aspect, wherein the predetermined shape of the figure is a circular or elliptical figure.

[0014] An eighth aspect of the present disclosure is an image processing apparatus according to any of the first to seventh aspects, comprising a statistical processing unit that calculates a feature quantity indicating the size of the region of an object identified by the identification unit, and statistically processes the feature quantity for a plurality of objects included in the target image.

[0015] A ninth aspect of the present disclosure is an image processing method, wherein a computer performs the following steps: binarizing a target image containing multiple objects into regions of objects and regions other than the regions of objects to generate a binary image; performing distance transformation on the binary image to generate a distance map; extracting multiple local maxima from the distance map; extracting multiple boundary points located at the boundary between the regions of objects and regions other than the regions of objects in the target image or the binary image; and determining multiple boundary points associated with each of the multiple boundary points by calculating the distance from each of the multiple local maxima to each of the multiple local maxima, and identifying the region of an object containing the local maxima by approximating the multiple boundary points associated with the local maxima using geometric information of a predetermined shape with the local maxima as the center point.

[0016] A tenth aspect of the present disclosure is an image processing program that causes a computer to perform the following steps: binarize a target image containing multiple objects into regions of objects and regions other than the regions of objects to generate a binary image; perform distance transformation on the binary image to generate a distance map; extract multiple local maxima from the distance map; extract multiple boundary points located at the boundary between the regions of objects and regions other than the regions of objects in the target image or the binary image; and determine multiple boundary points associated with each of the multiple boundary points by calculating the distance from each of the multiple local maxima to each of the multiple local maxima, and identify the region of an object containing the local maxima by approximating the multiple boundary points associated with the local maxima using geometric information of a predetermined shape with the local maxima as the center point.

[0017] This disclosure provides an algorithm for identifying the region of each object included in a target image.

[0018] Figure 1 shows an example of the system configuration of a process control system. Figure 2 shows an example of the hardware configuration of an image processing device. Figure 3 shows an example of a target image. Figure 4 shows an example of the functional configuration of the region identification unit of the image processing device. Figure 5 shows a specific example of distance map generation processing by the distance conversion unit. Figure 6 shows specific examples of local maximums and boundary points. Figure 7 shows specific examples of linking processing and geometric approximation processing by the identification unit. Figure 8 shows a specific example of display processing by the display unit. Figure 9A is a flowchart showing the flow of region identification processing by the region identification unit. Figure 9B is a flowchart showing the flow of linking processing and geometric approximation processing by the identification unit. Figure 10 shows an example of the processing result of region identification processing by the region identification unit.

[0019] Each embodiment will be described below with reference to the attached drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0020] [First Embodiment] <System Configuration of Process Control System> First, the overall system configuration of the process control system equipped with the image processing device according to the first embodiment will be described. Figure 1 is a diagram showing an example of the system configuration of the process control system.

[0021] As shown in Figure 1, the process control system 100 comprises a target process 110, a target image generation process, and an image processing device 130.

[0022] The target process 110 is an object group generation process, which is a process that generates multiple objects. In the first embodiment, the multiple objects generated by the object group generation process are assumed to be multiple spherical cells or multiple spherical particles. For example, these multiple objects may be multiple spheroids or the like generated by a culture process.

[0023] As shown in Figure 1, multiple objects 140 generated by the target process 110 are transported to the target image generation process 120.

[0024] The target image generation process 120 photographs the transported objects 140 under predetermined conditions using, for example, an optical microscope or an SEM (Scanning Electron Microscope). As a result, the target image generation process 120 generates an image containing multiple densely packed objects, which is then processed by the image processing device 130.

[0025] If the multiple objects 140 are, for example, multiple spheroids, the target image generation process 120 photographs the culture medium containing the multiple spheroids cultured in three dimensions using an optical microscope. This allows the target image generation process 120 to generate a target image containing multiple densely packed spheroids.

[0026] As shown in Figure 1, the target image generated by the target image generation process 120 is notified to the image processing device 130.

[0027] An image processing device 130 has an image processing program installed on it, and when this image processing program is executed, the image processing device 130 functions as a region identification unit 131 and a statistical processing unit 132.

[0028] The region identification unit 131 identifies the region of each object in a target image containing multiple densely packed objects, displays information about the region of each object superimposed on the target image, and notifies the statistical processing unit 132. In this embodiment, focusing on the fact that each object has a spherical shape (circular shape in a two-dimensional target image), the region identification unit 131 uses an algorithm that leverages domain knowledge that the shape of each object is circular to identify the region of each object. This allows for the appropriate identification of the region of each object and reduces the user's workload when identifying the region of each object.

[0029] The statistical processing unit 132 calculates a feature quantity indicating the size of each object (for example, the area of ​​each object) based on information about the region of each object, and performs statistical processing on the calculated feature quantity. This allows the statistical processing unit 132 to output the results of the statistical processing on the feature quantity indicating the size of each object for each target image. Furthermore, the target process 110 can appropriately change the process conditions by referring to the results of the statistical processing output by the statistical processing unit 132.

[0030] <Hardware Configuration of Image Processing Device> Next, the hardware configuration of the image processing device 130 will be described. Figure 2 is a diagram showing an example of the hardware configuration of the image processing device.

[0031] As shown in Figure 2, the image processing device 130 includes a processor 201, memory 202, auxiliary storage device 203, I / F (Interface) device 204, communication device 205, and drive device 206. The hardware components of the image processing device 130 are interconnected via a bus 207.

[0032] The processor 201 has various computing devices such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processor 201 reads various programs (for example, image processing programs, etc.) into the memory 202 and executes them.

[0033] Memory 202 has main memory devices such as ROM (Read Only Memory) and RAM (Random Access Memory). The processor 201 and memory 202 form a so-called computer, and the computer realizes various functions by having the processor 201 execute various programs read from memory 202.

[0034] The auxiliary storage device 203 stores various programs and various data used when these programs are executed by the processor 201.

[0035] The I / F device 204 is a connection device that connects the operating device 210 and the display device 220 to the image processing device 130. The operating device 210 is an operating device for the user to input various instructions to the image processing device 130, and the display device 220 is a display device for providing a display screen to the user. The communication device 205 is a communication device for communicating with external devices (not shown) via a network.

[0036] The drive device 206 is a device for setting the recording medium 230. The recording medium 230 here includes media that record information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, and magneto-optical disks. The recording medium 230 may also include semiconductor memory that records information electrically, such as ROMs and flash memory.

[0037] The various programs to be installed on the auxiliary storage device 203 are installed, for example, when the distributed recording medium 230 is set in the drive device 206 and the various programs recorded on the recording medium 230 are read by the drive device 206. Alternatively, the various programs to be installed on the auxiliary storage device 203 may be installed by downloading them from the network via the communication device 205.

[0038] <Details of the target image> Next, we will explain the details of the target image generated by the target image generation process 120. Figure 3 shows an example of a target image.

[0039] As shown in Figure 3, the target image 300 has the following characteristics: • It contains multiple densely packed objects. • These multiple objects are generally spherical in shape (generally circular in the two-dimensional target image 300). • The size of each object is not necessarily uniform. • Some objects are adjacent to other objects, or overlap with other objects in the direction of capture.

[0040] <Functional Configuration of Region Identification Unit> Next, the functional configuration of the region identification unit 131, which identifies the region of each object included in the target image 300, will be explained. As described above, the region identification unit 131 identifies the region of each object using an algorithm that takes advantage of domain knowledge that the shape of each object is circular.

[0041] Figure 4 shows an example of the functional configuration of the region identification unit of the image processing device. The region identification unit 131 implements the above algorithm with the functional configuration shown in Figure 4. As shown in Figure 4, the region identification unit 131 has: a binarization unit 401, a distance conversion unit 402, a boundary point extraction unit 403, a maximum point extraction unit 404, a identification unit 405, and a display unit 406.

[0042] The binarization unit 401 acquires the target image and executes "binarization processing". Specifically, the binarization unit 401 binarizes the target image into an object region and a background region (regions other than the object region) to generate a binary image. The binarization unit 401 notifies the generated binary image to the distance conversion unit 402 and the boundary point extraction unit 403.

[0043] The distance conversion unit 402 executes "distance map generation processing" to generate a distance map by performing distance conversion with the binary image notified from the binarization unit 401 as input. Specifically, the distance conversion unit 402 calculates the distance from each pixel (for example, each pixel in the white portion of the binary image) to the nearest pixel value 0 (for example, the distance to the nearest pixel in the black portion of the binary image) to generate a distance map. The distance conversion unit 402 notifies the generated distance map to the maximum point extraction unit 404.

[0044] The boundary point extraction unit 403 executes "boundary point extraction processing" on the binary image notified from the binarization unit 401. Specifically, the boundary point extraction unit 403 extracts boundary points indicating the boundary (edge) between the object region and the background region. The boundary point extraction unit 403 notifies the extracted plurality of boundary points to the identification unit 405. Note that the target for which the boundary point extraction unit 403 performs boundary point extraction processing is not limited to the binary image. For example, instead of the binary image, boundary point extraction processing may be executed on the target image.

[0045] The maximum point extraction unit 404 executes "maximum point extraction processing" on the distance map notified from the distance conversion unit 402. Specifically, the maximum point extraction unit 404 determines, for each region where pixels having pixel values equal to or greater than a predetermined threshold value in the distance map are aggregated, the pixel with the maximum pixel value and extracts it as a maximum point. The maximum point extraction unit 404 notifies the extracted plurality of maximum points to the identification unit 405.

[0046] The specific part 405 acquires the plurality of boundary points notified by the boundary point extraction part 403 and the plurality of maximum point extracted by the maximum point extraction part 404, and executes "association processing". Specifically, for each of the plurality of boundary points acquired from the boundary point extraction part 403, the specific part 405 calculates the distance between each of the plurality of maximum points acquired from the maximum point extraction part 404. For each of the plurality of boundary points, the specific part 405 determines the maximum point with the minimum distance among the plurality of maximum points, thereby determining the maximum point to be associated for each of the plurality of boundary points. At this time, the specific part 405 removes the maximum points for which the number of associated boundary points is less than or equal to a predetermined number.

[0047] The specific part 405 executes "figure approximation processing" using the maximum points and the boundary points associated with the maximum points. Specifically, the specific part 405 approximates the plurality of boundary points associated with each maximum point using circular figure information (approximate circle) centered on each maximum point. Thereby, the specific part 405 can specify "information on the region of the object" including the maximum point, the plurality of boundary points associated with the maximum point, and the approximate circle obtained by approximating the boundary points associated with the maximum point. The specific part 405 notifies the statistical processing part 132 and the display part 406 of the "information on the region of the object" specified for each maximum point.

[0048] The display part 406 executes "display processing" for displaying information on the region of the object. Specifically, the display part 406 acquires the target image, and superimposes and displays the information on the region of each object notified from the specific part 405 on the acquired target image.

[0049] <Specific examples of the processing of each part of the region specifying part> Next, specific examples of the processing by each part (here, the distance conversion part 402, the boundary point extraction part 403, the maximum point extraction part 404, the specific part 405, the display part 406) of the region specifying part 131 will be described. In the following, in order to clarify the "algorithm utilizing the domain knowledge that the shape of each object is circular" executed by the region specifying part 131, a case of processing an image including a plurality of densely packed coins (circular coins) as the target image will be described.

[0050] (1) Specific Example of Distance Map Generation Process by Distance Conversion Unit First, a specific example of the distance map generation process by the distance conversion unit 402 will be described. Figure 5 is a diagram showing a specific example of the distance map generation process by the distance conversion unit. As described above, the distance conversion unit 402 generates a distance map by performing a distance conversion on the binary image notified by the binarization unit 401 as input.

[0051] As shown in Figure 5, the target image 510 is an image containing multiple coins that are densely packed together. The binary image 520 generated by performing a binarization process on the target image 510 is an image in which the coin regions are white and the regions other than the coin regions (background regions) are black. When the distance conversion unit 402 is notified of the binary image 520, it performs a distance conversion on the binary image 520 and generates a distance map 530.

[0052] As shown in Figure 5, the distance map 530 shows that the pixel values ​​are highest near the center of each coin, and decrease as you approach the background area. Also, between coins that are placed adjacent to each other, the pixel values ​​along the line connecting their center positions are higher than zero.

[0053] (2) Specific Examples of Maximum Points and Boundary Points Next, specific examples of maximum points extracted by the maximum point extraction unit 404 performing a maximum point extraction process on the distance map 530, and specific examples of boundary points extracted by the boundary point extraction unit 403 performing a boundary point extraction process on the binary image 520 will be described.

[0054] As described above, the identification unit 405 acquires a plurality of boundary points notified by the boundary point extraction unit 403 and a plurality of local maximum points notified by the local maximum point extraction unit 404. Figure 6 is a diagram showing specific examples of local maximum points and boundary points. For the sake of explanation, the image 610 shown in Figure 6 shows the plurality of local maximum points acquired by the local maximum point extraction unit 404 and the plurality of boundary points acquired by the boundary point extraction unit 403 superimposed on the target image 510.

[0055] In Image 610, the points shown in pink indicate boundary points (reference numeral 611 shows an example of a boundary point). As is clear from Image 610, boundary points are plotted at the boundary between the coin region and the region outside the coin region (background region). However, when adjacent to other coins, boundary points are not extracted from the boundary near the adjacent position, and therefore no boundary points are plotted at that position.

[0056] In Image 610, the points shown in red indicate local maximums (reference numerals 621, 622, and 623 show examples of local maximums). As is clear from Image 610, local maximums are plotted at the center of the coin. However, if a coin is adjacent to another coin, a local maximum is also extracted at the adjacent position, and therefore a local maximum is plotted at that position as well.

[0057] (3) Specific Examples of Linking and Geometric Approximation Processing by the Specific Unit Next, specific examples of linking and geometric approximation processing by the specific unit 405 will be described. Figure 7 is a diagram showing specific examples of linking and geometric approximation processing by the specific unit. In Figure 7, image 710 shows two adjacent coins overlapping.

[0058] As shown in Figure 710, when two adjacent coins overlap, the identification unit 405 obtains boundary point 731 extracted for the left coin and boundary point 721 extracted for the right coin from the boundary point extraction unit 403. As shown in Figure 710, the boundary points 731 and 721 obtained by the identification unit 405 both include only the boundary points of the parts where the two coins do not overlap, and do not include the boundary points of the parts where the two coins overlap.

[0059] As shown in Figure 710, the identification unit 405 obtains the maximum point 732 extracted for the left coin and the maximum point 722 extracted for the right coin from the maximum point extraction unit 404. Note that the example in Figure 710 shows the case where the identification unit 405 did not obtain the maximum points at adjacent positions from the maximum point extraction unit 404.

[0060] Under these conditions, when the specific unit 405 performs the linking process, the boundary point 731 is linked to the maximum point 732, and the boundary point 721 is linked to the maximum point 722.

[0061] Next, the specific unit 405 performs a geometric approximation process on the local maximum point 732, approximating the boundary point 731 associated with the local maximum point 732 using circular geometric information (approximation circle 733) with the local maximum point 732 as its center.

[0062] Similarly, the specific unit 405 performs a geometric approximation process on the local maximum point 722 and approximates the boundary point 721 associated with the local maximum point 722 using circular geometric information (approximation circle 723) with the local maximum point 722 as its center.

[0063] The identification unit 405 identifies information regarding the region of a first object, including the maximum point 722, boundary point 721, and approximate circle 723 (an example of first information), and notifies the statistical processing unit 132 and the display unit 406. The identification unit 405 identifies information regarding the region of a second object, including the maximum point 732, boundary point 731, and approximate circle 733 (an example of second information), and notifies the statistical processing unit 132 and the display unit 406.

[0064] Thus, according to the identification unit 405, even if some of the boundary points of objects are not extracted due to objects being adjacent or adjacent objects overlapping, domain knowledge is utilized to approximate them using approximate circles. This makes it possible to appropriately identify information about the region of each object. In addition, users can obtain the added benefit of being able to easily calculate feature quantities that indicate the size of each object.

[0065] Although the above explanation did not describe the details of the geometric approximation process, the geometric approximation process is performed, for example, by following these steps: - The position of the center point of the approximation circle is made to coincide with the position of the local maximum point. - The error between the position coordinates of each point constituting the approximation circle and the position coordinates of the corresponding boundary points is calculated while changing the diameter in predetermined increments. - The diameter that minimizes the error is identified.

[0066] In this case, the position coordinates of each point constituting the approximate circle (for example, the relative position coordinates of each point with respect to the position coordinate of the center point of the approximate circle) may be prepared in advance as circular geometric information for each diameter. Alternatively, the position coordinates of each point constituting the approximate circle (for example, the relative position coordinates of each point with respect to the position coordinate of the center point of the approximate circle) may be calculated using a function capable of representing an approximate circle of any diameter.

[0067] Furthermore, the method of approximating the shape is not limited to fitting a predetermined shape, such as an approximate circle, while scaling it. For example, approximation may be performed by generating a predetermined shape by interpolating between multiple boundary points using spline interpolation with a function such as CubicSpline.

[0068] (4) Specific Examples of Display Processing by the Display Unit Next, a specific example of display processing by the display unit 406 will be described. Figure 8 is a diagram showing a specific example of display processing by the display unit. As described above, the display unit 406 acquires the target image and displays the information regarding the region of each object (maximum point, boundary point, approximate circle) notified by the identification unit 405 by superimposing it. The display image 810 in Figure 8 is an example of a display image in which information regarding the region of each object (maximum point, boundary point, approximate circle) is superimposed on the target image.

[0069] As shown in the display image 810, the display unit 406 displays information about the object's region (maximum point, boundary point, approximate circle) in the same color for each coin. For example, the maximum point 812, boundary point 811, and approximate circle 813 are displayed in the same color (light blue). Similarly, the maximum point 822, boundary point 821, and approximate circle 823 are displayed in the same color (pink).

[0070] On the other hand, between adjacent coins, the display unit 406 displays information about the object's region (maximum point, boundary point, approximate circle) in different colors (for example, light blue and pink). Furthermore, if two objects overlap, the display unit 406 displays both the approximate circle included in the region information of one object (for example, approximate circle 813) and the approximate circle included in the region information of the other object (for example, approximate circle 823) for the overlapping portion.

[0071] Displaying coins in this way has the added benefit of allowing users to properly understand the area of ​​each coin, even when adjacent coins are touching or overlapping.

[0072] As is clear from the explanation of each of the above specific examples, in realizing an algorithm that utilizes domain knowledge, by configuring the domain identification unit 131 with the functional configuration shown in Figure 4, the user does not need to perform any prior work. Specifically: - When executing the processing of each part (binarization processing, distance map generation processing, local maximum point extraction processing, boundary point extraction processing, linking processing, geometric approximation processing), it is no longer necessary to set parameters for each target image. - It is no longer necessary to generate training data or pre-train a model based on the generated training data.

[0073] Therefore, the area identification unit 131 can reduce the user's workload when identifying the area of ​​each object.

[0074] Furthermore, in order to realize an algorithm that leverages domain knowledge, by configuring the domain identification unit 131 as shown in Figure 4, the following can be achieved: - Even if some boundary points are not extracted due to adjacent objects or overlapping adjacent objects, the domain of each object will be approximated using an approximate circle. - In the area where objects overlap, both approximate circles will be displayed. This allows for the appropriate identification of the domain of each object. In addition, users can gain the added benefit of being able to appropriately grasp the domain of each object and making it easier to calculate feature quantities that indicate the size of each object.

[0075] <Flow of area identification processing by area identification unit> Next, the flow of area identification processing by area identification unit 131 will be explained. Figure 9A is a flowchart showing the flow of area identification processing by area identification unit.

[0076] In step S901, the binarization unit 401 acquires the target image.

[0077] In step S902, the binarization unit 401 performs a binarization process on the acquired target image to generate a binary image.

[0078] In step S903, the distance conversion unit 402 performs a distance map generation process on the generated binary image and generates a distance map.

[0079] In step S904, the maximal point extraction unit 404 performs a maximal point extraction process on the generated distance map and extracts the maximal points.

[0080] In step S905, the boundary point extraction unit 403 performs boundary point extraction processing on the generated binary image or target image and extracts boundary points.

[0081] In step S906, the identification unit 405 acquires multiple local maximums and multiple boundary points, and performs linking and geometric approximation processing. Details of the linking and geometric approximation processing in step S906 will be described later.

[0082] In step S907, the identification unit 405 notifies the statistical processing unit 132 of information regarding the area of ​​each object.

[0083] In step S908, the display unit 406 performs a display process and displays information about the area of ​​each object superimposed on the target image.

[0084] <Details of Linking Process and Geometric Approximation Process (Step S906)> Next, the details of the linking process and geometric approximation process (Step S906) in Figure 9A will be explained. Figure 9B is a flowchart showing the flow of the linking process and geometric approximation process by the specific unit.

[0085] In step S911, the identification unit 405 reads out the acquired boundary points and the acquired local maximum points.

[0086] In step S912, the identification unit 405 calculates the distance between one of the multiple boundary points read and each of the local maximum points read.

[0087] In step S913, the identification unit 405 determines the local maximum point to which a boundary point will be linked by determining the local maximum point that minimizes the distance between the boundary point and each of the read-out local maximum points.

[0088] In step S914, the identification unit 405 determines whether or not it has determined the local maximum point to be linked for all of the multiple boundary points that have been read. If there are any boundary points for which the local maximum point to be linked has not been determined in step S914 (i.e., the answer is NO in step S914), the process returns to step S912. On the other hand, if the local maximum point to be linked has been determined for all boundary points in step S914 (i.e., the answer is YES in step S914), the process proceeds to step S915.

[0089] In step S915, the identification unit 405 removes local maximums where the number of linked boundary points is less than or equal to a predetermined number.

[0090] In step S916, the identification unit 405 identifies information about the object's region by approximating the multiple boundary points associated with each local maximum point using circular geometric information (approximate circle) with each local maximum point as its center point, and then returns to step S907.

[0091] <Processing Results of Region Identification by Region Identification Unit> Next, the processing results of the region identification process by the region identification unit 131 will be explained in comparison with a comparative example. Figure 10 is a diagram showing an example of the processing results of the region identification process by the region identification unit. Of these, Figure 10(a) shows an example of a target image. Figure 10(b) shows, as a comparative example, how the regions of each object are identified using the watershed method. Figure 10(c) shows an example of the processing results of the region identification process by the region identification unit 131.

[0092] Comparing Figure 10(b) and Figure 10(c), the following can be observed: • In the comparative example's processing result, there are several instances where the object's region is incorrectly identified between adjacent objects (see reference numerals 1001-1004, etc.). • In the comparative example's processing result, at adjacent locations, only the region of one object is displayed. • In the case of the region identification processing result by the region identification unit 131, the object's region is appropriately identified (there are no instances where the object's region is incorrectly identified). • In the case of the region identification processing result by the region identification unit 131, at adjacent locations, the regions of both objects are displayed.

[0093] Thus, the region identification unit 131 can appropriately identify the regions of objects even between adjacent objects, compared to the case using the watershed method. Furthermore, the region identification unit 131 can output processing results that are more suitable for calculating feature quantities indicating the size of objects, compared to the case using the watershed method.

[0094] <Summary> As is clear from the above explanation, the image processing apparatus 130 according to the first embodiment: - Binarizes a target image containing multiple circular objects into regions of objects and regions other than the object regions to generate a binary image. - Performs distance transformation on the binary image to generate a distance map. - Extracts multiple local maxima from the distance map. - Extracts multiple boundary points located at the boundary between the object regions and regions other than the object regions in the binary image or target image. - Determines the multiple boundary points associated with each local maxima by calculating the distance from each of the multiple boundary points to each of the multiple local maxima. - Identifies the region of the object containing the local maxima by approximating the multiple boundary points associated with the local maxima using circular geometric information (approximate circle) with the local maxima as the center point.

[0095] Thus, in the first embodiment, we focus on the fact that the object has a spherical shape (circular shape in a two-dimensional target image), and use an algorithm that leverages domain knowledge that the shape of each object is circular to identify the region of each object. As a result, according to the first embodiment, in a target image containing multiple densely packed objects, the region of each object can be appropriately identified, and the workload of the user when identifying the region of each object can be reduced.

[0096] In other words, according to the first embodiment, an algorithm can be provided that can appropriately identify the region of each object included in the target image, and that can reduce the workload on the user.

[0097] [Second Embodiment] In the first embodiment described above, the case in which each object in the target image has a circular shape was explained. However, the shape of each object in the target image is not limited to a circular shape, but may be a predetermined shape. The predetermined shape here may be, for example, a shape that is point-symmetric with respect to a center point. If the shape is point-symmetric, the region of the object can be identified by arranging the figure such that the center point of the figure having that shape coincides with the local maximum point, and then approximating the figure having that shape by scaling it or rotating it around the center point. The point-symmetric figures here include, for example, elliptical figures and rectangular figures.

[0098] In the first embodiment described above, an image containing multiple coins of equal size was used as the target image 510. However, the same effect can be obtained even if the image contains multiple coins of different sizes, according to the region identification unit 131.

[0099] Furthermore, in the first embodiment described above, a case was explained in which the statistical processing unit 132 outputs an aggregated result obtained by aggregating feature quantities indicating the size of each object, but the items to be output as an aggregated result are not limited to this.

[0100] It should be noted that the present invention is not limited to the configurations shown in the above embodiments, including combinations with other elements. These aspects can be modified without departing from the spirit of the present invention and can be appropriately determined according to their application.

[0101] This application claims priority based on Japanese Patent Application No. 2024-219746, filed on 16 December 2024, which is incorporated herein by reference to the entire contents of the said Japanese Patent Application.

[0102] 100: Process control system 130: Image processing device 131: Region identification unit 132: Statistical processing unit 300: Target image 401: Binarization unit 402: Distance conversion unit 403: Boundary point extraction unit 404: Maximum point extraction unit 405: Identification unit 406: Display unit 530: Distance map 810: Display image

Claims

1. An image processing apparatus comprising: a binarization unit that binarizes a target image containing multiple objects into regions of objects and regions other than the regions of objects to generate a binary image; a distance transformation unit that transforms the binary image by distance to generate a distance map; a maximal point extraction unit that extracts multiple maximal points from the distance map; a boundary point extraction unit that extracts multiple boundary points located at the boundary between the regions of objects and regions other than the regions of objects in the target image or the binary image; and an identification unit that determines multiple boundary points associated with each of the multiple boundary points by calculating the distance from each of the multiple maximal points to each of the multiple maximal points, and identifies the region of an object containing the maximal points by approximating the multiple boundary points associated with the maximal points using geometric information of a predetermined shape with the maximal points as the center point.

2. The image processing apparatus according to claim 1, wherein the identifying unit identifies the region of an object with respect to the maximum points among the plurality of maximum points, excluding the maximum points for which the number of associated boundary points is less than or equal to a predetermined number.

3. A display unit that superimposes information relating to the region of an object, which includes the local maximum point, a plurality of boundary points associated with the local maximum point, and geometric information of a predetermined shape that approximates the plurality of boundary points associated with the local maximum point, onto the target image, as described in claim 1 or 2.

4. The image processing apparatus according to claim 3, wherein the display unit, when displaying first information relating to the region of a first object and second information relating to the region of a second object for two adjacent objects, displays them using different colors, and when the two objects overlap, displays both the graphic information included in the first information and the graphic information included in the second information for the overlapping portion.

5. The image processing apparatus according to any one of claims 1 to 4, wherein the graphic information of the predetermined shape includes the position coordinates of each point constituting the graphic of the predetermined shape, and is calculated by a function representing the graphic of the predetermined shape.

6. The image processing apparatus according to claim 5, wherein the figure of the predetermined shape is a figure that is point-symmetric with respect to a center point.

7. The image processing apparatus according to claim 5 or 6, wherein the predetermined shape is a circular or elliptical figure.

8. An image processing apparatus according to any one of claims 1 to 7, comprising: a statistical processing unit that calculates a feature quantity indicating the size of the region of an object identified by the identification unit, and statistically processes the feature quantity for a plurality of objects included in the target image.

9. An image processing method performed by a computer, comprising the steps of: binarizing a target image containing multiple objects into regions of objects and regions other than the regions of objects to generate a binary image; performing distance transformation on the binary image to generate a distance map; extracting multiple local maxima from the distance map; extracting multiple boundary points located at the boundary between the regions of objects and regions other than the regions of objects in the target image or the binary image; and determining multiple boundary points associated with each of the multiple boundary points by calculating the distance from each of the multiple local maxima to each of the multiple local maxima, and identifying the region of an object containing the local maxima by approximating the multiple boundary points associated with the local maxima using geometric information of a predetermined shape with the local maxima as the center point.

10. An image processing program for causing a computer to perform the following steps:

10. Binarizing a target image containing multiple objects into regions of objects and regions other than the regions of objects to generate a binary image; 20. Distancing the binary image to generate a distance map; 31. Extracting multiple local maxima from the distance map; 42. Extracting multiple boundary points located at the boundary between the regions of objects and regions other than the regions of objects in the target image or the binary image; 53. Determining multiple boundary points associated with each of the multiple boundary points by calculating the distance from each of the multiple local maxima, and 64. Identifying the region of an object containing the local maxima by approximating the multiple boundary points associated with the local maxima using geometric information of a predetermined shape with the local maxima as the center point.

Images