Image processing device, image processing method, and program

Abstract

It accurately detects the axes of fibers contained in an image. [Solution] The image processing device comprises: an image acquisition unit that acquires a target image in which multiple fibers are imaged; a first region division unit that generates a first region division result indicating the region in which each fiber included in the target image is detected using a region division model for each individual; a second region division unit that generates a second region division result indicating the region in which the fibers included in the target image are imaged; a thinning unit that generates a first thinning result and a second thinning result by thinning the first region division result and the second region division result, respectively; and a result output unit that outputs an analysis result indicating the axis of the fibers included in the target image by extracting pixels that are included in both the first thinning result and the second thinning result and excluding pixels that are included in only one of the first thinning result and the second thinning result.

Description

Technical Field

[0006] ,

[0003] ,

[0007] ,

[0001] The present disclosure relates to an image processing apparatus, an image processing method, and a program.

Background Art

[0002] A technique for detecting the axes of respective fibers based on an image in which a plurality of fibers are imaged is known. For example, Patent Document 1 discloses a line segment detection device that binarizes an image obtained by imaging a composite material in order to measure the fiber length in the composite material in which fillers are combined, thins the binarized image, applies a plurality of line segments to a set of feature points extracted from the thinned image, and integrates the applied plurality of line segments into one line segment.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the prior art has a problem that the accuracy of detecting the axes of individual fibers from an image in which a plurality of fibers are imaged is low. For example, when a plurality of fibers intersect and are interrupted, the axes of the fibers may be distorted. Also, when a plurality of fibers are dense, it is difficult to individually detect the axes of each fiber.

[0005] One aspect of the present disclosure aims to accurately detect the axes of fibers included in an image.

Means for Solving the Problems

[0006] The present disclosure includes the following configuration.

[0007] <1> An image acquisition unit configured to acquire a target image in which a plurality of fibers are imaged, A first region segmentation unit is configured to generate a first region segmentation result that indicates the region in which each of the fibers included in the target image is detected, using a region segmentation model for each individual; A second region division unit is configured to generate a second region division result indicating the region in which the fibers included in the target image are captured, A thinning unit is configured to generate a first thinning result and a second thinning result obtained by thinning the first region division result and the second region division result, respectively. A result output unit is configured to output an analysis result indicating the axis of the fibers contained in the target image by extracting pixels that are included in both the first thinning result and the second thinning result, and excluding pixels that are included in only one of the first thinning result and the second thinning result, An image processing device equipped with the following features.

[0008] <2> the above <1> The image processing apparatus described above, The analysis results exclude portions where the first thinning result and the second thinning result do not match in the region where the fibers are densely clustered during imaging. Image processing device.

[0009] <3> the above <1> The image processing apparatus described above, The analysis results include only the axes of the fibers that are not superimposed on other fibers. Image processing device.

[0010] <4> the above <1> The image processing apparatus described above, The analysis results are output as axes capable of measuring the length or thickness of the fibers. Image processing device.

[0011] <5> the above <1> The image processing apparatus described above, Pixels that are included in both the first and second thinning results are obtained by the logical AND of the first and second thinning results. Image processing device.

[0012] <6> Computers Procedure for acquiring a target image by imaging multiple fibers, A procedure for generating a first region segmentation result that shows the region in which each of the fibers included in the target image is detected, using a region segmentation model for each individual, A procedure for generating a second region segmentation result that shows the region in which the fibers included in the target image are captured, A procedure for generating a first thinning result and a second thinning result obtained by thinning the first region division result and the second region division result, respectively, A procedure for outputting an analysis result indicating the axis of the fibers contained in the target image by extracting pixels that are included in both the first thinning result and the second thinning result, and excluding pixels that are included in only one of the first thinning result and the second thinning result, An image processing method that performs this task.

[0013] <7> On the computer, Procedure for acquiring a target image by imaging multiple fibers, A procedure for generating a first region segmentation result that shows the region in which each of the fibers included in the target image is detected, using a region segmentation model for each individual, A procedure for generating a second region segmentation result that shows the region in which the fibers included in the target image are captured, A procedure for generating a first thinning result and a second thinning result obtained by thinning the first region division result and the second region division result, respectively, A procedure for outputting an analysis result indicating the axis of the fibers contained in the target image by extracting pixels that are included in both the first thinning result and the second thinning result, and excluding pixels that are included in only one of the first thinning result and the second thinning result, A program to execute. [Effects of the Invention]

[0014] According to one aspect of this disclosure, the axes of fibers contained in an image can be detected with high accuracy.

Brief Description of the Drawings

[0015] [Figure 1] FIG. 1 is a block diagram showing an example of the overall configuration of an image analysis system. [Figure 2] FIG. 2 is a block diagram showing an example of the hardware configuration of a computer. [Figure 3] FIG. 3 is a block diagram showing an example of the functional configuration of an image analysis system. [Figure 4] FIG. 4 is a flowchart showing an example of an image analysis method. [Figure 5] FIG. 5 is a diagram showing an example of the thinning result for each fiber. [Figure 6] FIG. 6 is a diagram showing an example of the thinning result of the fiber region. [Figure 7] FIG. 7 is a diagram showing an example of an analysis result.

Embodiments for Carrying Out the Invention

[0016] Hereinafter, each embodiment of the present disclosure will be described with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

[0017] [Embodiment] One embodiment of the present disclosure is an image analysis system that analyzes the shape of an object from an image obtained by imaging the object. Hereinafter, the image to be analyzed is also referred to as a "target image". The target image in the present embodiment is an image obtained by imaging a state in which a large number of fibers are dispersed on a metal surface. The target image may capture a state in which a plurality of fibers intersect or a state in which a large number of fibers are densely packed. The fibers in the present embodiment are, for example, carbon fibers. However, the fibers to be analyzed are not limited to carbon fibers, and may be fibers made of any material.

[0018] In this embodiment, in order to analyze the shape of fibers captured in an image, the region in which the fibers are captured is detected from the image. In the task of detecting a desired object from an image, segmentation using a machine learning model may be performed. Segmentation using a machine learning model includes instance segmentation and semantic segmentation based on deep learning.

[0019] Instance segmentation is the task of detecting individual objects contained in an image. In instance segmentation, the bounding boxes (rectangular regions) in which objects are captured may be detected individually from the image, or individual objects may be detected and identified on a pixel-by-pixel basis. The object detection results from instance segmentation may include, for each detected object, two-dimensional data (mask score) indicating the region in which the object was captured, a score indicating object-likeness, a mask obtained by binarizing the mask score using a threshold, and information indicating the bounding box. The size of the bounding box may be adjustable. The size of the mask for each individual may also be adjustable by setting a score threshold. By specifying a larger bounding box size or setting a lower score threshold, the region of each individual can be identified more broadly. Furthermore, the object detection results may include a confidence level for the object detection results.

[0020] A machine learning model that performs instance segmentation is an example of an "individual-level region segmentation model." Furthermore, the mask score included in the object detection results obtained through instance segmentation is an example of an "individual-level region segmentation result."

[0021] Machine learning models that perform instance segmentation can include Mask R-CNN (Mask Region-Based Convolutional Neural Networks) or YOLACT (You Only Look At CoefficienTs). Details of Mask R-CNN are disclosed in Reference 1.

[0022] [Reference 1] Kaiming He, Georgia Gkioxari, Piotr Dollar, Ross Girshick, "Mask R-CNN", Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017.

[0023] Semantic segmentation is the task of dividing an image into regions that represent one or more types (classes). Semantic segmentation predicts a class label for each unit (e.g., a pixel) in the image. The region segmentation results from semantic segmentation can include two-dimensional data (semantic scores) containing the class labels corresponding to each pixel in the image. Furthermore, the region segmentation results can include a confidence score for the segmentation.

[0024] A machine learning model that performs semantic segmentation is an example of a "domain segmentation model by type." Furthermore, the semantic score included in the domain segmentation results obtained by semantic segmentation is also an example of a "domain segmentation result by type."

[0025] For machine learning models used for semantic segmentation, U-Net or DeepLab can be used. Details of U-Net are disclosed in Reference 2. Details of DeepLab (DeepLabv3) are disclosed in Reference 3.

[0026] [Reference 2] Olaf Ronneberger, Philipp Fischer, and Thomas Brox, "U-Net: Convolutional Networks for Biomedical Image Segmentation", Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2015. [Reference 3] LC Chen, G. Papandreou, F. Schroff, and H. Adam, "Rethinking Atrous Convolution for Semantic Image Segmentation", European Conference on Computer Vision (ECCV), 2017.

[0027] To detect the axis of each fiber from an image containing multiple fibers, one approach is to thin the fiber regions recognized in the image. However, thinning the object detection results obtained by instance segmentation can distort the fiber axes when multiple fibers intersect and break. Furthermore, thinning the region division results obtained by semantic segmentation makes it impossible to individually detect the axis of each fiber when many fibers are densely packed together.

[0028] The image analysis system in this embodiment aims to accurately detect the axes of fibers contained in an image. The image analysis system thins the region division results for each object and the region division results for each type, and outputs analysis results indicating the axes of each fiber based on each thinning result.

[0029] In one aspect, this embodiment allows for the accurate detection of the axis of fibers contained in an image. For example, the image analysis system can correctly detect the axis even of intersecting fibers. Furthermore, the image analysis system can detect the axis of only fibers whose contours do not overlap with those of other fibers. As a result, this embodiment makes it possible to accurately measure the length or thickness of fibers captured in an image.

[0030] <Overall configuration of the image analysis system> The overall configuration of the image analysis system in this embodiment will be described with reference to Figure 1. Figure 1 is a block diagram showing an example of the overall configuration of the image analysis system in this embodiment.

[0031] As shown in Figure 1, the image analysis system 1 in this embodiment includes an image acquisition device 10, an image processing device 20, and a user terminal 30. The image acquisition device 10, the image processing device 20, and the user terminal 30 are connected via a communication network N1 such as a LAN (Local Area Network) or the Internet, enabling data communication.

[0032] The image acquisition device 10 is an optical instrument that acquires the target image to be analyzed. The image acquisition device 10 may be a digital camera that takes still images or a video camera that takes moving images. Depending on the size of the object to be detected, an optical microscope, scanning electron microscope (SEM), transmission electron microscope (TEM), etc., can be used as the image acquisition device 10. In addition, the image acquisition device 10 may be an information processing device such as a personal computer connected to various cameras, or an inspection device equipped with various cameras.

[0033] The image processing device 20 is an information processing device such as a personal computer, workstation, or server that analyzes the shape of an object from a target image acquired by the image acquisition device 10. The image processing device 20 receives the target image from the user terminal 30. The image processing device 20 analyzes the shape of the object from the acquired target image and transmits an output image showing the analysis result to the user terminal 30.

[0034] The user terminal 30 is an information processing terminal such as a personal computer, tablet, or smartphone operated by the user. The user terminal 30 acquires a target image from the image acquisition device 10 in response to the user's operation and transmits it to the image processing device 20. The user terminal 30 receives an output image showing the analysis results from the image processing device 20 and outputs it to the user.

[0035] Note that the overall configuration of the image analysis system 1 shown in Figure 1 is just one example, and various system configurations are possible depending on the application and purpose. For example, one or more of the image acquisition device 10, image processing device 20, and user terminal 30 may be included in the image analysis system 1 in multiple units. For example, the image processing device 20 may be implemented by multiple computers, or it may be implemented as a cloud computing service. The classification of devices such as the image acquisition device 10, image processing device 20, and user terminal 30 shown in Figure 1 is just one example.

[0036] <Hardware configuration of the image analysis system> The hardware configuration of the image analysis system 1 in this embodiment will be described with reference to Figure 2.

[0037] Computer Hardware Configuration In this embodiment, the image acquisition device 10, the image processing device 20, and the user terminal 30 are implemented, for example, by a computer. Figure 2 is a block diagram showing an example of the hardware configuration of the computer 500 in this embodiment.

[0038] As shown in Figure 2, the computer 500 includes a CPU (Central Processing Unit) 501, ROM (Read Only Memory) 502, RAM (Random Access Memory) 503, HDD (Hard Disk Drive) 504, input device 505, display device 506, communication interface 507, and external interface 508. The CPU 501, ROM 502, and RAM 503 form what is known as a computer. Each piece of hardware in the computer 500 is interconnected via a bus line 509. The input device 505 and display device 506 may also be used by connecting them to the external interface 508.

[0039] The CPU 501 is a processing unit that reads programs and data from a storage device such as the ROM 502 or HDD 504 onto the RAM 503 and executes processing, thereby realizing the overall control and functions of the computer 500. The computer 500 may have a GPU (Graphics Processing Unit) in addition to or instead of the CPU 501.

[0040] ROM502 is an example of non-volatile semiconductor memory (storage device) that can retain programs and data even when the power is turned off. ROM502 functions as the main memory, storing various programs and data necessary for the CPU501 to execute the programs installed on HDD504. Specifically, ROM502 stores boot programs such as BIOS (Basic Input / Output System) and EFI (Extensible Firmware Interface) that are executed when the computer 500 starts up, as well as OS (Operating System) settings, network settings, and other data.

[0041] RAM503 is an example of volatile semiconductor memory (storage device) whose programs and data are erased when the power is turned off. RAM503 includes, for example, DRAM (Dynamic Random Access Memory) and SRAM (Static Random Access Memory). RAM503 provides a working area that is expanded when various programs installed on HDD504 are executed by CPU501.

[0042] HDD504 is an example of a non-volatile storage device that stores programs and data. The programs and data stored in HDD504 include the operating system (OS), which is the basic software that controls the entire computer 500, and applications that provide various functions on the OS. Note that computer 500 may use a storage device that uses flash memory as its storage medium (e.g., SSD: Solid State Drive) instead of HDD504.

[0043] The input device 505 includes a touch panel used by the user to input various signals, operation keys and buttons, a keyboard and mouse, and a microphone for inputting sound data such as voice.

[0044] The display device 506 consists of a display such as a liquid crystal or organic EL (Electro-Luminescence) that displays a screen, and a speaker that outputs sound data such as audio.

[0045] Communication I / F 507 is an interface that connects to a communication network and allows computer 500 to perform data communication.

[0046] External I / F 508 is an interface for external devices. Examples of external devices include the drive device 510.

[0047] The drive device 510 is a device for setting the recording medium 511. The recording medium 511 here includes media that record information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, and magneto-optical disks. The recording medium 511 may also include semiconductor memory that records information electrically, such as ROMs and flash memory. This allows the computer 500 to read and / or write to the recording medium 511 via the external I / F 508.

[0048] The various programs to be installed on the HDD 504 are installed, for example, when the distributed recording medium 511 is set in a drive device 510 connected to an external I / F 508, and the various programs recorded on the recording medium 511 are read by the drive device 510. Alternatively, the various programs to be installed on the HDD 504 may be installed by downloading them via the communication I / F 507 from a communication network or another network different from the communication network.

[0049] <Functional Configuration of the Image Analysis System> The functional configuration of the image analysis system in this embodiment will be described with reference to Figure 3. Figure 3 is a block diagram showing an example of the functional configuration of the image analysis system 1.

[0050] ≪Functional Configuration of Image Acquisition Device≫ As shown in Figure 3, the image acquisition device 10 in this embodiment includes an imaging control unit 101 and an image storage unit 102.

[0051] The imaging control unit 101 is implemented by a camera connected to the external I / F 508 shown in Figure 2. The image storage unit 102 is implemented by the HDD 504 shown in Figure 2.

[0052] The imaging control unit 101 adjusts the camera's imaging conditions (angle of view, magnification, etc.) to enable imaging of multiple fibers and captures a target image. The imaging control unit 101 may capture a target image in response to user operation, or it may capture a target image when predetermined conditions are met. These predetermined conditions may include, for example, when an object to be inspected is transported to a predetermined position in the inspection device. The imaging control unit 101 may capture a still image, or it may capture a video and extract an image in which multiple fibers are captured.

[0053] The image storage unit 102 stores the target image captured by the imaging control unit 101. The image storage unit 102 may also store information related to the target image in association with the target image. This information may include, for example, the date and time of capture, information indicating the object being captured, and the imaging conditions.

[0054] ≪Functional Configuration of Image Processing Devices≫ As shown in Figure 3, the image processing apparatus 20 in this embodiment includes a model storage unit 200, an image acquisition unit 201, an object detection unit 202 (an example of a first region division unit), a region division unit 203 (an example of a second region division unit), a thinning unit 204, an expansion unit 205, a result integration unit 206, and a result output unit 207.

[0055] The model storage unit 200 is implemented by the HDD 504 shown in Figure 2. The image acquisition unit 201, object detection unit 202, region division unit 203, thinning unit 204, expansion unit 205, result integration unit 206, and result output unit 207 are implemented by a process in which the program, which is loaded from the HDD 504 shown in Figure 2 onto the RAM 503, is executed by the CPU 501.

[0056] The model storage unit 200 stores a trained object detection model and a trained region segmentation model.

[0057] The object detection model is a machine learning model that performs instance segmentation. The object detection model takes an image as input and outputs object detection results that individually detect objects contained in the image. The object detection results include region segmentation results for each individual object, recognizing the region in which each object is imaged. In this embodiment, the object detection model used is, as an example, Mask R-CNN. The object detection model is an example of a region segmentation model for each individual object.

[0058] The object detection model in this embodiment is trained to individually detect fibers captured in an input image. The object detection model is trained using training data in which training labels are assigned to images of a metal surface on which multiple fibers are dispersed. The training labels include information indicating the extent of the region in which each fiber is captured and identification information to identify the fiber.

[0059] A segmentation model is a machine learning model that performs semantic segmentation. The segmentation model takes an image as input and outputs segmentation results for each type, recognizing the region in the image where the desired object is captured. In this embodiment, DeepLabv3 is used as an example of the segmentation model. The segmentation model is an example of a segmentation model for each type.

[0060] The region segmentation model in this embodiment is trained to divide an input image into regions where any fiber is imaged and other regions. The region segmentation model is trained using training data in which training labels are assigned to images of a metal surface on which multiple fibers are dispersed. The training labels include information indicating the extent of the regions where the fibers are imaged.

[0061] The image acquisition unit 201 acquires the target image by receiving it from the user terminal 30. The image acquisition unit 201 may also acquire the target image from the image acquisition device 10 in response to a request from the user terminal 30.

[0062] The object detection unit 202 uses a trained object detection model read from the model storage unit 200 to detect individual fibers contained in the target image acquired by the image acquisition unit 201, thereby generating an object detection result (an example of a first region segmentation result) corresponding to each fiber contained in the target image.

[0063] The region segmentation unit 203 uses a trained region segmentation model read from the model storage unit 200 to recognize the region in which fibers are imaged (hereinafter also referred to as the "fiber region") from the target image acquired by the image acquisition unit 201, thereby generating a region segmentation result indicating the fiber region (an example of a second region segmentation result).

[0064] The thinning unit 204 thins the object detection results generated by the object detection unit 202 and the region division results generated by the region division unit 203. Specifically, the thinning unit 204 binarizes each object detection result corresponding to each fiber and then thins each of the binarized object detection results. The thinning unit 204 also superimposes the thinned object detection results corresponding to each fiber to generate a thinned result for each fiber (an example of a first thinned result). Furthermore, the thinning unit 204 binarizes the region division results indicating the fiber regions and then thins the binarized region division results. The region division results indicating the fiber regions after thinning are called the fiber region thinned results (an example of a second thinned result).

[0065] The expansion section 205 expands the thinning results for each fiber and the thinning results for the fiber region, respectively, generated by the thinning section 204.

[0066] The result integration unit 206 generates analysis results indicating the axes of fibers included in the target image based on the thinning results for each fiber after expansion and the thinning results of the fiber region after expansion generated by the expansion unit 205. The result integration unit 206 may also generate analysis results by calculating the logical AND of the thinning results for each fiber after expansion and the thinning results of the fiber region after expansion.

[0067] The result output unit 207 transmits an output image showing the analysis results generated by the result integration unit 206 to the user terminal 30.

[0068] ≪Functional Configuration of User Terminal 30≫ As shown in Figure 3, the user terminal 30 in this embodiment includes an image transmission unit 301 and a result display unit 302.

[0069] The image transmission unit 301 and the result display unit 302 are realized by a process in which a program loaded from the HDD 504 shown in Figure 2 onto the RAM 503 is executed by the CPU 501.

[0070] The image transmission unit 301 acquires a target image from the image acquisition device 10 in response to user operation. The image transmission unit 301 transmits the target image acquired from the image acquisition device 10 to the image processing device 20.

[0071] The result display unit 302 receives the output image from the image processing device 20. The result display unit 302 displays the received output image on the display device 506.

[0072] <Processing procedure of the image analysis system> The processing procedure of the image processing method performed by the image analysis system 1 in this embodiment will be described with reference to Figures 4 to 7. Figure 4 is a flowchart showing an example of the image processing method.

[0073] In step S1, the imaging control unit 101 of the image acquisition device 10 adjusts the camera's field of view so that multiple fibers are captured, and captures the target image. Next, the imaging control unit 101 stores the captured target image in the image storage unit 102.

[0074] In step S2, the image transmission unit 301 of the user terminal 30 transmits a request to the image acquisition device 10 to acquire a target image in response to the user's operation. The image acquisition device 10 reads the target image stored in the image storage unit 102 in response to the acquisition request received from the user terminal 30 and transmits it to the user terminal 30. The image transmission unit 301 transmits the target image received from the image acquisition device 10 to the image processing device 20.

[0075] In step S3, the image acquisition unit 201 of the image processing device 20 receives the target image from the user terminal 30. Next, the image acquisition unit 201 sends the received target image to the object detection unit 202 and the region division unit 203.

[0076] In step S4, the object detection unit 202 of the image processing device 20 receives the target image from the image acquisition unit 201. Next, the object detection unit 202 reads the trained object detection model from the model storage unit 200.

[0077] Next, the object detection unit 202 inputs the target image into the trained object detection model. The object detection model detects individual fibers contained in the target image and outputs an object detection result corresponding to each fiber. Thus, the object detection unit 202 obtains the object detection result corresponding to each fiber. The object detection unit 202 then sends the object detection result corresponding to each fiber to the thinning unit 204.

[0078] The object detection result for each fiber includes the detection result for each fiber detected from the target image, and the confidence level of that object detection result. The detection result for each fiber includes a mask score indicating the region in which the fiber was detected and a score indicating object-likeness. The detection result for each fiber may also include information indicating a mask obtained by binarizing the mask score using a threshold and a bounding box.

[0079] The object detection unit 202 may reject the detection results for each fiber if the score indicating object similarity (0 or more and 1 or less) is below a predetermined threshold. The threshold can be set arbitrarily, but for example, it can be set to 0.3.

[0080] In step S5, the region segmentation unit 203 of the image processing device 20 receives the target image from the image acquisition unit 201. Next, the region segmentation unit 203 reads the trained region segmentation model from the model storage unit 200.

[0081] Next, the region segmentation unit 203 inputs the target image into the trained region segmentation model. The region segmentation model recognizes the fiber regions from the target image and outputs region segmentation results indicating the fiber regions. Thus, the region segmentation unit 203 obtains region segmentation results indicating the fiber regions. The region segmentation unit 203 then sends the region segmentation results indicating the fiber regions to the thinning unit 204.

[0082] The region segmentation result indicating a fiber region includes a semantic score indicating the fiber region and a confidence level for that region segmentation result.

[0083] In step S6, the thinning unit 204 of the image processing device 20 receives object detection results corresponding to each fiber from the object detection unit 202. The thinning unit 204 also receives region division results indicating fiber regions from the region division unit 203.

[0084] Next, the thinning unit 204 thins the object detection results and region division results indicating the fiber regions corresponding to each fiber. This allows the thinning unit 204 to obtain thinning results for each fiber and thinning results for the fiber regions. The thinning unit 204 then sends these thinning results to the expansion unit 205.

[0085] Specifically, the thinning unit 204 thins the object detection results as follows: First, the thinning unit 204 binarizes each object detection result corresponding to each fiber. Next, the thinning unit 204 thins each of the binarized object detection results. Subsequently, the thinning unit 204 superimposes the thinned object detection results corresponding to each fiber to generate a thinned result for each fiber.

[0086] Furthermore, the thinning unit 204 thins the region division results as follows: First, the thinning unit 204 binarizes the region division results indicating the fiber region. Next, the thinning unit 204 thins the binarized region division results.

[0087] Figure 5 shows an example of the thinning results for each fiber in this embodiment. As shown in Figure 5, in the thinning results 900 for each fiber, the axis of each fiber is shown individually in the densely packed region 901. However, since the fibers detected in region 901 overlap in some areas, it is difficult to accurately measure the length and thickness of the fibers.

[0088] Figure 6 shows an example of the thinning result of the fiber region in this embodiment. As shown in Figure 6, in the thinning result 910 of the fiber region, the region 911 where fibers are densely packed shows the axis of a region consisting of multiple overlapping fibers. This axis does not coincide with the axis of any of the fibers imaged in region 911.

[0089] Comparing Figure 5 and Figure 6, it can be seen that in target images where parts of multiple fibers overlap, the fiber axes detected differ between the individual-specific region recognition model and the species-specific region recognition model.

[0090] Let's return to Figure 4 for explanation. In step S7, the expansion unit 205 of the image processing device 20 receives the thinning results for each fiber and the thinning results for the fiber region from the thinning unit 204. Next, the expansion unit 205 expands the thinning results for each fiber and the thinning results for the fiber region, respectively. Then, it sends the expanded thinning results for each fiber and the expanded thinning results for the fiber region to the result integration unit 206.

[0091] In step S8, the result integration unit 206 of the image processing device 20 receives the thinning results for each fiber after expansion and the thinning results for the fiber region after expansion from the expansion unit 205. Next, the result integration unit 206 generates analysis results based on the thinning results for each fiber and the thinning results for the fiber region. Then, the result integration unit 206 sends the analysis results to the result output unit 207.

[0092] For example, the results integration unit 206 may generate the analysis results by calculating the logical AND of the thinning results for each fiber and the thinning results for the fiber region. The analysis results will show the parts that are common to the thinning results for each fiber and the thinning results for the fiber region.

[0093] Figure 7 shows an example of the analysis results in this embodiment. As shown in Figure 7, in the analysis result 920, the fiber axes are not shown in any of the densely packed fiber regions 921. This is because the axes shown in region 901 in Figure 5 and the axes shown in region 911 in Figure 6 do not have any common parts. The analysis result 920 shows only the fiber axes that were detected in common in both the individual region recognition model and the species-specific region recognition model. Therefore, the fiber axes shown in the analysis result 920 do not overlap with other fibers and represent the shape of fibers that can be measured accurately in terms of length or thickness.

[0094] Let's return to Figure 4 for explanation. In step S9, the result output unit 207 of the image processing device 20 receives the analysis results from the result integration unit 206. Next, the result output unit 207 generates an output image showing the analysis results of the fiber axes captured in the target image, based on the analysis results. Then, the result output unit 207 transmits the output image to the user terminal 30. The analysis results may include the number and distribution of fiber length and fiber diameter for the fibers in which axes were detected, and may also include calculated statistical values ​​such as mean, median and standard deviation.

[0095] In step S10, the result display unit 302 of the user terminal 30 receives the output image from the image processing device 20. The result display unit 302 then displays the received output image on the display device 506.

[0096] <Effects of the Embodiment> In this embodiment, the image processing device 20 outputs an analysis result indicating the axis of the fibers contained in the target image, based on a first thinning result obtained by thinning the region in which each fiber contained in the target image is detected, and a second thinning result obtained by thinning the region in which the fibers contained in the target image are captured. By using thinning results based on different types of region division results, it is possible to detect the axis even of intersecting fibers, and to detect the axis only of fibers whose contours do not overlap with other fibers. In one aspect, according to this embodiment, the axis of the fibers contained in the image can be detected with high accuracy.

[0097] In this embodiment, the image processing device 20 may detect the fiber axis by performing a logical AND operation between the first thinning result and the second thinning result. The image processing device 20 may also expand the first thinning result and the second thinning result. According to this embodiment, since the axis shown in common to both thinning results is detected, it is possible to generate an analysis result showing only the axes that were correctly detected.

[0098] <First variation> In the above embodiment, an example was described in which the region division unit 203 divides the target image into fiber regions and other regions using a region division model for each type. The region division unit 203 may also divide the target image into fiber regions and other regions by binarization. For example, if the color of each fiber captured in the target image and the background color captured in the target image are uniform, the image can be divided into fiber regions and other regions by binarization. In this case, the thinning unit 204 can omit the binarization that is performed before thinning.

[0099] <Second variation> In the first modified example described above, an example was explained in which the region division unit 203 divides the target image into a fiber region and other regions by binarization. The region division unit 203 may also divide the target image into a fiber region and other regions by binarizing the first region division result. Similar to the first modified example, the image can be divided into a fiber region and other regions by binarization. In this case, the thinning unit 204 can omit the binarization that is performed before thinning.

[0100] Performing region segmentation using binarization can reduce computational complexity. For example, it eliminates the need to train a separate region segmentation model for each type of fiber. It also reduces the computational cost of recognizing fiber regions from the target image and binarizing the region segmentation results.

[0101] [supplement] Each of the embodiments described above can be implemented by one or more processing circuits. Hereinafter, "processing circuit" as used herein includes processors programmed to execute each function by software, such as CPUs (Central Processing Units) or GPUs (Graphics Processing Units) implemented by electronic circuits, as well as devices such as ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), FPGAs (Field Programmable Gate Arrays), and conventional circuit modules designed to execute each of the functions described above.

[0102] While embodiments of the present disclosure have been described in detail above, the embodiments disclosed herein are illustrative and not restrictive in all respects. The embodiments can be modified and improved in various ways without departing from the scope and spirit of the appended claims. The features described in the above embodiments can be combined in any way that is not inconsistent with other configurations.

[0103] This application claims priority to Japanese Patent Application No. 2023-150364, filed with the Japan Patent Office on 15 September 2023, which is incorporated herein by reference to its entire contents. [Explanation of Symbols]

[0104] 1: Image analysis system 10: Image acquisition device 101: Imaging Control Unit 102: Image storage unit 20: Image Processing Device 200: Model Memory Unit 201: Image acquisition unit 202: Object detection unit 203: Area division part 204: Thinning section 205: Expansion part 206: Results Integration Department 207: Result Output Section 30: User terminal 301: Image transmission unit 302:Result display section

Claims

1. An image acquisition unit configured to acquire a target image of multiple fibers, A first region segmentation unit is configured to generate a first region segmentation result that indicates the region in which each of the fibers included in the target image is detected, using a region segmentation model for each individual; A second region division unit is configured to generate a second region division result indicating the region in which the fibers included in the target image are captured, A thinning unit is configured to generate a first thinning result and a second thinning result obtained by thinning the first region division result and the second region division result, respectively. A result output unit is configured to output an analysis result indicating the axis of the fibers contained in the target image by extracting pixels that are included in both the first thinning result and the second thinning result, and excluding pixels that are included in only one of the first thinning result and the second thinning result, An image processing device equipped with the following features.

2. An image processing apparatus according to claim 1, The analysis results exclude portions where the first thinning result and the second thinning result do not match in the region where the fibers are densely clustered during imaging. Image processing device.

3. An image processing apparatus according to claim 1, The analysis results include only the axes of the fibers that are not superimposed on other fibers. Image processing device.

4. An image processing apparatus according to claim 1, The analysis results are output as axes capable of measuring the length or thickness of the fibers. Image processing device.

5. An image processing apparatus according to claim 1, Pixels that are included in both the first and second thinning results are obtained by the logical AND of the first and second thinning results. Image processing device.

6. Computers Procedure for acquiring a target image by imaging multiple fibers, A procedure for generating a first region segmentation result that shows the region in which each of the fibers included in the target image is detected, using a region segmentation model for each individual, A procedure for generating a second region segmentation result that shows the region in which the fibers included in the target image are captured, A procedure for generating a first thinning result and a second thinning result obtained by thinning the first region division result and the second region division result, respectively, A procedure for outputting an analysis result indicating the axis of the fibers contained in the target image by extracting pixels that are included in both the first thinning result and the second thinning result, and excluding pixels that are included in only one of the first thinning result and the second thinning result, An image processing method that performs this task.

7. On the computer, Procedure for acquiring a target image by imaging multiple fibers, A procedure for generating a first region segmentation result that shows the region in which each of the fibers included in the target image is detected, using a region segmentation model for each individual, A procedure for generating a second region segmentation result that shows the region in which the fibers included in the target image are captured, A procedure for generating a first thinning result and a second thinning result obtained by thinning the first region division result and the second region division result, respectively, A procedure for outputting an analysis result indicating the axis of the fibers contained in the target image by extracting pixels that are included in both the first thinning result and the second thinning result, and excluding pixels that are included in only one of the first thinning result and the second thinning result, A program to execute.

Images