Image analysis device, image analysis method, and image analysis program

Abstract

To make it possible to analyze orientation information of an object with high accuracy.SOLUTION: An image analysis device includes: a binarization processing part for receiving an image obtained by photographing materials including an object and creating a binarized image obtained by separating and binarizing the object in the image and a background; a contour extraction part for extracting a contour of the object from the binarized image; a calculation part for calculating an orientation in each pixel of the extracted contour; an enclosing contour detection part for detecting an enclosing contour enclosing the periphery of the pixel in the contour about each pixel inside the object by a prescribed detection method; and an orientation calculation part for calculating orientation information of the pixel by using the orientation of the enclosing contour about each pixel inside the object.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to an image analysis device, an image analysis method, and an image analysis program. [Background technology]

[0002] Conventionally, there are techniques related to image detection of an object.

[0003] For example, there is a technology for obtaining indexes related to particle angles (see Patent Document 1). There is also a technology related to image analysis that can calculate the feature amount of a three-dimensional structure with a small amount of calculation and display the distribution in a specific direction in an easy-to-understand manner (see Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-135824 [Patent Document 2] Japanese Patent Application Publication No. 2018-91765 Summary of the Invention [Problem to be solved by the invention]

[0005] Here, image analysis is being carried out on composite materials in which fillers are added to resins, etc. When the aspect ratio of the filler is small (for example, 10 or less), the contribution of the orientation of the filler edges increases with respect to the overall orientation of the filler. In assessing filler orientation, attention is often focused on the main orientation of the filler (the axial direction for fibers, or the normal direction to the main surface for plate-like particles), so the orientation at the edges is considered an error.

[0006] When using the technology of Patent Document 1, for example, image analysis of an object such as a fiber is envisioned. In image analysis of an object such as a fiber, the contour of the object is extracted from a 2D slice image extracted from a 3D image, and the orientation is calculated based on this. Because fibers often have a large aspect ratio, the influence of the edges on the overall orientation is small. However, as mentioned above, in the case of fillers with a small aspect ratio, the influence of the edges cannot be ignored and large errors occur, so a method of calculating the orientation of the filler from the orientation of the contour alone is not appropriate. Similarly, when using the technology of Patent Document 2, the orientation and length of the object are calculated based on the extracted contour, and the influence of the edges is a concern.

[0007] The present disclosure has been made in consideration of the above circumstances, and aims to provide an image analysis device, an image analysis method, and an image analysis program that can analyze orientation information of an object with high accuracy. [Means for solving the problem]

[0008] The image analysis device disclosed herein includes a binarization processing unit that receives an image of a material containing an object, separates the object from the background in the image, and creates a binarized image; a contour extraction unit that extracts the contour of the object from the binarized image; a calculation unit that calculates the orientation of each pixel of the extracted contour; a surrounding contour detection unit that detects, for each pixel inside the object, a surrounding contour that surrounds the pixel from the contour using a predetermined detection method; and an orientation calculation unit that calculates orientation information for each pixel inside the object using the orientation of the surrounding contour.

[0009] Furthermore, in the image analysis device of the present disclosure, the enclosing contour detection unit may, in detecting the enclosing contour, as a first detection process, group contour pixels according to their angle from a target object pixel, and further create subgroups within each group according to their distance from the target pixel, and select, from among the distance subgroups within each angle group, contour pixels of the subgroup closest to the target pixel as candidates for the enclosing contour, and as a second detection process, select, based on a direction vector from the target object pixel to each contour pixel and the contour orientation vector, contour pixels for which the angle or dot product between the two vectors satisfies a predetermined condition, and detect contour pixels selected as candidates in both the first detection process and the second detection process as the enclosing contour.

[0010] In the image analysis device of the present disclosure, the enclosing contour detection unit may set the detection radius in detecting the enclosing contour to a range that is at least a predetermined multiple of a predetermined length of the object end.

[0011] In addition, in the image analysis device of the present disclosure, the orientation calculation unit may calculate, for each pixel inside the object, the orientation vector of the pixel by calculating a predetermined tensor using the number of pixels in the enclosing contour and the orientation vector of the enclosing contour. [Effects of the Invention]

[0012] According to the image analysis device, image analysis method, and image analysis program of the present disclosure, it is possible to analyze the orientation information of an object with high accuracy. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram schematically illustrating the orientation of the normal direction of the surface of an object. [Figure 2] FIG. 2 is a block diagram showing the configuration of the image analysis device of this embodiment. [Figure 3] FIG. 3 is a block diagram showing the hardware configuration of the image analysis device. [Figure 4]FIG. 4 shows an example of the original image and the binarized image to be analyzed. [Figure 5] FIG. 5 shows an example of extracting the contour of a filler region separated by binarization processing, in which the contour of a white portion is extracted. [Figure 6] FIG. 6 shows an example of calculating the alignment vectors of each pixel of the contour, and the alignment vectors of the contour of the portion enclosed by the dotted line are calculated. [Figure 7] FIG. 7 shows an example of detecting the surrounding contour of a pixel of interest. [Figure 8] FIG. 8 is a diagram illustrating an example of grouping. [Figure 9] FIG. 9 shows the distribution of the obtained alignment vectors and the alignment angles calculated based on the alignment vectors. [Figure 10] FIG. 10 shows a comparative example of the orientation analysis results obtained by a general method. [Figure 11] FIG. 11 is a flowchart showing the flow of image analysis processing by the image analysis device. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, an embodiment of the present disclosure will be described in detail with reference to the drawings. First, an outline of the embodiment will be described.

[0015] Composite materials containing fillers, such as resins, are being analyzed. Composite materials containing fillers can impart or improve various properties by changing the type or combination of the matrix and filler. For example, hexagonal boron nitride particles (h-BN particles) are plate-shaped fillers consisting of a planar structure composed of a hexagonal network of nitrogen and boron atoms. Because of their high thermal conductivity in the planar direction, they are often used in applications such as heat dissipation materials. However, h-BN particles exhibit anisotropic thermal conductivity, resulting in low thermal conductivity in the layering direction. Therefore, for h-BN composite materials to exhibit good thermal conductivity, it is desirable for the h-BN particles to be oriented in the desired direction. For example, in the case of a heat sink, it is desirable to control the alignment of the h-BN particles so that their planar direction is perpendicular to the thickness direction of the heat sink.

[0016] Given the above background, controlling and evaluating the filler orientation is important for producing materials with the desired properties. Therefore, evaluation of filler orientation is important, and filler orientation evaluation is generally performed by image analysis. Note that the filler to be analyzed is not limited to the h-BN particles mentioned above.

[0017] FIG. 1 is a schematic diagram showing the orientation of the normal direction of an object surface. (d1) is the normal direction of the main surface, and (d2) is the normal direction of the side surface. Considering a disk-shaped or rectangular filler, the normal vectors of the filler's main surface and side surface are perpendicular. Therefore, when the aspect ratio of the filler is small, the proportion of orientation of the side surface (edge) increases. For example, h-BN particles have good thermal conductivity in the in-plane direction, so evaluation of the in-plane orientation (i.e., the orientation of the main surface) is important, and it is desirable to exclude the orientation of the side surface. However, because conventional orientation analysis does not distinguish between the main surface and the side surface, large orientation errors occur in fillers with low aspect ratios. Therefore, the image analysis of this embodiment uses a method that enables more accurate evaluation of filler orientation by considering the surrounding contour of the pixel of interest (the object pixel of interest). Because the surrounding contour contains many of the contours of the filler's main surface, it is possible to calculate an orientation close to the main orientation even for pixels near the edge.

[0018] Fig. 2 is a block diagram showing the configuration of the image analysis device of this embodiment. As shown in Fig. 2, the image analysis device 100 includes a binarization processing unit 110, a contour extraction unit 112, a calculation unit 114, a surrounding contour detection unit 116, and an orientation calculation unit 118.

[0019] Fig. 3 is a block diagram showing the hardware configuration of the image analyzing device 100. As shown in Fig. 3, the image analyzing device 100 has a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a display interface (I / F) 16, and a communication interface (I / F) 17. Each component is connected to each other via a bus 19 so as to be able to communicate with each other.

[0020] The CPU 11 is a central processing unit that executes various programs and controls each part. That is, the CPU 11 reads a program from the ROM 12 or the storage 14 and executes the program using the RAM 13 as a work area. The CPU 11 controls each of the above components and performs various arithmetic processing in accordance with the program stored in the ROM 12 or the storage 14. In this embodiment, the ROM 12 or the storage 14 stores an image analysis program.

[0021] The ROM 12 stores various programs and various data. The RAM 13 temporarily stores programs or data as a working area. The storage 14 is configured with a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) and stores various programs including the operating system and various data.

[0022] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to perform various inputs.

[0023] The display interface 16 is, for example, a liquid crystal display, and displays various information. The display interface 16 may function as the input unit 15 by adopting a touch panel system.

[0024] The communication interface 17 is an interface for communicating with other devices such as terminals, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).

[0025] Next, a description will be given of each functional configuration of the image analysis device 100. Each functional configuration is realized by the CPU 11 reading out an image analysis program stored in the ROM 12 or storage 14, expanding it in the RAM 13, and executing it.

[0026] In this embodiment, the image analysis device 100 receives an image of a cross section of a material containing a filler as an object and performs image analysis. The application procedure of this embodiment will be described using a two-dimensional image of a cross section of a composite material with a resin base material and h-BN particles as a filler, taken by a scanning electron microscope.

[0027] FIG. 4 shows examples of an original image and a binarized image to be analyzed. (a1) is the original image, and (a2) is a binarized image obtained by performing the binarization process described below on the original image. The number of pixels in the image is 145 x 124, and the thickness of the L-shaped filler in the center is approximately 20 pixels. Note that the filler in the composite material is an example of an object in the present disclosure, and the resin portion corresponds to the background.

[0028] The binarization processing unit 110 separates the object from the background in the image and creates a binarized image. The binarization processing unit 110 performs binarization based on brightness values ​​to separate the resin and filler. Here, to achieve a clearer separation, an averaging filter is applied to the original image, and binarization is performed based on the calculated average brightness value (a2). Note that small white areas remain in the resin area and small black areas remain in the filler area, but these can be removed according to their area (or volume).

[0029] The contour extraction unit 112 extracts the contours of objects from the binarized image. FIG. 5 shows an example of contour extraction for a filler region separated by binarization processing, where the contours of the white parts are extracted. Here, contour extraction is performed based on the gradient of brightness values. After calculating the gradient of brightness values ​​for each pixel, pixels with a gradient magnitude equal to or greater than a threshold are extracted as contours. Note that in FIG. 5, processing is performed to remove contours with small areas, so the contours of small black regions within the filler are not extracted.

[0030] The calculation unit 114 calculates the orientation of each pixel of the extracted contour. Here, an orientation vector is calculated for each pixel of the extracted contour. In this embodiment, the orientation of the contour refers to the local normal vector at each position of the contour. FIG. 6 shows an example of calculating the orientation vector of each pixel of the contour, where the orientation vector of the contour in the portion surrounded by the dotted line is calculated. Here, the brightness value gradient calculated in the previous step is simply used as the orientation vector. Therefore, the orientation vector of the contour points inward (towards the inside of the filler).

[0031] For each pixel inside the object, the enclosing contour detection unit 116 detects the enclosing contour that surrounds the pixel from among the contours using a predetermined detection method. The detection method uses the following first and second detection processes to select candidates for the enclosing contour, and detects the contour pixels selected as candidates as the enclosing contour. Note that the inside of the object refers to the area inside the contour extracted by the contour extraction unit 112.

[0032] Figure 7 shows an example of detecting the surrounding contour for a pixel of interest. Note that if the surrounding contour is detected simply based on the distance from the filler pixel of interest, extraneous contours (contours of different fillers) will also be detected. To alleviate this, the following two processes are applied, and contour pixels that simultaneously satisfy these conditions are considered to be the surrounding contour. Note that the search radius for detecting contour pixels is set to 50 pixels. When detecting the surrounding contour, the detection radius should be set to a range at least twice the specified length of the object edge (side length, plate thickness, and diameter).

[0033] In the first detection process, contour pixels are first grouped according to their angle from the pixel of interest. FIG. 8 is a diagram showing an example of grouping. In the example shown, the pixels are divided into eight angle groups every 45°. Next, subgroups are created for the contour pixels in each angle group according to their distance from the pixel of interest. In each angle group, the contour pixels in the subgroup closest to the pixel of interest are designated as candidates for the enclosing contour. This is to select contours that are close in distance as the enclosing contour. In this way, in the first detection process, the enclosing contour detection unit 116 creates subgroups within each group according to their distance from the pixel of interest, and selects the contour pixels in the subgroup closest to the pixel of interest from among the distance subgroups in each angle group as candidates for the enclosing contour.

[0034] The second detection process uses pixel vectors. First, a direction vector from the filler pixel of interest toward each contour pixel is calculated. The dot product of this direction vector and the contour orientation vector calculated by the calculation unit 114 is calculated, and if the value is negative, the pixel is designated as a candidate for the enclosing contour. In this way, in the second detection process, the enclosing contour detection unit 116 selects contour pixels that satisfy a predetermined condition (for example, the dot product value is negative) as candidates for the enclosing contour based on the angle between the direction vector from the pixel of interest toward each contour pixel and the contour orientation vector.

[0035] The enclosing contour detection unit 116 detects candidate contour pixels that simultaneously satisfy the selection criteria of the first detection process and the second detection process for each pixel inside the object, as an enclosing contour.

[0036] The orientation calculation unit 118 calculates orientation information for each pixel inside the object using the orientation of the surrounding contour. Here, for each pixel of the filler, an orientation vector is calculated based on the orientation vector of the surrounding contour of the pixel. An example of a method for calculating an orientation vector is shown below. First, the following second-order tensor A is calculated for each pixel of the filler using the following equation (1).

number

[0037] However, n edge is the number of pixels in the enclosing contour, p edge is the orientation vector of the enclosing contour. The eigenvalues ​​and eigenvectors of A are calculated, and the eigenvector corresponding to the maximum eigenvalue is set as the orientation vector of the filler pixel. Figure 9 shows the distribution of the obtained orientation vectors and the orientation angles calculated based on the orientation vectors. It can be seen that the orientation of each filler is calculated well. As described above, the orientation calculation unit 118 calculates the orientation vector of each pixel inside the object by calculating a predetermined tensor using the number of pixels in the enclosing contour and the orientation vector of the enclosing contour.

[0038] For reference, a comparative example of the orientation analysis results by a general method is shown in Figure 10. In this method, first, the orientation tensor A of the following equation (2) is calculated for each pixel of the filler.

number

[0039] Here, N is the number of pixels surrounding the pixel of interest. The number of surrounding pixels does not distinguish between filler, contours, and background. The surrounding area here is a rectangular area of ​​L pixels x L pixels centered on the pixel of interest. ￣I (￣ is a superscript) is the brightness value gradient, calculated based on the brightness value before binarization. The eigenvalues ​​and eigenvectors of A are found, and the eigenvector corresponding to the maximum eigenvalue is used as the orientation vector of the filler pixel.

[0040] As shown in Figure 10, when the surrounding area is changed, in two cases where the surrounding area is changed, (b1) L = 3 and (b2) L = 21, a generally good orientation is obtained by setting the surrounding area large (b2). However, in the center of the thick filler on the left and the edge area indicated by the dotted circle, an orientation different from the intended orientation of the method of this embodiment is calculated. Setting the surrounding area wider than in these cases further mitigates the influence of the edge, but in this case, it is susceptible to the influence of the orientation of nearby different fillers, which is not appropriate. On the other hand, the method of this embodiment can detect contours over a wide range while mitigating false detection of different fillers, as shown in Figure 7, and therefore calculates an intended orientation even at the edge, as shown in Figure 9.

[0041] Next, a description will be given of the operation of the image analysis device 100 according to this embodiment. Fig. 11 is a flowchart showing the flow of image analysis processing by the image analysis device 100. The CPU 11 reads out an image analysis program from the ROM 12 or storage 14, expands it into the RAM 13, and executes it, thereby performing the image analysis processing.

[0042] In step S100, the CPU 11 functions as the binarization processing unit 110 to separate an object from a background in an image and to create a binarized image.

[0043] In step S102, the CPU 11 functions as the contour extraction unit 112 to extract the contour of the object from the binarized image.

[0044] In step S104, the CPU 11, functioning as the calculation unit 114, calculates the orientation of each pixel of the extracted contour.

[0045] In step S106, the CPU 11, functioning as the surrounding contour detection unit 116, groups the contour pixels according to the angle from the pixel of interest.

[0046] In step S108, the CPU 11, functioning as the enclosing contour detection unit 116, creates subgroups for contour pixels in each angle group according to the distance from the pixel of interest, and selects the subgroup closest to the pixel of interest as a candidate for the enclosing contour.

[0047] In step S110, the CPU 11, as the surrounding contour detection unit 116, selects contour pixels whose inner product value satisfies a predetermined condition based on the angle between the direction vector from the pixel of interest to each pixel of the contour and the orientation vector of the contour as candidates for the surrounding contour.

[0048] In step S112, the CPU 11 functions as the enclosing contour detection unit 116 to detect, for each pixel inside the object, candidate contour pixels that simultaneously satisfy the selection criteria of the first detection process and the second detection process, as enclosing contours.

[0049] In step S114, the orientation calculation unit 118 of the CPU 11 calculates the orientation information of each pixel inside the object using the orientation of the surrounding contour, using the above formula (1).

[0050] As described above, the image analyzing device according to this embodiment can analyze the orientation information of an object with high accuracy.

[0051] The present disclosure is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the spirit and scope of the present invention.

[0052] In the above embodiment, the image analysis process executed by the CPU after reading the software (program) may be executed by various processors other than the CPU. Examples of such processors include programmable logic devices (PLDs) (such as field-programmable gate arrays (FPGAs)) whose circuit configuration can be changed after fabrication, application-specific integrated circuits (ASICs), graphics processing units (GPUs), and other dedicated electrical circuits that are processors with circuit configurations specifically designed to execute specific processes. The image analysis process may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). The hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor devices.

[0053] In the above embodiment, the image analysis processing program is pre-stored (installed) in a ROM or storage device, but the present invention is not limited to this. The program may be provided in a form recorded on a non-transitory recording medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The program may also be downloaded from an external device via a network. [Explanation of symbols]

[0054] 100 Image analysis device 110 Binarization processing unit 112 Contour extraction unit 114 Calculation Unit 116 Enclosing contour detection unit 118 Orientation calculation unit

Claims

1. a binarization processing unit that receives an image of a material including an object, separates the object from a background in the image, and binarizes the image; a contour extraction unit that extracts a contour of an object from the binarized image; a calculation unit for calculating an orientation of each pixel of the extracted contour; an enclosing contour detection unit that applies a first detection process and a second detection process as predetermined detection methods for detecting an enclosing contour that surrounds each pixel within the object from among the contours, and detects contour pixels selected as candidates in both the first detection process and the second detection process as the enclosing contour; an orientation calculation unit that calculates orientation information of each pixel inside the object using the orientation of the surrounding contour; Equipped with As the first detection process, the contour pixels are grouped according to an angle from a target object pixel, and further subgroups are created within each group according to a distance from the target pixel, and a contour pixel of a subgroup that is closest to the target pixel among distance subgroups within each angle group is selected as a candidate for the enclosing contour; As the second detection process, an image analysis device selects contour pixels whose inner product value satisfies predetermined conditions as candidates for the enclosing contour based on a direction vector from the object pixel of interest to each pixel of the contour and an orientation vector of the contour.

2. 2. The image analysis device according to claim 1, wherein the enclosing contour detection unit, in detecting the enclosing contour, sets a detection radius that defines a range for detecting each length of the target related to the side length, plate thickness, and diameter of the object end to a range that is at least twice the length of each of the targets at the object end.

3. 2. The image analysis device according to claim 1, wherein the orientation calculation unit calculates the orientation vector of each pixel of a filler by determining a tensor for each pixel of the filler based on the orientation vector of the enclosing contour of the pixel in a predetermined calculation using the number of pixels of the enclosing contour and the orientation vector of the enclosing contour for each pixel inside the object.

4. Accepting an image of a material including an object, separating the object from the background in the image, and creating a binarized image; Extracting an object contour from the binarized image; Calculating the orientation at each pixel of the extracted contour; applying a first detection process and a second detection process as predetermined detection methods for detecting a surrounding contour that surrounds each pixel within the object from among the contours, thereby detecting contour pixels selected as candidates in both the first detection process and the second detection process as the surrounding contour; calculating, for each pixel within the object, orientation information for that pixel using the orientation of the enclosing contour; An image analysis method in which a computer executes a process of: As the first detection process, the contour pixels are grouped according to an angle from a target object pixel, and further subgroups are created within each group according to a distance from the target pixel, and a contour pixel of a subgroup that is closest to the target pixel among distance subgroups within each angle group is selected as a candidate for the enclosing contour; As the second detection process, an image analysis method is provided in which contour pixels whose inner product value satisfies predetermined conditions are selected as candidates for the enclosing contour based on a direction vector pointing from the object pixel of interest to each pixel of the contour and the orientation vector of the contour.

5. Accepting an image of a material including an object, separating the object from the background in the image, and creating a binarized image; Extracting an object contour from the binarized image; Calculating the orientation at each pixel of the extracted contour; applying a first detection process and a second detection process as predetermined detection methods for detecting a surrounding contour that surrounds each pixel within the object from among the contours, thereby detecting contour pixels selected as candidates in both the first detection process and the second detection process as the surrounding contour; calculating, for each pixel within the object, orientation information for that pixel using the orientation of the enclosing contour; An image analysis program that causes a computer to execute the following process: As the first detection process, the contour pixels are grouped according to an angle from a target object pixel, and further subgroups are created within each group according to a distance from the target pixel, and a contour pixel of a subgroup that is closest to the target pixel among distance subgroups within each angle group is selected as a candidate for the enclosing contour; As the second detection process, an image analysis program selects contour pixels whose inner product value satisfies predetermined conditions as candidates for the enclosing contour based on a direction vector from the object pixel of interest to each pixel of the contour and the orientation vector of the contour.

Images