Image processing apparatus, image processing method, and program

Abstract

To provide an image processing apparatus, an image processing method, and a program capable of appropriately deriving the form of a root while reducing a computational load.SOLUTION: An image processing apparatus includes: an acquisition unit which acquires a three-dimensional transparent image which is obtained by imaging a soil block containing roots of crops with electromagnetic waves; an edge image generation unit which generates an edge image by filtering the transparent image; a kernel filtering unit which performs convolution operation, using a ring-shaped kernel filter, on a plurality of slice images which are two-dimensional images obtained by slicing the edge image on surfaces orthogonal to three-axis directions; a size filtering unit which performs size filtering on three three-dimensional images obtained by combining the slice images subjected to the convolution operation, in a three-dimensional manner; and an integration processing unit which combines the size-filtered three-dimensional images in the three-axis directions, and outputs a three-dimensional root system image that shows the presence of the roots.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] Conventionally, research has been conducted on a technology for capturing three-dimensional perspective images of soil blocks containing crop roots using an X-ray CT device or the like, and grasping the morphology of the roots through image analysis. Non-Patent Document 1 describes the extraction of linear fragments using a three-dimensional filter. Non-Patent Documents 2 and 3 describe the use of a trained model based on machine learning to perform image analysis. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] “An improved method for the segmentation of roots from X-ray computed tomography 3D images: Rootine v.2”, Plant Methods, Phalempin et al. (2021) [Non-Patent Document 2] “Three Dimensional Root CT Segmentation Using Multi-Resolution Encoder-Decoder Networks”, IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL 29, 6667-6679, Soltaninejad et al. (2020) [Non-Patent Document 3] “Transfer Learning from Synthetic Data Applied to Soil-Root Segmentation in X-Ray Tomography Images”, Journal of Imaging, 4, 65, Douarre et al. (2018) Summary of the Invention [Problem to be solved by the invention]

[0004] The technology described in Non-Patent Document 1 has a problem in that the computational load is high because a three-dimensional filter is used. The technologies described in Non-Patent Documents 2 and 3 have a problem in that the computational load is even higher because a trained model is used, and it is time-consuming to collect training data.

[0005] The present invention has been made in consideration of the above circumstances, and one of its objects is to provide an image processing device, an image processing method, and a program that can appropriately derive the root shape while reducing the computational load. [Means for solving the problem]

[0006] An image processing device according to an embodiment of the present invention includes an acquisition unit that acquires a three-dimensional perspective image obtained by imaging a soil block containing crop roots with electromagnetic waves; an edge image generation unit that generates an edge image by filtering the perspective image; a kernel filter processing unit that performs a convolution operation using a circular kernel filter on a plurality of slice images, which are two-dimensional images obtained by slicing the edge image through planes perpendicular to each of three axial directions; a size filter processing unit that performs size filter processing on three three-dimensional images obtained by three-dimensionally combining the plurality of slice images that have undergone the convolution operation; and an integration processing unit that combines the three-dimensional images that have undergone the size filter processing in the three axial directions to output a three-dimensional root system image that shows the presence of the roots. Effect of the Invention

[0007] According to each of the above aspects, it is possible to appropriately derive the root form while reducing the computational load. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of a usage environment of an image processing device 100. [Diagram 2] 1 is a diagram illustrating a configuration of an image processing device 100. [Diagram 3] 10 is a diagram for explaining the contents of processing performed by an edge image generating unit 120. FIG. [Figure 4] 10 is a diagram for explaining the contents of the processing performed by the kernel filter processor 130 and the size filter processor 140. FIG. [Diagram 5] FIG. 11 is a diagram illustrating an example of the contents of a kernel filter. [Figure 6] 10 is a diagram for explaining the contents of processing by an integration processing unit 150. FIG. [Figure 7] 13 is a diagram for explaining the content of processing by the quantification processing unit 160. FIG. [Figure 8] FIG. 13 is a diagram showing an example of how a skeleton image SC# appears when viewed transparently. [Figure 9] FIG. 1 shows a comparison of root length, dry mass, and diameter derived by the method of the present invention with those measured by manual censoring and root system analysis software (comparison method in the figure). [Figure 10] FIG. 13 is a configuration diagram of an image processing device 100A according to a modified example. [Figure 11] FIG. 13 is a diagram showing a process of generating a vector without branching. [Figure 12] FIG. 13 is a diagram showing how the root of an adjacent individual is eliminated by eliminating vectors that are not aligned in a straight line. [Figure 13] FIG. 13 shows images of the root system before and after the removal of roots from adjacent individuals. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of an image processing device, an image processing method, and a program according to the present invention will be described with reference to the drawings.

[0010] FIG. 1 is a diagram showing an example of an environment in which the image processing device 100 is used. The image processing device 100 acquires a three-dimensional perspective image obtained by imaging a soil block SB (cylindrical monolith) containing the roots of a crop with electromagnetic waves. The soil block SB is, for example, a soil containing the roots of a crop carved into a cylindrical shape using an agricultural tool or the like. The soil block SB is placed on a turntable TT and imaged by an X-ray CT device 1, and a perspective image reconstructed by a processing computer of the X-ray CT device 1 is provided to the image processing device 100. The perspective image is a pixel value based on the transmittance of X-rays associated with each pixel having three-dimensional coordinates, and may be referred to as voxel data. In the perspective image, the pixel value of the part where the roots of the crop are present is larger than the pixel value of the other part. To obtain this characteristic, a pixel value inversion process (a process of making the pixel value closer to zero as the pixel value becomes larger, and making the pixel value larger as the pixel value becomes closer to zero) may be performed.

[0011] FIG. 2 is a configuration diagram of the image processing device 100. The image processing device 100 includes, for example, an acquisition unit 110, an edge image generation unit 120, a kernel filter processing unit 130, a size filter processing unit 140, an integration processing unit 150, and a quantification processing unit 160. These components are realized by, for example, a processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by cooperation between software and hardware. The program may be stored in advance in a storage device (a storage device having a non-transient storage medium) such as an HDD (Hard Disk Drive) or a flash memory, or may be stored in a removable storage medium (a non-transient storage medium) such as a DVD or a CD-ROM, and installed in the storage device by mounting the storage medium in a drive device.

[0012] The acquisition unit 110 acquires a three-dimensional perspective image as described above. Hereinafter, the acquired perspective image is referred to as a perspective image CI. The three-dimensional perspective image has three-dimensional coordinates represented by the X, Y, and Z axes, and each side has the same length (i.e., it is a cube). However, the present invention is not limited to this, and the length of each side of the three-dimensional perspective image may be different for each axis.

[0013] The edge image generating unit 120 generates an edge image by performing a filter process on the perspective image CI. FIG. 3 is a diagram for explaining the contents of the process of the edge image generating unit 120. For example, the edge image generating unit 120 extracts edges by raster scanning an edge extraction filter for each of a plurality of planes Sxy(1) to Sxy(n) obtained by slicing the perspective image CI along a plane perpendicular to the Z-axis direction, that is, along the XY plane. n is the number of pixels on one side of the perspective image. The edge extraction filter is a filter that outputs a larger value as the difference in pixel value between adjacent pixels (for example, four pixels vertically and horizontally, or eight pixels including diagonals) increases. For example, the edge image generating unit 120 replaces the pixel value of a pixel whose output value is equal to or greater than a threshold value with 1, and the pixel value of a pixel whose output value is less than the threshold value with zero, and generates a three-dimensional edge image CI# by combining them again in three dimensions. By this process, segments of soil whose pixel values ​​do not change disappear, and segments of impurities such as roots and straw, cracks, air bubbles, etc. are extracted.

[0014] FIG. 4 is a diagram for explaining the contents of the processing of the kernel filter processing unit 130 and the size filter processing unit 140. The kernel filter processing unit 130 performs a convolution operation using an annular kernel filter on each of a plurality of slice images Sxy#(1) to Sxy#(n), Syz#(1) to Syz#(n), and Sxz#(1) to Sxz#(n), which are two-dimensional images obtained by slicing the edge image CI# along planes perpendicular to each of the three axial directions. The kernel filter processing unit 130 performs a convolution operation by raster scanning the kernel filter, binarizes the operation result using a threshold value (for example, replaces the result with 1 if the result is equal to or greater than the threshold value, and replaces the result with zero if the result is less than the threshold value), and integrates them in the slice direction to output three-dimensional images Ixy*, Iyz*, and Ixz*. FIG. 5 is a diagram showing an example of a kernel filter. The kernel filter includes a center portion CP to which a positive coefficient is set, an annular portion RP to which a negative coefficient is set, and an intermediate portion MP that exists between the center portion CP and the annular portion RP and to which a coefficient of zero is set. In other words, the kernel filter is a filter that outputs a larger value when the pixel value of the pixel is large and the pixel values ​​of the surrounding pixels are small. By applying this, the pixel values ​​of the part reflecting the roots of the crop, which extend at an angle close to perpendicular to the slice plane, are emphasized and output as a large value. On the other hand, the pixels of the part reflecting the impurities and cracks that are not roots and are composed of large segments that overlap with the ring part RP to which a negative coefficient is set are output as a small value.

[0015] Here, the diameter 2rC of the ring-shaped portion RP of the kernel filter (not a diameter in the strict sense because it is composed of pixels in a matrix, but the ring-shaped portion RP is set based on this diameter) will be described. First, the maximum diameter of the root of the target crop to be processed by the image processing device 100 is R. If this R is set so that it fits inside the ring-shaped portion RP, the center of the root can be captured suitably. However, the root does not necessarily exist perpendicular to the set plane (xy, yz, xz), and if the inclination with respect to the plane is θ, it is necessary to assume a range of θ=45 degrees to 90 degrees with respect to each plane (for the area of ​​θ<45 degrees, it appears in a form closer to perpendicular in other planes, so it is not necessary to assume it). For this reason, it is sufficient to set 2rC to a value equal to or greater than R×(1 / sin45°). Therefore, it is suitable to set the diameter of the ring-shaped portion RP to a value of about 1.4 times the maximum diameter R of the root of the target crop plus several pixels (for example, 2 to 4 pixels).

[0016] Returning to Fig. 4, the size filter processing unit 140 performs size filter processing on three three-dimensional images Ixy*, Iyz*, and Ixz* obtained by three-dimensionally combining multiple slice images that have been subjected to convolution operations. The size filter is a process in which, for a group of pixels in which adjacent pixels have a pixel value of 1, if the number of pixels is equal to or greater than a reference value, it is left as is, and if the number of pixels is less than the reference value, it is replaced with zero. By applying the size filter, small impurities that are not roots and segments such as air bubbles are removed.

[0017] The integration processing unit 150 combines the size-filtered three-dimensional images Ixy**, Iyz**, and Ixz** in the three-axis directions to output a three-dimensional root system image CI## indicating the presence of roots. FIG. 6 is a diagram for explaining the contents of the processing of the integration processing unit 150. For example, for each three-dimensional pixel, the integration processing unit 150 sets the pixel value to 1 if the pixel value of any of the three-dimensional images Ixy**, Iyz**, and Ixz** is 1, and sets the pixel value to zero if all pixel values ​​of the three-dimensional images Ixy**, Iyz**, and Ixz** are zero. That is, the integration processing unit 150 generates a three-dimensional root system image CI## by OR-combining the size-filtered three-dimensional images Ixy**, Iyz**, and Ixz**.

[0018] FIG. 7 is a diagram for explaining the contents of the processing of the quantification processing unit 160. The processing of the quantification processing unit 160 can be implemented using freeware such as ImageJFiji. The quantification processing unit 160 first converts the root system image CI## into a skeleton image SC represented by thin lines passing through the center of the roots by morphological processing. Next, the quantification processing unit 160 performs expansion processing on each thin line of the skeleton image SC until it becomes thicker than the maximum diameter R of the root of the target crop, and converts it into a root system image CI### to which diameter information has been added by performing an AND operation with the edge image CI#. The quantification processing unit 160 performs distance conversion on the root system image CI### and performs an AND operation with the skeleton image SC to generate a skeleton image SC# to which diameter information has been added. The skeleton image SC# is a skeleton image in which root diameter information is added to a skeleton image showing the length of the root and three-dimensional distribution information for each root, and this makes it possible to quantify the root system. FIG. 8 is a diagram showing an example of how the skeleton image SC# appears when viewed transparently. The function of the quantification processing unit 160 is not the core of the present invention, so a detailed description thereof will be omitted.

[0019] FIG. 9 is a diagram showing a comparison of the root length, dry mass, and diameter derived by the method of the present invention with those measured by manually sorting the values ​​and using root system analysis software (comparison method in the figure). In this comparison, the layers were divided into A layer (0-5 [cm]), B layer (5-10 [cm]), C layer (10-15 [cm]), and D layer (15-20 [cm]) based on the depth from the region, and the same sample was measured for each layer using the present invention and the comparison method, and the correspondence between the two methods was mapped. The black circle represents the A layer, the triangle represents the B layer, the square represents the C layer, "+" represents the D layer, and r represents the correlation coefficient. In the A layer, the roots are densely packed, so the values ​​of the two often differed, but a good correlation was observed below the B layer.

[0020] By using the root system image CI### with diameter information created by the method of the present invention, more complex root system traits can be calculated. For example, FIG. 8 is a root system image created by the method of the present invention, but the roots of adjacent individuals are mixed in. Therefore, the trait value of each individual cannot be measured. To measure the trait value of each individual, it is possible to convert it into a vector, which is information on how points are connected. The following describes an image processing device according to a modified example. FIG. 10 is a configuration diagram of an image processing device 100A according to a modified example.

[0021] The adjacent individual removal processing unit 170 first performs the above-mentioned conversion to vectors. The conversion to vectors can be implemented, for example, using freeware such as Python. The vectorized root system is divided into information for each root. Depending on the root system image, for example, a protruding protrusion may be vectorized, resulting in a vector with branched roots. To convert to a single root vector without branches, for example, it is sufficient to search for the longest node pair from each vector. The path between nodes can be extracted by, for example, a depth-first search or a breadth-first search to find the shortest path. FIG. 11 is a diagram showing the process of generating a vector without branches.

[0022] The root system of rice is a fibrous root, which extends radially from a certain origin to the lower part of the ground. Therefore, by providing the position information of the origin, it is possible to calculate the extension angle of the root from an unbranched vector. For example, a root vector generated from a certain origin should be aligned approximately in a straight line with the origin. For example, the adjacent individual removal processing unit 170 performs the relevant processing using the angle of the angle OAB when the origin is O, the end closer to the origin of the unbranched vector is A, and the end farther from the origin is B. When the points OAB are aligned in a straight line, the angle is 180°. Strictly speaking, since there is a measurement error, it is possible to remove the vector originating from the adjacent individual by judging whether the vector is generated from the origin within a range of, for example, 150° to 180°. FIG. 12 is a diagram showing how the roots of adjacent individuals are removed by removing vectors that are not aligned in a straight line. FIG. 13 is a diagram showing root system images before and after the removal of the roots of adjacent individuals.

[0023] By removing the adjacent individuals, the traits of the roots of the individual can be measured. The quantification processing unit 180 after removing the influence of adjacent individuals measures the trait value of each individual. For example, the quantification processing unit 180 after removing the influence of adjacent individuals calculates the angle between the plane XY and the line AB (if they do not intersect, it is set to 0°) when the plane of the earth's surface is the plane XY and the line AB is the straight line including both ends of each vector, and determines the root extension angle of the vector. Since multiple vectors may be generated from one root, it is not preferable to calculate the average extension angle of each vector. For example, the problem of multiple vectors being generated from one root can be avoided by calculating a weighted root extension angle weighted by the length of each vector. If there are n roots, the extension angle of the i-th root is Di, the length of the i-th root is Li, and the sum of the lengths of the roots from 1 to n is TL, the weighted root extension angle A is calculated by the following formula (1).

[0024]

number

[0025] According to the embodiment described above, it is possible to appropriately derive the shape of the root while reducing the computation load. First, since a simple convolution operation is used, it is less platform-dependent and can be implemented in various languages. In addition, since the convolution operation related to the kernel filter is performed in two dimensions, the demands on memory and machine power are low. Furthermore, by changing the size of the kernel, it is possible to change the shape (thickness) of the root to be extracted.

[0026] The present invention is easy to implement and does not require model learning, making it easy for research institutes and companies such as seed companies to use in the start-up stage to analyze root systems using X-ray CT images. In addition, since it uses edge extraction and filtering by kernel size, it can be used as is for clay soils without changing settings even if the region or land changes. In addition, since it can calculate the distribution of root system length and thickness in soil, it can provide information that is suitable for research institutes and companies that want to obtain such information.

[0027] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0028] 100 Image processing device 110 Acquisition Department 120 Edge image generation unit 130 Kernel filter processing unit 140 Size filter processing section 150 Integrated Processing Unit 160 Quantification Processing Section 170 Adjacent individual removal processing unit 180 Quantification processing unit after removing the influence of adjacent individuals

Claims

1. an acquisition unit that acquires a three-dimensional perspective image obtained by imaging a soil block including the roots of a crop using electromagnetic waves; an edge image generating unit that generates an edge image by performing a filter process on the perspective image; and a kernel filter processing unit that performs a convolution operation using a ring-shaped kernel filter on a plurality of slice images, which are two-dimensional images obtained by slicing the edge image with planes perpendicular to each of the three axial directions. a size filter processing unit that performs a size filter process on three three-dimensional images obtained by three-dimensionally combining the plurality of slice images that have been subjected to the convolution operation; an integration processing unit that combines the size-filtered three-dimensional images in three axial directions to output a three-dimensional root system image showing the presence of the roots; An image processing device comprising:

2. the kernel filter includes a central portion having a positive coefficient, an annular portion having a negative coefficient, and an intermediate portion located between the central portion and the annular portion and having a zero coefficient; 2. The image processing device according to claim 1.

3. The annular portion has a circular shape, and a diameter of the annular portion is set to a value obtained by adding a few pixels to 1.4 times the maximum diameter of the root of the target crop.

3. The image processing device according to claim 2.

4. The image processing device A process of acquiring a three-dimensional perspective image obtained by imaging the soil block containing the crop roots with electromagnetic waves; generating an edge image by filtering the perspective image; A process of performing a convolution operation using a ring-shaped kernel filter on a plurality of slice images, which are two-dimensional images obtained by slicing the edge image along planes perpendicular to each of the three axial directions; a process of performing a size filter process on three three-dimensional images obtained by three-dimensionally combining the plurality of slice images subjected to the convolution operation; a process of combining the size-filtered three-dimensional images in three axial directions to output a three-dimensional root system image showing the presence of the roots; An image processing method that performs

5. The processor of the image processing device A process of acquiring a three-dimensional perspective image obtained by imaging the soil block containing the crop roots with electromagnetic waves; generating an edge image by filtering the perspective image; A process of performing a convolution operation using a ring-shaped kernel filter on a plurality of slice images, which are two-dimensional images obtained by slicing the edge image along planes perpendicular to each of the three axial directions; a process of performing a size filter process on three three-dimensional images obtained by three-dimensionally combining the plurality of slice images subjected to the convolution operation; a process of combining the size-filtered three-dimensional images in three axial directions to output a three-dimensional root system image showing the presence of the roots; A program for executing the above.

Images