Particle size distribution estimation device, particle size distribution estimation method, and particle size distribution estimation program

Abstract

To provide a particle size distribution estimation device, a particle size distribution estimation method, and a particle size distribution estimation program for estimating a particle size distribution with high accuracy.SOLUTION: In the particle size distribution estimation device including a photographing device and an information processing device, the information processing device 20 includes a binarized image generation part 203 for generating a binarized image by performing binarization processing on a photographed image, a corrected binarized image generation part 204 for generating a corrected binarized image by performing opening processing on the binarized image, a boundary information generation part 205 for generating boundary information by performing boundary extraction processing on the corrected binarized image, a contour group information generation part 206 for generating contour group information indicating a contour group which is a group of contour candidates not satisfying a rejection condition among a group of contour candidates which are a group of closed curves included in the boundary by rejecting contour candidates satisfying the rejection condition from the contour candidate group, and a particle size distribution estimation part 207 for estimating the particle size distribution on the basis of the contour group information.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a particle size distribution estimation device, a particle size distribution estimation method, and a particle size distribution estimation program. [Background technology]

[0002] There is known a particle size distribution estimation device that estimates a particle size distribution, which is the distribution of particle sizes of a plurality of granular bodies (e.g., stones, etc.), based on a captured image, which is an image generated by capturing an image of a target area including a plurality of granular bodies at least partially overlapping one another. For example, a particle size distribution estimation device described in Patent Document 1 generates a binary image by performing a binarization process on the captured image, extracts the contours of each granular body based on the binary image, and estimates the particle size of each granular body based on the extracted contours. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-95644 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the plurality of granular bodies at least partially overlap each other. Furthermore, the plurality of granular bodies often have similar colors. Therefore, in a binary image, the entire group of granular bodies that are close to each other may be erroneously extracted as the outline of a single granular body. Therefore, the particle size distribution estimation device described above cannot extract the outline of each granular body with high accuracy, and as a result, there is a risk that the particle size distribution cannot be estimated with high accuracy.

[0005] One of the objects of the present invention is to estimate particle size distribution with high accuracy. [Means for solving the problem]

[0006] In one aspect, the particle size distribution estimation device includes a photographed image acquisition unit, a binary image generation unit, a corrected binary image generation unit, a boundary information generation unit, a contour group information generation unit, and a particle size distribution estimation unit.

[0007] The photographed image acquisition unit acquires a photographed image, which is an image generated by photographing a target area including a plurality of granular bodies at least partially overlapping each other. The binary image generation unit generates a binary image by performing a binary process on the captured image, converting the values ​​of each of the multiple pixels that make up the captured image into a first value or a second value based on a threshold value. The corrected binary image generating unit generates a corrected binary image by performing opening processing on the binary image, which includes erosion processing and dilation processing, which are filter processing using a kernel.

[0008] The boundary information generating unit generates boundary information representing the boundary by performing a boundary extraction process on the corrected binarized image to extract a boundary between an area having a first value and an area having a second value. The contour group information generation unit performs a rejection process on the boundary information to reject contour candidates that satisfy a predetermined rejection condition from a contour candidate group, which is a group of closed curves included in the boundary represented by the boundary information, thereby generating contour group information representing a contour candidate group that does not satisfy the rejection condition from the contour candidate group. The particle size distribution estimation unit estimates a particle size distribution, which is a distribution of particle sizes of the plurality of granular bodies, based on the contour group information.

[0009] In another aspect, a method for estimating particle size distribution includes: Acquire a captured image, which is an image generated by capturing an image of a target area including a plurality of granular bodies at least partially overlapping one another; performing a binarization process on the captured image to convert the values ​​of each of a plurality of pixels constituting the captured image into a first value or a second value based on a threshold value, thereby generating a binarized image; performing an opening process including erosion and dilation, which are filter processes using a kernel, on the binarized image to generate a corrected binarized image; performing a boundary extraction process on the corrected binarized image to extract a boundary between an area having a first value and an area having a second value, thereby generating boundary information representing the boundary; performing a rejection process on the boundary information to reject contour candidates that satisfy a predetermined rejection condition from among a contour candidate group that is a group of closed curves included in the boundary represented by the boundary information, thereby generating contour group information that represents contour candidates that do not satisfy the rejection condition from among the contour candidate group; A particle size distribution, which is a distribution of particle sizes of the plurality of granular bodies, is estimated based on the contour group information. This includes:

[0010] In another aspect, the particle size distribution estimation program comprises: Acquire a captured image, which is an image generated by capturing an image of a target area including a plurality of granular bodies at least partially overlapping one another; performing a binarization process on the captured image to convert the values ​​of each of a plurality of pixels constituting the captured image into a first value or a second value based on a threshold value, thereby generating a binarized image; performing an opening process including erosion and dilation, which are filter processes using a kernel, on the binarized image to generate a corrected binarized image; performing a boundary extraction process on the corrected binarized image to extract a boundary between an area having a first value and an area having a second value, thereby generating boundary information representing the boundary; performing a rejection process on the boundary information to reject contour candidates that satisfy a predetermined rejection condition from among a contour candidate group that is a group of closed curves included in the boundary represented by the boundary information, thereby generating contour group information that represents contour candidates that do not satisfy the rejection condition from among the contour candidate group; A particle size distribution, which is a distribution of particle sizes of the plurality of granular bodies, is estimated based on the contour group information. The computer is caused to perform a process including the steps of: [Effects of the Invention]

[0011] The particle size distribution can be estimated with high accuracy. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram illustrating a configuration of a particle size distribution estimation device according to a first embodiment. [Figure 2] FIG. 2 is a block diagram illustrating functions of the information processing apparatus according to the first embodiment. [Figure 3] 4 is a flowchart showing a process executed by the information processing apparatus of the first embodiment. [Figure 4] FIG. 2 is a block diagram showing the configuration of a particle size distribution estimation device according to a first modified example of the first embodiment. [Figure 5] 10 is a flowchart showing a process executed by an information processing device according to a first modified example of the first embodiment. [Figure 6] FIG. 10 is a block diagram showing the configuration of a particle size distribution estimation device according to a second modified example of the first embodiment. [Figure 7] 10 is a flowchart illustrating a part of a process executed by an information processing device according to a second modified example of the first embodiment. [Figure 8] 10 is a flowchart showing a process executed by an information processing apparatus according to a second embodiment. [Figure 9] 10 is a flowchart showing a process executed by an information processing device according to a first modified example of the second embodiment. [Figure 10] 10A and 10B are explanatory diagrams illustrating an example of a captured image and a binarized image after preprocessing. [Figure 11] 10A and 10B are explanatory diagrams illustrating an example of a binarized image before opening processing is performed and a corrected binarized image after opening processing is performed. [Figure 12] 10A and 10B are explanatory diagrams illustrating an example of a corrected binarized image before shape rejection processing is performed, and a corrected binarized image reflecting contour candidates rejected by the shape rejection processing. [Figure 13] FIG. 10 is an explanatory diagram illustrating an example of an image in which a group of contours represented by contour group information is drawn in solid lines on a captured image after preprocessing has been performed. [Figure 14] 1 is a graph showing an example of particle size distribution. [Figure 15] 10 is a flowchart illustrating a process executed by an information processing apparatus according to a third embodiment. [Figure 16] 10 is a flowchart showing a process executed by an information processing apparatus according to a fourth embodiment. [Figure 17] 13 is a flowchart illustrating a part of a process executed by an information processing apparatus according to a fifth embodiment. [Figure 18] 1A is an explanatory diagram showing an example of a line used to obtain a luminance change parameter, and FIG. 1B is an example of a graph showing the change in luminance with respect to pixel position on the line. [Figure 19] 10A and 10B are explanatory diagrams each showing an example of a line used to obtain a luminance change parameter. [Figure 20] 13 is a flowchart illustrating a part of a process executed by an information processing device according to a first modified example of the fifth embodiment. [Figure 21] 13 is a flowchart illustrating a part of a process executed by an information processing device according to a second modified example of the fifth embodiment. [Figure 22] 10 is a graph showing an example of a change in resistance force with respect to a moving distance of the bucket. [Figure 23] 13 is a flowchart illustrating a part of a process executed by an information processing device according to a third modified example of the fifth embodiment. [Figure 24] 13A and 13B are flowcharts illustrating a part of a process executed by an information processing device according to a fourth modified example of the fifth embodiment. [Figure 25] 13 is a flowchart illustrating a part of a process executed by an information processing apparatus according to a sixth embodiment. [Figure 26] 13 is a flowchart showing a process executed by an information processing apparatus according to a seventh embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of a particle size distribution estimation device, a particle size distribution estimation method, and a particle size distribution estimation program according to the present invention will be described with reference to FIGS.

[0014] First Embodiment (overview) The particle size distribution estimation device of the first embodiment includes a captured image acquisition unit, a binary image generation unit, a corrected binary image generation unit, a boundary information generation unit, an outline group information generation unit, and a particle size distribution estimation unit.

[0015] The photographed image acquisition unit acquires a photographed image, which is an image generated by photographing a target area including a plurality of granular bodies at least partially overlapping each other. The binary image generation unit generates a binary image by performing a binary process on the captured image, converting the values ​​of each of the multiple pixels that make up the captured image into a first value or a second value based on a threshold value. The corrected binary image generating unit generates a corrected binary image by performing opening processing on the binary image, which includes erosion processing and dilation processing, which are filter processing using a kernel.

[0016] The boundary information generating unit generates boundary information representing the boundary by performing a boundary extraction process on the corrected binarized image to extract a boundary between an area having a first value and an area having a second value. The contour group information generation unit performs a rejection process on the boundary information to reject contour candidates that satisfy a predetermined rejection condition from a contour candidate group, which is a group of closed curves included in the boundary represented by the boundary information, thereby generating contour group information representing a contour candidate group that does not satisfy the rejection condition from the contour candidate group. The particle size distribution estimation unit estimates a particle size distribution, which is a distribution of particle sizes of the plurality of granular bodies, based on the contour group information.

[0017] This allows the contours of each granular body to be reflected with high accuracy in the boundaries extracted by the boundary extraction process, because an opening process is performed. Furthermore, the discarding process allows contour candidates corresponding to the overall contour of a group of granular bodies that are close to each other to be appropriately discarded. This allows the contours of each granular body to be extracted with high accuracy. As a result, the particle size distribution can be estimated with high accuracy. Next, the particle size distribution estimation device of the first embodiment will be described in more detail.

[0018] (composition) As shown in FIG. 1, the particle size distribution estimation device 1 includes an imaging device 11 and an information processing device 20. In this example, the particle size distribution estimation device 1 is mounted on construction machinery. For example, the construction machinery is an excavator or a loader, such as a hydraulic excavator, a power shovel, a shovel loader, a shovel dozer, or a wheel loader. The construction machinery may also be a transport machinery, such as a truck or a belt conveyor, or a bulldozer. Only a part of the particle size distribution estimation device 1 may be mounted on the construction machinery. Furthermore, the particle size distribution estimation device 1 may constitute a part of the construction machinery.

[0019] In this example, the photographing device 11 and the information processing device 20 are separate entities. However, the photographing device 11 and the information processing device 20 may be integrated.

[0020] For example, the information processing device 20 may be represented as a computer. For example, the information processing device 20 may be at least a part of a smartphone, a television set, or the like. For example, the information processing device 20 may be a desktop computer, a laptop computer, a tablet computer, a smartphone, or the like. Note that the information processing device 20 may be composed of multiple devices connected to each other so as to be able to communicate with each other.

[0021] In response to a photography command, the photography device 11 photographs the target area, generates a photographed image that represents the photographed target area, and outputs the generated photographed image. In this example, the target area is a predetermined area on a mounting surface on which a plurality of granular bodies, at least some of which overlap each other, can be placed. In other words, in this example, the target area includes a plurality of granular bodies, at least some of which overlap each other. For example, the target area includes a plurality of granular bodies loaded on the bed of a truck. In this example, the granular material is crushed stone, but the granular material may also be rock blocks, gravel, or sand.

[0022] In this example, the captured image is a visible light image. The visible light image represents the intensity of visible light reflected from a target region for each of a plurality of pixels. In this example, the plurality of pixels included in the visible light image are arranged in a grid pattern.

[0023] In this example, the photographing device 11 includes a color camera or an RGB (Red Green Blue) camera. Note that the photographing device 11 may also be a black and white camera. In this example, the photographed image is a still image. Note that the photographed image may be a moving image instead of a still image. In this case, the information processing device 20 may generate a still image from the moving image.

[0024] The information processing device 20 includes a processing device 21, a storage device 22, and a connection device 23, which are connected to each other via a bus. The processing device 21 executes a program stored in the storage device 22 to control the storage device 22 and the connection device 23. In this way, the processing device 21 realizes the functions described below.

[0025] In this example, the processing device 21 is a central processing unit (CPU). Note that the processing device 21 may include a micro processing unit (MPU), a graphics processing unit (GPU), or a digital signal processor (DSP) instead of or in addition to the CPU.

[0026] In this example, the storage device 22 includes a volatile memory and a non-volatile memory, such as at least one of a random access memory (RAM), a read only memory (ROM), a semiconductor memory, an organic memory, a hard disk drive (HDD), and a solid state drive (SSD).

[0027] The connection device 23 is connected to devices external to the information processing device 20 so as to be able to communicate with them via wired or wireless communication. In this example, the connection device 23 is connected to the photographing device 11 so as to be able to communicate with them via wired communication. The connection device 23 transmits a photographing command to the photographing device 11. The photographing command is information commanding the photographing of a target area. The connection device 23 receives a photographed image output by the photographing device 11, thereby inputting the photographed image.

[0028] (function)

[0029] 3, the functions of the information processing device 20 include a captured image acquisition unit 201, a preprocessing unit 202, a binary image generation unit 203, a corrected binary image generation unit 204, a boundary information generation unit 205, an outline group information generation unit 206, and a particle size distribution estimation unit 207. Note that the functions of the information processing device 20 do not necessarily include the preprocessing unit 202.

[0030] The photographed image acquisition unit 201 transmits a photographing command to the photographing device 11, and receives the photographed image output from the photographing device 11, thereby acquiring the photographed image (in other words, accepting the photographed image).

[0031] The pre-processing unit 202 performs predetermined pre-processing on the captured image acquired by the captured image acquisition unit 201. In this example, the pre-processing includes a trimming process and a filtering process that is executed following the trimming process.

[0032] The trimming process is a process of cutting out a part of the captured image so that no objects other than the granular material are included. In this example, the captured image after the trimming process has a short side of 500 pixels and a long side of 730 pixels. Note that the size of the captured image after the trimming process may be different from that in this example.

[0033] The filtering process is a process of applying a bilateral filter to the captured image after the trimming process so as to suppress shadows due to the unevenness of the surface of the granular material. In this example, the filtering process involves repeatedly applying the bilateral filter multiple times. Note that the pre-processing does not necessarily have to include either the trimming process or the filtering process.

[0034] The binary image generating unit 203 generates a binary image by performing a binary process on the captured image that has been preprocessed by the preprocessing unit 202 . The binarization process converts the values ​​of each of the pixels that make up a captured image into a first or second value based on a threshold. In this example, the first value represents black, and the second value represents white. For example, the first value is 0, and the second value is 255.

[0035] In this example, the threshold is set in advance. For example, the threshold is a value between 100 and 150. The binary image generation unit 203 may determine the threshold based on the captured image. For example, the binary image generation unit 203 may determine the threshold according to a method called the p-tile method, the mode method, Otsu's binarization, or an adaptive binarization method.

[0036] The corrected binary image generation unit 204 generates a corrected binary image by performing opening processing on the binary image generated by the binary image generation unit 203. In this example, the opening processing includes N rounds of erosion processing and N rounds of dilation processing executed following the N rounds of erosion processing. N represents an integer equal to or greater than 1. In this example, N represents 1.

[0037] Erosion processing is a filtering process that replaces the value of a target pixel with a first value when at least one of the pixels included in a kernel area centered on the target pixel has a first value for each of the multiple pixels that make up a binary image. In other words, erosion processing is a filtering process that uses a kernel. Erosion processing is also referred to as erosion.

[0038] The dilation process is a filtering process that replaces the value of a target pixel with a second value when at least one of the pixels included in a kernel area centered on the target pixel has a second value for each of the multiple pixels that make up a binary image. In other words, the dilation process is a filtering process that uses a kernel. The dilation process is also referred to as dilation.

[0039] In this example, the kernel is a square. The kernel may be rectangular, circular, elliptical, or other shapes other than a square. In this example, the size of the kernel is set in advance. For example, the size of the kernel is set so that one side of the kernel has 10 to 40 pixels. The size of the kernel may be set according to the size of the binarized image. The size of the kernel may also be expressed as the kernel size.

[0040] The boundary information generating unit 205 generates boundary information by performing boundary extraction processing on the corrected binary image generated by the corrected binary image generating unit 204. The boundary extraction process extracts the boundary between a region having a first value and a region having a second value. The boundary information represents the boundary between the region having the first value and the region having the second value.

[0041] The contour group information generating unit 206 generates contour group information by performing a rejection process on the boundary information generated by the boundary information generating unit 205 . The rejection process is a process of rejecting contour candidates that satisfy a predetermined rejection condition from a group of contour candidates, which are a group of closed curves included in the boundary represented by the boundary information. The contour group information represents a group of contour candidates that do not satisfy the rejection condition.

[0042] In this example, the rejection process is a shape rejection process that rejects contour candidates based on their shapes. The shape rejection process is a process that rejects contour candidates that satisfy a predetermined shape rejection condition from a group of contour candidates, which are a group of closed curves included in the boundary represented by the boundary information. In this example, the shape rejection condition includes a first shape rejection condition and a second shape rejection condition. Note that the shape rejection condition may include only one of the first shape rejection condition and the second shape rejection condition. In this example, the shape rejection condition corresponds to the rejection condition.

[0043] In this example, the shape rejection condition is satisfied when at least one of the first shape rejection condition and the second shape rejection condition is satisfied. Note that the shape rejection condition may also be satisfied when both the first shape rejection condition and the second shape rejection condition are satisfied.

[0044] The first shape rejection condition is a condition that the circularity parameter, which increases as the shape of the contour candidate to be rejected approaches a circle, is outside the first range. FU is expressed by Equation 1. The circularity parameter may be expressed as an unevenness coefficient.

number

[0045] A represents the area inside the contour candidate. L represents the perimeter of the contour candidate. The first range is a range that is greater than the first lower limit value and less than the first upper limit value. In this example, the first lower limit value is 55% and the first upper limit value is 85%. Note that at least one of the first lower limit value and the first upper limit value may be different from this example.

[0046] The second shape rejection condition is a condition that the rectangularity parameter, which increases as the shape of the contour candidate to be rejected approaches a rectangle, is outside the second range. RE is expressed by Equation 2. The rectangularity parameter may be expressed as rectangularity.

number

[0047] A rect represents the area inside the smallest rectangle that encloses the contour candidate. The second range is a range that is larger than the second lower limit value and smaller than the second upper limit value. In this example, the second lower limit value is 60% and the second upper limit value is 90%. Note that at least one of the second lower limit value and the second upper limit value may be different from this example.

[0048] The particle size distribution estimation unit 207 estimates the particle size distribution, which is the distribution of particle sizes of multiple granular materials in the target region, based on the contour group information generated by the contour group information generation unit 206. In this example, the particle size distribution is a particle size accumulation curve. The particle size accumulation curve represents the change in the passing mass percentage with respect to particle size. The passing mass percentage for a certain particle size is the ratio of the mass of granular materials smaller than that particle size to the mass of all granular materials.

[0049] In this example, the particle size distribution estimation unit 207 estimates the particle size accumulation curve by estimating the mass and particle size of the granular material corresponding to the contour based on the area inside each contour that constitutes the contour group represented by the contour group information. The mass of the granules is estimated by multiplying the area inside the outline by a predetermined first factor, and the particle size of the granules is estimated by multiplying the square root of that area by a predetermined second factor, so that the diameter of the granules corresponds to the diameter of a circle having the same area as the area inside the outline.

[0050] At least one of the first coefficient and the second coefficient may be set according to the number of pixels in the captured image of the reference length. Furthermore, the information processing device 20 may output (for example, display on a display) the estimated particle size distribution.

[0051] (operation) Next, the operation of the particle size distribution estimation device 1 will be described with reference to FIG. The particle size distribution estimation device 1 starts to execute the process shown in the flowchart of FIG. First, the information processing device 20 transmits a shooting command to the photographing device 11, and receives a photographed image output from the photographing device 11, thereby acquiring the photographed image (step S101).

[0052] Next, the information processing device 20 performs preprocessing on the captured image acquired in step S101 (step S102). Next, the information processing device 20 performs binarization processing on the captured image after the preprocessing in step S102 to generate a binarized image (step S103).

[0053] Next, the information processing device 20 performs an opening process on the binarized image generated in step S103 to generate a corrected binarized image (step S104). Next, the information processing device 20 performs a boundary extraction process on the corrected binarized image generated in step S104 to generate boundary information (step S105).

[0054] Next, the information processing device 20 generates contour group information by performing a shape rejection process on the boundary information generated in step S105 (step S106). Next, the information processing device 20 estimates a particle size distribution based on the contour group information generated in step S106 (step S107). As a result, the information processing device 20 ends the execution of the process shown in FIG.

[0055] As described above, the particle size distribution estimation device 1 of the first embodiment includes the photographed image acquisition unit 201, the binary image generation unit 203, the corrected binary image generation unit 204, the boundary information generation unit 205, the contour group information generation unit 206, and the particle size distribution estimation unit 207.

[0056] The photographed image acquisition unit 201 acquires a photographed image, which is an image generated by photographing a target area including a plurality of granular bodies at least partially overlapping each other. The binary image generation unit 203 generates a binary image by performing a binary process on the captured image, converting the values ​​of each of the multiple pixels that make up the captured image into a first value or a second value based on a threshold value. The corrected binary image generating unit 204 generates a corrected binary image by performing opening processing on the binary image, which includes erosion processing and dilation processing, which are filter processing using a kernel.

[0057] The boundary information generating unit 205 performs boundary extraction processing on the corrected binarized image to extract the boundary between the area having the first value and the area having the second value, thereby generating boundary information representing the boundary. The contour group information generation unit 206 performs a rejection process on the boundary information to reject contour candidates that satisfy a predetermined rejection condition from a contour candidate group, which is a group of closed curves included in the boundary represented by the boundary information, thereby generating contour group information representing a contour candidate group that does not satisfy the rejection condition from the contour candidate group. The particle size distribution estimation unit 207 estimates the particle size distribution, which is the distribution of particle sizes of the plurality of granular bodies, based on the contour group information.

[0058] This allows the contours of each granular body to be reflected with high accuracy in the boundaries extracted by the boundary extraction process, because an opening process is performed. Furthermore, the discarding process allows contour candidates corresponding to the overall contour of a group of granular bodies that are close to each other to be appropriately discarded. This allows the contours of each granular body to be extracted with high accuracy. As a result, the particle size distribution can be estimated with high accuracy.

[0059] Furthermore, in the particle size distribution estimation device 1 of the first embodiment, the rejection condition includes at least one of a first shape rejection condition that a circularity parameter that increases as the shape of the contour candidate to be judged approaches a circle is outside a first range, and a second shape rejection condition that a rectangularity parameter that increases as the shape of the contour candidate to be judged approaches a rectangle is outside a second range.

[0060] Incidentally, the contour shape of a granular material often has a circularity parameter within a predetermined range. Furthermore, the contour shape of a granular material often has a rectangularity parameter within a predetermined range. In contrast, according to the particle size distribution estimation device 1, a contour candidate is rejected when the circularity parameter is outside a first range. Furthermore, a contour candidate is rejected when the rectangularity parameter is outside a second range. Therefore, a contour candidate that is unlikely to be the contour shape of a granular material is rejected. This makes it possible to appropriately reject a contour candidate that corresponds to the overall contour of a group of granular materials that are close to each other. Therefore, the contour of each granular material can be extracted with high accuracy. As a result, the particle size distribution can be estimated with high accuracy.

[0061] <First Modification of First Embodiment> Next, a particle size distribution estimation device according to a first modification of the first embodiment will be described. The particle size distribution estimation device according to the first modification of the first embodiment differs from the particle size distribution estimation device according to the first embodiment in that it estimates particle size distribution based on a plurality of captured images generated by capturing images of a target area in a plurality of states in which the positions of at least some of a plurality of granular bodies are different from one another. The following description will focus on the differences. In the description of the first modification of the first embodiment, components that are assigned the same reference numerals as those used in the first embodiment are the same or substantially similar.

[0062] (composition) As shown in FIG. 4, a particle size distribution estimation device 1A of the first modified example of the first embodiment includes a granular material moving device 12 in addition to the components included in the particle size distribution estimation device 1 of the first embodiment. The granular material moving device 12 moves at least some of the granular materials in the target area. In this example, the granular material moving device 12 corresponds to a granular material moving section.

[0063] In this example, the granular material moving device 12 is a bucket of a construction machine. In this example, the photographing device 11 is positioned opposite the granular material moving device 12. The photographing device 11 may also be mounted on the granular material moving device 12.

[0064] (function) The captured image acquisition unit 201 of the first variant of the first embodiment acquires multiple captured images that are each generated by photographing the target area in multiple states in which the positions of at least some of the multiple granular bodies are different from each other due to the movement of the granular bodies by the granular body moving device 12.

[0065] The pre-processing unit 202 of the first modified example of the first embodiment performs pre-processing on each of the multiple captured images acquired by the captured image acquisition unit 201 . The binary image generating unit 203 of the first modified example of the first embodiment generates a plurality of binary images by performing a binary process on each of a plurality of captured images after preprocessing by the preprocessing unit 202.

[0066] The corrected binary image generation unit 204 of the first modified example of the first embodiment generates multiple corrected binary images by performing opening processing on each of the multiple binary images generated by the binary image generation unit 203. The boundary information generating unit 205 of the first modified example of the first embodiment performs boundary extraction processing on each of the plurality of corrected binary images generated by the corrected binary image generating unit 204, thereby generating a plurality of pieces of boundary information.

[0067] The contour group information generating unit 206 of the first modified example of the first embodiment performs rejection processing on each of the plurality of boundary information generated by the boundary information generating unit 205, thereby generating a plurality of pieces of contour group information. The particle size distribution estimation unit 207 of the first modified example of the first embodiment estimates the particle size distribution based on a plurality of pieces of contour group information generated by the contour group information generation unit 206.

[0068] (operation) The particle size distribution estimation device 1A of the first modified example of the first embodiment starts to execute the process shown in FIG. 5 instead of the process shown in FIG. The granular material moving device 12 moves at least some of the plurality of granular materials in a target area (step S201). The information processing device 20 acquires the M photographed images by receiving the M photographed images output from the photographing device 11 by transmitting M photographing commands to the photographing device 11 at M different times during the period in which the granular material is being moved by the granular material moving device 12 (step S202). M represents an integer of 2 or greater. In this example, M represents 3.

[0069] Next, the information processing device 20 executes a loop process (steps S203 to S209) in which each of the M captured images acquired in step S202 is used as a processing target one by one in turn.

[0070] In the loop processing, the information processing device 20 performs preprocessing, binarization processing, opening processing, boundary extraction processing, and shape rejection processing on the captured image to be processed, similarly to steps S102 to S106 in FIG. 3 (steps S204 to S208). Then, the information processing device 20 executes the above loop processing (steps S203 to S209) for all of the acquired M photographed images, and then proceeds to step S210.

[0071] Next, the information processing device 20 estimates the particle size distribution based on the contour group information generated in step S208 for each of the M captured images (step S210). As a result, the information processing device 20 ends the execution of the process shown in FIG.

[0072] As described above, according to the particle size distribution estimating device 1A of the first modified example of the first embodiment, the same actions and effects as those of the particle size distribution estimating device 1 of the first embodiment can be achieved. Furthermore, the particle size distribution estimation device 1A of the first modified example of the first embodiment includes a granular material moving section (in this example, a granular material moving device 12) that moves at least some of the granular materials in the target area.

[0073] The photographed image acquisition unit 201 acquires a plurality of photographed images generated by photographing a target area in a plurality of states in which the positions of at least some of the plurality of granular bodies are different from one another due to the movement of the granular bodies by the granular body moving device 12. The binary image generation unit 203 generates a plurality of binary images by performing a binarization process on each of the plurality of photographed images.

[0074] The corrected binary image generating unit 204 generates a plurality of corrected binary images by performing opening processing on each of the plurality of binary images. The boundary information generating unit 205 generates a plurality of pieces of boundary information by performing boundary extraction processing on each of the plurality of corrected binary images.

[0075] The contour group information generating unit 206 generates a plurality of pieces of contour group information by performing rejection processing on each of the plurality of boundary information. The particle size distribution estimating unit 207 estimates the particle size distribution based on the plurality of pieces of contour group information.

[0076] This allows the contours of each granular body to be extracted based on multiple captured images of the granular bodies, each with different overlapping states, thereby increasing the number of granular bodies whose contours are extracted and enabling the particle size distribution to be estimated with high accuracy.

[0077] <Second Modification of First Embodiment> Next, a particle size distribution estimation device according to a second modified example of the first embodiment will be described. The particle size distribution estimation device according to the second modified example of the first embodiment differs from the particle size distribution estimation device according to the first modified example of the first embodiment in that the particle size distribution estimation device according to the second modified example of the first embodiment detects the resistance force that the particle movement unit receives during the movement of the particle material, and determines the size of the kernel based on the detected resistance force. The following description will focus on the differences. In the description of the second modified example of the first embodiment, components that are assigned the same reference numerals as those used in the first modified example of the first embodiment are the same or substantially similar.

[0078] (composition) As shown in FIG. 6, a particle size distribution estimation device 1B of the second modified example of the first embodiment includes a granular material moving device 12B instead of the granular material moving device 12 of the first modified example of the first embodiment. The granular material moving device 12B moves at least some of the plurality of granular materials in the target area, similar to the granular material moving device 12. In this example, the granular material moving device 12B corresponds to a granular material moving section.

[0079] In this example, the granular material moving device 12B is a bucket of a construction machine. In this example, the granular material moving device 12B is driven hydraulically. Note that the granular material moving device 12B may also be driven electrically. In this example, the photographing device 11 is positioned opposite the granular material moving device 12B. Note that the photographing device 11 may also be mounted on the granular material moving device 12B.

[0080] The granular material moving device 12B is equipped with a resistance force detector 121. The resistance force detector 121 detects the resistance force that the granular material moving device 12B receives when moving the granular material. In this example, the granular material moving device 12B detects the resistance force by detecting the magnitude of the hydraulic pressure that drives the granular material moving device 12B. In this example, the resistance force detector 121 corresponds to a resistance force detection unit.

[0081] (function) The corrected binary image generation unit 204 of the second modified example of the first embodiment determines the size of the kernel used in the opening process based on the resistance force detected by the resistance force detector 121. In this example, the corrected binary image generation unit 204 determines the kernel size so that it becomes smaller as the detected resistance force becomes smaller.

[0082] For example, if the detected resistance is greater than a predetermined resistance threshold, the corrected binary image generation unit 204 determines the size of the kernel so that one side of the kernel has a first number of pixels, and if the resistance is equal to or less than the resistance threshold, the corrected binary image generation unit 204 determines the size of the kernel so that one side of the kernel has a second number of pixels that is smaller than the first number of pixels. Note that the size of the kernel may be determined so that one side of the kernel has a number of pixels selected from a group of three or more pixel numbers. In this case, two or more resistance thresholds may be provided.

[0083] (operation) The particle size distribution estimation device 1B of the second modified example of the first embodiment executes a process in which the processes of steps S2011 and S2012 shown in FIG. 7 are added between steps S201 and S202 of the process shown in FIG.

[0084] Therefore, after the granular material moving device 12B starts moving at least some of the multiple granular materials in the target area in step S201, the resistance force detector 121 detects the resistance force that the granular material moving device 12B receives in the movement of the granular materials during the period in which the granular material moving device 12B is moving the granular materials (step S2011).

[0085] Next, the information processing device 20 determines the size of the kernel to be used in the opening process based on the resistance force detected in step S2011 (step S2012). Next, the information processing device 20 executes the processes of steps S202 to S210, similarly to the information processing device 20 of the first modified example of the first embodiment.

[0086] As described above, according to the particle size distribution estimating device 1B of the second modified example of the first embodiment, the same actions and effects as those of the particle size distribution estimating device 1A of the first modified example of the first embodiment can be achieved. Furthermore, the particle size distribution estimation device 1B of the second modified example of the first embodiment includes a resistance force detection unit (resistance force detector 121 in this example) that detects the resistance force that the granular material moving unit (granular material moving device 12B in this example) receives during the movement of the granular material. The corrected binary image generation unit 204 determines the kernel size so that it becomes smaller as the detected resistance force becomes smaller.

[0087] Incidentally, the smaller the average particle size, the smaller the resistance force in many cases. Furthermore, the smaller the average particle size, the smaller the appropriate kernel size in many cases. Therefore, the particle size distribution estimation device 1B can extract the contours of each granular body with high accuracy. As a result, the particle size distribution can be estimated with high accuracy.

[0088] The particle size distribution estimation device 1B of the second modified example of the first embodiment may estimate the particle size distribution based on one captured image.

[0089] Second Embodiment Next, a particle size distribution estimation device of a second embodiment will be described. The particle size distribution estimation device of the second embodiment differs from the particle size distribution estimation device of the first embodiment in that it generates multiple binary images by performing binarization processing for each of multiple thresholds that are different from each other, and estimates the particle size distribution based on the generated multiple binary images. The following mainly describes the differences. In the description of the second embodiment, parts that are assigned the same reference numerals as those used in the first embodiment are the same or substantially similar.

[0090] (function) The binary image generating unit 203 of the second embodiment generates multiple binary images by performing a binarization process for each of multiple different thresholds. The multiple thresholds are set so that the difference between two consecutive thresholds in the luminance range of the captured image matches a predetermined difference amount. In this example, the difference amount is 5. Note that the difference amount may be a value other than 5. For example, if the luminance range of the captured image is 110 to 120, the multiple thresholds are set to 110, 115, and 120. Note that the multiple thresholds may be set in advance.

[0091] The corrected binary image generating unit 204 of the second embodiment performs opening processing on each of the plurality of binary images generated by the binary image generating unit 203, thereby generating a plurality of corrected binary images. The boundary information generating unit 205 of the second embodiment performs boundary extraction processing on each of the plurality of corrected binary images generated by the corrected binary image generating unit 204, thereby generating a plurality of pieces of boundary information.

[0092] The contour group information generating unit 206 of the second embodiment generates contour group information by performing a rejection process on a plurality of pieces of boundary information generated by the boundary information generating unit 205. The rejection process is a process of rejecting contour candidates that satisfy a predetermined rejection condition from a group of contour candidates, which are a group of closed curves included in the boundary represented by the boundary information. The contour group information represents a group of contour candidates that do not satisfy the rejection condition.

[0093] In this example, the rejection process includes a shape rejection process that rejects contour candidates based on their shapes, and a duplicate rejection process that rejects duplicate contour candidates.

[0094] In this example, the contour group information generation unit 206 generates contour candidate group information by performing a shape rejection process on each of the plurality of boundary information generated by the boundary information generation unit 205. The contour candidate group information represents a group of contour candidates that do not satisfy the shape rejection condition, among the group of contour candidates, which are a group of closed curves included in the boundary represented by the boundary information.

[0095] The shape rejection process is a process of rejecting contour candidates that satisfy a predetermined shape rejection condition from a group of contour candidates, which are a group of closed curves included in the boundary represented by the boundary information. In this example, the shape rejection condition includes a first shape rejection condition and a second shape rejection condition. The first shape rejection condition and the second shape rejection condition are similar to the first shape rejection condition and the second shape rejection condition of the first embodiment. Note that the shape rejection condition may include only one of the first shape rejection condition and the second shape rejection condition. In this example, the shape rejection condition corresponds to a part of the rejection condition.

[0096] In this example, the shape rejection condition is satisfied when at least one of the first shape rejection condition and the second shape rejection condition is satisfied. Note that the shape rejection condition may also be satisfied when both the first shape rejection condition and the second shape rejection condition are satisfied.

[0097] In this example, the contour group information generation unit 206 generates contour group information by performing overlap rejection processing on each of the contour candidate group information pairs consisting of two contour candidate group information generated for two consecutive thresholds in the multiple contour candidate group information generated by the shape rejection processing.

[0098] The overlap rejection process is a process of rejecting contour candidates that satisfy a predetermined overlap rejection condition from among the contour candidate group represented by the contour candidate group information, so as to reject contour candidates that overlap between contour candidate group information constituting a contour candidate group information pair. The overlap rejection condition is a condition that a contour candidate being determined whether to reject contains the center of another contour candidate. In this example, the overlap rejection condition corresponds to part of the rejection condition.

[0099] In this example, in the overlap rejection process, the contour candidate that constitutes the contour candidate group represented by the contour candidate group information based on the binary image generated based on the larger threshold value of the two contour candidate group information that constitute the contour candidate group information pair is the subject of a determination as to whether to reject it or not.

[0100] In this example, in the duplicate rejection process, whether or not the duplicate rejection condition is satisfied is determined according to the Crossing Number Algorithm. Note that in the duplicate rejection process, whether or not the duplicate rejection condition is satisfied may be determined according to the Winding Number Algorithm instead of the Crossing Number Algorithm.

[0101] (operation) The particle size distribution estimation device 1 of the second embodiment starts to execute the process shown in FIG. 8 instead of the process shown in FIG. The information processing device 20 acquires a captured image and performs preprocessing on the captured image (steps S301 and S302), similar to steps S101 and S102 in FIG.

[0102] Next, the information processing device 20 executes a loop process (steps S303 to S308) in which each of the P threshold values ​​is used as a processing target one by one in turn, where P represents an integer of 2 or greater.

[0103] In the loop processing, the information processing device 20 executes binarization processing, opening processing, and boundary extraction processing for the threshold value to be processed (steps S304 to S306), similarly to steps S103 to S105 in Fig. 3. Next, in the loop processing, the information processing device 20 generates contour candidate group information by executing shape rejection processing on the boundary information generated in step S306 for the threshold value to be processed (step S307).

[0104] Then, the information processing device 20 executes the above loop process (steps S303 to S308) for all of the P threshold values, and then proceeds to step S309.

[0105] Next, the information processing device 20 generates contour group information by performing overlap rejection processing on the contour candidate group information generated in step S307 for each of the P threshold values ​​(step S309). Next, the information processing device 20 estimates the particle size distribution based on the contour group information generated in step S309 (step S310). As a result, the information processing device 20 ends the execution of the process shown in FIG. The process of step S309 may be executed immediately after step S307 in the loop process.

[0106] As described above, according to the particle size distribution estimating device 1 of the second embodiment, the same actions and effects as those of the particle size distribution estimating device 1 of the first embodiment can be achieved. Furthermore, in the particle size distribution estimation device 1 of the second embodiment, the binary image generation unit 203 generates a plurality of binary images by performing a binary process for each of a plurality of different thresholds. The corrected binary image generation unit 204 generates a plurality of corrected binary images by performing an opening process for each of the plurality of binary images.

[0107] The boundary information generation unit 205 generates multiple pieces of boundary information by performing boundary extraction processing on each of the multiple corrected binarized images. The contour group information generation unit 206 generates contour group information by performing rejection processing on the multiple pieces of boundary information. The rejection condition includes an overlap rejection condition that the center of one contour candidate is contained within the center of another contour candidate.

[0108] However, the granules whose contours are extracted often vary depending on the threshold value used in the binarization process. Therefore, by performing binarization for each of multiple different threshold values, the number of granules whose contours are extracted can be increased. Furthermore, a contour candidate is rejected if the center of another contour candidate is contained within the candidate, preventing multiple contours from being extracted for the same granule. This allows for appropriate rejection of contour candidates corresponding to the overall contour of a group of closely spaced granules. Therefore, the contours of each granule can be extracted with high accuracy. As a result, the particle size distribution can be estimated with high accuracy.

[0109] <First Modification of Second Embodiment> Next, a particle size distribution estimation device according to a first modified example of the second embodiment will be described. The particle size distribution estimation device according to the first modified example of the second embodiment differs from the particle size distribution estimation device according to the second embodiment in that the particle size distribution is estimated based on a plurality of captured images generated by capturing images of a target area in a plurality of states in which the positions of at least some of a plurality of granular bodies are different from one another. The following description will focus on the differences. In the description of the first modified example of the second embodiment, components that are assigned the same reference numerals as those used in the second embodiment are the same or substantially similar.

[0110] (composition) As shown in FIG. 4, a particle size distribution estimation device 1A of the first modified example of the second embodiment includes a granular material moving device 12 in addition to the components included in the particle size distribution estimation device 1 of the second embodiment. The granular material moving device 12 moves at least some of the granular materials in the target area. In this example, the granular material moving device 12 corresponds to a granular material moving section.

[0111] In this example, the granular material moving device 12 is a bucket of a construction machine. In this example, the photographing device 11 is positioned opposite the granular material moving device 12. The photographing device 11 may also be mounted on the granular material moving device 12.

[0112] (function) The captured image acquisition unit 201 of the first variant of the second embodiment acquires multiple captured images that are each generated by photographing the target area in multiple states in which the positions of at least some of the multiple granular bodies are different from each other due to the movement of the granular bodies by the granular body moving device 12.

[0113] The pre-processing unit 202 of the first modified example of the second embodiment performs pre-processing on each of the multiple captured images acquired by the captured image acquisition unit 201 .

[0114] The binary image generation unit 203 of the first variant of the second embodiment generates multiple binary images by performing binary processing using multiple different thresholds on the captured image after preprocessing has been performed by the preprocessing unit 202 for each captured image.

[0115] The corrected binary image generation unit 204 of the first variant of the second embodiment generates multiple corrected binary images by performing opening processing on each of the multiple binary images generated by the binary image generation unit 203 for each captured image.

[0116] The boundary information generation unit 205 of the first variant of the second embodiment generates multiple pieces of boundary information by performing boundary extraction processing on each of the multiple corrected binary images generated by the corrected binary image generation unit 204 for each captured image.

[0117] The contour group information generating unit 206 of the first modified example of the second embodiment generates contour group information by performing a rejection process on each of the plurality of boundary information generated by the boundary information generating unit 205 for each captured image. The particle size distribution estimation unit 207 of the first modified example of the second embodiment estimates the particle size distribution based on a plurality of pieces of contour group information generated by the contour group information generation unit 206.

[0118] (operation) The particle size distribution estimation device 1A of the first modified example of the second embodiment starts to execute the process shown in FIG. 9 instead of the process shown in FIG. The granular material moving device 12 moves at least some of the plurality of granular materials in the target area (step S401). The information processing device 20 acquires the M photographed images by receiving the M photographed images output from the photographing device 11 by transmitting M photographing commands to the photographing device 11 at M different times during the period in which the granular material is being moved by the granular material moving device 12 (step S402). M represents an integer of 2 or greater. In this example, M represents 3.

[0119] Next, the information processing device 20 executes a first loop process (steps S403 to S412) in which each of the M captured images acquired in step S402 is used as a processing target one by one in turn.

[0120] In the first loop process, the information processing device 20 performs pre-processing on the captured image to be processed (step S404), similar to step S302 in FIG. Next, in the first loop process, the information processing device 20 executes a second loop process (steps S405 to S410) in which each of the P threshold values ​​is used as a processing target one by one in turn, where P represents an integer of 2 or greater.

[0121] In the second loop process, the information processing device 20 executes binarization processing, opening processing, boundary extraction processing, and shape rejection processing on the threshold value to be processed, similarly to steps S304 to S307 in FIG. 8 (steps S406 to S409).

[0122] Then, the information processing device 20 executes the second loop process (steps S405 to S410) for all of the P threshold values, and then proceeds to step S411.

[0123] Next, the information processing device 20 performs overlap rejection processing on the contour candidate group information generated in step S409 for each of the P thresholds, thereby generating contour group information for the photographed image to be processed (step S411). Then, the information processing device 20 executes the first loop process (steps S403 to S412) for all of the acquired M photographed images, and then proceeds to step S413.

[0124] Next, the information processing device 20 estimates the particle size distribution based on the contour group information generated for each of the M captured images in step S411 (step S413). As a result, the information processing device 20 ends the execution of the process shown in FIG. The process of step S411 may be executed immediately after step S409 in the first loop process.

[0125] Here, an example of the results of the processing executed by the information processing device 20 will be described with reference to FIGS. Fig. 10(A) shows an example of a captured image after preprocessing. Fig. 10(B) shows an example of a binarized image when the threshold value is 100. Fig. 10(C) shows an example of a binarized image when the threshold value is 125. Fig. 10(D) shows an example of a binarized image when the threshold value is 150.

[0126] Fig. 11A shows an example of a binarized image before the opening process is performed, and Fig. 11B shows an example of a binarized image after the opening process is performed (in other words, a corrected binarized image). Fig. 12A shows an example of a corrected binarized image before the shape rejection process is performed, and Fig. 12B shows an example of a corrected binarized image that reflects the contour candidates rejected by the shape rejection process.

[0127] FIG. 13 shows an example of an image in which the contour groups represented by the contour group information are drawn in solid lines on the captured image after preprocessing. Fig. 14 shows an example of particle size distribution (in this example, a particle size accumulation curve). In Fig. 14, curve C1 shows the measured particle size accumulation curve, and curve C2 shows the estimated particle size accumulation curve.

[0128] As described above, according to the particle size distribution estimating device 1A of the first modified example of the second embodiment, the same actions and effects as those of the particle size distribution estimating device 1 of the second embodiment can be achieved. Furthermore, the particle size distribution estimation device 1A of the first modified example of the second embodiment includes a granular material moving unit (in this example, a granular material moving device 12) that moves at least some of the plurality of granular materials in a target area. The photographed image acquisition unit 201 acquires a plurality of photographed images that are respectively generated by photographing the target area in a plurality of states in which the positions of at least some of the plurality of granular materials are different from each other due to the movement of the granular materials by the granular material moving device 12.

[0129] The binary image generation unit 203 generates a plurality of binary images by performing a binary process on each of the plurality of captured images. The corrected binary image generation unit 204 generates a plurality of corrected binary images by performing an opening process on each of the plurality of binary images.

[0130] The boundary information generation unit 205 generates a plurality of pieces of boundary information by performing boundary extraction processing on each of the plurality of corrected binarized images. The contour group information generation unit 206 generates a plurality of pieces of contour group information by performing rejection processing on each of the plurality of pieces of boundary information. The particle size distribution estimation unit 207 estimates the particle size distribution based on the plurality of pieces of contour group information.

[0131] This allows the contours of each granular body to be extracted based on multiple captured images of the granular bodies, each with different overlapping states, thereby increasing the number of granular bodies whose contours are extracted and enabling the particle size distribution to be estimated with high accuracy.

[0132] <Third embodiment> Next, a particle size distribution estimation device of a third embodiment will be described. The particle size distribution estimation device of the third embodiment differs from the particle size distribution estimation device of the first embodiment in that it generates a plurality of corrected binary images by performing opening processing on each of a plurality of kernel sizes that are different from each other, and estimates the particle size distribution based on the generated plurality of corrected binary images. The following mainly describes the differences. In the description of the third embodiment, parts that are assigned the same reference numerals as those used in the first embodiment are the same or substantially similar.

[0133] (function) The corrected binary image generation unit 204 of the third embodiment generates a plurality of corrected binary images by performing opening processing on each of a plurality of different kernel sizes (in other words, kernel sizes). The plurality of kernel sizes are set in advance. In this example, the plurality of kernel sizes are set to a kernel size having 17 pixels on one side and a kernel size having 31 pixels on one side. Note that the plurality of kernel sizes may be set to sizes different from those in this example. For example, the plurality of kernel sizes may be set to a kernel size having 19 pixels on one side and a kernel size having 31 pixels on one side. The number of kernel sizes may be three or more. For example, the plurality of kernel sizes may be set to a kernel size having 19 pixels on one side, a kernel size having 27 pixels on one side, and a kernel size having 31 pixels on one side.

[0134] The boundary information generating unit 205 of the third embodiment performs boundary extraction processing on each of the plurality of corrected binary images generated by the corrected binary image generating unit 204, thereby generating a plurality of pieces of boundary information.

[0135] The contour group information generation unit 206 of the third embodiment, like the contour group information generation unit 206 of the second embodiment, generates contour group information by performing a rejection process on multiple pieces of boundary information generated by the boundary information generation unit 205.

[0136] (operation) The particle size distribution estimation device 1 of the third embodiment starts to execute the process shown in FIG. 15 instead of the process shown in FIG. Similar to steps S101 to S103 in FIG. 3, the information processing device 20 acquires a captured image and performs preprocessing and binarization processing on the captured image (steps S501 to S503).

[0137] Next, the information processing device 20 executes a loop process (steps S504 to S508) in which each of the Q kernel sizes is used as a processing target one by one in turn, where Q represents an integer of 2 or greater.

[0138] In the loop processing, the information processing device 20 executes opening processing and boundary extraction processing for the kernel size to be processed (steps S505 and S506), similar to steps S104 and S105 in Fig. 3. Next, in the loop processing, the information processing device 20 generates contour candidate group information by executing shape rejection processing on the boundary information generated in step S506 for the kernel size to be processed (step S507).

[0139] Then, the information processing device 20 executes the above loop process (steps S504 to S508) for all of the Q kernel sizes, and then proceeds to step S509.

[0140] Next, the information processing apparatus 20 generates contour group information by performing overlap rejection processing on the contour candidate group information generated in step S507 for each of the Q kernel sizes (step S509). Next, the information processing device 20 estimates the particle size distribution based on the contour group information generated in step S509 (step S510). As a result, the information processing device 20 ends the execution of the process shown in FIG. The process of step S509 may be executed immediately after step S507 in the loop process.

[0141] As described above, according to the particle size distribution estimating device 1 of the third embodiment, the same actions and effects as those of the particle size distribution estimating device 1 of the first embodiment can be achieved. Furthermore, in the particle size distribution estimation device 1 of the third embodiment, the corrected binary image generating unit 204 generates a plurality of corrected binary images by performing opening processing on each of a plurality of kernel sizes that are different from one another.

[0142] The boundary information generation unit 205 generates multiple pieces of boundary information by performing boundary extraction processing on each of the multiple corrected binarized images. The contour group information generation unit 206 generates contour group information by performing rejection processing on the multiple pieces of boundary information. The rejection condition includes an overlap rejection condition that the center of one contour candidate is contained within the center of another contour candidate.

[0143] The size of the kernel used in the opening process often determines the number of granules whose contours are extracted. Therefore, by performing the opening process for each of multiple different kernel sizes, the number of granules whose contours are extracted can be increased. Furthermore, a contour candidate is rejected if the center of another contour candidate is included within the candidate, preventing multiple contours from being extracted for the same granule. This allows for appropriate rejection of contour candidates corresponding to the overall contour of a group of closely spaced granules. Therefore, the contours of each granule can be extracted with high accuracy. As a result, the particle size distribution can be estimated with high accuracy.

[0144] The particle size distribution estimation device 1 of the first modified example of the third embodiment may estimate the particle size distribution based on a plurality of captured images generated by photographing a target area in a plurality of states in which the positions of at least some of the plurality of granular bodies are different from each other, similar to the particle size distribution estimation device 1A of the first modified example of the first embodiment. Furthermore, the particle size distribution estimation device 1 of the second modified example of the third embodiment may, like the particle size distribution estimation device 1B of the second modified example of the first embodiment, detect the resistance force that the granular material moving section receives during the movement of the granular material, and determine the size of the kernel based on the detected resistance force.

[0145] <Fourth embodiment> Next, a particle size distribution estimation device of a fourth embodiment will be described. The particle size distribution estimation device of the fourth embodiment differs from the particle size distribution estimation device of the second embodiment in that it generates a plurality of corrected binary images by performing opening processing for each of a plurality of different kernel sizes for each binary image generated for each of a plurality of different thresholds, and estimates the particle size distribution based on the generated corrected binary images. The following mainly describes the differences. In the description of the fourth embodiment, parts that are assigned the same reference numerals as those used in the second embodiment are the same or substantially similar.

[0146] (function) The corrected binary image generation unit 204 of the fourth embodiment generates multiple corrected binary images by performing opening processing on each of multiple different kernel sizes (in other words, kernel sizes) for each binary image generated by the binary image generation unit 203. The multiple kernel sizes are set in advance. In this example, the multiple kernel sizes are set to a kernel size with 17 pixels on one side and a kernel size with 31 pixels on one side. Note that the multiple kernel sizes may be set to sizes different from those in this example. For example, the multiple kernel sizes may be set to a kernel size with 19 pixels on one side and a kernel size with 31 pixels on one side. Furthermore, the number of kernel sizes may be three or more. For example, the multiple kernel sizes may be set to a kernel size with 19 pixels on one side, a kernel size with 27 pixels on one side, and a kernel size with 31 pixels on one side.

[0147] The boundary information generating unit 205 of the fourth embodiment generates a plurality of pieces of boundary information by performing boundary extraction processing on each of a plurality of corrected binary images generated for each binary image. The contour group information generating unit 206 of the fourth embodiment generates contour group information by performing a rejection process on a plurality of pieces of boundary information generated for each binarized image.

[0148] (operation) The particle size distribution estimation device 1 of the fourth embodiment starts to execute the process shown in FIG. 16 instead of the process shown in FIG. The information processing device 20 acquires a captured image and performs preprocessing on the captured image (steps S601 and S602), similar to steps S301 and S302 in FIG.

[0149] Next, the information processing device 20 executes a first loop process (steps S603 to S610) in which each of the P threshold values ​​is used as a processing target one by one in turn, where P represents an integer of 2 or greater.

[0150] In the first loop process, the information processing device 20 executes binarization processing on the threshold value to be processed (step S604), similarly to step S304 in FIG.

[0151] Next, the information processing device 20 executes a second loop process (steps S605 to S609) in which each of the Q kernel sizes is used as a processing target one by one in turn, where Q represents an integer of 2 or greater.

[0152] In the second loop process, the information processing device 20 executes opening processing and boundary extraction processing for the kernel size to be processed (steps S606 and S607), similarly to steps S305 and S306 in Fig. 8. Next, in the second loop process, the information processing device 20 generates contour candidate group information by executing shape rejection processing on the boundary information generated in step S607 for the kernel size to be processed (step S608).

[0153] Then, the information processing device 20 executes the second loop process (steps S605 to S609) for all of the Q kernel sizes, and then proceeds to step S610. Then, the information processing device 20 executes the first loop process (steps S603 to S610) for all of the P threshold values, and then proceeds to step S611.

[0154] Next, the information processing apparatus 20 generates contour group information by performing overlap rejection processing on the contour candidate group information generated in step S608 for each of the Q kernel sizes for each threshold value (step S611). Next, the information processing device 20 estimates the particle size distribution based on the contour group information generated in step S611 (step S612). As a result, the information processing device 20 ends the execution of the process shown in FIG. The process of step S611 may be executed immediately after the second loop process in the first loop process, or may be executed immediately after step S608 in the second loop process.

[0155] As described above, according to the particle size distribution estimating device 1 of the fourth embodiment, the same actions and effects as those of the particle size distribution estimating device 1 of the second embodiment can be achieved. Furthermore, in the particle size distribution estimation device 1 of the fourth embodiment, the corrected binary image generating unit 204 generates a plurality of corrected binary images by performing opening processing on each of a plurality of kernel sizes different from each other.

[0156] The boundary information generation unit 205 generates multiple pieces of boundary information by performing boundary extraction processing on each of the multiple corrected binarized images. The contour group information generation unit 206 generates contour group information by performing rejection processing on the multiple pieces of boundary information. The rejection condition includes an overlap rejection condition that the center of one contour candidate is contained within the center of another contour candidate.

[0157] The size of the kernel used in the opening process often determines the number of granules whose contours are extracted. Therefore, by performing the opening process for each of multiple different kernel sizes, the number of granules whose contours are extracted can be increased. Furthermore, a contour candidate is rejected if the center of another contour candidate is included within the candidate, preventing multiple contours from being extracted for the same granule. This allows for appropriate rejection of contour candidates corresponding to the overall contour of a group of closely spaced granules. Therefore, the contours of each granule can be extracted with high accuracy. As a result, the particle size distribution can be estimated with high accuracy.

[0158] The particle size distribution estimation device 1 of the first modified example of the fourth embodiment may estimate the particle size distribution based on a plurality of captured images generated by photographing a target area in a plurality of states in which the positions of at least some of the plurality of granular bodies are different from each other, similar to the particle size distribution estimation device 1A of the first modified example of the first embodiment. Furthermore, the particle size distribution estimation device 1 of the second modified example of the fourth embodiment may, like the particle size distribution estimation device 1B of the second modified example of the first embodiment, detect the resistance force that the granular material moving section receives during the movement of the granular material, and determine the size of the kernel based on the detected resistance force.

[0159] Fifth Embodiment Next, a particle size distribution estimation device of a fifth embodiment will be described. The particle size distribution estimation device of the fifth embodiment differs from the particle size distribution estimation device of the first embodiment in that the size of the kernel is determined based on the total distance of edges detected in a captured image. The following description will focus on the differences. In the description of the fifth embodiment, components that are assigned the same reference numerals as those used in the first embodiment are the same or substantially similar.

[0160] (function) The corrected binary image generation unit 204 of the fifth embodiment detects edges in the captured image after preprocessing has been performed by the preprocessing unit 202, and obtains a total edge distance, which is the total distance of the detected edges. In this example, the corrected binary image generation unit 204 detects edges using the Canny method (in other words, a Canny edge detector).

[0161] The corrected binary image generating unit 204 may detect edges using a Sobel filter, a Prewitt filter, a Roberts filter, a Laplacian filter, a Laplacian of Gaussians filter, or a Difference of Gaussians filter instead of the Canny method.

[0162] Furthermore, the corrected binary image generation unit 204 may detect edges using a method using deep learning (for example, Holistically Nested Edge Detection, Richly Activated Convolutional Features, or DexiNed).

[0163] The corrected binary image generation unit 204 determines the size of the kernel used in the opening process based on the acquired total edge distance. In this example, the corrected binary image generation unit 204 determines the kernel size so that it becomes smaller as the acquired total edge distance becomes longer.

[0164] (operation) The particle size distribution estimation device 1 of the fifth embodiment executes a process in which the processes of steps S1021 and S1022 shown in FIG. 17 are added between steps S102 and S103 of the process shown in FIG.

[0165] Therefore, the information processing device 20 performs preprocessing on the captured image, detects edges in the captured image after the preprocessing, and obtains the total edge distance, which is the total distance of the detected edges (step S1021).

[0166] Next, the information processing device 20 determines the size of the kernel to be used in the opening process based on the total edge distance acquired in step S1021 (step S1022). Next, the information processing device 20 executes the processes of steps S103 to S107, similarly to the information processing device 20 of the first embodiment.

[0167] As described above, according to the particle size distribution estimating device 1 of the fifth embodiment, the same actions and effects as those of the particle size distribution estimating device 1 of the first embodiment can be achieved. Furthermore, in the particle size distribution estimation device 1 of the fifth embodiment, the corrected binary image generation unit 204 determines the size of the kernel based on the total distance of the edges detected in the captured image.

[0168] Incidentally, there is a strong correlation between the average particle size and the width of the particle size distribution (in other words, the range of particle sizes in the particle size distribution) and the total distance of the edge. There is also a strong correlation between the average particle size and the width of the particle size distribution and the appropriate kernel size. Therefore, the particle size distribution estimation device 1 can extract the contour of each granular body with high accuracy. As a result, the particle size distribution can be estimated with high accuracy.

[0169] Furthermore, in the particle size distribution estimation device 1 of the fifth embodiment, the corrected binary image generation unit 204 determines the size of the kernel so that the size decreases as the total distance of the edges detected in the captured image increases.

[0170] Incidentally, the smaller the average particle size, the longer the total distance of the edge often becomes. Furthermore, the smaller the average particle size, the smaller the appropriate kernel size often becomes. Therefore, the particle size distribution estimation device 1 can extract the contour of each granular body with high accuracy. As a result, the particle size distribution can be estimated with high accuracy.

[0171] In this example, the opening process is performed for one kernel size (in other words, kernel size), so the processing load can be reduced compared to when the opening process is performed for each of a plurality of kernel sizes.

[0172] Note that, similarly to the particle size distribution estimation device 1 of the second embodiment, the particle size distribution estimation device 1 of the fifth embodiment may generate a plurality of binary images by performing binarization processing for each of a plurality of threshold values ​​different from each other, and estimate the particle size distribution based on the generated binary images.

[0173] Furthermore, the particle size distribution estimation device 1 of the fifth embodiment may estimate the particle size distribution based on a plurality of captured images generated by photographing a target area in a plurality of states in which the positions of at least some of the plurality of granular bodies are different from each other, similar to the particle size distribution estimation device 1A of the first modified example of the first embodiment.

[0174] In this case, the corrected binary image generation unit 204 may obtain the size of the kernel for each of the multiple captured images based on the total distance of the edges detected in the captured image, and use the size of the kernel obtained for the captured image in the opening processing for the captured image.

[0175] In this case, the corrected binary image generation unit 204 may obtain the size of the kernel for each of the multiple captured images based on the total distance of the edges detected in that captured image, and determine the average value of the multiple kernel sizes obtained for each of the multiple captured images as the size of the kernel to be used in the opening process for all of the multiple captured images.

[0176] <First Modification of Fifth Embodiment> Next, a particle size distribution estimation device according to a first modified example of the fifth embodiment will be described. The particle size distribution estimation device according to the first modified example of the fifth embodiment differs from the particle size distribution estimation device according to the fifth embodiment in that the size of the kernel is determined based on a brightness change parameter acquired based on a change in brightness along a predetermined line in a captured image. The following description will focus on the differences. In the description of the first modified example of the fifth embodiment, components that are assigned the same reference numerals as those used in the fifth embodiment are the same or substantially similar.

[0177] (function) The corrected binary image generation unit 204 of the first modified example of the fifth embodiment acquires a brightness change parameter based on a change in brightness along a predetermined line in the captured image after preprocessing by the preprocessing unit 202.

[0178] In this example, as represented by the dotted lines in (A) of Figure 18, the predetermined line in the captured image is made up of multiple (eight in this example) straight lines extending in the vertical direction. In this example, the multiple straight lines are positioned at equal intervals in the horizontal direction. Note that the predetermined line in the captured image may be made up of one straight line, or may be made up of multiple straight lines other than eight.

[0179] 19A, the predetermined line in the captured image may be made up of multiple (five in this example) straight lines extending in the horizontal direction. In this case, too, the predetermined line in the captured image may be made up of one straight line, or may be made up of multiple straight lines other than five.

[0180] Furthermore, the predetermined line in the captured image may be a curved line. For example, as shown by the dotted line in (B) of Fig. 19, the predetermined line in the captured image may be a curved line formed by connecting multiple straight lines extending in the vertical direction at their ends in the vertical direction. Furthermore, the predetermined line in the captured image may be a combination of straight lines and curved lines.

[0181] 18B is a graph showing an example of a change in luminance with respect to a pixel position on a line. The pixel position on a line represents the position of a pixel along a predetermined line in a captured image. In this example, the pixel position on a line represents the number of pixels counted from the start point to the end point of the predetermined line in the captured image.

[0182] In this example, the brightness change parameter is the number of peaks in the brightness change relative to the pixel positions on the line (in other words, the number of brightness change peaks). Note that the brightness change parameter may be a frequency range in a power spectrum obtained by performing frequency analysis on the brightness change relative to the pixel positions on the line. For example, the frequency range may be a full width at half maximum, or a frequency range in which the power is equal to or greater than a predetermined threshold.

[0183] The corrected binary image generation unit 204 determines the size of the kernel used in the opening process based on the acquired brightness change parameter. In this example, the corrected binary image generation unit 204 determines the kernel size so that it becomes smaller as the acquired brightness change parameter becomes larger.

[0184] (operation) The particle size distribution estimation device 1 of the first modified example of the fifth embodiment executes a process in which the processes of step S1021A and step S1022 shown in FIG. 20 are added between step S102 and step S103 of the process shown in FIG.

[0185] Therefore, the information processing device 20 performs preprocessing on the captured image, and then acquires the brightness change parameters of the captured image after the preprocessing (step S1021A).

[0186] Next, the information processing device 20 determines the size of the kernel to be used in the opening process based on the luminance change parameter acquired in step S1021A (step S1022). Next, the information processing device 20 executes the processes of steps S103 to S107, similarly to the information processing device 20 of the first embodiment.

[0187] As described above, according to the particle size distribution estimating device 1 of the first modified example of the fifth embodiment, the same actions and effects as those of the particle size distribution estimating device 1 of the first embodiment can be achieved. Furthermore, in the particle size distribution estimation device 1 of the first modified example of the fifth embodiment, the corrected binary image generation unit 204 determines the size of the kernel based on a brightness change parameter acquired based on a change in brightness along a predetermined line in the captured image.

[0188] Incidentally, there is a strong correlation between the average particle size and the width of the particle size distribution (in other words, the particle size range in the particle size distribution) and the brightness change parameter. There is also a strong correlation between the average particle size and the width of the particle size distribution and the appropriate kernel size. Therefore, the particle size distribution estimation device 1 can extract the contour of each granular body with high accuracy. As a result, the particle size distribution can be estimated with high accuracy.

[0189] Furthermore, in the particle size distribution estimation device 1 of the first modified example of the fifth embodiment, the corrected binary image generation unit 204 determines the size of the kernel so that it becomes smaller as the number of brightness change peaks acquired in the captured image increases.

[0190] Incidentally, the smaller the average particle size, the larger the number of brightness change peaks in many cases. Furthermore, the smaller the average particle size, the smaller the appropriate kernel size in many cases. Therefore, the particle size distribution estimation device 1 can extract the contours of each granular body with high accuracy. As a result, the particle size distribution can be estimated with high accuracy.

[0191] In this example, the opening process is performed for one kernel size (in other words, kernel size), so the processing load can be reduced compared to when the opening process is performed for each of a plurality of kernel sizes.

[0192] Note that, like the particle size distribution estimation device 1 of the second embodiment, the particle size distribution estimation device 1 of the first modified example of the fifth embodiment may generate multiple binary images by performing binarization processing for each of multiple threshold values ​​different from each other, and estimate the particle size distribution based on the generated multiple binary images.

[0193] Furthermore, the particle size distribution estimation device 1 of the first modified example of the fifth embodiment may estimate the particle size distribution based on a plurality of captured images generated by photographing a target area in a plurality of states in which the positions of at least some of the plurality of granular bodies are different from each other, similar to the particle size distribution estimation device 1A of the first modified example of the first embodiment.

[0194] In this case, the corrected binary image generation unit 204 may obtain the size of the kernel for each of the multiple captured images based on the brightness change parameters obtained in the captured image, and use the size of the kernel obtained for the captured image in the opening processing for the captured image.

[0195] In this case, the corrected binary image generation unit 204 may obtain the size of the kernel for each of the multiple captured images based on the brightness change parameters obtained in the captured image, and determine the average value of the multiple kernel sizes obtained for each of the multiple captured images as the size of the kernel to be used in the opening process for all of the multiple captured images.

[0196] <Second Modification of Fifth Embodiment> Next, a particle size distribution estimation device according to a second modified example of the fifth embodiment will be described. The particle size distribution estimation device according to the second modified example of the fifth embodiment differs from the particle size distribution estimation device according to the first modified example of the fifth embodiment in that the kernel size is determined based on the total edge distance in addition to the brightness change parameter. The following description will focus on the differences. In the description of the second modified example of the fifth embodiment, components that are assigned the same reference numerals as those used in the first modified example of the fifth embodiment are the same or substantially similar.

[0197] (function) The corrected binary image generation unit 204 of the second variant of the fifth embodiment, like the corrected binary image generation unit 204 of the fifth embodiment, detects edges in the captured image after preprocessing has been performed by the preprocessing unit 202, and obtains the total edge distance, which is the total distance of the detected edges.

[0198] Furthermore, the corrected binary image generation unit 204 of the second modified example of the fifth embodiment, like the corrected binary image generation unit 204 of the first modified example of the fifth embodiment, acquires a brightness change parameter based on a change in brightness along a predetermined line in the captured image after preprocessing has been performed by the preprocessing unit 202.

[0199] The corrected binary image generation unit 204 determines the size of the kernel used in the opening process based on both the acquired total edge distance and the acquired brightness change parameter. In this example, the corrected binary image generation unit 204 determines the kernel size so that it becomes smaller as the acquired total edge distance becomes longer and so that it becomes smaller as the acquired brightness change parameter becomes larger.

[0200] (operation) The particle size distribution estimation device 1 of the second modified example of the fifth embodiment executes a process in which the processes of step S1021, step S1021A, and step S1022 shown in FIG. 21 are added between step S102 and step S103 of the process shown in FIG. 3.

[0201] Therefore, after performing preprocessing on the captured image, the information processing device 20 detects edges in the captured image after the preprocessing and obtains the total edge distance, which is the total distance of the detected edges (step S1021). Next, the information processing device 20 obtains brightness change parameters in the captured image after the preprocessing (step S1021A). Note that the information processing device 20 may perform the processes of step S1021 and step S1021A in the reverse order to the order shown in FIG. 21.

[0202] Next, the information processing device 20 determines the size of the kernel to be used in the opening process based on the total edge distance acquired in step S1021 and the brightness change parameter acquired in step S1021A (step S1022). Next, the information processing device 20 executes the processes of steps S103 to S107, similarly to the information processing device 20 of the first embodiment.

[0203] As described above, according to the particle size distribution estimating device 1 of the second modified example of the fifth embodiment, the same actions and effects as those of the particle size distribution estimating device 1 of the first embodiment can be achieved. Furthermore, in the particle size distribution estimation device 1 of the second modified example of the fifth embodiment, the corrected binary image generation unit 204 determines the size of the kernel based on the total distance of edges detected in the captured image and a brightness change parameter acquired based on a change in brightness along a predetermined line in the captured image.

[0204] Incidentally, there is a strong correlation between the average particle size and the width of the particle size distribution (in other words, the range of particle sizes in the particle size distribution) and the total edge distance or brightness change parameter. There is also a strong correlation between the average particle size and the width of the particle size distribution and the appropriate kernel size. Therefore, the particle size distribution estimation device 1 can extract the contour of each granular body with high accuracy. As a result, the particle size distribution can be estimated with high accuracy.

[0205] In this example, the opening process is performed for one kernel size (in other words, kernel size), so the processing load can be reduced compared to when the opening process is performed for each of a plurality of kernel sizes.

[0206] Note that, like the particle size distribution estimation device 1 of the second embodiment, the particle size distribution estimation device 1 of the second modified example of the fifth embodiment may generate multiple binary images by performing binarization processing for each of multiple threshold values ​​that are different from each other, and estimate the particle size distribution based on the generated multiple binary images.

[0207] Furthermore, the particle size distribution estimation device 1 of the second modified example of the fifth embodiment may estimate the particle size distribution based on a plurality of captured images generated by photographing a target area in a plurality of states in which the positions of at least some of the plurality of granular bodies are different from each other, similar to the particle size distribution estimation device 1A of the first modified example of the first embodiment.

[0208] In this case, the corrected binary image generation unit 204 may obtain the size of the kernel for each of the multiple captured images based on the acquired total edge distance and brightness change parameters, and use the size of the kernel obtained for the captured image in the opening processing for the captured image.

[0209] In this case, the corrected binary image generation unit 204 may acquire the size of the kernel for each of the multiple captured images based on the acquired total edge distance and brightness change parameters, and determine the average value of the multiple kernel sizes acquired for each of the multiple captured images as the size of the kernel to be used in the opening process for all of the multiple captured images.

[0210] <Third Modification of Fifth Embodiment> Next, a particle size distribution estimation device according to a third modified example of the fifth embodiment will be described. The particle size distribution estimation device according to the third modified example of the fifth embodiment differs from the particle size distribution estimation device according to the first modified example of the first embodiment in that the particle size distribution estimation device detects the resistance force that the particle movement unit receives during the movement of the particle material and determines the size of the kernel based on the detected resistance force. The following description will focus on the differences. In the description of the third modified example of the fifth embodiment, components that are assigned the same reference numerals as those used in the first modified example of the first embodiment are the same or substantially similar.

[0211] (composition) As shown in FIG. 6, a particle size distribution estimation device 1B of the third modified example of the fifth embodiment includes a granular material moving device 12B instead of the granular material moving device 12 of the first modified example of the first embodiment. The granular material moving device 12B moves at least some of the plurality of granular materials in the target area, similar to the granular material moving device 12. In this example, the granular material moving device 12B corresponds to a granular material moving section.

[0212] In this example, the granular material moving device 12B is a bucket of a construction machine. In this example, the granular material moving device 12B is driven hydraulically. Note that the granular material moving device 12B may also be driven electrically. In this example, the photographing device 11 is positioned opposite the granular material moving device 12B. Note that the photographing device 11 may also be mounted on the granular material moving device 12B.

[0213] The granular material moving device 12B is equipped with a resistance force detector 121. The resistance force detector 121 detects the resistance force that the granular material moving device 12B receives when moving the granular material. In this example, the granular material moving device 12B detects the resistance force by detecting the magnitude of the hydraulic pressure that drives the granular material moving device 12B. In this example, the resistance force detector 121 corresponds to a resistance force detection unit. The resistance detector 121 may detect the resistance using a force sensor, a pressure sensor, or a load sensor.

[0214] In this example, resistance force detector 121 detects the resistance force as well as the travel distance of the bucket. In this example, resistance force detector 121 detects the drive amount (e.g., rotation angle, displacement amount, etc.) of a drive unit (e.g., an arm, a boom, etc.) for moving the bucket, and detects the travel distance of the bucket based on the detected drive amount. For example, resistance force detector 121 may detect the drive amount by using a rotation angle sensor and a displacement sensor. In this way, in this example, the resistance force detector 121 detects the resistance force in association with the moving distance of the bucket.

[0215] (function) The corrected binary image generating unit 204 of the third modified example of the fifth embodiment determines the size of the kernel used in the opening process based on the resistance force detected by the resistance force detector 121.

[0216] In this example, corrected binary image generation unit 204 acquires a resistance force change parameter based on the change in the detected resistance force with respect to the travel distance of the bucket. Fig. 22 is a graph showing an example of the change in resistance force with respect to the travel distance of the bucket.

[0217] In this example, the resistance change parameter is an average resistance, which is an average value of the resistance over a scooping interval. The scooping interval is a range of the bucket travel distance where the resistance is greater than a predetermined threshold resistance. In this example, the corrected binary image generating unit 204 determines the size of the kernel so that it becomes smaller as the acquired resistance force change parameter becomes smaller.

[0218] The resistance force change parameter may be, for example, the wave slope or the number of waves instead of the average resistance force. The wave slope is the average value obtained by dividing the difference between successive minimum and maximum values ​​in the change in resistance force by the distance difference, which is the difference between the travel distances of the two buckets corresponding to the minimum and maximum values, respectively, over the scooping interval. The number of waves is the number of maximum values ​​in the change in resistance force over the scooping interval. The resistance force change parameter may also be a parameter determined based on a power spectrum obtained by performing frequency analysis on the change in resistance force with respect to the travel distance of the bucket.

[0219] (operation) The particle size distribution estimation device 1B of the third modified example of the fifth embodiment executes a process in which the processes of step S2011, step S2011A, and step S2012 shown in FIG. 23 are added between step S201 and step S202 of the process shown in FIG. 5.

[0220] Therefore, after the granular material moving device 12B starts moving at least some of the multiple granular materials in the target area in step S201, the resistance force detector 121 detects the resistance force that the granular material moving device 12B receives in the movement of the granular materials during the period in which the granular material is being moved by the granular material moving device 12B, in correspondence with the movement distance of the bucket (step S2011).

[0221] Next, information processing device 20 acquires a resistance force change parameter based on the resistance force detected in association with the moving distance of the bucket (step S2011A). Next, the information processing device 20 determines the size of the kernel to be used in the opening process based on the resistance force change parameter acquired in step S2011A (step S2012). Next, the information processing device 20 executes the processes of steps S202 to S210, similar to the information processing device 20 of the first modified example of the first embodiment.

[0222] As described above, according to the particle size distribution estimating device 1B of the third modified example of the fifth embodiment, the same actions and effects as those of the particle size distribution estimating device 1A of the first modified example of the first embodiment are achieved. Furthermore, the particle size distribution estimation device 1B of the third modified example of the fifth embodiment includes a resistance force detection unit (resistance force detector 121 in this example) that detects the resistance force that the granular material moving unit (granular material moving device 12B in this example) receives during the movement of the granular material. The corrected binary image generation unit 204 determines the size of the kernel based on the detected resistance force.

[0223] Incidentally, there is a strong correlation between the average particle size and the resistance. Furthermore, the smaller the average particle size, the smaller the appropriate kernel size often becomes. Therefore, the particle size distribution estimation device 1B can extract the contours of each granular body with high accuracy. As a result, the particle size distribution can be estimated with high accuracy.

[0224] Furthermore, in the particle size distribution estimation device 1B of the third modified example of the fifth embodiment, the corrected binary image generating unit 204 determines the size of the kernel so that it becomes smaller as the acquired average resistance force becomes smaller.

[0225] Incidentally, the smaller the average particle size, the smaller the average resistance force often becomes. Furthermore, the smaller the average particle size, the smaller the appropriate kernel size often becomes. Therefore, the particle size distribution estimation device 1B can extract the contours of each granular body with high accuracy. As a result, the particle size distribution can be estimated with high accuracy.

[0226] In this example, the opening process is performed for one kernel size (in other words, kernel size), so the processing load can be reduced compared to when the opening process is performed for each of a plurality of kernel sizes.

[0227] Note that, like the particle size distribution estimation device 1A of the first modified example of the second embodiment, the particle size distribution estimation device 1B of the third modified example of the fifth embodiment may generate multiple binary images by performing binarization processing for each of multiple threshold values ​​that are different from each other, and estimate the particle size distribution based on the generated multiple binary images.

[0228] Furthermore, the particle size distribution estimation device 1B of the third modified example of the fifth embodiment may scoop up the granular material with the bucket multiple times. In this case, the corrected binary image generation unit 204 may acquire the size of the kernel for each of the multiple scoopings based on the resistance force detected in association with the scooping, and may use the size of the kernel acquired for the scooping in the opening process for the multiple captured images acquired in association with the scooping.

[0229] In this case, the corrected binary image generation unit 204 may obtain the size of the kernel for each of the multiple scoopings based on the resistance force detected in connection with the scooping, and may determine the average value of the multiple kernel sizes obtained for each of the multiple scoopings as the kernel size to be used in the opening process for all of the multiple captured images obtained in connection with the multiple scoopings.

[0230] <Fourth Modification of Fifth Embodiment> Next, a particle size distribution estimation device according to a fourth modified example of the fifth embodiment will be described. The particle size distribution estimation device according to the fourth modified example of the fifth embodiment differs from the particle size distribution estimation device according to the third modified example of the fifth embodiment in that the kernel size is determined based on the total edge distance in addition to the resistance force change parameter. The following description will focus on the differences. In the description of the fourth modified example of the fifth embodiment, components that are assigned the same reference numerals as those used in the third modified example of the fifth embodiment are the same or substantially similar.

[0231] (function) The corrected binary image generation unit 204 of the fourth modified example of the fifth embodiment, like the corrected binary image generation unit 204 of the third modified example of the fifth embodiment, acquires a resistance force change parameter based on the change in the resistance force detected by the resistance force detector 121 relative to the bucket movement distance. Furthermore, the corrected binary image generation unit 204 of the fourth variant of the fifth embodiment, like the corrected binary image generation unit 204 of the fifth embodiment, detects edges in the captured image after preprocessing has been performed by the preprocessing unit 202, and obtains the total edge distance, which is the total distance of the detected edges.

[0232] The corrected binary image generation unit 204 determines the size of the kernel used in the opening process based on both the acquired resistance force change parameter and the acquired total edge distance. In this example, the corrected binary image generation unit 204 determines the kernel size so that it becomes smaller as the acquired resistance force change parameter becomes smaller and so that it becomes smaller as the acquired total edge distance becomes longer.

[0233] (operation) The particle size distribution estimation device 1B of the fourth modified example of the fifth embodiment executes a process in which the processes of step S2011 and step S2011A shown in FIG. 24A are added between step S201 and step S202 of the process shown in FIG. 5, and the processes of step S2041 and step S2042 shown in FIG. 24B are added between step S204 and step S205 of the process shown in FIG. 5.

[0234] Therefore, after the granular material moving device 12B starts moving at least some of the multiple granular materials in the target area in step S201, the resistance force detector 121 detects the resistance force that the granular material moving device 12B receives in the movement of the granular materials during the period in which the granular material is being moved by the granular material moving device 12B, in correspondence with the movement distance of the bucket (step S2011).

[0235] Next, information processing device 20 acquires a resistance force change parameter based on the resistance force detected in association with the moving distance of the bucket (step S2011A). Next, similar to the information processing device 20 of the first modified example of the first embodiment, the information processing device 20 performs the processes of steps S202 to S204, and then detects edges in the captured image after preprocessing, and obtains the total edge distance, which is the total distance of the detected edges (step S2041).

[0236] Next, the information processing device 20 determines the size of the kernel to be used in the opening process based on the resistance force change parameter acquired in step S2011A and the total edge distance acquired in step S2041 (step S2042). Next, the information processing device 20 executes the processes of steps S205 to S210, similar to the information processing device 20 of the first modified example of the first embodiment.

[0237] As described above, according to the particle size distribution estimating device 1B of the fourth modified example of the fifth embodiment, the same actions and effects as those of the particle size distribution estimating device 1B of the third modified example of the fifth embodiment can be achieved. Furthermore, in the particle size distribution estimation device 1B of the fourth modified example of the fifth embodiment, the corrected binary image generation unit 204 determines the size of the kernel based on the total distance of the edges detected in the captured image in addition to the resistance force change parameter.

[0238] Incidentally, the smaller the average particle size, the smaller the average resistance force often becomes. Also, the smaller the average particle size, the smaller the appropriate kernel size often becomes. Furthermore, there is a strong correlation between the average particle size and the width of the particle size distribution (in other words, the range of particle sizes in the particle size distribution) and the total distance of the edge. There is also a strong correlation between the average particle size and the width of the particle size distribution and the appropriate kernel size. Therefore, the particle size distribution estimation device 1B can extract the contour of each granular body with high accuracy. As a result, the particle size distribution can be estimated with high accuracy.

[0239] In this example, the opening process is performed for one kernel size (in other words, kernel size), so the processing load can be reduced compared to when the opening process is performed for each of a plurality of kernel sizes.

[0240] The particle size distribution estimation device 1B of the fourth modified example of the fifth embodiment may use a brightness change parameter instead of the total distance of the edges.

[0241] Furthermore, similar to the particle size distribution estimation device 1A of the first modified example of the second embodiment, the particle size distribution estimation device 1B of the fourth modified example of the fifth embodiment may generate multiple binary images by performing binarization processing for each of multiple threshold values ​​that are different from each other, and estimate the particle size distribution based on the generated multiple binary images.

[0242] In addition, the corrected binary image generation unit 204 may obtain the size of the kernel for each of the multiple captured images based on the total distance of the edges detected in the captured image and the resistance force change parameter, and determine the average value of the multiple kernel sizes obtained for each of the multiple captured images as the size of the kernel to be used in the opening process for all of the multiple captured images.

[0243] Furthermore, the particle size distribution estimation device 1B of the fourth modified example of the fifth embodiment may scoop up the granular material with the bucket multiple times. In this case, the corrected binary image generation unit 204 may acquire the size of the kernel for each of the multiple scoopings based on the resistance force detected in association with the scooping, and may use the size of the kernel acquired for the scooping in the opening process for the multiple captured images acquired in association with the scooping.

[0244] In this case, the corrected binary image generation unit 204 may obtain the size of the kernel for each of the multiple scoopings based on the resistance force detected in connection with the scooping, and may determine the average value of the multiple kernel sizes obtained for each of the multiple scoopings as the kernel size to be used in the opening process for all of the multiple captured images obtained in connection with the multiple scoopings.

[0245] Sixth Embodiment Next, a particle size distribution estimation device of the sixth embodiment will be described. The particle size distribution estimation device of the sixth embodiment differs from the particle size distribution estimation device of the third embodiment in that the first kernel size is determined based on the total distance of edges, and the second kernel size is determined based on a brightness change parameter. The following description will focus on the differences. In the description of the sixth embodiment, components that are assigned the same reference numerals as those used in the third embodiment are the same or substantially similar.

[0246] (function) The corrected binary image generation unit 204 of the sixth embodiment, like the corrected binary image generation unit 204 of the fifth embodiment, detects edges in the captured image after preprocessing has been performed by the preprocessing unit 202, and obtains the total edge distance, which is the total distance of the detected edges.

[0247] Furthermore, the corrected binary image generation unit 204 of the sixth embodiment, like the corrected binary image generation unit 204 of the first modified example of the fifth embodiment, acquires a brightness change parameter based on the change in brightness along a predetermined line in the captured image after preprocessing has been performed by the preprocessing unit 202.

[0248] In this example, the multiple kernel sizes used by the corrected binary image generation unit 204 include a first kernel size and a second kernel size. In other words, in this example, the number of kernel sizes used by the corrected binary image generation unit 204 is two. In this example, the first kernel size is larger than the second kernel size.

[0249] The corrected binary image generation unit 204 determines the first kernel size based on the acquired total edge distance. In this example, the corrected binary image generation unit 204 determines the first kernel size so that the first kernel size decreases as the acquired total edge distance increases.

[0250] Furthermore, the corrected binary image generation unit 204 determines a second kernel size based on the determined first kernel size and the acquired brightness change parameter. In this example, the corrected binary image generation unit 204 determines the second kernel size so that the smaller the determined first kernel size is, and the smaller the acquired brightness change parameter is. The corrected binary image generating unit 204 may determine the second kernel size based on the acquired brightness change parameter, without based on the determined first kernel size.

[0251] (operation) The particle size distribution estimation device 1 of the sixth embodiment executes processing in which the processes of steps S5021, S5022, and S5023 shown in FIG. 25 are added between steps S502 and S503 of the processing shown in FIG.

[0252] Therefore, after performing preprocessing on the captured image, the information processing device 20 detects edges in the captured image after the preprocessing and obtains the total edge distance, which is the total distance of the detected edges (step S5021). Next, the information processing device 20 obtains brightness change parameters in the captured image after the preprocessing (step S5022). Note that the information processing device 20 may perform the processes of steps S5021 and S5022 in the reverse order to the order shown in FIG. 25.

[0253] Next, the information processing device 20 determines each kernel size to be used in the opening process based on the total edge distance acquired in step S5021 and the brightness change parameter acquired in step S5022 (step S5023). Next, the information processing device 20 executes the processes of steps S503 to S510, similarly to the information processing device 20 of the third embodiment.

[0254] As described above, according to the particle size distribution estimating device 1 of the sixth embodiment, the same actions and effects as those of the particle size distribution estimating device 1 of the third embodiment can be achieved. Furthermore, in the particle size distribution estimation device 1 of the sixth embodiment, the multiple kernel sizes include a first kernel size and a second kernel size. The corrected binarized image generation unit 204 determines the first kernel size based on one of the total distance of edges detected in the captured image and a brightness change parameter acquired based on a change in brightness along a predetermined line in the captured image, and determines the second kernel size based on the other of the total distance of edges and the brightness change parameter.

[0255] The inventors of the present application have discovered that when the width of the particle size distribution is relatively wide, the contours of each granular body can be extracted with sufficiently high accuracy by performing opening processing for each of the two kernel sizes.

[0256] Incidentally, there is a strong correlation between the average particle size and the width of the particle size distribution (in other words, the range of particle sizes in the particle size distribution) and the total distance of the edge or the brightness change parameter. There is also a strong correlation between the average particle size and the width of the particle size distribution and the appropriate kernel size. Therefore, when the width of the particle size distribution is relatively wide, the particle size distribution estimation device 1 can extract the contour of each granular object with sufficiently high accuracy. As a result, the particle size distribution can be estimated with high accuracy.

[0257] Furthermore, in the particle size distribution estimation device 1 of the sixth embodiment, the corrected binary image generating unit 204 determines the first kernel size so that it becomes smaller as the total distance of edges detected in the captured image becomes longer.

[0258] Incidentally, the smaller the average particle size, the longer the total distance of the edge often becomes. Furthermore, the smaller the average particle size, the smaller the appropriate kernel size often becomes. Therefore, the particle size distribution estimation device 1 can extract the contour of each granular body with high accuracy. As a result, the particle size distribution can be estimated with high accuracy.

[0259] Furthermore, in the particle size distribution estimation device 1 of the sixth embodiment, the corrected binary image generation unit 204 determines the second kernel size so that the second kernel size decreases as the brightness change parameter acquired based on the change in brightness along a predetermined line in the captured image increases.

[0260] Incidentally, the wider the width of the particle size distribution, the larger the brightness change parameter in many cases. Furthermore, the wider the width of the particle size distribution, the smaller the kernel size appropriate for the second kernel size in many cases. Therefore, the particle size distribution estimation device 1 can extract the contours of each granular body with high accuracy. As a result, the particle size distribution can be estimated with high accuracy.

[0261] The particle size distribution estimation device 1 of the sixth embodiment may determine the first kernel size based on a brightness change parameter instead of the total edge distance. In this case, the particle size distribution estimation device 1 may determine the second kernel size based on the determined first kernel size and the total edge distance. In this case, the particle size distribution estimation device 1 may determine the second kernel size based on the total edge distance, not based on the determined first kernel size.

[0262] Furthermore, similar to the particle size distribution estimation device 1 of the fourth embodiment, the particle size distribution estimation device 1 of the sixth embodiment may generate multiple binary images by performing binarization processing for each of multiple thresholds that are different from each other, and estimate the particle size distribution based on the generated multiple binary images.

[0263] Furthermore, the particle size distribution estimation device 1 of the sixth embodiment may estimate the particle size distribution based on a plurality of captured images generated by photographing a target area in a plurality of states in which the positions of at least some of the plurality of granular bodies are different from each other, similar to the particle size distribution estimation device 1A of the first modified example of the first embodiment.

[0264] In this case, the corrected binary image generation unit 204 may obtain a first kernel size and a second kernel size for each of the multiple captured images based on the total distance of edges detected in the captured image and a brightness change parameter obtained based on the change in brightness along a predetermined line in the captured image, and may use the first kernel size and the second kernel size obtained for the captured image in the opening process for the captured image.

[0265] In this case, the corrected binary image generation unit 204 may acquire a first kernel size and a second kernel size for each of the multiple captured images based on the total distance of edges detected in the captured image and a brightness change parameter acquired based on a change in brightness along a predetermined line in the captured image. Furthermore, in this case, the corrected binary image generation unit 204 may determine an average value of the multiple first kernel sizes acquired for each of the multiple captured images as the first kernel size to be used in the opening process for all of the multiple captured images. In addition, in this case, the corrected binary image generation unit 204 may determine an average value of the multiple second kernel sizes acquired for each of the multiple captured images as the second kernel size to be used in the opening process for all of the multiple captured images.

[0266] Seventh Embodiment Next, a particle size distribution estimation device of the seventh embodiment will be described. The particle size distribution estimation device of the seventh embodiment differs from the particle size distribution estimation device of the fourth embodiment in that it estimates particle size distribution based on a plurality of captured images generated by capturing images of a target area in a plurality of states in which the positions of at least some of a plurality of granular materials are different from one another, and in that it detects the resistance force that a granular material moving unit experiences during the movement of the granular materials, determines a first kernel size based on the detected resistance force, and determines a second kernel size based on the total edge distance. The following description will focus on the differences. Note that in the description of the seventh embodiment, parts that are assigned the same reference numerals as those used in the fourth embodiment are the same or substantially similar.

[0267] (composition) As shown in FIG. 6, the particle size distribution estimation device 1B of the seventh embodiment includes, in addition to the components of the particle size distribution estimation device 1 of the fourth embodiment, a granular material moving device 12B, similar to the particle size distribution estimation device 1B of the third modified example of the fifth embodiment. The granular material moving device 12B moves at least some of the granular materials in the target area. In this example, the granular material moving device 12B corresponds to a granular material moving section.

[0268] In this example, the granular material moving device 12B is a bucket of a construction machine. In this example, the granular material moving device 12B is driven hydraulically. Note that the granular material moving device 12B may also be driven electrically. In this example, the photographing device 11 is positioned opposite the granular material moving device 12B. Note that the photographing device 11 may also be mounted on the granular material moving device 12B.

[0269] The granular material moving device 12B is equipped with a resistance force detector 121. The resistance force detector 121 detects the resistance force that the granular material moving device 12B receives when moving the granular material. In this example, the granular material moving device 12B detects the resistance force by detecting the magnitude of the hydraulic pressure that drives the granular material moving device 12B. In this example, the resistance force detector 121 corresponds to a resistance force detection unit. The resistance detector 121 may detect the resistance using a force sensor, a pressure sensor, or a load sensor.

[0270] In this example, resistance force detector 121 detects the resistance force as well as the travel distance of the bucket. In this example, resistance force detector 121 detects the drive amount (e.g., rotation angle, displacement amount, etc.) of a drive unit (e.g., an arm, a boom, etc.) for moving the bucket, and detects the travel distance of the bucket based on the detected drive amount. For example, resistance force detector 121 may detect the drive amount by using a rotation angle sensor and a displacement sensor. In this way, in this example, the resistance force detector 121 detects the resistance force in association with the moving distance of the bucket.

[0271] (function) The captured image acquisition unit 201 of the seventh embodiment acquires multiple captured images that are each generated by photographing the target area in multiple states in which the positions of at least some of the multiple granular bodies are different from each other due to the movement of the granular bodies by the granular body moving device 12B.

[0272] The pre-processing unit 202 of the seventh embodiment performs pre-processing on each of the multiple captured images acquired by the captured image acquisition unit 201.

[0273] The binary image generation unit 203 of the seventh embodiment generates multiple binary images by performing binary processing using multiple different thresholds on the captured image after preprocessing by the preprocessing unit 202 for each captured image.

[0274] The corrected binary image generation unit 204 of the seventh embodiment generates multiple corrected binary images by performing opening processing for each of multiple different kernel sizes (in other words, kernel sizes) for each binary image generated by the binary image generation unit 203 for each captured image.

[0275] The boundary information generating unit 205 of the seventh embodiment generates a plurality of pieces of boundary information by performing boundary extraction processing on each of a plurality of corrected binary images generated for each binary image for each captured image.

[0276] The contour group information generation unit 206 of the seventh embodiment generates contour group information for each captured image by performing a rejection process on each of the multiple pieces of boundary information generated for each of the multiple corrected binary images generated for each binary image.

[0277] The particle size distribution estimation unit 207 of the seventh embodiment estimates the particle size distribution based on a plurality of pieces of contour group information generated by the contour group information generation unit 206.

[0278] In this example, the multiple kernel sizes used by the corrected binary image generation unit 204 include a first kernel size and a second kernel size. In other words, in this example, the number of kernel sizes used by the corrected binary image generation unit 204 is two. In this example, the first kernel size is larger than the second kernel size.

[0279] The corrected binary image generation unit 204 determines the first kernel size based on the resistance force detected by the resistance force detector 121. In this example, the corrected binary image generation unit 204 acquires a resistance force change parameter based on a change in the detected resistance force relative to the moving distance of the bucket.

[0280] In this example, the resistance change parameter is an average resistance, which is an average value of the resistance over a scooping interval. The scooping interval is a range of the bucket travel distance where the resistance is greater than a predetermined threshold resistance. In this example, the corrected binary image generating unit 204 determines the first kernel size so that it decreases as the acquired resistance force change parameter decreases.

[0281] The resistance force change parameter may be, for example, the wave slope or the number of waves instead of the average resistance force. The wave slope is the average value obtained by dividing the difference between successive minimum and maximum values ​​in the change in resistance force by the distance difference, which is the difference between the travel distances of the two buckets corresponding to the minimum and maximum values, respectively, over the scooping interval. The number of waves is the number of maximum values ​​in the change in resistance force over the scooping interval. The resistance force change parameter may also be a parameter determined based on a power spectrum obtained by performing frequency analysis on the change in resistance force with respect to the travel distance of the bucket.

[0282] Furthermore, similar to the corrected binary image generation unit 204 of the fifth embodiment, the corrected binary image generation unit 204 detects edges in the captured image after preprocessing has been performed by the preprocessing unit 202, and obtains the total edge distance, which is the total distance of the detected edges.

[0283] The corrected binary image generation unit 204 determines a second kernel size based on the determined first kernel size and the acquired total edge distance. In this example, the corrected binary image generation unit 204 determines the second kernel size so that the smaller the determined first kernel size is, and the smaller the acquired total edge distance is. The corrected binary image generating unit 204 may determine the second kernel size based on the acquired total edge distance, without based on the determined first kernel size.

[0284] (operation) The particle size distribution estimation device 1B of the seventh embodiment starts to execute the process shown in FIG. 26 instead of the process shown in FIG. The granular material moving device 12B moves at least some of the plurality of granular materials in the target area (step S701).

[0285] During the period in which the granular material is being moved by the granular material moving device 12B, the resistance force detector 121 detects the resistance force that the granular material moving device 12B receives in the movement of the granular material in association with the moving distance of the bucket (step S702).

[0286] Next, information processing device 20 acquires a resistance change parameter based on the resistance detected in association with the moving distance of the bucket (step S703).

[0287] In parallel with the processing of steps S702 and S703, the information processing device 20 transmits M photographing commands to the photographing device 11 at M different times during the period in which the granular material is being moved by the granular material moving device 12, and thereby receives M photographed images output from the photographing device 11, thereby acquiring the M photographed images (step S704), where M represents an integer of 2 or greater.

[0288] Next, the information processing device 20 executes a first loop process (steps S705 to S710) in which each of the M captured images acquired in step S704 is used as a processing target one by one. In the first loop process, the information processing device 20 performs pre-processing on the captured image to be processed (step S706), similarly to step S602 in FIG.

[0289] Next, similarly to the information processing device 20 of the fifth embodiment, the information processing device 20 detects edges in the photographed image after preprocessing and obtains the total edge distance, which is the total distance of the detected edges (step S707).

[0290] Next, the information processing device 20 determines each kernel size used in the opening process based on the resistance force change parameter acquired in step S703 and the total edge distance acquired in step S707 (step S708). Next, the information processing device 20 executes the processes of steps S603 to S611 in FIG. 16, similarly to the information processing device 20 of the fourth embodiment (step S709).

[0291] Then, the information processing device 20 executes the first loop process (steps S705 to S710) for all of the acquired M photographed images, and then proceeds to step S711. Next, the information processing device 20 estimates a particle size distribution based on the contour group information generated for each of the M photographed images (step S711). As a result, the information processing device 20 ends the execution of the process shown in FIG.

[0292] As described above, according to the particle size distribution estimating device 1B of the seventh embodiment, the same actions and effects as those of the particle size distribution estimating device 1 of the fourth embodiment can be achieved. Furthermore, the particle size distribution estimation device 1B of the seventh embodiment includes a granular material moving section (in this example, a granular material moving device 12B) that moves at least a portion of the plurality of granular materials in the target area, and a resistance force detection section (in this example, a resistance force detector 121) that detects the resistance force that the granular material moving section receives during movement.

[0293] The plurality of kernel sizes include a first kernel size and a second kernel size. The corrected binary image generation unit 204 determines the first kernel size based on the detected resistance force, and determines the second kernel size based on a brightness change parameter obtained based on the total distance of an edge detected in the captured image or a brightness change along a predetermined line in the captured image.

[0294] The inventors of the present application have discovered that when the width of the particle size distribution is relatively wide, the contours of each granular body can be extracted with sufficiently high accuracy by performing opening processing for each of the two kernel sizes.

[0295] Incidentally, there is a strong correlation between the average particle size and resistance. Also, the smaller the average particle size, the smaller the appropriate kernel size often becomes. Furthermore, there is a strong correlation between the width of the particle size distribution (in other words, the range of particle sizes in the particle size distribution) and the total edge distance or brightness change parameter. There is also a strong correlation between the width of the particle size distribution and the appropriate kernel size.

[0296] Therefore, when the width of the particle size distribution is relatively wide, the particle size distribution estimation device 1B can extract the contour of each granular material with sufficiently high accuracy, thereby estimating the particle size distribution with high accuracy.

[0297] Furthermore, in the particle size distribution estimation device 1B of the seventh embodiment, the corrected binarized image generating unit 204 determines the first kernel size so that it becomes smaller as the resistance change parameter becomes smaller.

[0298] Incidentally, the smaller the average particle size, the smaller the resistance change parameter often becomes. Furthermore, the smaller the average particle size, the smaller the appropriate kernel size often becomes. Therefore, the particle size distribution estimation device 1B can extract the contours of each granular body with high accuracy. As a result, the particle size distribution can be estimated with high accuracy.

[0299] Furthermore, in the particle size distribution estimation device 1B of the seventh embodiment, the corrected binary image generating unit 204 determines the second kernel size so that it becomes smaller as the total distance of edges detected in the captured image becomes longer.

[0300] Incidentally, the wider the width of the particle size distribution, the longer the total distance of the edges often becomes. Furthermore, the wider the width of the particle size distribution, the smaller the kernel size appropriate for the second kernel size often becomes. Therefore, the particle size distribution estimation device 1B can extract the contours of each granular body with high accuracy. As a result, the particle size distribution can be estimated with high accuracy.

[0301] The particle size distribution estimation device 1B of the seventh embodiment may determine the second kernel size based on a brightness change parameter instead of the total edge distance. Furthermore, the particle size distribution estimation device 1B of the seventh embodiment may estimate the particle size distribution by performing binarization processing on only one threshold value.

[0302] In addition, the corrected binary image generation unit 204 may obtain a second kernel size for each of the multiple captured images based on the total distance of the edges detected in the captured image, and determine the average value of the multiple second kernel sizes obtained for each of the multiple captured images as the second kernel size to be used in the opening processing for all of the multiple captured images.

[0303] Furthermore, the particle size distribution estimation device 1B of the seventh embodiment may scoop up the granular material with the bucket multiple times. In this case, the corrected binary image generation unit 204 may acquire a first kernel size for each of the multiple scoopings based on the resistance force detected in association with the scooping, and may use the first kernel size acquired for the scooping in the opening process for the multiple captured images acquired in association with the scooping.

[0304] In this case, the corrected binary image generation unit 204 may acquire a first kernel size for each of the multiple scoopings based on the resistance force detected in association with the scooping, and determine the average value of the multiple first kernel sizes acquired for each of the multiple scoopings as the first kernel size to be used in the opening process for all of the multiple captured images acquired in association with the multiple scoopings.

[0305] The present invention is not limited to the above-described embodiment. For example, various modifications that can be understood by a person skilled in the art may be made to the above-described embodiment without departing from the spirit of the present invention. [Explanation of symbols]

[0306] 1,1A,1B Particle size distribution estimation device 11 Imaging equipment 12,12B Granule moving device 121 Resistance Detector 20 Information processing equipment 21 Processing equipment 22 Storage device 23 Connection Device 201 Image acquisition unit 202 Pre-processing section 203 Binarized Image Generation Unit 204 Corrected binary image generation unit 205 Boundary information generation unit 206 Contour group information generation unit 207 Particle size distribution estimation section

Claims

1. a captured image acquisition unit that acquires a captured image, which is an image generated by photographing a target area including a plurality of granular bodies at least partially overlapping each other; a binary image generating unit that generates a binary image by performing a binarization process on the captured image to convert values ​​of each of a plurality of pixels that constitute the captured image into a first value or a second value based on a threshold value; a corrected binary image generation unit that generates a corrected binary image by performing an opening process including an erosion process and an expansion process, which are filter processes using a kernel, on the binary image; a boundary information generating unit that generates boundary information representing the boundary by performing a boundary extraction process on the corrected binarized image to extract a boundary between a region having the first value and a region having the second value; a contour group information generation unit that performs a rejection process on the boundary information to reject contour candidates that satisfy a predetermined rejection condition from a contour candidate group that is a group of closed curves included in the boundary represented by the boundary information, thereby generating contour group information that represents contour candidates that do not satisfy the rejection condition from among the contour candidate group; a particle size distribution estimation unit that estimates a particle size distribution, which is a distribution of particle sizes of the plurality of granular bodies, based on the contour group information; A particle size distribution estimation device comprising:

2. The particle size distribution estimation device according to claim 1, a granular material moving unit that moves at least some of the plurality of granular materials in the target area; a resistance force detection unit that detects a resistance force that the granular material moving unit receives during the movement; Equipped with The particle size distribution estimation device, wherein the corrected binary image generation unit determines the size of the kernel based on the detected resistance force.

3. The particle size distribution estimation device according to claim 2, the corrected binary image generation unit determines the size of the kernel based on a total distance of edges detected in the captured image or a brightness change parameter acquired based on a change in brightness along a predetermined line in the captured image.

4. The particle size distribution estimation device according to claim 1, the corrected binary image generation unit determines the size of the kernel based on at least one of a total distance of edges detected in the captured image and a brightness change parameter acquired based on a change in brightness along a predetermined line in the captured image.

5. The particle size distribution estimation device according to any one of claims 1 to 4, the binary image generating unit generates a plurality of the binary images by performing the binarization process for each of a plurality of the threshold values ​​that are different from one another; the corrected binary image generation unit generates a plurality of the corrected binary images by performing the opening process on each of the plurality of binary images; the boundary information generation unit generates a plurality of pieces of boundary information by performing the boundary extraction process on each of the plurality of corrected binarized images; the contour group information generation unit generates the contour group information by performing the rejection process on the plurality of pieces of boundary information; The particle size distribution estimation device, wherein the rejection condition includes an overlap rejection condition that the center of another contour candidate is included inside the contour candidate being judged.

6. The particle size distribution estimation device according to claim 1, the corrected binary image generation unit generates a plurality of the corrected binary images by performing the opening process on each of a plurality of kernel sizes that are different from one another, the kernel sizes being the sizes of the kernels; the boundary information generation unit generates a plurality of pieces of boundary information by performing the boundary extraction process on each of the plurality of corrected binarized images; the contour group information generation unit generates the contour group information by performing the rejection process on the plurality of pieces of boundary information; The particle size distribution estimation device, wherein the rejection condition includes an overlap rejection condition that the center of another contour candidate is included inside the contour candidate being judged.

7. The particle size distribution estimation device according to claim 6, a granular material moving unit that moves at least some of the plurality of granular materials in the target area; a resistance force detection unit that detects a resistance force that the granular material moving unit receives during the movement; Equipped with the plurality of kernel sizes includes a first kernel size and a second kernel size; the corrected binary image generation unit determines the first kernel size based on the detected resistance, and determines the second kernel size based on a total distance of edges detected in the captured image or a brightness change parameter acquired based on a change in brightness along a predetermined line in the captured image.

8. The particle size distribution estimation device according to claim 6, the plurality of kernel sizes includes a first kernel size and a second kernel size; the corrected binary image generation unit determines the first kernel size based on one of a total distance of edges detected in the captured image and a brightness change parameter acquired based on a change in brightness along a predetermined line in the captured image, and determines the second kernel size based on the other of the total distance of the edges and the brightness change parameter.

9. The particle size distribution estimation device according to any one of claims 1 to 4 and claims 6 to 8, the rejection condition includes at least one of a first shape rejection condition that a circularity parameter, which increases as the shape of the contour candidate to be determined approaches a circle, is outside a first range, and a second shape rejection condition that a rectangularity parameter, which increases as the shape of the contour candidate to be determined approaches a rectangle, is outside a second range.

10. The particle size distribution estimation device according to any one of claims 1 to 8, further comprising: a granular material moving unit that moves at least some of the granular materials in the target area; The photographed image acquisition unit acquires a plurality of photographed images generated by photographing the target area in a plurality of states in which positions of at least some of the plurality of granular bodies are different from each other, by the movement, the binary image generation unit generates a plurality of binary images by performing a binary process on each of the plurality of captured images; the corrected binary image generation unit generates a plurality of the corrected binary images by performing the opening process on each of the plurality of binary images; the boundary information generation unit generates a plurality of pieces of boundary information by performing the boundary extraction process on each of the plurality of corrected binarized images; the contour group information generation unit generates a plurality of pieces of contour group information by performing the rejection process on each of the plurality of pieces of boundary information; The particle size distribution estimation unit estimates the particle size distribution based on the plurality of pieces of contour group information.

11. Acquire a captured image, which is an image generated by capturing an image of a target area including a plurality of granular bodies at least partially overlapping one another; performing a binarization process on the captured image to convert values ​​of each of a plurality of pixels constituting the captured image into a first value or a second value based on a threshold value, thereby generating a binarized image; performing an opening process including an erosion process and an expansion process, which are filter processes using a kernel, on the binarized image to generate a corrected binarized image; performing a boundary extraction process on the corrected binarized image to extract a boundary between an area having the first value and an area having the second value, thereby generating boundary information representing the boundary; performing a rejection process on the boundary information to reject contour candidates that satisfy a predetermined rejection condition from a contour candidate group that is a group of closed curves included in the boundary represented by the boundary information, thereby generating contour group information that represents contour candidates that do not satisfy the rejection condition from the contour candidate group; estimating a particle size distribution, which is a distribution of particle sizes of the plurality of granular bodies, based on the contour group information; A method for estimating particle size distribution, comprising:

12. Acquire a captured image, which is an image generated by capturing an image of a target area including a plurality of granular bodies at least partially overlapping one another; performing a binarization process on the captured image to convert values ​​of each of a plurality of pixels constituting the captured image into a first value or a second value based on a threshold value, thereby generating a binarized image; performing an opening process including an erosion process and an expansion process, which are filter processes using a kernel, on the binarized image to generate a corrected binarized image; performing a boundary extraction process on the corrected binarized image to extract a boundary between an area having the first value and an area having the second value, thereby generating boundary information representing the boundary; performing a rejection process on the boundary information to reject contour candidates that satisfy a predetermined rejection condition from a contour candidate group that is a group of closed curves included in the boundary represented by the boundary information, thereby generating contour group information that represents contour candidates that do not satisfy the rejection condition from the contour candidate group; estimating a particle size distribution, which is a distribution of particle sizes of the plurality of granular bodies, based on the contour group information; A particle size distribution estimation program that causes a computer to execute processing including the above.

Images