Aggregate quality estimation method, quality estimation program, aggregate quality estimation device, ready-mixed concrete manufacturing method, and ready-mixed concrete manufacturing system

Abstract

To improve prediction accuracy of a prediction model constructed by machine learning.SOLUTION: Acquiring color captured image data obtained by capturing an image of an aggregate, generating input image data by sequentially performing first processing for grayscaling, second processing for darkening, and third processing for binarization on the captured image data, acquiring input information including the input image data, and using a prediction model constructed in advance by machine learning so as to output a physical property value of the aggregate according to an input of the input information, A prediction step of acquiring the physical property value corresponding to the input information acquired in the second acquisition step.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to an aggregate quality prediction method, a quality prediction program, an aggregate quality prediction device, a ready-mix concrete manufacturing method, and a ready-mix concrete manufacturing system. [Background technology]

[0002] Patent Document 1 discloses an aggregate quality determination system for determining the quality of aggregate, and an aggregate quality determination method. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-137651 Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides an aggregate quality prediction method, a quality prediction program, an aggregate quality prediction device, a ready-mix concrete manufacturing method, and a ready-mix concrete manufacturing system that are useful for improving the prediction accuracy of a prediction model constructed using machine learning. [Means for solving the problem]

[0005] [1] A method for predicting the quality of aggregates, comprising: a first acquisition step of acquiring color image data obtained by imaging aggregates; a pre-processing step of generating input image data by sequentially performing a first process of grayscaling the image data, a second process of darkening the image data, and a third process of binarizing the image data; a second acquisition step of acquiring input information including the input image data; and a prediction step of acquiring the physical property values ​​corresponding to the input information acquired in the second acquisition step, using a prediction model previously constructed by machine learning to output the physical property values ​​of the aggregates according to the input of the input information.

[0006] [2] In the second process, a darkening process is performed using any one of the following equations (1), (2), (3), and (4). Bi = Ai × C (1) Bi=MAX[(Ai-D),0] (2) Bi=MAX[(Ai-E),0] (3) Bi = MAX[(Ai × CE),0] (4) Ai: pixel value of each pixel in the image before processing, Bi: pixel value of each pixel in the image after processing, C: value obtained by dividing B0 by A0, D: value obtained by subtracting A0 by B0, A0: average pixel value in the image before processing, B0: target value set in advance for pixel value, E: arbitrarily specified value, MAX[a,b]: operation that outputs the larger of the numerical value a and the numerical value b.

[0007] [3] The method for predicting quality of aggregate according to [1] or [2] above, wherein the third process involves performing binarization by adaptive binarization or Otsu's binarization.

[0008] [4] A method for predicting the quality of aggregates described in any one of [1] to [3] above, further comprising: a preparation step of preparing training data including training input information corresponding to the input information and correct answer information for the physical property values ​​associated with the training input information; and a construction step of constructing the prediction model by machine learning based on the training data, wherein the preparation step includes generating extended image data by applying a conversion process and the first process in any order to training captured image data corresponding to the captured image data, wherein the conversion process includes randomly changing the brightness and contrast of the image, and the input image data in the training input information includes image data obtained by applying the second process and the third process in order to the extended image data.

[0009] [5] The method for predicting quality of aggregates according to [4] above, wherein the transformation process further includes at least one of flipping the image and rotating the image.

[0010] [6] A quality prediction program for causing a computer to execute the aggregate quality prediction method according to any one of [1] to [5] above.

[0011] [7] An aggregate quality prediction device comprising: a first acquisition unit that acquires color image data obtained by imaging aggregate; a pre-processing unit that generates input image data by sequentially performing a first process of grayscaling the image data, a second process of darkening the image data, and a third process of binarizing the image data; a second acquisition unit that acquires input information including the input image data; and a prediction unit that acquires the physical property values ​​according to the input information acquired by the second acquisition unit using a prediction model that has been constructed in advance by machine learning to output the physical property values ​​of the aggregate according to the input information.

[0012] [8] A method for producing ready-mixed concrete, comprising: a manufacturing process for mixing materials including aggregate to produce ready-mixed concrete; and a quality prediction process for predicting the quality of at least a portion of the aggregate used in the manufacturing process, wherein the quality prediction process comprises: a first acquisition process for acquiring color image data obtained by imaging the aggregate; a pre-processing process for generating input image data by sequentially performing a first process for grayscaling the image data, a second process for darkening the image data, and a third process for binarizing the image data; a second acquisition process for acquiring input information including the input image data; and a prediction process for acquiring the physical property values ​​according to the input information acquired in the second acquisition process, using a prediction model previously constructed by machine learning to output the physical property values ​​of the aggregate according to the input of the input information.

[0013] [9] A ready-mixed concrete manufacturing system comprising: a manufacturing apparatus for kneading materials including aggregate to manufacture ready-mixed concrete; and a quality prediction device for predicting the quality of at least a portion of the aggregate used by the manufacturing apparatus, wherein the quality prediction device has: a first acquisition unit that acquires color image data obtained by imaging the aggregate; a pre-processing unit that generates input image data by sequentially performing a first process of grayscaling the image data, a second process of darkening the image data, and a third process of binarizing the image data; a second acquisition unit that acquires input information including the input image data; and a prediction unit that acquires the physical property values ​​according to the input information acquired by the second acquisition unit using a prediction model that has been constructed in advance by machine learning to output the physical property values ​​of the aggregate according to the input of the input information. [Effects of the Invention]

[0014] According to the present disclosure, there are provided an aggregate quality prediction method, a quality prediction program, an aggregate quality prediction device, a ready-mix concrete manufacturing method, and a ready-mix concrete manufacturing system that are useful for improving the prediction accuracy of a prediction model constructed by machine learning. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a schematic diagram showing an example of a ready-mix concrete manufacturing system. [Figure 2] FIG. 2 is a block diagram showing an example of the hardware configuration of a calculation device included in the quality prediction device. [Figure 3] FIG. 3 is a block diagram illustrating an example of the functional configuration of the arithmetic device. [Figure 4] FIG. 4 is a diagram for explaining the problems that arise when a threshold value is fixed during binarization. [Figure 5] Fig. 5(a) is a diagram illustrating the process of the preprocessing step, and Fig. 5(b) is a diagram illustrating the calculation process of the prediction step. [Figure 6] FIG. 6 is a diagram showing an example of captured image data for training. [Figure 7]FIG. 7 is a flowchart illustrating a series of processes in the training phase. [Figure 8] FIG. 8 is a diagram illustrating how extended image data is created. [Figure 9] FIG. 9 is a flowchart illustrating a series of processes in the evaluation phase. [Figure 10] Figure 10(a) is a graph illustrating the results of verification using test data with the same gradation level as the training aggregate, and Figure 10(b) is a graph illustrating the results of verification using test data with a different gradation level than the training aggregate. [Figure 11] Figure 11(a) is a graph illustrating the results of verification using test data with the same gradation level as the training aggregate, and Figure 11(b) is a graph illustrating the results of verification using test data with a different gradation level than the training aggregate. [Figure 12] Figure 12(a) is a graph illustrating the results of verification using test data with the same gradation level as the training aggregate, and Figure 12(b) is a graph illustrating the results of verification using test data with a different gradation level than the training aggregate. DETAILED DESCRIPTION OF THE INVENTION

[0016] An embodiment will be described below with reference to the drawings. In the description, the same elements or elements having the same functions are designated by the same reference numerals, and redundant description will be omitted.

[0017] [Ready-mix concrete manufacturing system] Fig. 1 schematically shows a ready-mixed concrete manufacturing system according to one embodiment. The manufacturing system 1 shown in Fig. 1 is a system for manufacturing ready-mixed concrete. The manufacturing system 1 is installed, for example, in a factory for manufacturing ready-mixed concrete. The manufacturing system 1 manufactures (produces) ready-mixed concrete by mixing concrete materials.

[0018] The concrete materials used in the manufacturing system 1 include cement, admixtures, coarse aggregate, fine aggregate, water, and admixtures. Examples of coarse aggregate include gravel, crushed stone, slag coarse aggregate, lightweight coarse aggregate, recycled coarse aggregate, recovered aggregate, and coarse aggregates made from a mixture of these. Examples of gravel include mountain gravel, land gravel, river gravel, and sea gravel. Examples of slag coarse aggregate include blast furnace slag aggregate, ferronickel slag aggregate, electric arc furnace oxidizing slag aggregate, and coal gasification slag aggregate. Examples of lightweight coarse aggregate include natural lightweight aggregate, by-product lightweight aggregate, and artificial lightweight aggregate. Examples of coarse aggregate include crushed rock or crushed limestone.

[0019] Examples of fine aggregates include sand, crushed sand, slag fine aggregate, lightweight fine aggregate, recycled fine aggregate, recovered aggregate, and fine aggregates made from mixtures of these. Sand includes mountain sand, land sand, river sand, and sea sand. Slag fine aggregates include blast furnace slag aggregate, ferronickel slag aggregate, copper slag aggregate, electric furnace oxidizing slag aggregate, and coal gasification slag aggregate. Lightweight fine aggregates include natural lightweight aggregate, by-product lightweight aggregate, and artificial lightweight aggregate.

[0020] Examples of rock types for crushed stone and crushed sand include igneous rocks, sedimentary rocks, metamorphic rocks, quartzite, limestone, domalite, and peridotite. Igneous rocks include granite, diorite, gabbro, porphyrite, diabase, rhyolite, andesite, basalt, and serpentinite. Sedimentary rocks include conglomerate, sandstone, shale, slate, and tuff. Metamorphic rocks include gneiss and schist.

[0021] In addition to the function of producing ready-mixed concrete, the manufacturing system 1 also has the function of predicting the quality of aggregate contained in concrete materials. In this disclosure, coarse aggregate and fine aggregate may be collectively referred to as "aggregate." In this case, "aggregate" refers to coarse aggregate, fine aggregate, or both coarse aggregate and fine aggregate.

[0022] The manufacturing system 1 loads the manufactured ready-mixed concrete onto a transport vehicle C. After the ready-mixed concrete has been loaded onto the transport vehicle C, the transport vehicle C transports the ready-mixed concrete to the site where the ready-mixed concrete will be used (for example, a construction site). Examples of the transport vehicle C include an agitator vehicle (mixer vehicle) or a dump truck. The manufacturing system 1 may manufacture ready-mixed concrete from concrete materials so as to satisfy a target quality (required quality) set for each site. For example, an operator of the manufacturing system 1 determines the mix of concrete materials so as to satisfy the control quality at the time of shipping based on the target quality at the site, and inputs operating instructions to the manufacturing system 1.

[0023] The manufacturing system 1 includes, for example, a material storage area 2, a transport device 8, a manufacturing device 10, and a quality prediction device 50.

[0024] The material storage yard 2 is a place where concrete materials are stored. The material storage yard 2 includes a plurality of silos 4. The plurality of silos 4 are containers that store at least a portion of the concrete materials by material type. The plurality of silos 4 include a silo 4 that stores coarse aggregate, a silo 4 that stores fine aggregate, and a silo 4 that stores cement.

[0025] The transporting device 8 is a device that transports the concrete materials stored in the multiple silos 4 to the manufacturing device 10. The transporting device 8 includes, for example, a belt conveyor that transports the concrete materials. The transporting device 8 may transport the concrete materials by type at different times. In one example, based on an operation instruction from a control device included in the manufacturing system 1, a specific material from among the various concrete materials is transferred to the transporting device 8 and transported to the manufacturing device 10.

[0026] The manufacturing apparatus 10 is an apparatus that mixes concrete materials including aggregate to manufacture ready-mixed concrete. The manufacturing apparatus 10 operates based on operation instructions from a control device included in the manufacturing system 1. The manufacturing apparatus 10 includes, for example, a storage bottle 12, a measuring bottle 14, a collecting hopper 16, a mixer 20, and a loading hopper 30.

[0027] Storage bottles 12 temporarily store various types of concrete materials. Various types of concrete materials are transported (conveyed) to storage bottles 12 from material storage area 2 by transport device 8. Storage bottles 12 are configured to store various types of concrete materials individually. Hereinafter, "concrete materials" may be simply referred to as "materials." Various materials stored in storage bottles 12 are supplied to measuring bottles 14 as needed.

[0028] The measuring bottle 14 is disposed below the storage bottle 12. The measuring bottle 14 operates based on operational instructions from the control device of the manufacturing system 1, and individually measures various materials. When the measuring bottle 14 detects the target amount of material instructed by the control device, it supplies the material to the collecting hopper 16. When water is supplied to the measuring bottle 14, an admixture may be mixed into the water. The collecting hopper 16 is disposed below the measuring bottle 14. The collecting hopper 16 collects the various materials discharged from the measuring bottle 14 and supplies the collected materials to the mixer 20.

[0029] The mixer 20 is disposed below the collecting hopper 16. The mixer 20 is a device that mixes concrete materials. The mixer 20 produces ready-mixed concrete by mixing (kneading) aggregate, cement, water, admixtures, etc. The ready-mixed concrete is discharged from the bottom of the mixer 20 into the loading hopper 30. The loading hopper 30 is disposed below the mixer 20 and temporarily stores the ready-mixed concrete. The loading hopper 30 supplies the temporarily stored ready-mixed concrete to the transport vehicle C.

[0030] The manufacturing apparatus 10 described above is an example of a ready-mixed concrete manufacturing apparatus, and the ready-mixed concrete manufacturing apparatus may be configured in any way as long as it is capable of mixing concrete materials and manufacturing ready-mixed concrete. For example, the manufacturing apparatus 10 does not need to be equipped with the collection hopper 16, and various materials measured in the measuring bottles 14 may be supplied from the measuring bottles 14 to the mixer 20.

[0031] (Quality prediction device) The quality prediction device 50 (aggregate quality prediction device) is a device that predicts the quality of aggregate contained in concrete material. The quality prediction device 50 may predict the quality of at least some of the aggregate used by the manufacturing apparatus 10. It is known that the quality of the ready-mixed concrete manufactured by the manufacturing apparatus 10 is affected by the quality of the concrete material. Specifically, the particle size of the aggregate used to manufacture the ready-mixed concrete affects the fluidity (e.g., slump and slump flow) of the ready-mixed concrete. For example, for the same unit water content, the larger the particle size of the aggregate, the smaller the surface area of ​​the aggregate, which reduces the water retention capacity and may increase the fluidity of the ready-mixed concrete. Conversely, the smaller the particle size of the aggregate, the larger the surface area of ​​the aggregate, which increases the water retention capacity and may decrease the fluidity of the ready-mixed concrete. Thus, in order to stabilize the quality, including the fluidity, of the ready-mixed concrete, it is necessary to control the particle size of the aggregate.

[0032] The gradation of aggregate can be expressed by the coarse-grained fraction. In order to stabilize the quality of ready-mixed concrete, it is necessary to control, for example, whether the coarse-grained fraction of aggregate is within a desired range. The quality prediction device 50 may predict the coarse-grained fraction of aggregate as the quality of aggregate. The quality prediction device 50 may be configured to predict the coarse-grained fraction of aggregate from information including image data based on image data obtained by capturing an image of the aggregate, using a prediction model constructed by machine learning.

[0033] At least a part of the quality prediction device 50 is configured by one or more computers. The computer that configures at least a part of the quality prediction device 50 may be a personal computer, a tablet computer (tablet terminal), a smartphone, a wearable device, a workstation, a server computer, or a general-purpose computer.

[0034] When the quality prediction device 50 includes two or more computers, these computers may be communicatively connected to each other. The quality prediction device 50 may be communicatively connected to a control device included in the manufacturing system 1. The quality prediction device 50 may be a part of the function of the control device included in the manufacturing system 1 (i.e., may be included in the control device).

[0035] The quality prediction device 50 includes, for example, a calculation device 52, an input / output device 54, and an imaging device 58. The calculation device 52 is a device that performs calculations to predict the quality of aggregate. The calculation device 52 may be a computer main body (a main body portion of a computer) that executes the main functions of the computer that constitutes the quality prediction device 50.

[0036] The input / output device 54 is connected to the arithmetic unit 52. The input / output device 54 has a function of inputting information indicating instructions from a user such as a worker to the arithmetic unit 52, and a function of outputting information from the arithmetic unit 52 to the user such as a worker. The input / output device 54 may include a keyboard, an operation panel, or a mouse as an input device, and may include a monitor (e.g., a liquid crystal display) as an output device. The input / output device 54 may be a touch panel in which an input device and an output device are integrated. The arithmetic unit 52 and the input / output device 54 may be integrated.

[0037] The imaging device 58 is a device (camera) capable of capturing an image of the aggregate. The imaging device 58 generates color image data by, for example, capturing an image of the aggregate. Hereinafter, the color image data generated by the imaging device 58 will be referred to as "captured image data P1." The imaging device 58 is a digital camera or video camera that captures an image within its field of view (imaging range) using visible light. The color captured image data P1 generated by the imaging device 58 is used to predict the quality of the aggregate. When the imaging device 58 generates video data, still image data included in the video may be acquired as the captured image data P1.

[0038] In the quality prediction device 50, the arithmetic device 52, the input / output device 54, and the imaging device 58 may be integrated into one device, such as a smartphone with a built-in (mounted) camera or a tablet computer with a built-in camera. In this case, the quality prediction device 50 may be portable by a worker or the like. In one example, the quality prediction device 50 is configured by installing an application for predicting the quality of aggregate on a smartphone or a tablet computer.

[0039] The imaging device 58 may be arranged in the manufacturing system 1 so as to be able to capture an image of the aggregate at a specific location. For example, the imaging device 58 is arranged so as to be able to capture an image of the aggregate while it is being transported by the transporting device 8. The imaging device 58 may be mounted on a mobile object such as a drone, and the imaging device 58 may capture an image of the aggregate by, for example, operating the mobile object by a user. The imaging device 58 may capture an image of the aggregate in a room where the imaging environment, such as illuminance, is adjusted.

[0040] 2, the arithmetic device 52 includes a circuit 91. The circuit 91 has a processor 92, a memory 93, a storage 94, and an input / output port 95. The storage 94 is configured with one or more non-volatile memory devices such as a flash memory or a hard disk. The storage 94 stores at least a quality prediction program for causing a computer to execute a quality prediction process, which will be described later. The storage 94 stores a quality prediction program for configuring each functional block, which will be described later, of the arithmetic device 52.

[0041] The memory 93 is composed of one or more volatile memory devices such as a random access memory. The memory 93 temporarily stores a quality prediction program loaded from the storage 94. The processor 92 is composed of one or more arithmetic devices such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The processor 92 configures each functional block of the arithmetic unit 52 by executing the quality prediction program loaded into the memory 93. The results of arithmetic operations by the processor 92 are temporarily stored in the memory 93. The input / output port 95 inputs and outputs information to and from the input / output device 54, the imaging device 58, etc. in response to a request from the processor 92.

[0042] The circuit 91 is not necessarily limited to one in which each function is configured by a program. For example, the circuit 91 may have at least some of its functions configured by a dedicated logic circuit or an ASIC (Application Specific Integrated Circuit) that integrates such a dedicated logic circuit. The quality prediction program may be provided by being permanently recorded on a tangible recording medium such as a CD-ROM, a DVD-ROM, or a semiconductor memory. Alternatively, the quality prediction program may be provided via a communication network as a data signal superimposed on a carrier wave.

[0043] 3 shows an example of functional components (referred to as "functional blocks" in this disclosure) included in the arithmetic device 52. The arithmetic device 52 has, as functional blocks, an imaging data acquisition unit 62 (first acquisition unit), a preprocessing unit 64, an input data acquisition unit 66 (second acquisition unit), a model construction unit 68, a model holding unit 70, a prediction unit 72, and an output unit 74. The processing performed by these functional blocks corresponds to the processing performed by the arithmetic device 52 (quality prediction device 50).

[0044] The imaging data acquisition unit 62 is configured to acquire captured image data P1, which is color image data obtained by capturing an image of an aggregate whose quality is to be evaluated. Hereinafter, the aggregate whose quality is to be evaluated will be referred to as the "aggregate to be evaluated." The imaging data acquisition unit 62 acquires, for example, captured image data P1 from the imaging device 58, which is obtained by imaging with the imaging device 58.

[0045] In the color captured image data P1, pixel values ​​for the red, green, and blue color components are determined for each pixel. If the pixel value of the red component, the pixel value of the green component, and the pixel value of the blue component are written as "R," "G," and "B," respectively, the pixel value for each pixel is written as (R, G, B). The pixel value is also called the brightness value, density value, shading value, tone value, or color value. The pixel value is expressed as a number ranging from 0 to 255, for example.

[0046] The pre-processing unit 64 generates pre-processed image data by performing predetermined pre-processing on the captured image data P1. The pre-processing unit 64 generates pre-processed image data by sequentially performing a grayscale conversion process, a darkening process, and a binarization process on the captured image data P1. Hereinafter, the grayscale conversion process, the darkening process, and the binarization process will be referred to as the “first process,” the “second process,” and the “third process,” respectively. Performing the first process, the second process, and the third process in sequence means that the first process is performed, followed by the second process, and then the third process. Another process may be included between the first process and the second process, or between the second process and the third process, or at least one of the first process and the second process and the third process. The first process, the second process, and the third process may be performed consecutively in this order, without a separate process being included between the first process and the second process or between the second process and the third process.

[0047] Here, the significance of performing preprocessing in which the first process, the second process, and the third process are performed in that order will be explained with reference to Fig. 4. The inventors have confirmed that the following problems exist when predicting the aggregate particle size (coarse particle ratio) using a prediction model constructed by machine learning so as to output physical property values ​​related to the particle size of aggregate in response to input of color captured image data P1. They have confirmed that the prediction accuracy decreases and the variance in the prediction results increases when the imaging environment, such as illuminance or camera position, differs between training and evaluation, and / or when the particle size of the aggregate to be evaluated is a particle size that was not used during training.

[0048] Therefore, the inventors have investigated various image preprocessing methods to prevent a decrease in prediction accuracy even when the imaging environment differs. In FIG. 4, image data a1 to a7 with different image brightnesses are shown as pre-processed images. Image data a4 is the original captured image data, and other image data is generated by adjusting the brightness of the image data a4. Each of the image data a1 to a7 is subjected to a process different from the preprocessing described above to generate image data b1 to b7. The pre-processed image data an (n is an integer between 1 and 7) corresponds to the processed image data bn. The image data bn is generated by sequentially subjecting the image data an to a grayscale process and a binarization process in which a constant threshold is set between images. As can be seen from observing the images based on the image data b1 to b7, the contours of each particle contained in the aggregate disappear depending on the brightness because the threshold for binarization is constant. Through such verification, the inventors came up with the idea of ​​performing pre-processing on the captured image data P1, which involves sequentially performing a first process to convert the image to grayscale, a second process to darken the image, and a third process to binarize the image. In the following description, "pre-processing" refers to the sequential execution of the first process, the second process, and the third process.

[0049] FIG. 5(a) illustrates image data obtained in the preprocessing process. To summarize the preprocessing process, the preprocessing unit 64 first performs a first process of grayscaling the captured image data P1, thereby generating grayscaled image data (grayscale image data P2). Next, the preprocessing unit 64 performs a second process of darkening the grayscaled image data, thereby generating darkened image data (darkened image data P3). After that, the preprocessing unit 64 performs a third process of binarizing the darkened image data, thereby generating preprocessed image data (input image data P4). Note that the image data illustrated in FIG. 5(a) has been subjected to Otsu's binarization A, which will be described later, as the third process. The processing performed by the preprocessing unit 64 will be described in detail below.

[0050] The pre-processing unit 64 performs a first grayscale conversion process on the captured image data P1 to generate grayscaled image data (hereinafter referred to as "grayscale image data P2"). In the first process, the pre-processing unit 64 converts a combination of red, green, and blue pixel values ​​(R, G, B) for each pixel into a single pixel value. When the pixel value for each pixel in the grayscale image data P2 is represented as "Y", the pre-processing unit 64 may perform grayscale conversion using the following equation (0): Y=0.299×R+0.587×G+0.114×B (0)

[0051] The pre-processing unit 64 performs a second darkening process on the grayscale image data P2 to generate darkened image data (hereinafter referred to as "darkened image data P3"). In the present disclosure, the darkening process refers to a process of reducing the pixel value of each pixel so that the entire image is darker than the image before the process. Reducing the pixel value is equivalent to bringing the image closer to black. The pre-processing unit 64 may perform the darkening process by any method.

[0052] In one example, the pre-processing unit 64 performs the darkening process using any one of the following equations (1), (2), (3), and (4). Bi = Ai × C (1) Bi=MAX[(Ai-D),0] (2) Bi=MAX[(Ai-E),0] (3) Bi = MAX[(Ai × CE),0] (4) In formulas (1) to (4), Ai, Bi, C, D, E, and MAX[a, b], as well as A0 and B0 included in the definitions of C and D, have the following meanings. Ai: pixel value of each pixel in the image before processing Bi: Pixel value of each pixel in the processed image A0: Average pixel value in the image before processing (grayscale image data P2) B0: Pre-set target value for pixel value C: The value obtained by dividing B0 by A0 (=B0 / A0) D: The value obtained by subtracting A0 from B0 (=A0-B0) E: Any specified value MAX[a,b]: An operation that outputs the larger value of number a and number b.

[0053] In the above-described formula (2), when (Ai-D) is less than 0, Bi=0; when (Ai-D) is 0 or greater, Bi=(Ai-D). In formulas (3) and (4), the calculation of MAX[a,b] has the same meaning as formula (2). The preprocessing unit 64 performs the calculation represented by any one of formulas (1), (2), (3), and (4) for each pixel to generate the darkened image data P3. The target value B0 may be set in advance by the user of the quality prediction device 50, the provider of the quality prediction device 50, or the provider of the quality prediction program. The target value B0 is set so that the entire image after processing becomes dark. For example, the target value B0 is 10 to 150 or 15 to 45. By using a value C or a value D including the target value B0 and the average pixel value A0, the amount of reduction in pixel value for each pixel can be varied depending on the brightness of the image to be processed. The value E may be set in advance by the user of the quality prediction device 50, the provider of the quality prediction device 50, or the provider of the quality prediction program. The value E is set so that the entire image after processing becomes dark. The value E is, for example, 1 to 200, or 10 to 100.

[0054] The pre-processing unit 64 performs a third process of binarizing the blackout image data P3 to generate pre-processed image data (hereinafter referred to as "input image data P4"). Binarization means converting each pixel into two colors, black and white, by comparing the grayscale pixel value (for example, a numerical value in the range of 0 to 255) with a threshold value. For example, for each pixel, if the pixel value before processing is equal to or less than the threshold value, the pre-processing unit 64 converts the pixel value of the pixel to 0 (black), and if the pixel value before processing is greater than the threshold value, the pre-processing unit 64 converts the pixel value of the pixel to 255 (white).

[0055] The pre-processing unit 64 dynamically sets a threshold value based on pixel values ​​of the image data to be binarized, and then performs the binarization process. The pre-processing unit 64 may perform the binarization process using adaptive binarization or Otsu's binarization.

[0056] "Adaptive binarization" is a binarization technique that sets a threshold for each pixel based on statistical information about pixel values ​​in a surrounding region (peripheral region) that includes the pixel (pixel of interest) at its center. Adaptive binarization is also referred to as adaptive binarization. When performing binarization using adaptive binarization, the pre-processing unit 64 may set, for each pixel, the arithmetic mean of pixel values ​​in the surrounding region that includes the pixel at its center as the binarization threshold. The pre-processing unit 64 may also set, for each pixel, a weighted mean of pixel values ​​in the surrounding region weighted by a Gaussian function as the binarization threshold. For ease of explanation, adaptive binarization that uses the arithmetic mean of pixel values ​​in the surrounding region as the binarization threshold will be referred to as "adaptive binarization A." Furthermore, adaptive binarization that uses the weighted mean of pixel values ​​in the surrounding region weighted by a Gaussian function as the binarization threshold will be referred to as "adaptive binarization B."

[0057] In the present disclosure, "Otsu's binarization" includes not only a method of dividing the histogram of pixel values ​​of an entire image into two classes, calculating the degree of separation between the two classes, and setting the pixel value that maximizes the degree of separation as a threshold, but also a method of setting a threshold (one threshold) based on statistical information of the entire image. In the Otsu's binarization in the third process, the pre-processing unit 64 may use a threshold determined based on a histogram of pixel values ​​in the blackout image data P3 (image before processing). In the Otsu's binarization in the third process, the pre-processing unit 64 may use the average (arithmetic mean) of pixel values ​​of all pixels in the blackout image data P3 (image before processing) as a threshold, instead of a threshold determined based on a histogram. Hereinafter, for ease of explanation, Otsu's binarization that performs binarization using the arithmetic mean of pixel values ​​of the entire image as a threshold will be referred to as "Otsu's binarization A." Also, Otsu's binarization, which performs binarization using a threshold determined based on a histogram of pixel values ​​of the entire image, is referred to as "Otsu's binarization B."

[0058] 3, the input data acquisition unit 66 acquires input information including input image data P4. The input data acquisition unit 66 may acquire only the input image data P4 as input information. The input data acquisition unit 66 may also acquire information that may affect the granularity of the aggregate in addition to the input image data P4 as input information.

[0059] The model construction unit 68 constructs a model (hereinafter referred to as the "prediction model M") for predicting the particle size of aggregate. The prediction model M is a model constructed by machine learning to output physical property values ​​of aggregate in response to input information including input image data P4. The physical property values ​​of aggregate are physical property values ​​related to the particle size of aggregate. Specific examples of physical property values ​​related to the particle size of aggregate include the mass fraction retained between successive sieves (mass fraction for each particle size division), the mass fraction retained on each sieve, the mass fraction passing through each sieve, and the coarse particle fraction of aggregate, as defined in JIS A 1102:2014 "Test Method for Sieving of Aggregates"; the fine particle content of aggregate, as defined in JIS A 1102:2014 "Test Method for Fine Particle Content of Aggregates"; and the oversized particle content of aggregate (the amount of fine aggregate retained on a 5 mm sieve) and the undersized particle content of aggregate (the amount of coarse aggregate passing through a 5 mm sieve). The prediction unit 72 constructs a prediction model M by machine learning based on, for example, input information including input image data P4 and correct answer information of the mass fraction remaining between each successive sieve associated with the input information.

[0060] Machine learning is a technique in which a machine (computer) autonomously finds laws or rules by repeatedly learning based on given information. A predictive model M can be constructed using an algorithm and a data structure. A predictive model M is realized, for example, using a neural network, which is an information processing model that mimics the mechanism of the human brain and nerves. There are no particular restrictions on the specific algorithm of machine learning used when constructing a predictive model M. A neural network has an input layer, one or more intermediate layers, and an output layer. By including one or more intermediate layers, a more complex predictive model M can be constructed, thereby improving prediction accuracy.

[0061] The model construction unit 68 may construct the prediction model M by machine learning using a convolutional neural network (CNN). Using a convolutional neural network makes it possible to more appropriately capture image features. The model construction unit 68 may construct the prediction model M by machine learning based on the input image data P4 and correct answer information on the mass fraction remaining between successive sieves. The prediction model M constructed by the model construction unit 68 may output a value indicating the mass fraction remaining between successive sieves in accordance with the input image data P4. In other words, the prediction model M may output multiple values ​​indicating the mass fraction for multiple particle size classes.

[0062] The model construction unit 68 may autonomously construct a prediction model M by performing machine learning using data provided as machine learning input and correct answer information of the machine learning output (such as correct values ​​of the mass fraction remaining between successive sieves). The machine learning input is various data sets of input information including input image data P4. The machine learning output is data (numerical values) indicating physical property values ​​such as the mass fraction remaining between successive sieves. The model construction unit 68 iteratively trains a model that outputs physical property values ​​related to the granularity of aggregate using multiple combinations of data sets of input information and correct values ​​of physical property values.

[0063] The stage of autonomously constructing the predictive model M corresponds to the training phase. The training phase is also called the learning phase. The same (mutually corresponding) information and aggregates are used in the training phase and the evaluation phase. Hereinafter, the terms "for training" and "for evaluation" may be used to distinguish between the information and aggregates used in the training phase and the information and aggregates used in the evaluation phase.

[0064] The model storage unit 70 stores the prediction model M constructed by the model construction unit 68. The prediction model M, which is a trained model, may be transferable between computers. Therefore, the model storage unit 70 may store the prediction model M constructed in a device different from the quality prediction device 50. In this case, the calculation device 52 does not need to have the model construction unit 68.

[0065] In the evaluation phase, the prediction unit 72 predicts the quality of the aggregate to be evaluated. The prediction unit 72 uses the prediction model M to acquire physical property values ​​corresponding to the input information for evaluation acquired by the input data acquisition unit 66. For example, the prediction unit 72 inputs the input information for evaluation to the prediction model M, and then acquires values ​​indicating the mass fraction remaining between successive sieves that are output from the prediction model M. The prediction unit 72 may calculate a predicted value of the coarse particle fraction of the aggregate to be evaluated by converting the values ​​indicating the mass fraction remaining between successive sieves into a coarse particle fraction. The prediction unit 72 may correct the values ​​indicating the multiple mass fractions output from the prediction model M before converting them into a coarse particle fraction.

[0066] The output unit 74 outputs the quality prediction result by the prediction unit 72 to the monitor of the input / output device 54. The output unit 74 may display the predicted value of the coarse particle ratio calculated by the prediction unit 72 on the monitor. In addition to the predicted value of the coarse particle ratio, the output unit 74 may display the value obtained from the prediction model M regarding the mass fraction remaining between successive sieves on the monitor. The output unit 74 may display a graph of the particle size distribution on the monitor based on the predicted value of the mass fraction remaining between successive sieves.

[0067] <Prediction model M> Here, the prediction model M used to predict the coarse particle ratio will be further described with reference to Figures 5(b) and 6. Figure 5(b) schematically shows the calculation process for predicting the coarse particle ratio using the prediction model M from input image data P4, which is at least a part of the input information to the prediction model M. The input to the prediction model M is, for example, only the input image data P4, and the output of the prediction model M is, for example, the mass fraction remaining between each successive sieve.

[0068] In the training phase for constructing the prediction model M, the same type of aggregate as the aggregate to be evaluated is used as the training aggregate. When constructing the prediction model M, training data for machine learning is prepared. The training data includes training input information and correct answer information for physical property values ​​associated with the training input information. The training data is composed of multiple data sets, and each of the multiple data sets includes, for example, training input image data P4 and correct answer information for the mass fraction remaining between each successive sieve associated with the input image data P4. The training input image data P4 differs among the multiple data sets.

[0069] When preparing various training input image data P4, for example, various types of aggregates are prepared, each having a known mass fraction remaining between successive sieves. The above-mentioned various types of aggregates do not refer to different types of aggregates, but rather to the same type of aggregates with different particle sizes. In one example, the various training aggregates are stacked on a flat tray and individually imaged by the imaging device 58. The stacked state (piled state) refers to a state in which some particles contained in the aggregate overlap other particles when viewed vertically from above.

[0070] The imaging device 58 captures an image of an area including the aggregate on the tray, from a position a predetermined distance away from the tray on which the training aggregate is piled (for example, a predetermined position above). The imaging by the imaging device 58 may be performed indoors or outdoors. The amount of aggregate placed on the tray when imaging is performed is not particularly limited, but may be approximately 0.1 kg to 10 kg. At least a portion of the aggregate being transported on the belt conveyor included in the transporting device 8 may be used as the training aggregate. In this case, the imaging device 58 may capture an image of the aggregate being transported on the belt conveyor of the transporting device 8. The imaging device 58 may be installed so that the amount of aggregate being transported included in the imaging field of the imaging device 58 is approximately 0.1 kg to 10 kg.

[0071] In one example, various samples are prepared as training aggregates under the conditions shown in Table 1 below, and the prepared samples are imaged to obtain training image data P1. The mass fraction and coarse particle fraction remaining between successive sieves for the various samples prepared under the conditions shown in Table 1 are known, and at least a portion of these known values ​​are used as ground truth information in machine learning.

[0072] [Table 1]

[0073] The fineness fraction (FM) can be calculated, for example, by the following formula (5) specified in JIS A 1102:2014 "Sieve Analysis Test Method for Aggregates." Note that in the "Standard Specifications for Building Construction and Commentary JASS5 Reinforced Concrete Construction," nine sieves, excluding the 80 mm sieve, are used in formula (5). The fineness fraction predicted in this disclosure may be specified by either method.

[0074]

number

[0075] In Table 1, this means that aggregates of 10 levels of gradation (coarseness ratio) are prepared. Each level of aggregate is sieved successively in order of coarseness of the sieve and classified into categories I to IV. Category I represents the particle size category for particles greater than 10 mm and smaller than 20 mm. Particles that make up Category I are those that are retained on a 10 mm sieve but pass through a 20 mm sieve. Category II represents the particle size category for particles greater than 5 mm and smaller than 10 mm. Particles that make up Category II are those that are retained on a 5 mm sieve but pass through a 10 mm sieve. Category III represents the particle size category for particles larger than 2.5 mm and smaller than 5 mm. Particles that make up Category III are those that are retained on a 2.5 mm sieve but pass through a 5 mm sieve. Category IV represents the particle size category for particles greater than 1.2 mm and smaller than 2.5 mm. Particles that make up Category IV are those that are retained on a 1.2 mm sieve but pass through a 2.5 mm sieve. The mass fraction retained on each successive sieve means that, for example, level 1 in Table 1, the particles in category I were 85 wt%, the particles in category II were 15 wt%, the particles in category III were 0 wt%, and the particles in category IV were 0 wt%.

[0076] Image data T1 to T10 shown in FIG. 6 are examples of image data obtained by imaging training aggregates prepared according to the conditions of each of Levels 1 to 10 in Table 1. The image data illustrated in FIG. 6 was obtained by imaging the training aggregates placed on a sheet that simulates the transportation of aggregates on a belt conveyor. Instead of such a sheet, the training aggregates may be placed on a flat tray, and image data related to the training aggregates prepared according to the conditions of each of Levels 1 to 10 in Table 1 may be obtained. Level k (k is an integer greater than or equal to 1) corresponds to image data Tk. For example, image data T1 is obtained by imaging training aggregates prepared according to Level 1. Training aggregates according to each Level k contain particles of multiple gradation classes.

[0077] For each level k, multiple image data Tk may be prepared, each showing a different state (more specifically, pile state) of the training aggregate. In one example, a first image is taken with the training aggregate placed on a tray, thereby acquiring one image data Tk. Then, the training aggregate is temporarily removed from the tray, and the same training aggregate (same sample) is placed on the tray again so that the pile state of the multiple particles is different from that in the first image. After that, a second image is taken. By repeating the above-described image capture, multiple image data Tk may be prepared for each level k.

[0078] In addition to the conditions shown in Table 1, various samples may be prepared as training bone materials under the conditions shown in Tables 2 and 3 below, and training captured image data P1 (image data Tk) may be prepared.

[0079] [Table 2]

[0080] [Table 3]

[0081] In one example, one data set in the training data is composed of one image data Tk and the correct answer information for the mass fraction at level k associated with that image data Tk. Multiple data sets, each consisting of a combination of image data Tk and correct answer information for the mass fraction, may be prepared as training data. Then, machine learning using such training data may be performed to construct a prediction model M that outputs predicted values ​​for four mass fractions for four particle size classes. The mass fractions remaining between successive sieves at each level in the training data, the number of levels (the value of k), and the particle size classes may be set as desired, without being limited to the examples shown in Tables 1 to 3.

[0082] An example of the prediction model M will be described using simplified formulas for ease of understanding. The prediction model M constructed by the model construction unit 68 can be simply expressed as, for example, the following formulas (6) and (7).

number

number

[0083] In equation (7), Y represents the output value of the physical property, for example, the output value of the mass fraction retained between successive sieves. In this case, equations (6) and (7) are calculated for each particle size classification. N is an integer of 2 or greater and represents the number of data sets in the input information. x represents various input values ​​included in the input information, and the input information includes at least input image data P4 based on captured image data P1.

[0084] wi is a weight (coefficient), and b is a bias term (coefficient). f(U) represents the activation function. The activation function can be a linear function (identity function) or a nonlinear function such as a polynomial, absolute value, step function, sigmoid function, hardsigmoid function, logsigmoid function, softmax function, logsoftmax function, softmin function, softplus function, softsign function, tanh function, tanhShrink function, hardtanh function, tanhexp function, ReLU function, ReLU6 function, Leaky-ReLU function, PReLU function, ELU function, SELU function, CELU function, Swith function, Mish function, or ACON function.

[0085] The model construction unit 68 may use training data to repeatedly evaluate the error (loss) between Y (predicted value) obtained by equation (7) and the correct information on the physical properties, and determine the weights wi and bias term b in equation (6) so as to minimize the error. The model construction unit 68 may use any type of loss function as a function for evaluating the error between Y (output value from an intermediate model at an intermediate stage in constructing the prediction model M) obtained by equation (7) and the correct information on the physical properties. The loss function is a function that calculates a loss value based on the error between the predicted value and the correct value, and the weights wi are updated so as to minimize the loss value. Specific examples of loss functions include the Huber loss function, the mean absolute error, the mean squared error, and the ε-allowable loss function.

[0086] The model construction unit 68 may repeatedly update the weights wi using a gradient method so as to minimize the loss value evaluated by the loss function. The model construction unit 68 may use any type of update formula (weight update formula) when updating the weights wi. Specific examples of weight update formulas include Adam, AdamW, AdamBelief, Adamax, AdaBound, Adagrad, AMSGRAD, AMSBound, Lars, RMSprop, Adadelta, Sgd, SgdW, Momentum, and Nesterov. The weight update formula is also referred to as an optimization algorithm or an optimization method.

[0087] [Ready-mix concrete manufacturing method] Next, an example of a method for producing ready-mixed concrete executed in the production system 1 will be described. The method for producing ready-mixed concrete includes a production process and a quality prediction process. The production process is a process for kneading materials including aggregate to produce ready-mixed concrete. The quality prediction process is a process for predicting the quality of at least a portion of the aggregate used in the production process. The quality prediction process may be executed during a period that overlaps with at least a portion of the period during which the production process is executed.

[0088] (manufacturing process) The manufacturing process includes, for example, a transporting process, a measuring process, a feeding process, a mixing process, a discharging process, and a loading process. In the transporting process, various concrete materials are transported by a transporting device 8 to storage bottles 12, and the various materials are individually supplied to storage bottles 12. In the measuring process, the various materials are individually supplied from storage bottles 12 to measuring bottles 14, and the various materials are measured in measuring bottles 14. In the measuring process, when the measured amount of each material reaches a predetermined set amount, the material is discharged into a collecting hopper 16.

[0089] In the charging process, after all types of materials have been collected in the collecting hopper 16, the materials in the collecting hopper 16 are charged (supplied) into the mixer 20. In the mixing process, multiple types of concrete materials are mixed in the mixer 20. In the discharging process, after mixing of the concrete materials is completed in the mixer 20, ready-mixed concrete is discharged from the mixer 20 into the loading hopper 30. In the loading process, the ready-mixed concrete discharged into the loading hopper 30 is loaded onto the transport vehicle C.

[0090] (Quality prediction process) The quality prediction process (aggregate quality prediction method) includes a model construction process in a training phase and a quality evaluation process in an evaluation phase. In the quality prediction process, the model construction process is executed before the quality evaluation process.

[0091] The model construction process includes, for example, a preparation process and a construction process. The preparation process and the construction process may be executed by the model construction unit 68 of the calculation device 52. The preparation process is a process of preparing training data including training input information corresponding to the evaluation input information and correct answer information for physical property values ​​associated with the training input information. The construction process is a process of constructing a prediction model M by machine learning based on the training data prepared in the preparation process.

[0092] The preparation process may include generating extended image data by performing a conversion process and a first grayscale process in any order on the training image data P1 corresponding to the evaluation image data P1. The term "extension" in the extended image data refers to extending the initially prepared training data (increasing the number of data sets in the training data). The conversion process in the preparation process may include randomly changing the brightness and contrast of the image. The conversion process in the preparation process may include at least one of inverting and rotating the image, in addition to changing the brightness and contrast. That is, the conversion process may include either inverting or rotating the image, or both inverting and rotating the image. The input image data P4 in the training input information may include image data obtained by sequentially performing a second process and a third process on the extended image data. Details of the conversion process for extending the training data will be described later.

[0093] The quality evaluation process includes, for example, a first acquisition process, a preprocessing process, a second acquisition process, a prediction process, and an output process. The first acquisition process is a process of acquiring captured image data P1, which is color image data obtained by capturing an image of the aggregate for evaluation. The captured image data acquisition unit 62 may execute the first acquisition process. The preprocessing process is a process of generating input image data P4 by sequentially performing a first process of grayscaling, a second process of darkening (second darkening process), and a third process of binarizing on the captured image data P1 acquired in the first acquisition process.

[0094] In the first process, grayscale conversion may be performed using the above-described formula (0). In the second process, darkening processing may be performed using any of the above-described formulas (1), (2), (3), and (4). In the third process, binarization processing may be performed by adaptive binarization or Otsu's binarization. When binarization by adaptive binarization is performed in the third process, binarization may be performed by the above-described adaptive binarization A or adaptive binarization B. When binarization by Otsu's binarization is performed in the third process, binarization may be performed by the above-described Otsu's binarization A or Otsu's binarization B. The preprocessing unit 64 may execute a preprocessing step.

[0095] The second acquisition step is a step of acquiring input information including the input image data P4 generated in the preprocessing step. The input data acquisition unit 66 may execute the second acquisition step. The prediction step is a step of acquiring physical property values ​​(physical property values ​​related to the particle size of the aggregate) corresponding to the input information acquired in the second acquisition step, using a prediction model M previously constructed by machine learning so as to output physical property values ​​of the aggregate in response to input of the input information. In the prediction step, the quality of the aggregate, such as the coarseness ratio, is predicted based on the predicted values ​​of the physical property values ​​output from the prediction model M. The prediction unit 72 may execute the prediction step. The output step is a step of outputting the quality of the aggregate predicted in the prediction step to the input / output device 54. The output unit 74 may execute the output step.

[0096] An example of the model construction process and an example of the quality evaluation process will be described below with reference to Figures 7 to 9. Note that the description will be given using an example in which the aggregate coarseness fraction is predicted as the quality of the aggregate, and the input information to the prediction model M includes only input image data P4. Also, the description will be given using an example in which the prediction model M outputs a predicted value of the mass fraction for each of the four particle size classifications.

[0097] <Model building process> 7 is a flowchart showing an example of a series of processes executed in the model construction step. In the model construction step, a portion of the aggregate (e.g., coarse aggregate) used in the production of ready-mixed concrete in the production apparatus 10 may be extracted, and the extracted aggregate may be used as training aggregate.

[0098] In the model building process, step S11 is first executed. In step S11, for example, a worker or the like acquires information necessary for preparing training data for machine learning. In one example, the worker or the like prepares various types of aggregates whose mass fractions remaining between successive sieves are known. The worker or the like may prepare various types of aggregates having gradations of levels 1 to 30 in Tables 1 to 3 described above. The worker or the like may prepare M pieces of training image data P1, each different from the other in at least one of gradation level and deposition state. M may be 10 to 100,000, or may be 100 to 10,000. The M pieces of training image data P1 are each associated with a known mass fraction for each gradation level as correct information for the physical property value.

[0099] Next, step S12 is executed. In step S12, for example, the model construction unit 68 creates (generates) extended image data based on the M pieces of training captured image data P1 prepared in step S11 in accordance with a predetermined calculation procedure in order to extend the training data. Fig. 8 schematically shows the process of creating extended image data from the training captured image data P1. In Fig. 8, the training captured image data P1 is indicated by "TP".

[0100] For example, the model construction unit 68 generates Q pieces of image data TP1 to TPQ by randomly changing the brightness and contrast of each of the M pieces of image data TP (for each piece of image data TP). Q is an integer equal to or greater than 2, and FIG. 8 illustrates an example where Q is 7. As a result of randomly changing the brightness and contrast, only the brightness or only the contrast may be changed in some of the image data TP1 to TPQ. As a result of randomly changing the brightness and contrast, the image data TP1 to TPQ may include images for which neither the brightness nor the contrast has been changed. The model construction unit 68 may generate new image data based on the image data TP so that one of the image data TP1 to TPQ includes the image data TP as is without any change processing. Any one of the image data TP1 to TPQ will be referred to as "image data TPq" (q is an integer from 1 to Q).

[0101] The model construction unit 68 generates R pieces of image data TPq1 to TPqR by, for example, at least one of image flipping and image rotation for each of the image data TP1 to TPQ (for each piece of image data TPq). R is an integer equal to or greater than 2, and FIG. 8 illustrates an example where R is 4. While FIG. 8 illustrates an example where new image data is generated based on the image data TP3, new image data is similarly generated for each of the image data TP1, TP2, TP4 to TP7. The model construction unit 68 may generate new image data based on the image data TPq so that the image data TPq is included as is in one of the image data TPq1 to TPqR.

[0102] In one example, the model construction unit 68 generates four pieces of image data from the image data TPq. In Fig. 8, image data TP31 is the same as the image data TP3, and image data TP32 is data generated by flipping the image data TP3 horizontally. Image data TP33 is data generated by flipping the image data TP3 vertically, and image data TP34 is data generated by flipping the image data TP3 both vertically and horizontally.

[0103] In the inversion for generating new image data based on one image data, the inversion may be performed about any axis, for example, horizontal inversion, vertical inversion, or both horizontal and vertical inversion. In the rotation for generating new image data based on one image data, the rotation may be performed at any angle in the range of 0° to 360°, and the rotation direction may be either clockwise or counterclockwise. At least one of the following rotations may be performed: rotating the image clockwise by 90°, rotating the image counterclockwise by 90°, and rotating the image 180°. After rotating at an angle other than 90° or 180°, correction may be performed so that the outer edges of the image are aligned vertically and horizontally while maintaining the rotated state.

[0104] By generating new image data as described above, the model construction unit 68 may generate (M×Q×R) pieces of image data based on M pieces of image data TP. The model construction unit 68 maintains a state in which correct answer information for physical property values ​​is associated with the newly generated image data. The correct answer information associated between the newly generated image data and the image data on which the newly generated image data is based is the same. As described above, M training datasets may be expanded to (M×Q×R) training datasets.

[0105] Next, step S13 is executed. In step S13, for example, the model construction unit 68 performs the first grayscale conversion process on each of the (M×Q×R) pieces of image data prepared in step S12, according to a predetermined calculation procedure. The model construction unit 68 may perform grayscale conversion using the above-mentioned equation (0) on each piece of image data prepared in step S12. As a result, grayscaled extended image data is obtained.

[0106] Next, step S14 is executed. In step S14, for example, the model construction unit 68 performs a second process of darkening each of the (M×Q×R) pieces of image data grayscaled in step S13 according to a predetermined calculation procedure. The model construction unit 68 may generate the darkened image data by performing the calculation shown in the above-mentioned formula (1), formula (2), formula (3), or formula (4) for each piece of image data grayscaled in step S13.

[0107] Next, step S15 is executed. In step S15, for example, the model construction unit 68 performs a third process of binarizing each of the (M×Q×R) pieces of image data darkened in step S14 in accordance with a predetermined calculation procedure. The model construction unit 68 may generate binarized image data by performing calculations using any one of adaptive binarization A, adaptive binarization B, Otsu's binarization A, and Otsu's binarization B for each piece of image data darkened in step S14.

[0108] In adaptive binarization A, for each pixel, the arithmetic mean of pixel values ​​in a surrounding region including the pixel at its center is set as a binarization threshold, and the pixel value is compared with the threshold to perform binarization. In adaptive binarization B, for each pixel, a weighted mean using a Gaussian function of pixel values ​​in a surrounding region including the pixel at its center is set as a binarization threshold, and the pixel value is compared with the threshold to perform binarization. In both adaptive binarization A and adaptive binarization B, a threshold is calculated for each pixel, and comparison for binarization is performed. The size of the surrounding region may be set in advance, and the number of vertical and horizontal pixels (number of pixels) may be the same. In one example, the number of vertical and horizontal pixels in the surrounding region is an odd number between 3 and 21. In areas near the edge of the image, the threshold may be calculated using pixel values ​​within the available range included in the size of the surrounding region.

[0109] The Gaussian function used for weighting in adaptive binarization B is also referred to as a normal distribution, normal distribution function, Gaussian function, Gaussian distribution, or Gaussian distribution. Weighting using a Gaussian function is performed so that the influence on the pixel of interest (the pixel for which binarization is determined) is increased. A threshold is dynamically set by adaptive binarization A or adaptive binarization B, so that binarization processing is performed under different conditions for each image data. Binarization processing by adaptive binarization A or adaptive binarization B can be performed using a known library such as OpenCV.

[0110] The threshold in Otsu's binarization A is set to a value obtained by calculating the arithmetic mean of the pixel values ​​of all pixels in the image data. The threshold in Otsu's binarization B is set based on a histogram of the pixel values ​​of all pixels in the image data. Specifically, the histogram of the pixel values ​​of the entire image is divided into two classes, the degree of separation between the two classes is calculated, and the pixel value that maximizes the degree of separation is set as the threshold. By dynamically setting the threshold using Otsu's binarization A or Otsu's binarization B, binarization processing is performed under different conditions for each image data. Binarization processing using Otsu's binarization A or Otsu's binarization B can be performed using a known library such as OpenCV.

[0111] By executing steps S11 to S15, (M×Q×R) pieces of training input image data P4 are prepared. The (M×Q×R) pieces of training input image data P4 have been subjected to a conversion process to expand the training data, but have been generated by sequentially performing the first process, the second process, and the third process on the training captured image data P1. Each of the (M×Q×R) pieces of training input image data P4 is associated with a known mass fraction for each particle size classification as ground truth information for the physical property value.

[0112] As described above, the model construction unit 68 may prepare a plurality of training data sets as training data. Each of the plurality of training data sets (each training data set) includes training input image data P4 and correct answer information on the mass fraction remaining between successive sieves, which is associated with the input image data P4.

[0113] Next, steps S16 and S17 are executed. In step S16, for example, the model construction unit 68 constructs a prediction model M by machine learning based on the training data prepared up to the execution of step S15. The model construction unit 68 may construct a prediction model M that outputs multiple mass fraction values ​​for multiple particle size classes in response to input of input image data P4. In step S17, for example, the model holding unit 70 stores the prediction model M constructed in step S16. This completes the series of processes executed in the model construction process.

[0114] <Quality evaluation process> 9 is a flowchart showing an example of a series of processes executed in the quality evaluation step. This quality evaluation step is performed, for example, during a period that overlaps with at least a part of the period during which the above-mentioned production step is executed by the production apparatus 10. In the quality evaluation step, a part of the aggregate used in the production of ready-mixed concrete by the production apparatus 10 may be extracted, and the quality of the aggregate may be evaluated as the evaluation target.

[0115] In the quality evaluation process, step S21 is executed first. In step S21, for example, the imaging data acquisition unit 62 acquires evaluation image data P1 obtained by imaging the aggregate for evaluation. The imaging data acquisition unit 62 may acquire the image data P1 from the imaging device 58. The imaging of the aggregate for evaluation by the imaging device 58 may be performed autonomously by a device including the quality prediction device 50, or may be performed while some of the work is being performed by a worker or the like. Step S21 may be executed at a predetermined evaluation timing. The evaluation timing may be set to a certain time of day, or may be a timing in response to an instruction from a worker or the like. The conditions for imaging by the imaging device 58 may be set to approximately match the conditions in the model construction process, to the extent that this can be set.

[0116] Next, step S22 is executed. In step S22, for example, the pre-processing unit 64 performs the first process of grayscaling the evaluation captured image data P1 acquired in step S21, thereby generating grayscale image data P2. The pre-processing unit 64 performs grayscale conversion using the same calculation procedure (same image processing procedure) as the first process in the model construction step. The pre-processing unit 64 may perform grayscale conversion on the evaluation captured image data P1 using the above-mentioned equation (0).

[0117] Next, step S23 is executed. In step S23, for example, the pre-processing unit 64 performs a second process of darkening the grayscale image data P2 generated in step S22, thereby generating darkened image data P3. The pre-processing unit 64 performs the darkening process using the same calculation procedure (same image processing procedure) as the second process in the model construction step. When any of the above-mentioned formulas (1), (2), and (4) is used, the setting of the target value B0 is the same between the model construction step and the quality evaluation step.

[0118] Next, step S24 is executed. In step S24, for example, the pre-processing unit 64 performs the third process of binarizing the blackout image data P3 generated in step S23, thereby generating input image data P4 for evaluation. The pre-processing unit 64 performs the binarization process using the same calculation procedure (the same image processing procedure) as the third process in the model construction step. The method of setting the threshold for binarization is the same between the model construction step and the quality evaluation step.

[0119] Next, steps S25 and S26 are executed. In step S25, for example, the prediction unit 72 inputs the evaluation input image data P4 generated in step S24 into the prediction model M held by the model holding unit 70, and then obtains a predicted value of the mass fraction remaining between successive sieves output from the prediction model M. In step S26, for example, the prediction unit 72 converts the predicted value of the mass fraction remaining between successive sieves obtained in step S25 into a coarse particle fraction, thereby calculating a predicted value of the coarse particle fraction.

[0120] Next, step S27 is executed. In step S27, for example, the output unit 74 outputs the predicted value of the coarse-grained fraction calculated in step S26 to the monitor of the input / output device 54. In addition to the predicted value of the coarse-grained fraction, the output unit 74 may also output the predicted value of the mass fraction remaining between each successive sieve obtained in step S25 to the monitor of the input / output device 54. By executing step S27, a worker or the like can check the predicted result of the coarse-grained fraction. If the coarse-grained fraction is outside the control range, the worker or the like who has checked the predicted result of the coarse-grained fraction may take measures to maintain the quality of the ready-mixed concrete (for example, adjusting the gradation of the aggregate, correcting (modifying) the mix, temporarily suspending production, investigating the cause, etc.).

[0121] This completes the series of processes for predicting the coarse particle ratio once. After step S27 is executed, the series of processes from steps S21 to S27 may be repeated. In the above example, some aggregates are extracted and quality evaluation (sampling inspection) is performed, but all aggregates used in the production of ready-mixed concrete may also be inspected.

[0122] [Verification of prediction accuracy] Next, the results of verifying the prediction accuracy of a prediction model M constructed by machine learning using a convolutional neural network will be described with reference to Figures 10 to 12. In this verification, prediction models M were constructed using different preprocessing procedures, and then a comparison was made between the prediction results and the correct values ​​for each prediction model M using a test dataset in which the correct values ​​were known and not used in training when the model was constructed.

[0123] (Verification example 1: Blackout → Otsu's binarization A) First, 10 levels of aggregate with different coarseness ratios were prepared according to the conditions in Table 1 above. For each level, 4 kg of aggregate was placed on a flat tray (a rimmed tray). Ten images were taken indoors, with the aggregate being placed in different positions on the tray. This resulted in 100 image data sets according to the conditions in Table 1. Each set of 100 data sets was obtained, each corresponding to the correct value for the mass fraction remaining between successive sieves. In addition to the correct value for the mass fraction remaining between successive sieves, the correct value for the coarseness ratio was also associated with the image data for comparative verification.

[0124] Next, 10 levels of aggregate with different coarseness ratios were prepared according to the conditions in Table 2 above. Using a method similar to the preparation according to the conditions in Table 1, 100 image data were obtained according to the conditions in Table 2, and 100 data sets were obtained in which the image data was associated with the correct value for the mass fraction remaining between each successive sieve. Furthermore, 10 levels of aggregate with different coarseness ratios were prepared according to the conditions in Table 3 above. Using a method similar to the preparation according to the conditions in Table 1, 100 image data were obtained according to the conditions in Table 3, and 100 data sets were obtained in which the image data was associated with the correct value for the mass fraction remaining between each successive sieve.

[0125] After obtaining 300 data sets according to the conditions in Tables 1, 2, and 3, new image data was created to expand the training data by performing the process in step S12 described above. Specifically, when randomly changing the brightness and contrast, 11 patterns of new image data were generated from one original image data. When changing the image by flipping or rotating it, four patterns of new image data were generated from one original image data. As a result, 300 image data were expanded to (300 x 11 x 4 = 13,200) image data. When randomly changing the brightness and contrast, random numbers were generated by computer processing, and the change conditions were varied. When changing the image by flipping or rotating it, new image data was generated by performing four conversion patterns: no flip, vertical flip, horizontal flip, and vertical and horizontal flip.

[0126] Next, the 13,200 pieces of image data were each subjected to the first process (grayscaling) in step S13, the second process (darkening) in step S14, and the third process (Otsu's binarization A) in step S15, in that order, to generate 13,200 pieces of input image data P4. In the first process, grayscale conversion was performed using the above-mentioned equation (0). In the second process, a well-known library (OpenCV) was used to perform the darkening process using the calculation procedure of the above-mentioned equation (1). The target value B0, which represents the target pixel value, was set to 30.

[0127] In the third process, a known library (OpenCV) was used to perform binarization using Otsu's binarization A, which uses the arithmetic mean of the pixel values ​​of the entire image as the binarization threshold. As a result, 13,200 data sets were obtained in which the input image data P4 was associated with the correct mass fraction information.

[0128] Of the 13,200 datasets, 11,880 datasets were used as training datasets (training data) for building a prediction model M through machine learning. The remaining 1,320 datasets were used as test datasets for comparing and verifying the predicted values ​​by the prediction model M with the correct values. In addition to the 1,320 datasets (hereinafter referred to as "test data 1"), 100 datasets (hereinafter referred to as "test data 2") were prepared as test datasets. In preparing test data 2, 10 levels of aggregate with different coarseness ratios were prepared according to the conditions shown in Table 4 below.

[0129] [Table 4]

[0130] Under the conditions shown in Table 4, the combinations of mass fractions for multiple granularity classes were set differently from the combinations under the conditions of Tables 1, 2, and 3. After preparing aggregates according to the conditions of Table 4, 100 image data (imaged image data P1) were obtained according to the conditions of Table 4 using a method similar to the preparation (imaging) according to the conditions of Table 1. 100 data sets were obtained, each corresponding to the correct value of the mass fraction remaining between successive sieves. Then, without expanding the data set, the imaged image data P1 was subjected to the first process, the second process, and the third process using Otsu's binarization A in that order to prepare 100 test data sets as test data 2. Test data 1 was based on the same granularity level as the training data, while test data 2 was based on a different granularity level than the training data.

[0131] Using test data 1 and test data 2, a comparative verification was performed for each test dataset using the following procedure. First, input image data P4 (input information) included in the dataset was input into prediction model M to obtain a predicted value of the mass fraction (mass fraction of each of multiple particle size divisions) retained between successive sieves. Then, the predicted value of the mass fraction retained between successive sieves was converted to a coarse particle fraction to obtain a predicted value of the coarse particle fraction using prediction model M. After that, the error between the predicted value of the coarse particle fraction by prediction model M and the correct value of the coarse particle fraction for the dataset was calculated.

[0132] In the comparative verification, the error between the predicted value of the coarse grain ratio and the correct value for each of test data 1 and test data 2 was evaluated using the following three indicators. Coarse-grained accuracy rate (%): The ratio of the number of correct datasets to the total number of datasets, assuming that a dataset with an error of within ±0.10 is defined as correct. ·RMSE: Root Mean Square Error. · Sum of mean and standard deviation: The arithmetic sum of the mean of the errors and the standard deviation of the errors. The third indicator, "the sum of the mean and standard deviation," means that the error is evaluated taking into account the variability. In other words, even if the error itself (mean) is small, if the standard deviation is large, it means that the variability of the error is large, which is not desirable. Also, even if the standard deviation of the error is small, if the mean is large, it means that the error itself is large, which is not desirable.

[0133] (Verification example 2: Blackout → Otsu's binarization B) Comparative verification was performed in the same manner as in Verification Example 1, except that in the third process in preparing all data sets, namely, training data, test data 1, and test data 2, binarization processing was performed using Otsu's binarization B, which sets a binarization threshold based on a histogram of pixel values ​​of all pixels, instead of Otsu's binarization A. The binarization processing using Otsu's binarization B was performed using a publicly known library (OpenCV).

[0134] (Verification example 3: Blackout → Adaptive binarization A) Comparative verification was performed in the same manner as in Verification Example 1, except that in the third step in preparing all data sets (training data, test data 1, and test data 2), binarization processing was performed using adaptive binarization A, in which the arithmetic mean of pixel values ​​in the surrounding area of ​​a pixel of interest was set as the threshold, instead of Otsu's binarization A. The binarization processing using adaptive binarization A was performed using a publicly known library (OpenCV). In this case, the surrounding area was set to an area having 11 vertical x 11 horizontal pixels centered around the pixel of interest.

[0135] (Verification example 4: Blackout → Adaptive binarization B) Comparative verification was performed in the same manner as in Verification Example 1, except that in the third process in preparing all data sets (training data, test data 1, and test data 2), binarization processing was performed using adaptive binarization B, which sets the threshold value to a weighted average based on a Gaussian function of pixel values ​​in a region surrounding a pixel of interest, instead of Otsu's binarization A. The binarization processing using adaptive binarization B was performed using a publicly known library (OpenCV). In this case, the surrounding region was set to an area having 11 vertical x 11 horizontal pixels centered around the pixel of interest.

[0136] (Reference Example 1: No pre-treatment) Comparative verification was performed in the same manner as in Verification Example 1, except that the preprocessing of sequentially performing the first process, the second process, and the third process was not performed in preparing all of the data sets of training data, test data 1, and test data 2. That is, in Reference Example 1, color image data was used as input information.

[0137] (Reference Example 2: Adaptive Thresholding A) Comparative verification was performed in the same manner as in Verification Example 1, except that in preparing all data sets, namely, training data, test data 1, and test data 2, binarization processing by adaptive binarization A was performed instead of the second and third processes. That is, in the verification of Reference Example 2, image data obtained by performing the first process and binarization processing by adaptive binarization A on captured image data P1 was used as input information. The binarization processing by adaptive binarization A was performed under the same conditions as the third process in Verification Example 3.

[0138] FIG. 10(a) is a graph showing the accuracy rate of the coarse grain ratio when comparatively verifying test data 1 for each of verification examples 1 to 4 and reference examples 1 and 2. FIG. 10(b) is a graph showing the accuracy rate of the coarse grain ratio when comparatively verifying test data 2 for each of verification examples 1 to 4 and reference examples 1 and 2. From the graph shown in FIG. 10(a), it can be seen that the accuracy rate exceeds 90% for verification examples 1 to 4 and reference examples 1 and 2 when using test data 1. On the other hand, from the graph shown in FIG. 10(b), it can be seen that, for test data 2, the accuracy rate for reference examples 1 and 2 drops significantly, while the drop in accuracy rate is small for verification examples 1 to 4. It can also be seen that the drop in verification example 3 is particularly small among verification examples 1 to 4. From the above, it can be seen that even if the aggregate used for evaluation has a different particle size level from the training data used to build the prediction model M, predictions can be made with a high accuracy rate by performing preprocessing including darkening and binarization using either Otsu's binarization A, Otsu's binarization B, adaptive binarization A, or adaptive binarization B.

[0139] FIG. 11(a) is a graph showing the RMSE when Verification Examples 1 to 4 and Reference Examples 1 and 2 are compared and verified using Test Data 1. FIG. 11(b) is a graph showing the RMSE when Verification Examples 1 to 4 and Reference Examples 1 and 2 are compared and verified using Test Data 2. From the graph shown in FIG. 11(a), it can be seen that with Test Data 1, the RMSE is 0.06 or less in Verification Examples 1 to 4 and Reference Examples 1 and 2. On the other hand, from the graph shown in FIG. 11(b), it can be seen that with Test Data 2, the RMSE of Reference Examples 1 and 2 increases significantly, while the increase in RMSE is small in Verification Examples 1 to 4. It can also be seen that the increase in Verification Example 3 among Verification Examples 1 to 4 is particularly small. As described above, even if the aggregate used for evaluation has a different particle size level from the training data used to build the prediction model M, it is possible to reduce the error between the predicted value and the correct value by performing preprocessing including darkening and binarization using either Otsu's binarization A, Otsu's binarization B, adaptive binarization A, or adaptive binarization B.

[0140] FIG. 12(a) is a graph showing the average value and the sum of standard deviations when comparing and verifying test data 1 for each of Verification Examples 1 to 4 and Reference Examples 1 and 2. FIG. 12(b) is a graph showing the average value and the sum of standard deviations when comparing and verifying test data 2 for each of Verification Examples 1 to 4 and Reference Examples 1 and 2. From the graph shown in FIG. 12(a), it can be seen that for Test Data 1, the sum of the average value and the standard deviation is 0.086 or less in Verification Examples 1 to 4 and Reference Examples 1 and 2. On the other hand, from the graph shown in FIG. 12(b), it can be seen that for Test Data 2, the sum of the average value and the standard deviation increases significantly in Reference Examples 1 and 2, whereas the increase in the sum of the average value and the standard deviation is small in Verification Examples 1 to 4. It can also be seen that the increase in Verification Example 3 is particularly small among Verification Examples 1 to 4. As described above, even if the aggregate used for evaluation has a different particle size level from the training data used to construct the prediction model M, it is clear that the error index value, taking variability into account, can be reduced by performing preprocessing including darkening and binarization using either Otsu's binarization A, Otsu's binarization B, adaptive binarization A, or adaptive binarization B.

[0141] [Variations] The series of processes shown in Figures 7 and 9 are examples and can be modified as appropriate. In the series of processes, one step and the next step may be executed in parallel, or some steps may be executed in an order different from that of the examples described above. Steps different from those of the examples described above may be executed instead of or in addition to at least some of the steps of the series of processes described above.

[0142] In the example shown in FIG. 7 , after step S11 is performed, the first grayscaling process of step S13 may be performed, followed by the generation of extended image data of step S12. Training input image data P4 may be generated by subjecting training captured image data P1 to the first process, a transformation process for extending the training data, a second process, and a third process in this order. The transformation process for extending the training data may not include transformation including at least one of image inversion and rotation. Training data extension involving the generation of new image data may be performed by a method different from the extension method exemplified in step S12. In preparing the training data, extension of the training data by the method of step S12 or another method may not be performed.

[0143] In step S16, the prediction model M may be configured to output a coarse particle ratio in response to input of input image data P4. In this case, in the evaluation phase, instead of steps S25 and S26, the coarse particle ratio output when the evaluation input image data P4 is input to the prediction model M is acquired as a prediction result. The prediction model M may be configured to output two or more physical property values ​​related to the aggregate particle size. For example, the prediction model M may be configured to output the mass fraction and coarse particle ratio remaining between successive sieves in response to input information including the input image data P4. In this case, the prediction unit 72 may calculate the coarse particle ratio based on the predicted value of the mass fraction from the prediction model M and the predicted value of the coarse particle ratio output from the prediction model M as quality prediction results. The input information, which is data input to the prediction model M, may include any information that may be correlated with aggregate quality in addition to the input image data P4.

[0144] In addition to the quality prediction device 50 predicting the quality of the aggregate, the manufacturing system 1 may periodically (e.g., several times a day) measure the physical properties of the aggregate. In this case, the prediction model M may be updated based on the actual measured values ​​of the physical properties of the aggregate and input information (e.g., input image data P4) when the actual measured values ​​were obtained. In the prediction step, the physical properties of the aggregate may be predicted using the updated prediction model M. Note that even when the updated prediction model M is used, the step of predicting the physical properties of the aggregate is still performed based on the prediction model M and the input information acquired in the second acquisition step.

[0145] The quality prediction device 50 may be used in a location other than the manufacturing system 1 (other than a process inspection in the manufacturing process of ready-mixed concrete). The quality prediction device 50 may be used in an acceptance inspection when a factory that manufactures ready-mixed concrete receives raw materials. In one example, the quality prediction device 50 is used for quality prediction (acceptance inspection) performed on a transport truck or a transport ship.

[0146] The imaging device 58 may obtain color captured image data P1 by capturing an image, and then perform a process of grayscaling the captured image data P1 to generate grayscale image data P2. In this way, some of the above-described functional blocks may be configured by the imaging device 58 instead of the arithmetic device 52. In one example of the various examples described above, at least some of the matters described in the other examples may be combined.

[0147] Summary of this disclosure The aggregate quality prediction method described above includes a first acquisition step of acquiring color image data (P1) obtained by imaging the aggregate, a pre-processing step of generating input image data (P4) by sequentially performing a first process of grayscaling the image data (P1), a second process of darkening the image, and a third process of binarizing the image data (P1), a second acquisition step of acquiring input information including the input image data (P4), and a prediction step of acquiring physical property values ​​corresponding to the input information acquired in the second acquisition step using a prediction model (M) previously constructed by machine learning to output physical property values ​​of the aggregate according to the input of the input information.

[0148] A technology for predicting aggregate quality from aggregate image data using a prediction model constructed by machine learning has been investigated. It was discovered that differences in imaging conditions or granularity levels, which represent the size of aggregate particles, between the training phase and the evaluation phase can lead to reduced prediction accuracy. After extensive investigation, the inventors discovered that the degradation of prediction accuracy can be reduced by preprocessing the image data to be input to the prediction model, sequentially converting it to grayscale, darkening, and binarizing it. In the quality prediction method, input image data (P4) included in the input information is obtained by sequentially converting captured image data (P1) to grayscale, darkening, and binarizing it. The prediction model (M) is then constructed to output physical property values ​​from such input image data (P4). Based on the above, the aggregate quality prediction method including the preprocessing step is useful for improving the prediction accuracy of a prediction model constructed by machine learning.

[0149] In the aggregate quality prediction method described above, the second step may include a darkening process using any one of the above-described formulas (1), (2), (3), and (4). By performing such darkening as part of preprocessing, it has been confirmed that even when the granularity level differs between the training phase and the evaluation phase, the deterioration in prediction accuracy is smaller than when no preprocessing is performed or when adaptive binarization A is performed as part of preprocessing without darkening. Therefore, the quality prediction method described above is useful for improving the accuracy of predictions using a prediction model for predicting aggregate quality from image data.

[0150] In the aggregate quality prediction method described above, the third step may involve binarization using adaptive binarization or Otsu's binarization. By performing such binarization as part of preprocessing, it has been confirmed that even when the granularity level differs between the training phase and the evaluation phase, the deterioration in prediction accuracy is smaller than when no preprocessing is performed or when adaptive binarization A is performed as part of preprocessing without darkening. Therefore, the quality prediction method described above is useful for improving the accuracy of predictions using a prediction model for predicting aggregate quality from image data.

[0151] The aggregate quality prediction method described above may further include a preparation step of preparing training data including training input information corresponding to the input information and correct information on physical properties associated with the training input information, and a construction step of constructing a prediction model (M) through machine learning based on the training data. The preparation step may include generating extended image data by performing a conversion process and a first process in any order on training captured image data corresponding to the captured image data (P1). The conversion process may include randomly changing the brightness and contrast of the image. The input image data (P4) in the training input information may include image data obtained by sequentially performing a second process and a third process on the extended image data. Even if the correct values ​​of the physical properties are the same, when evaluated using a trained model, the image data may be recognized as having different characteristics, resulting in reduced prediction accuracy. Therefore, it is possible to prepare various training image data after adjusting the imaging environment and then perform machine learning, but this complicates the preparation of the training data. In contrast, in the quality prediction method described above, the brightness and contrast of the image are randomly changed to generate new image data and extend the training data. Therefore, it is useful for improving prediction accuracy while avoiding the complication of preparing training data for machine learning.

[0152] In the aggregate quality prediction method described above, the transformation process may further include at least one of flipping and rotating the image. Even if the correct values ​​of the physical properties are the same, differences in the deposition state of the imaged aggregate may result in the image being recognized as having different characteristics when evaluated using a trained model, resulting in reduced prediction accuracy. Therefore, while it is conceivable to perform machine learning by preparing a large number of training image data with different deposition states, this would complicate the preparation of the training data. In contrast, the quality prediction method described above involves at least one of flipping and rotating the image, generating new image data, and expanding the training data. Therefore, this method is even more useful for improving prediction accuracy while avoiding the complication of preparing training data for machine learning.

[0153] The quality prediction program described above is a program for causing a computer to execute the aggregate quality prediction method described above. Like the aggregate quality prediction method, this quality prediction program is useful for improving the prediction accuracy of a prediction model constructed by machine learning.

[0154] The quality prediction device (50) described above includes a first acquisition unit (62) that acquires color image data (P1) obtained by capturing an image of an aggregate, a pre-processing unit (64) that generates input image data (P4) by sequentially performing a first process of grayscaling the captured image data (P1), a second process of darkening the image, and a third process of binarizing the image data (P1), a second acquisition unit (66) that acquires input information including the input image data (P4), and a prediction unit (72) that acquires physical property values ​​corresponding to the input information acquired by the second acquisition unit (66) using a prediction model (M) previously constructed by machine learning to output physical property values ​​of the aggregate in response to input of the input information. This quality prediction device (50), like the above-described aggregate quality prediction method, is useful for improving the prediction accuracy of a prediction model constructed by machine learning.

[0155] The above-described method for producing ready-mixed concrete includes a production step of kneading materials, including aggregate, to produce ready-mixed concrete, and a quality prediction step of predicting the quality of at least a portion of the aggregate used in the production step. The quality prediction step includes a first acquisition step of acquiring color image data (P1) obtained by capturing an image of the aggregate, a preprocessing step of generating input image data (P4) by sequentially performing a first grayscale conversion process, a second darkening process, and a third binarization process on the captured image data (P1), a second acquisition step of acquiring input information including the input image data (P4), and a prediction step of acquiring physical property values ​​corresponding to the input information acquired in the second acquisition step using a prediction model (M) previously constructed by machine learning to output physical property values ​​of the aggregate in response to input of the input information. This method for producing ready-mixed concrete, like the above-described method for predicting aggregate quality, is useful for improving the prediction accuracy of the prediction model constructed by machine learning.

[0156] The ready-mixed concrete manufacturing system (1) described above includes a manufacturing apparatus (10) that mixes materials including aggregate to manufacture ready-mixed concrete, and a quality prediction device (50) that predicts the quality of at least a portion of the aggregate used by the manufacturing apparatus (10). The quality prediction device (50) includes a first acquisition unit (62) that acquires color image data (P1) obtained by capturing an image of the aggregate, a pre-processing unit (64) that generates input image data (P4) by sequentially performing a first process of grayscaling the captured image data (P1), a second process of darkening the captured image data (P1), and a third process of binarizing the captured image data (P1), a second acquisition unit (66) that acquires input information including the input image data (P4), and a prediction unit (72) that acquires physical property values ​​according to the input information acquired by the second acquisition unit (66) using a prediction model (M) previously constructed by machine learning to output physical property values ​​of the aggregate according to the input of the input information. This manufacturing system (1) is useful for improving the prediction accuracy of a prediction model constructed by machine learning, similar to the above-mentioned aggregate quality prediction method. [Explanation of symbols]

[0157] 1... manufacturing system, 10... manufacturing equipment, 50... quality prediction device, 62... imaging data acquisition unit, 64... pre-processing unit, 66... ​​input data acquisition unit, 68... model construction unit, 72... prediction unit, P1... imaging image data, P4... input image data, M... prediction model.

Claims

1. a first acquisition step of acquiring color image data obtained by imaging the aggregate; a pre-processing step of generating input image data by sequentially performing a first process of grayscaling, a second process of darkening, and a third process of binarizing the captured image data; a second acquisition step of acquiring input information including the input image data; a prediction step of acquiring the physical property values ​​corresponding to the input information acquired in the second acquisition step by using a prediction model previously constructed by machine learning so as to output the physical property values ​​of the aggregate in response to the input of the input information, Methods for predicting aggregate quality.

2. In the second process, Ai, Bi, A0, B0, C, D, E, and MAX[a, b] are respectively set as follows: Ai: pixel value of each pixel in the image before processing Bi: pixel value of each pixel in the processed image A0: average pixel value in the image before processing B0: A target value set in advance for the pixel value C: Value obtained by dividing B0 by A0 D: The value obtained by subtracting A0 from B0 E: Any specified value MAX[a, b]: An operation that outputs the larger value of the numerical value a and the numerical value b. When the above definition is made, the darkening process is performed using one of the following formulas (1), (2), (3), and (4): Bi=Ai×C...(1) Bi=MAX[(Ai-D),0]...(2) Bi=MAX[(Ai-E), 0]...(3) Bi=MAX[(Ai×C-E),0]...(4) The method for predicting aggregate quality according to claim 1.

3. In the third processing, binarization processing is performed by adaptive binarization or Otsu's binarization. The method for predicting aggregate quality according to claim 1 or 2.

4. a preparation step of preparing training data including training input information corresponding to the input information and correct answer information of the physical property value associated with the training input information; and a construction step of constructing the prediction model by machine learning based on the training data, The preparation step includes generating extended image data by performing a conversion process and the first process in any order on training captured image data corresponding to the captured image data, the transformation process includes randomly changing the brightness and contrast of the image; The input image data in the training input information includes image data obtained by sequentially performing the second processing and the third processing on the extended image data. The method for predicting aggregate quality according to claim 1 or 2.

5. The transformation process further includes at least one of flipping the image and rotating the image. The method for predicting aggregate quality according to claim 4.

6. A quality prediction program for causing a computer to execute the aggregate quality prediction method according to claim 1 or 2.

7. a first acquisition unit that acquires color image data obtained by capturing an image of the aggregate; a pre-processing unit that generates input image data by sequentially performing a first process of grayscaling, a second process of darkening, and a third process of binarizing the captured image data; a second acquisition unit that acquires input information including the input image data; a prediction unit that acquires the physical property values ​​according to the input information acquired by the second acquisition unit by using a prediction model that has been constructed in advance by machine learning so as to output physical property values ​​of the aggregate according to the input information, Aggregate quality prediction device.

8. A manufacturing process in which materials including aggregate are mixed to produce ready-mix concrete; a quality prediction step of predicting the quality of at least a portion of the aggregate used in the manufacturing step; Including, The quality prediction step includes: a first acquisition step of acquiring color image data obtained by imaging the aggregate; a pre-processing step of generating input image data by sequentially performing a first process of grayscaling, a second process of darkening, and a third process of binarizing the captured image data; a second acquisition step of acquiring input information including the input image data; a prediction step of acquiring the physical property values ​​corresponding to the input information acquired in the second acquisition step by using a prediction model previously constructed by machine learning so as to output the physical property values ​​of the aggregate in response to the input of the input information, Method for manufacturing ready-mix concrete.

9. A manufacturing device that mixes materials including aggregate to manufacture ready-mix concrete; a quality prediction device that predicts the quality of at least a portion of the aggregate used by the manufacturing device; The quality prediction device includes: a first acquisition unit that acquires color image data obtained by capturing an image of the aggregate; a pre-processing unit that generates input image data by sequentially performing a first process of grayscaling, a second process of darkening, and a third process of binarizing the captured image data; a second acquisition unit that acquires input information including the input image data; a prediction unit that acquires the physical property values ​​according to the input information acquired by the second acquisition unit by using a prediction model that has been constructed in advance by machine learning so as to output physical property values ​​of the aggregate according to the input information, Ready-mix concrete manufacturing system.

Images