Aggregate quality prediction method, quality prediction program, aggregate quality prediction device, ready-mix concrete manufacturing method, and ready-mix concrete manufacturing system

Abstract

Improve the accuracy of quality predictions by using predictive models built using machine learning. [Solution] The method for predicting the quality of aggregate includes a first acquisition step of acquiring color image data obtained by imaging an aggregate to be evaluated, a pre-processing step of generating input image data by grayscaling the image data, a second acquisition step of acquiring input information including the input image data, a prediction step of acquiring physical property values ​​corresponding to the input information acquired in the second acquisition step using a prediction model previously constructed to output physical property values ​​of the aggregate according to the input of the input information, and a construction step of constructing the prediction model by machine learning based on training data. When the illuminance of the environmental conditions when imaging the training aggregate is IA[lx] and the illuminance of the environmental conditions when imaging the aggregate to be evaluated is IB[lx], IB[lx] is 0.2 times or more than IA[lx] and 5 times or less than IA[lx].

Description

[Technical field]

[0001] The present disclosure relates to an aggregate quality prediction method, a quality prediction program, an aggregate quality prediction device, a ready-mix concrete manufacturing method, and a ready-mix concrete manufacturing system. [Background technology]

[0002] Patent Document 1 discloses an aggregate quality assessment system for assessing the quality of aggregate, and an aggregate quality assessment method. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2023-137651 A Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides an aggregate quality prediction method, a quality prediction program, an aggregate quality prediction device, a ready-mix concrete manufacturing method, and a ready-mix concrete manufacturing system, which are useful for improving prediction accuracy when predicting quality using a prediction model constructed by machine learning. [Means for solving the problem]

[0005] a prediction step of acquiring physical property values ​​corresponding to the input information acquired in the second acquisition step by using a prediction model previously constructed so as to output physical property values ​​of the aggregate in response to input of the input information; and a construction step of constructing the prediction model by machine learning based on training data including training input information corresponding to the input information and ground truth information for the physical property values ​​corresponding to the training input information, wherein an illuminance of an environmental condition when an image of a training aggregate is captured to prepare the training input information is defined as IA[lx], and an illuminance of an environmental condition when an image of the aggregate is captured to obtain the image data is defined as IB[lx], where IB[lx] is equal to or greater than 0.2 times IA[lx] and equal to or less than 5 times IA[lx].

[0006] [2] The method for predicting quality of aggregate described in [1] above, wherein the absolute value of the difference between IA [lx], which represents the illuminance among the environmental conditions when the training aggregate is imaged to prepare the training input information, and IB [lx], which represents the illuminance among the environmental conditions when the aggregate to be evaluated is imaged to obtain the image data, is 2500 lx or less.

[0007] [3] The method for predicting quality of aggregate described in [1] or [2] above, wherein IB [lx], which represents the illuminance among the environmental conditions when imaging the aggregate to be evaluated to obtain the image data, is 2000 lx or more.

[0008] [4] In the pre-processing step, when the pixel values ​​of the red, green, and blue color components of each pixel in the color image data are represented as R, G, and B, and the pixel value of each pixel in the grayscale image data is represented as Y, grayscale conversion is performed using the following formula (1): Y=0.299×R+0.587×G+0.114×B (1) The method for predicting aggregate quality according to any one of the above [1] to [3].

[0009] [5] A method for predicting the quality of aggregate described in any one of [1] to [4] above, further comprising a preparation step of preparing the training data, the preparation step including generating extended image data by subjecting training captured image data corresponding to the captured image data to a conversion process and a grayscale process in any order, the conversion process including randomly changing the brightness and contrast of an image, and the input image data in the training input information including the extended image data.

[0010] [6] The method for predicting quality of aggregate described in [5] above, wherein the conversion process further includes at least one of inverting the image and rotating the image.

[0011] [7] The method for predicting quality of aggregate described in any one of [1] to [6] above, wherein the construction step includes: preparing, separately from the training data, model evaluation data including input information for model evaluation corresponding to the input information and correct answer information for the physical properties corresponding to the input information for model evaluation; continuously updating the intermediate model using the training data so as to reduce an error with respect to the correct answer information for the physical properties; and determining, using the model evaluation data, one of the intermediate models generated during the update as the prediction model.

[0012] [8] A quality prediction program for causing a computer to execute the aggregate quality prediction method according to any one of [1] to [7] above.

[0013] a prediction unit that acquires physical property values ​​corresponding to the input information acquired by the second acquisition unit using a prediction model previously constructed so as to output physical property values ​​of the aggregate in response to input of the input information; and a construction unit that constructs the prediction model by machine learning based on training data including training input information corresponding to the input information and ground truth information for the physical property values ​​associated with the training input information, wherein an illuminance of an environmental condition when an image of a training aggregate is captured to prepare the training input information is defined as IA[lx], and an illuminance of an environmental condition when an image of the aggregate is captured to obtain the image data is defined as IB[lx], where IB[lx] is equal to or greater than 0.2 times IA[lx] and equal to or less than 5 times IA[lx].

[0014]

[10] A method for producing ready-mix concrete by mixing materials including aggregate, and a quality prediction process for predicting the quality of at least a part of the aggregate used in the production process, the quality prediction process including a first acquisition process for acquiring color image data obtained by imaging the aggregate to be evaluated, a pre-processing process for generating input image data by grayscaling the captured image data, a second acquisition process for acquiring input information including the input image data, and a prediction model for predicting the quality of the input information acquired in the second acquisition process using a prediction model previously constructed to output physical property values ​​of the aggregate in response to input of the input information. a prediction step of acquiring the physical property values ​​corresponding to the input information and a construction step of constructing the prediction model by machine learning based on training data including training input information corresponding to the input information and correct answer information for the physical property values ​​associated with the training input information, wherein when an illuminance among the environmental conditions when an image of a training aggregate is taken to prepare the training input information is defined as IA[lx], and an illuminance among the environmental conditions when an image of the aggregate to be evaluated is taken to obtain the captured image data is defined as IB[lx], IB[lx] is not less than 0.2 times IA[lx] and not more than 5 times IA[lx].

[0015]

[11] A manufacturing apparatus for manufacturing ready-mixed concrete by mixing materials including aggregate, and a quality prediction device for predicting the quality of at least a portion of the aggregate used by the manufacturing apparatus, wherein the quality prediction device includes a first acquisition unit that acquires color image data obtained by imaging an aggregate to be evaluated, a pre-processing unit that generates input image data by grayscaling the image data, a second acquisition unit that acquires input information including the input image data, and a prediction model that is constructed in advance to output physical property values ​​of the aggregate in response to input of the input information, and a quality prediction device that predicts the quality of at least a portion of the aggregate used by the manufacturing apparatus. a prediction unit that acquires the physical property values, and a construction unit that constructs the prediction model by machine learning based on training data including training input information corresponding to the input information and correct answer information for the physical property values ​​associated with the training input information, wherein when an illuminance among the environmental conditions when an image of a training aggregate is captured to prepare the training input information is defined as IA[lx], and an illuminance among the environmental conditions when an image of the aggregate to be evaluated is captured to obtain the captured image data is defined as IB[lx], IB[lx] is equal to or greater than 0.2 times IA[lx] and equal to or less than 5 times IA[lx]. Effect of the Invention

[0016] According to the present disclosure, there are provided an aggregate quality prediction method, a quality prediction program, an aggregate quality prediction device, a ready-mix concrete manufacturing method, and a ready-mix concrete manufacturing system, which are useful for improving prediction accuracy when predicting quality using a prediction model constructed by machine learning. [Brief description of the drawings]

[0017] [Figure 1] FIG. 1 is a schematic diagram showing an example of a ready-mix concrete manufacturing system. [Diagram 2] FIG. 2 is a block diagram showing an example of a hardware configuration of a calculation device included in the quality prediction device. [Diagram 3] FIG. 3 is a block diagram illustrating an example of a functional configuration of the arithmetic device. [Figure 4]FIG. 4 is a diagram for explaining problems that arise when the imaging environment is different. [Diagram 5] FIG. 5 is a diagram illustrating an example of a calculation process when predicting quality using a prediction model. [Figure 6] FIG. 6 is a diagram showing an example of captured image data for training. [Figure 7] FIG. 7 is a flowchart illustrating a series of processes in the training phase. [Figure 8] FIG. 8 is a diagram illustrating an example of how extended image data is generated. [Figure 9] FIG. 9 is a flowchart illustrating a series of processes executed during model construction. [Figure 10] FIG. 10 is a diagram illustrating the calculation process when constructing a model. [Figure 11] FIG. 11 is a flowchart illustrating a series of processes in the evaluation phase. [Figure 12] 12(a) and 12(b) are graphs illustrating the results of comparing predicted values ​​with ground truth values ​​in a test data set. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, an embodiment will be described with reference to the drawings. In the description, the same elements or elements having the same functions are denoted by the same reference numerals, and duplicated description will be omitted.

[0019] [Ready-mix concrete manufacturing system] Fig. 1 shows a schematic diagram of a ready-mix concrete manufacturing system according to one embodiment. The manufacturing system 1 shown in Fig. 1 is a system for manufacturing ready-mix concrete. The manufacturing system 1 is installed, for example, in a factory for manufacturing ready-mix concrete. The manufacturing system 1 manufactures (produces) ready-mix concrete by mixing concrete materials.

[0020] The concrete materials used in the manufacturing system 1 include cement, admixtures, coarse aggregates, fine aggregates, water, and admixtures. Examples of the coarse aggregates include gravel, crushed stone, slag coarse aggregates, lightweight coarse aggregates, recycled coarse aggregates, recovered aggregates, and coarse aggregates made by mixing these. The gravel includes mountain gravel, land gravel, river gravel, and sea gravel. The slag coarse aggregates include blast furnace slag aggregates, ferro-nickel slag aggregates, electric furnace oxidized slag aggregates, and coal gasification slag aggregates. The lightweight coarse aggregates include natural lightweight aggregates, by-product lightweight aggregates, and artificial lightweight aggregates. The coarse aggregates may include crushed rock, or limestone crushed stone.

[0021] Examples of fine aggregates include sand, crushed sand, slag fine aggregate, lightweight fine aggregate, recycled fine aggregate, recovered aggregate, and fine aggregates made by mixing these. Sand includes mountain sand, land sand, river sand, sea sand, etc. Slag fine aggregates include blast furnace slag aggregate, ferronickel slag aggregate, copper slag aggregate, electric furnace oxidized slag aggregate, coal gasification slag aggregate, etc. Lightweight fine aggregates include natural lightweight aggregate, by-product lightweight aggregate, artificial lightweight aggregate, etc.

[0022] Rock types of the crushed stone and crushed sand include, for example, igneous rocks, sedimentary rocks, metamorphic rocks, quartzite, limestone, domalite, or peridotite. Igneous rocks include granite, diorite, gabbro, porphyrite, diabase, rhyolite, andesite, basalt, or serpentinite. Sedimentary rocks include conglomerate, sandstone, shale, slate, or tuff. Metamorphic rocks include gneiss or schist.

[0023] The manufacturing system 1 has a function of predicting the quality of aggregate contained in concrete materials in addition to a function of manufacturing ready-mix concrete. In this disclosure, coarse aggregate and fine aggregate may be collectively referred to as "aggregate." In this case, "aggregate" means coarse aggregate, fine aggregate, or both coarse aggregate and fine aggregate.

[0024] The manufacturing system 1 loads the manufactured ready-mix concrete onto a transport vehicle C. After the ready-mix concrete has been loaded, the transport vehicle C transports the ready-mix concrete to a site (e.g., a construction site) where the ready-mix concrete will be used. Examples of the transport vehicle C include an agitator vehicle (mixer vehicle) or a dump truck. The manufacturing system 1 may manufacture ready-mix concrete from concrete materials so as to satisfy a target quality (required quality) set for each site. For example, an operator of the manufacturing system 1 determines the mix of concrete materials so as to satisfy the management quality at the time of shipment based on the target quality at the site, and inputs operation instructions to the manufacturing system 1.

[0025] The manufacturing system 1 includes, for example, a material storage yard 2, a transport device 8, a manufacturing device 10, and a quality prediction device 50.

[0026] The material storage yard 2 is a place where concrete materials are stored. The material storage yard 2 includes a plurality of silos 4. The plurality of silos 4 are containers that store at least a portion of the concrete materials by material type. The plurality of silos 4 include a silo 4 that stores coarse aggregate, a silo 4 that stores fine aggregate, and a silo 4 that stores cement.

[0027] The transporting device 8 is a device that transports the concrete materials stored in the multiple silos 4 to the manufacturing equipment 10. The transporting device 8 includes, for example, a belt conveyor that transports the concrete materials. The transporting device 8 may transport the concrete materials by type at different times. In one example, based on an operation instruction from a control device included in the manufacturing system 1, a specific material from among the various concrete materials is transferred to the transporting device 8 and transported to the manufacturing equipment 10.

[0028] The manufacturing apparatus 10 is an apparatus that manufactures ready-mixed concrete by mixing concrete materials including aggregate. The manufacturing apparatus 10 operates based on an operation instruction from a control device included in the manufacturing system 1. The manufacturing apparatus 10 includes, for example, a storage bottle 12, a measuring bottle 14, a collecting hopper 16, a mixer 20, and a loading hopper 30.

[0029] Storage bottle 12 temporarily stores various types of concrete materials. Various types of concrete materials are transported (conveyed) to storage bottle 12 from material storage yard 2 by transport device 8. Storage bottle 12 is configured to store various types of concrete materials individually. Hereinafter, "concrete material" may be simply referred to as "material". Various materials stored in storage bottle 12 are supplied to measuring bottle 14 as required.

[0030] The measuring bottle 14 is disposed below the storage bottle 12. The measuring bottle 14 operates based on operational instructions from a control device of the manufacturing system 1, and weighs each material individually. When the measuring bottle 14 detects a target amount of material instructed by the control device, it supplies the material to a collecting hopper 16. When water is supplied to the measuring bottle 14, an admixture may be mixed into the water. The collecting hopper 16 is disposed below the measuring bottle 14. The collecting hopper 16 collects the various materials discharged from the measuring bottle 14, and supplies the collected materials to a mixer 20.

[0031] The mixer 20 is disposed below the collecting hopper 16. The mixer 20 is a device that mixes concrete materials. The mixer 20 produces ready-mixed concrete by mixing (kneading) aggregate, cement, water, admixtures, etc. The ready-mixed concrete is discharged from the bottom of the mixer 20 into the loading hopper 30. The loading hopper 30 is disposed below the mixer 20, and temporarily stores the ready-mixed concrete. The loading hopper 30 supplies the temporarily stored ready-mixed concrete to the transport vehicle C.

[0032] The manufacturing apparatus 10 described above is an example of a ready-mix concrete manufacturing apparatus, and the ready-mix concrete manufacturing apparatus may be configured in any manner as long as it is capable of mixing concrete materials and manufacturing ready-mix concrete. For example, the manufacturing apparatus 10 does not need to include the collection hopper 16, and various materials measured in the measuring bottle 14 may be supplied from the measuring bottle 14 to the mixer 20.

[0033] (Quality Prediction Device) The quality prediction device 50 (aggregate quality prediction device) is a device that predicts the quality of aggregate contained in concrete material. The quality prediction device 50 may predict the quality of at least a part of the aggregate used by the manufacturing device 10. It is known that the quality of the ready-mix concrete manufactured by the manufacturing device 10 is affected by the quality of the concrete material. Specifically, the particle size of the aggregate used to manufacture the ready-mix concrete affects the fluidity (e.g., slump and slump flow) of the ready-mix concrete. For example, with the same unit water content, the larger the particle size of the aggregate, the smaller the surface area of ​​the aggregate, which reduces the water retention capacity, and the greater the fluidity of the ready-mix concrete. Conversely, the smaller the particle size of the aggregate, the larger the surface area of ​​the aggregate, which increases the water retention capacity, and the less the fluidity of the ready-mix concrete. Thus, in order to stabilize the quality including the fluidity of the ready-mix concrete, it is necessary to manage the particle size of the aggregate.

[0034] The grading of aggregate can be expressed by a coarse aggregate ratio, and in order to stabilize the quality of ready-mixed concrete, it is necessary to manage, for example, whether the coarse aggregate ratio is within a desired range. The quality prediction device 50 may predict the coarse aggregate ratio as the quality of the aggregate. The quality prediction device 50 may be configured to predict the coarse aggregate ratio from information including image data based on captured image data obtained by capturing an image of the aggregate, using a prediction model constructed by machine learning.

[0035] At least a part of the quality prediction device 50 is configured by one or more computers. The computer that configures at least a part of the quality prediction device 50 may be a personal computer, a tablet computer (tablet terminal), a smartphone, a wearable device, a workstation, a server computer, or a general-purpose computer.

[0036] When the quality prediction device 50 includes two or more computers, these computers may be communicatively connected to each other. The quality prediction device 50 may be communicatively connected to a control device included in the manufacturing system 1. The quality prediction device 50 may be a part of the function of the control device included in the manufacturing system 1 (i.e., may be included in the control device).

[0037] The quality prediction device 50 includes, for example, a calculation device 52, an input / output device 54, and an imaging device 58. The calculation device 52 is a device that performs calculations for predicting the quality of aggregate. The calculation device 52 may be a computer main body (a main body of a computer) that executes the main functions of the computer that constitutes the quality prediction device 50.

[0038] The input / output device 54 is connected to the arithmetic unit 52. The input / output device 54 is a device having a function of inputting information indicating instructions from a user such as a worker to the arithmetic unit 52, and a function of outputting information from the arithmetic unit 52 to the user such as a worker. The input / output device 54 may include a keyboard, an operation panel, or a mouse as an input device, and may include a monitor (e.g., a liquid crystal display) as an output device. The input / output device 54 may be a touch panel in which an input device and an output device are integrated. The arithmetic unit 52 and the input / output device 54 may be integrated.

[0039] The imaging device 58 is a device (camera) capable of capturing an image of the aggregate. The imaging device 58 generates color image data by, for example, capturing an image of the aggregate. Hereinafter, the color image data generated by the imaging device 58 will be referred to as "captured image data P1." The imaging device 58 is a digital camera or a video camera that captures an image within a field of view (imaging range) based on visible light. The color captured image data P1 generated by the imaging device 58 is used when predicting the quality of the aggregate. When the imaging device 58 generates video data, still image data included in the video may be acquired as the captured image data P1.

[0040] In the quality prediction device 50, the arithmetic device 52, the input / output device 54, and the imaging device 58 may be integrated together, such as a smartphone with a built-in (mounted) camera or a tablet computer with a built-in camera. In this case, the quality prediction device 50 may be portable by a worker or the like. In one example, the quality prediction device 50 is configured by installing an application for predicting the quality of aggregate in the smartphone or tablet computer.

[0041] The imaging device 58 may be disposed in the manufacturing system 1 so as to be able to capture an image of an aggregate at a specific location. For example, the imaging device 58 is disposed so as to be able to capture an image of the aggregate while it is being transported by the transporting device 8. The imaging device 58 may be mounted on a moving body such as a drone, and the imaging device 58 may capture an image of the aggregate by, for example, operating the moving body by a user. The imaging device 58 may capture an image of the aggregate in a room where the imaging environment, such as illuminance, is adjusted.

[0042] 2, the arithmetic device 52 includes a circuit 91. The circuit 91 includes a processor 92, a memory 93, a storage 94, and an input / output port 95. The storage 94 is configured with one or more non-volatile memory devices such as a flash memory or a hard disk. The storage 94 stores at least a quality prediction program for causing a computer to execute a quality prediction process described later. The storage 94 stores a quality prediction program for configuring each functional block of the arithmetic device 52 described later.

[0043] The memory 93 is composed of one or more volatile memory devices such as a random access memory. The memory 93 temporarily stores a quality prediction program loaded from the storage 94. The processor 92 is composed of one or more arithmetic devices such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The processor 92 configures each functional block of the arithmetic device 52 by executing the quality prediction program loaded into the memory 93. The calculation results by the processor 92 are temporarily stored in the memory 93. The input / output port 95 inputs and outputs information between the input / output device 54, the imaging device 58, etc. in response to a request from the processor 92.

[0044] The circuit 91 is not necessarily limited to one that configures each function by a program. For example, the circuit 91 may configure at least some of its functions by a dedicated logic circuit or an ASIC (Application Specific Integrated Circuit) that integrates the dedicated logic circuit. The quality prediction program may be provided in a state of being fixedly recorded on a tangible recording medium such as a CD-ROM, a DVD-ROM, or a semiconductor memory. Alternatively, the quality prediction program may be provided via a communication network as a data signal superimposed on a carrier wave.

[0045] 3 shows an example of functional components (referred to as "functional blocks" in this disclosure) included in the arithmetic device 52. The arithmetic device 52 has, as functional blocks, an imaging data acquisition unit 62 (first acquisition unit), a preprocessing unit 64, an input data acquisition unit 66 (second acquisition unit), a model construction unit 68 (construction unit), an imaging condition holding unit 69, a model holding unit 70, a prediction unit 72, and an output unit 74. The processes executed by these functional blocks correspond to the processes executed by the arithmetic device 52 (quality prediction device 50).

[0046] The imaging data acquisition unit 62 is configured to acquire captured image data P1, which is color image data obtained by capturing an image of an aggregate to be evaluated for quality. Hereinafter, the aggregate to be evaluated for quality is referred to as the "aggregate to be evaluated." The imaging data acquisition unit 62 acquires, for example, captured image data P1 obtained by capturing an image with the imaging device 58 from the imaging device 58.

[0047] In the color captured image data P1, pixel values ​​of the red, green, and blue color components are determined for each pixel. If the pixel value of the red component, the pixel value of the green component, and the pixel value of the blue component of each pixel are expressed as "R," "G," and "B," respectively, the pixel value of each pixel is expressed as (R,G,B). The pixel value is also called the brightness value, density value, shade value, gradation value, or color value. The pixel value is expressed as a numerical value ranging from 0 to 255, for example.

[0048] The pre-processing unit 64 generates pre-processed image data by performing a predetermined pre-processing on the captured image data P1. The pre-processing unit 64 generates pre-processed image data (hereinafter referred to as "input image data P2") by performing a grayscale process on the captured image data P1. The pre-processing unit 64 may perform other processes as part of the pre-processing, at least either before or after the grayscale process, while maintaining the input image data P2 as grayscale image data. The pre-processing unit 64 may perform only the grayscale process as the pre-processing.

[0049] Here, the significance of performing preprocessing including grayscaling processing will be described with reference to Fig. 4. The present inventors have confirmed that there are the following problems when predicting aggregate granularity (coarse granularity ratio) using a prediction model constructed by machine learning so as to output physical property values ​​related to aggregate granularity in response to input of color captured image data P1. It has been confirmed that the prediction accuracy decreases or the variance of prediction results increases when imaging conditions (imaging environment) such as illuminance or camera position are different between training and evaluation, and / or when the aggregate granularity to be evaluated is a granularity not used during training.

[0050] In FIG. 4, image data a1 to a7 with different image brightness are shown as color captured image data. Image data a4 is the original captured image data, and other image data are generated by adjusting the brightness of the image data a4. As illustrated in FIG. 4, if the brightness is different, the image features may be different even if the captured object is the same aggregate. Therefore, the inventors have verified various image preprocessing so as not to cause a decrease in prediction accuracy even if the imaging environment is different, and have come up with the idea of ​​performing preprocessing including a grayscale process on the captured image data P1 and using the grayscale image data as input data for the prediction model. On the other hand, the inventors have confirmed that simply grayscaling is not preferable from the viewpoint of quality prediction accuracy when they have performed further verification, but this point will be described in detail later. In the following description, "preprocessing" means preprocessing including a grayscale process, which generates grayscale image data as processed image data.

[0051] 5 shows a schematic diagram of a calculation process in prediction using a prediction model for predicting quality. The pre-processing unit 64 generates input image data P2 by performing pre-processing (for example, only grayscale processing) on ​​the captured image data P1. In the pre-processing, the pre-processing unit 64 converts a combination of red, green, and blue pixel values ​​(R, G, B) for each pixel into a single pixel value. When the pixel value of each pixel in the input image data P2 (grayscale image data) is represented as "Y", the pre-processing unit 64 may perform grayscale conversion for each pixel using the following formula (1). Y=0.299×R+0.587×G+0.114×B (1)

[0052] 3, the input data acquisition unit 66 acquires input information including the input image data P2. The input data acquisition unit 66 may acquire only the input image data P2 as the input information. The input data acquisition unit 66 may acquire information that may affect the granularity of the aggregate in addition to the input image data P2 as the input information.

[0053] The model construction unit 68 constructs a model (hereinafter, referred to as "prediction model M") for predicting the granularity of aggregate. The prediction model M is a model constructed by machine learning so as to output the physical property value of aggregate according to input information including the input image data P2. The physical property value of aggregate is a physical property value related to the granularity of aggregate. Specific examples of the physical property value related to the granularity of aggregate include the mass fraction remaining between successive sieves (mass fraction for each granularity classification), the mass fraction remaining on each sieve, the mass fraction passing through each sieve, and the coarse aggregate fraction of aggregate, as specified in JIS A 1102:2014 "Test method for sieving aggregate", the fine aggregate content of aggregate, as specified in JIS A 1102:2014 "Test method for fine aggregate content", and the excessive aggregate content (amount of fine aggregate remaining on a 5 mm sieve) and the excessive aggregate content (amount of coarse aggregate passing through a 5 mm sieve). The prediction unit 72 constructs a prediction model M by machine learning based on, for example, input information including the input image data P2 and correct answer information of the mass fraction remaining between each successive sieve associated with the input information.

[0054] Machine learning refers to a method in which a machine (computer) autonomously finds laws or rules by repeatedly learning based on given information. The prediction model M can be constructed using an algorithm and a data structure. The prediction model M is realized, for example, using a neural network, which is an information processing model that mimics the mechanism of the human brain. There is no particular limitation on the specific algorithm of the machine learning performed when constructing the prediction model M. The neural network has an input layer, one or more intermediate layers, and an output layer. By including one or more intermediate layers, a more complex prediction model M can be constructed, and prediction accuracy can be improved.

[0055] The model construction unit 68 may construct the prediction model M by machine learning using a convolutional neural network (CNN). By using a convolutional neural network, it is possible to more appropriately capture the features of the image. The model construction unit 68 may construct the prediction model M by machine learning based on the input image data P2 and correct answer information of the mass fraction remaining between each successive sieve. The prediction model M constructed by the model construction unit 68 may output a value indicating the mass fraction remaining between each successive sieve in accordance with the input image data P2. In other words, the prediction model M may output a plurality of values ​​each indicating a mass fraction for a plurality of particle size divisions.

[0056] The model construction unit 68 may autonomously construct a prediction model M by performing machine learning using data provided as input for machine learning and correct answer information of the output of the machine learning (such as correct answer values ​​of the mass fraction remaining between each successive sieve). The input for the machine learning is various data sets of input information including the input image data P2. The output of the machine learning is data (numerical values) indicating physical property values ​​such as the mass fraction remaining between each successive sieve. The model construction unit 68 iteratively trains a model that outputs physical property values ​​related to the granularity of the aggregate using multiple combinations of the data sets of input information and correct answer values ​​of the physical property values.

[0057] The stage of autonomously constructing the prediction model M corresponds to the training phase. The training phase is also called the learning phase. The same (mutually corresponding) information and aggregates are used in the training phase and the evaluation phase. Hereinafter, the terms "for training" and "for evaluation" may be used to distinguish the information and aggregates used in the training phase from the information and aggregates used in the evaluation phase.

[0058] The imaging condition holding unit 69 holds information representing the environmental conditions (imaging conditions) when the training bone material is imaged in the training phase. The information representing the environmental conditions held (stored) by the imaging condition holding unit 69 includes the illuminance when the training bone material is imaged. The model holding unit 70 holds the prediction model M constructed by the model construction unit 68. The prediction model M, which is a trained model, may be transferable between computers. The prediction model M constructed in the quality prediction device 50 may also be used in other devices.

[0059] In the evaluation phase, the prediction unit 72 predicts the quality of the aggregate to be evaluated. The prediction unit 72 uses the prediction model M to acquire physical property values ​​according to the input information for evaluation acquired by the input data acquisition unit 66. For example, the prediction unit 72 inputs the input information for evaluation to the prediction model M, and then acquires values ​​indicating the mass fraction remaining between each successive sieve that are output from the prediction model M. The prediction unit 72 may calculate a predicted value of the coarse fraction of the aggregate to be evaluated by converting the values ​​indicating the mass fraction remaining between each successive sieve into a coarse fraction. The prediction unit 72 may correct the values ​​indicating the mass fractions output from the prediction model M before converting them into a coarse fraction.

[0060] The output unit 74 outputs the quality prediction result by the prediction unit 72 to the monitor of the input / output device 54. The output unit 74 may cause the monitor to display the predicted value of the coarse particle ratio calculated by the prediction unit 72. The output unit 74 may cause the monitor to display, in addition to the predicted value of the coarse particle ratio, a value obtained from the prediction model M regarding the mass fraction remaining between each successive sieve. The output unit 74 may cause the monitor to display a graph of the particle size distribution based on the predicted value of the mass fraction remaining between each successive sieve.

[0061] <Environmental conditions when capturing image data P1> In the quality prediction using the prediction model M by the quality prediction device 50, the relationship of illuminance among the environmental conditions when imaging the aggregate to obtain the captured image data P1 is regulated between the training phase and the evaluation phase. In the training phase, the illuminance in the vicinity of the training aggregate imaged by the imaging device 58 may be measured. For example, when the imaging device 58 images the training aggregate stacked on a tray, a illuminometer may be installed on the imaging table on which the tray is placed, and the illuminance may be measured as one of the environmental conditions (imaging environment).

[0062] Here, "illuminance IA" and "illuminance IB" are defined as follows: Each of illuminance IA and illuminance IB represents an illuminance in the vicinity of the bone material to be imaged. · Illuminance IA: Illuminance [lux: lx], which is one of the environmental conditions when imaging the training aggregate to prepare training input information including the training image data P1 during the training phase. Illuminance IB: Illuminance [lux: lx], which is one of the environmental conditions when capturing an image of the aggregate to be evaluated in order to obtain the captured image data P1 for evaluation during the evaluation phase.

[0063] From the viewpoint of suppressing a decrease in prediction accuracy by the prediction model M, the illuminance IB is 0.2 times or more and 5 times or less than the illuminance IA. That is, the imaging data acquisition unit 62 acquires, as the captured image data P1, image data obtained by capturing an image under environmental conditions (imaging conditions) in which the illuminance IB is 0.2 times or more and 5 times or less than the illuminance IA. In this case, in the evaluation phase, the imaging device 58 captures an image of the aggregate for evaluation under environmental conditions (imaging conditions) in which the illuminance IB is 0.2 times or more and 5 times or less than the illuminance IA.

[0064] The minimum value of the range defining the illuminance IB may be 0.3 times the illuminance IA, 0.4 times the illuminance IA, or 0.5 times the illuminance IA, from the viewpoint of further suppressing a decrease in the prediction accuracy of the prediction model M. The maximum value of the range defining the illuminance IB may be 4 times the illuminance IA, 3 times the illuminance IA, 2 times the illuminance IA, or 1.5 times the illuminance IA, from the viewpoint of further suppressing a decrease in the prediction accuracy of the prediction model M. In one example, the illuminance IB is 0.5 times or more and 2 times or less than the illuminance IA. When the illuminance IA is 3000lx to 4000lx, the illuminance IB may be 0.5 times or more and 2 times or less than the illuminance IA.

[0065] From the viewpoint of suppressing a decrease in prediction accuracy by the prediction model M, the absolute value of the difference between the illuminance IA and the illuminance IB may be 2500 lx or less. From the viewpoint of further suppressing a decrease in prediction accuracy by the prediction model M, the absolute value of the difference between the illuminance IA and the illuminance IB may be 2400 lx, 2300 lx, 2200 lx, 2100 lx, or 2000 lx or less. In one example, the illuminance IB is equal to or greater than a value obtained by subtracting 2000 lx from the illuminance IA and equal to or less than a value obtained by adding 2000 lx to the illuminance IA (within the range of IA±2000 lx). The illuminance IA may be greater than 2500 lx. The illuminance IA may be greater than 2600 lx, 2700 lx, 2800 lx, or 2900 lx. When the illuminance IA is 3000 lx to 4000 lx, the absolute value of the difference between the illuminance IA and the illuminance IB may be within 2000 lx.

[0066] The illuminance IB may be 2000 lx or more. The illuminance IB may be in the range of 2000 lx to 6000 lx. Generally, the illuminance at a location 1 m inside from a south window in a room during a sunny day is set to 3000 lx to 5000 lx. Therefore, by setting the illuminance IB within the range of 2000 lx to 6000 lx, it is easy to adjust the environmental conditions when capturing images indoors. The illuminance IB may be 2200 lx or more, 2400 lx or more, 2500 lx or more, 2600 lx or more, 2700 lx or more, 2800 lx or more, or 2900 lx or more. The illuminance IB may be 6000lx or less, 5800lx or less, 5600lx or less, 5500lx or less, 5400lx or less, 5300lx or less, 5200lx or less, or 5100lx or less. In one example, the illuminance IB is in the range of 2500lx to 5500lx, or in the range of 2900lx to 5100lx. Both the illuminance IA and the illuminance IB may be in the range of 2000lx to 6000lx. When the illuminance IA is 3000lx to 5000lx, the illuminance IB may be 2000lx to 6000lx, or 2500lx to 5500lx. When the illuminance IA is 3000 lx to 4000 lx, the illuminance IB may be 2000 lx to 6000 lx, or 2500 lx to 5500 lx.

[0067] To summarize the range of the illuminance IB, the range of the illuminance IB at least satisfies the following condition 1. The range of the illuminance IB may also satisfy both of the following conditions 1 and 2, both of the following conditions 1 and 3, or all of the following conditions 1, 2, and 3. Condition 1: Illuminance must be at least 0.2 times the illuminance IA and not more than 5 times the illuminance IA. Condition 2: The absolute value of the difference between illuminance IA and illuminance IB is 2500 lx or less. Condition 3: 2000lx or more. For example, when both condition 1 and condition 2 are satisfied, the minimum value of the range of illuminance IB is determined by the greater of the minimum values ​​of condition 1 and condition 2, and the maximum value of the range of illuminance IB is determined by the smaller of the maximum values ​​of condition 1 and condition 2. When both condition 1 and condition 3 are satisfied, the minimum value of the range of illuminance IB is determined by the greater of the minimum values ​​of condition 1 and condition 3, and the maximum value of the range of illuminance IB is determined by the maximum value of condition 1. When all of condition 1, condition 2, and condition 3 are satisfied, the minimum and maximum values ​​of the range of illuminance IB are determined in the same way as when both condition 1 and condition 2 are satisfied.

[0068] <Prediction model M> Next, the prediction model M used to predict the coarse grain ratio will be further described with reference to Figures 5 and 6. A part of Figure 5 shows a schematic diagram of a calculation process for predicting the coarse grain ratio using the prediction model M from input image data P2, which is at least a part of the input information to the prediction model M. The input of the prediction model M is, for example, only the input image data P2, and the output of the prediction model M is, for example, the mass fraction remaining between each successive sieve.

[0069] In the training phase for constructing the prediction model M, the same type of aggregate as the aggregate that can be the subject of evaluation is used as the training aggregate. When constructing the prediction model M, training data for machine learning (hereinafter, referred to as "training data TD") is prepared. The training data TD includes training input information and correct answer information of physical property values ​​associated with the training input information. The training data TD is composed of a plurality of data sets, and each of the plurality of data sets includes, for example, training input image data P2 and correct answer information of the mass fraction remaining between each successive sieve associated with the input image data P2. The training input image data P2 differs among the plurality of data sets.

[0070] When preparing various types of training input image data P2, for example, various types of aggregates are prepared, the mass fractions of which remaining between successive sieves are known. The above-mentioned various types of aggregates do not mean different types of aggregates, but the same type of aggregates with different grain sizes. In one example, the various types of training aggregates are individually imaged by the imaging device 58 while stacked on a flat tray. The stacked state (stacked state) refers to a state in which some grains contained in the aggregates overlap other grains when viewed vertically from above.

[0071] The imaging device 58 captures an image of an area including the aggregates on the trays on which the training aggregates are piled up, from a position (for example, a predetermined position above) a predetermined distance away from the trays. The imaging device 58 may capture images indoors or outdoors. The amount of aggregates placed on the trays when capturing images is not particularly limited, but may be about 0.1 kg to 10 kg. At least a part of the aggregates being transported on the belt conveyor included in the transport device 8 may be used as the training aggregates. In this case, the imaging device 58 may capture an image of the aggregates being transported on the belt conveyor of the transport device 8. The imaging device 58 may be installed so that the amount of the aggregates being transported included in the imaging field of view of the imaging device 58 is about 0.1 kg to 10 kg.

[0072] In one example, various samples are prepared as training aggregates under the conditions shown in Table 1 below, and the prepared samples are imaged to obtain training image data P1. The mass fractions remaining between successive sieves and the coarse particle fractions in the various samples prepared under the conditions shown in Table 1 are known, and at least a portion of the known values ​​is used as correct answer information in machine learning.

[0073] [Table 1]

[0074] The coarse aggregate fraction (FM) is calculated, for example, by the following formula (2) specified in JIS A 1102:2014 "Sieve analysis test method for aggregates". In addition, in "Standard Specifications for Building Construction and Commentary JASS5 Reinforced Concrete Construction", 9 sieves excluding 80 mm in formula (2) are used. The coarse aggregate fraction predicted in this disclosure may be specified by any method.

[0075]

number

[0076] In Table 1, this means that aggregates of 10 different grain sizes (coarse grain ratios) are prepared. Each level of aggregate is sieved in order of coarseness of the sieve and classified into categories I to IV. Category I represents the particle size category of particles larger than 10 mm and smaller than 20 mm. Particles that make up Category I are those that are retained on a 10 mm sieve but pass through a 20 mm sieve. Category II represents the particle size category for particles 5 mm or larger and smaller than 10 mm. Particles that make up Category II are those that are retained on a 5 mm sieve but pass through a 10 mm sieve. Category III represents the particle size category for particles 2.5 mm or larger and smaller than 5 mm. Particles that make up category III are those that are retained on a 2.5 mm sieve but pass through a 5 mm sieve. Category IV is the particle size category for particles 1.2 mm or larger and smaller than 2.5 mm. Particles that make up category IV are those that are retained on a 1.2 mm sieve but pass through a 2.5 mm sieve. The mass fraction retained in each successive sieve means that, for example, level 1 in Table 1, the particles in category I were 85 wt%, the particles in category II were 15 wt%, the particles in category III were 0 wt%, and the particles in category IV were 0 wt%.

[0077] Image data T1 to T10 shown in FIG. 6 are examples of image data obtained by imaging training aggregates prepared according to the conditions of each of levels 1 to 10 in Table 1. The image data exemplified in FIG. 6 is data obtained by imaging training aggregates placed on a sheet simulating the transport of aggregates on a belt conveyor. Instead of such a sheet, training aggregates may be placed on a flat tray to obtain image data related to training aggregates prepared according to the conditions of each of levels 1 to 10 in Table 1. Level k (k is an integer of 1 or more) corresponds to image data Tk. For example, image data T1 is obtained by imaging training aggregates prepared according to level 1. Training aggregates according to each level k contain grains of multiple grain size classifications.

[0078] For each level k, multiple image data Tk in which the states (more specifically, pile states) of the training aggregate are different from each other may be prepared. In one example, a first image is taken with the training aggregate placed on a tray, and one image data Tk is acquired. Then, the training aggregate is temporarily removed from the tray, and the same training aggregate (same sample) is placed on the tray again so that the pile state of the multiple particles is different from that at the time of the first image. After that, a second image is taken. By repeating the above-mentioned image capture, multiple image data Tk may be prepared for each level k.

[0079] In addition to the conditions shown in Table 1, various samples may be prepared as training bone materials under the conditions shown in Tables 2 and 3 below, respectively, and training captured image data P1 (image data Tk) may be prepared.

[0080] [Table 2]

[0081] [Table 3]

[0082] In one example, one data set in the training data is composed of one image data Tk and the correct answer information of the mass fraction at the level k associated with the image data Tk. As the training data, a plurality of data sets each consisting of a combination of the image data Tk and the correct answer information of the mass fraction may be prepared. Then, machine learning using such training data may be performed to construct a prediction model M that outputs predicted values ​​of four mass fractions for four grain size classes. The method of setting the mass fractions remaining between each successive sieve at each level in the training data, the number of levels (the value of k), and the grain size classes is not limited to the examples shown in Tables 1 to 3, and may be set arbitrarily.

[0083] An example of the prediction model M will be described using simplified formulas for ease of understanding. The prediction model M constructed by the model construction unit 68 can be simply expressed as, for example, the following formulas (3) and (4).

number

number

[0084] In formula (4), Y represents the output value of the physical property value, for example, the output value of the mass fraction remaining between successive sieves, and in this case, formulas (3) and (4) are calculated for each particle size division. N is an integer of 2 or more, and represents the number of data sets in the input information. xl represents various input values ​​included in the input information, and the input information includes at least input image data P2 based on captured image data P1.

[0085] wl is a weight (coefficient), and b is a bias term (coefficient). f(U) represents an activation function. The activation function is a linear function (identity function) or a nonlinear function such as a polynomial, absolute value, step function, sigmoid function, hardsigmoid function, logsigmoid function, softmax function, logsoftmax function, softmin function, softplus function, softsign function, tanh function, tanhShrink function, hardtanh function, tanhexp function, ReLU function, ReLU6 function, Leaky-ReLU function, PReLU function, ELU function, SELU function, CELU function, Swith function, Mish function, ACON function, etc.

[0086] The model construction unit 68 may repeatedly evaluate the error (loss) between Y (predicted value) obtained by formula (4) and the correct information of the physical property value using the training data, and may determine the weight wl and the bias term b in formula (3) so that the error is minimized. The model construction unit 68 may use any type of loss function as a function for evaluating the error between Y (output value from an intermediate model at an intermediate stage of constructing the prediction model M) obtained by formula (4) and the correct information of the physical property value. The loss function is a function for calculating a loss value based on the error between the predicted value and the correct value, and the weight wl is updated so that the loss value is minimized. Specific examples of the loss function include the Huber loss function, the mean absolute error, the mean square error, and the ε-allowable loss function.

[0087] The model construction unit 68 may repeatedly update the weight wl by the gradient method so as to minimize the loss value evaluated by the loss function. The model construction unit 68 may use any type of update formula (weight update formula) when updating the weight wl. Specific examples of the weight update formula include Adam, AdamW, AdamBelief, Adamax, AdaBound, Adagrad, AMSGRAD, AMSBound, Lars, RMSprop, Adadelta, Sgd, SgdW, Momentum, and Nesterov. The weight update formula is also called an optimization algorithm or an optimization method.

[0088] [Method of manufacturing ready-mix concrete] Next, an example of a method for producing ready-mixed concrete executed in the production system 1 will be described. The method for producing ready-mixed concrete includes a production process and a quality prediction process. The production process is a process for kneading materials including aggregate to produce ready-mixed concrete. The quality prediction process is a process for predicting the quality of at least a portion of the aggregate used in the production process. The quality prediction process may be executed during a period overlapping with at least a portion of the period during which the production process is executed.

[0089] (manufacturing process) The manufacturing process includes, for example, a transporting process, a weighing process, a feeding process, a mixing process, a discharging process, and a loading process. In the transporting process, various concrete materials are transported by a transport device 8 to a storage bottle 12, and the various materials are individually supplied to the storage bottle 12. In the weighing process, various materials are individually supplied from the storage bottle 12 to a weighing bottle 14, and the various materials are weighed in the weighing bottle 14. In the weighing process, when the measured amount of each material reaches a predetermined set amount, the material is discharged into a collecting hopper 16.

[0090] In the charging process, after all types of materials are collected in the collecting hopper 16, the materials in the collecting hopper 16 are charged (supplied) to the mixer 20. In the mixing process, multiple types of concrete materials are mixed in the mixer 20. In the discharging process, after mixing of the concrete materials is completed in the mixer 20, ready-mixed concrete is discharged from the mixer 20 to the loading hopper 30. In the loading process, the ready-mixed concrete discharged into the loading hopper 30 is loaded onto the transport vehicle C.

[0091] (Quality prediction process) The quality prediction process (aggregate quality prediction method) includes a model construction process in a training phase and a quality evaluation process in an evaluation phase. In the quality prediction process, the model construction process is executed before the quality evaluation process.

[0092] The model construction process includes, for example, a preparation process and a construction process. The model construction unit 68 of the calculation device 52 may execute the preparation process and the construction process. The preparation process is a process of preparing training data TD including training input information corresponding to the input information for evaluation and correct answer information of physical property values ​​associated with the training input information. The construction process is a process of constructing a prediction model M by machine learning based on the training data TD prepared in the preparation process.

[0093] The preparation process may include generating extended image data by performing a conversion process and a grayscale process in any order on the training captured image data P1 corresponding to the evaluation captured image data P1. The "extension" in the extended image data means extending the data set used in the training phase prepared initially (increasing the number of data sets). The conversion process in the preparation process may include randomly changing the brightness and contrast of the image. The conversion process in the preparation process may include at least one of inverting the image and rotating the image in addition to changing the brightness and contrast. That is, the conversion process may include either one of inverting the image and rotating the image, or both inverting the image and rotating the image. The input image data P2 in the training input information may include the extended image data, which is a grayscale image. Details of the conversion process for extending the training data TD will be described later.

[0094] The construction process may include preparing model evaluation data MED separately from the training data TD. The model evaluation data MED includes input information for model evaluation and correct answer information of physical property values ​​associated with the input information for model evaluation. The model evaluation data MED is composed of a plurality of data sets of the same type as the plurality of data sets included in the training data TD. The "model evaluation" in the model evaluation data MED means evaluating an intermediate model generated at the stage of constructing a prediction model when constructing the prediction model in the training phase, unlike evaluating quality in the evaluation phase. The construction process may include continuously updating the intermediate model using the training data TD so that the error with the correct answer information of the physical property values ​​is small, and determining one of the intermediate models generated during the update as the prediction model M using the model evaluation data MED. A specific example of a method for determining the prediction model M using the model evaluation data MED will be described later.

[0095] The quality evaluation process includes, for example, a first acquisition process, a preprocessing process, a second acquisition process, a prediction process, and an output process. The first acquisition process is a process of acquiring captured image data P1, which is color image data obtained by capturing an image of the aggregate to be evaluated. The captured image data acquisition unit 62 may execute the first acquisition process. The preprocessing process is a process of generating input image data P2 by grayscaling the captured image data P1 acquired in the first acquisition process. The preprocessing process may be executed by the preprocessing unit 64.

[0096] The second acquisition step is a step of acquiring input information including the input image data P2 generated in the pre-processing step. The input data acquisition unit 66 may execute the second acquisition step. The prediction step is a step of acquiring a physical property value (physical property value related to the grain size of the aggregate) corresponding to the input information acquired in the second acquisition step, using a prediction model M previously constructed by machine learning so as to output a physical property value of the aggregate according to the input of the input information. In the prediction step, the quality of the aggregate, such as the coarse aggregate ratio, is predicted based on the predicted value of the physical property value output from the prediction model M. The prediction unit 72 may execute the prediction step. The output step is a step of outputting the quality of the aggregate predicted in the prediction step to the input / output device 54. The output unit 74 may execute the output step.

[0097] An example of the model construction step and an example of the quality evaluation step will be described below with reference to Figs. 7 to 11. Note that the description will be given using an example in which the aggregate coarseness ratio is predicted as the quality of the aggregate, and the input information to the prediction model M includes only the input image data P2. In addition, the description will be given using an example in which the prediction model M outputs a predicted value of the mass fraction for each of the four granularity classes.

[0098] <Model building process> 7 is a flowchart showing an example of a series of processes executed in the model construction step. In the model construction step, a part of aggregate (e.g., coarse aggregate) used in manufacturing ready-mix concrete in the manufacturing apparatus 10 may be extracted, and the part of aggregate may be used as an aggregate for training.

[0099] In the model construction process, step S11 is executed first. In step S11, for example, a worker or the like acquires information necessary for preparing training data TD for machine learning and model evaluation data MED used in constructing a prediction model M. In one example, a worker or the like prepares various aggregates whose mass fractions remaining between successive sieves are known. A worker or the like may prepare various aggregates having grain sizes of levels 1 to 30 in Tables 1 to 3 described above. A worker or the like may prepare M pieces of training image data P1 in which at least one of grain size levels and deposition states is different from each other. M may be 10 to 100,000, or may be 100 to 10,000. A known mass fraction for each grain size division is associated with each of the M pieces of training image data P1 as correct answer information of the physical property value.

[0100] In imaging to obtain the M pieces of training captured image data P1, the training bone may be imaged by the imaging device 58 under substantially the same illuminance conditions. The illuminance IA, which is one of the environmental conditions when imaging the training bone to obtain the training captured image data P1, may be within the range exemplified in the above-mentioned "environmental conditions when acquiring captured image data P1".

[0101] Next, step S12 is executed. In step S12, for example, the environmental conditions when an image of the training aggregate is captured by a worker or the like to obtain the training captured image data P1 are stored in the image capturing condition storage unit 69. The worker or the like may input the illuminance IA, which is one of the environmental conditions, to the arithmetic unit 52 by operating an input device of the input / output device 54.

[0102] Next, step S13 is executed. In step S13, for example, the model construction unit 68 creates (generates) extended image data based on the M pieces of training captured image data P1 prepared in step S11 in order to extend the data used in the training phase according to a predetermined calculation procedure. Fig. 8 shows a schematic diagram of the process of creating extended image data from the training captured image data P1. In Fig. 8, the training captured image data P1 is indicated by "TP".

[0103] The model construction unit 68 generates image data TP1 to TPQ, which are Q pieces of image data, by randomly changing the brightness and contrast of the image data TP for each of the M pieces of image data TP (for each piece of image data TP). Q is an integer of 2 or more, and FIG. 8 illustrates a case where Q is 7. As a result of randomly changing the brightness and contrast, only the brightness or only the contrast may be changed in some of the image data TP1 to TPQ. As a result of randomly changing the brightness and contrast, the image data TP1 to TPQ may include an image in which both the brightness and the contrast are unchanged. The model construction unit 68 may generate new image data based on the image data TP so that one of the image data TP1 to TPQ includes the image data TP as it is without performing a process of changing it. Any one of the images of the image data TP1 to TPQ is expressed as "image data TPq" (q is an integer of 1 to Q).

[0104] The model construction unit 68 generates R pieces of image data, ie, image data TPq1 to TPqR, by performing at least one of image inversion and image rotation for each of the image data TP1 to TPQ (for each piece of image data TPq). R is an integer equal to or greater than 2, and FIG. 8 illustrates an example in which R is 4. FIG. 8 illustrates an example in which new image data is generated based on the image data TP3, but new image data is generated in the same manner for each of the image data TP1, TP2, TP4 to TP7. The model construction unit 68 may generate new image data based on the image data TPq so that the image data TPq is included as is in one of the image data TPq1 to TPqR.

[0105] In one example, the model construction unit 68 generates four pieces of image data from the image data TPq. In Fig. 8, the image data TP31 is the same as the image data TP3, and the image data TP32 is generated by inverting the image data TP3 horizontally. The image data TP33 is generated by inverting the image data TP3 vertically, and the image data TP34 is generated by inverting both the image data TP3 vertically and horizontally.

[0106] In the inversion for generating new image data based on one image data, the inversion may be performed about any axis, for example, left-right inversion, up-down inversion, or both left-right and up-down inversion. In the rotation for generating new image data based on one image data, the rotation may be performed at any angle within the range of 0° to 360°, and the rotation direction may be either clockwise or counterclockwise. At least one of the following may be performed: rotating the image clockwise by 90°, rotating the image counterclockwise by 90°, and rotating the image 180°. After rotating at an angle other than 90° and 180°, correction may be performed so that the outer edge of the image is aligned vertically and horizontally while maintaining the rotated state.

[0107] By generating new image data as described above, the model construction unit 68 generates (M×Q×R) pieces of image data based on the M pieces of image data TP. The model construction unit 68 maintains a state in which correct answer information of physical property values ​​is associated with the newly generated image data. The correct answer information associated between the newly generated image data and the image data on which the image data is based is the same. As described above, M data sets may be expanded to (M×Q×R) data sets.

[0108] Next, step S14 is executed. In step S14, for example, the model construction unit 68 performs a grayscale process on each of the (M×Q×R) pieces of image data prepared in step S13 according to a predetermined calculation procedure. The model construction unit 68 may perform grayscale conversion according to the above-mentioned formula (1) on each piece of image data prepared in step S13. As a result, (M×Q×R) pieces of extended image data, which are grayscaled image data, are obtained, and this extended image data after grayscale conversion is used as the input image data P2.

[0109] By executing steps S11, S13, and S14, (M×Q×R) pieces of training input image data P2 are prepared. The (M×Q×R) pieces of training input image data P2 are generated by performing grayscale processing on the training captured image data P1, although conversion processing has been performed to expand the data set used in the training phase. Each of the (M×Q×R) pieces of training input image data P2 is associated with a known mass fraction for each grain size division as correct answer information for the physical property value.

[0110] As described above, the model construction unit 68 may prepare (M×Q×R) data sets as data to be used in the training phase. Each of the (M×Q×R) data sets (each data set) includes training input image data P2 and correct answer information of the mass fraction remaining between each successive sieve, which is associated with the input image data P2. A part of the (M×Q×R) data sets may be used as the training data TD, and the remaining part may be used as the model evaluation data MED.

[0111] Next, steps S15 and S16 are executed. In step S15, for example, the model construction unit 68 constructs a prediction model M by machine learning based on the training data TD prepared up to the execution of step S14. In the process of constructing the prediction model M, the model construction unit 68 may determine the prediction model M to be used in the quality evaluation process by using the model evaluation data MED. The model construction unit 68 may construct a prediction model M that outputs a plurality of mass fraction values ​​for a plurality of grain size classes in response to the input of the input image data P2. In step S16, for example, the model holding unit 70 stores the prediction model M constructed in step S15. With the above, a series of processes executed in the model construction process is completed.

[0112] FIG. 9 is a flowchart showing an example of a series of processes executed when constructing a prediction model M in step S15. FIG. 10 is a diagram for explaining the series of processes shown in FIG. 9. In machine learning for constructing a prediction model M, the number of epochs (total number of learning times) is represented as "K", and the number of epochs is represented as i (i is an integer of 2 or more). The number of divisions of the training data TD in one epoch (each epoch) is represented as "L", and each piece of data obtained by dividing the training data TD into L pieces is called "batch data". Each batch data in the training data TD is represented as "batch data Bj" (j is an integer of 2 or more).

[0113] For ease of understanding of the description, it is assumed that the training data TD is 3000 data sets, and L, which represents the division number of the training data for each epoch, is 10. In this case, each batch data is 300 data sets, and the training data TD is divided into batch data B1 to B10. It is also assumed that K, which is the number of epochs meaning the number of times of learning (total number), is a multiple of 10. K may be 30 to 100,000, 100 to 10,000, or 1,000 to 5,000. L and K may be arbitrarily set by an operator or the like.

[0114] Steps S51 and S52 are executed in a state where L and K have already been set. In step S51, for example, the model construction unit 68 sets each of the variables i and j to 1. In step S52, for example, the model construction unit 68 randomly selects 300 pieces of data (data sets) from the training data TD to prepare batch data B1 to B10.

[0115] Next, steps S53 and S54 are executed. In step S53, for example, the model construction unit 68 uses the batch data Bj to update the weight wl as exemplified by the above formula (2). The model construction unit 68 uses the batch data Bj to determine the weight wl so that the above-mentioned loss value is minimized. In step S54, for example, the model construction unit 68 increments j.

[0116] Next, step S55 is executed. In step S55, for example, the model construction unit 68 judges whether j is greater than L (=10). If it is judged that j is equal to or less than L (step S55: NO), the process returns to step S53. In this case, the model construction unit 68 executes the series of processes from steps S53 to S55 again. The model construction unit 68 repeats the series of processes from steps S53 to S55 until j exceeds L. A part of FIG. 10 illustrates how ten weight updates are performed in the first epoch (i=1). In the first weight update, batch data B1 is used, in the second weight update, batch data B2 is used, and in the tenth weight update, batch data B10 is used. Although omitted, corresponding batch data Bj is also used in the third to ninth weight updates.

[0117] In step S55, if it is determined that j is greater than L (step S55: YES), the process proceeds to step S56. In step S56, for example, the model construction unit 68 stores the model Mi defined by the updated weight wl as an intermediate model of the prediction model M. The model Mi means the intermediate model constructed at the time when the i-th epoch (i epochs in total) is executed.

[0118] Next, steps S57 and S58 are executed. In step S57, for example, the model construction unit 68 increments i and sets j to 1. In step S58, for example, the model construction unit 68 determines whether i is greater than K. If it is determined that i is equal to or less than K (step S58: NO), the process returns to step S52. In step S52, which is executed again, the model construction unit 68 randomly reselects 300 pieces of data (data set) from the training data TD to newly prepare batch data B1 to B10. As a result, different batch data B1 to B10 are prepared between the i-th epoch and the (i-1)-th epoch.

[0119] The model construction unit 68 repeats a series of processes from steps S52 to S58 until i exceeds K. As a result, updating of the weight wl using the batch data B1 to B10 obtained by random selection is repeated for each epoch until training in the Kth epoch is completed, as shown in Fig. 10. Note that Fig. 10 illustrates only the case where the weight wl is updated using the batch data B10 in the second and subsequent epochs, and omits the state where the weight wl is updated using each of the batch data B1 to B9.

[0120] In step S58, if it is determined that i is greater than K (step S58: YES), the process proceeds to step S59. In step S59, for example, the model construction unit 68 calculates a loss value in the latest model Mi obtained at that time point using the model evaluation data MED for every arbitrarily set number of epochs (for example, every 1 to 10 epochs). FIG. 10 illustrates an example in which the loss value in the model Mi is evaluated using the model evaluation data MED for every 10 epochs. In this case, the model Mi obtained at the time when training (weight update) is completed at each of the 10th, 20th, . . . , Kth epochs is evaluated for the degree of loss value using the model evaluation data MED.

[0121] In the evaluation of the loss value using the model evaluation data MED, the loss value may be calculated for each data set included in the model evaluation data MED, and statistical information (e.g., arithmetic mean) of the loss values ​​may be used as an evaluation index. The model construction unit 68 may determine (select) the model Mi having the smallest loss value, which is the evaluation index, as the prediction model M from among the multiple models Mi that have been subject to evaluation using the model evaluation data MED.

[0122] <Quality evaluation process> 11 is a flowchart showing an example of a series of processes executed in the quality evaluation process. This quality evaluation process is performed, for example, during a period overlapping with at least a part of a period during which the above-mentioned manufacturing process is executed by the manufacturing apparatus 10. In the quality evaluation process, a part of aggregate used in manufacturing ready-mix concrete by the manufacturing apparatus 10 may be extracted, and the quality of the aggregate may be evaluated as an evaluation target.

[0123] In the quality evaluation process, step S21 is executed first. In step S21, the imaging data acquisition unit 62 may display the range of the illuminance IB when the captured image data P1 is obtained in the quality evaluation process on the monitor of the input / output device 54 based on the illuminance IA stored in the imaging condition storage unit 69. The range of the illuminance IB is the same as the range exemplified in the above-mentioned "environmental conditions when acquiring the captured image data P1". Note that, if it is guaranteed that the illuminance IB when the captured image data P1 is obtained in the quality evaluation process is within the range, step S21 may be omitted.

[0124] Next, step S22 is executed. In step S22, for example, the imaging data acquisition unit 62 acquires the captured image data P1 for evaluation obtained by capturing an image of the aggregate for evaluation. The imaging data acquisition unit 62 may acquire the captured image data P1 from the imaging device 58. The imaging device 58 may capture the image of the aggregate to be evaluated while a part of the work is being performed by a worker or the like. In this case, the worker or the like may check whether the illuminance IB is within the range displayed on the monitor in step S21. Step S22 may be executed at a predetermined evaluation timing. The evaluation timing may be set to a certain time in a day, or may be a timing when an instruction is received from a worker or the like. The conditions other than the illuminance among the conditions when the imaging device 58 captures the image may be set to approximately match the conditions in the model construction process within a settable range.

[0125] Next, step S23 is executed. In step S23, for example, the pre-processing unit 64 performs a grayscale conversion process on the captured image data P1 for evaluation acquired in step S22, thereby generating input image data P2. The pre-processing unit 64 performs grayscale conversion using the same calculation procedure (same image processing procedure) as the grayscale conversion process in the model construction step. The pre-processing unit 64 may perform grayscale conversion on the captured image data P1 for evaluation using the above-mentioned formula (1).

[0126] Next, steps S24 and S25 are executed. In step S24, for example, the prediction unit 72 inputs the input image data P2 for evaluation generated in step S23 into the prediction model M held by the model holding unit 70, and obtains a predicted value of the mass fraction remaining between each successive sieve output from the prediction model M. In step S25, for example, the prediction unit 72 calculates a predicted value of the coarse particle fraction by converting the predicted value of the mass fraction remaining between each successive sieve obtained in step S24 into a coarse particle fraction.

[0127] Next, step S26 is executed. In step S26, for example, the output unit 74 outputs the predicted value of the coarse aggregate ratio calculated in step S25 to the monitor of the input / output device 54. In addition to the predicted value of the coarse aggregate ratio, the output unit 74 may output the predicted value of the mass fraction remaining between each successive sieve obtained in step S24 to the monitor of the input / output device 54. By executing step S26, a worker or the like can check the predicted result of the coarse aggregate ratio. If the coarse aggregate ratio is out of the control range, the worker or the like who has checked the predicted result of the coarse aggregate ratio may take measures to maintain the quality of the ready-mixed concrete (for example, adjusting the granularity of the aggregate, correcting (modifying) the mix, temporarily suspending production, investigating the cause, etc.).

[0128] This completes a series of processes for predicting the coarse aggregate ratio once. After step S26 is executed, the series of processes from steps S21 to S26 may be repeated. In the above example, some aggregate is extracted and the quality is evaluated (sampling inspection), but all aggregate used in the production of ready-mix concrete may be inspected.

[0129] [Verification of prediction accuracy] Next, the results of verifying the prediction accuracy of the prediction model M constructed by machine learning using a convolutional neural network will be described with reference to Figures 12(a) and 12(b). In this verification, a data set in which the correct answer value is known was used to compare the prediction result by the prediction model M with the correct answer value.

[0130] (Verification example 1A: air-dry condition, illuminance IB=3220lx) First, 10 levels of training aggregates with different coarseness ratios were prepared according to the conditions in Table 1 above. For each level, 4 kg of training aggregate was placed on a flat tray (a bat with a rim) and 10 pieces of image data were obtained by taking 10 images indoors so that the state of the aggregate on the tray was different from each other. As the training aggregate, air-dried aggregate, which means a dry state in a natural environment, was used. The illuminance IA, which is one of the environmental conditions during imaging, was set to 3420 lx. By taking 10 images, 100 pieces of image data were obtained according to the conditions in Table 1, and 100 data sets were obtained in which the image data was associated with the correct value of the mass fraction remaining between each successive sieve.

[0131] Next, 10 levels of aggregate (air-dried state) with different coarse grain ratios were prepared according to the conditions in Table 2 above. 100 pieces of image data were obtained according to the conditions in Table 2 using a method similar to the preparation according to the conditions in Table 1, and 100 data sets were obtained in which the image data and the correct answer value of the mass fraction remaining between each successive sieve were associated with each other. Furthermore, 10 levels of aggregate (air-dried state) with different coarse grain ratios were prepared according to the conditions in Table 3 above. 100 pieces of image data were obtained according to the conditions in Table 3 using a method similar to the preparation according to the conditions in Table 1, and 100 data sets were obtained in which the image data and the correct answer value of the mass fraction remaining between each successive sieve were associated with each other.

[0132] After obtaining 300 data sets according to the conditions in Tables 1, 2, and 3, new image data was created to expand the data set used in the training phase by performing the process of step S13 described above. Specifically, when the brightness and contrast were randomly changed, 11 patterns of new image data were generated from one original image data. When the image was inverted or rotated to change, 4 patterns of new image data were generated from one original image data. As a result, 300 image data were expanded to (300 x 11 x 4 = 13,200) image data. When the brightness and contrast were randomly changed, random numbers were generated in computer processing, and the change conditions were varied. When the image was inverted or rotated, new image data was generated by performing four conversion patterns: no inversion, up-down inversion, left-right inversion, and up-down and left-right inversion.

[0133] Next, the 13,200 pieces of image data were each subjected to a grayscale process in step S14 to generate 13,200 pieces of input image data P2. In the grayscale process, grayscale conversion was performed for each pixel using the above-mentioned formula (1). As a result, 13,200 data sets were obtained in which the input image data P2 was associated with the correct information on the mass fraction.

[0134] Of the 13,200 datasets, 9,240 datasets were used as training data TD, and 2,640 datasets were used as model evaluation data MED. The remaining 1,320 datasets were not used in the comparative verification according to the present disclosure. A series of processes in steps S51 to S59 in step S15 were performed to construct a prediction model M from the training data TD and the model evaluation data MED. K, which represents the number of epochs (total number of learning times), was set to 10,000, and the model Mi, which is an intermediate model being updated, was evaluated every 10 epochs using the model evaluation data MED.

[0135] In addition to the training dataset, a dataset (hereinafter referred to as "test data") was prepared to compare and verify the predicted values ​​by the prediction model M with the correct values. In preparing the test data, 10 levels of aggregates with different coarse aggregate ratios were prepared according to the conditions shown in Table 4 below.

[0136] [Table 4]

[0137] Under the conditions shown in Table 4, the combination of mass fractions in a plurality of granularity classes is set to be different from the combinations under the conditions in Tables 1, 2, and 3. After preparing aggregates under the conditions in Table 4 in an air-dried state, 100 pieces of image data (imaged image data P1) under the conditions in Table 4 were obtained in the same manner as the preparation (imaging) under the conditions in Table 1. However, the illuminance IB, which is one of the environmental conditions during imaging, was set to 3220 lx. The illuminances IA and IB were measured by placing a digital illuminance meter (product name: CANA-0010, manufactured by Tokyo Koden Co., Ltd.) on the imaging platform on which the tray containing the aggregates to be imaged was placed.

[0138] By carrying out a grayscale process on the captured image data P1 in a manner similar to the preparation according to the conditions in Table 1, 100 data sets were obtained as test data, each of which corresponded to the image data and the correct value of the mass fraction remaining between each successive sieve. This test data was data based on aggregates having a different grading level from the grading level of the aggregates used in the preparation of the training data TD. In the 100 data sets included in the test data, in addition to the correct value of the mass fraction remaining between each successive sieve, the correct value of the coarse aggregate ratio was also stored in correspondence with the image data for comparative verification.

[0139] Using the test data, comparative verification was performed for each data set included in the test data by the following procedure. First, input image data P2 (input information) included in the data set was input into prediction model M to obtain a predicted value of the mass fraction (mass fraction of each of a plurality of particle size divisions) remaining between successive sieves. Then, the predicted value of the mass fraction remaining between successive sieves was converted to a coarse particle fraction to obtain a predicted value of the coarse particle fraction using prediction model M. After that, the error between the predicted value of the coarse particle fraction by prediction model M and the correct value of the coarse particle fraction in the data set was calculated.

[0140] In the comparative verification, the error between the predicted value of the coarse grain ratio and the correct value was evaluated using the following two indicators. - Coarse-grained accuracy rate (%): The ratio of the number of correct datasets to the total number of datasets, when the correct answer is defined as a dataset with an error of within ±0.10. ·RMSE: Root Mean Square Error.

[0141] (Verification example 1B: surface dry, illuminance IB=3220lx) The same prediction model M as in Verification Example 1A was used, but the method of preparing the test data was different from that in Verification Example 1A. In Verification Example 1B, instead of the air-dried aggregate, test data was prepared using aggregate that simply mimics surface-dried aggregate, which means that the internal voids are filled with water and the surface is dry. Specifically, the surface-dried aggregate was prepared by adding 0.25 wt% (10 ml) of water to 4 kg of air-dried aggregate and stirring the aggregate until the moisture covers the entire aggregate surface. In the explanation of this comparative verification, for convenience of explanation, the state that simply mimics the surface-dried state is simply referred to as the "surface-dried state". In other words, the surface-dried aggregate means aggregate that simply mimics the surface-dried aggregate. The test data was prepared in the same manner as in Verification Example 1A, except that the surface-dried aggregate was used.

[0142] (Verification example 2A: air-dry condition, illuminance IB=3600lx) The same prediction model M as in Verification Example 1A was used, but the method of preparing the test data was different from that in Verification Example 1A. In Verification Example 2A, the test data was prepared with the illuminance IB set to 3600 lx. The test data was prepared in the same manner as in Verification Example 1A, except that the illuminance IB was set to 3600 lx.

[0143] (Verification example 2B: surface dry, illuminance IB=3600lx) The same prediction model M as in Verification Example 1A was used, but the method of preparing the test data was different from that in Verification Example 2A. In Verification Example 2B, test data was prepared using surface-dried aggregate instead of air-dried aggregate. The test data was prepared in the same manner as in Verification Example 2A, except that surface-dried aggregate was used.

[0144] (Verification example 3A: air-dry condition, illuminance IB=4190lx) The same prediction model M as in Verification Example 1A was used, but the method of preparing the test data was different from that in Verification Example 1A. In Verification Example 3A, the test data was prepared with the illuminance IB set to 4190 lx. The test data was prepared in the same manner as in Verification Example 1A, except that the illuminance IB was set to 4190 lx.

[0145] (Verification example 3B: surface dry, illuminance IB=4190lx) The same prediction model M as in Verification Example 1A was used, but the method of preparing the test data was different from that in Verification Example 3A. In Verification Example 3B, test data was prepared using surface-dried aggregate instead of air-dried aggregate. The test data was prepared in the same manner as in Verification Example 3A, except that surface-dried aggregate was used.

[0146] (Verification example 4A: air-dry condition, illuminance IB=4810lx) The same prediction model M as in Verification Example 1A was used, but the method of preparing the test data was different from that in Verification Example 1A. In Verification Example 4A, the test data was prepared with the illuminance IB set to 4810 lx. The test data was prepared in the same manner as in Verification Example 1A, except that the illuminance IB was set to 4810 lx.

[0147] (Verification example 4B: surface dry, illuminance IB=4810lx) The same prediction model M as in Verification Example 1A was used, but the method of preparing the test data was different from that in Verification Example 4A. In Verification Example 4B, test data was prepared using surface-dried aggregate instead of air-dried aggregate. The test data was prepared in the same manner as in Verification Example 4A, except that surface-dried aggregate was used.

[0148] (Reference example A: air-dry condition, illuminance IB=610lx) The same prediction model M as in Verification Example 1A was used, but the method of preparing the test data was different from that in Verification Example 1A. In Reference Example A, the test data was prepared with the illuminance IB set to 610 lx. The test data was prepared in the same manner as in Verification Example 1A, except that the illuminance IB was set to 610 lx.

[0149] (Reference example B: surface dry, illuminance IB=610lx) The same prediction model M as in Verification Example 1A was used, but the method of preparing the test data was different from that in Reference Example A. In Reference Example B, test data was prepared using surface-dried aggregate instead of air-dried aggregate. The test data was prepared in the same manner as Reference Example A, except that surface-dried aggregate was used.

[0150] (Difference between air-dried and surface-dried) The color of the aggregate in the image was different between the air-dry state and the surface-dry state. Specifically, the aggregate in the image was whitish in the air-dry state, whereas the aggregate in the image was blackish in the surface-dry state. That is, in Reference Example A and Verification Examples 1A to 4A, the color of the aggregate in the image was the same between the training phase and the evaluation phase, whereas in Reference Example B and Verification Examples 1B to 4B, the color of the aggregate in the image was different between the training phase and the evaluation phase.

[0151] (Verification results) Tables 5 and 6 below show the verification results regarding errors in each verification example and each reference example.

[0152] [Table 5]

[0153] [Table 6]

[0154] In Fig. 12(a), the evaluation results regarding the accuracy rate are plotted on a graph, and in Fig. 12(b), the evaluation results regarding the RMSE are plotted on a graph. In the graphs of Fig. 12(a) and 12(b), the circles and solid lines correspond to the results shown in Table 5, and the triangles and dashed lines correspond to the results shown in Table 6.

[0155] From the results shown in Tables 5 and 6 and FIG. 12(a), it can be seen that the accuracy rate is higher in the verification examples 1A to 4A and 1B to 4B in which the illuminance IA is 3220lx, 3600lx, 4190lx, and 4810lx than in the reference examples A and B in which the illuminance IB is 610lx while the illuminance IA is 3420lx. Also, from the results shown in Tables 5 and 6 and FIG. 12(b), it can be seen that the RMSE is smaller in the verification examples 1A to 4A and 1B to 4B in which the illuminance IA is 3220lx, 3600lx, 4190lx, and 4810lx than in the reference examples A and B in which the illuminance IB is 610lx while the illuminance IA is 3420lx. Also, it can be seen that the error index can be significantly reduced in the verification examples 1B to 4B in terms of the surface dry state compared to the reference example B.

[0156] [Variations] The series of processes shown in each of Figures 7, 9, and 11 are examples and can be modified as appropriate. In the series of processes, one step and the next step may be executed in parallel, or some steps may be executed in an order different from the above-mentioned example. Instead of at least some of the steps of the series of processes, or in addition to the series of processes, steps with content different from the above-mentioned example may be executed. In step S15, the model evaluation data MED may not be used, and the model at the time when the training (weight update) of the set number of epochs is completed may be determined as the prediction model M.

[0157] In the example shown in FIG. 7, after step S11, the grayscale process of step S14 may be performed, and then the generation of the extended image data of step S13 may be performed. The training input image data P2 may be generated by performing the grayscale process and the conversion process for extending the training data on the training captured image data P1 in this order. In the conversion process for extending the training data, conversion including at least one of image inversion and rotation may not be performed. The extension of the training data with the generation of new image data may be performed by a method different from the extension method exemplified in step S13. In the preparation of the training data, the extension of the training data by the method of step S13 or another method may not be performed.

[0158] In step S15, the prediction model M may be constructed to output the coarse grain ratio in response to the input of the input image data P2. In this case, in the evaluation phase, instead of steps S24 and S25, the coarse grain ratio output when the input image data P2 for evaluation is input to the prediction model M is acquired as a prediction result. The prediction model M may be constructed to output two or more physical property values ​​related to the grain size of the aggregate. The prediction model M may be constructed to output the mass fraction and the coarse grain ratio remaining between each successive sieve in response to input information including the input image data P2. In this case, the prediction unit 72 may calculate the coarse grain ratio based on the predicted value of the mass fraction from the prediction model M and the predicted value of the coarse grain ratio output from the prediction model M as a prediction result of the quality. The input information, which is data input to the prediction model M, may include any information that may be correlated with the quality of the aggregate in addition to the input image data P2.

[0159] In addition to the quality prediction device 50 predicting the quality of the aggregate, the manufacturing system 1 may actually measure the physical properties of the aggregate periodically (e.g., several times a day). In this case, the prediction model M may be updated based on the actual measured values ​​of the physical properties of the aggregate and input information (e.g., input image data P2) at the time when the actual measured values ​​were obtained. In the prediction step, the physical properties of the aggregate may be predicted by the updated prediction model M. Note that even when the updated prediction model M is used, the step of predicting the physical properties of the aggregate is still performed based on the prediction model M and the input information acquired in the second acquisition step.

[0160] The quality prediction device 50 may be used in a place other than the manufacturing system 1 (other than a process inspection in the manufacturing process of ready-mix concrete). The quality prediction device 50 may be used in an acceptance inspection when a factory that manufactures ready-mix concrete accepts raw materials. In one example, the quality prediction device 50 is used for quality prediction (acceptance inspection) performed on a transport truck or a transport ship.

[0161] The imaging device 58 may autonomously capture the image of the aggregate for evaluation while checking the illuminance IB of the device including the quality prediction device 50. In this case, the device including the quality prediction device 50 may determine whether or not to execute prediction using the prediction model M based on the illuminance IB, and execute prediction using the prediction model M if the illuminance IB is appropriate. The imaging device 58 may obtain color captured image data P1 by imaging, and then perform a process of grayscaling the captured image data P1 to generate grayscale image data. In this way, a part of the above-mentioned functional blocks may be configured by the imaging device 58 instead of the arithmetic device 52. In one example of the various examples described above, at least a part of the matters described in the other examples may be combined.

[0162] [Summary of this disclosure] The aggregate quality prediction method described above includes a first acquisition step of acquiring color image data (P1) obtained by imaging the aggregate to be evaluated, a pre-processing step of generating input image data (P2) by grayscaling the captured image data (P1), a second acquisition step of acquiring input information including the input image data (P2), a prediction step of acquiring physical property values ​​corresponding to the input information acquired in the second acquisition step using a prediction model (M) previously constructed to output physical property values ​​of the aggregate according to the input of the input information, and a construction step of constructing the prediction model (M) by machine learning based on training data (TD) including training input information corresponding to the above input information and ground truth information of the physical property values ​​associated with the training input information. When the illuminance among the environmental conditions when imaging the training bone material to prepare the training input information is IA [lx], and the illuminance among the environmental conditions when imaging the bone material to be evaluated to obtain the captured image data (P1) is IB [lx], IB [lx] is greater than or equal to 0.2 times IA [lx] and less than or equal to 5 times IA [lx].

[0163] A technology for predicting the quality of aggregates from image data of the aggregates using a prediction model constructed by machine learning has been studied. It was found that the prediction accuracy may decrease if the imaging conditions or the granularity level representing the size of the aggregates differ between the training phase and the evaluation phase. As a result of various studies, the inventors came up with the idea of ​​performing a preprocessing of grayscaling on the image data to be input to the prediction model. As a result of further verification, the inventors confirmed a new problem that the prediction accuracy may decrease even if the preprocessing of grayscaling is performed when the illuminance increases between the training phase and the evaluation phase. In contrast, in the above quality prediction method, the illuminance IB in the evaluation phase is 0.2 times or more and 5 times or less than the illuminance IA in the training phase. Therefore, since the illuminance (IB) when obtaining image data whose physical properties are unknown in the evaluation phase is close to the illuminance (IA) when preparing the training data (TD) in the training phase, it is possible to suppress the prediction accuracy of the prediction model (M) from decreasing due to the difference in illuminance, assuming that grayscaling is performed as a preprocessing. Therefore, the above quality prediction method is useful for improving prediction accuracy when predicting quality using a prediction model constructed by machine learning.

[0164] In the above-described aggregate quality prediction method, the absolute value of the difference between IA [lx], which represents the illuminance among the environmental conditions when the training aggregate is imaged to prepare the training input information, and IB [lx], which represents the illuminance among the environmental conditions when the evaluation target aggregate is imaged to obtain the image data (P1), may be 2500 lx or less. It has been confirmed that the decrease in prediction accuracy is suppressed when the difference between the illuminance IB and the illuminance IA is 2500 lx or less, compared to when the difference between the illuminance IB and the illuminance IA is greater than 2500. Therefore, the above quality prediction method is useful for improving the accuracy of prediction using a prediction model for predicting the quality of aggregate from image data.

[0165] In the above-described aggregate quality prediction method, IB [lx], which represents the illuminance among the environmental conditions when the aggregate to be evaluated is imaged to obtain the image data (P1), may be 2000 lx or more. In this case, the illuminance IB is close to the range of general illuminance indoors on a sunny day. Therefore, it is easy to adjust the environmental conditions at the time of imaging to a desired range in the evaluation phase.

[0166] In the above-described aggregate quality prediction method, in the pre-processing step, when the pixel values ​​of the red, green, and blue color components of each pixel in the color image data are expressed as R, G, and B, and the pixel value of each pixel in the grayscale image data is expressed as Y, grayscale conversion may be performed using the above formula (1). In this case, the grayscale conversion is performed taking into account the sensitivity of the human eye. This increases the possibility that the prediction model can capture and predict the differences perceived by humans as features from the grayscale image data.

[0167] The aggregate quality prediction method described above may further include a preparation step of preparing training data (TD). The preparation step may include generating extended image data by performing a conversion process and a grayscale process in any order on the training captured image data (P1) corresponding to the captured image data (P1). The conversion process may include randomly changing the brightness and contrast of the image. The input image data (P2) in the training input information may include extended image data. The brightness and contrast of the captured image data may change depending on the surrounding environment at the time of capturing. In this case, even if the correct answer value of the physical property value is the same, and even if the difference in illuminance between the training phase and the evaluation phase is reduced, the trained prediction model may recognize the image data as having different characteristics, and the prediction accuracy may decrease. Therefore, it is possible to prepare various training image data after adjusting the environment at the time of capturing and perform machine learning, but the preparation of the training data becomes complicated. In contrast, in the above quality prediction method, the brightness and contrast of the image are randomly changed to generate new image data, and the training data is expanded. Therefore, it is useful for improving prediction accuracy while avoiding the complication of preparing training data for machine learning.

[0168] In the above-described aggregate quality prediction method, the conversion process may further include at least one of inverting the image and rotating the image. Even if the correct answer value of the physical property value is the same, and even if the difference in illuminance between the training phase and the evaluation phase is reduced, the trained prediction model may recognize the image data as having different characteristics due to differences in the deposition state of the aggregate to be imaged, and the prediction accuracy may decrease. Therefore, it is possible to prepare a large number of training image data with different deposition states and perform machine learning, but the preparation of the training data becomes complicated. In contrast, in the above-described quality prediction method, at least one of the conversions of inversion and rotation of the image is performed, new image data is generated, and the training data is expanded. Therefore, it is further useful for improving the prediction accuracy while avoiding the complication of the preparation of training data to be used in machine learning.

[0169] In the above-described aggregate quality prediction method, the construction step may include preparing model evaluation data (MED) including input information for model evaluation corresponding to the input information and correct information of physical properties associated with the input information for model evaluation, separately from the training data (TD); continuously updating the intermediate model (Mi) using the training data (TD) so that the error with the correct information of the physical properties is reduced; and using the model evaluation data (MED) to determine one of the intermediate models (Mi) generated during the update as the prediction model (M). During training in machine learning, adjustment of parameters such as weights is continued, and the intermediate model is continuously updated. In such machine learning, it is also possible to use a model generated at the time when the training of a set number of learning times (epoch number) is completed as a prediction model to be used in the evaluation phase. However, the prediction accuracy of the model at the time when the training is completed is not necessarily higher than the prediction accuracy of the model being updated before that. In contrast, in the above quality prediction method, the intermediate models (Mi) generated during the update are evaluated using model evaluation data (MED) separate from the training data (TD), and one of the intermediate models (Mi) is determined as the prediction model (M). As a result, even if an intermediate model generated during the update is preferable in terms of prediction accuracy, the intermediate model can be used as the prediction model (M) for prediction in the evaluation phase. Therefore, it is useful for improving the prediction accuracy of a prediction model for predicting aggregate quality.

[0170] The above-described aggregate quality prediction program is a program for causing a computer to execute the above-described quality prediction methods [1] to [7]. This quality prediction program, like the above-described aggregate quality prediction method, is useful for improving prediction accuracy when predicting quality using a prediction model constructed by machine learning.

[0171] The quality prediction device (50) described above includes a first acquisition unit (62) that acquires color captured image data (P1) obtained by capturing an image of the aggregate to be evaluated, a pre-processing unit (64) that generates input image data (P2) by grayscaling the captured image data (P1), a second acquisition unit (66) that acquires input information including the input image data (P2), a prediction unit (72) that acquires physical property values ​​corresponding to the input information acquired by the second acquisition unit (66) using a prediction model (M) that has been constructed in advance to output physical property values ​​of the aggregate in response to input of the input information, and a construction unit (68) that constructs the prediction model (M) by machine learning based on training data (TD) that includes training input information corresponding to the input information and ground truth information of the physical property values ​​associated with the training input information. When the illuminance among the environmental conditions when an image of a training aggregate is taken to prepare input information for training is IA [lx], and the illuminance among the environmental conditions when an image of an aggregate to be evaluated is taken to obtain captured image data (P1) is IB [lx], IB [lx] is 0.2 times or more than IA [lx] and 5 times or less than IA [lx]. This quality prediction device (50) is useful for improving prediction accuracy when predicting quality using a prediction model constructed by machine learning, similar to the above-mentioned aggregate quality prediction method.

[0172] The manufacturing method of ready-mixed concrete described above includes a manufacturing process of manufacturing ready-mixed concrete by mixing materials including aggregate, and a quality prediction process of predicting the quality of at least a part of the aggregate used in the manufacturing process. The quality prediction process includes a first acquisition process of acquiring color image data (P1) obtained by imaging the aggregate to be evaluated, a pre-processing process of generating input image data (P2) by grayscaling the image data (P1), a second acquisition process of acquiring input information including the input image data (P2), a prediction process of acquiring physical property values ​​corresponding to the input information acquired in the second acquisition process using a prediction model (M) previously constructed to output physical property values ​​of the aggregate according to the input of the input information, and a construction process of constructing the prediction model (M) by machine learning based on training data (TD) including training input information corresponding to the input information and correct answer information of the physical property values ​​associated with the training input information. When the illuminance among the environmental conditions when an image of a training aggregate is taken to prepare input information for training is IA [lx], and the illuminance among the environmental conditions when an image of an aggregate to be evaluated is taken to obtain captured image data (P1) is IB [lx], IB [lx] is 0.2 times or more than IA [lx] and 5 times or less than IA [lx]. This manufacturing method for ready-mixed concrete is useful for improving prediction accuracy when predicting quality using a prediction model constructed by machine learning, similar to the above-mentioned aggregate quality prediction method.

[0173] The manufacturing system (1) described above includes a manufacturing apparatus (10) that manufactures ready-mixed concrete by mixing materials including aggregate, and a quality prediction device (50) that predicts the quality of at least a part of the aggregate used by the manufacturing apparatus (10). The quality prediction device (50) includes a first acquisition unit (62) that acquires color captured image data (P1) obtained by capturing an image of an aggregate to be evaluated, a pre-processing unit (64) that generates input image data (P2) by grayscaling the captured image data (P1), a second acquisition unit (66) that acquires input information including the input image data (P2), a prediction unit (72) that acquires physical property values ​​corresponding to the input information acquired by the second acquisition unit (66) using a prediction model (M) that has been previously constructed to output physical property values ​​of the aggregate in response to input of the input information, and a construction unit (68) that constructs the prediction model (M) by machine learning based on training data (TD) that includes training input information corresponding to the input information and correct answer information of the physical property values ​​associated with the training input information. When the illuminance among the environmental conditions when an image of a training aggregate is taken to prepare input information for training is IA [lx], and the illuminance among the environmental conditions when an image of an aggregate to be evaluated is taken to obtain captured image data (P1) is IB [lx], IB [lx] is 0.2 times or more than IA [lx] and 5 times or less than IA [lx]. This manufacturing system (1) is useful for improving prediction accuracy when predicting quality using a prediction model constructed by machine learning, similar to the above-mentioned aggregate quality prediction method. [Explanation of symbols]

[0174] 1... manufacturing system, 10... manufacturing equipment, 50... quality prediction device, 62... imaging data acquisition unit, 64... pre-processing unit, 66... ​​input data acquisition unit, 68... model construction unit, 72... prediction unit, P1... imaging image data, P2... input image data, TD... training data, MED... model evaluation data, M... prediction model, Mi... model (intermediate model).

Claims

1. A first acquisition step of acquiring color image data obtained by imaging an aggregate to be evaluated; a pre-processing step of generating input image data by grayscaling the captured image data; a second acquisition step of acquiring input information including at least the input image data; a prediction process for acquiring physical property values ​​according to the input information acquired in the second acquisition process by utilizing a prediction model previously constructed to output physical property values ​​related to the granularity of aggregates in response to input of the input information; A construction process of constructing the prediction model by machine learning based on training data including training input information corresponding to the input information and correct answer information of the physical property value associated with the training input information, When the illuminance when imaging the training bone material to prepare the training input information is IA [lx], and the illuminance when imaging the evaluation target bone material to obtain the image data is IB [lx], IB [lx] is 0.2 times or more and 5 times or less of IA [lx], The absolute value of the difference between IA [lx] and IB [lx] is 2500 lx or less. Methods for predicting aggregate quality.

2. A first acquisition step of acquiring color image data obtained by imaging an aggregate to be evaluated; a pre-processing step of generating input image data by grayscaling the captured image data; a second acquisition step of acquiring input information including at least the input image data; a prediction process for acquiring physical property values ​​according to the input information acquired in the second acquisition process by utilizing a prediction model previously constructed to output physical property values ​​related to the granularity of aggregates in response to input of the input information; A construction process of constructing the prediction model by machine learning based on training data including training input information corresponding to the input information and correct answer information of the physical property value associated with the training input information, When the illuminance when imaging the training bone material to prepare the training input information is IA [lx], and the illuminance when imaging the evaluation target bone material to obtain the image data is IB [lx], IB [lx] is 0.2 times or more and 5 times or less of IA [lx], IB [lx] is 2000 lx or more. Methods for predicting aggregate quality.

3. In the pre-processing step, when the pixel values ​​of the red, green, and blue color components of each pixel in the color image data are expressed as R, G, and B, and the pixel value of each pixel in the grayscale image data is expressed as Y, grayscale conversion is performed using the following formula (1): Y=0.299×R+0.587×G+0.114×B...(1) The method for predicting aggregate quality according to claim 1 or 2.

4. preparing the training data; The preparation step includes generating extended image data by performing a conversion process and a grayscale process on training captured image data corresponding to the captured image data in any order; The transformation process includes randomly modifying the brightness and contrast of the image; The input image data in the training input information includes the extended image data. The method for predicting aggregate quality according to claim 1 or 2.

5. The transformation process further includes at least one of flipping the image and rotating the image. The method for predicting aggregate quality according to claim 4.

6. The construction step includes: preparing model evaluation data including input information for model evaluation corresponding to the input information and correct answer information of the physical property value associated with the input information for model evaluation separately from the training data; continuously updating the intermediate model using the training data so that an error between the intermediate model and the correct answer information of the physical property value is reduced; and determining, using the model evaluation data, one of the intermediate models generated during the update as the predictive model. The method for predicting aggregate quality according to claim 1 or 2.

7. A first acquisition step of acquiring color image data obtained by imaging an aggregate to be evaluated; a pre-processing step of generating input image data by grayscaling the captured image data; a second acquisition step of acquiring input information including at least the input image data; a prediction process for acquiring physical property values ​​according to the input information acquired in the second acquisition process by utilizing a prediction model previously constructed to output physical property values ​​related to the granularity of aggregates in response to input of the input information; a preparation step of preparing training data including training input information corresponding to the input information and correct answer information of the physical property value associated with the training input information; A construction step of constructing the prediction model by machine learning based on the training data; Including, When an illuminance when imaging the training bone material to prepare the training input information is IA [lx], and an illuminance when imaging the evaluation target bone material to obtain the captured image data is IB [lx], IB [lx] is 0.2 times or more of IA [lx] and 5 times or less of IA [lx], The preparation step includes generating extended image data by performing a conversion process and a grayscale process on training captured image data corresponding to the captured image data in any order; The transformation process includes randomly modifying the brightness and contrast of the image; The input image data in the training input information includes the extended image data. Methods for predicting aggregate quality.

8. A quality prediction program for causing a computer to execute the aggregate quality prediction method according to any one of claims 1 to 7.

9. A manufacturing process for producing ready-mix concrete by mixing materials including aggregate; a quality prediction step of predicting the quality of at least a part of the aggregate used in the production step by the aggregate quality prediction method according to any one of claims 1, 2 and 7; Including, A method for producing ready-mix concrete.

10. A first acquisition unit that acquires color image data obtained by imaging the aggregate to be evaluated; a pre-processing unit that generates input image data by gray-scaling the captured image data; A second acquisition unit that acquires input information including at least the input image data; a prediction unit that acquires the physical property values ​​according to the input information acquired by the second acquisition unit by using a prediction model that is constructed in advance so as to output physical property values ​​related to the granularity of the aggregate in response to the input of the input information; A construction unit that constructs the prediction model by machine learning based on training data including training input information corresponding to the input information and correct answer information of the physical property value associated with the training input information, When the illuminance when imaging the training bone material to prepare the training input information is IA [lx], and the illuminance when imaging the evaluation target bone material to obtain the image data is IB [lx], IB [lx] is 0.2 times or more and 5 times or less of IA [lx], The absolute value of the difference between IA [lx] and IB [lx] is 2500 lx or less. Aggregate quality prediction device.

11. A first acquisition unit that acquires color image data obtained by imaging an aggregate to be evaluated; a pre-processing unit that generates input image data by gray-scaling the captured image data; A second acquisition unit that acquires input information including at least the input image data; a prediction unit that acquires the physical property values ​​according to the input information acquired by the second acquisition unit by using a prediction model that is constructed in advance so as to output physical property values ​​related to the granularity of the aggregate in response to the input of the input information; A construction unit that constructs the prediction model by machine learning based on training data including training input information corresponding to the input information and correct answer information of the physical property value associated with the training input information, When the illuminance when imaging the training bone material to prepare the training input information is IA [lx], and the illuminance when imaging the evaluation target bone material to obtain the image data is IB [lx], IB [lx] is 0.2 times or more and 5 times or less of IA [lx], IB [lx] is 2000 lx or more. Aggregate quality prediction device.

12. A first acquisition unit that acquires color image data obtained by imaging an aggregate to be evaluated; a pre-processing unit that generates input image data by gray-scaling the captured image data; A second acquisition unit that acquires input information including at least the input image data; a prediction unit that acquires the physical property values ​​according to the input information acquired by the second acquisition unit by using a prediction model that is constructed in advance so as to output physical property values ​​related to the granularity of the aggregate in response to the input of the input information; A construction unit that prepares training data including training input information corresponding to the input information and correct answer information of the physical property value associated with the training input information, and constructs the prediction model by machine learning based on the training data, When an illuminance when imaging the training bone material to prepare the training input information is IA [lx], and an illuminance when imaging the evaluation target bone material to obtain the captured image data is IB [lx], IB [lx] is 0.2 times or more of IA [lx] and 5 times or less of IA [lx], When preparing the training data, the construction unit generates extended image data by performing a conversion process and a grayscale process on the training captured image data corresponding to the captured image data in any order; The transformation process includes randomly modifying the brightness and contrast of the image; The input image data in the training input information includes the extended image data. Aggregate quality prediction device.

13. A manufacturing device for mixing materials including aggregate to manufacture ready-mix concrete; The quality prediction device according to any one of claims 10 to 12, The quality prediction device predicts the quality of at least a portion of the aggregate used by the manufacturing device. Ready-mix concrete manufacturing system.

Images