Quality prediction program, quality prediction method, and quality prediction apparatus

Abstract

To provide a quality prediction program, a quality prediction method, and a quality prediction apparatus for aggregate, configured to improve prediction accuracy.SOLUTION: A quality prediction program causes a quality prediction apparatus to execute: an acquisition step of acquiring evaluation input information based on captured image data obtained by imaging aggregate to be evaluated; a specifying step of specifying a type of the aggregate to be evaluated, from among a plurality of aggregate types; a model selection step of selecting a corresponding prediction model pertaining to the type of the aggregate to be evaluated specified in the specifying step, from among a plurality of prediction models corresponding to the multiple aggregate types; and a prediction step. Each of the prediction models is built so as to output a value indicating a fineness modulus according to an input of input information based on learning image data obtained by imaging aggregate for leaning, by machine learning based on the input information and based on a ground truth value of fineness modulus associated with the input information.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present disclosure relates to a quality prediction program, a quality prediction method, and a quality prediction device. [Background technology]

[0002] Patent Document 1 discloses a method for estimating the particle size distribution of soil. Patent Document 2 discloses a method for estimating the quality of aggregate. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-117625 [Patent Document 2] JP 2021-135199 A Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a quality prediction program, a quality prediction method, and a quality prediction device that are useful for improving prediction accuracy. [Means for solving the problem]

[0005] [1] A program for causing a computer to execute the following steps: an acquisition step for acquiring evaluation input information based on image data obtained by imaging an aggregate to be evaluated; an identification step for identifying the type of aggregate to be evaluated from a plurality of aggregate types; a model selection step for selecting a corresponding prediction model corresponding to the type of aggregate to be evaluated identified in the identification step from a plurality of prediction models corresponding to the plurality of aggregate types; and a prediction step, wherein each of the plurality of prediction models is constructed to output a value indicating a coarse grain ratio in response to input of input information by machine learning based on input information based on learning image data obtained by imaging an aggregate for training and a ground truth value of the coarse grain ratio associated with the input information, and in the prediction step, the coarse grain ratio of the aggregate to be evaluated is calculated based on the corresponding prediction model and the evaluation input information. When a prediction model is constructed by machine learning after preparing learning data using a specific type of aggregate, if other types of aggregate are evaluated, the prediction accuracy using the prediction model may decrease due to differences in characteristics appearing in the image. In response to this, the above program selects a prediction model corresponding to the type of aggregate to be evaluated from among multiple prediction models, and then performs prediction using the selected prediction model. This makes it possible to avoid a decrease in prediction accuracy due to differences in aggregate types between the learning phase and the evaluation phase. This is therefore useful for improving prediction accuracy.

[0006] [2] A program for causing a computer to execute the following steps: an acquiring step of acquiring evaluation input information based on image data obtained by imaging an aggregate to be evaluated; an identifying step of identifying the type of the aggregate to be evaluated from a plurality of aggregate types; a model selecting step of selecting a corresponding prediction model corresponding to the type of aggregate to be evaluated identified in the identifying step from a plurality of prediction models corresponding to the plurality of aggregate types; and a predicting step, wherein each of the plurality of prediction models is constructed to output a value indicating a mass fraction for each division included in the plurality of granularity divisions in response to input of the input information by machine learning based on input information based on learning image data obtained by imaging an aggregate for training and ground truth values ​​of mass fractions for each of a plurality of granularity divisions of the aggregate corresponding to the input information; and This quality prediction program, like the program described in [1] above, is useful for improving prediction accuracy.

[0007] [3] The quality prediction program described in [1] or [2] above, wherein in the identification process, the type of aggregate to be evaluated is identified based on user input indicating the type of the aggregate to be evaluated. In this case, the user can know the quality prediction result after understanding the type of aggregate that he / she selected, which is useful for improving user convenience.

[0008] [4] The quality prediction program described in any one of [1] to [3] above, further causing a computer to execute a result display process of displaying on a monitor the calculation result of the coarse aggregate ratio in the prediction process and a control range representing an appropriate range of the coarse aggregate ratio according to the type of aggregate to be evaluated. In this case, the user can immediately understand whether the prediction result is appropriate or not, which is useful for improving user convenience.

[0009] [5] A quality prediction method comprising: an acquisition step of acquiring evaluation input information based on image data obtained by imaging an aggregate to be evaluated; an identification step of identifying the type of the aggregate to be evaluated from a plurality of aggregate types; a model selection step of selecting a corresponding prediction model corresponding to the type of aggregate to be evaluated identified in the identification step from a plurality of prediction models corresponding to the plurality of aggregate types; and a prediction step, wherein each of the plurality of prediction models is constructed so as to output a value indicating a coarse grain ratio in response to input of the input information by machine learning based on input information based on learning image data obtained by imaging an aggregate for training and a ground truth value of the coarse grain ratio corresponding to the input information, and in the prediction step, the coarse grain ratio of the aggregate to be evaluated is calculated based on the corresponding prediction model and the evaluation input information. This quality prediction method, like the program described in [1] above, is useful for improving prediction accuracy.

[0010] [6] A quality prediction method including: an acquiring step of acquiring evaluation input information based on image data obtained by imaging an aggregate to be evaluated; an identifying step of identifying the type of the aggregate to be evaluated from a plurality of aggregate types; a model selecting step of selecting a corresponding prediction model corresponding to the type of the aggregate to be evaluated identified in the identifying step from a plurality of prediction models corresponding to the plurality of aggregate types; and a predicting step, wherein each of the plurality of prediction models is constructed to output a value indicating a mass fraction for each classification included in the plurality of granularity classifications in response to input of the input information by machine learning based on input information based on learning image data obtained by imaging an aggregate for training and ground truth values ​​of mass fractions for each of a plurality of granularity classifications of the aggregate corresponding to the input information, and the predicting step sequentially includes the steps of: calculating a mass fraction for each of the classifications in the aggregate to be evaluated based on the corresponding prediction model and the evaluation input information; and calculating a coarse grain ratio of the aggregate to be evaluated from the calculation results of the mass fraction for each classification. This quality prediction method, like the program described in [2] above, is useful for improving prediction accuracy.

[0011] [7] The quality prediction method described in [5] or [6] above, further comprising a model construction step of constructing the plurality of predictive models by executing the machine learning for each type of aggregate contained in the plurality of aggregate types. In this case, a plurality of prediction models corresponding to a plurality of aggregate types can be obtained.

[0012] [8] A quality prediction device comprising: an evaluation information acquisition unit that acquires evaluation input information based on captured image data obtained by imaging an aggregate to be evaluated; a type identification unit that identifies the type of the aggregate to be evaluated from a plurality of aggregate types; a model selection unit that selects a corresponding prediction model corresponding to the type of aggregate to be evaluated identified by the type identification unit from a plurality of prediction models corresponding to the plurality of aggregate types; and a prediction calculation unit, wherein each of the plurality of prediction models is constructed to output a value indicating a coarse grain ratio in response to input of input information by machine learning based on learning image data obtained by imaging an aggregate for training and a ground truth value of a coarse grain ratio corresponding to the input information, and the prediction calculation unit calculates the coarse grain ratio of the aggregate to be evaluated based on the corresponding prediction model and the evaluation input information. This quality prediction device is useful for improving prediction accuracy, similar to the program described in [1] above.

[0013] [9] A quality prediction device comprising: an evaluation information acquisition unit that acquires evaluation input information based on captured image data obtained by imaging an aggregate to be evaluated; a type identification unit that identifies the type of the aggregate to be evaluated from a plurality of aggregate types; a model selection unit that selects a corresponding prediction model corresponding to the type of the aggregate to be evaluated identified by the type identification unit from a plurality of prediction models corresponding to the plurality of aggregate types; and a prediction calculation unit, wherein each of the plurality of prediction models is constructed to output a value indicating a mass fraction for each classification included in the plurality of granularity classifications in response to input of the input information by machine learning based on input information based on learning image data obtained by imaging an aggregate for training and ground truth values ​​of mass fractions for each of a plurality of granularity classifications of the aggregate corresponding to the input information, and the prediction calculation unit sequentially executes the following steps: calculates a mass fraction for each of the classifications in the aggregate to be evaluated based on the corresponding prediction model and the evaluation input information; and calculates a coarse grain ratio of the aggregate to be evaluated from the calculation results of the mass fraction for each of the classifications. This quality prediction device is useful for improving prediction accuracy, similar to the program described in [2] above.

[0014]

[10] The quality prediction device described in [8] or [9] above, further comprising a model construction unit that constructs the multiple prediction models by executing the machine learning for each type of aggregate included in the multiple aggregate types. In this case, a plurality of prediction models corresponding to a plurality of aggregate types can be obtained. Effect of the Invention

[0015] According to the present disclosure, a quality prediction program, a quality prediction method, and a quality prediction device that are useful for improving prediction accuracy are provided. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a schematic diagram illustrating a ready-mix concrete manufacturing system and a quality prediction device. [Diagram 2] FIG. 2 is a block diagram illustrating an example of a hardware configuration of the quality prediction device. [Diagram 3] FIG. 3 is a block diagram illustrating an example of a functional configuration of the quality prediction device. [Figure 4] Fig. 4(a) is a schematic diagram showing an example of a predictive calculation using a predictive model, and Fig. 4(b) is a cross-sectional view showing a schematic diagram of acquiring image data for learning. [Diagram 5] FIG. 5 is a diagram showing an example of image data for learning. [Figure 6] 6(a) and 6(b) are graphs showing an example of the comparison result between the predicted value and the correct value. [Figure 7] FIG. 7 is a flow diagram showing an example of a series of processes executed by the quality prediction device. [Figure 8] 8(a) and 8(b) are schematic diagrams showing an example of a display screen on a monitor. [Figure 9] 9(a) and 9(b) are schematic diagrams showing an example of a display screen on a monitor. [Figure 10] 10(a) and 10(b) are schematic diagrams showing an example of a display screen on a monitor. [Figure 11] FIG. 11 is a schematic diagram showing an example of a display screen on a monitor. [Figure 12] FIG. 12 is a schematic diagram showing an example of a display screen on a monitor. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, an embodiment will be described with reference to the drawings. In the description, the same elements or elements having the same functions are given the same reference numerals, and duplicated descriptions will be omitted. Fig. 1 shows a schematic diagram of a quality prediction device according to an embodiment, and a ready-mix concrete manufacturing system in which the quality prediction device is used.

[0018] [Ready-mix concrete manufacturing system] First, an overview of a ready-mix concrete manufacturing system will be described. The manufacturing system 100 shown in Fig. 1 is a system that manufactures ready-mix concrete. The manufacturing system 100 kneads and mixes concrete materials to manufacture (produce) ready-mix concrete.

[0019] The concrete materials used in the manufacturing system 100 include cement, admixtures, coarse aggregates, fine aggregates, water, and admixtures. Examples of the coarse aggregates include gravel, crushed stone, slag coarse aggregates, lightweight coarse aggregates, recycled coarse aggregates, reclaimed aggregates, and coarse aggregates made by mixing these. Examples of the gravel include mountain gravel, land gravel, river gravel, and sea gravel. Examples of the slag coarse aggregates include blast furnace slag aggregates, ferronickel slag aggregates, electric furnace oxidized slag aggregates, and coal gasification slag aggregates. Examples of the lightweight coarse aggregates include natural lightweight aggregates, by-product lightweight aggregates, and artificial lightweight aggregates. The coarse aggregates may include crushed rock, or limestone crushed stone.

[0020] Examples of fine aggregates include sand, crushed sand, slag fine aggregate, lightweight fine aggregate, recycled fine aggregate, recovered aggregate, and fine aggregates made by mixing these. Sand includes mountain sand, land sand, river sand, sea sand, etc. Slag fine aggregates include blast furnace slag aggregate, ferro-nickel slag aggregate, copper slag aggregate, electric furnace oxidized slag aggregate, coal gasification slag aggregate, etc. Lightweight fine aggregates include natural lightweight aggregate, by-product lightweight aggregate, artificial lightweight aggregate, etc.

[0021] Examples of rock types of crushed stone and crushed sand include igneous rocks, sedimentary rocks, metamorphic rocks, quartzite, limestone, domalite, and peridotite. Igneous rocks include granite, diorite, gabbro, porphyrite, diabase, rhyolite, andesite, basalt, and serpentine. Sedimentary rocks include conglomerate, sandstone, shale, slate, and tuff. Metamorphic rocks include gneiss and schist. In the present disclosure, coarse aggregate and fine aggregate may be collectively referred to as "aggregate". In this case, "aggregate" refers to coarse aggregate, fine aggregate, or both coarse aggregate and fine aggregate.

[0022] The manufacturing system 100 loads the manufactured ready-mix concrete onto a transport vehicle 200. After the ready-mix concrete is loaded, the transport vehicle 200 transports the ready-mix concrete to a site (e.g., a construction site) where the ready-mix concrete is to be used. Examples of the transport vehicle 200 include an agitator vehicle (mixer vehicle) or a dump truck. The manufacturing system 100 may manufacture ready-mix concrete from concrete materials so as to satisfy a target quality (required quality) set for each site. In one example, the target quality of the ready-mix concrete includes a target value for slump or slump flow. The manufacturing system 100 includes, for example, a material storage yard 101, a transport device 104, and a manufacturing device 110.

[0023] The material yard 101 is a place where concrete materials are stored. The material yard 101 includes a plurality of silos 102. The plurality of silos 102 are containers that store at least a portion of the concrete materials by material type. The plurality of silos 102 include a silo 102 that stores coarse aggregate, a silo 102 that stores fine aggregate, and a silo 102 that stores cement.

[0024] The transporting device 104 is a device that transports the concrete materials stored in the multiple silos 102 to the manufacturing device 110. The transporting device 104 includes, for example, a belt conveyor that transports the concrete materials. The transporting device 104 may transport the concrete materials by type at different times. In one example, based on an operation instruction from a control device included in the manufacturing system 100, a specific material from among the various concrete materials is transferred to the transporting device 104 and transported to the manufacturing device 110.

[0025] The manufacturing apparatus 110 is an apparatus for manufacturing ready-mixed concrete by mixing concrete materials including aggregate. The manufacturing apparatus 110 includes, for example, a storage bottle 111, a measuring bottle 112, a collecting hopper 113, a mixer 114, and a loading hopper 115.

[0026] Storage bottle 111 temporarily stores various types of concrete materials. Various types of concrete materials are transported (conveyed) to storage bottle 111 from material storage yard 101 by transport device 104. Storage bottle 111 is configured to store various types of concrete materials individually. Hereinafter, "concrete material" may be simply referred to as "material". Various materials stored in storage bottle 111 are supplied to measuring bottle 112 as necessary.

[0027] The measuring bottle 112 is disposed below the storage bottle 111. The measuring bottle 112 operates based on an operation command from the control device, and weighs each material individually. When the measuring bottle 112 detects a target amount of material instructed by the control device, it supplies the material to the collecting hopper 113. When water is supplied to the measuring bottle 112, an admixture may be mixed into the water. The collecting hopper 113 is disposed below the measuring bottle 112. The collecting hopper 113 collects each material discharged from the measuring bottle 112, and supplies the collected materials to the mixer 114. Note that the collecting hopper 113 does not have to be provided, and in this case, each material is supplied from the measuring bottle 112 to the mixer 114.

[0028] The mixer 114 is disposed below the collecting hopper 113. The mixer 114 is a device that mixes concrete materials. The mixer 114 produces ready-mixed concrete by mixing (kneading) aggregate, cement, water, admixtures, and the like. The ready-mixed concrete is discharged from the bottom of the mixer 114 into the loading hopper 115. The loading hopper 115 is disposed below the mixer 114, and temporarily stores the ready-mixed concrete. The loading hopper 115 supplies the temporarily stored ready-mixed concrete to the transport vehicle 200.

[0029] The manufacturing system 100 described above is one example of a ready-mix concrete manufacturing system, and the ready-mix concrete manufacturing system may be configured in any manner as long as it is capable of mixing concrete materials and manufacturing ready-mix concrete.

[0030] [Quality prediction device] The quality prediction device 1 is a device that predicts the coarse aggregate ratio as the quality of aggregate contained in concrete material. The quality prediction device 1 is configured with one or more computers. The computers that configure the quality prediction device 1 may be a personal computer, a tablet computer (tablet terminal), a smartphone, a wearable device, a workstation, a server computer, or a general-purpose computer. The quality prediction device 1 may be connected to a communication network via a wired or wireless connection.

[0031] The quality prediction device 1 includes, for example, a main body 10, an input device 12, a monitor 14, and a camera 16. The main body 10 is a device (computer main body) that executes the main functions of the computer that constitutes the quality prediction device 1.

[0032] The input device 12 is a device for inputting information to the main body 10. The input device 12 may be any device capable of inputting desired information, and specific examples thereof include operation interfaces such as a keypad, a mouse, and an operation controller.

[0033] The monitor 14 is a device for displaying information output from the main body 10. The monitor 14 may be any device capable of displaying graphics, a specific example of which is a liquid crystal panel. The monitor 14 and the input device 12 may be integrated as a touch panel.

[0034] The camera 16 is a device capable of capturing an image of the aggregate. The camera 16 captures an image of the aggregate to generate image data. The camera 16 is, for example, a digital camera that captures an image of the imaging range based on visible light, or a video camera. The image data generated by the camera 16 is used to predict the quality of the aggregate. When the camera 16 generates a video of the imaging target, still images included in the video may be acquired as image data.

[0035] The camera 16 may be disposed in the manufacturing system 100 so as to be able to capture an image of an aggregate at a specific location. For example, the camera 16 may be disposed so as to be able to capture an image of an aggregate being transported by the transport device 104. The camera 16 may be portable, and a user such as a worker may carry the camera 16 to an imaging position. The camera 16 may be mounted on a mobile body such as a drone, and the camera 16 may capture an image at any location in the manufacturing system 100 by a user operating the mobile body, etc.

[0036] In the quality prediction device 1, the main body 10, the input device 12, the monitor 14, and the camera 16 may be integrated into one device, such as a smartphone with a built-in (mounted) camera or a tablet computer with a built-in camera. In one example, the quality prediction device 1 is configured by installing an application for predicting the quality of aggregate on a smartphone.

[0037] As shown in FIG. 2, the main body 10 includes a circuit 50. The circuit 50 includes a processor 51, a memory 52, a storage 53, and an input / output port 54. The storage 53 is configured with one or more non-volatile memory devices such as a flash memory or a hard disk. The storage 53 stores a quality prediction program that causes a computer to execute at least an acquisition step, a specification step, a model selection step, and a prediction step. The quality prediction program may cause a computer to execute a specification step, an image display step, an acquisition step, a prediction step, and a result display step. Details of these steps will be described later. The storage 53 stores a quality prediction program for configuring each functional block of the main body 10.

[0038] The memory 52 is composed of one or more volatile memory devices such as a random access memory. The memory 52 temporarily stores a quality prediction program loaded from the storage 53. The processor 51 is composed of one or more arithmetic devices such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The processor 51 configures each functional block of the main body 10 by executing the quality prediction program loaded into the memory 52. ​​The calculation results by the processor 51 are temporarily stored in the memory 52. ​​The input / output port 54 inputs and outputs information between the input device 12, the monitor 14, the camera 16, etc. in response to a request from the processor 51.

[0039] The quality prediction program may be provided in a state of being fixedly recorded on a tangible recording medium such as a CD-ROM, a DVD-ROM, or a semiconductor memory. Alternatively, the quality prediction program may be provided via a communication network as a data signal superimposed on a carrier wave. The main body 10 (quality prediction device 1) may be composed of a plurality of computers communicably connected to each other. The main body 10 (quality prediction device 1) may be a part of a control device included in the manufacturing system 100.

[0040] 3 shows an example of functional components (referred to as "functional blocks" in this disclosure) included in the main body 10. The main body 10 includes, as functional blocks, a user input acquisition unit 22, an image data acquisition unit 24, an evaluation information acquisition unit 26, a prediction model storage unit 28, a model construction unit 38, a model selection unit 29, a prediction calculation unit 30, a display output unit 32, and an aggregate information storage unit 34. The processes executed by these functional blocks correspond to the processes executed by the main body 10 (quality prediction device 1).

[0041] The user input acquisition unit 22 is configured to acquire user input information indicating an input from a user required for predicting the coarse grain ratio of the aggregate to be evaluated. The user input acquisition unit 22 may acquire the user input information based on a user input to an interface screen displayed on the monitor 14.

[0042] The image data acquisition unit 24 is configured to acquire image data generated by capturing an image of the aggregate to be evaluated (hereinafter, referred to as "captured image data"). The image data acquisition unit 24 acquires captured image data (captured image) captured by the camera 16.

[0043] The evaluation information acquiring unit 26 is configured to acquire evaluation input information based on captured image data acquired by capturing an image of the aggregate to be evaluated. The evaluation information acquiring unit 26 may acquire, for example, the captured image data as is as the evaluation input information, or may acquire image data acquired by performing a predetermined processing on the captured image data as the evaluation input information. A user input may determine whether or not to process the captured image data in order to obtain the evaluation input information, and how to process the captured image data.

[0044] The prediction model holding unit 28 is configured to hold a plurality of prediction models M corresponding to a plurality of aggregate types. Each of the plurality of prediction models M is a model for predicting the coarse grain ratio of the aggregate to be evaluated. In the prediction model holding unit 28, each of the plurality of prediction models M is associated with a specific aggregate type. The prediction model M is a model constructed to output a predicted value of the quality value in response to input of the input information by machine learning based on input information based on learning image data (learning image data) obtained by imaging the learning aggregate and a correct answer value of a quality value indicating the quality of the aggregate associated with the input information. The quality value is, for example, the coarse grain ratio, the mass fraction for each grain size division included in the grain size division of the aggregate, or both the coarse grain ratio and the mass fraction for each grain size division.

[0045] The multiple types of prediction models M held by the prediction model holding unit 28 are respectively associated with multiple types of aggregate. Each of the multiple types of prediction models M may be constructed by machine learning based on learning data obtained from the corresponding type of aggregate. The type of aggregate may be determined in any way. The type of aggregate is preset by the user or the provider of the quality prediction device 1.

[0046] The following are definitions (methods of classification) of aggregate types: Classification by aggregate type, such as gravel, sand, crushed stone, crushed sand, slag aggregate, artificial lightweight aggregate, recycled aggregate, etc., or by the mixture ratio of these. Classification by rock type (e.g. limestone, hard sandstone, etc.) Classification by place of origin (e.g. mountain sand, river gravel, etc.) Classification by type of molten slag (e.g. blast furnace slag aggregate, electric furnace oxidized slag aggregate, etc.) Classification by aggregate quality such as bone dry density and volume fraction (e.g. artificial lightweight aggregate MA-417, recycled aggregate H, etc.) Classification by aggregate particle size (e.g. coarse aggregate, fine aggregate, crushed stone 2005, etc.; crushed stone 2005 means a maximum particle size of 20 mm and a minimum particle size of 5 mm) Classification by origin or product name of aggregate Classification by aggregate manufacturer -Two or more of the definitions given above may be combined (e.g. limestone crushed sand, recycled coarse aggregate H, mountain sand (Mine City, Yamaguchi Prefecture), etc.)

[0047] The model construction unit 38 is configured to construct multiple prediction models M by executing machine learning for each type of aggregate contained in the multiple aggregate types. Machine learning refers to a method in which a machine (computer) autonomously finds laws or rules by repeatedly learning based on given information. Each of the multiple prediction models M can be constructed using an algorithm and a data structure. Each of the multiple prediction models M is realized, for example, using a neural network, which is an information processing model that mimics the mechanism of the human brain and nerves. The specific algorithm of the machine learning performed when constructing each of the multiple prediction models M is not particularly limited. The neural network has an input layer, one or more intermediate layers, and an output layer. By including one or more intermediate layers, a more complex prediction model M can be constructed, and prediction accuracy can be improved.

[0048] Each of the multiple prediction models M may be constructed by machine learning using a convolutional neural network (CNN). Each of the multiple prediction models M is constructed by machine learning based on a correct value of the quality value. Each of the multiple prediction models M may output, according to the input information, as the quality value, only the coarse grain ratio, only a value indicating the mass fraction for each grain size division included in the multiple grain size divisions, or both the coarse grain ratio and the mass fraction for each grain size division. When outputting the mass fraction for each grain size division, each prediction model M outputs multiple values ​​indicating mass fractions for the multiple grain size divisions.

[0049] The model selection unit 29 is configured to specify the type of aggregate to be evaluated from among a plurality of aggregate types, and then select a prediction model M corresponding to the specified aggregate to be evaluated from among a plurality of prediction models M. That is, the model selection unit 29 also functions as a type specification unit that specifies the type of aggregate to be evaluated. Hereinafter, the prediction model M corresponding to the aggregate to be evaluated is referred to as a "corresponding prediction model MA". The model selection unit 29 may specify the type of aggregate to be evaluated based on information from a device other than the quality prediction device 1 (for example, a control device provided in the manufacturing system 100). The model selection unit 29 may specify the type of aggregate to be evaluated based on a user input specifying the type of aggregate, or may specify the type of aggregate by a program that reads an image. For example, the program that reads an image may include a prediction model for predicting the type of aggregate by machine learning, and an image may be input to the prediction model to determine the type of aggregate. Alternatively, the program that reads an image may specify the type of aggregate from features of the aggregate, such as color tone, grain size, and shape. This makes it possible to use a program for reading an image instead of a user selection in step S05 of the processing flow described later.

[0050] In one example, when aggregate types are classified according to the maximum and minimum dimensions of the aggregate, when the model selection unit 29 identifies the type of aggregate to be evaluated as "crushed stone 2005", it selects the prediction model M associated with the crushed stone 2005 as the corresponding prediction model MA from among the multiple prediction models M held (stored) in the prediction model holding unit 28. Note that crushed stone 2005 means a maximum dimension of 20 mm and a minimum dimension of 5 mm.

[0051] The prediction calculation unit 30 is configured to input evaluation input information to the corresponding prediction model MA and obtain a predicted value output from the corresponding prediction model MA. For example, the prediction calculation unit 30 obtains a predicted value indicating a coarse grain ratio from the corresponding prediction model MA. The prediction calculation unit 30 may also obtain a plurality of predicted values ​​indicating mass fraction values ​​for a plurality of grain size divisions from the corresponding prediction model MA. The prediction calculation unit 30 may calculate the coarse grain ratio from a predicted value (a plurality of predicted values) regarding the mass fraction for each grain size division. For reliability evaluation, the prediction calculation unit 30 may obtain both the coarse grain ratio and a plurality of predicted values ​​indicating mass fraction values ​​from the corresponding prediction model MA and compare the coarse grain ratios predicted for each corresponding prediction model MA. As described above, the calculation (prediction) of the coarse grain ratio is performed by two different methods using the corresponding prediction model MA (prediction model M), so that the reliability of the predicted value of the coarse grain ratio can be evaluated.

[0052] The display output unit 32 is configured to display an image obtained by imaging an aggregate to be evaluated, whose quality values ​​such as the coarse aggregate ratio are unknown, on the monitor 14. The display output unit 32 may display a prediction result by the prediction calculation unit 30 on the monitor 14. The display output unit 32 may display a predicted value of the coarse aggregate ratio calculated by the prediction calculation unit 30 and a control range indicating an appropriate range of the coarse aggregate ratio according to the type of aggregate to be evaluated on the monitor 14.

[0053] The aggregate information storage unit 34 is configured to store at least information necessary for displaying the above-mentioned control range. The aggregate information storage unit 34 stores, for example, information indicating the above-mentioned control ranges associated with each of a plurality of types of aggregate. That is, the aggregate information storage unit 34 may store information indicating the corresponding control range for each of a plurality of types of aggregate. The control range of the coarse aggregate ratio is set in advance for each type of aggregate by, for example, a user or a provider of the quality prediction device 1.

[0054] <Multiple prediction models> Here, the multiple prediction models M used to predict the coarse grain ratio will be further described with reference to Figures 4 and 5. In the following, a case will be illustrated in which an arbitrary one of the multiple prediction models M outputs both the coarse grain ratio and the mass fraction for each grain size class, with attention focused on.

[0055] The prediction model M is constructed to predict the coarse aggregate ratio and mass fraction by performing machine learning using, for example, data provided as input for machine learning and correct answer data (correct answer values ​​for the coarse aggregate ratio and mass fraction) of the machine learning output. The input for the machine learning is various data sets of input information based on learning image data obtained by imaging learning aggregate. In at least some of the various data sets of input information, at least the image data included in the input information is different from each other. The output of the machine learning is data (numerical values) indicating the coarse aggregate ratio and the mass fraction for each grain size division. When constructing the prediction model M, a model that outputs predicted values ​​for the coarse aggregate ratio, etc. is iteratively trained using multiple combinations of input information and correct answer values ​​for the coarse aggregate ratio, etc.

[0056] The stage in which the prediction model M is autonomously constructed corresponds to the learning phase. The learning phase may be performed before the production phase in which ready-mix concrete is manufactured in the manufacturing system 100, or may be performed at an early stage of the production phase. The prediction model M, which is a trained model, is portable between computers. Therefore, instead of the model construction unit 38 (own device), a prediction model M constructed in another device may be stored in the prediction model holding unit 28. The stage in which the quality prediction device 1 predicts the coarse grain ratio of the aggregate to be evaluated using the prediction model M (corresponding prediction model MA) corresponds to the evaluation phase.

[0057] In the learning phase for constructing the prediction model M, aggregate of the same type as the aggregate that is used in the manufacturing system 100 and that can be evaluated is used as the learning aggregate. Fig. 4(a) shows a schematic diagram of the calculation process for predicting the coarse particle ratio using the prediction model M from the image Im, which is input information. The input of the prediction model M is the image Im, and the output of the prediction model M is the predicted values ​​of the mass fraction of each of the multiple particle size classes and the predicted value of the coarse particle ratio.

[0058] When constructing a prediction model M, learning data for machine learning is prepared. The learning data includes various image data (input information) obtained by imaging the learning aggregate, and answer data associated with each of the various image data. In preparing the learning images, for example, samples of various aggregates whose mass fractions for each granularity classification are known are prepared. The above-mentioned samples of various aggregates prepared to construct one prediction model M do not mean different types of aggregate, but samples (various samples) obtained from the same type of aggregate but with different granularities.

[0059] In one example, various learning samples are individually imaged by the camera 130 while stacked on the measurement sheet 132 (see FIG. 4(b)). A stacked state (piled state) refers to a state in which some particles contained in the aggregate sample overlap other particles when viewed vertically from above. By individually imaging the samples by the camera 130, learning image data is obtained for each of the various samples. The camera 130 may be a digital camera, and images an area including the stacked samples from a position a predetermined distance away from the samples on the measurement sheet 132 (for example, a predetermined position above).

[0060] The sheet 132 may be a rubber sheet. As shown in FIG. 4(b), the sheet 132 may be a sheet formed to be curved to imitate the belt conveyor of the transport device 104, or may be a sheet formed in a flat shape, unlike the example shown in FIG. 4(b). The amount of sample (aggregate) placed on the sheet 132 is not particularly limited, but may be about 1 kg to 10 kg. FIG. 4(b) shows a schematic example of the state in which the sample is piled on the sheet 132, and shows the piled state when the amount of the sample is 2 kg, 5 kg, and 8 kg. Note that the sheet 132 can be a container other than a rubber sheet. If the container has edges, it is preferable from the viewpoint that it can be used to uniformly adjust the size of the captured sample (grain) by correcting the size of the image data based on the calculation result of the distance after calculating the distance from the camera to the sample.

[0061] In one example, various samples are prepared under the conditions shown in Table 1 below, and images for learning are obtained by imaging the prepared samples. The mass fraction and coarse particle ratio for each particle size division in the various samples prepared under the conditions shown in Table 1 are known, and the known values ​​are the correct answer data in machine learning.

[0062] [Table 1]

[0063] The coarse grain ratio (FM) is calculated, for example, by the following formula (1) specified in the Standard Method for Concrete. In the measurement specified in the "Standard Specifications for Building Construction and Commentary JASS5 Reinforced Concrete Construction", 9 sieves are used excluding the 80 mm sieve in formula (1). The coarse grain ratio predicted in this disclosure may be specified by any method.

number

[0064] In Table 1, this means that aggregates of 10 levels of gradation (coarse aggregate ratio) are prepared. Category I represents a gradation category of particles smaller than 2.5 mm. Category I includes particles that cannot be retained on a 2.5 mm sieve. Category II represents a gradation category of particles 2.5 mm or larger and smaller than 5 mm. Category II includes particles that can be retained on a 2.5 mm sieve but cannot be retained on a 5 mm sieve. Category III represents a gradation category of particles 5 mm or larger and smaller than 10 mm. Category III includes particles that can be retained on a 5 mm sieve but cannot be retained on a 10 mm sieve. Category IV represents a gradation category of particles 10 mm or larger and smaller than 20 mm. Category IV includes particles that can be retained on a 10 mm sieve but cannot be retained on a 20 mm sieve.

[0065] For level 1, the coarse aggregate ratio can be calculated from the mass fractions of each grain size category (mass fractions of each of categories I to IV) as follows. For level 1 aggregate, the mass percentages of the sample that are retained on the 80 mm, 40 mm, and 20 mm sieves are 0, the mass percentages of the sample that are retained on the 10 mm sieve are 85, and the mass percentages of the sample that are retained on the 5 mm sieve are 100 (=15+85). Also, since all the grains are retained on the 5 mm sieve, the mass percentages of the sample that are retained on the 2.5 mm, 1.2 mm, 600 μm, 300 μm, and 150 μm sieves are 100. Therefore, the coarse aggregate ratio is calculated as shown in the following formula (2).

number

[0066] For levels 2 to 10, the coarse aggregate ratio is calculated in the same manner as for level 1. The conditions of the learning aggregate shown in Table 1 are merely examples, and the number of levels, the coarse aggregate ratio and mass fraction for each grading class at each level, and the method of setting the grading classes may be set arbitrarily. As with the multiple grading classes shown in Table 1, the grading classes in multiple classes including the smallest class (<150 μm) may be combined into one grading class. Grain sizes above a certain class (e.g., 20 to 40 mm) may not be included in multiple grading classes. Also, unlike the setting method in Table 1, grain sizes below a certain class (e.g., 1.2 to 2.5 μm) may not be included in multiple grading classes.

[0067] FIG. 5 shows images T1 to T10 obtained by imaging learning aggregates prepared according to the conditions of each of levels 1 to 10 in Table 1. Level n (n is an integer from 1 to 10) corresponds to image Tn. For example, image T1 is obtained by imaging learning aggregates prepared according to level 1. Learning aggregates according to each level n contain grains of multiple grain size divisions. In this way, learning aggregates may be prepared so as to contain grains of two or more grain size divisions, and input information may be prepared based on image data obtained by imaging the learning aggregates.

[0068] For each level n, multiple images Tn in which the states (more specifically, pile states) of the learning aggregate are different from each other may be prepared. In one example, a first image is taken with the learning aggregate placed on the measurement sheet 132, to obtain one image Tn. Then, the learning aggregate is temporarily removed from the sheet 132, and the same learning aggregate (same sample) is placed again on the sheet 132 so that the pile state of the multiple grains is different from that in the first image. Then, a second image is taken. By repeating the above-mentioned image taking, multiple images Tn may be prepared for each level n.

[0069] One data set is composed of one image Tn and the correct answer data of the mass fraction and the coarse grain ratio at the level n corresponding to the image Tn. As learning data, a plurality of data sets each consisting of a combination of the image Tn and the correct answer data of the mass fraction and the coarse grain ratio are prepared. Then, a prediction model M is constructed by performing machine learning using such learning data. The prediction model M constructed as described above outputs, for example, four mass fraction values ​​and a coarse grain ratio value for four grain size classifications. The model construction unit 38 repeats the process of constructing the prediction model M in the same manner by changing the type of aggregate for learning. In this way, a plurality of prediction models M corresponding to a plurality of aggregate types are constructed.

[0070] Focusing on the point of outputting the value of the coarse grain ratio, each of the multiple prediction models M is constructed to output a value indicating the coarse grain ratio in response to input of the input information by machine learning based on input information based on learning image data obtained by imaging the learning aggregate and the correct value of the coarse grain ratio associated with the input information. Focusing on the point of outputting the value of the mass fraction for each grain size division, each of the multiple prediction models M is constructed to output a value indicating the mass fraction for each grain size division in response to input of the input information by machine learning based on input information based on learning image data obtained by imaging the learning aggregate and the correct value of the mass fraction for each of the multiple grain size divisions of the aggregate associated with the input information.

[0071] In order to verify the method of predicting the coarse aggregate ratio using the prediction model M based on machine learning using a convolutional neural network, a comparison was made between the prediction results by the prediction model M and the correct values ​​using a dataset in which the correct values ​​were known. For the learning aggregate, 10 levels of aggregate with different coarse aggregate ratios were prepared according to the conditions in Table 1 above. For each level, images were taken 10 times indoors with the aggregate piled up on the measurement sheet 132 in different states, to obtain 10 images.

[0072] In addition, a total of 100 images (10 levels x 10 shots) were prepared according to the standard conditions shown in Table 2 below. As a result, 100 data sets were prepared as learning data, each of which corresponds to an image and a correct answer value.

[0073] [Table 2]

[0074] The prediction model M was constructed using 80 of the 100 datasets, and 20 datasets were used for validation. At each level, 8 data were used for model construction, and 2 datasets were used for validation. In the validation using 20 datasets, the prediction results calculated using the image and the prediction model M for each dataset were compared with the correct values ​​such as the coarse grain rate associated with the image. The prediction model M was constructed using a convolutional neural network with 6 intermediate layers consisting of a convolutional layer and a pooling layer, followed by 4 fully connected layers. The Huber function was used as the loss function. In each dataset, the number of images was increased by the Image Augmentation layer.

[0075] FIG. 6(a) is a graph showing the relationship between the predicted values ​​of the mass ratio (%) in the divisions III and IV in Table 1 and the corresponding correct values ​​of the mass fraction (%). In the graph shown in FIG. 6(a), the horizontal axis is the correct value of the mass fraction, and the vertical axis is the predicted value of the mass fraction output from the prediction model M. The prediction results in division III (5 mm to 10 mm) are plotted with △ marks, and the prediction results in division IV (10 mm to 20 mm) are plotted with ◯ marks. A solid line where the correct value and the predicted value match is drawn, and the closer the plotted results are to the line, the higher the prediction accuracy. From the results shown in FIG. 6(a), the prediction results are plotted near the line, and it can be seen that the mass ratio can be predicted with high accuracy by the prediction model M. Note that the coarse particle ratio can be calculated from the mass fraction, so if the mass fraction can be predicted with high accuracy by the prediction model M, the coarse particle ratio can also be predicted with high accuracy.

[0076] FIG. 6(b) is a graph showing the relationship between the predicted value of the coarse grain ratio output from the prediction model M and the correct value of the coarse grain ratio. In the graph shown in FIG. 6(b), the horizontal axis is the correct value of the coarse grain ratio, and the vertical axis is the predicted value of the coarse grain ratio output from the prediction model M. The prediction results are plotted with square marks. The solid line is the line where the correct value and the predicted value match, and a dashed upper limit line shifted vertically by +0.10 from the solid line and a lower limit line shifted vertically by -0.10 are drawn. The prediction results included in the area between the upper limit line and the lower limit line are defined as being within the allowable range. With reference to the error in the coarse grain ratio allowed in current ready-mix concrete plants, a range of ±0.10 was set as the allowable range. It can be seen that all of the prediction results using the prediction model M are included in the allowable range, and the prediction model M can predict the coarse grain ratio with high accuracy.

[0077] [Quality prediction method] Next, a quality prediction method executed in the quality prediction device 1 will be described with reference to Figs. 7 to 12. Hereinafter, the quality prediction method will be described using an example in which the quality prediction device 1 is configured by a smartphone with a camera. This quality prediction method includes, for example, a model construction step, a specification step, a model selection unit, an image display step, an acquisition step, a prediction step, and a result display step. The model construction step is a step executed in the learning phase. The specification step, the model selection unit, the image display step, the acquisition step, the prediction step, and the result display step are all steps executed in the evaluation phase. Each of the model construction step, specification step, image display step, acquisition step, prediction step, and result display step may be executed by a corresponding functional block in the quality prediction device 1.

[0078] The model construction step is a step of constructing a plurality of prediction models M by executing machine learning for each type of aggregate contained in the plurality of aggregate types. In the machine learning executed for each type of aggregate, training data prepared using the corresponding type of training aggregate is used.

[0079] The identification process is a process of identifying the type of aggregate to be evaluated, the coarse aggregate ratio of which is the prediction target, from among a plurality of aggregate types. In the identification process, the type of aggregate to be evaluated may be identified based on a user input indicating the type of aggregate to be evaluated. In the identification process, the type of aggregate to be evaluated may be identified based on information obtained from a device other than the device itself (quality prediction device 1).

[0080] The model selection step is a step of selecting a corresponding prediction model MA corresponding to the type of aggregate to be evaluated identified in the identification step from among a plurality of prediction models M corresponding to a plurality of aggregate types. For example, in the prediction model holding unit 28, the type of aggregate is linked to each prediction model M, and the corresponding prediction model M is selected by referring to the result of identifying the type of aggregate.

[0081] The image display step is a step of displaying an image obtained by capturing an image of the aggregate to be evaluated on the monitor 14 before the acquisition step. The image display step may include adjusting the captured image displayed on the monitor 14 when a user input instructing image adjustment is input. The adjustment of the captured image performed in the image display step may include at least one of enlarging an image, reducing an image, cropping an image, translating an image, changing the brightness of an image, rotating an image, changing the clarity of an image, changing the contrast of an image, and grayscaling an image.

[0082] In one example, when a user input instructing to rotate the image is input in the image display process, the captured image displayed on the monitor 14 is rotated. In the rotation of the captured image displayed on the monitor 14, for example, the image is rotated 90° to the right or left so that the vertical and horizontal directions on the image are switched. When the outer frame of the captured image is a square, the coordinates are converted in accordance with the rotation of the captured image. For example, when a device such as a smartphone is used as a computer constituting the quality prediction device 1, a gyro sensor mounted on the device may rotate the shooting screen, resulting in a captured image that is not intended by the photographer. By rotating the captured image to make the orientation of the learning image data and the orientation of the captured image the same, this is preferable from the viewpoint of improving the prediction accuracy of the coarse grain ratio.

[0083] The acquisition step is a step of acquiring evaluation input information based on evaluation image data obtained by imaging the aggregate to be evaluated. In the acquisition step, the evaluation input information is determined based on the state of the captured image displayed on the monitor 14 at the time when a user input instructing the determination of the evaluation input information is input. In one example, in the image display step, data indicating the current captured image (at the time of performing the acquisition step) displayed on the monitor 14 is acquired as the evaluation input information. Since pixel values ​​for each coordinate are different in at least a part of the image between an unadjusted image (e.g., an image not rotated by 90°) and an adjusted image (e.g., an image rotated by 90°), which may be the same captured image, different evaluation input information is obtained.

[0084] The prediction step is a step of calculating the coarse grain ratio of the aggregate to be evaluated based on the corresponding prediction model MA (the prediction model M selected in the model selection step) and the evaluation input information. In the prediction step, the evaluation input information is input to the corresponding prediction model MA, and a predicted value for the coarse grain ratio output from the corresponding prediction model MA is obtained. In addition, in the prediction step, the following steps are sequentially executed: calculating a mass fraction for each of the above-mentioned gradation classes in the aggregate to be evaluated based on the corresponding prediction model MA and the evaluation input information; and calculating the coarse grain ratio of the aggregate to be evaluated from the calculation results of the mass fraction for each gradation class. In the calculation of the mass fraction for each gradation class, the evaluation input information is input to the corresponding prediction model MA, and a predicted value for the mass fraction output from the corresponding prediction model MA is obtained for each gradation class.

[0085] The result display step is a step of displaying the prediction result of the coarse aggregate ratio in the prediction step on the monitor 14. In the result display step, the predicted value (coarse aggregate ratio) obtained in the prediction step and a control range indicating an appropriate range of the coarse aggregate ratio according to the aggregate type identified in the identification step may be displayed on the monitor 14.

[0086] <Processing flow> FIG. 7 is a flow diagram illustrating a series of processes executed in the quality prediction method. A user holds the quality prediction device 1 and executes step S01 at any timing when the user wants to predict the coarse aggregate ratio of the aggregate to be evaluated. In step S01, for example, the user starts an application for predicting quality by operating on a screen displayed on the monitor 14. In one example, as shown in FIG. 8(a), in a state where a home screen is displayed on the monitor 14 of the quality prediction device 1, the user clicks, taps, or presses an icon 60 representing an application for predicting quality.

[0087] When the quality prediction device 1 receives the start request of step S01, it executes step S02. In step S02, for example, the display output unit 32 of the quality prediction device 1 displays a home screen of an application that executes quality prediction on the monitor 14. An example of the home screen is shown in Fig. 8(b), and on the home screen, an execute button 62 for switching to a mode for photographing (taking an image of) an aggregate and an execute button 64 for switching to a mode for executing another function (for example, a mode for viewing past prediction results) are displayed.

[0088] Next, the user executes step S03. In step S03, for example, the user presses (e.g., taps) the execute button 62 displayed on the monitor 14 to select a mode for photographing the aggregate. When the selection of step S03 is made, the quality prediction device 1 executes step S04. In step S04, for example, the display output unit 32 switches the display content on the monitor 14 to a screen corresponding to the mode for photographing the aggregate. FIG. 9(a) shows an example of the screen after switching in step S04.

[0089] Next, the user executes step S05. In step S05, for example, the user operates the monitor 14 to select a type corresponding to the aggregate to be evaluated from among a plurality of aggregate types. As also shown in FIG. 9(b), the display output unit 32 may display a pull-down menu 66 on the monitor 14 from which the user can select the type of aggregate to be evaluated. The user may select an aggregate type that matches the type of aggregate to be evaluated from a plurality of aggregate types previously determined in the pull-down menu 66. FIG. 10(a) shows an example of a display screen after the type of aggregate to be evaluated has been selected. Note that step S05 may not be performed manually by the user, but may be automatically determined by a program that reads an image.

[0090] After the selection in step S05 is performed, the quality prediction device 1 executes step S06. In step S06, for example, the model selection unit 29 identifies the type of aggregate to be evaluated according to the selection result in step S05. Then, the model selection unit 29 selects, as a corresponding prediction model MA, a prediction model M corresponding to the identified aggregate type from among the multiple prediction models M held in the prediction model holding unit 28 according to the identification result of the type of aggregate to be evaluated.

[0091] After execution of steps S05 and S06, the display output unit 32 displays a start button 68 for inputting an instruction to start the camera 16. Before execution of steps S05 and S06, the display output unit 32 may display the start button 68 on the monitor 14 in a state in which the user cannot tap it. In FIG. 9(a) and the like, the button shown with a dashed line represents a state in which the user cannot tap it. Alternatively, the display output unit 32 may not display the start button 68 on the monitor 14 before execution of steps S05 and S06, and may display the start button 68 on the monitor 14 after execution of steps S05 and S06.

[0092] Next, the user executes step S07. In step S07, for example, the user instructs the camera 16 to be started, and then the camera 16 performs image capture. In one example, when the user taps the start button 68, the quality prediction device 1 starts the camera 16, and as shown in FIG. 10(b), the display output unit 32 displays a preview image 70 captured by the camera 16 and an image capture execution button 71 for instructing the camera 16 to perform image capture on the monitor 14. When the user taps the image capture execution button 71 while viewing the preview image 70 on the monitor 14, the camera 16 captures an image of the aggregate to be evaluated.

[0093] After the imaging in step S07, the quality prediction device 1 executes step S08. In step S08, for example, the image data acquisition unit 24 acquires captured image data generated by imaging using the camera 16, and the display output unit 32 displays an image 72 (captured image) based on the captured image data on the monitor 14. Fig. 11 illustrates a state in which the image 72 based on the captured image data, a rotation execution button 74 for instructing rotation, which is an example of adjustment of the image 72, a decision button 75 for instructing decision of an image (evaluation input information) to be input to the prediction model M, and a prediction execution button 76 for instructing execution of prediction by the prediction model M are displayed on the monitor 14.

[0094] Next, the user executes step S09. In step S09, for example, the user taps the rotation execution button 74 as necessary so as to rotate the image 72 on the monitor 14 by 90° per tap. If the user does not need to rotate the image 72 displayed on the monitor 14, he or she does not tap the rotation execution button 74. In one example, when the image 72 is captured by the camera 16 mounted on a device such as a smartphone constituting the quality prediction apparatus 1, the vertical and horizontal directions of the image 72 may be automatically converted. In this case, the user can adjust the image 72 to match the vertical direction, horizontal direction, up and down, and left and right within the field of view of the camera 16 by tapping the rotation execution button 74 one or more times.

[0095] In step S09, when the user taps the rotation execution button 74, the quality prediction device 1 receives an instruction to change the image angle (adjust the image) and executes step S10. In step S10, for example, the display output unit 32 rotates the captured image displayed on the monitor 14 by 90 degrees. This allows the user to check the captured image after rotation.

[0096] Next, in step S11, for example, when the user judges that the image 72 displayed on the monitor 14 is acceptable as an input to the prediction model M, the user taps the decision button 75 and then taps the prediction execution button 76. When the decision button 75 is tapped, the evaluation information acquisition unit 26 acquires evaluation input information to be input to the corresponding prediction model MA based on the state (e.g., rotation state) of the image 72 displayed on the monitor 14 at the time when the decision button 75 is tapped. The evaluation information acquisition unit 26 acquires, for example, pixel values ​​for each coordinate as at least a part of the evaluation input information from the image 72 displayed at the time when the decision button 75 is tapped. The display output unit 32 may display the decision button 75 and the prediction execution button 76 on the monitor 14 in a state in which the user cannot tap before step S08 is executed. Alternatively, the display output unit 32 may not display the decision button 75 and the prediction execution button 76 on the monitor 14 before step S08 is executed, and may display the decision button 75 and the prediction execution button 76 on the monitor 14 after step S08 is executed.

[0097] In step S11, when the user taps the prediction execution button 76, the quality prediction device 1 receives an execution instruction regarding the prediction and executes step S12. In step S12, for example, the prediction calculation unit 30 inputs the acquired evaluation input information into the corresponding prediction model MA and acquires the predicted values ​​of the coarse grain fraction and the mass fraction for each grain size division output from the corresponding prediction model MA. The prediction calculation unit 30 acquires the predicted values ​​of the coarse grain fraction and the mass fraction for each grain size division using the corresponding prediction model MA identified in step S06. The prediction calculation unit 30 calculates the coarse grain fraction from the mass fraction for each grain size division.

[0098] The display output unit 32 may not display the decision button 75. In this case, the evaluation information acquisition unit 26 acquires evaluation input information to be input to the corresponding prediction model MA based on the state (e.g., rotation state) of the image 72 displayed on the monitor 14 at the time when the user taps the prediction execution button 76. That is, the user input indicating an instruction to execute prediction may include (or may serve as) a user input indicating an instruction to decide the evaluation input information.

[0099] Next, the quality prediction device 1 executes step S13. In step S13, for example, the display output unit 32 displays the result of the prediction calculation executed in step S12 on the monitor 14. Fig. 12 illustrates a state in which the result of the prediction calculation in step S10 is displayed on the monitor 14. In one example, the display output unit 32 causes the monitor 14 to display information 82a indicating the identified aggregate type, information 82b indicating the prediction result of the coarse aggregate ratio, and information 82c indicating the management range for the coarse aggregate ratio.

[0100] The "coarse grain ratio (direct)" in the information 82b represents the result of prediction obtained by acquiring the coarse grain ratio from the prediction model M, and the "coarse grain ratio (indirect)" represents the result of prediction obtained by calculating the coarse grain ratio from the mass fraction after predicting the mass fraction for each grain size division from the prediction model M. The information 82b may describe one or both of the "coarse grain ratio (direct)" and the "coarse grain ratio (indirect)" according to the quality control item. The information 82b may also display the prediction result of the mass fraction for each grain size division obtained from the prediction model M. The display output unit 32 may display on the monitor 14 the image 84 (for example, the captured image as it is, or the image after adjusting the captured image) input to the prediction model M when the result of the coarse grain ratio is obtained.

[0101] [Variations] 7 is an example and can be modified as appropriate. For example, after executing step S07, the quality prediction device 1 may execute step S12 without going through steps S08 to S10.

[0102] The quality prediction device 1 may be a server connected to a terminal device such as a smartphone that can be operated by a user via a communication network. In this case, the terminal device may execute steps S01, S03, S05, S07, S09, and S11, and the quality prediction device 1 as the server may execute steps S02, S04, S06, S08, S10, S12, and S13. The display output unit 32 included in the quality prediction device 1 may display various information on a monitor included in the terminal device, which is a device other than the quality prediction device 1.

[0103] The quality prediction device 1 as the server may exchange information with the terminal device by presenting to the user a Web page or the like that can be viewed and accessed from the terminal device operated by the user. In this case, in step S01, login may be executed instead of starting an application. Alternatively, instead of a Web page, an application that cooperates with the quality prediction device 1 may be prepared in the terminal device, and the quality prediction device 1 as the server may provide information to the terminal device.

[0104] The prediction model M described above is an example, and the prediction model M may be any model as long as it can output at least a plurality of predicted values ​​of mass fractions for a plurality of gradation classes or coarse aggregate ratios from information based on image data of aggregates. The input data to the prediction model M may include information indicating physical quantities that may affect the mass fraction or coarse aggregate ratio, in addition to image data obtained by imaging the aggregate.

[0105] In one example among the various examples described above, at least a part of the matters described in the other examples may be combined. [Explanation of symbols]

[0106] 1...quality prediction device, 14...monitor, 24...image data acquisition unit, 26...evaluation information acquisition unit, 29...model selection unit, 30...prediction calculation unit, 32...display output unit, 38...model construction unit, M...prediction model, MA...corresponding prediction model.

Claims

1. an acquisition step of acquiring evaluation input information based on captured image data obtained by capturing an image of the aggregate to be evaluated; A step of identifying a type of aggregate to be evaluated from among a plurality of aggregate types; a model selection step of selecting a corresponding prediction model corresponding to the type of aggregate to be evaluated identified in the identification step from among a plurality of prediction models corresponding to the plurality of aggregate types; A prediction process; A program for causing a computer to execute the following: Each of the plurality of prediction models is constructed to output a value indicating a coarse aggregate ratio in response to input of the input information by machine learning based on input information based on learning image data obtained by imaging an aggregate for learning and a correct answer value of a coarse aggregate ratio associated with the input information; A quality prediction program, in which in the prediction step, a coarse grain ratio of the aggregate to be evaluated is calculated based on the corresponding prediction model and the evaluation input information.

2. an acquisition step of acquiring evaluation input information based on captured image data obtained by capturing an image of the aggregate to be evaluated; A step of identifying a type of aggregate to be evaluated from among a plurality of aggregate types; a model selection step of selecting a corresponding prediction model corresponding to the type of aggregate to be evaluated identified in the identification step from among a plurality of prediction models corresponding to the plurality of aggregate types; A prediction process; A program for causing a computer to execute the following: Each of the plurality of prediction models is constructed to output a value indicating a mass fraction for each of the plurality of granularity classifications in response to input of the input information by machine learning based on input information based on learning image data obtained by imaging an aggregate for learning and on a correct answer value of a mass fraction for each of a plurality of granularity classifications of the aggregate associated with the input information; In the prediction step, Calculating a mass fraction for each of the sections in the aggregate to be evaluated based on the corresponding prediction model and the evaluation input information; Calculating the coarse aggregate ratio of the aggregate to be evaluated from the calculation results of the mass fraction for each category.

3. The quality prediction program according to claim 1 , wherein in the specifying step, the type of the aggregate to be evaluated is specified based on a user input indicating the type of the aggregate to be evaluated.

4. The quality prediction program of claim 1 or 2, further causing a computer to execute a result display process of displaying on a monitor the calculation result of the coarse aggregate ratio in the prediction process and a control range representing an appropriate range of the coarse aggregate ratio according to the type of aggregate to be evaluated.

5. an acquisition step of acquiring evaluation input information based on captured image data obtained by capturing an image of the aggregate to be evaluated; A step of identifying a type of aggregate to be evaluated from among a plurality of aggregate types; a model selection step of selecting a corresponding prediction model corresponding to the type of aggregate to be evaluated identified in the identification step from among a plurality of prediction models corresponding to the plurality of aggregate types; A prediction step, Each of the plurality of prediction models is constructed to output a value indicating a coarse aggregate ratio in response to input of the input information by machine learning based on input information based on learning image data obtained by imaging an aggregate for learning and a correct answer value of a coarse aggregate ratio associated with the input information; A quality prediction method, wherein in the prediction step, a coarse grain ratio of the aggregate to be evaluated is calculated based on the corresponding prediction model and the evaluation input information.

6. an acquisition step of acquiring evaluation input information based on captured image data obtained by capturing an image of the aggregate to be evaluated; A step of identifying a type of aggregate to be evaluated from among a plurality of aggregate types; a model selection step of selecting a corresponding prediction model corresponding to the type of aggregate to be evaluated identified in the identification step from among a plurality of prediction models corresponding to the plurality of aggregate types; A prediction step, Each of the plurality of prediction models is constructed to output a value indicating a mass fraction for each of the plurality of granularity classifications in response to input of the input information by machine learning based on input information based on learning image data obtained by imaging an aggregate for learning and on a correct answer value of a mass fraction for each of a plurality of granularity classifications of the aggregate associated with the input information; In the prediction step, Calculating a mass fraction for each of the sections in the aggregate to be evaluated based on the corresponding prediction model and the evaluation input information; Calculating the coarse aggregate ratio of the aggregate to be evaluated from the calculation results of the mass fraction for each category.

7. The quality prediction method according to claim 5 or 6, further comprising a model construction step of constructing the plurality of prediction models by executing the machine learning for each type of aggregate included in the plurality of aggregate types.

8. an evaluation information acquisition unit that acquires evaluation input information based on captured image data obtained by capturing an image of the aggregate to be evaluated; A type identification unit that identifies a type of the aggregate to be evaluated from among a plurality of aggregate types; a model selection unit that selects a corresponding prediction model corresponding to the type of aggregate to be evaluated identified by the type identification unit from a plurality of prediction models corresponding to the plurality of aggregate types; A prediction calculation unit, Each of the plurality of prediction models is constructed to output a value indicating a coarse aggregate ratio in response to input of the input information by machine learning based on input information based on learning image data obtained by imaging an aggregate for learning and a correct answer value of a coarse aggregate ratio associated with the input information; The prediction calculation unit calculates the coarse grain ratio of the aggregate to be evaluated based on the corresponding prediction model and the evaluation input information.

9. an evaluation information acquisition unit that acquires evaluation input information based on captured image data obtained by capturing an image of the aggregate to be evaluated; A type identification unit that identifies a type of the aggregate to be evaluated from among a plurality of aggregate types; a model selection unit that selects a corresponding prediction model corresponding to the type of aggregate to be evaluated identified by the type identification unit from a plurality of prediction models corresponding to the plurality of aggregate types; A prediction calculation unit, Each of the plurality of prediction models is constructed to output a value indicating a mass fraction for each of the plurality of granularity classifications in response to input of the input information by machine learning based on input information based on learning image data obtained by imaging an aggregate for learning and on a correct answer value of a mass fraction for each of a plurality of granularity classifications of the aggregate associated with the input information; The prediction calculation unit is Calculating a mass fraction for each of the sections in the aggregate to be evaluated based on the corresponding prediction model and the evaluation input information; Calculating the coarse aggregate ratio of the aggregate to be evaluated from the calculation results of the mass fraction for each category.

10. The quality prediction device according to claim 8 or 9, further comprising a model construction unit that constructs the plurality of prediction models by executing the machine learning for each type of aggregate included in the plurality of aggregate types.

Images