Quality prediction program, quality prediction method, and quality prediction device

Abstract

To improve reliability of quality prediction.SOLUTION: A quality prediction program is a program allowing a computer to execute an image acquisition step of acquiring captured image data obtained by capturing an aggregate of an evaluation object, an image evaluation step of evaluating a captured image related to the captured image data, and a prediction step of inputting the evaluation input information based on the captured image data to a prediction model constructed to output a value indicating a fineness modulus associated with the input information in response to input of the input information, by machine learning based on input information based on learning image data obtained by capturing an aggregate for learning and a correct answer value of the fineness modulus, after the image evaluation step, and acquiring a prediction value output from the prediction model. The image evaluation step evaluates whether or not the captured image is suitable for prediction by the prediction model.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present disclosure relates to a quality prediction program, a quality prediction method, and a quality prediction device. [Background technology]

[0002] Patent Document 1 discloses a method for estimating the particle size distribution of soil. Patent Document 2 discloses a method for estimating the quality of aggregate. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-117625 [Patent Document 2] JP 2021-135199 A Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a quality prediction program, a quality prediction method, and a quality prediction device that are useful for improving the reliability of quality prediction. [Means for solving the problem]

[0005] [1] A quality prediction program that causes a computer to execute the following steps: an image acquisition step of acquiring captured image data obtained by imaging an aggregate to be evaluated; an image evaluation step of evaluating the captured image related to the captured image data; and a prediction step of inputting evaluation input information based on the captured image data after the image evaluation step, to acquire a predicted value to be output from the prediction model, the prediction model being constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for training and a ground truth value of the coarse grain ratio corresponding to the input information, the prediction model being constructed so as to output a value indicating the coarse grain ratio in response to the input of the input information, wherein in the image evaluation step, it is evaluated whether the captured image is suitable for prediction by the prediction model. The prediction model constructed by machine learning is constructed by machine learning based on image data obtained by imaging an aggregate sample under certain conditions in the learning phase. Therefore, for example, if there is a large deviation between various conditions and / or various states when imaging the aggregate to be evaluated in the evaluation phase and various conditions and / or various states in the learning phase, the accuracy of the prediction result by the prediction model may decrease. In response to this, in the above program, before the prediction step in which prediction is made using the prediction model, it is evaluated whether the captured image is suitable for prediction by the model. Therefore, when the accuracy of the prediction result may decrease, corrective measures (re-photographing, correction, etc.) can be taken. This can improve the reliability of the quality prediction result using the prediction model M. Therefore, it is useful for improving the reliability of quality prediction.

[0006] [2] A quality prediction program that causes a computer to execute the following steps: an image acquisition step of acquiring captured image data obtained by imaging an aggregate to be evaluated; an image evaluation step of evaluating the captured image related to the captured image data; and a prediction step of inputting evaluation input information based on the captured image data after the image evaluation step, for a prediction model constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for training and ground truth values ​​of mass fractions for each of a plurality of particle size classifications of the aggregate corresponding to the input information, so as to output a value indicating a mass fraction for each classification included in the plurality of particle size classifications in response to input of the input information, and acquiring a predicted value for each classification to be output from the prediction model, wherein in the image evaluation step, it is evaluated whether the captured image is suitable for prediction by the prediction model. This quality prediction program, like the program described in [1] above, is useful for improving the reliability of quality prediction.

[0007] [3] The quality prediction program according to [1] or [2] above, further comprising a warning display step of causing a computer to execute, before the prediction step, a warning to a user that the captured image is unsuitable for prediction by the prediction model when the captured image is determined to be unsuitable in the image evaluation step. In this case, by displaying a warning to the user, the user can be made aware that the captured image is not appropriate and can be prompted to take corrective measures such as re-capturing the image, which is further useful for improving the reliability of quality prediction.

[0008] [4] The quality prediction program according to any one of [1] to [3] above, further causing the computer to execute an image display process of displaying the captured image on a monitor before the prediction process when the captured image is determined in the image evaluation process to be unsuitable for prediction by the prediction model. Even if the captured image is judged to be inappropriate, it may be judged by the human eye that there is no problem with prediction using the prediction model. By performing the image display process, the user can decide whether to take corrective action or to continue with the prediction as is. Therefore, it is useful for improving both the reliability and efficiency of quality prediction.

[0009] [5] The quality prediction program according to any one of [1] to [4] above, further causing the computer to execute an image correction process for correcting the captured image before the prediction process when the captured image is determined in the image evaluation process to be unsuitable for prediction by the prediction model. This program is capable of correcting a captured image that is determined to be inappropriate so as to be suitable for prediction by a prediction model, which is useful for improving both the reliability and efficiency of quality prediction.

[0010] [6] A quality prediction method comprising: an image acquisition step of acquiring captured image data obtained by imaging an aggregate to be evaluated; an image evaluation step of evaluating the captured image related to the captured image data; and a prediction step of inputting evaluation input information based on the captured image data after the image evaluation step, into a prediction model constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for training and a ground truth value of the coarse grain ratio corresponding to the input information, so as to output a value indicating the coarse grain ratio in response to the input of the input information, and acquiring a predicted value to be output from the prediction model, wherein in the image evaluation step, it is evaluated whether the captured image is suitable for prediction by the prediction model. This quality prediction method, like the program described in [1] above, is useful for improving the reliability of quality prediction.

[0011] [7] A quality prediction method comprising: an image acquisition step of acquiring captured image data obtained by imaging an aggregate to be evaluated; an image evaluation step of evaluating the captured image related to the captured image data; and a prediction step of inputting evaluation input information based on the captured image data after the image evaluation step, to obtain predicted values ​​output from the prediction model for each of the multiple particle size classifications, the prediction model being constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for training and ground truth values ​​of mass fractions for each of a plurality of particle size classifications of the aggregate corresponding to the input information, so as to output a value indicating a mass fraction for each classification included in the multiple particle size classifications in response to input of the input information, wherein in the image evaluation step, it is evaluated whether the captured image is suitable for prediction by the prediction model. This quality prediction method, like the program described in [2] above, is useful for improving the reliability of quality prediction.

[0012] [8] A quality prediction device comprising: an image data acquisition unit that acquires captured image data obtained by imaging an aggregate to be evaluated; an image evaluation unit that evaluates the captured image related to the captured image data; and a prediction calculation unit that inputs evaluation input information based on the captured image data after evaluation by the image evaluation unit into a prediction model constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for training and a ground truth value of the coarse grain ratio corresponding to the input information, and acquires a predicted value output from the prediction model, wherein the image evaluation unit evaluates whether the captured image is suitable for prediction by the prediction model. This quality prediction device, like the program described in [1] above, is useful for improving the reliability of quality prediction.

[0013] [9] A quality prediction device comprising: an image data acquisition unit that acquires captured image data obtained by imaging an aggregate to be evaluated; an image evaluation unit that evaluates the captured image related to the captured image data; and a prediction calculation unit that executes a prediction step in which, after evaluation by the image evaluation unit, evaluation input information based on the captured image data is input to a prediction model constructed by machine learning based on input information based on training image data obtained by imaging an aggregate for training and ground truth values ​​of mass fractions for each of a plurality of particle size classifications of the aggregate corresponding to the input information, and acquires predicted values ​​to be output from the prediction model for each classification, wherein the image evaluation unit evaluates whether the captured image is suitable for prediction by the prediction model. This quality prediction device, like the program described in [2] above, is useful for improving the reliability of quality prediction. Effect of the Invention

[0014] According to the present disclosure, a quality prediction program, a quality prediction method, and a quality prediction device are provided that are useful for improving the reliability of quality prediction. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a schematic diagram illustrating a ready-mix concrete manufacturing system and a quality prediction device. [Diagram 2] FIG. 2 is a block diagram illustrating an example of a hardware configuration of the quality prediction device. [Diagram 3] FIG. 3 is a block diagram illustrating an example of a functional configuration of the quality prediction device. [Figure 4] Fig. 4(a) is a schematic diagram showing an example of a predictive calculation using a predictive model, and Fig. 4(b) is a cross-sectional view showing a schematic diagram of acquiring image data for learning. [Diagram 5] FIG. 5 is a diagram showing an example of image data for learning. [Figure 6] 6(a) and 6(b) are graphs showing an example of the comparison result between the predicted value and the correct value. [Figure 7] FIG. 7 is a flow diagram showing an example of a series of processes executed by the quality prediction device. [Figure 8] 8(a) and 8(b) are schematic diagrams showing an example of a display screen on a monitor. [Figure 9] 9(a) and 9(b) are schematic diagrams showing an example of a display screen on a monitor. [Figure 10] 10(a) and 10(b) are schematic diagrams showing an example of a display screen on a monitor. [Figure 11] FIG. 11 is a schematic diagram showing an example of a display screen on a monitor. [Figure 12] FIG. 12 is a flow diagram showing an example of a series of processes executed by the quality prediction device. [Figure 13] FIG. 13 is a schematic diagram showing an example of a display screen on a monitor. [Figure 14] FIG. 14 is a flow diagram showing an example of a series of processes executed by the quality prediction device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, an embodiment will be described with reference to the drawings. In the description, the same elements or elements having the same functions are given the same reference numerals, and duplicated descriptions will be omitted. Fig. 1 shows a schematic diagram of a quality prediction device according to an embodiment, and a ready-mix concrete manufacturing system in which the quality prediction device is used.

[0017] [Ready-mix concrete manufacturing system] First, an overview of a ready-mix concrete manufacturing system will be described. The manufacturing system 100 shown in Fig. 1 is a system that manufactures ready-mix concrete. The manufacturing system 100 kneads and mixes concrete materials to manufacture (produce) ready-mix concrete.

[0018] The concrete materials used in the manufacturing system 100 include cement, admixtures, coarse aggregates, fine aggregates, water, and admixtures. Examples of the coarse aggregates include gravel, crushed stone, slag coarse aggregates, lightweight coarse aggregates, recycled coarse aggregates, reclaimed aggregates, and coarse aggregates made by mixing these. Examples of the gravel include mountain gravel, land gravel, river gravel, and sea gravel. Examples of the slag coarse aggregates include blast furnace slag aggregates, ferronickel slag aggregates, electric furnace oxidized slag aggregates, and coal gasification slag aggregates. Examples of the lightweight coarse aggregates include natural lightweight aggregates, by-product lightweight aggregates, and artificial lightweight aggregates. The coarse aggregates may include crushed rock, or limestone crushed stone.

[0019] Examples of fine aggregates include sand, crushed sand, slag fine aggregate, lightweight fine aggregate, recycled fine aggregate, recovered aggregate, and fine aggregates made by mixing these. Sand includes mountain sand, land sand, river sand, sea sand, etc. Slag fine aggregates include blast furnace slag aggregate, ferro-nickel slag aggregate, copper slag aggregate, electric furnace oxidized slag aggregate, coal gasification slag aggregate, etc. Lightweight fine aggregates include natural lightweight aggregate, by-product lightweight aggregate, artificial lightweight aggregate, etc.

[0020] Examples of rock types of crushed stone and crushed sand include igneous rocks, sedimentary rocks, metamorphic rocks, quartzite, limestone, domalite, and peridotite. Igneous rocks include granite, diorite, gabbro, porphyrite, diabase, rhyolite, andesite, basalt, and serpentine. Sedimentary rocks include conglomerate, sandstone, shale, slate, and tuff. Metamorphic rocks include gneiss and schist. In the present disclosure, coarse aggregate and fine aggregate may be collectively referred to as "aggregate". In this case, "aggregate" refers to coarse aggregate, fine aggregate, or both coarse aggregate and fine aggregate.

[0021] The manufacturing system 100 loads the manufactured ready-mix concrete onto a transport vehicle 200. After the ready-mix concrete is loaded, the transport vehicle 200 transports the ready-mix concrete to a site (e.g., a construction site) where the ready-mix concrete is to be used. Examples of the transport vehicle 200 include an agitator vehicle (mixer vehicle) or a dump truck. The manufacturing system 100 may manufacture ready-mix concrete from concrete materials so as to satisfy a target quality (required quality) set for each site. In one example, the target quality of the ready-mix concrete includes a target value for slump or slump flow. The manufacturing system 100 includes, for example, a material storage yard 101, a transport device 104, and a manufacturing device 110.

[0022] The material yard 101 is a place where concrete materials are stored. The material yard 101 includes a plurality of silos 102. The plurality of silos 102 are containers that store at least a portion of the concrete materials by material type. The plurality of silos 102 include a silo 102 that stores coarse aggregate, a silo 102 that stores fine aggregate, and a silo 102 that stores cement.

[0023] The transporting device 104 is a device that transports the concrete materials stored in the multiple silos 102 to the manufacturing device 110. The transporting device 104 includes, for example, a belt conveyor that transports the concrete materials. The transporting device 104 may transport the concrete materials by type at different times. In one example, based on an operation instruction from a control device included in the manufacturing system 100, a specific material from among the various concrete materials is transferred to the transporting device 104 and transported to the manufacturing device 110.

[0024] The manufacturing apparatus 110 is an apparatus for manufacturing ready-mixed concrete by mixing concrete materials including aggregate. The manufacturing apparatus 110 includes, for example, a storage bottle 111, a measuring bottle 112, a collecting hopper 113, a mixer 114, and a loading hopper 115.

[0025] Storage bottle 111 temporarily stores various types of concrete materials. Various types of concrete materials are transported (conveyed) to storage bottle 111 from material storage yard 101 by transport device 104. Storage bottle 111 is configured to store various types of concrete materials individually. Hereinafter, "concrete material" may be simply referred to as "material". Various materials stored in storage bottle 111 are supplied to measuring bottle 112 as necessary.

[0026] The measuring bottle 112 is disposed below the storage bottle 111. The measuring bottle 112 operates based on an operation command from the control device, and weighs each material individually. When the measuring bottle 112 detects a target amount of material instructed by the control device, it supplies the material to the collecting hopper 113. When water is supplied to the measuring bottle 112, an admixture may be mixed into the water. The collecting hopper 113 is disposed below the measuring bottle 112. The collecting hopper 113 collects each material discharged from the measuring bottle 112, and supplies the collected materials to the mixer 114. Note that the collecting hopper 113 does not have to be provided, and in this case, each material is supplied from the measuring bottle 112 to the mixer 114.

[0027] The mixer 114 is disposed below the collecting hopper 113. The mixer 114 is a device that mixes concrete materials. The mixer 114 produces ready-mixed concrete by mixing (kneading) aggregate, cement, water, admixtures, and the like. The ready-mixed concrete is discharged from the bottom of the mixer 114 into the loading hopper 115. The loading hopper 115 is disposed below the mixer 114, and temporarily stores the ready-mixed concrete. The loading hopper 115 supplies the temporarily stored ready-mixed concrete to the transport vehicle 200.

[0028] The manufacturing system 100 described above is one example of a ready-mix concrete manufacturing system, and the ready-mix concrete manufacturing system may be configured in any manner as long as it is capable of mixing concrete materials and manufacturing ready-mix concrete.

[0029] [Quality prediction device] The quality prediction device 1 is a device that predicts the coarse aggregate ratio as the quality of aggregate contained in concrete material. The quality prediction device 1 is configured with one or more computers. The computers that configure the quality prediction device 1 may be a personal computer, a tablet computer (tablet terminal), a smartphone, a wearable device, a workstation, a server computer, or a general-purpose computer. The quality prediction device 1 may be connected to a communication network via a wired or wireless connection.

[0030] The quality prediction device 1 includes, for example, a main body 10, an input device 12, a monitor 14, and a camera 16. The main body 10 is a device (computer main body) that executes the main functions of the computer that constitutes the quality prediction device 1.

[0031] The input device 12 is a device for inputting information to the main body 10. The input device 12 may be any device capable of inputting desired information, and specific examples thereof include operation interfaces such as a keypad, a mouse, and an operation controller.

[0032] The monitor 14 is a device for displaying information output from the main body 10. The monitor 14 may be any device capable of displaying graphics, a specific example of which is a liquid crystal panel. The monitor 14 and the input device 12 may be integrated as a touch panel.

[0033] The camera 16 is a device capable of capturing an image of the aggregate. The camera 16 captures an image of the aggregate to generate image data. The camera 16 is, for example, a digital camera that captures an image of the imaging range based on visible light, or a video camera. The image data generated by the camera 16 is used to predict the quality of the aggregate. When the camera 16 generates a video of the imaging target, still images included in the video may be acquired as image data.

[0034] The camera 16 may be disposed in the manufacturing system 100 so as to be able to capture an image of an aggregate at a specific location. For example, the camera 16 may be disposed so as to be able to capture an image of an aggregate being transported by the transport device 104. The camera 16 may be portable, and a user such as a worker may carry the camera 16 to an imaging position. The camera 16 may be mounted on a mobile body such as a drone, and the camera 16 may capture an image at any location in the manufacturing system 100 by a user operating the mobile body, etc.

[0035] In the quality prediction device 1, the main body 10, the input device 12, the monitor 14, and the camera 16 may be integrated into one device, such as a smartphone with a built-in (mounted) camera or a tablet computer with a built-in camera. In one example, the quality prediction device 1 is configured by installing an application for predicting the quality of aggregate on a smartphone.

[0036] As shown in FIG. 2, the main body 10 includes a circuit 50. The circuit 50 includes a processor 51, a memory 52, a storage 53, and an input / output port 54. The storage 53 is configured with one or more non-volatile memory devices such as a flash memory or a hard disk. The storage 53 stores a quality prediction program that causes a computer to execute at least an image acquisition step, an image evaluation step, and a prediction step. The quality prediction program may further cause a computer to execute at least one of a warning display step, an image display step, and an image correction step, a specification step, an evaluation information acquisition step, and a result display step, in addition to the image acquisition step, the image evaluation step, and the prediction step. Details of these steps will be described later. The storage 53 stores a quality prediction program for configuring each functional block of the main body 10.

[0037] The memory 52 is composed of one or more volatile memory devices such as a random access memory. The memory 52 temporarily stores a quality prediction program loaded from the storage 53. The processor 51 is composed of one or more arithmetic devices such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The processor 51 configures each functional block of the main body 10 by executing the quality prediction program loaded into the memory 52. ​​The calculation results by the processor 51 are temporarily stored in the memory 52. ​​The input / output port 54 inputs and outputs information between the input device 12, the monitor 14, the camera 16, etc. in response to a request from the processor 51.

[0038] The quality prediction program may be provided in a state of being fixedly recorded on a tangible recording medium such as a CD-ROM, a DVD-ROM, or a semiconductor memory. Alternatively, the quality prediction program may be provided via a communication network as a data signal superimposed on a carrier wave. The main body 10 (quality prediction device 1) may be composed of a plurality of computers communicably connected to each other. The main body 10 (quality prediction device 1) may be a part of a control device included in the manufacturing system 100.

[0039] 3 shows an example of functional components (referred to as "functional blocks" in this disclosure) included in the main body 10. The main body 10 includes, as functional blocks, a user input acquisition unit 22, an image data acquisition unit 24, an image evaluation unit 26, a prediction model storage unit 28, a prediction calculation unit 30, a display output unit 32, and an aggregate information storage unit 34. The processes executed by these functional blocks correspond to the processes executed by the main body 10 (quality prediction device 1).

[0040] The user input acquisition unit 22 is configured to acquire user input information indicating an input from a user required for predicting the coarse grain ratio of the aggregate to be evaluated. The user input acquisition unit 22 may acquire the user input information based on a user input to an interface screen displayed on the monitor 14.

[0041] The image data acquiring unit 24 is configured to acquire image data (hereinafter referred to as "captured image data") generated by capturing an image of the aggregate to be evaluated. The image data acquiring unit 24 acquires captured image data (captured image) captured by the camera 16. The image evaluating unit 26 is configured to evaluate the captured image related to the captured image data. The image evaluating unit 26 evaluates whether or not the captured image is suitable for prediction by a prediction model M described later.

[0042] The prediction model holding unit 28 is configured to hold a model for predicting the coarse grain ratio of the aggregate to be evaluated (hereinafter, referred to as "prediction model M"). The prediction model M is a model constructed by machine learning based on input information based on learning image data (learning image data) obtained by imaging the learning aggregate and a ground truth value of a quality value indicating the quality of the aggregate associated with the input information, so as to output a predicted value of the quality value in response to input of the input information. The quality value is, for example, the coarse grain ratio, the mass fraction for each grain size division included in the grain size division of the aggregate, or both the coarse grain ratio and the mass fraction for each grain size division. The construction of the prediction model M will be described later.

[0043] The prediction model holding unit 28 may hold (store) a plurality of types of prediction models M. The plurality of types of prediction models M held by the prediction model holding unit 28 may be associated with a plurality of types of aggregate, respectively. Each of the plurality of types of prediction models M may be constructed by machine learning based on learning data obtained from the corresponding type of aggregate. For example, when the type of aggregate to be evaluated is "crushed stone 2005", the quality value may be predicted using the prediction model M associated with the crushed stone 2005 among the plurality of types of prediction models M.

[0044] Machine learning refers to a method in which a machine (computer) autonomously finds laws or rules by repeatedly learning based on given information. The prediction model M can be constructed using an algorithm and a data structure. The prediction model M is realized, for example, using a neural network, which is an information processing model that mimics the mechanism of the human brain. There is no particular limitation on the specific algorithm of the machine learning performed when constructing the prediction model M. The neural network has an input layer, one or more intermediate layers, and an output layer. By including one or more intermediate layers, a more complex prediction model M can be constructed, and prediction accuracy can be improved.

[0045] The prediction model M may be constructed by machine learning using a convolutional neural network (CNN). The prediction model M is constructed by machine learning based on the correct value of the quality value. The prediction model M may output only the coarse grain ratio, only a value indicating the mass fraction for each grain size division included in the multiple grain size divisions, or both the coarse grain ratio and the mass fraction for each grain size division, as the quality value, according to the input information. When outputting the mass fraction for each grain size division, the prediction model M outputs multiple values ​​each indicating a mass fraction for the multiple grain size divisions. The following illustrates an example in which the prediction model M outputs both the coarse grain ratio and the mass fraction for each grain size division.

[0046] The prediction model M is constructed to predict the above quality value, for example, by performing machine learning using data provided as input for machine learning and correct answer data (correct answer value of the above quality value) of the machine learning output. The input for the machine learning is various data sets of input information based on learning image data obtained by imaging the learning aggregate. In the various data sets of input information, at least the image data included in the input information is different from each other. The output of the machine learning is data (numerical value) indicating a quality value such as the coarse grain ratio. When constructing the prediction model M, a model that outputs a predicted value of the above quality value is iteratively trained using multiple combinations of input information and correct answer values ​​such as the coarse grain ratio.

[0047] The stage in which the prediction model M is autonomously constructed corresponds to the learning phase. The learning phase may be performed before the production phase in which ready-mix concrete is manufactured in the manufacturing system 100, or may be performed at an early stage of the production phase. The prediction model M, which is a trained model, is portable between computers. Therefore, the prediction model M constructed in another device may be stored in the prediction model holding unit 28. The stage in which the quality prediction device 1 predicts the coarse particle ratio of the aggregate to be evaluated using the prediction model M corresponds to the evaluation phase.

[0048] The prediction calculation unit 30 is configured to predict the coarse grain ratio of the aggregate to be evaluated using the prediction model M. The aggregate to be evaluated is an aggregate whose coarse grain ratio is unknown. The prediction calculation unit 30 may include an evaluation information acquisition unit that performs an evaluation information acquisition step. The evaluation information acquisition unit may be configured separately from the prediction calculation unit 30, and the image evaluation unit 26 may function as the evaluation information acquisition unit. The evaluation information acquisition unit acquires evaluation input information based on captured image data obtained by capturing an image of the aggregate to be evaluated. The evaluation information acquisition unit may acquire, for example, the captured image data as it is as the evaluation input information, or may acquire image data obtained by performing a predetermined processing on the captured image data as the evaluation input information. Whether or not to process the captured image data in order to obtain the evaluation input information and how to process it may be determined by a user input.

[0049] After the evaluation by the image evaluation unit 26, the prediction calculation unit 30 inputs the evaluation input information to the prediction model M and obtains a predicted value output from the prediction model M. For example, the prediction calculation unit 30 obtains a predicted value indicating the coarse grain ratio from the prediction model M. Also, after the evaluation by the image evaluation unit 26, the prediction calculation unit 30 may obtain a plurality of predicted values ​​each indicating a mass fraction value for a plurality of grain size divisions from the prediction model M. The prediction calculation unit 30 may calculate the coarse grain ratio from a predicted value (a plurality of predicted values) regarding the mass fraction for each grain size division. Note that, for reliability evaluation, the prediction calculation unit 30 may obtain both the coarse grain ratio and a plurality of predicted values ​​each indicating a mass fraction value from the prediction model M and compare the coarse grain ratio predicted for each prediction model M. As described above, the calculation (prediction) of the coarse grain ratio is performed by two different methods using the prediction model M, so that the reliability of the predicted value of the coarse grain ratio can be evaluated.

[0050] The display output unit 32 is configured to display the prediction result by the prediction calculation unit 30 on the monitor 14. The display output unit 32 may display the prediction result on the monitor 14 in any format, but for example, at least displays the calculation result (prediction result) of the coarse grain rate using the prediction model M on the monitor 14. The display output unit 32 may display various information on the monitor 14 to present to the user or to receive operations by the user when prediction is performed by the prediction calculation unit 30.

[0051] The aggregate information storage unit 34 is configured to store information necessary for the evaluation by the image evaluation unit 26. The aggregate information storage unit 34 stores information necessary for the evaluation (e.g., judgment thresholds, etc.) that is associated with each of the multiple types of aggregate. The aggregate information storage unit 34 may store corresponding information (information necessary for evaluation) for each of the multiple types of aggregate. The information stored by the aggregate information storage unit 34 is set in advance for each type of aggregate by, for example, a user or a provider of the quality prediction device 1.

[0052] <Prediction model M> Here, the prediction model M used to predict the coarse grain ratio will be further described with reference to Figs. 4 and 5. In the learning phase for constructing the prediction model M, aggregate of the same type as the aggregate that is used in the manufacturing system 100 and can be evaluated is used as the learning aggregate. Fig. 4(a) shows a schematic diagram of the calculation process for predicting the coarse grain ratio using the prediction model M from the image Im, which is input information. The input of the prediction model M is the image Im, and the output of the prediction model M is the predicted value of the mass fraction of each of the multiple grain size classes and the predicted value of the coarse grain ratio.

[0053] When constructing the prediction model M, learning data for machine learning is prepared. The learning data includes various image data (input information) obtained by imaging the learning aggregate, and answer data associated with each of the various image data. In preparing the learning images, for example, various aggregates whose mass fractions for each granularity classification are known are prepared. The above various aggregates do not mean different types of aggregate, but mean the same type of aggregates with different granularities.

[0054] In one example, various types of learning aggregates are individually imaged by camera 130 in a stacked state on measurement sheet 132 (see FIG. 4(b)). A stacked state (piled state) refers to a state in which some particles contained in the aggregate overlap other particles when viewed vertically from above. By individually imaging the various types of aggregates by camera 130, learning image data is obtained for each type of aggregate. Camera 130 may be a digital camera, and images an area including the stacked aggregates from a position a predetermined distance away from the aggregates on measurement sheet 132 (for example, a predetermined position above).

[0055] The sheet 132 may be a rubber sheet. As shown in FIG. 4(b), the sheet 132 may be a sheet formed to be curved in imitation of the belt conveyor of the conveying device 104, or may be a sheet formed in a flat shape, unlike the example shown in FIG. 4(b). The amount of aggregate placed on the sheet 132 is not particularly limited, but may be about 1 kg to 10 kg. FIG. 4(b) shows a schematic example of aggregate piled up on the sheet 132, and shows piled up states when the amount of aggregate is 2 kg, 5 kg, and 8 kg. Note that the sheet 132 may be a container other than a rubber sheet. If it is a container with a rim, it is preferable from the viewpoint that it can be used to uniformly adjust the size of the captured aggregate by correcting the size of the image data based on the distance calculation result after calculating the distance from the camera to the aggregate.

[0056] In one example, samples of various aggregates are prepared under the conditions shown in Table 1 below, and images for learning are obtained by photographing the prepared samples. The mass fraction and coarse aggregate ratio for each grain size division of the various aggregates prepared under the conditions shown in Table 1 are known, and the known values ​​are the correct answer data for machine learning.

[0057] [Table 1]

[0058] The coarse grain ratio (FM) is calculated, for example, by the following formula (1) specified in the Standard Method for Concrete. In the measurement specified in the "Standard Specifications for Building Construction and Commentary JASS5 Reinforced Concrete Construction", 9 sieves are used excluding the 80 mm sieve in formula (1). The coarse grain ratio predicted in this disclosure may be specified by any method.

number

[0059] In Table 1, this means that aggregates of 10 levels of gradation (coarse aggregate ratio) are prepared. Category I represents a gradation category of particles smaller than 2.5 mm. Category I includes particles that cannot be retained on a 2.5 mm sieve. Category II represents a gradation category of particles 2.5 mm or larger and smaller than 5 mm. Category II includes particles that can be retained on a 2.5 mm sieve but cannot be retained on a 5 mm sieve. Category III represents a gradation category of particles 5 mm or larger and smaller than 10 mm. Category III includes particles that can be retained on a 5 mm sieve but cannot be retained on a 10 mm sieve. Category IV represents a gradation category of particles 10 mm or larger and smaller than 20 mm. Category IV includes particles that can be retained on a 10 mm sieve but cannot be retained on a 20 mm sieve.

[0060] For level 1, the coarse aggregate ratio can be calculated from the mass fractions of each grain size category (mass fractions of each of categories I to IV) as follows. For level 1 aggregate, the mass percentages of the sample that are retained on the 80 mm, 40 mm, and 20 mm sieves are 0, the mass percentages of the sample that are retained on the 10 mm sieve are 85, and the mass percentages of the sample that are retained on the 5 mm sieve are 100 (=15+85). Also, since all the grains are retained on the 5 mm sieve, the mass percentages of the sample that are retained on the 2.5 mm, 1.2 mm, 600 μm, 300 μm, and 150 μm sieves are 100. Therefore, the coarse aggregate ratio is calculated as shown in the following formula (2).

number

[0061] For levels 2 to 10, the coarse aggregate ratio is calculated in the same manner as for level 1. The conditions of the learning aggregate shown in Table 1 are merely examples, and the number of levels, the coarse aggregate ratio and mass fraction for each grading class at each level, and the method of setting the grading classes may be set arbitrarily. As with the multiple grading classes shown in Table 1, the grading classes in multiple classes including the smallest class (<150 μm) may be combined into one grading class. Grain sizes above a certain class (e.g., 20 to 40 mm) may not be included in multiple grading classes. Also, unlike the setting method in Table 1, grain sizes below a certain class (e.g., 1.2 to 2.5 μm) may not be included in multiple grading classes.

[0062] FIG. 5 shows images T1 to T10 obtained by imaging learning aggregates prepared according to the conditions of each of levels 1 to 10 in Table 1. Level n (n is an integer from 1 to 10) corresponds to image Tn. For example, image T1 is obtained by imaging learning aggregates prepared according to level 1. Learning aggregates according to each level n contain grains of multiple grain size divisions. In this way, learning aggregates may be prepared so as to contain grains of two or more grain size divisions, and input information may be prepared based on image data obtained by imaging the learning aggregates.

[0063] For each level n, multiple images Tn in which the states (more specifically, pile states) of the learning aggregate are different from each other may be prepared. In one example, a first image is taken with the learning aggregate placed on the measurement sheet 132, to obtain one image Tn. Then, the learning aggregate is temporarily removed from the sheet 132, and the same learning aggregate (same sample) is placed again on the sheet 132 so that the pile state of the multiple grains is different from that in the first image. Then, a second image is taken. By repeating the above-mentioned image taking, multiple images Tn may be prepared for each level n.

[0064] One data set is composed of one image Tn and the correct answer data of the mass fraction and the coarse grain ratio at the level n associated with the image Tn. As learning data, a plurality of data sets each consisting of a combination of the image Tn and the correct answer data of the mass fraction and the coarse grain ratio are prepared. Then, a prediction model M is constructed by performing machine learning using such learning data. The prediction model M configured as above outputs, for example, four mass fraction values ​​and a coarse grain ratio value for four grain size classifications.

[0065] Focusing on the point of outputting a value of the coarse grain ratio, the prediction model M is a model constructed by machine learning based on input information based on learning image data obtained by imaging the learning aggregate and the correct value of the coarse grain ratio associated with the input information, so as to output a value indicating the coarse grain ratio in response to the input of the input information. Focusing on the point of outputting a value of the mass fraction for each grain size division, the prediction model M is a model constructed by machine learning based on input information based on learning image data obtained by imaging the learning aggregate and the correct value of the mass fraction for each of the multiple grain size divisions of the aggregate associated with the input information, so as to output a value indicating the mass fraction for each of the multiple grain size divisions in response to the input of the input information.

[0066] In order to verify the method of predicting the coarse aggregate ratio using the prediction model M based on machine learning using a convolutional neural network, a comparison was made between the prediction results by the prediction model M and the correct values ​​using a dataset in which the correct values ​​were known. For the learning aggregate, 10 levels of aggregate with different coarse aggregate ratios were prepared according to the conditions in Table 1 above. For each level, images were taken 10 times indoors with the aggregate piled up on the measurement sheet 132 in different states, to obtain 10 images.

[0067] In addition, a total of 100 images (10 levels x 10 shots) were prepared according to the standard conditions shown in Table 2 below. As a result, 100 data sets were prepared as learning data, each of which corresponds to an image and a correct answer value.

[0068] [Table 2]

[0069] The prediction model M was constructed using 80 of the 100 datasets, and 20 datasets were used for validation. At each level, 8 data were used for model construction, and 2 datasets were used for validation. In the validation using 20 datasets, the prediction results calculated using the image and the prediction model M for each dataset were compared with the correct values ​​such as the coarse grain rate associated with the image. The prediction model M was constructed using a convolutional neural network with 6 intermediate layers consisting of a convolutional layer and a pooling layer, followed by 4 fully connected layers. The Huber function was used as the loss function. In each dataset, the number of images was increased by the Image Augmentation layer.

[0070] FIG. 6(a) is a graph showing the relationship between the predicted values ​​of the mass ratio (%) in the divisions III and IV in Table 1 and the corresponding correct values ​​of the mass fraction (%). In the graph shown in FIG. 6(a), the horizontal axis is the correct value of the mass fraction, and the vertical axis is the predicted value of the mass fraction output from the prediction model M. The prediction results in division III (5 mm to 10 mm) are plotted with △ marks, and the prediction results in division IV (10 mm to 20 mm) are plotted with ◯ marks. A solid line where the correct value and the predicted value match is drawn, and the closer the plotted results are to the line, the higher the prediction accuracy. From the results shown in FIG. 6(a), the prediction results are plotted near the line, and it can be seen that the mass ratio can be predicted with high accuracy by the prediction model M. Note that the coarse particle ratio can be calculated from the mass fraction, so if the mass fraction can be predicted with high accuracy by the prediction model M, the coarse particle ratio can also be predicted with high accuracy.

[0071] FIG. 6(b) is a graph showing the relationship between the predicted value of the coarse grain ratio output from the prediction model M and the correct value of the coarse grain ratio. In the graph shown in FIG. 6(b), the horizontal axis is the correct value of the coarse grain ratio, and the vertical axis is the predicted value of the coarse grain ratio output from the prediction model M. The prediction results are plotted with square marks. The solid line is the line where the correct value and the predicted value match, and a dashed upper limit line shifted vertically by +0.10 from the solid line and a lower limit line shifted vertically by -0.10 are drawn. The prediction results included in the area between the upper limit line and the lower limit line are defined as being within the allowable range. With reference to the error in the coarse grain ratio allowed in current ready-mix concrete plants, a range of ±0.10 was set as the allowable range. It can be seen that all of the prediction results using the prediction model M are included in the allowable range, and the prediction model M can predict the coarse grain ratio with high accuracy.

[0072] [Quality prediction method] Next, a quality prediction method executed in the quality prediction device 1 will be described with reference to Figs. 7 to 11. Hereinafter, the quality prediction method will be described using an example in which the quality prediction device 1 is configured by a smartphone with a camera. This quality prediction method includes, for example, a specification step, an image acquisition step, an image evaluation step, an evaluation information acquisition step, a prediction step, a result display step, and a warning display step. The specification step, the image acquisition step, the evaluation information acquisition step, the image evaluation step, the prediction step, the result display step, and the warning display step are all steps executed in the evaluation phase. That is, these steps are executed after the prediction model M has already been constructed. These steps may be executed by corresponding functional blocks in the quality prediction device 1. In the learning phase, a plurality of prediction models M may be constructed according to a plurality of types of aggregate (a plurality of aggregate types).

[0073] The identification step is a step of identifying the type of aggregate to be evaluated from among a plurality of aggregate types based on a user input indicating the type of aggregate to be evaluated. The image acquisition step is a step of acquiring captured image data obtained by capturing an image of the aggregate to be evaluated by the camera 16.

[0074] The image evaluation process is a process of evaluating whether or not a captured image related to the captured image data is suitable for prediction by the prediction model M. The prediction model M is constructed by machine learning based on image data obtained by imaging an aggregate sample under certain conditions in the learning phase. Therefore, for example, if there is a large deviation between the various conditions and / or various states when imaging the aggregate to be evaluated in the evaluation phase and the various conditions and / or various states in the learning phase, the accuracy of the prediction result by the prediction model M may decrease. In the image evaluation process, for example, if it is determined that the accuracy of the prediction result by the prediction model M may decrease when a certain captured image is input, the captured image is evaluated as not suitable for prediction by the prediction model M.

[0075] Examples of criteria (evaluation indexes) for determining whether a captured image is suitable in the image evaluation process include the following. In one example, the suitability of a captured image is evaluated by comparing at least one of the evaluation indexes exemplified below with the settings in the learning phase. For at least a part of these evaluation indexes, a model that outputs an evaluation index or a judgment result of suitability according to the evaluation index in response to an input of a captured image may be constructed by machine learning, separate from the prediction model M. - Shooting environment detected from pixel values ​​of captured images (e.g., ambient brightness) -Color difference of aggregate detected from pixel values ​​of captured images - Unmeasurable color (for example, if a prediction model M is built using dry aggregate and wet aggregate is used as the evaluation target, an unmeasurable color may be detected.) The number of aggregate particles captured in the captured image (specifically, to detect whether the particle count is abnormally high or low) - The degree of focus and / or blur of the captured image · Detection of abnormal objects (foreign objects other than aggregates, or types of aggregates other than the aggregate being evaluated) through image recognition The position of a part of the component that supports the aggregate to be evaluated (e.g., a rubber sheet, a container frame, or a conveyor belt frame) in the captured image The pile-up state of aggregates in the captured image (for example, detecting the outer edge of the aggregate mass and evaluating whether the outer edge is within a standard range or not, or evaluating the deviation of the center position of the aggregate mass from the center of the image). Imaging distance (for example, information indicating the distance between the camera position and the aggregate being photographed)

[0076] The evaluation information acquisition step is a step of acquiring evaluation input information based on captured image data relating to a captured image obtained by capturing an image of the aggregate to be evaluated. For example, when the captured image is determined to be suitable for prediction by the prediction model M in the image evaluation step, the evaluation information acquisition step is executed. In one example, in the image evaluation step, one or more evaluation indexes among the above-mentioned various evaluation indexes are used, and when the (those) evaluation indexes are within a range of a set value in the learning phase ± an allowable target value, the captured image is determined to be suitable for prediction by the prediction model M, and the evaluation input information is acquired.

[0077] The prediction step is a step of inputting evaluation input information to the prediction model M after the image evaluation step, and acquiring a predicted value of the coarse aggregate ratio from the prediction model M. Also, in the prediction step, after the image evaluation step, evaluation input information is input to the prediction model M, and a predicted value of the mass fraction output from the prediction model M is acquired for each granularity division. Then, the coarse aggregate ratio of the aggregate to be evaluated is calculated from the predicted value of the mass fraction for each granularity division. When calculating the coarse aggregate ratio from the predicted value of the mass fraction for each granularity division, the coarse aggregate ratio is calculated, for example, according to the above formula (1). In the prediction step, a prediction model M according to the result of identifying the type of aggregate to be evaluated in the identifying step may be selected from among the multiple prediction models M, and a predicted value of the coarse aggregate ratio, etc. may be acquired using the selected prediction model M.

[0078] The result display step is a step of displaying the prediction result of the coarse aggregate ratio in the prediction step on the monitor 14. In the result display step, two types of predicted values ​​regarding the coarse aggregate ratio obtained in the prediction step and information indicating the aggregate type identified in the identification step may be displayed on the monitor 14.

[0079] The warning display step is a step of displaying a warning to the user that the captured image is unsuitable for prediction by the prediction model M on the monitor 14 before the prediction step when the captured image is determined to be unsuitable for prediction by the prediction model M in the image evaluation step. The warning display step is not executed unless the captured image is determined to be unsuitable for prediction by the prediction model M in the image evaluation step. In one example, in the image evaluation step, one or more evaluation indexes among the above-mentioned various evaluation indexes are used, and when the (those) evaluation indexes fall outside the range of the set value in the learning phase ± the allowable target value, the captured image is determined to be unsuitable for prediction by the prediction model M, and the warning display step is acquired. When the warning display step is executed, for example, the user redoes acquisition of the captured image.

[0080] <Processing flow> FIG. 7 is a flow diagram illustrating a series of processes executed in the quality prediction method. A user holds the quality prediction device 1 and executes step S01 at any timing when the user wants to predict the coarse aggregate ratio of the aggregate to be evaluated. In step S01, for example, the user starts an application for predicting quality by operating on a screen displayed on the monitor 14. In one example, as shown in FIG. 8(a), in a state where a home screen is displayed on the monitor 14 of the quality prediction device 1, the user clicks, taps, or presses an icon 60 representing an application for predicting quality.

[0081] When the quality prediction device 1 receives the start request of step S01, it executes step S02. In step S02, for example, the display output unit 32 of the quality prediction device 1 displays a home screen of an application that executes quality prediction on the monitor 14. An example of the home screen is shown in Fig. 8(b), and on the home screen, an execute button 62 for switching to a mode for photographing (taking an image of) an aggregate and an execute button 64 for switching to a mode for executing another function (for example, a mode for viewing past prediction results) are displayed.

[0082] Next, the user executes step S03. In step S03, for example, the user presses (e.g., taps) the execute button 62 displayed on the monitor 14 to select a mode for photographing the aggregate. When the selection of step S03 is made, the quality prediction device 1 executes step S04. In step S04, for example, the display output unit 32 switches the display content on the monitor 14 to a screen corresponding to the mode for photographing the aggregate. FIG. 9(a) shows an example of the screen after switching in step S04.

[0083] Next, the user executes step S05. In step S05, for example, the user operates monitor 14 to select a type corresponding to the aggregate to be evaluated from among a plurality of aggregate types. As also shown in FIG. 9(b), display output unit 32 may display on monitor 14 a pull-down menu 66 from which the user can select the type of aggregate to be evaluated. In pull-down menu 66, the user may select one that matches the type of aggregate to be evaluated from a plurality of aggregate types previously determined. FIG. 10(a) shows an example of a display screen after the type of aggregate to be evaluated has been selected.

[0084] When the selection of step S05 is performed, the quality prediction device 1 executes step S06. In step S06, for example, the prediction calculation unit 30 and the display output unit 32 identify the type of aggregate to be evaluated according to the selection result of step S05. After the execution of steps S05 and S06, the display output unit 32 displays a start button 68 for inputting an instruction to start the camera 16. Before the execution of steps S05 and S06, the display output unit 32 may display the start button 68 on the monitor 14 in a state in which the user cannot tap it. In FIG. 9(a) and the like, the button shown by the dashed line represents a state in which the user cannot tap it. Alternatively, the display output unit 32 may not display the start button 68 on the monitor 14 before the execution of steps S05 and S06, and may display the start button 68 on the monitor 14 after the execution of steps S05 and S06.

[0085] Next, the user executes step S07. In step S07, for example, the user instructs the camera 16 to be started by operating the screen on the monitor 14. When the quality prediction device 1 receives the instruction to start in step S07, it executes step S08. In step S08, the display output unit 32 displays a preview image 70 by the camera 16 on the monitor 14. At the time of executing step S08, as shown in FIG. 10(b), the display output unit 32 may display an execution button 76 for instructing the execution of shooting by the camera 16 on the monitor 14 in addition to the preview image 70 by the camera 16. Note that the execution button 76 for instructing the execution of prediction using the prediction model M may be displayed on the monitor 14, and the execution button 76 for executing shooting by the camera 16 and executing prediction using the prediction model M by pressing the execution button 76 may be displayed on the monitor 14.

[0086] Next, the user executes step S09. In step S09, for example, the user operates the screen on the monitor 14 to instruct the camera 16 to take an image and to execute a prediction using the prediction model M. In one example, the user taps the execute button 76 while looking at the preview image 70 on the monitor 14, whereby the camera 16 takes an image of the aggregate to be evaluated, and then the process executed by the quality prediction device 1 proceeds to the next step.

[0087] Upon receiving the instruction in step S09, the quality prediction device 1 executes step S11. In step S11, for example, the image evaluation unit 26 evaluates whether or not the captured image captured by the camera 16 in response to a user instruction including the imaging instruction in step S09 is suitable for prediction by the prediction model M. The image evaluation unit 26 may use one or more evaluation indexes among the various evaluation indexes described above to determine whether or not the (those) evaluation indexes are within a range of a set value in the learning phase ± an allowable target value.

[0088] 11 illustrates an example of a state in which a captured image 74 obtained by the camera 16 is displayed on the monitor 14, in which the position of the frame of the container supporting the aggregate or the position of the outer edge of the aggregate mass in the image significantly deviates from the settings in the learning phase in the captured image 74. In such a case, the image evaluation unit 26 may determine that the captured image 74 is not suitable for prediction by the prediction model M.

[0089] If it is determined in step S11 that the captured image 74 is unsuitable for prediction by the prediction model M, the quality prediction device 1 executes step S12. In step S12, for example, the display output unit 32 causes the monitor 14 to display warning information 72 notifying the user that the captured image 74 is unsuitable. The warning information 72 may be information in the form of a message as shown in Fig. 11, or may be information in which a figure and a message are combined. In this case, the display output unit 32 may cause the monitor 14 to display an execution button 69 for re-executing shooting by the camera 16.

[0090] By executing step S12, the user can understand that the captured image 74 is not suitable for prediction by the prediction model M. When step S11 is executed, for example, the user taps the execution button 69 on the monitor 14. In this case, the execution content of the processing flow returns to step S07. When it is not determined in step S11 that the captured image 74 is not suitable for prediction by the prediction model M, the execution of step S12 is omitted.

[0091] Next, the quality prediction device 1 executes step S21. In step S21, for example, the prediction calculation unit 30 inputs evaluation input information based on the captured image 74 that is not determined to be unsuitable for prediction by the prediction model M to the prediction model M, and obtains predicted values ​​of the coarse aggregate ratio and the mass fraction for each gradation class output from the prediction model M. The prediction calculation unit 30 may obtain predicted values ​​of the coarse aggregate ratio and the mass fraction for each gradation class using the prediction model M associated with the type of aggregate to be evaluated that was identified in step S06. The prediction calculation unit 30 calculates the coarse aggregate ratio from the mass fraction for each gradation class.

[0092] Next, the quality prediction device 1 executes step S22. In step S22, for example, the display output unit 32 displays the result of the prediction calculation executed in step S21 on the monitor 14. In one example, the display output unit 32 causes the monitor 14 to display information indicating the identified aggregate type, information indicating the coarse aggregate ratio directly obtained from the prediction model M, and information indicating the coarse aggregate ratio calculated from the mass fraction obtained from the prediction model M.

[0093] [Variations] The process flow shown in Fig. 7 is an example and can be modified as appropriate. When there is one type of aggregate whose quality is to be predicted by the quality prediction device 1, steps S05 and S06 may be omitted. The quality prediction device 1 may execute the process flow shown in Fig. 12 instead of the process flow shown in Fig. 7. The process flow shown in Fig. 12 differs from the process flow shown in Fig. 7 in that steps S13 and S14 are executed instead of steps S11 and S12.

[0094] 12 is executed, an image display step is executed in place of the warning display step in the quality prediction method. The image display step is a step of displaying a captured image on a monitor before the prediction step when the captured image is determined in the image evaluation step to be unsuitable for prediction by the prediction model M. In this case, the user can determine whether or not to execute prediction by the prediction model M using such a captured image while checking the contents of the captured image.

[0095] 12, when it is determined in step S11 that the captured image 74 is not suitable for prediction using the prediction model M, the quality prediction device 1 executes step S13. In step S13, for example, the display output unit 32 displays the captured image 74 that is the subject of the determination on the monitor 14 as shown in FIG. 13. At this time, the display output unit 32 may display auxiliary information 72A indicating the result by the prediction model M and guidance to the user on the monitor 14. In addition to the auxiliary information 72A and the captured image 74, the display output unit 32 may display an execute button 69 for re-shooting and an execute button 76 for continuing prediction using the prediction model M on the monitor 14.

[0096] For example, when the user looks at the contents of the captured image 74 and determines that prediction using the prediction model M should not be continued, the user taps the execute button 69 on the monitor 14. In this case, the process flow returns to step S07. When the user looks at the contents of the captured image 74 and determines that there is no problem in continuing prediction using the prediction model M, the user executes step S14. In step S14, for example, the user taps the execute button 69 to instruct the quality prediction device 1 to continue prediction using the prediction model M. Note that, if the captured image 74 is not determined to be inappropriate in step S11, steps S13 and S14 are not executed.

[0097] The quality prediction device 1 may execute a process flow shown in Fig. 14 instead of the process flow shown in Fig. 7. The process flow shown in Fig. 14 differs from the process flow shown in Fig. 7 in that step S16 is executed instead of steps S11 and S12.

[0098] When the process flow shown in Fig. 14 is executed, an image correction step is executed in the quality prediction method instead of the warning display step. The image correction step is a step of correcting the captured image before the prediction step when the captured image is determined in the image evaluation step to be unsuitable for prediction by the prediction model M. In the image correction step, the captured image may be corrected so as to approach various conditions and various states when the image used in the learning phase was obtained. The image correction step may be executed autonomously by the quality prediction device 1, or may be executed by the quality prediction device 1 based on a user input.

[0099] In one example, the captured image is corrected so that one or more of the various evaluation indices described above approach the set values ​​in the learning phase. For example, if the imaging distance is greater than the set value in the learning phase, the captured image may be enlarged, and if the imaging distance is less than the set value in the learning phase, the captured image may be reduced. For example, the pixel values ​​of brightness and / or contrast of the captured image may be calculated, and the captured image may be corrected so that they approach the set values ​​in the learning phase. Instead of or in addition to the correction using the evaluation indices, the captured image may be corrected so that aggregates are supplemented in areas where aggregates are not present, by image processing using a machine-learned model or the like.

[0100] 14, if it is determined in step S11 that the captured image 74 is unsuitable for prediction by the prediction model M, the quality prediction device 1 executes step S16. In step S16, for example, the prediction calculation unit 30 (a part that functions as an image correction unit) executes the image correction process. After execution of step S16, steps S21 and S22 are executed. Note that if it is not determined in step S11 that the captured image 74 is unsuitable, step S16 is not executed.

[0101] When the captured image is unsuitable, the quality prediction device 1 may execute one process selected from two or more processes of a warning display process, an image display process, and a correction process. For example, the quality prediction device 1 may select and execute one process from the two or more processes according to the level of deviation between the evaluation index and a set value (± an acceptable target value).

[0102] The quality prediction device 1 may be a server connected to a terminal device such as a smartphone that can be operated by a user via a communication network. In this case, the terminal device may execute steps S01, S03, S05, S07, and S09, and the quality prediction device 1 as the server may execute steps S02, S04, S06, S08, S11, S12, S21, and S22. The display output unit 32 included in the quality prediction device 1 may display various information on a monitor included in a terminal device that is a device other than the quality prediction device 1.

[0103] The quality prediction device 1 as the server may exchange information with the terminal device by presenting to the user a Web page or the like that can be viewed and accessed from the terminal device operated by the user. In this case, in step S01, login may be executed instead of starting an application. Alternatively, instead of a Web page, an application that cooperates with the quality prediction device 1 may be prepared in the terminal device, and the quality prediction device 1 as the server may provide information to the terminal device.

[0104] The prediction model M described above is an example, and the prediction model M may be any model as long as it can output at least a plurality of predicted values ​​of mass fractions for a plurality of gradation classes or coarse aggregate ratios from information based on image data of aggregates. The input data to the prediction model M may include information indicating physical quantities that may affect the mass fraction or coarse aggregate ratio, in addition to image data obtained by imaging the aggregate.

[0105] In one example among the various examples described above, at least a part of the matters described in the other examples may be combined. [Explanation of symbols]

[0106] 1...quality prediction device, 14...monitor, 24...image data acquisition section, 26...image evaluation section, 30...prediction calculation section, 32...display output section, M...prediction model.

Claims

1. an image acquiring step of acquiring captured image data obtained by imaging an aggregate to be evaluated; an image evaluation step of evaluating a captured image related to the captured image data; a prediction step of inputting evaluation input information based on the captured image data after the image evaluation step, to a prediction model constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for learning and a correct answer value of the coarse aggregate ratio corresponding to the input information, to output a value indicating the coarse aggregate ratio in response to the input of the input information, and acquiring a predicted value output from the prediction model; A program for causing a computer to execute the following: A quality prediction program, wherein in the image evaluation step, whether or not the captured image is suitable for prediction using the prediction model is evaluated.

2. an image acquiring step of acquiring captured image data obtained by imaging the aggregate to be evaluated; an image evaluation step of evaluating a captured image related to the captured image data; a prediction process in which, after the image evaluation process, evaluation input information based on the captured image data is input to a prediction model constructed by machine learning based on input information based on learning image data obtained by photographing an aggregate for learning and on correct values ​​of mass fractions for each of a plurality of granularity classifications of the aggregate associated with the input information, so as to output a value indicating a mass fraction for each classification included in the plurality of granularity classifications in response to the input of the input information, and a prediction process in which, after the image evaluation process, evaluation input information based on the captured image data is input to the prediction model to obtain a predicted value for each classification that is output from the prediction model; A program for causing a computer to execute the following: A quality prediction program, wherein in the image evaluation step, whether or not the captured image is suitable for prediction using the prediction model is evaluated.

3. 3. The quality prediction program according to claim 1, further comprising a warning display step of displaying a warning to a user on a monitor before the prediction step, when the captured image is determined to be unsuitable for prediction by the prediction model in the image evaluation step.

4. 3. The quality prediction program according to claim 1, further comprising: an image display step of displaying the captured image on a monitor before the prediction step when the captured image is determined in the image evaluation step to be unsuitable for prediction by the prediction model.

5. 3. The quality prediction program according to claim 1, further comprising: if the captured image is determined in the image evaluation step to be unsuitable for prediction by the prediction model, the quality prediction program further causes the computer to execute an image correction step of correcting the captured image before the prediction step.

6. an image acquiring step of acquiring captured image data obtained by imaging the aggregate to be evaluated; an image evaluation step of evaluating a captured image related to the captured image data; a prediction step of inputting evaluation input information based on the captured image data after the image evaluation step, to a prediction model constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for learning and a correct answer value of the coarse grain ratio corresponding to the input information, so as to output a value indicating the coarse grain ratio in response to the input of the input information, and acquiring a predicted value output from the prediction model; A quality prediction method, wherein in the image evaluation step, it is evaluated whether or not the captured image is suitable for prediction using the prediction model.

7. an image acquiring step of acquiring captured image data obtained by imaging the aggregate to be evaluated; an image evaluation step of evaluating a captured image related to the captured image data; a prediction step of inputting evaluation input information based on the captured image data after the image evaluation step, to a prediction model constructed to output a value indicating a mass fraction for each of a plurality of granularity classifications included in the plurality of granularity classifications in response to input of the input information by machine learning based on input information based on learning image data obtained by imaging an aggregate for learning and on a ground truth value of the mass fraction for each of a plurality of granularity classifications of the aggregate associated with the input information, and obtaining a predicted value output from the prediction model for each of the classifications; A quality prediction method, wherein in the image evaluation step, it is evaluated whether or not the captured image is suitable for prediction using the prediction model.

8. an image data acquisition unit that acquires captured image data obtained by imaging the aggregate to be evaluated; an image evaluation unit that evaluates a captured image related to the captured image data; a prediction calculation unit that, after evaluation by the image evaluation unit, inputs evaluation input information based on the captured image data into a prediction model that is constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for learning and a correct answer value of a coarse grain ratio associated with the input information, and acquires a predicted value output from the prediction model; The image evaluation unit evaluates whether the captured image is suitable for prediction using the prediction model.

9. an image data acquisition unit that acquires captured image data obtained by imaging the aggregate to be evaluated; an image evaluation unit that evaluates a captured image related to the captured image data; a prediction calculation unit that executes a prediction step of inputting evaluation input information based on the captured image data, after evaluation by the image evaluation unit, into a prediction model that is constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for learning and on ground-truth values ​​of mass fractions for each of a plurality of granularity classifications of the aggregate corresponding to the input information, to output a value indicating a mass fraction for each classification included in the plurality of granularity classifications in response to the input of the input information, and acquiring a predicted value output from the prediction model for each classification, The image evaluation unit evaluates whether the captured image is suitable for prediction using the prediction model.

Images