Quality prediction program, quality prediction method, and quality prediction device

Abstract

To improve convenience of a user.SOLUTION: A quality prediction program is a program allowing a computer to execute an image display step of allowing a monitor to display a captured image obtained by capturing an aggregate of an evaluation object, an acquisition step of acquiring evaluation input information based on captured image data related to the captured image, and a prediction step of inputting the evaluation input information to a prediction model constructed to output a value indicating a fineness modulus associated with the input information in response to input of the input information, by machine learning based on input information based on learning image data obtained by capturing an aggregate for learning and a correct answer value of the fineness modulus, and acquiring a prediction value output from the prediction model. The acquisition step determines the evaluation input information, on the basis of the state of the captured image displayed on the monitor when use input instructing determination of the evaluation input information is input.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present disclosure relates to a quality prediction program, a quality prediction method, and a quality prediction device. [Background technology]

[0002] Patent Document 1 discloses a method for estimating the particle size distribution of soil. Patent Document 2 discloses a method for estimating the quality of aggregate. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-117625 [Patent Document 2] JP 2021-135199 A Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a quality prediction program, a quality prediction method, and a quality prediction device that are useful for improving user convenience. [Means for solving the problem]

[0005] [1] A quality prediction program that causes a computer to execute the following steps: an image display step of displaying an image obtained by photographing an aggregate to be evaluated on a monitor; an acquisition step of acquiring evaluation input information based on image data relating to the captured image; and a prediction step of inputting the evaluation input information into a prediction model constructed by machine learning based on input information based on learning image data obtained by photographing an aggregate for training and a ground truth value of the coarse grain ratio corresponding to the input information, and acquiring a predicted value output from the prediction model, wherein in the acquisition step, the evaluation input information is determined based on the state of the captured image displayed on the monitor at the time a user input instructing the determination of the evaluation input information is input. According to the quality program, the information to be input to the prediction model is determined by inputting a user input instructing the determination of the evaluation input information, so that the user can not only obtain a prediction result, but also easily understand what input the prediction result was based on, which is useful for improving user convenience.

[0006] [2] The quality prediction program described in [1] above, further causing the computer to execute a result display process of displaying on the monitor the predicted value obtained in the prediction process and a control range representing an appropriate range of coarse aggregate ratio according to the type of aggregate to be evaluated. In this case, the user can immediately know whether the prediction result is appropriate or not, which is further useful for improving user convenience.

[0007] [3] A program for causing a computer to execute the steps of: an image display step of displaying an image obtained by photographing an aggregate to be evaluated on a monitor; an acquisition step of acquiring evaluation input information based on image data related to the captured image; and a first prediction step of inputting the evaluation input information into a prediction model constructed by machine learning based on input information based on training image data obtained by photographing an aggregate for training and ground truth values ​​of mass fractions for each of a plurality of particle size classifications of the aggregate corresponding to the input information, so as to output a value indicating a mass fraction for each classification included in the plurality of particle size classifications in response to input of the input information, and acquiring a predicted value output from the prediction model for each classification, wherein in the acquisition step, the evaluation input information is determined based on the state of the captured image displayed on the monitor at the time a user input instructing the determination of the evaluation input information is input. According to the quality program, the information to be input to the prediction model is determined by inputting a user input instructing the determination of the evaluation input information, so that the user can not only obtain a prediction result, but also easily understand what input the prediction result was based on, which is useful for improving user convenience.

[0008] [4] The quality prediction program described in [3] above further causes the computer to execute a second prediction step of calculating the coarse aggregate ratio of the aggregate to be evaluated from the predicted values ​​for each category obtained in the first prediction step, and a result display step of displaying on the monitor the coarse aggregate ratio calculated in the second prediction step and a control range representing an appropriate range of coarse aggregate ratio according to the type of aggregate to be evaluated. In this case, the user can immediately know whether the prediction result is appropriate or not, which is further useful for improving user convenience.

[0009] [5] The quality prediction program according to any one of [1] to [4] above, wherein the image display process includes adjusting the captured image displayed on the monitor when a user input instructing adjustment of the image is input. In this case, the user can adjust the image and then decide on the input to the prediction model, which is further useful for improving user convenience.

[0010] [6] The quality prediction program described in [5] above, wherein the adjustment of the captured image performed in the image display process includes at least one of enlarging the image, reducing the image, cropping the image, changing the brightness of the image, rotating the image, changing the clarity of the image, and changing the contrast of the image. In this case, the user can apply various types of processing to the image before deciding to input it to the prediction model, which is further useful for improving user convenience.

[0011] [7] The quality prediction program according to any one of [1] to [6] above, further causing the computer to execute an auxiliary display step of displaying on the monitor information indicating the conditions under which the training aggregate was imaged when the prediction model was constructed, or information indicating precautions to take when imaging the aggregate to be evaluated. In this case, the user can easily obtain captured images suitable for input to the prediction model, which is useful for improving the reliability of quality prediction using the prediction model.

[0012] [8] A quality prediction method including: an image display step of displaying an image obtained by photographing an aggregate to be evaluated on a monitor; an acquisition step of acquiring evaluation input information based on image data relating to the captured image; and a prediction step of inputting the evaluation input information into a prediction model constructed by machine learning based on input information based on learning image data obtained by photographing an aggregate for training and a ground truth value of the coarse grain ratio corresponding to the input information, and acquiring a predicted value output from the prediction model, wherein in the acquisition step, the evaluation input information is determined based on the state of the captured image displayed on the monitor at the time a user input instructing the determination of the evaluation input information is input. This quality prediction method is useful for improving user convenience, similar to the above-mentioned quality prediction program.

[0013] [9] A quality prediction method comprising: an image display step of displaying an image obtained by photographing an aggregate to be evaluated on a monitor; an acquisition step of acquiring evaluation input information based on image data relating to the captured image; and a first prediction step of inputting the evaluation input information into a prediction model constructed by machine learning based on input information based on training image data obtained by photographing an aggregate for training and on ground-truth values ​​of mass fractions for each of a plurality of particle size classifications of the aggregate corresponding to the input information, so as to output a value indicating a mass fraction for each classification included in the plurality of particle size classifications in response to input of the input information, and acquiring a predicted value output from the prediction model for each classification, wherein in the acquisition step, the evaluation input information is determined based on the state of the captured image displayed on the monitor at the time a user input instructing the determination of the evaluation input information is input. This quality prediction method is useful for improving user convenience, similar to the above-mentioned quality prediction program.

[0014]

[10] A quality prediction device comprising: a display output unit which displays an image obtained by imaging an aggregate to be evaluated on a monitor; an evaluation information acquisition unit which acquires evaluation input information based on image data relating to the captured image; and a prediction calculation unit which inputs the evaluation input information to a prediction model constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for training and a ground truth value of the coarse grain ratio corresponding to the input information, and acquires a predicted value output from the prediction model, wherein the evaluation information acquisition unit determines the evaluation input information based on the state of the captured image displayed on the monitor at the time a user input instructing the determination of the evaluation input information is input. This quality prediction device is useful for improving user convenience, similar to the above-mentioned quality prediction program.

[0015]

[11] A quality prediction device comprising: a display output unit that displays an image obtained by imaging an aggregate to be evaluated on a monitor; an evaluation information acquisition unit that acquires evaluation input information based on image data related to the captured image; and a prediction calculation unit that executes a first prediction step of inputting the evaluation input information to a prediction model constructed by machine learning based on input information based on training image data obtained by imaging an aggregate for training and ground truth values ​​of mass fractions for each of a plurality of particle size classifications of the aggregate corresponding to the input information, and acquires predicted values ​​output from the prediction model for each classification, wherein the evaluation input information is input to a prediction model, the prediction model being constructed to output a value indicating a mass fraction for each classification included in the plurality of particle size classifications in response to input of the input information, and the evaluation information acquisition unit determines the evaluation input information based on a state of the captured image displayed on the monitor at the time a user input instructing the determination of the evaluation input information is input. This quality prediction device is useful for improving user convenience, similar to the above-mentioned quality prediction program. Effect of the Invention

[0016] According to the present disclosure, a quality prediction program, a quality prediction method, and a quality prediction device are provided that are useful for improving user convenience. [Brief description of the drawings]

[0017] [Figure 1] FIG. 1 is a schematic diagram illustrating a ready-mix concrete manufacturing system and a quality prediction device. [Diagram 2] FIG. 2 is a block diagram illustrating an example of a hardware configuration of the quality prediction device. [Diagram 3] FIG. 3 is a block diagram illustrating an example of a functional configuration of the quality prediction device. [Figure 4] Fig. 4(a) is a schematic diagram showing an example of a predictive calculation using a predictive model, and Fig. 4(b) is a cross-sectional view showing a schematic diagram of acquiring image data for learning. [Diagram 5] FIG. 5 is a diagram showing an example of image data for learning. [Figure 6]6(a) and 6(b) are graphs showing an example of the comparison result between the predicted value and the correct value. [Figure 7] FIG. 7 is a flow diagram showing an example of a series of processes executed by the quality prediction device. [Figure 8] 8(a) and 8(b) are schematic diagrams showing an example of a display screen on a monitor. [Figure 9] 9(a) and 9(b) are schematic diagrams showing an example of a display screen on a monitor. [Figure 10] 10(a) and 10(b) are schematic diagrams showing an example of a display screen on a monitor. [Figure 11] FIG. 11 is a schematic diagram showing an example of a display screen on a monitor. [Figure 12] FIG. 12 is a schematic diagram showing an example of a display screen on a monitor. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, an embodiment will be described with reference to the drawings. In the description, the same elements or elements having the same functions are given the same reference numerals, and duplicated descriptions will be omitted. Fig. 1 shows a schematic diagram of a quality prediction device according to an embodiment, and a ready-mix concrete manufacturing system in which the quality prediction device is used.

[0019] [Ready-mix concrete manufacturing system] First, an overview of a ready-mix concrete manufacturing system will be described. The manufacturing system 100 shown in Fig. 1 is a system that manufactures ready-mix concrete. The manufacturing system 100 kneads and mixes concrete materials to manufacture (produce) ready-mix concrete.

[0020] The concrete materials used in the manufacturing system 100 include cement, admixtures, coarse aggregates, fine aggregates, water, and admixtures. Examples of the coarse aggregates include gravel, crushed stone, slag coarse aggregates, lightweight coarse aggregates, recycled coarse aggregates, reclaimed aggregates, and coarse aggregates made by mixing these. Examples of the gravel include mountain gravel, land gravel, river gravel, and sea gravel. Examples of the slag coarse aggregates include blast furnace slag aggregates, ferronickel slag aggregates, electric furnace oxidized slag aggregates, and coal gasification slag aggregates. Examples of the lightweight coarse aggregates include natural lightweight aggregates, by-product lightweight aggregates, and artificial lightweight aggregates. The coarse aggregates may include crushed rock, or limestone crushed stone.

[0021] Examples of fine aggregates include sand, crushed sand, slag fine aggregate, lightweight fine aggregate, recycled fine aggregate, recovered aggregate, and fine aggregates made by mixing these. Sand includes mountain sand, land sand, river sand, sea sand, etc. Slag fine aggregates include blast furnace slag aggregate, ferronickel slag aggregate, copper slag aggregate, electric furnace oxidized slag aggregate, coal gasification slag aggregate, etc. Lightweight fine aggregates include natural lightweight aggregate, by-product lightweight aggregate, artificial lightweight aggregate, etc.

[0022] Examples of rock types of crushed stone and crushed sand include igneous rocks, sedimentary rocks, metamorphic rocks, quartzite, limestone, domalite, and peridotite. Igneous rocks include granite, diorite, gabbro, porphyrite, diabase, rhyolite, andesite, basalt, and serpentine. Sedimentary rocks include conglomerate, sandstone, shale, slate, and tuff. Metamorphic rocks include gneiss and schist. In the present disclosure, coarse aggregate and fine aggregate may be collectively referred to as "aggregate". In this case, "aggregate" refers to coarse aggregate, fine aggregate, or both coarse aggregate and fine aggregate.

[0023] The manufacturing system 100 loads the manufactured ready-mix concrete onto a transport vehicle 200. After the ready-mix concrete is loaded, the transport vehicle 200 transports the ready-mix concrete to a site (e.g., a construction site) where the ready-mix concrete is to be used. Examples of the transport vehicle 200 include an agitator vehicle (mixer vehicle) or a dump truck. The manufacturing system 100 may manufacture ready-mix concrete from concrete materials so as to satisfy a target quality (required quality) set for each site. In one example, the target quality of the ready-mix concrete includes a target value for slump or slump flow. The manufacturing system 100 includes, for example, a material storage yard 101, a transport device 104, and a manufacturing device 110.

[0024] The material yard 101 is a place where concrete materials are stored. The material yard 101 includes a plurality of silos 102. The plurality of silos 102 are containers that store at least a portion of the concrete materials by material type. The plurality of silos 102 include a silo 102 that stores coarse aggregate, a silo 102 that stores fine aggregate, and a silo 102 that stores cement.

[0025] The transporting device 104 is a device that transports the concrete materials stored in the multiple silos 102 to the manufacturing device 110. The transporting device 104 includes, for example, a belt conveyor that transports the concrete materials. The transporting device 104 may transport the concrete materials by type at different times. In one example, based on an operation instruction from a control device included in the manufacturing system 100, a specific material from among the various concrete materials is transferred to the transporting device 104 and transported to the manufacturing device 110.

[0026] The manufacturing apparatus 110 is an apparatus for manufacturing ready-mixed concrete by mixing concrete materials including aggregate. The manufacturing apparatus 110 includes, for example, a storage bottle 111, a measuring bottle 112, a collecting hopper 113, a mixer 114, and a loading hopper 115.

[0027] Storage bottle 111 temporarily stores various types of concrete materials. Various types of concrete materials are transported (conveyed) to storage bottle 111 from material storage yard 101 by transport device 104. Storage bottle 111 is configured to store various types of concrete materials individually. Hereinafter, "concrete material" may be simply referred to as "material". Various materials stored in storage bottle 111 are supplied to measuring bottle 112 as necessary.

[0028] The measuring bottle 112 is disposed below the storage bottle 111. The measuring bottle 112 operates based on an operation command from the control device, and weighs each material individually. When the measuring bottle 112 detects a target amount of material instructed by the control device, it supplies the material to the collecting hopper 113. When water is supplied to the measuring bottle 112, an admixture may be mixed into the water. The collecting hopper 113 is disposed below the measuring bottle 112. The collecting hopper 113 collects each material discharged from the measuring bottle 112, and supplies the collected materials to the mixer 114. Note that the collecting hopper 113 does not have to be provided, and in this case, each material is supplied from the measuring bottle 112 to the mixer 114.

[0029] The mixer 114 is disposed below the collecting hopper 113. The mixer 114 is a device that mixes concrete materials. The mixer 114 produces ready-mixed concrete by mixing (kneading) aggregate, cement, water, admixtures, and the like. The ready-mixed concrete is discharged from the bottom of the mixer 114 into the loading hopper 115. The loading hopper 115 is disposed below the mixer 114, and temporarily stores the ready-mixed concrete. The loading hopper 115 supplies the temporarily stored ready-mixed concrete to the transport vehicle 200.

[0030] The manufacturing system 100 described above is one example of a ready-mix concrete manufacturing system, and the ready-mix concrete manufacturing system may be configured in any manner as long as it is capable of mixing concrete materials and manufacturing ready-mix concrete.

[0031] [Quality prediction device] The quality prediction device 1 is a device that predicts the coarse aggregate ratio as the quality of aggregate contained in concrete material. The quality prediction device 1 is configured with one or more computers. The computers that configure the quality prediction device 1 may be a personal computer, a tablet computer (tablet terminal), a smartphone, a wearable device, a workstation, a server computer, or a general-purpose computer. The quality prediction device 1 may be connected to a communication network via a wired or wireless connection.

[0032] The quality prediction device 1 includes, for example, a main body 10, an input device 12, a monitor 14, and a camera 16. The main body 10 is a device (computer main body) that executes the main functions of the computer that constitutes the quality prediction device 1.

[0033] The input device 12 is a device for inputting information to the main body 10. The input device 12 may be any device capable of inputting desired information, and specific examples thereof include operation interfaces such as a keypad, a mouse, and an operation controller.

[0034] The monitor 14 is a device for displaying information output from the main body 10. The monitor 14 may be any device capable of displaying graphics, a specific example of which is a liquid crystal panel. The monitor 14 and the input device 12 may be integrated as a touch panel.

[0035] The camera 16 is a device capable of capturing an image of the aggregate. The camera 16 captures an image of the aggregate to generate image data. The camera 16 is, for example, a digital camera that captures an image of the imaging range based on visible light, or a video camera. The image data generated by the camera 16 is used to predict the quality of the aggregate. When the camera 16 generates a video of the imaging target, still images included in the video may be acquired as image data.

[0036] The camera 16 may be disposed in the manufacturing system 100 so as to be able to capture an image of an aggregate at a specific location. For example, the camera 16 may be disposed so as to be able to capture an image of an aggregate being transported by the transport device 104. The camera 16 may be portable, and a user such as a worker may carry the camera 16 to an imaging position. The camera 16 may be mounted on a mobile body such as a drone, and the camera 16 may capture an image at any location in the manufacturing system 100 by a user operating the mobile body, etc.

[0037] In the quality prediction device 1, the main body 10, the input device 12, the monitor 14, and the camera 16 may be integrated into one device, such as a smartphone with a built-in (mounted) camera or a tablet computer with a built-in camera. In one example, the quality prediction device 1 is configured by installing an application for predicting the quality of aggregate on a smartphone.

[0038] As shown in FIG. 2, the main body 10 includes a circuit 50. The circuit 50 includes a processor 51, a memory 52, a storage 53, and an input / output port 54. The storage 53 is configured with one or more non-volatile memory devices such as a flash memory or a hard disk. The storage 53 stores a quality prediction program that causes a computer to execute at least an image display step, an acquisition step, and a prediction step, or an image display step, an acquisition step, a first prediction step, and a second prediction step. The quality prediction program may cause a computer to execute a specification step, an auxiliary display step, an image display step, an acquisition step, a prediction step (or a first prediction step and a second prediction step), and a result display step. Details of these steps will be described later. The storage 53 stores a quality prediction program for configuring each functional block of the main body 10.

[0039] The memory 52 is composed of one or more volatile memory devices such as a random access memory. The memory 52 temporarily stores a quality prediction program loaded from the storage 53. The processor 51 is composed of one or more arithmetic devices such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The processor 51 configures each functional block of the main body 10 by executing the quality prediction program loaded into the memory 52. ​​The calculation results by the processor 51 are temporarily stored in the memory 52. ​​The input / output port 54 inputs and outputs information between the input device 12, the monitor 14, the camera 16, etc. in response to a request from the processor 51.

[0040] The quality prediction program may be provided in a state of being fixedly recorded on a tangible recording medium such as a CD-ROM, a DVD-ROM, or a semiconductor memory. Alternatively, the quality prediction program may be provided via a communication network as a data signal superimposed on a carrier wave. The main body 10 (quality prediction device 1) may be composed of a plurality of computers communicably connected to each other. The main body 10 (quality prediction device 1) may be a part of a control device included in the manufacturing system 100.

[0041] 3 shows an example of functional components (referred to as "functional blocks" in this disclosure) included in the main body 10. The main body 10 includes, as functional blocks, a user input acquisition unit 22, an image data acquisition unit 24, an evaluation information acquisition unit 26, a prediction model storage unit 28, a prediction calculation unit 30, a display output unit 32, and an aggregate information storage unit 34. The processes executed by these functional blocks correspond to the processes executed by the main body 10 (quality prediction device 1).

[0042] The user input acquisition unit 22 is configured to acquire user input information indicating an input from a user required for predicting the coarse grain ratio of the aggregate to be evaluated. The user input acquisition unit 22 may acquire the user input information based on a user input to an interface screen displayed on the monitor 14.

[0043] The image data acquisition unit 24 is configured to acquire image data generated by capturing an image of the aggregate to be evaluated (hereinafter, referred to as "captured image data"). The image data acquisition unit 24 acquires captured image data (captured image) captured by the camera 16.

[0044] The evaluation information acquiring unit 26 is configured to acquire evaluation input information based on captured image data relating to an image obtained by capturing an image of the aggregate to be evaluated. The evaluation information acquiring unit 26 may acquire, for example, the captured image data as is as the evaluation input information, or may acquire image data obtained by subjecting the captured image data to a predetermined processing as the evaluation input information. A user input may determine whether or not to process the captured image data in order to obtain the evaluation input information, and how to process the captured image data.

[0045] The prediction model holding unit 28 is configured to hold a model for predicting the coarse grain ratio of the aggregate to be evaluated (hereinafter referred to as "prediction model M"). The prediction model M is a model constructed by machine learning based on input information based on learning image data (learning image data) obtained by imaging the learning aggregate and a ground truth value of a quality value indicating the quality of the aggregate associated with the input information, so as to output a predicted value of the quality value in response to the input of the input information. The quality value is, for example, the coarse grain ratio, the mass fraction for each grain size division included in the grain size division of the aggregate, or both the coarse grain ratio and the mass fraction for each grain size division. The construction of the prediction model M will be described later.

[0046] The prediction model holding unit 28 may hold (store) a plurality of types of prediction models M. The plurality of types of prediction models M held by the prediction model holding unit 28 may be associated with a plurality of types of aggregate, respectively. Each of the plurality of types of prediction models M may be constructed by machine learning based on learning data obtained from the corresponding type of aggregate. For example, when the type of aggregate to be evaluated is "crushed stone 2005", the quality value may be predicted using the prediction model M associated with the crushed stone 2005 among the plurality of types of prediction models M.

[0047] Machine learning refers to a method in which a machine (computer) autonomously finds laws or rules by repeatedly learning based on given information. The prediction model M can be constructed using an algorithm and a data structure. The prediction model M is realized, for example, using a neural network, which is an information processing model that mimics the mechanism of the human brain. There is no particular limitation on the specific algorithm of the machine learning performed when constructing the prediction model M. The neural network has an input layer, one or more intermediate layers, and an output layer. By including one or more intermediate layers, a more complex prediction model M can be constructed, and prediction accuracy can be improved.

[0048] The prediction model M may be constructed by machine learning using a convolutional neural network (CNN). The prediction model M is constructed by machine learning based on the correct value of the quality value. The prediction model M may output only the coarse grain ratio, only a value indicating the mass fraction for each grain size division included in the multiple grain size divisions, or both the coarse grain ratio and the mass fraction for each grain size division, as the quality value, according to the input information. When outputting the mass fraction for each grain size division, the prediction model M outputs multiple values ​​each indicating a mass fraction for the multiple grain size divisions. The following illustrates an example in which the prediction model M outputs both the coarse grain ratio and the mass fraction for each grain size division.

[0049] The prediction model M is constructed to predict the above quality value, for example, by performing machine learning using data provided as input for machine learning and correct answer data (correct answer value of the above quality value) of the machine learning output. The input for the machine learning is various data sets of input information based on learning image data obtained by imaging the learning aggregate. In the various data sets of input information, at least the image data included in the input information is different from each other. The output of the machine learning is data (numerical value) indicating a quality value such as the coarse grain ratio. When constructing the prediction model M, a model that outputs a predicted value of the above quality value is iteratively trained using multiple combinations of input information and correct answer values ​​such as the coarse grain ratio.

[0050] The stage in which the prediction model M is autonomously constructed corresponds to the learning phase. The learning phase may be performed before the production phase in which ready-mix concrete is manufactured in the manufacturing system 100, or may be performed at an early stage of the production phase. The prediction model M, which is a trained model, is portable between computers. Therefore, the prediction model M constructed in another device may be stored in the prediction model holding unit 28. The stage in which the quality prediction device 1 predicts the coarse particle ratio of the aggregate to be evaluated using the prediction model M corresponds to the evaluation phase.

[0051] The prediction calculation unit 30 is configured to input evaluation input information to the prediction model M and obtain a predicted value output from the prediction model M. For example, the prediction calculation unit 30 obtains a predicted value indicating a coarse grain ratio from the prediction model M. The prediction calculation unit 30 may also obtain a plurality of predicted values ​​indicating mass fraction values ​​for a plurality of grain size divisions from the prediction model M. The prediction calculation unit 30 may calculate the coarse grain ratio from a predicted value (a plurality of predicted values) regarding the mass fraction for each grain size division. For reliability evaluation, the prediction calculation unit 30 may obtain both the coarse grain ratio and a plurality of predicted values ​​each indicating a mass fraction value from the prediction model M and compare the coarse grain ratio predicted for each prediction model M. As described above, the calculation (prediction) of the coarse grain ratio is performed by two different methods using the prediction model M, so that the reliability of the predicted value of the coarse grain ratio can be evaluated.

[0052] The display output unit 32 is configured to display an image obtained by imaging an aggregate to be evaluated, whose quality values ​​such as the coarse aggregate ratio are unknown, on the monitor 14. The display output unit 32 may display a prediction result by the prediction calculation unit 30 on the monitor 14. The display output unit 32 may display a predicted value of the coarse aggregate ratio calculated by the prediction calculation unit 30 and a control range indicating an appropriate range of the coarse aggregate ratio according to the type of aggregate to be evaluated on the monitor 14.

[0053] The aggregate information storage unit 34 is configured to store at least information necessary for displaying the above-mentioned control range. The aggregate information storage unit 34 stores, for example, information indicating the above-mentioned control ranges associated with each of a plurality of types of aggregate. That is, the aggregate information storage unit 34 may store information indicating the corresponding control range for each of a plurality of types of aggregate. The control range of the coarse aggregate ratio is set in advance for each type of aggregate by, for example, a user or a provider of the quality prediction device 1.

[0054] <Prediction model M> Here, the prediction model M used to predict the coarse grain ratio will be further described with reference to Figs. 4 and 5. In the learning phase for constructing the prediction model M, aggregate of the same type as the aggregate that is used in the manufacturing system 100 and can be evaluated is used as the learning aggregate. Fig. 4(a) shows a schematic diagram of the calculation process for predicting the coarse grain ratio using the prediction model M from the image Im, which is input information. The input of the prediction model M is the image Im, and the output of the prediction model M is the predicted value of the mass fraction of each of the multiple grain size classes and the predicted value of the coarse grain ratio.

[0055] When constructing the prediction model M, learning data for machine learning is prepared. The learning data includes various image data (input information) obtained by imaging the learning aggregate, and answer data associated with each of the various image data. In preparing the learning images, for example, various aggregates whose mass fractions for each granularity classification are known are prepared. The above various aggregates do not mean different types of aggregate, but mean the same type of aggregates with different granularities.

[0056] In one example, various types of learning aggregates are individually imaged by camera 130 in a stacked state on measurement sheet 132 (see FIG. 4(b)). A stacked state (piled state) refers to a state in which some particles contained in the aggregate overlap other particles when viewed vertically from above. By individually imaging the various types of aggregates by camera 130, learning image data is obtained for each type of aggregate. Camera 130 may be a digital camera, and images an area including the stacked aggregates from a position a predetermined distance away from the aggregates on measurement sheet 132 (for example, a predetermined position above).

[0057] The sheet 132 may be a rubber sheet. As shown in FIG. 4(b), the sheet 132 may be a sheet formed to be curved in imitation of the belt conveyor of the conveying device 104, or may be a sheet formed in a flat shape, unlike the example shown in FIG. 4(b). The amount of aggregate placed on the sheet 132 is not particularly limited, but may be about 1 kg to 10 kg. FIG. 4(b) shows a schematic example of aggregate piled up on the sheet 132, and shows piled up states when the amount of aggregate is 2 kg, 5 kg, and 8 kg. Note that the sheet 132 may be a container other than a rubber sheet. If it is a container with a rim, it is preferable from the viewpoint that it can be used to uniformly adjust the size of the captured aggregate by correcting the size of the image data based on the distance calculation result after calculating the distance from the camera to the aggregate.

[0058] In one example, samples of various aggregates are prepared under the conditions shown in Table 1 below, and images for learning are obtained by photographing the prepared samples. The mass fraction and coarse aggregate ratio for each grain size division of the various aggregates prepared under the conditions shown in Table 1 are known, and the known values ​​are the correct answer data for machine learning.

[0059] [Table 1]

[0060] The coarse grain ratio (FM) is calculated, for example, by the following formula (1) specified in the Standard Method for Concrete. In the measurement specified in the "Standard Specifications for Building Construction and Commentary JASS5 Reinforced Concrete Construction", 9 sieves are used excluding the 80 mm sieve in formula (1). The coarse grain ratio predicted in this disclosure may be specified by any method.

number

[0061] In Table 1, this means that aggregates of 10 levels of gradation (coarse aggregate ratio) are prepared. Category I represents a gradation category of particles smaller than 2.5 mm. Category I includes particles that cannot be retained on a 2.5 mm sieve. Category II represents a gradation category of particles 2.5 mm or larger and smaller than 5 mm. Category II includes particles that can be retained on a 2.5 mm sieve but cannot be retained on a 5 mm sieve. Category III represents a gradation category of particles 5 mm or larger and smaller than 10 mm. Category III includes particles that can be retained on a 5 mm sieve but cannot be retained on a 10 mm sieve. Category IV represents a gradation category of particles 10 mm or larger and smaller than 20 mm. Category IV includes particles that can be retained on a 10 mm sieve but cannot be retained on a 20 mm sieve.

[0062] For level 1, the coarse aggregate ratio can be calculated from the mass fractions of each grain size category (mass fractions of each of categories I to IV) as follows. For level 1 aggregate, the mass percentages of the sample that are retained on the 80 mm, 40 mm, and 20 mm sieves are 0, the mass percentages of the sample that are retained on the 10 mm sieve are 85, and the mass percentages of the sample that are retained on the 5 mm sieve are 100 (=15+85). Also, since all the grains are retained on the 5 mm sieve, the mass percentages of the sample that are retained on the 2.5 mm, 1.2 mm, 600 μm, 300 μm, and 150 μm sieves are 100. Therefore, the coarse aggregate ratio is calculated as shown in the following formula (2).

number

[0063] For levels 2 to 10, the coarse aggregate ratio is calculated in the same manner as for level 1. The conditions of the learning aggregate shown in Table 1 are merely examples, and the number of levels, the coarse aggregate ratio and mass fraction for each grading class at each level, and the method of setting the grading classes may be set arbitrarily. As with the multiple grading classes shown in Table 1, the grading classes in multiple classes including the smallest class (<150 μm) may be combined into one grading class. Grain sizes above a certain class (e.g., 20 to 40 mm) may not be included in multiple grading classes. Also, unlike the setting method in Table 1, grain sizes below a certain class (e.g., 1.2 to 2.5 μm) may not be included in multiple grading classes.

[0064] FIG. 5 shows images T1 to T10 obtained by imaging learning aggregates prepared according to the conditions of each of levels 1 to 10 in Table 1. Level n (n is an integer from 1 to 10) corresponds to image Tn. For example, image T1 is obtained by imaging learning aggregates prepared according to level 1. Learning aggregates according to each level n contain grains of multiple grain size divisions. In this way, learning aggregates may be prepared so as to contain grains of two or more grain size divisions, and input information may be prepared based on image data obtained by imaging the learning aggregates.

[0065] For each level n, multiple images Tn in which the states (more specifically, pile states) of the learning aggregate are different from each other may be prepared. In one example, a first image is taken with the learning aggregate placed on the measurement sheet 132, to obtain one image Tn. Then, the learning aggregate is temporarily removed from the sheet 132, and the same learning aggregate (same sample) is placed again on the sheet 132 so that the pile state of the multiple grains is different from that in the first image. Then, a second image is taken. By repeating the above-mentioned image taking, multiple images Tn may be prepared for each level n.

[0066] One data set is composed of one image Tn and the correct answer data of the mass fraction and the coarse grain ratio at the level n associated with the image Tn. As learning data, a plurality of data sets each consisting of a combination of the image Tn and the correct answer data of the mass fraction and the coarse grain ratio are prepared. Then, a prediction model M is constructed by performing machine learning using such learning data. The prediction model M constructed as described above outputs, for example, four mass fraction values ​​and a coarse grain ratio value for four grain size classifications.

[0067] Focusing on the point of outputting a value of the coarse grain ratio, the prediction model M is a model constructed by machine learning based on input information based on learning image data obtained by imaging the learning aggregate and the correct value of the coarse grain ratio associated with the input information, so as to output a value indicating the coarse grain ratio in response to the input of the input information. Focusing on the point of outputting a value of the mass fraction for each grain size division, the prediction model M is a model constructed by machine learning based on input information based on learning image data obtained by imaging the learning aggregate and the correct value of the mass fraction for each of the multiple grain size divisions of the aggregate associated with the input information, so as to output a value indicating the mass fraction for each of the multiple grain size divisions in response to the input of the input information.

[0068] In order to verify the method of predicting the coarse aggregate ratio using the prediction model M based on machine learning using a convolutional neural network, a comparison was made between the prediction results by the prediction model M and the correct values ​​using a dataset in which the correct values ​​were known. For the learning aggregate, 10 levels of aggregate with different coarse aggregate ratios were prepared according to the conditions in Table 1 above. For each level, images were taken 10 times indoors with the aggregate piled up on the measurement sheet 132 in different states, to obtain 10 images.

[0069] In addition, a total of 100 images (10 levels x 10 shots) were prepared according to the standard conditions shown in Table 2 below. As a result, 100 data sets were prepared as learning data, each of which corresponds to an image and a correct answer value.

[0070] [Table 2]

[0071] The prediction model M was constructed using 80 of the 100 datasets, and 20 datasets were used for validation. At each level, 8 data were used for model construction, and 2 datasets were used for validation. In the validation using 20 datasets, the prediction results calculated using the image and the prediction model M for each dataset were compared with the correct values ​​such as the coarse grain rate associated with the image. The prediction model M was constructed using a convolutional neural network with 6 intermediate layers consisting of a convolutional layer and a pooling layer, followed by 4 fully connected layers. The Huber function was used as the loss function. In each dataset, the number of images was increased by the Image Augmentation layer.

[0072] FIG. 6(a) is a graph showing the relationship between the predicted values ​​of the mass ratio (%) in the divisions III and IV in Table 1 and the corresponding correct values ​​of the mass fraction (%). In the graph shown in FIG. 6(a), the horizontal axis is the correct value of the mass fraction, and the vertical axis is the predicted value of the mass fraction output from the prediction model M. The prediction results in division III (5 mm to 10 mm) are plotted with △ marks, and the prediction results in division IV (10 mm to 20 mm) are plotted with ◯ marks. A solid line where the correct value and the predicted value match is drawn, and the closer the plotted results are to the line, the higher the prediction accuracy. From the results shown in FIG. 6(a), the prediction results are plotted near the line, and it can be seen that the mass ratio can be predicted with high accuracy by the prediction model M. Note that the coarse particle ratio can be calculated from the mass fraction, so if the mass fraction can be predicted with high accuracy by the prediction model M, the coarse particle ratio can also be predicted with high accuracy.

[0073] FIG. 6(b) is a graph showing the relationship between the predicted value of the coarse grain ratio output from the prediction model M and the correct value of the coarse grain ratio. In the graph shown in FIG. 6(b), the horizontal axis is the correct value of the coarse grain ratio, and the vertical axis is the predicted value of the coarse grain ratio output from the prediction model M. The prediction results are plotted with square marks. The solid line is the line where the correct value and the predicted value match, and a dashed upper limit line shifted vertically by +0.10 from the solid line and a lower limit line shifted vertically by -0.10 are drawn. The prediction results included in the area between the upper limit line and the lower limit line are defined as being within the allowable range. With reference to the error in the coarse grain ratio allowed in current ready-mix concrete plants, a range of ±0.10 was set as the allowable range. It can be seen that all of the prediction results using the prediction model M are included in the allowable range, and the prediction model M can predict the coarse grain ratio with high accuracy.

[0074] [Quality prediction method] Next, a quality prediction method executed in the quality prediction device 1 will be described with reference to Figs. 7 to 12. Hereinafter, the quality prediction method will be described using an example in which the quality prediction device 1 is configured by a smartphone with a camera. This quality prediction method includes, for example, a specification step, an auxiliary display step, an image display step, an acquisition step, a prediction step, and a result display step. The specification step, the auxiliary display step, the image display step, the acquisition step, the prediction step, and the result display step are all steps executed in the evaluation phase. That is, these steps are executed after the prediction model M has already been constructed. The quality prediction method may include a first prediction step and a second prediction step instead of the prediction step. Each of these steps may be executed by a corresponding functional block in the quality prediction device 1. In the learning phase, a plurality of prediction models M may be constructed according to a plurality of types of aggregate (a plurality of aggregate types).

[0075] The identification step is a step of identifying the type of aggregate to be evaluated from among a plurality of aggregate types based on a user input indicating the type of aggregate to be evaluated. The auxiliary display step is a step of displaying on the monitor 14 information indicating the conditions when the learning aggregate was imaged when constructing the prediction model M, or information indicating precautions when imaging the aggregate to be evaluated. The information indicating the conditions when the learning aggregate was imaged and the information indicating precautions when imaging the aggregate to be evaluated may include information that overlaps with each other, and such information will be referred to as "photography auxiliary information" hereinafter. For example, the photography auxiliary information may be displayed on the monitor 14 while a preview by the camera 16 is displayed on the monitor 14, or before imaging by the camera 16 is started.

[0076] Examples of the auxiliary information for photography include the brightness of the environment around the camera (e.g., whether indoors or outdoors, whether lighting is present, weather, etc.), the height of the camera (more specifically, the distance from the aggregate to be imaged to the camera), the type and origin of the aggregate, the brightness of the aggregate resulting from the surface condition of the aggregate (e.g., the degree of dryness or wetness), the pile condition of the aggregate (e.g., whether it is near the center or not, whether it is biased with respect to the central axis, the average height of the aggregate, the pile type such as a heap or a flat pile, the pile shape such as a regular cone shape or an elliptical cone shape), the movement speed of the aggregate when imaging is performed during transportation, the range of evaluable aggregate dimensions, the threshold value of the difference in the evaluation results when imaging is performed two or more times in succession, and the condition of the sheet or container on which the aggregate to be imaged is placed (e.g., the presence or absence of a handle, the presence or absence of a conveyor belt frame). At least one of the above exemplified information may be displayed on the monitor 14 as the auxiliary information for photography.

[0077] The image display step is a step of displaying an image obtained by capturing an image of the aggregate to be evaluated on the monitor 14 before the acquisition step. The image display step may include adjusting the captured image displayed on the monitor 14 when a user input instructing image adjustment is input. The adjustment of the captured image performed in the image display step may include at least one of enlarging an image, reducing an image, cropping an image, translating an image, changing the brightness of an image, rotating an image, changing the clarity of an image, changing the contrast of an image, and grayscaling an image.

[0078] In one example, when a user input instructing to rotate the image is input in the image display process, the captured image displayed on the monitor 14 is rotated. In the rotation of the captured image displayed on the monitor 14, for example, the image is rotated 90° to the right or left so that the vertical and horizontal directions on the image are switched. When the outer frame of the captured image is a square, the coordinates are converted in accordance with the rotation of the captured image. For example, when a device such as a smartphone is used as a computer constituting the quality prediction device 1, a gyro sensor mounted on the device may rotate the shooting screen, resulting in a captured image that is not intended by the photographer. By rotating the captured image to make the orientation of the learning image data and the orientation of the captured image the same, this is preferable from the viewpoint of improving the prediction accuracy of the coarse grain ratio.

[0079] The acquisition step is a step of acquiring evaluation input information based on captured image data obtained by capturing an image of the aggregate to be evaluated. In the acquisition step, the evaluation input information is determined based on the state of the captured image displayed on the monitor 14 at the time when a user input instructing the determination of the evaluation input information is input. In one example, in the image display step, data indicating the current captured image (at the time of performing the acquisition step) displayed on the monitor 14 is acquired as the evaluation input information. Since pixel values ​​for each coordinate are different in at least a part of the image between an unadjusted image (e.g., an image not rotated by 90°) and an adjusted image (e.g., an image rotated by 90°), which may be the same captured image, different evaluation input information is obtained.

[0080] The prediction step is a step of inputting evaluation input information to the prediction model M and acquiring a predicted value for the coarse aggregate ratio from the prediction model M. Instead of or in addition to the prediction step, a first prediction step and a second prediction step may be performed. The first prediction step is a step of inputting evaluation input information to the prediction model M and acquiring a predicted value for the mass fraction output from the prediction model M for each granularity division. The second prediction step is a step of calculating the coarse aggregate ratio of the aggregate to be evaluated from the predicted value for each granularity division acquired in the first prediction step. In the second prediction step, for example, the coarse aggregate ratio is calculated from the predicted value for each granularity division according to the above formula (1). In each of the prediction step and the first prediction step, a prediction model M according to the result of identifying the type of aggregate to be evaluated in the identifying step may be selected from among a plurality of prediction models M, and a predicted value such as the coarse aggregate ratio may be acquired using the selected prediction model M.

[0081] The result display step is a step of displaying the prediction result of the coarse aggregate ratio in the prediction step or / and the second prediction step on the monitor 14. In the result display step, the predicted value (coarse aggregate ratio) obtained in the prediction step and a control range indicating an appropriate range of the coarse aggregate ratio according to the aggregate type identified in the identification step may be displayed on the monitor 14.

[0082] <Processing flow> FIG. 7 is a flow diagram illustrating a series of processes executed in the quality prediction method. A user holds the quality prediction device 1 and executes step S01 at any timing when the user wants to predict the coarse aggregate ratio of the aggregate to be evaluated. In step S01, for example, the user starts an application for predicting quality by operating on a screen displayed on the monitor 14. In one example, as shown in FIG. 8(a), in a state where a home screen is displayed on the monitor 14 of the quality prediction device 1, the user clicks, taps, or presses an icon 60 representing an application for predicting quality.

[0083] When the quality prediction device 1 receives the start request of step S01, it executes step S02. In step S02, for example, the display output unit 32 of the quality prediction device 1 displays a home screen of an application that executes quality prediction on the monitor 14. An example of the home screen is shown in Fig. 8(b), and on the home screen, an execute button 62 for switching to a mode for photographing (taking an image of) an aggregate and an execute button 64 for switching to a mode for executing another function (for example, a mode for viewing past prediction results) are displayed.

[0084] Next, the user executes step S03. In step S03, for example, the user presses (e.g., taps) the execute button 62 displayed on the monitor 14 to select a mode for photographing the aggregate. When the selection of step S03 is made, the quality prediction device 1 executes step S04. In step S04, for example, the display output unit 32 switches the display content on the monitor 14 to a screen corresponding to the mode for photographing the aggregate. FIG. 9(a) shows an example of the screen after switching in step S04.

[0085] Next, the user executes step S05. In step S05, for example, the user operates monitor 14 to select a type corresponding to the aggregate to be evaluated from among a plurality of aggregate types. As also shown in FIG. 9(b), display output unit 32 may display on monitor 14 a pull-down menu 66 from which the user can select the type of aggregate to be evaluated. In pull-down menu 66, the user may select one that matches the type of aggregate to be evaluated from a plurality of aggregate types previously determined. FIG. 10(a) shows an example of a display screen after the type of aggregate to be evaluated has been selected.

[0086] When the selection of step S05 is performed, the quality prediction device 1 executes step S06. In step S06, for example, the prediction calculation unit 30 and the display output unit 32 identify the type of aggregate to be evaluated according to the selection result of step S05. After the execution of steps S05 and S06, the display output unit 32 displays a start button 68 for inputting an instruction to start the camera 16. Before the execution of steps S05 and S06, the display output unit 32 may display the start button 68 on the monitor 14 in a state in which the user cannot tap it. In FIG. 9(a) and the like, the button shown by the dashed line represents a state in which the user cannot tap it. Alternatively, the display output unit 32 may not display the start button 68 on the monitor 14 before the execution of steps S05 and S06, and may display the start button 68 on the monitor 14 after the execution of steps S05 and S06.

[0087] Next, the user executes step S07. In step S07, for example, the user instructs the camera 16 to be started, and then the camera 16 performs image capture. In one example, when the user taps the start button 68, the quality prediction device 1 starts the camera 16, and as shown in FIG. 10(b), the display output unit 32 displays a preview image 70 captured by the camera 16 and an image capture execution button 71 for instructing the camera 16 to perform image capture on the monitor 14. When the user taps the image capture execution button 71 while viewing the preview image 70 on the monitor 14, the camera 16 captures an image of the aggregate to be evaluated.

[0088] After the camera 16 is started up and before shooting is performed, the display output unit 32 may display the above-mentioned shooting auxiliary information (information indicated by "79" in FIG. 10(b)) on the monitor 14. The display output unit 32 may display the shooting auxiliary information 79 in a message format or a table format on the monitor 14. After the camera 16 is started up and before the preview image 70 is displayed, the display output unit 32 may display the shooting auxiliary information 79 on the monitor 14.

[0089] After the imaging in step S07, the quality prediction device 1 executes step S08. In step S08, for example, the image data acquisition unit 24 acquires captured image data generated by imaging using the camera 16, and the display output unit 32 displays an image 72 (captured image) based on the captured image data on the monitor 14. Fig. 11 illustrates a state in which the image 72 based on the captured image data, a rotation execution button 74 for instructing rotation, which is an example of adjustment of the image 72, a decision button 75 for instructing decision of an image (evaluation input information) to be input to the prediction model M, and a prediction execution button 76 for instructing execution of prediction by the prediction model M are displayed on the monitor 14.

[0090] Next, the user executes step S09. In step S09, for example, the user taps the rotation execution button 74 as necessary so as to rotate the image 72 on the monitor 14 by 90° per tap. If the user does not need to rotate the image 72 displayed on the monitor 14, he or she does not tap the rotation execution button 74. In one example, when the image 72 is captured by the camera 16 mounted on a device such as a smartphone constituting the quality prediction apparatus 1, the vertical and horizontal directions of the image 72 may be automatically converted. In this case, the user can adjust the image 72 to match the vertical direction, horizontal direction, up and down, and left and right within the field of view of the camera 16 by tapping the rotation execution button 74 one or more times.

[0091] In step S09, when the user taps the rotation execution button 74, the quality prediction device 1 receives an instruction to change the image angle (adjust the image) and executes step S10. In step S10, for example, the display output unit 32 rotates the captured image displayed on the monitor 14 by 90 degrees. This allows the user to check the captured image after rotation.

[0092] Next, in step S11, for example, when the user judges that the image 72 displayed on the monitor 14 is acceptable as an input to the prediction model M, the user taps the decision button 75 and then taps the prediction execution button 76. When the decision button 75 is tapped, the evaluation information acquisition unit 26 acquires evaluation input information to be input to the prediction model M based on the state (e.g., rotation state) of the image 72 displayed on the monitor 14 at the time the decision button 75 is tapped. The evaluation information acquisition unit 26 acquires, for example, pixel values ​​for each coordinate as at least a part of the evaluation input information from the image 72 displayed at the time the decision button 75 is tapped. The display output unit 32 may display the decision button 75 and the prediction execution button 76 on the monitor 14 in a state in which the user cannot tap before step S08 is executed. Alternatively, the display output unit 32 may not display the decision button 75 and the prediction execution button 76 on the monitor 14 before step S08 is executed, and may display the decision button 75 and the prediction execution button 76 on the monitor 14 after step S08 is executed.

[0093] In step S11, when the user taps the prediction execution button 76, the quality prediction device 1 receives an execution instruction regarding the prediction and executes step S12. In step S12, for example, the prediction calculation unit 30 inputs the acquired evaluation input information to the prediction model M and acquires the predicted values ​​of the coarse aggregate ratio and the mass fraction for each particle size division output from the prediction model M. The prediction calculation unit 30 acquires the predicted values ​​of the coarse aggregate ratio and the mass fraction for each particle size division using the prediction model M associated with the type of aggregate to be evaluated identified in step S06. The prediction calculation unit 30 calculates the coarse aggregate ratio from the mass fraction for each particle size division.

[0094] The display output unit 32 may not display the decision button 75. In this case, the evaluation information acquisition unit 26 acquires evaluation input information to be input to the prediction model M based on the state (e.g., rotation state) of the image 72 displayed on the monitor 14 at the time when the user taps the prediction execution button 76. That is, the user input indicating an instruction to execute prediction may include (or may serve as) a user input indicating an instruction to decide the evaluation input information.

[0095] Next, the quality prediction device 1 executes step S13. In step S13, for example, the display output unit 32 displays the result of the prediction calculation executed in step S12 on the monitor 14. Fig. 12 illustrates a state in which the result of the prediction calculation in step S10 is displayed on the monitor 14. In one example, the display output unit 32 causes the monitor 14 to display information 82a indicating the identified aggregate type, information 82b indicating the prediction result of the coarse aggregate ratio, and information 82c indicating the management range for the coarse aggregate ratio.

[0096] The "coarse grain ratio (direct)" in the information 82b represents the result of prediction obtained by acquiring the coarse grain ratio from the prediction model M, and the "coarse grain ratio (indirect)" represents the result of prediction obtained by calculating the coarse grain ratio from the mass fraction after predicting the mass fraction for each grain size division from the prediction model M. The information 82b may describe one or both of the "coarse grain ratio (direct)" and the "coarse grain ratio (indirect)" according to the quality control item. The information 82b may also display the prediction result of the mass fraction for each grain size division obtained from the prediction model M. The display output unit 32 may display on the monitor 14 the image 84 (for example, the captured image as it is, or the image after adjusting the captured image) input to the prediction model M when the result of the coarse grain ratio is obtained.

[0097] [Variations] The process flow shown in Fig. 7 is an example and can be modified as appropriate. For example, the quality prediction device 1 may execute step S12 after executing step S07, without going through steps S08 to S10. When there is only one type of aggregate whose quality is to be predicted by the quality prediction device 1, steps S05 and S06 may be omitted. Even if there is only one type of aggregate, the management range related to the coarse aggregate ratio displayed in this case remains the range corresponding to that one type.

[0098] The quality prediction device 1 may be a server connected to a terminal device such as a smartphone that can be operated by a user via a communication network. In this case, the terminal device may execute steps S01, S03, S05, S07, S09, and S11, and the quality prediction device 1 as the server may execute steps S02, S04, S06, S08, S10, S12, and S13. The display output unit 32 included in the quality prediction device 1 may display various information on a monitor included in the terminal device, which is a device other than the quality prediction device 1.

[0099] The quality prediction device 1 as the server may exchange information with the terminal device by presenting to the user a Web page or the like that can be viewed and accessed from the terminal device operated by the user. In this case, in step S01, login may be executed instead of starting an application. Alternatively, instead of a Web page, an application that cooperates with the quality prediction device 1 may be prepared in the terminal device, and the quality prediction device 1 as the server may provide information to the terminal device.

[0100] The prediction model M described above is an example, and the prediction model M may be any model as long as it can output at least a plurality of predicted values ​​of mass fractions for a plurality of gradation classes or coarse aggregate ratios from information based on image data of aggregates. The input data to the prediction model M may include information indicating physical quantities that may affect the mass fraction or coarse aggregate ratio, in addition to image data obtained by imaging the aggregate.

[0101] In one example among the various examples described above, at least a part of the matters described in the other examples may be combined. [Explanation of symbols]

[0102] 1...quality prediction device, 14...monitor, 24...image data acquisition section, 26...evaluation information acquisition section, 30...prediction calculation section, 32...display output section, M...prediction model.

Claims

1. an image display step of imaging the aggregate to be evaluated and displaying the image on a monitor; acquiring evaluation input information based on captured image data relating to the captured image; a prediction step of inputting the evaluation input information into a prediction model constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for learning and a correct answer value of the coarse aggregate ratio associated with the input information, so as to output a value indicating the coarse aggregate ratio in response to the input of the input information, and acquiring a predicted value output from the prediction model; A program for causing a computer to execute the following: A quality prediction program in which, in the acquisition process, the evaluation input information is determined based on the state of the captured image displayed on the monitor at the time a user input instructing the determination of the evaluation input information is input.

2. The quality prediction program of claim 1, further causing the computer to execute a result display process of displaying on the monitor the predicted value obtained in the prediction process and a control range representing an appropriate range of coarse aggregate ratio according to the type of aggregate to be evaluated.

3. an image display step of imaging the aggregate to be evaluated and displaying the image on a monitor; acquiring evaluation input information based on captured image data relating to the captured image; a first prediction step of inputting the evaluation input information into a prediction model constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for learning and on correct values ​​of mass fractions of a plurality of granularity classifications of the aggregate associated with the input information, so as to output a value indicating a mass fraction for each classification included in the plurality of granularity classifications in response to input of the input information, and acquiring a predicted value output from the prediction model for each classification; A program for causing a computer to execute the following: A quality prediction program in which, in the acquisition process, the evaluation input information is determined based on the state of the captured image displayed on the monitor at the time a user input instructing the determination of the evaluation input information is input.

4. The quality prediction program includes: a second prediction step of calculating a coarse aggregate ratio of the aggregate to be evaluated from the predicted value for each of the categories obtained in the first prediction step; a result display step of displaying on the monitor the coarse aggregate ratio calculated in the second prediction step and a control range that indicates an appropriate range of the coarse aggregate ratio according to the type of aggregate to be evaluated; The quality prediction program according to claim 3 , further comprising:

5. The quality prediction program according to any one of claims 1 to 4, wherein the image display step includes adjusting the captured image displayed on the monitor when a user input instructing adjustment of the image is input.

6. 6. The quality prediction program according to claim 5, wherein the adjustment of the captured image performed in the image display process includes at least one of enlarging an image, reducing an image, cropping an image, changing the brightness of an image, rotating an image, changing the sharpness of an image, and changing the contrast of an image.

7. The quality prediction program according to any one of claims 1 to 4, further causing a computer to execute an auxiliary display step of displaying on the monitor information indicating the conditions under which the training bone material was imaged when constructing the prediction model, or information indicating precautions to take when imaging the bone material to be evaluated.

8. an image display step of imaging the aggregate to be evaluated and displaying the image on a monitor; acquiring evaluation input information based on captured image data relating to the captured image; a prediction step of inputting the evaluation input information into a prediction model constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for learning and a correct answer value of the coarse aggregate ratio associated with the input information, so as to output a value indicating the coarse aggregate ratio in response to the input of the input information, and acquiring a predicted value output from the prediction model; A quality prediction method in which, in the acquisition process, the evaluation input information is determined based on the state of the captured image displayed on the monitor at the time a user input instructing the determination of the evaluation input information is input.

9. an image display step of imaging the aggregate to be evaluated and displaying the image on a monitor; acquiring evaluation input information based on captured image data relating to the captured image; a first prediction step of inputting the evaluation input information into a prediction model constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for learning and on a ground truth value of the mass fraction of each of a plurality of granularity classifications of the aggregate associated with the input information, so as to output a value indicating a mass fraction for each classification included in the plurality of granularity classifications in response to input of the input information, and acquiring a predicted value output from the prediction model for each classification; A quality prediction method in which, in the acquisition process, the evaluation input information is determined based on the state of the captured image displayed on the monitor at the time a user input instructing the determination of the evaluation input information is input.

10. a display output unit that displays an image obtained by imaging the aggregate to be evaluated on a monitor; an evaluation information acquisition unit that acquires evaluation input information based on captured image data relating to the captured image; a prediction calculation unit that inputs the evaluation input information to a prediction model that is constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for learning and a correct answer value of the coarse aggregate ratio associated with the input information, and obtains a predicted value output from the prediction model; The quality prediction device, wherein the evaluation information acquisition unit determines the evaluation input information based on the state of the captured image displayed on the monitor at the time a user input instructing the determination of the evaluation input information is input.

11. a display output unit that displays an image obtained by imaging the aggregate to be evaluated on a monitor; an evaluation information acquisition unit that acquires evaluation input information based on captured image data relating to the captured image; a prediction calculation unit that inputs the evaluation input information to a prediction model that is constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for learning and on ground truth values ​​of mass fractions of a plurality of granularity classifications of the aggregate associated with the input information, so as to output a value indicating a mass fraction for each classification included in the plurality of granularity classifications in response to input of the input information, and executes a first prediction step of acquiring a predicted value output from the prediction model for each classification, The quality prediction device, wherein the evaluation information acquisition unit determines the evaluation input information based on the state of the captured image displayed on the monitor at the time a user input instructing the determination of the evaluation input information is input.

Images