Quality prediction method, quality prediction program, and quality prediction apparatus

Abstract

To simplify the operation for predicting quality.SOLUTION: A quality prediction method includes: an image acquisition step of acquiring captured image data obtained by imaging aggregate to be evaluated; a distance acquisition step of acquiring distance data representing a distance between a camera and the aggregate to be evaluated, the distance data being obtained when imaging the aggregate to be evaluated; an image correction step of acquiring a corrected image by correcting a captured image related to the captured image data based on the distance data; and a prediction step of calculating a fineness modulus of the aggregate to be evaluated, on the basis of evaluation input information based on the corrected image and a prediction model which is constructed to output a value indicating a fineness modulus in accordance with an input of input information, by machine learning based on the input information based on learning image data obtained by imaging aggregate for learning and based on a ground truth value of fineness modulus associated with the input information.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present disclosure relates to a quality prediction method, a quality prediction program, and a quality prediction device. [Background technology]

[0002] Patent Document 1 discloses a method for estimating the particle size distribution of soil. Patent Document 2 discloses a method for estimating the quality of aggregate. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-117625 [Patent Document 2] JP 2021-135199 A Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a quality prediction method, a quality prediction program, and a quality prediction device that are useful for simplifying the work of predicting quality. [Means for solving the problem]

[0005] [1] A quality prediction method comprising: an image acquisition step of acquiring captured image data obtained by imaging an aggregate to be evaluated; a distance acquisition step of acquiring distance data indicating a distance between the aggregate to be evaluated and a camera when the aggregate to be evaluated is imaged; an image correction step of acquiring a corrected image by correcting the captured image related to the captured image data based on the distance data; and a prediction step of calculating the coarse grain ratio of the aggregate to be evaluated based on evaluation input information based on the corrected image, the prediction model being constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for training and a ground truth value of the coarse grain ratio corresponding to the input information, the prediction model being configured to output a value indicating the coarse grain ratio in response to input of the input information, and evaluation input information based on the corrected image. From the viewpoint of the accuracy of quality prediction by the prediction model, it is desirable to adjust the position of the camera relative to the aggregate to be evaluated when acquiring the captured image data, in accordance with the distance between the camera and the learning aggregate in the learning phase in which the prediction model is constructed. However, the task of adjusting the camera position may be cumbersome, and there may be restrictions on the position of the camera when capturing the image of the aggregate to be evaluated. In contrast, in the above method, the captured image is corrected based on the distance data to obtain a corrected image, and information based on the corrected image is input to the prediction model. In this case, the effect on the prediction accuracy by the prediction model can be reduced by correcting the difference in the distance data. As a result, the work of arranging the camera is reduced when capturing the image of the aggregate to be evaluated, and there are fewer restrictions. Therefore, it is useful for simplifying the work of predicting quality.

[0006] [2] A quality prediction method comprising: an image acquiring step of acquiring image data obtained by imaging an aggregate to be evaluated; a distance acquiring step of acquiring distance data indicating a distance between the aggregate to be evaluated and a camera when the aggregate to be evaluated is imaged; an image correction step of acquiring a corrected image by correcting the image data based on the distance data; and a prediction step of calculating a coarse grain ratio of the aggregate to be evaluated based on evaluation input information based on the corrected image, the prediction model being constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for training and on ground-truth values ​​of mass fractions of each of a plurality of granularity classifications of the aggregate corresponding to the input information, the prediction model being configured to output a value indicating a mass fraction for each classification included in the plurality of granularity classifications in response to input of the input information, and evaluation input information based on the corrected image. This quality prediction method, like the method described in [1] above, is useful for simplifying the work of predicting quality.

[0007] [3] The quality prediction method according to [1] or [2] above, wherein in the image correction process, the corrected image is obtained by extracting a range of the captured image according to the distance data. In this case, the difference in relative aggregate grain size in the image caused by the difference in distance data can be reduced.

[0008] [4] A quality prediction method described in any one of [1] to [3] above, wherein in the image acquisition process, a marker member of a predetermined size is imaged together with the aggregate to be evaluated to acquire the captured image data, and in the distance acquisition process, the distance data is acquired based on the size of the marker member in the captured image. In this case, when acquiring the captured image data, it is sufficient to simply capture the image so that the marker component is captured, which can improve the efficiency of the work of predicting quality using a prediction model.

[0009] [5] A quality prediction program that causes a computer to execute the following steps: an image acquisition step of acquiring captured image data obtained by imaging an aggregate to be evaluated; a distance acquisition step of acquiring distance data indicating a distance between the aggregate to be evaluated and a camera when the aggregate to be evaluated is imaged; an image correction step of acquiring a corrected image by correcting the captured image related to the captured image data based on the distance data; and a prediction step of calculating the coarse grain ratio of the aggregate to be evaluated based on evaluation input information based on the corrected image, the prediction model being constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for training and a ground truth value of the coarse grain ratio associated with the input information, so as to output a value indicating the coarse grain ratio in response to input of the input information. This quality prediction program, like the method described in [1] above, is useful for simplifying the work of predicting quality.

[0010] [6] A quality prediction program that causes a computer to execute the following steps: an image acquisition step of acquiring image data obtained by imaging an aggregate to be evaluated; a distance acquisition step of acquiring distance data indicating a distance between the aggregate to be evaluated and a camera when the aggregate to be evaluated is imaged; an image correction step of acquiring a corrected image by correcting the image data based on the distance data; and a prediction step of calculating the coarse grain ratio of the aggregate to be evaluated based on evaluation input information based on the corrected image, the prediction model being constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for training and on ground truth values ​​of mass fractions of each of a plurality of grain size classifications of the aggregate associated with the input information, so as to output a value indicating a mass fraction for each classification included in the plurality of grain size classifications in response to input of the input information. This quality prediction program, like the method described in [2] above, is useful for simplifying the work of predicting quality.

[0011] [7] A quality prediction device comprising: an image data acquisition unit that acquires captured image data obtained by imaging an aggregate to be evaluated; a distance data acquisition unit that acquires distance data indicating a distance between the aggregate to be evaluated and a camera when the aggregate to be evaluated is imaged; an image correction unit that acquires a corrected image by correcting the captured image related to the captured image data based on the distance data; and a prediction calculation unit that calculates the coarse grain ratio of the aggregate to be evaluated based on evaluation input information based on the corrected image, the prediction model being constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for training and a ground truth value of the coarse grain ratio associated with the input information, the prediction model being constructed to output a value indicating the coarse grain ratio in response to input of the input information, and evaluation input information based on the corrected image. This quality prediction device, like the method described in [1] above, is useful for simplifying the work of predicting quality.

[0012] [8] A quality prediction device comprising: an image data acquisition unit that acquires captured image data obtained by imaging an aggregate to be evaluated; a distance data acquisition unit that acquires distance data indicating a distance between the aggregate to be evaluated and a camera when the aggregate to be evaluated is imaged; an image correction unit that acquires a corrected image by correcting the captured image related to the captured image data based on the distance data; and a prediction calculation unit that calculates a coarse grain ratio of the aggregate to be evaluated based on evaluation input information based on the corrected image, the prediction model being constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for training and on ground truth values ​​of mass fractions of each of a plurality of grain size classifications of the aggregate corresponding to the input information, the prediction model being configured to output a value indicating a mass fraction for each classification included in the plurality of grain size classifications in response to input of the input information, and evaluation input information based on the corrected image. This quality prediction device, like the method described in [1] above, is useful for simplifying the work of predicting quality. Effect of the Invention

[0013] According to the present disclosure, a quality prediction method, a quality prediction program, and a quality prediction device are provided that are useful for simplifying the work of predicting quality. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a schematic diagram illustrating a ready-mix concrete manufacturing system and a quality prediction device. [Diagram 2] FIG. 2 is a block diagram illustrating an example of a hardware configuration of the quality prediction device. [Diagram 3] FIG. 3 is a block diagram illustrating an example of a functional configuration of the quality prediction device. [Figure 4] Fig. 4(a) is a schematic diagram showing an example of a predictive calculation using a predictive model, and Fig. 4(b) is a cross-sectional view showing a schematic diagram of acquiring image data for learning. [Diagram 5] FIG. 5 is a diagram showing an example of image data for learning. [Figure 6]6(a) and 6(b) are graphs showing an example of the comparison result between the predicted value and the correct value. [Figure 7] FIG. 7 is a flow diagram showing an example of a series of processes executed by the quality prediction device. [Figure 8] FIG. 8 is a diagram for explaining image correction. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, an embodiment will be described with reference to the drawings. In the description, the same elements or elements having the same functions are given the same reference numerals, and duplicated descriptions will be omitted. Fig. 1 shows a schematic diagram of a quality prediction device according to an embodiment, and a ready-mix concrete manufacturing system in which the quality prediction device is used.

[0016] [Ready-mix concrete manufacturing system] First, an overview of a ready-mix concrete manufacturing system will be described. The manufacturing system 100 shown in Fig. 1 is a system that manufactures ready-mix concrete. The manufacturing system 100 kneads and mixes concrete materials to manufacture (produce) ready-mix concrete.

[0017] The concrete materials used in the manufacturing system 100 include cement, admixtures, coarse aggregates, fine aggregates, water, and admixtures. Examples of the coarse aggregates include gravel, crushed stone, slag coarse aggregates, lightweight coarse aggregates, recycled coarse aggregates, reclaimed aggregates, and coarse aggregates made by mixing these. Examples of the gravel include mountain gravel, land gravel, river gravel, and sea gravel. Examples of the slag coarse aggregates include blast furnace slag aggregates, ferronickel slag aggregates, electric furnace oxidized slag aggregates, and coal gasification slag aggregates. Examples of the lightweight coarse aggregates include natural lightweight aggregates, by-product lightweight aggregates, and artificial lightweight aggregates. The coarse aggregates may include crushed rock, or limestone crushed stone.

[0018] Examples of fine aggregates include sand, crushed sand, slag fine aggregate, lightweight fine aggregate, recycled fine aggregate, recovered aggregate, and fine aggregates made by mixing these. Sand includes mountain sand, land sand, river sand, sea sand, etc. Slag fine aggregates include blast furnace slag aggregate, ferro-nickel slag aggregate, copper slag aggregate, electric furnace oxidized slag aggregate, coal gasification slag aggregate, etc. Lightweight fine aggregates include natural lightweight aggregate, by-product lightweight aggregate, artificial lightweight aggregate, etc.

[0019] Examples of rock types of crushed stone and crushed sand include igneous rocks, sedimentary rocks, metamorphic rocks, quartzite, limestone, domalite, and peridotite. Igneous rocks include granite, diorite, gabbro, porphyrite, diabase, rhyolite, andesite, basalt, and serpentine. Sedimentary rocks include conglomerate, sandstone, shale, slate, and tuff. Metamorphic rocks include gneiss and schist. In the present disclosure, coarse aggregate and fine aggregate may be collectively referred to as "aggregate". In this case, "aggregate" refers to coarse aggregate, fine aggregate, or both coarse aggregate and fine aggregate.

[0020] The manufacturing system 100 loads the manufactured ready-mix concrete onto a transport vehicle 200. After the ready-mix concrete is loaded, the transport vehicle 200 transports the ready-mix concrete to a site (e.g., a construction site) where the ready-mix concrete is to be used. Examples of the transport vehicle 200 include an agitator vehicle (mixer vehicle) or a dump truck. The manufacturing system 100 may manufacture ready-mix concrete from concrete materials so as to satisfy a target quality (required quality) set for each site. In one example, the target quality of the ready-mix concrete includes a target value for slump or slump flow. The manufacturing system 100 includes, for example, a material storage yard 101, a transport device 104, and a manufacturing device 110.

[0021] The material yard 101 is a place where concrete materials are stored. The material yard 101 includes a plurality of silos 102. The plurality of silos 102 are containers that store at least a portion of the concrete materials by material type. The plurality of silos 102 include a silo 102 that stores coarse aggregate, a silo 102 that stores fine aggregate, and a silo 102 that stores cement.

[0022] The transporting device 104 is a device that transports the concrete materials stored in the multiple silos 102 to the manufacturing device 110. The transporting device 104 includes, for example, a belt conveyor that transports the concrete materials. The transporting device 104 may transport the concrete materials by type at different times. In one example, based on an operation instruction from a control device included in the manufacturing system 100, a specific material from among the various concrete materials is transferred to the transporting device 104 and transported to the manufacturing device 110.

[0023] The manufacturing apparatus 110 is an apparatus for manufacturing ready-mixed concrete by mixing concrete materials including aggregate. The manufacturing apparatus 110 includes, for example, a storage bottle 111, a measuring bottle 112, a collecting hopper 113, a mixer 114, and a loading hopper 115.

[0024] Storage bottle 111 temporarily stores various types of concrete materials. Various types of concrete materials are transported (conveyed) to storage bottle 111 from material storage yard 101 by transport device 104. Storage bottle 111 is configured to store various types of concrete materials individually. Hereinafter, "concrete material" may be simply referred to as "material". Various materials stored in storage bottle 111 are supplied to measuring bottle 112 as necessary.

[0025] The measuring bottle 112 is disposed below the storage bottle 111. The measuring bottle 112 operates based on an operation command from the control device, and weighs each material individually. When the measuring bottle 112 detects a target amount of material instructed by the control device, it supplies the material to the collecting hopper 113. When water is supplied to the measuring bottle 112, an admixture may be mixed into the water. The collecting hopper 113 is disposed below the measuring bottle 112. The collecting hopper 113 collects each material discharged from the measuring bottle 112, and supplies the collected materials to the mixer 114. Note that the collecting hopper 113 does not have to be provided, and in this case, each material is supplied from the measuring bottle 112 to the mixer 114.

[0026] The mixer 114 is disposed below the collecting hopper 113. The mixer 114 is a device that mixes concrete materials. The mixer 114 produces ready-mixed concrete by mixing (kneading) aggregate, cement, water, admixtures, and the like. The ready-mixed concrete is discharged from the bottom of the mixer 114 into the loading hopper 115. The loading hopper 115 is disposed below the mixer 114, and temporarily stores the ready-mixed concrete. The loading hopper 115 supplies the temporarily stored ready-mixed concrete to the transport vehicle 200.

[0027] The manufacturing system 100 described above is one example of a ready-mix concrete manufacturing system, and the ready-mix concrete manufacturing system may be configured in any manner as long as it is capable of mixing concrete materials and manufacturing ready-mix concrete.

[0028] [Quality prediction device] The quality prediction device 1 is a device that predicts the coarse aggregate ratio as the quality of aggregate contained in concrete material. The particle size of aggregate, the quality of which is predicted by the quality prediction device 1, is on a different order from the particle size of soil. The quality prediction device 1 is composed of one or more computers. The computers that constitute the quality prediction device 1 may be a personal computer, a tablet computer (tablet terminal), a smartphone, a wearable device, a workstation, a server computer, or a general-purpose computer. The quality prediction device 1 may be connected to a communication network via a wired or wireless connection.

[0029] The quality prediction device 1 includes, for example, a main body 10, an input device 12, a monitor 14, and a camera 16. The main body 10 is a device (computer main body) that executes the main functions of the computer that constitutes the quality prediction device 1.

[0030] The input device 12 is a device for inputting information to the main body 10. The input device 12 may be any device capable of inputting desired information, and specific examples thereof include operation interfaces such as a keypad, a mouse, and an operation controller.

[0031] The monitor 14 is a device for displaying information output from the main body 10. The monitor 14 may be any device capable of displaying graphics, a specific example of which is a liquid crystal panel. The monitor 14 and the input device 12 may be integrated as a touch panel.

[0032] The camera 16 is a device capable of capturing an image of the aggregate. The camera 16 captures an image of the aggregate to generate image data. The camera 16 is, for example, a digital camera that captures an image of the imaging range based on visible light, or a video camera. The image data generated by the camera 16 is used to predict the quality of the aggregate. When the camera 16 generates a video of the imaging target, still images included in the video may be acquired as image data.

[0033] The camera 16 may be disposed in the manufacturing system 100 so as to be able to capture an image of an aggregate at a specific location. For example, the camera 16 may be disposed so as to be able to capture an image of an aggregate being transported by the transport device 104. The camera 16 may be portable, and a user such as a worker may carry the camera 16 to an imaging position. The camera 16 may be mounted on a mobile body such as a drone, and the camera 16 may capture an image at any location in the manufacturing system 100 by a user operating the mobile body, etc.

[0034] In the quality prediction device 1, the main body 10, the input device 12, the monitor 14, and the camera 16 may be integrated into one device, such as a smartphone with a built-in (mounted) camera or a tablet computer with a built-in camera. In one example, the quality prediction device 1 is configured by installing an application for predicting the quality of aggregate on a smartphone.

[0035] As shown in FIG. 2, the main body 10 includes a circuit 50. The circuit 50 includes a processor 51, a memory 52, a storage 53, and an input / output port 54. The storage 53 is configured with one or more non-volatile memory devices such as a flash memory or a hard disk. The storage 53 stores a quality prediction program that causes a computer to execute at least an image acquisition step, a distance acquisition step, an image correction step, and a prediction step. The quality prediction program may further cause a computer to execute a specification step and a result display step. Details of these steps will be described later. The storage 53 stores a quality prediction program for configuring each functional block of the main body 10.

[0036] The memory 52 is composed of one or more volatile memory devices such as a random access memory. The memory 52 temporarily stores a quality prediction program loaded from the storage 53. The processor 51 is composed of one or more arithmetic devices such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The processor 51 configures each functional block of the main body 10 by executing the quality prediction program loaded into the memory 52. ​​The calculation results by the processor 51 are temporarily stored in the memory 52. ​​The input / output port 54 inputs and outputs information between the input device 12, the monitor 14, the camera 16, etc. in response to a request from the processor 51.

[0037] The quality prediction program may be provided in a state of being fixedly recorded on a tangible recording medium such as a CD-ROM, a DVD-ROM, or a semiconductor memory. Alternatively, the quality prediction program may be provided via a communication network as a data signal superimposed on a carrier wave. The main body 10 (quality prediction device 1) may be composed of a plurality of computers communicably connected to each other. The main body 10 (quality prediction device 1) may be a part of a control device included in the manufacturing system 100.

[0038] 3 shows an example of functional components (referred to as "functional blocks" in this disclosure) included in the main body 10. The main body 10 includes, as functional blocks, a user input acquisition unit 22, an image data acquisition unit 24, a distance data acquisition unit 38, an evaluation information acquisition unit 26, a prediction model holding unit 28, a prediction calculation unit 30, and a display output unit 32. The processes executed by these functional blocks correspond to the processes executed by the main body 10 (quality prediction device 1).

[0039] The user input acquisition unit 22 is configured to acquire user input information indicating an input from a user required for predicting the coarse grain ratio of the aggregate to be evaluated. The user input acquisition unit 22 may acquire the user input information based on a user input to an interface screen displayed on the monitor 14.

[0040] The image data acquisition unit 24 is configured to acquire image data relating to a captured image PI generated by capturing an image of the aggregate to be evaluated (hereinafter, referred to as "captured image data"). The image data acquisition unit 24 acquires captured image data (captured image PI) captured by the camera 16.

[0041] The distance data acquisition unit 38 acquires distance data indicating the distance between the aggregate to be evaluated and the camera 16 when the aggregate to be evaluated is imaged (when the captured image PI is obtained). The distance data acquisition unit 38 may acquire the distance data by image processing of the captured image PI, or by a trained model constructed by machine learning so as to output information indicating the distance from the camera in response to an input of an image. The distance data acquisition unit 38 may acquire the distance data from a measurement result by a measuring device 19 (see FIG. 1 ) provided next to the camera 16. Examples of the measuring device 19 include depth sensors such as a TOF camera, stereo vision, and structured light, and distance sensors such as a laser displacement meter and an ultrasonic sensor.

[0042] The evaluation information acquisition unit 26 acquires a corrected image EI by correcting the captured image PI based on the distance data, and acquires evaluation input information based on the corrected image EI. The evaluation information acquisition unit 26 functions as an image correction unit that corrects the captured image PI to acquire the corrected image EI. The correction of the captured image PI will be described later. The evaluation information acquisition unit 26 may acquire, for example, image data related to the corrected image EI as is as the evaluation input information, or may acquire information obtained by processing or adding data to the image data related to the corrected image EI as the evaluation input information.

[0043] The prediction model holding unit 28 is configured to hold a model for predicting the coarse grain ratio of the aggregate to be evaluated (hereinafter, referred to as the "prediction model M"). The prediction model M is a model constructed to output a predicted value of the quality value in response to input of the input information by machine learning based on input information based on learning image data (learning image data) obtained by imaging the learning aggregate and a ground truth value of a quality value indicating the quality of the aggregate associated with the input information. The quality value is, for example, the coarse grain ratio, the mass fraction for each grain size division included in the grain size division of the aggregate, or both the coarse grain ratio and the mass fraction for each grain size division.

[0044] Machine learning refers to a method in which a machine (computer) autonomously finds laws or rules by repeatedly learning based on given information. The prediction model M can be constructed using an algorithm and a data structure. The prediction model M is realized, for example, using a neural network, which is an information processing model that mimics the mechanism of the human brain. There is no particular limitation on the specific algorithm of the machine learning performed when constructing the prediction model M. The neural network has an input layer, one or more intermediate layers, and an output layer. By including one or more intermediate layers, a more complex prediction model M can be constructed, and prediction accuracy can be improved.

[0045] The prediction model M may be constructed by machine learning using a convolutional neural network (CNN). The prediction model M is constructed by machine learning based on the correct value of the quality value. The prediction model M may output only the coarse grain ratio, only a value indicating the mass fraction for each grain size division included in the multiple grain size divisions, or both the coarse grain ratio and the mass fraction for each grain size division, as the quality value, according to the input information. When outputting the mass fraction for each grain size division, the prediction model M outputs multiple values ​​each indicating a mass fraction for the multiple grain size divisions. The following illustrates an example in which the prediction model M outputs both the coarse grain ratio and the mass fraction for each grain size division.

[0046] The prediction model M is constructed to predict the above quality value, for example, by performing machine learning using data provided as input for machine learning and correct answer data (correct answer value of the above quality value) of the machine learning output. The input for the machine learning is various data sets of input information based on learning image data obtained by imaging the learning aggregate. In the various data sets of input information, at least the image data included in the input information is different from each other. The output of the machine learning is data (numerical value) indicating a quality value such as the coarse grain ratio. When constructing the prediction model M, a model that outputs a predicted value of the above quality value is iteratively trained using multiple combinations of input information and correct answer values ​​such as the coarse grain ratio.

[0047] The stage in which the prediction model M is autonomously constructed corresponds to the learning phase. The learning phase may be performed before the production phase in which ready-mix concrete is manufactured in the manufacturing system 100, or may be performed at an early stage of the production phase. The prediction model M, which is a trained model, is portable between computers. Therefore, the prediction model M constructed in another device may be stored in the prediction model holding unit 28. The stage in which the quality prediction device 1 predicts the coarse particle ratio of the aggregate to be evaluated using the prediction model M corresponds to the evaluation phase.

[0048] The prediction calculation unit 30 is configured to calculate the coarse aggregate ratio of the aggregate to be evaluated based on the evaluation input information and the prediction model M. For example, the prediction calculation unit 30 inputs the evaluation input information to the prediction model M and obtains a predicted value output from the prediction model M. For example, the prediction calculation unit 30 obtains a predicted value indicating the coarse aggregate ratio from the prediction model M. The prediction calculation unit 30 may also obtain a plurality of predicted values ​​indicating mass fraction values ​​for a plurality of particle size divisions from the prediction model M. The prediction calculation unit 30 may calculate the coarse aggregate ratio from a predicted value (a plurality of predicted values) regarding the mass fraction for each particle size division. For reliability evaluation, the prediction calculation unit 30 may obtain both the coarse aggregate ratio and a plurality of predicted values ​​each indicating a mass fraction value from the prediction model M and compare the coarse aggregate ratio predicted for each prediction model M. As described above, the calculation (prediction) of the coarse aggregate ratio is performed by two different methods using the prediction model M, so that the reliability of the predicted value of the coarse aggregate ratio can be evaluated.

[0049] The display output unit 32 displays the prediction result by the prediction calculation unit 30 on the monitor 14. The display output unit 32 may display the predicted value of the coarse grain rate calculated by the prediction calculation unit 30 on the monitor 14. The display output unit 32 may display the distance data and the corrected image EI obtained from the captured image PI on the monitor 14 before the prediction calculation unit 30 executes the prediction.

[0050] <Prediction model M> Here, the prediction model M used to predict the coarse grain ratio will be further described with reference to Figs. 4 and 5. In the learning phase for constructing the prediction model M, aggregate of the same type as the aggregate that is used in the manufacturing system 100 and can be evaluated is used as the learning aggregate. Fig. 4(a) shows a schematic diagram of the calculation process for predicting the coarse grain ratio using the prediction model M from the image Im, which is input information. The input of the prediction model M is the image Im, and the output of the prediction model M is the predicted value of the mass fraction of each of the multiple grain size classes and the predicted value of the coarse grain ratio.

[0051] When constructing the prediction model M, learning data for machine learning is prepared. The learning data includes various image data (input information) obtained by imaging the learning aggregate, and answer data associated with each of the various image data. In preparing the learning images, for example, various aggregates whose mass fractions for each granularity classification are known are prepared. The above various aggregates do not mean different types of aggregate, but mean the same type of aggregates with different granularities.

[0052] In one example, the various types of learning aggregates are individually imaged by the camera 130 in a state where they are stacked on the measurement sheet 132 (see FIG. 4(b)). The stacked state (stacked state) refers to a state where some particles contained in the aggregates overlap other particles when viewed vertically from above. By individually imaging the various types of aggregates by the camera 130, learning image data is obtained for each of the various types of aggregates. The camera 130 may be a digital camera, and captures an image of an area including the stacked aggregates from a position a predetermined distance away from the aggregates on the measurement sheet 132 (for example, a predetermined position above). The quality prediction device 1 (for example, the evaluation information acquisition unit 26) stores information indicating the distance between the camera 130 and the learning aggregates when the learning image data is obtained as a reference imaging distance.

[0053] The sheet 132 may be a rubber sheet. As shown in FIG. 4(b), the sheet 132 may be a sheet formed to be curved in imitation of the belt conveyor of the conveying device 104, or may be a sheet formed in a flat shape, unlike the example shown in FIG. 4(b). The amount of aggregate placed on the sheet 132 is not particularly limited, but may be about 1 kg to 10 kg. FIG. 4(b) shows a schematic example of aggregate piled up on the sheet 132, and shows piled up states when the amount of aggregate is 2 kg, 5 kg, and 8 kg. Note that the sheet 132 may be a container other than a rubber sheet. If it is a container with a rim, it is preferable from the viewpoint that it can be used to uniformly adjust the size of the captured aggregate by correcting the size of the image data based on the distance calculation result after calculating the distance from the camera to the aggregate.

[0054] In one example, samples of various aggregates are prepared under the conditions shown in Table 1 below, and images for learning are obtained by photographing the prepared samples. The mass fraction and coarse aggregate ratio for each grain size division of the various aggregates prepared under the conditions shown in Table 1 are known, and the known values ​​are the correct answer data for machine learning.

[0055] [Table 1]

[0056] The coarse grain ratio (FM) is calculated, for example, by the following formula (1) specified in the Standard Method for Concrete. In the measurement specified in the "Standard Specifications for Building Construction and Commentary JASS5 Reinforced Concrete Construction", 9 sieves are used excluding the 80 mm sieve in formula (1). The coarse grain ratio predicted in this disclosure may be specified by any method.

number

[0057] In Table 1, this means that aggregates of 10 levels of gradation (coarse aggregate ratio) are prepared. Category I represents a gradation category of particles smaller than 2.5 mm. Category I includes particles that cannot be retained on a 2.5 mm sieve. Category II represents a gradation category of particles 2.5 mm or larger and smaller than 5 mm. Category II includes particles that can be retained on a 2.5 mm sieve but cannot be retained on a 5 mm sieve. Category III represents a gradation category of particles 5 mm or larger and smaller than 10 mm. Category III includes particles that can be retained on a 5 mm sieve but cannot be retained on a 10 mm sieve. Category IV represents a gradation category of particles 10 mm or larger and smaller than 20 mm. Category IV includes particles that can be retained on a 10 mm sieve but cannot be retained on a 20 mm sieve.

[0058] For level 1, the coarse aggregate ratio can be calculated from the mass fractions of each grain size category (mass fractions of each of categories I to IV) as follows. For level 1 aggregate, the mass percentages of the sample that are retained on the 80 mm, 40 mm, and 20 mm sieves are 0, the mass percentages of the sample that are retained on the 10 mm sieve are 85, and the mass percentages of the sample that are retained on the 5 mm sieve are 100 (=15+85). Also, since all the grains are retained on the 5 mm sieve, the mass percentages of the sample that are retained on the 2.5 mm, 1.2 mm, 600 μm, 300 μm, and 150 μm sieves are 100. Therefore, the coarse aggregate ratio is calculated as shown in the following formula (2).

number

[0059] For levels 2 to 10, the coarse aggregate ratio is calculated in the same manner as for level 1. The conditions of the learning aggregate shown in Table 1 are merely examples, and the number of levels, the coarse aggregate ratio and mass fraction for each grading class at each level, and the method of setting the grading classes may be set arbitrarily. As with the multiple grading classes shown in Table 1, the grading classes in multiple classes including the smallest class (<150 μm) may be combined into one grading class. Grain sizes above a certain class (e.g., 20 to 40 mm) may not be included in multiple grading classes. Also, unlike the setting method in Table 1, grain sizes below a certain class (e.g., 1.2 to 2.5 μm) may not be included in multiple grading classes.

[0060] FIG. 5 shows images T1 to T10 obtained by imaging learning aggregates prepared according to the conditions of each of levels 1 to 10 in Table 1. Level n (n is an integer from 1 to 10) corresponds to image Tn. For example, image T1 is obtained by imaging learning aggregates prepared according to level 1. Learning aggregates according to each level n contain grains of multiple grain size divisions. In this way, learning aggregates may be prepared so as to contain grains of two or more grain size divisions, and input information may be prepared based on image data obtained by imaging the learning aggregates.

[0061] For each level n, multiple images Tn in which the states (more specifically, pile states) of the learning aggregate are different from each other may be prepared. In one example, a first image is taken with the learning aggregate placed on the measurement sheet 132, to obtain one image Tn. Then, the learning aggregate is temporarily removed from the sheet 132, and the same learning aggregate (same sample) is placed again on the sheet 132 so that the pile state of the multiple grains is different from that in the first image. Then, a second image is taken. By repeating the above-mentioned image taking, multiple images Tn may be prepared for each level n.

[0062] One data set is composed of one image Tn and the correct answer data of the mass fraction and the coarse grain ratio at the level n associated with the image Tn. As learning data, a plurality of data sets each consisting of a combination of the image Tn and the correct answer data of the mass fraction and the coarse grain ratio are prepared. Then, a prediction model M is constructed by performing machine learning using such learning data. The prediction model M constructed as described above outputs, for example, four mass fraction values ​​and a coarse grain ratio value for four grain size classifications.

[0063] Focusing on the point of outputting a value of the coarse grain ratio, the prediction model M is a model constructed to output a value indicating the coarse grain ratio in response to input of the input information by machine learning based on input information based on learning image data obtained by imaging the learning aggregate and the correct value of the coarse grain ratio associated with the input information. Focusing on the point of outputting a value of the mass fraction for each grain size division, the prediction model M is a model constructed to output a value indicating the mass fraction for each division included in a plurality of grain size divisions in response to input of the input information by machine learning based on input information based on learning image data obtained by imaging the learning aggregate and the correct value of the mass fraction for each of the plurality of grain size divisions of the aggregate associated with the input information.

[0064] In order to verify the method of predicting the coarse aggregate ratio using the prediction model M based on machine learning using a convolutional neural network, a comparison was made between the prediction results by the prediction model M and the correct values ​​using a dataset in which the correct values ​​were known. For the learning aggregate, 10 levels of aggregate with different coarse aggregate ratios were prepared according to the conditions in Table 1 above. For each level, images were taken 10 times indoors with the aggregate piled up on the measurement sheet 132 in different states, to obtain 10 images.

[0065] In addition, a total of 100 images (10 levels x 10 shots) were prepared according to the standard conditions shown in Table 2 below. As a result, 100 data sets were prepared as learning data, each of which corresponds to an image and a correct answer value.

[0066] [Table 2]

[0067] The prediction model M was constructed using 80 of the 100 datasets, and 20 datasets were used for validation. At each level, 8 data were used for model construction, and 2 datasets were used for validation. In the validation using 20 datasets, the prediction results calculated using the image and the prediction model M for each dataset were compared with the correct values ​​such as the coarse grain rate associated with the image. The prediction model M was constructed using a convolutional neural network with 6 intermediate layers consisting of a convolutional layer and a pooling layer, followed by 4 fully connected layers. The Huber function was used as the loss function. In each dataset, the number of images was increased by the Image Augmentation layer.

[0068] FIG. 6(a) is a graph showing the relationship between the predicted values ​​of the mass ratio (%) in the divisions III and IV in Table 1 and the corresponding correct values ​​of the mass fraction (%). In the graph shown in FIG. 6(a), the horizontal axis is the correct value of the mass fraction, and the vertical axis is the predicted value of the mass fraction output from the prediction model M. The prediction results in division III (5 mm to 10 mm) are plotted with △ marks, and the prediction results in division IV (10 mm to 20 mm) are plotted with ◯ marks. A solid line where the correct value and the predicted value match is drawn, and the closer the plotted results are to the line, the higher the prediction accuracy. From the results shown in FIG. 6(a), the prediction results are plotted near the line, and it can be seen that the mass ratio can be predicted with high accuracy by the prediction model M. Note that the coarse particle ratio can be calculated from the mass fraction, so if the mass fraction can be predicted with high accuracy by the prediction model M, the coarse particle ratio can also be predicted with high accuracy.

[0069] FIG. 6(b) is a graph showing the relationship between the predicted value of the coarse grain ratio output from the prediction model M and the correct value of the coarse grain ratio. In the graph shown in FIG. 6(b), the horizontal axis is the correct value of the coarse grain ratio, and the vertical axis is the predicted value of the coarse grain ratio output from the prediction model M. The prediction results are plotted with square marks. The solid line is the line where the correct value and the predicted value match, and a dashed upper limit line shifted vertically by +0.10 from the solid line and a lower limit line shifted vertically by -0.10 are drawn. The prediction results included in the area between the upper limit line and the lower limit line are defined as being within the allowable range. With reference to the error in the coarse grain ratio allowed in current ready-mix concrete plants, a range of ±0.10 was set as the allowable range. It can be seen that all of the prediction results using the prediction model M are included in the allowable range, and the prediction model M can predict the coarse grain ratio with high accuracy.

[0070] [Quality prediction method] Next, a quality prediction method executed in the quality prediction device 1 will be described with reference to Figs. 7 and 8. Hereinafter, the quality prediction method will be described using an example in which the quality prediction device 1 is configured by a smartphone with a camera. This quality prediction method includes, for example, an image acquisition step, a distance acquisition step, an image correction step (evaluation information acquisition step), a prediction step, and a result display step. All of these steps are executed in the evaluation phase. Each of these steps may be executed by a corresponding functional block in the quality prediction device 1. In the learning phase, a prediction model M is constructed, and the distance between the camera and the aggregate when an image of the aggregate is captured to construct the prediction model M is recorded.

[0071] The image acquiring step is a step of acquiring captured image data obtained by imaging the aggregate to be evaluated. In the image acquiring step, a marker member MK (see FIG. 8) of a predetermined size may be imaged together with the aggregate to be evaluated to acquire captured image data. The marker member MK may have any shape as long as it can be distinguished from the aggregate to be evaluated, and has a size that fits within the angle of view of the camera 16 and does not affect the prediction by the prediction model M. The marker member MK may be a block body having a rectangular or columnar shape, or may be a plate shape (for example, a disk shape), or may be the edge of a container that contains the aggregate to be evaluated (a sample thereof), or the edge of a conveyor belt that transports the aggregate to be evaluated.

[0072] The distance acquisition process is a process of acquiring distance data representing the distance between the aggregate to be evaluated and the camera 16 when the aggregate to be evaluated is imaged. In the distance acquisition process, the distance data may be acquired based on, for example, the size of the marker member MK in the captured image PI (the size of the area occupied by the marker member MK in the captured image PI). When the marker member MK is used, the image is not captured by the camera 16 so that the size of the marker member MK in the image is adjusted to a certain standard, but the image is captured without considering the size of the marker member MK in the image. Therefore, the arrangement of the camera 16 when capturing the image of the aggregate to be evaluated can be set more flexibly than when the size of the marker member MK in the image is adjusted when capturing the image. Instead of or in addition to the marker member MK, the distance data may be acquired based on the measurement result of the distance by the measuring device 19. In the distance acquisition process, the distance data may be acquired based on a user input, or the distance data may be acquired according to the position information possessed by the camera 16 itself.

[0073] The image correction process is a process of obtaining a corrected image EI by correcting the captured image PI related to the captured image data based on the distance data. The captured image PI is corrected in accordance with the difference between a reference value of the imaging distance (the imaging distance when the learning phase is executed) and the distance indicated by the distance data, so as to reduce the influence of the difference. That is, even if the grains are the same size, their sizes in the captured image PI (the area that the grains occupy in the captured image PI) differ depending on the distance data. The captured image PI is corrected so as to reduce the difference in grain size in the captured image PI.

[0074] For example, in the image correction process, a range (a part of the image) according to the distance data is extracted from the captured image PI to obtain a corrected image EI. A set of pixel values ​​within the range from the captured image PI is extracted (generated) as the corrected image EI. As the distance indicated by the distance data increases, the range is extracted so as to become smaller, and as the distance indicated by the distance data decreases, the range is extracted so as to become larger. In the image correction process, evaluation input information may be acquired based on the corrected image EI.

[0075] The prediction step is a step of calculating the coarse aggregate ratio of the aggregate to be evaluated based on the prediction model M and evaluation input information based on the corrected image EI. In the prediction step, for example, an evaluation input image based on the corrected image EI is input to the prediction model M, and a predicted value of the coarse aggregate ratio output from the prediction model M is obtained. In the prediction step, an evaluation input image based on the corrected image EI may be input to the prediction model M, and a predicted value of the mass fraction for each particle size division output from the prediction model M may be obtained. Then, the coarse aggregate ratio may be calculated from the predicted value of the mass fraction for each particle size division.

[0076] The result display step is a step of displaying the prediction result of the coarse particle ratio in the prediction step on the monitor 14. In the result display step, at least one of the predicted value of the coarse particle ratio directly output from the prediction model M and the predicted value of the coarse particle ratio calculated from the mass fraction for each particle size class output from the prediction model M may be displayed on the monitor 14.

[0077] 7 shows a process flow illustrating a series of processes executed by the quality prediction device 1. The quality prediction device 1 first executes step S11. In step S11, for example, the image data acquisition unit 24 acquires captured image data related to the captured image PI obtained by capturing an image of the aggregate to be evaluated by the camera 16.

[0078] Next, the quality prediction device 1 executes step S12. In step S12, for example, the distance data acquisition unit 38 acquires distance data indicating the distance between the camera 16 and the aggregate to be evaluated when step S11 is executed based on the captured image PI or based on the measurement result by the measuring device 19. As shown in Fig. 8, the distance data acquisition unit 38 detects the size of the marker member MK in the captured image PI and acquires distance data according to the size. Note that the distance data acquisition unit 38 may acquire information indicating the size of the marker member MK itself as distance data indicating the imaging distance (correlated with the imaging distance).

[0079] Next, the quality prediction device 1 executes step S13. In step S13, for example, the evaluation information acquisition unit 26 corrects the captured image PI based on the distance data acquired in step S12 to generate a corrected image EI. In one example, the evaluation information acquisition unit 26 cuts out a range according to the distance data (for example, the size of the marker member MK in the image) from the captured image PI to generate the corrected image EI.

[0080] 8 illustrates the captured image PI, the size of the marker member MK, and the range (cutout range) of the corrected image EI in each of the cases where the imaging distance is greater than a reference value, where the imaging distance is approximately equal to the reference value, and where the imaging distance is smaller than the reference value. When the corrected images EI in the three cases of different imaging distances are enlarged or reduced so that they have approximately the same size, the difference in size of the aggregate particles in the corrected image EI is reduced more than the difference in particle size in the original captured image PI. The corrected images EI may be images that have been enlarged or reduced so that they have approximately the same size after cutting.

[0081] Next, the quality prediction device 1 executes steps S14 and S15. In step S14, for example, the prediction calculation unit 30 calculates a predicted value of the coarse aggregate ratio of the aggregate to be evaluated based on the evaluation input information based on the corrected image EI obtained in step S13 and the prediction model M. In step S15, for example, the display output unit 32 causes the monitor 14 to display the predicted value of the coarse aggregate ratio obtained in step S14.

[0082] [Variations] The series of processes shown in Fig. 7 is an example and can be changed as appropriate. In the series of processes shown in Fig. 7, the quality prediction device 1 may execute one step and the next step in parallel, may execute some steps in an order different from the example shown in Fig. 7, or may execute steps with contents different from the example shown in Fig. 7.

[0083] The quality prediction device 1 may be a server connected to a terminal device such as a smartphone that can be operated by a user via a communication network. The display output unit 32 included in the quality prediction device 1 may display various information on a monitor of a terminal device that is a device other than the quality prediction device 1. The quality prediction device 1 as a server may exchange information with the terminal device by presenting to the user a Web page or the like that can be viewed and accessed from the terminal device operated by the user. Alternatively, instead of a Web page, an application that cooperates with the quality prediction device 1 may be prepared in the terminal device, and the quality prediction device 1 as a server may provide information to the terminal device.

[0084] The prediction model M described above is an example, and the prediction model M may be any model that can output at least a plurality of predicted values ​​of mass fractions for a plurality of granularity classes or coarse grain ratios from information based on image data of aggregates. The input data to the prediction model M may include, in addition to image data related to the corrected image EI, information indicating physical quantities that may affect the mass fraction or coarse grain ratio. Examples of such physical quantities include the amount of aggregates and the conveying speed of the belt conveyor.

[0085] In one example among the various examples described above, at least a part of the matters described in the other examples may be combined. [Explanation of symbols]

[0086] 1...quality prediction device, 14...monitor, 24...image data acquisition unit, PI...captured image, 26...evaluation information acquisition unit (image correction unit), EI...corrected image, 30...prediction calculation unit, M...prediction model, 38...distance data acquisition unit, MK...marker member.

Claims

1. an image acquiring step of acquiring captured image data obtained by imaging the aggregate to be evaluated; a distance acquisition step of acquiring distance data representing a distance between the aggregate to be evaluated and a camera when the aggregate to be evaluated is imaged; an image correction step of correcting a captured image related to the captured image data based on the distance data to obtain a corrected image; a prediction process for calculating the coarse grain ratio of the aggregate to be evaluated based on input information based on learning image data obtained by imaging an aggregate for training and a prediction model constructed by machine learning based on a correct value of the coarse grain ratio corresponding to the input information, the prediction model being constructed so as to output a value indicating the coarse grain ratio in response to input of the input information, and evaluation input information based on the corrected image; Quality prediction methods.

2. an image acquiring step of acquiring captured image data obtained by imaging the aggregate to be evaluated; a distance acquisition step of acquiring distance data representing a distance between the aggregate to be evaluated and a camera when the aggregate to be evaluated is imaged; an image correction step of correcting a captured image related to the captured image data based on the distance data to obtain a corrected image; a prediction process for calculating the coarse grain ratio of the aggregate to be evaluated based on input information based on learning image data obtained by imaging an aggregate for training and machine learning based on a correct value of the mass fraction of each of a plurality of granularity classes of the aggregate associated with the input information, the prediction model being constructed so as to output a value indicating a mass fraction for each of the plurality of granularity classes in response to input of the input information, and evaluation input information based on the corrected image; Quality prediction methods.

3. The quality prediction method according to claim 1 , wherein in the image correction step, a range of the captured image corresponding to the distance data is extracted, thereby obtaining the corrected image.

4. In the image acquisition step, a marker member of a predetermined size is imaged together with the aggregate to be evaluated, and the captured image data is acquired. The quality prediction method according to claim 1 , wherein in the distance acquisition step, the distance data is acquired based on a size of the marker member in the captured image.

5. an image acquiring step of acquiring captured image data obtained by imaging the aggregate to be evaluated; a distance acquisition step of acquiring distance data representing a distance between the aggregate to be evaluated and a camera when the aggregate to be evaluated is imaged; an image correction step of correcting a captured image related to the captured image data based on the distance data to obtain a corrected image; a prediction step of calculating the coarse grain ratio of the aggregate to be evaluated based on input information based on learning image data obtained by imaging the learning aggregate and a prediction model constructed by machine learning based on a correct value of the coarse grain ratio corresponding to the input information so as to output a value indicating the coarse grain ratio in response to the input of the input information, and on evaluation input information based on the corrected image; A quality prediction program that runs the above on a computer.

6. an image acquiring step of acquiring captured image data obtained by imaging the aggregate to be evaluated; a distance acquisition step of acquiring distance data representing a distance between the aggregate to be evaluated and a camera when the aggregate to be evaluated is imaged; an image correction step of correcting a captured image related to the captured image data based on the distance data to obtain a corrected image; a prediction process for calculating the coarse grain ratio of the aggregate to be evaluated based on input information based on learning image data obtained by imaging an aggregate for training and machine learning based on the corrected values ​​of the mass fractions of the aggregate corresponding to the input information, the prediction model being constructed so as to output a value indicating a mass fraction for each of the plurality of grain size divisions in response to input of the input information, and evaluation input information based on the corrected image; A quality prediction program that runs the above on a computer.

7. an image data acquisition unit that acquires captured image data obtained by imaging the aggregate to be evaluated; a distance data acquisition unit that acquires distance data representing a distance between the aggregate to be evaluated and a camera when the aggregate to be evaluated is imaged; an image correction unit that corrects a captured image related to the captured image data based on the distance data to obtain a corrected image; a prediction model constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for learning and a correct value of the coarse grain ratio associated with the input information, so as to output a value indicating the coarse grain ratio in response to input of the input information, and a prediction calculation unit that calculates the coarse grain ratio of the aggregate to be evaluated based on evaluation input information based on the corrected image; Quality prediction device.

8. an image data acquisition unit that acquires captured image data obtained by imaging the aggregate to be evaluated; a distance data acquisition unit that acquires distance data representing a distance between the aggregate to be evaluated and a camera when the aggregate to be evaluated is imaged; an image correction unit that corrects a captured image related to the captured image data based on the distance data to obtain a corrected image; a prediction model constructed by machine learning based on input information based on learning image data obtained by imaging an aggregate for learning and on a correct value of the mass fraction of each of a plurality of granularity classifications of the aggregate associated with the input information, so as to output a value indicating a mass fraction for each classification included in the plurality of granularity classifications in response to input of the input information, and a prediction calculation unit that calculates a coarse grain fraction of the aggregate to be evaluated based on evaluation input information based on the corrected image; Quality prediction device.

Images