Kneaded body quality prediction model generation method and quality prediction program

Abstract

To provide a kneaded body quality prediction model generation method capable of generating a quality prediction model whose prediction accuracy is further enhanced while suppressing a time required for machine learning, and a quality prediction program for executing quality prediction of the kneaded body by the quality prediction model generated in the same method.SOLUTION: A kneaded body quality prediction model generation method comprises: a step for receiving input of learning input data which includes multiple pieces of time series image data acquired by an imaging part which images a kneaded body which is kneaded by a kneading part and quality related data which is data related to quality of the kneaded body after kneading; a step for determining a main region in the image data acquired by the imaging part for generating main region image data; a step for selecting, from the image data, at least one sub region in which the time series change of luminance is substantially equal to that in the main region, cutting out the sub region, and generating sub region image data; and a step for applying the main region image data, the sub region image data, and the quality related data for executing machine learning.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a quality prediction model generating method and a quality prediction program, and more particularly to a quality prediction model generating method and a quality prediction program for a kneaded body. [Background technology]

[0002] A kneaded body is a mixture of liquid materials such as water, materials such as powder, and solid materials such as aggregates, for example, a mixture of water, hydraulic hardening materials such as cement, fly ash, and blast furnace slag, and aggregates. Specific examples of the kneaded body include fresh concrete, fresh mortar, geopolymer, and fresh paste. The properties required for the kneaded body vary depending on the purpose of use and the environment of the site where it is used. Therefore, in the conventional manufacturing process of the kneaded body, the kneaded state was visually checked based on intuition and experience so that the kneaded body to be manufactured would meet the required properties, and then the product was shipped.

[0003] One of the important characteristics related to the quality of a concrete mixture is the slump value. The slump value is a value used as an index of the consistency of concrete before hardening, and generally, when an unhardened concrete mixture is shipped or unloaded, it is evaluated and inspected to see if its slump value is within the required range.

[0004] The evaluation results and inspection results of the slump value of the kneaded body are the standard for judging whether the manufactured kneaded body can be shipped or used at the unloading site. Therefore, it is expected that the slump value of the kneaded body can be predicted with high accuracy, as far as possible, without depending on the feeling or experience of the worker, before the kneaded body is shipped from the manufacturing plant.

[0005] Therefore, in recent years, in order to predict the slump value of a kneaded body with higher accuracy, a method has been proposed for predicting the slump value of a kneaded body using a prediction model generated by machine learning that applies training data that associates image data of the kneaded body in the kneading section with the slump value (see Patent Document 1 below). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2023-065743 A Summary of the Invention [Problem to be solved by the invention]

[0007] The prediction method described in Patent Document 1 can predict the slump value of a kneaded body with sufficiently high accuracy. Here, the present inventors have been earnestly studying ways to further improve the accuracy and introduce the method to each production base, and have found the following problems.

[0008] It is generally considered preferable that the predictive model used to predict the quality of the kneaded body should be based on machine learning using as much training data as possible in order to find correlations between image data and data related to the quality of the kneaded body and to achieve predictions with higher accuracy.

[0009] Furthermore, it is preferable that the image data used as training data is image data captured or cut out of a part that exhibits relatively large movements during kneading, such that it is relatively strongly associated with data related to the quality of the kneaded body.

[0010] However, the kneading time of the kneaded body is predetermined according to the specifications of the kneaded body, and the time during which the kneaded body can be photographed during kneading is limited. In other words, when the area to be used as image data is specified in advance, the number of image data that can be obtained by photographing the kneaded body during kneading is relatively small.

[0011] In order to solve the above problems, it is conceivable to extract as many areas as possible from the image data captured by the imaging unit and include them in the teacher data. However, when this method is adopted, the movement of the kneaded body is too small, and a large amount of noise-like image data that is hardly associated with data related to the quality of the kneaded body is included in the teacher data.

[0012] In other words, simply increasing the amount of image data used as training data may improve, albeit very slightly, the accuracy of predictions made by the quality prediction model, but it may also have the major disadvantage of requiring large amounts of data processing and an enormous amount of time for machine learning.

[0013] In view of the above problems, the present invention aims to provide a method for creating a quality prediction model of a kneaded body that can create a quality prediction model with improved prediction accuracy while reducing the time required for machine learning, and a quality prediction program that performs quality prediction of a kneaded body using the quality prediction model created by the method. [Means for solving the problem]

[0014] The method for creating a quality prediction model of a kneaded body of the present invention includes the steps of: A step (A) of receiving input of learning input data including a plurality of time-series image data acquired by an imaging unit that captures an image of the kneaded body being kneaded in the kneading unit and quality-related data that is data related to the quality of the kneaded body after kneading; (B) determining a main region in the image data acquired by the imaging unit and generating main region image data; (C) selecting at least one sub-region from the image data, the sub-region having a time-series change in luminance substantially identical to that of the main region, and extracting the sub-region to generate sub-region image data; The method further includes a step (D) of performing machine learning by applying the main region image data, the sub-region image data, and the quality-related data.

[0015] In this specification, the term "image data" is used to include still image data and video data. Video data is a collection of image data acquired at a specific frame rate, and still image data can be acquired by extracting one frame from video data. Therefore, video data and still image data are collectively referred to as "image data."

[0016] In addition, in this specification, the term "time series changes in luminance are substantially the same" is used to refer to a range that is determined to be equivalent using a dedicated application or the like based on parameters such as the distribution of luminance, changes over time, fluctuation range, and correlation with the operation of the kneading section.

[0017] To give specific examples of each, possible parameters that can be used include, with respect to the brightness distribution within a single piece of image data, the average brightness of the entire distribution, the percentage of pixels exceeding a threshold, the percentage of pixels classified into each of the classified brightness levels, the difference between the maximum and minimum brightness values ​​that change due to blending, the frequency of the brightness that changes due to blending, and the period (number of frames) until the brightness increases and decreases and vice versa.

[0018] Then, whether these parameters are equivalent is determined based on the numerical value of the parameter in the main region. Regarding the standard for whether these parameters are equivalent, if a standard is set in advance when using a commercially available application, whether these parameters are equivalent may be determined based on that standard. If there is no preset standard, whether these parameters are equivalent is determined based on whether the error is at least 30% or less, using the numerical value of the parameter in the main region as the standard.

[0019] The following are detailed specific examples that are considered desirable. As an example, in the luminance distribution during one rotation of the mixer shaft, it is desirable that the error in the ratio of pixels classified into classes classified by luminance levels is within 20%. This is because it is desirable that the luminance change in one cycle (one rotation) is equal, since the luminance changes periodically with the rotation of the kneaded body. As another example, it is desirable that the error between the maximum value and the minimum value in the luminance distribution is within 20%. This is because, when there is a portion with extremely high luminance or extremely low luminance, it is relatively likely that a foreign object other than the kneaded body (for example, a component of the kneading unit) is reflected. Note that an example of a specific standard is also shown in the section "Mode for carrying out the invention".

[0020] The mixing section is a component where the mixture is mixed, and includes a housing in which the mixture is housed and a stirring member for mixing the mixture. For example, the mixing section in the case of a mixture containing cement such as fresh concrete corresponds to a mixer in a ready-mixed concrete plant (concrete plant) or a concrete product plant, or an on-site plant installed at a construction site.

[0021] The quality prediction model created by the above-mentioned creation method is not based on learning input data in which only main region image data is associated with quality-related data, but on learning input data in which main region image data, sub-region image data surrounding the main region image data, and quality-related data are associated.

[0022] Here, the sub-region image data is image data in which a luminance change that is substantially the same as a luminance change that appears in the main region image data appears, and the main region is a region that is considered to show a relatively high correlation with data related to the quality of the kneaded body.

[0023] In other words, like the main region, the sub-region is a region that shows a relatively high correlation with data related to the quality of the kneaded body, and like the main region image data, the sub-region image data is considered to be data that is likely to have a relatively strong impact on improving the accuracy of the quality prediction model of the kneaded body.

[0024] Therefore, in the above-mentioned creation method, machine learning is performed using not only the main region image data but also sub-region image data whose relevance to quality-related data can be evaluated as being equivalent to that of the main region image data, so that a quality prediction model with higher prediction accuracy than a quality prediction model created by a conventional creation method can be generated. Furthermore, the above-mentioned creation method does not use image data whose contribution to quality prediction accuracy is unknown or whose contribution is predicted to be extremely small as training data, so that the effect on the processing speed during machine learning is minimized.

[0025] In the above quality prediction model creation method, The step (C) A step (C1) of identifying a structure region in which the structure of the kneading unit is reflected in the image data based on the time-series change in luminance derived in the step (B), The step (B) may be a step of selecting, as at least one sub-region, a region in which the time series change in luminance derived in the step (B) is substantially the same as that of the main region and which is located in the vicinity of the structure region.

[0026] In this specification, the term "vicinity of the structure region" is used to mean any of (1) a portion of both rotating shafts, (2) at least one portion of the tip portions of the multiple agitator blades, or (3) at least one portion of the fixed portions of the rotating shaft and the multiple agitator blades, as well as an area into which the kneaded body or the material of the kneaded body may be reflected.

[0027] According to the above method, even when the material of the kneaded body is scattered and the area where the structure is reflected is difficult to distinguish from the appearance, the area where the structure is reflected can be relatively accurately determined. Moreover, the larger the movement of the kneaded body, the more likely the characteristics of the kneaded body during kneading are to appear. And, the quality prediction model created by the above-mentioned creation method can determine, as the main area, the area near the structure where the kneaded body that moves relatively greatly due to the kneading operation is likely to be reflected.

[0028] In the above quality prediction model creation method, The step (C) may be a step of selecting at least one sub-region in which the time series change in luminance derived in the step (B) is substantially the same as that of the main region and which is spaced at approximately the same distance from the rotation axis of the stirring member provided in the kneading section.

[0029] In this specification, the term "approximately the same distance" is used to mean that the distance between the rotation axis of the stirring member and the sub-region is within the range of 95% to 105% of the distance between the rotation axis of the stirring member and the main region.

[0030] According to the above method, the sub-region image data obtained by photographing the kneaded body in the region that is kneaded in the same manner as the kneaded body located in the main region by the stirring member can be applied to machine learning as training data. Therefore, according to the above method, it is possible to generate a quality prediction model for the kneaded body with improved prediction accuracy while suppressing an increase in the amount of training data.

[0031] The quality prediction program of the present invention is A program executed by a computer to predict the quality of a kneaded body, A step of photographing the kneaded body being kneaded in the kneading section to obtain a plurality of image data; inputting the plurality of image data into the quality prediction model; and outputting quality-related information regarding the quality-related data corresponding to the plurality of image data. Effect of the Invention

[0032] According to the present invention, a method for creating a quality prediction model of a kneaded body is provided that can create a quality prediction model with improved prediction accuracy while reducing the time required for machine learning, and a quality prediction program is provided that performs quality prediction of a kneaded body using the quality prediction model created by the method. [Brief description of the drawings]

[0033] [Figure 1] FIG. 1 is a block diagram showing an overall configuration of an embodiment of a quality prediction model creation system. [Diagram 2] 1 is a diagram showing a schematic view of a scene in a mixer in which a kneaded material is being mixed. [Diagram 3] 1 is a diagram showing a schematic view of a scene in a mixer in which a kneaded material is being mixed. [Figure 4] 1 is a diagram showing a virtual representation of a pixel area for image data of a shooting area. [Diagram 5] 1 is a diagram showing a schematic view of a scene in a mixer in which a kneaded material is being mixed. [Figure 6] 1 is a diagram showing a virtual representation of a pixel area for image data of a shooting area. [Figure 7] 10 is a flowchart showing a processing operation for creating learning input data executed by a calculation processing unit. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034] The method for creating a quality prediction model for a kneaded body of the present invention will be described below with reference to the drawings as appropriate. Note that the drawings are all schematic illustrations, and the numbers on the drawings do not necessarily correspond to the actual numbers.

[0035] The method for creating a quality prediction model of a kneaded body of the present invention is a method for creating a quality prediction model of a kneaded body by performing machine learning based on image data obtained by an imaging unit photographing the kneaded body being kneaded in a mixer corresponding to a kneading unit, and learning input data including quality-related data of the kneaded body. Note that, although the following embodiment will be described on the premise that the quality information data is a slump value, the quality information data may be other information.

[0036] Specifically, the quality prediction data is data including information on the fluidity of the mixture (for example, slump flow and air content). The quality prediction information of fresh concrete in the following description of the embodiment is a slump value, but other information related to the quality of fresh concrete may be adopted. The quality prediction model generated by the quality prediction model creation system 1 of the embodiment described below can capture changes in the state of fresh concrete more accurately, and therefore can adequately predict indices related to the consistency and fluidity of fresh concrete. Furthermore, the quality prediction model generated by the quality prediction model creation system 1 can predict other qualities besides the indices related to the consistency and fluidity of fresh concrete based on the above associations.

[0037] First, the relationship between information extracted from image data (hereinafter referred to as "image extraction information") and the quality of the mixture will be described. As described above, the mixture is a mixture of liquid materials such as water, materials such as powder, and solid materials such as aggregates, for example, a mixture of water and construction materials such as cement, fly ash, and blast furnace slag. Specific examples of the mixture include fresh concrete, fresh mortar, geopolymer, and fresh paste.

[0038] [Image extraction information] The image extraction information is information extracted from image data of the kneaded body being kneaded in the kneading section, and is an element for grasping the viscosity, water content, etc. of the kneaded body and predicting the state of the kneaded body after a predetermined time has passed. Therefore, quality prediction information related to the predicted value of the quality of the kneaded body can be associated with the image extraction information.

[0039] Therefore, by applying a quality prediction model for predicting the quality of a kneaded body generated by machine learning using learning input data including image extraction information and learning output data including quality prediction information, it is possible to predict the quality of the kneaded body to be manufactured.

[0040] Specifically, the image extraction information includes the amplitude of the surface of the kneaded body, the depth to which the kneaded body is sunk by the blades of the stirring member, the size and shape of the aggregate, etc. The image extraction information applied to the quality prediction model may be one of the above image extraction information or a combination of two or more of them.

[0041] The number of samples of teacher data used for training the trained model is preferably 100 or more, more preferably 1,000 or more, even more preferably 10,000 or more, even more preferably 50,000 or more, and particularly preferably 100,000 or more, from the viewpoint of extracting features necessary for deriving slump information from the training input data and further improving the prediction accuracy. Furthermore, the number of training times for each trained model can be, for example, 1,000 or more, but is not particularly limited.

[0042] Specifically, in the case of a kneaded body, it is preferable to obtain image data of multiple batches of the kneaded body and use it as input data for learning, but it is also acceptable to adopt image data of only one batch of the kneaded body as training data.

[0043] Next, the machine learning method used in the quality prediction model creation system of the present invention will be described. Examples of the machine learning method used in the quality prediction model creation system of the present invention include neural networks, linear regression, decision trees, support vector regression, ensemble methods, support vector machines, discriminant analysis, naive Bayes methods, nearest neighbor methods, etc. These methods may be used alone or in combination of two or more.

[0044] Among these methods, machine learning using a neural network is preferably selected from the viewpoint of being able to predict quality with higher accuracy. As the neural network, a hierarchical neural network having one or more intermediate layers between an input layer and an output layer is preferable from the viewpoint of being able to predict quality with higher accuracy.

[0045] Examples of neural networks include convolutional neural networks (CNNs) such as 3D Convolutional Neural Networks (3DCNNs), deep neural networks (DNNs), recurrent neural networks (RNNs), and long short-term memory (LSTM) neural networks (recurrent neural networks improved using LSTM).

[0046] Among these, a three-dimensional convolutional neural network (a neural network having a convolutional layer, a pooling layer, or the like as an intermediate layer) is more suitable, since it has excellent performance in the field of image recognition and is related to the time axis. A three-dimensional convolutional neural network can detect features (including features that change with time) from multiple image data acquired at different times, and create a quality prediction model capable of performing classification or regression using the features. In a convolutional neural network, the number of layers consisting of a combination of a convolutional layer and a pooling layer is preferably two or more, more preferably three or more, from the viewpoint of being able to perform prediction with higher accuracy.

[0047] In addition, as a tool for performing machine learning, for example, "TensorFlow (registered trademark)", a software library developed by Google, or "IBM Watson (registered trademark)", a system developed by IBM, can be used.

[0048] Next, a specific configuration of the quality prediction model creation system 1 will be described.

[0049] The configuration of this embodiment of the quality prediction model creation system 1 of the present invention will be described. Fig. 1 is a block diagram showing the overall configuration of one embodiment of the quality prediction model creation system 1. As shown in Fig. 1, the quality prediction model creation system 1 is composed of a data input unit 10, a calculation processing unit 11, and a model storage unit 12.

[0050] Fig. 2 is a diagram showing a schematic view of a state in the mixer 2 where the kneaded material C1 is being mixed. Fig. 3 is a diagram showing a schematic view of a state in the mixer 2 where the kneaded material C1 is being mixed, as viewed from a different direction from that in Fig. 2.

[0051] In reality, the quality prediction model creation system 1 would capture images of the inside of the mixer 2 during mixing to obtain image data. However, for ease of explanation, Figures 2 and 3 show a schematic diagram of the mixer 2 with mixing stopped so that the condition inside the mixer 2 can be easily confirmed.

[0052] 1, the data input unit 10 of this embodiment accepts input of image data d1 acquired by the imaging unit 20 as learning input data. Then, the data input unit 10 converts the image data d1 into image data d2 applicable to the arithmetic processing unit 11 and inputs it to the arithmetic processing unit 11. If the arithmetic processing unit 11 is configured to be able to process the image data d1 as is, the data input unit 10 may be configured to input the image data d1 as is to the arithmetic processing unit 11 as image data d2.

[0053] In this embodiment, the image data d1 is directly input from the imaging unit 20 to the data input unit 10, but the image data d1 input to the data input unit 10 may be data stored in a memory unit provided in a data server, a cloud server, a smartphone or a PC, or a dedicated storage device.

[0054] 3, the imaging unit 20 is fixed at a position where the height from the liquid surface when stationary is h1. The height h1 is set arbitrarily depending on the shape and size of the mixer 2, and is set within a range of 50 cm to 300 cm, for example.

[0055] The imaging unit 20 may be, for example, a camera that acquires still image data at a predetermined time interval, or a video camera that acquires moving image data. The image data acquired by the imaging unit 20 may be color image data or grayscale image data, as long as the brightness of each pixel or the average brightness in a unit area can be derived. In this embodiment, the imaging unit 20 captures kneading for 20 seconds per batch at a frame rate of 30 fps (flames per second), acquires grayscale moving image data (with brightness levels expressed in the range of 0 to 255), and inputs the image data constituting each frame to the data input unit 10 as image data d1.

[0056] 1, the arithmetic processing unit 11 includes a main region determination unit 11a, a main region image generation unit 11b, a sub-region image generation unit 11c, and a learning unit 11d. Note that the arithmetic processing unit 11 is configured by an arithmetic processing device such as a CPU or an MPU, but does not necessarily have to be configured by one device or IC chip, and may be configured by combining multiple devices or multiple IC chips.

[0057] Fig. 4 is a drawing that virtually represents the main region 21 and the sub-region 22 for image data d2 of the photographed region 20a in Fig. 2. Fig. 5 is a drawing that typically shows a scene in a mixer 2 having a different configuration from that in Fig. 2, and Fig. 6 is a drawing that virtually represents the main region 21 and the sub-region 22 for image data d2 of the photographed region 20a in Fig. 5. For convenience of explanation, Figs. 4 to 6 show image data d2 obtained by photographing a state in which the kneaded body has been removed.

[0058] When the image data d2 is input, the main region determination unit 11a determines a structure region A1 in which the structure of the mixer 2 is reflected. The structure region A1 may be determined simply based on the position of the stirring member 3 reflected in the image data d2, but it is assumed that the material of the kneaded body is scattered and the stirring member 3 in the image data d2 is difficult to determine by appearance. In such a case, for example, the region in which the structure is reflected can be determined by referring to a plurality of time-series image data d2 and referring to the magnitude of the change in brightness over time. As a mere example, the structure region A1 is adopted as the rotation axis 3a of the stirring member 3 as shown in FIG. 4.

[0059] The main region determining unit 11a determines a region in the image data d2 as the main region 21, which is located at a distance L1 away from the rotation axis 3a of the stirring member 3, as shown by a dashed line in Fig. 4, and outputs image data d4 to which information about the main region 21 is added, to the main region image generating unit 11b. Note that this determination process may be performed on image data obtained by cutting out a portion of the image data of the shooting region 20a. Also, the main region 21 may be determined as a pre-specified region, rather than being determined based on the structure region A1 or the position of the structure.

[0060] The main region image generating unit 11b cuts out the region determined as the main region 21 in the image data d4, and outputs the main region image data d3 to the learning unit 11d. After completing generation of the main region image data d3, the main region image generating unit 11b outputs the image data d4 to the sub region image generating unit 11c. Furthermore, the main region image generating unit 11b derives transition data of the luminance change over a predetermined period in the region of the image data d4, and outputs the transition data to the sub region image generating unit 11c.

[0061] The sub-region image generating unit 11c selects a plurality of regions on the image data d4 that are candidates for the sub-region 22. Then, the sub-region image generating unit 11c cuts out each of the selected sub-regions 22 to generate sub-region image data d5. In this embodiment, the candidate regions for the sub-region 22 are selected such that the distance (L2, L3, L4) from the rotation shaft 3a of the stirring member 3, which is a structure, is approximately the same as the distance L1 between the main region 21 and the rotation shaft 3a. This is because it is assumed that the stirring blade 3b of the stirring member 3 operates in the same way, but the positional relationship of each region and structure that is focused on when selecting the candidates for the sub-region 22 is arbitrary.

[0062] Here, even in the mixer 2 having the configuration shown in Figures 5 and 6, if the structural region A1 is set as the rotation axis 3a of the stirring member 3, the main region 21 and candidates for the sub-region 22 can be selected in the same manner, as shown in Figure 6.

[0063] When the sub-region image generating unit 11c determines the candidates for the sub-region 22, it derives transition data of luminance change in a predetermined period for each of the selected candidates for the sub-region 22 in the image data d4, and compares it with the transition data of luminance change of the main region 21 input from the main region image generating unit 11b. Note that the region finally selected as the sub-region 22 may be one region or multiple regions.

[0064] The learning unit 11d executes machine learning based on the main region image data d3 input from the main region image generating unit 11b, the multiple sub region image data d5 input from the sub region image generating unit 11c, and the slump value data d6 of the kneaded body depicted in the image data d1, which is one of the learning input data and is read from the data server 30. The learning unit 11d executes machine learning to generate a quality prediction model M1 and stores it in the model storage unit 12.

[0065] The method of comparing the data on the transition of luminance change in the main region 21 with the data on the transition of luminance change in the sub-region 22 is to check whether or not each parameter calculated from the luminance distribution in the target region within a predetermined period matches at least 80%. The target parameter here is at least one of the maximum luminance value, the minimum luminance value, and the total number of pixels belonging to each range when the full scale of luminance is divided at regular intervals, and all of these parameters are adopted in this embodiment.

[0066] Next, with reference to the drawings, an operation related to the generation of image data (d3, d5) included in the learning input data in the arithmetic processing unit 11 will be described.

[0067] FIG. 7 is a flowchart showing the processing operation for creating learning input data executed by the arithmetic processing unit 11, and more specifically, a flowchart showing the process of creating image data (d3, d5) to be input to the learning unit 11d.

[0068] Image data d1 is input from the imaging unit 20 to the data input unit 10. The step of inputting this image data d1 corresponds to process (A). When image data d2 is input from the data input unit 10 to the calculation processing unit 11, creation of learning input data is started. As shown in FIG. 7, when creation of learning input data starts, the main region determination unit 11a identifies a structure region A1 in the image data d2 (step S1). This step S1 corresponds to process (C1). Note that the structure region A1 here is the region in which the rotation shaft 3a of the stirring member 3 shown in FIG. 2 is captured, as described above.

[0069] After the structure region A1 is specified by step S1, the main region determining unit 11a determines a region in which the kneaded body or the material of the kneaded body may be reflected, together with any one of (1) parts of both rotating shafts, (2) at least one part of the tip parts of the plurality of stirring blades, or (3) at least one part of the fixed parts of the rotating shaft and the plurality of stirring blades, as the main region 21 (step S2). Note that steps S1 and S2 may not be performed, and the main region 21 may be a region previously determined in the image data d1.

[0070] The main region image generating unit 11b cuts out the region determined to be the main region 21 from the image data d4 in step S2, and generates main region image data d3 (step S3). This step S3 corresponds to process (B). Then, the main region image generating unit 11b outputs the main region image data d3 generated in step S4 to the learning unit 11d, and outputs the image data d4 to the sub-region image generating unit 11c.

[0071] Thereafter, the main region image generating unit 11b derives luminance transition data in the time series of the main region 21 on the image data d2 (step S4). The luminance transition data of the main region 21 is output to the sub-region image generating unit 11c together with the image data d4.

[0072] When the sub-region image generating unit 11c receives image data d4 and brightness trend data of the main region 21, it selects each region in the image data d4 that is located at a position whose distance from the rotation axis 3a of the stirring member 3 is approximately the same as distance L1, i.e., the region indicated by the dotted line in Figure 4, as a candidate for the sub-region 22 (step S5).

[0073] Furthermore, the subregion image generating unit 11c derives the luminance transition data of the region selected as a candidate for the subregion 22, just as it derives the luminance transition data of the main region 21 in step S4. Then, it is determined whether the luminance transition data of the main region 21 and the luminance transition data of the region selected as a candidate for the subregion 22 are substantially identical to each other (step S7).

[0074] Whether the luminance transition data is substantially the same or not is determined based on whether the luminance fluctuation range and the luminance distribution are the same. An example will be described in detail below.

[0075] First, the maximum and minimum luminance values ​​in each region are extracted by referring to the time series luminance distribution in the regions selected as candidates for the main region 21 and the sub-region 22. Regarding the luminance fluctuation range, if the maximum luminance value in the main region 21 is 140 and the minimum luminance value is 10, then if the region has a maximum value in the range of 126 to 154 and a minimum value in the range of 9 to 11, it is determined that the luminance fluctuation range is substantially the same and that the region satisfies one of the criteria for the sub-region 22.

[0076] In the description herein, the maximum and minimum luminance values ​​are preferably within ±20%, and more preferably within ±10%, of the maximum and minimum luminance values ​​of the main region 21. In this case, if the cycle (the difference between the frame in which the luminance switches between increasing and decreasing and the next frame in which the luminance switches between increasing and decreasing) is 3 frames, it may be between 2.7 and 3.3 frames.

[0077] Regarding the luminance distribution, the full scale of luminance (for example, 0 to 255) is divided into intervals starting from 0 and 10, and for the regions selected as candidates for the main region 21 and the sub-region 22, the number of pixels belonging to each interval is counted.

[0078] For example, if the number of pixels belonging to each section in the main region 21 is expressed as 500 for the section with brightness 0 to 10, 600 for the section with brightness 11 to 20, and 400... for the section with brightness 21 to 30, then for a region selected as a candidate for the sub-region 22, if the number of pixels belonging to each section is within a range of ±10% of the number of pixels belonging to the corresponding section in the main region 21, such as 450 to 550 for the section with brightness 0 to 10, 540 to 660 for the section with brightness 11 to 20, and 360 to 440..., then the brightness distribution is judged to be substantially identical.

[0079] In the description herein, the number of pixels belonging to each section is preferably within ±20% of the number of pixels belonging to the corresponding section of the main region 21 as a reference, and more preferably within ±10%.

[0080] If at least one of the luminance fluctuation range and the luminance distribution is substantially identical, it may be determined that the time series changes in luminance are substantially identical. However, in this embodiment, if both of the above conditions are satisfied, the time series changes in luminance are deemed to be substantially identical.

[0081] If the number of regions in which the compared luminance transition data is determined to be substantially the same is equal to or greater than a target value, the subregions 22 are cut out from the image data d4 to generate subregion image data d5 (step S8).

[0082] If the number of regions determined to have substantially the same luminance transition data is less than the target value, the process returns to step S5, and a process of selecting regions to be candidates for subregions 22 is performed. Steps S5 to S7 are then repeated until the number of subregions 22 determined to have substantially the same luminance transition data as the luminance transition data of the main region 21 reaches the target value. The target value for the number of subregions 22 can be set arbitrarily.

[0083] These steps S5 to S8 correspond to process (C).The sub-region image generating unit 11c outputs the sub-region image data d5 generated in step S8 to the learning unit 11d.

[0084] Through the above steps, image data (d3, d5) to be input to the learning unit 11d is created.

[0085] [Verification experiment] Here, a verification experiment was conducted using the above-mentioned quality prediction model creation system 1 to confirm whether a quality prediction model created by the above-mentioned creation method can make predictions with the same accuracy as conventional quality prediction models with a smaller number of image data d1. Details of the experiment are described below.

[0086] (Example) The embodiment is a quality prediction model created by the above-mentioned quality prediction model creation system 1. When creating the quality prediction model of the embodiment, information on image data applied to machine learning is as follows. Total number of image data d1: 6,000 (=30fps x 20 seconds x 10 times) Total number of main area image data d3: 6,000 (same number as image data d1) Total number of sub-area image data d5: 12,000 (two images from one image data d1) -Number of data applied to machine learning: 18,000

[0087] (Comparative Example) The comparative example is the same as the quality prediction model of the embodiment, except that it was created by performing machine learning only on the main region image data d3, and the total number of image data d1 is 18,000 so that the number of data applied to the machine learning is the same. Therefore, when creating the quality prediction model of the comparative example, the information on the image data applied to the machine learning is as follows.

[0088] Total number of image data d1: 18,000 (= 30 fps x 20 seconds x 30 times) Total number of main area image data d3: 18,000 (same number as image data d1) Total number of sub-area image data d5: 0 -Number of data applied to machine learning: 18,000

[0089] [result] First, it was confirmed that the accuracy rate of the working example was 96.6% and the accuracy rate of the comparative example was 95.8%, and both achieved the same accuracy. In other words, the quality prediction model creation system 1 requires significantly less image data d1 to achieve high prediction accuracy than the conventional system. In other words, if the total number of image data d1 to be acquired is the same, the quality prediction model creation system 1 can create a quality prediction model that can predict the quality of the kneaded body with higher accuracy than the conventional system.

[0090] As described above, the above-mentioned creation method performs machine learning using not only the main region image data d3 but also sub-region image data d5 whose correlation with quality-related data can be evaluated as being equivalent to that of the main region image data d3, and therefore can generate a quality prediction model with higher prediction accuracy than a quality prediction model created by a conventional creation method.

[0091] In addition, the above-mentioned creation method does not use image data that is unknown or predicted to have a very small contribution to quality prediction accuracy as training data, so that the impact on the processing speed during machine learning is minimized. Furthermore, since less image data is expected to be required for machine learning until the prediction accuracy reaches the target value, this also leads to improved efficiency in generating a quality prediction model.

[0092] The configuration of the quality prediction model creation system 1 described above is merely an example, and the present invention is not limited to the illustrated configurations. For example, the quality prediction model creation system 1 may be realized by installing a quality prediction model creation program that executes the above-mentioned steps (S1 to S8) in a computer such as a PC. The quality prediction model creation program may be configured to cause a computer to execute the steps required by the quality prediction model creation system 1 among the above-mentioned steps (S1 to S8).

[0093] Furthermore, a computer can be configured to configure a program for predicting the quality of a mixed material by utilizing the quality prediction model created by the above-mentioned quality prediction model creation system 1. By way of example only, when installed in a computer, the program is configured to cause the computer to execute a series of operations, such as photographing a mixed material being mixed in a mixing section of a ready-mix concrete manufacturing plant or the like to obtain multiple pieces of image data, inputting the multiple pieces of image data into the quality prediction model created by the quality prediction model creation system 1, and outputting quality-related information regarding the quality-related data corresponding to the multiple pieces of image data.

[0094] It should be noted that the terms "executed by a computer" and "installed on a computer" mentioned here do not only refer to a case where one terminal executes all processing on its own, but also to a complex system that can be essentially regarded as a single computer, consisting of multiple terminals connected via a network. [Explanation of symbols]

[0095] 1: Quality prediction model creation system 2: Mixer 3: Stirring member 3a: Rotation axis 3b: Stirring blade 10: Data entry section 11: Processing section 11a: Main area determination part 11b: Main area image generation unit 11c: Sub-area image generation unit 11d: Learning Department 12: Model storage section 20: Imaging section 20a: Shooting area 21 : Main area 22 : Sub area 30: Data Server A1: Structure area C1: Kneaded body M1: Quality prediction model

Claims

1. A step (A) of receiving input of learning input data including a plurality of time-series image data acquired by an imaging unit that captures an image of the kneaded body being kneaded in the kneading unit and quality-related data that is data related to the quality of the kneaded body after kneading; A step (B) of determining a main region in the image data acquired by the imaging unit and generating main region image data; (C) selecting at least one sub-region from the image data, the sub-region having substantially the same time series change in luminance as the main region, and extracting the sub-region to generate sub-region image data; A method for generating a quality prediction model of a kneaded body, comprising a step (D) of performing machine learning by applying the main region image data, the sub-region image data, and the quality-related data.

2. The step (C) A step (C1) of identifying a structure region in which a structure provided in the kneading unit is reflected in the image data based on a time-series change in luminance derived in the step (B), A method for generating a quality prediction model of a kneaded body as described in claim 1, characterized in that a region in which the time-series change in brightness derived in step (B) is substantially identical to that of the main region and which is located in the vicinity of the structure region is selected as at least one sub-region.

3. The method for generating a quality prediction model of a kneaded body as described in claim 1, characterized in that the step (C) selects as at least one sub-region a region in which the time-series change in luminance derived in the step (B) is substantially the same as that of the main region and which has a distance from the rotation axis of the stirring member provided in the kneading section that is approximately the same as that of the main region.

4. A program for predicting the quality of a kneaded body, which is executed by a computer, A step of photographing the kneaded body being kneaded in the kneading section to obtain a plurality of image data; A step of inputting the plurality of image data into the quality prediction model according to claim 1; and outputting quality-related information regarding the quality-related data corresponding to the plurality of image data.

Images