Mixing state determination device, mixing state determination method, and mixing state determination program

Abstract

To achieve refinement that reduces the variability in judgments due to experience levels and individual differences. [Solution] A kneading state determination device according to one aspect of this embodiment includes an acquisition unit, an estimation unit, and a recommendation generation unit. The acquisition unit acquires an image or video of the material being kneaded, which is the subject of the determination. The estimation unit inputs the image or video of the material being kneaded, and inputs the target image or video to a first trained model that has been trained to estimate a determination result regarding the kneading state of the material, and obtains a determination result regarding the target image or video. The recommendation generation unit generates recommendations regarding kneading based on the determination result.

Description

Technical Field

[0001] The present invention relates to a kneading state determination device, a kneading state determination method, and a kneading state determination program.

Background Art

[0002] In the kneading of concrete, while referring to a moving image taken by a camera of the kneading state inside the mixer, the person in charge of kneading determines whether the desired kneaded state has been achieved. As an index for evaluating the kneading, the slump value is often used. Furthermore, the kneaded concrete is placed in a formwork, and the workability including the consistency and compactness of the concrete is evaluated by the person in charge of placing based on the feeling during placement. As described above, since the kneading and evaluation of concrete are performed by humans, there is a possibility of variation due to experience values and individual differences. Therefore, there is also a method of determining the blending amount of concrete materials using a machine learning model based on the slump value and creating concrete that satisfies the desired quality (see, for example, Patent Document 1 and Patent Document 2).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the workability of concrete is often difficult to evaluate solely by its slump value. When evaluating concrete products, especially concrete with a slump value of around "3-5 cm" used in precast concrete piles—so-called stiff concrete—or so-called ultra-stiff concrete that does not easily deform under its own weight, the degree of voids within the concrete has a greater impact than in ordinary concrete. Therefore, it is difficult to evaluate it solely by its slump value, and there is a problem of quality variability.

[0005] The present invention aims to achieve refinement that reduces variability in judgment due to experience and individual differences. [Means for solving the problem]

[0006] (1) A kneading state determination device according to one aspect of the present invention includes an acquisition unit, an estimation unit, and a recommendation generation unit. The acquisition unit acquires an image or video of a material being kneaded, which is the subject of determination. The estimation unit inputs the image or video of the material being kneaded, and inputs the image or video to a first trained model that has been trained to estimate a determination result regarding the kneading state of the material, and obtains a determination result regarding the image or video. The recommendation generation unit generates recommendations regarding kneading based on the determination result. According to the above configuration (1), the refinement state can be estimated using a trained model, and the user can understand the recommendation of additional steps as needed, thereby achieving refinement that reduces variability in judgment due to experience and individual differences.

[0007] (2) In some embodiments, in (1) above, The acquisition unit acquires feedback information from the user regarding the properties of the material in its previous kneaded state. The kneading state determination device further includes an update unit that updates the recommendation according to the feedback information. According to the above configuration (2), by providing feedback information regarding the properties of the material in its mixed state, including qualitative evaluations by the user, it is possible to suggest additional processes in the mixing of subsequent batches, taking into account the operator's intuition. (3) In some embodiments, in (1) above, The mixing state determination device further includes a determination unit that determines whether or not an additional process is required for mixing based on the determination result. The recommendation generation unit further generates recommendations regarding the additional process. According to the above configuration (3), it is possible to easily determine whether or not additional steps are necessary through recommendations.

[0008] (4) In some embodiments, in the above configuration (1), The recommendation generation unit takes production data relating to the type and amount of the material, operation data of the equipment that performs the mixing of the material, the judgment result, and information regarding the desired mixed state as input data, and inputs at least one of the production data, the operation data, the judgment result relating to the target image, and information regarding the desired mixed state to a second trained model that has been trained with the information of the additional process as ground truth data, in order to obtain the recommendation. According to the above configuration (4), the decisions made by skilled personnel regarding additional processes such as additional mixing and the addition of additional materials, taking into account the proportions of materials and the operating status of the machinery, can be incorporated into the second trained model.

[0009] (5) In some embodiments, in the above configuration (4), The acquisition unit acquires feedback information from the user, which is information regarding the properties of the material in its previous mixed state, and is a relative evaluation based on the user's perception. The update unit updates the parameters of the second trained model again to estimate additional steps that will result in the desired refined state, based on the relative evaluation. According to the above configuration (5), it is possible to output recommendations based on the judgment of a skilled person regarding additional processes such as additional mixing and adding of additional materials, taking into account the proportion of materials used and the operating status of the machine.

[0010] (6) In some embodiments, in any of the above configurations (1)-(5), The recommendation includes information on additional ingredients, the time to add the additional ingredients, and the mixing time. According to the above configuration (6), the user can easily understand the additional materials, the timing of their addition, the additional mixing time, etc., and identify the necessary additional steps.

[0011] (7) In some embodiments, in the above configuration (1), The mixing state determination device further includes a display control unit that displays the determination result and the recommendation. According to the above configuration (7), users can easily view the judgment results and recommendations and grasp the necessary information.

[0012] (8) In one aspect of the present invention, a method for determining the mixing state includes: an acquisition means that acquires an image or video of the material being mixed, which is the subject of determination; an estimation means that inputs the image or video of the material being mixed, to a first trained model that has been trained to estimate the determination result regarding the mixing state of the material, and obtains a determination result regarding the image or video; and a recommendation generation means that generates recommendations regarding mixing based on the determination result. According to the above configuration (8), the refinement state can be estimated using a trained model, and the user can understand the recommendation of additional steps as needed, thereby achieving refinement that reduces variability in judgment due to experience and individual differences.

[0013] (9) The kneading state determination program according to one aspect of the present invention causes a computer to function as an acquisition means for acquiring a target image or a target moving image to be determined, which is an image of the kneaded material in the kneading state; an estimation means for inputting an image or a moving image of the kneaded material and obtaining a determination result regarding the target image or the target moving image by inputting the target image or the target moving image to a first learned model learned to estimate a determination result regarding the kneading state of the material; and a recommendation generation means for generating a recommendation regarding the kneading based on the determination result. According to the above configuration (9), by estimating the kneaded state using a learned model and enabling the user to grasp the recommendations for additional processes as needed, it is possible to achieve kneading with reduced variation in determination due to empirical values and individual differences.

Effect of the Invention

[0014] According to the present invention, it is possible to achieve kneading with reduced variation in determination due to empirical values and individual differences.

Brief Description of the Drawings

[0015] [Figure 1] FIG. 1 is a conceptual diagram showing a kneading state determination system according to the present embodiment. [Figure 2] FIG. 2 is a block diagram showing a kneading state determination device according to the present embodiment. [Figure 3] FIG. 3 is a flowchart showing an operation example of the kneading state determination device according to the present embodiment. [Figure 4] FIG. 4 is a conceptual diagram showing the learning process and the inference process of the first learned model. [Figure 5] FIG. 5 is a conceptual diagram showing the learning process and the inference process of the second learned model. [Figure 6] FIG. 6 is a diagram showing an example of feedback information. [Figure 7] FIG. 7 is a diagram showing an example of a display screen output by the display control unit. [Modes for carrying out the invention]

[0016] Hereinafter, with reference to the drawings, a kneading state determination device, a kneading state determination method, and a kneading state determination program according to embodiments of the present invention will be described in detail. In the following embodiments, parts with the same number perform the same operation, and therefore repeated explanations will be omitted.

[0017] The mixing state determination system according to this embodiment will be described with reference to the conceptual diagram in Figure 1. The mixing state determination system 1 shown in Figure 1 includes a mixing state determination device 10, a camera 20, a mixer 21, a control device 22, a staff terminal 23, and a display device 24.

[0018] The mixing state determination device 10 outputs a determination result that estimates the mixing state of the mixer 21, and generates recommendations regarding mixing and recommendations regarding additional steps to reach the desired mixing state by comparing it with the desired state. In this embodiment, the recommendations for additional steps include information such as the type and amount or mixing ratio of additional materials, the additional work to be done, and the time required for that work. The mixing state determination device 10 also receives feedback information regarding the properties of the concrete from the operator terminal 23 and reflects it in the mixing of subsequent batches.

[0019] Camera 20 captures the mixing state of the materials in the mixer 21 and transmits the image or video to the mixing state determination device 10. Camera 20 only needs to be capable of acquiring RGB images or monochrome images. Furthermore, camera 20 may also be capable of acquiring depth images in addition to RGB images or monochrome images. Note that camera 20 is not limited to capturing images of the entire mixed material in the mixer 21, but may also capture images or video of a part of the mixer 21, for example, the vicinity of the blades or shaft. In addition, the image or video transmitted to the mixing state determination device 10 may be an image or video of the entire mixing process, or it may be an image or video of a part of the mixing process, for example, a partial image or video after the mixing has stabilized.

[0020] Mixer 21 mixes the concrete materials that have been put in. The concrete materials include cement, coarse aggregate, fine aggregate, water, admixtures, and other admixtures (e.g., fly ash). Admixtures may include those that improve the workability and freeze resistance of the concrete, such as AE (Air Entraining) agents, those that increase strength, such as water-reducing agents, those that adjust the hardening time or setting time, such as hardening accelerators and setting retarders, or those that provide a waterproofing effect, such as waterproofing agents.

[0021] The control device 22 controls the mixer 21 and the material feeding device (not shown) to control the rotation speed of the mixer 21 and the feeding of materials into the mixer 21.

[0022] The operator terminal 23 is, for example, a desktop PC, notebook PC, tablet PC, smartphone, or a dedicated terminal with a display. The operator terminal 23 displays the judgment result regarding the mixing state estimated by the mixing state determination device 10, and input screens for feedback regarding test results or estimated results regarding the physical properties of the finished product, such as slump value, and inspection results of visual inspection. The operator terminal 23 transmits the feedback information entered by the user operating the operator terminal 23 to the mixing state determination device 10. In addition, as long-term feedback, inspection results such as strength tests after mixing may be included as feedback information.

[0023] The display device 24 is a so-called monitor and displays the data output from the mixing state determination device 10.

[0024] Furthermore, communication between the mixing state determination device 10, the camera 20, the mixer 21, the control device 22, the staff terminal 23, and the display device 24 may be conducted via wired or wireless communication. In addition, the control device 22 may be operated manually.

[0025] In the example shown in Figure 1, there is one camera 20, one mixer 21, and one operator terminal 23, but there may be multiple units. Also, the mixing state determination device 10 and the display device 24 may be configured as a single unit, or the mixing state determination device 10 and the control device 22 may be configured as a single unit.

[0026] Next, the details of the kneading state determination device 10 according to this embodiment will be described with reference to the block diagram in Figure 2. The kneading state determination device 10 according to this embodiment includes a processing circuit 11, a storage unit 12, and a communication interface 13, all of which are connected via a bus.

[0027] The processing circuit 11 is a processor such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), NPU (Neural Network Processing Unit), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), or any combination thereof. The processing circuit 11 includes an acquisition unit 111, an estimation unit 112, a determination unit 113, a recommendation generation unit 114, a display control unit 115, and an update unit 116.

[0028] The acquisition unit 111 acquires target images or video images from the camera 20 that capture the mixing state of the material in the mixing stage, which are to be judged. The acquisition unit 111 also acquires feedback information from the user regarding the properties of the material in its mixed state. The feedback information includes items evaluated by the user, such as the person in charge of concrete pouring, from inspections such as compressive strength tests, concrete bending strength tests, and visual inspections of the mixed concrete product.

[0029] The estimation unit 112 uses the first trained model to input a target image or target video and obtains a judgment result regarding the target image or target video. The first trained model is a machine learning model that has been trained to take an image or video of the mixing state of a material as input and estimate a judgment result regarding the finished state of the material.

[0030] Based on the determination result, the decision unit 113 determines whether or not additional steps are necessary to reach the desired mixed state.

[0031] The recommendation generation unit 114 generates recommendations regarding the mixing process based on the judgment result, for example, using a second trained model. The second trained model is a machine learning model that has been trained using production data regarding the type and amount of materials, operation data of the equipment that performs the mixing of the materials, the judgment result, and information regarding the desired mixed state as input data, and information on the additional steps that were performed as ground truth data. The recommendation generation unit 114 also generates further recommendations regarding additional steps depending on whether or not additional steps were performed. The recommendation generation unit 114 or the decision unit 113 may have a function to change the recommendation generation conditions or determination conditions according to the feedback information.

[0032] The display control unit 115 displays the judgment result and recommendations on a display (not shown) or the like.

[0033] The update unit 116 updates the recommendations according to the feedback information and updates any additional steps presented as recommendations as necessary.

[0034] The storage unit 12 consists of an HDD (Hard Disk Drive), an SSD (Solid State Drive), and the like. The storage unit 12 stores sensor data, captured images, inspection data, and judgment results that can be measured in the equipment and environment related to mixing materials.

[0035] The communication interface 13 is an interface for sending and receiving data with the device shown in Figure 1, which conforms to a predetermined communication standard. The communication standard may be a wireless network such as a Wi-Fi® compliant wireless LAN (Local Area Network) or Bluetooth®, or a wired network using signal lines, LAN cables, etc.

[0036] Next, an example of the operation of the kneading state determination device 10 according to this embodiment will be explained with reference to the flowchart in Figure 3. Although a target image is described here, the same processing can be applied to a target video.

[0037] In step SA1, the acquisition unit 111 acquires target images of the concrete material being mixed in the mixer 21, which are captured by the camera 20.

[0038] In step SA2, the estimation unit 112 inputs the target image into the first trained model and estimates a determination result regarding the mixing state of the material.

[0039] In step SA3, the determination unit 113 determines whether the judgment result estimated in step SA2 indicates that the mixture is in the desired state. The determination unit 113 can make this determination at a time before the predetermined mixing time, at a time when additional processes can be carried out. For example, if the mixing time is 120 seconds, the determination in step SA3 can be made 90 seconds after the start of mixing. This allows additional processes to be carried out within 30 seconds after the determination. Alternatively, the determination may be made when the desired mixing time has elapsed. If the mixture is in the desired state, the process proceeds to step SA4. On the other hand, if the mixture is not in the desired state, the process proceeds to step SA5.

[0040] In step SA4, if the decision unit 113 determines that no additional steps are necessary, the recommendation generation unit 114 generates recommendations regarding the mixing process. For example, it may generate recommendations indicating that no additional steps are needed, such as that there are no additional materials to add or that no additional mixing is required. In step SA5, if the decision unit 113 determines that an additional step is necessary, the recommendation generation unit 114 generates a recommendation regarding the additional step required to reach the desired mixing state. Specifically, the second trained model can output information about the additional step required to reach the desired mixing state as a recommendation.

[0041] In step SA6, the display control unit 115 displays the judgment result estimated in step SA2 and the recommendation generated in step SA4 or step SA5 on the display device 24.

[0042] Subsequently, the material is mixed, and the resulting product undergoes visual inspection, compressive strength testing, and bending strength testing.

[0043] In step SA7, the determination unit 113 determines whether or not the acquisition unit 111 has obtained feedback information from the user regarding the inspection. If feedback information has been obtained from the user, the process proceeds to step SA8. On the other hand, if no feedback information has been obtained from the user, the processing for the current batch is terminated in order to process the next batch.

[0044] In step SA8, the update unit 116 updates the additional process based on the acquired feedback information. Specifically, for example, if the feedback information indicates that more strength is desired, the additional process is updated to generate a recommendation that suggests an additional process that will result in a harder mixture. Specifically, the parameters of the second trained model (e.g., weights and biases in the neural network) should be updated again so that a recommendation is generated to reduce the amount of water added or to add a water-reducing agent. Alternatively, the additional process may be updated rule-based in response to the feedback information without updating the second trained model itself. For example, if the initial recommendation is "add 2 kg of water within 10 seconds" and there is feedback indicating an intention to make the mixture harder, the recommendation may be changed to "add 1 kg of water within 10 seconds," updating the content of the additional process by increasing or decreasing the numerical value. In other words, the recommendation regarding the additional process only needs to include information on at least one of the following: the additional material, the time until the additional material is added, and the mixing time. The information regarding mixing time includes the total mixing time for the entire mixing process, or the mixing time after adding additional ingredients.

[0045] Next, an example of the first pre-trained model will be explained with reference to Figure 4. The machine learning model 40 is trained using the captured image 41 as input data and the result 42 of the determination of the kneaded state, which is determined by a user such as a mixing staff member based on the captured image 41, as the ground truth data, thereby generating the first trained model 45. Of course, the captured image 41 may also be a video. The learning method can be general supervised learning, so a detailed explanation is omitted.

[0046] By learning in this way, the image-based judgment of the finished state made by skilled personnel can be incorporated into the first trained model 45.

[0047] The judgment result 42 used as the correct answer data is assumed to be selected from multiple state categories classified according to the hardness of the concrete. For example, if the hardness of the concrete is classified using a five-level index, "A: Hard, B: Slightly Hard, C: Good, D: Slightly Soft, E: Soft", then categories such as "B: Slightly Hard" and "C: Good" can be used as judgment result 42. Note that one index may be subdivided. For example, "B: Slightly Hard" may be subdivided into "Hard," "Medium," and "Soft." Note that hardness is not limited to five levels; it can be classified into multiple levels. Note that the judgment result 42 may also use workability indicators such as consistency and compactibility, in addition to the hardness of the concrete.

[0048] During inference by the first trained model 45, the target image 43 is input to the first trained model 45, and a judgment result 44 for the target image 43 is output.

[0049] Next, an example of a second pre-trained model will be explained with reference to Figure 5. A second trained model 55 is generated by training the machine learning model 50 with the judgment result 44 estimated by the first trained model, classification information 51, production data 52, and operation data 53 as input data, and additional process information 54 provided by users such as mixing personnel as ground truth data.

[0050] Classification information 51 is information regarding the desired mixed state classification. Production data 52 includes data on the proportions of concrete materials and process information related to mixing. Production data 52 may also include environmental data such as temperature, room temperature, and humidity. Environmental data can be obtained from sensor data such as temperature sensors and humidity sensors. The operation data 53 is data related to the operation of equipment that performs mixing of materials, such as the mixer 21 and the material input device. For example, it can be obtained from the control device 22 shown in Figure 1.

[0051] The additional process information 54 includes, for example, information about the content of the additional process and the timing of its implementation, which is determined by the person in charge of mixing the image when the judgment result 42 was obtained. The learning method can be the same as the first trained model, using general supervised learning, so a detailed explanation is omitted.

[0052] By learning in this way, the decisions made by skilled personnel regarding additional processes such as additional mixing and the addition of additional materials, taking into account the proportions of materials and the operating status of the machinery, can be incorporated into the second trained model 55.

[0053] During inference by the second pre-trained model 55, the judgment result 44 for the target image, the classification information 51, the production data 52, and the operation data 53 are input to the second pre-trained model 55, and a recommendation 56 for additional steps based on the state of the judgment result 44 is output. Alternatively, at least one of the judgment result 44 for the target image, the classification information 51, the production data 52, and the operation data 53 may be input to the second pre-trained model 55, and the recommendation 56 may be output.

[0054] Next, an example of feedback information will be explained with reference to Figure 6. As shown in Figure 6, the feedback information 60 is an evaluation value expressed on a multi-level scale (e.g., 5 levels) for quantitative evaluation items such as particle size, weight, density, strength, and viscosity, as well as qualitative evaluation items such as gloss, color tone, unevenness, tactile feel, vibration (e.g., vibration), and falling behavior. In this embodiment, particle size refers to the size of particles when cement and aggregate are bound together by moisture. For tactile feel, for example, human touch and sensation such as being mushy or crumbly can be quantified relatively. For falling behavior, for example, the behavior during impact when falling can be assumed. As a specific example, when pouring using a chute, the sound produced by the particles, especially the pitch, frequency, volume, and proportion of sound produced by contact between particles, can be used as evaluation items. Note that the evaluation items are not limited to these, and items that can be evaluated quantitatively or qualitatively may be added as appropriate. In addition, information regarding the slump value may or may not be included in the feedback information 60. For ease of input, it is assumed that evaluation values ​​will be entered by clicking or touching the corresponding numerical value, but this is not limited to this method; a free-text field may also be provided to allow text input. If text is entered, for example, a trained model capable of interpreting text may be used to convert the intent of the text into an evaluation value, determining which evaluation value corresponds to which text.

[0055] The evaluation value may be an evaluation of the product related to the current batch, or it may be an evaluation value that quantifies the difference between the product related to the current batch and the ideal product, that is, an evaluation value corresponding to feedback such as "make the particles a little finer" or "make it a little harder."

[0056] For example, if feedback information 60 is generated that includes evaluation values ​​corresponding to assessments such as low strength, low yield strength, and being too soft, and evaluation values ​​indicating an intention to make it a little firmer, then the parameters of the second trained model should be updated so that recommendations such as reducing the amount of water added or increasing the amount of water-reducing agent in the mixing process are output.

[0057] Furthermore, the timing for updating the parameters of the second trained model based on feedback information 60 may be adjusted as follows: if negative feedback indicating that the concrete mixture deviates significantly from the desired mixture is received, for example, if the score of an evaluation item falls below a threshold, the parameters of the second trained model may be updated again. Alternatively, if feedback information 60 containing the above-mentioned negative feedback is collected a predetermined number of times or more, the parameters of the second trained model may be updated again. Alternatively, if negative feedback information 60 indicating a difference from the desired mixture is received multiple times consecutively with respect to a specific evaluation item, the parameters of the second trained model may be updated again. In other words, the parameters of the second trained model should be updated while considering the balance between the accuracy of the output of the second trained model and the stability of the model.

[0058] Next, an example of a display screen by the display control unit 115 will be described with reference to Figure 7. Figure 7 shows an example of a display screen 70 that displays the mixing state and recommendations. Here, the desired mixing state category 71, the judgment result 72 regarding the target image, the recommendation 73, and the target image 74 are displayed. Although this example shows each piece of information displayed in one window, the target image 74 and the recommendation 73 may be displayed in separate windows or on separate display devices. Furthermore, since there may not be enough time to decide whether or not to perform additional steps, the recommendation 73 may be displayed in bold, blinking, or other highlighted format to make it easier to distinguish from other information.

[0059] According to the embodiment described above, the estimation unit obtains a determination result regarding the mixing state of the material from the target image using the first trained model, the determination unit determines whether additional steps are necessary to reach the desired mixing state based on the determination result, and the recommendation generation unit generates a recommendation regarding the additional steps if additional steps are necessary. In addition, the acquisition unit obtains feedback information regarding the properties of the material in its mixed state, including qualitative evaluations by the user, and the update unit updates the recommendation based on the feedback information.

[0060] This allows information obtained through the sensory functions of experienced personnel to be incorporated into a trained model, enabling the system to recommend additional processes based on the personnel's intuition during batch mixing after feedback. As a result, the desired mixing state can be achieved without variations in judgment due to experience levels or individual differences.

[0061] In this embodiment, it is assumed that feedback information is reflected in the recommendation, but the mixing state determination device 10 may also automatically incorporate the additional steps that reflect the feedback information into the mixing process. In other words, in the above embodiment, the user can select whether or not to perform the additional steps by displaying the modified additional steps based on the feedback information on the display device 24. On the other hand, if, for example, feedback information is obtained indicating that the strength of the concrete was weak and the user wants to increase the strength, the update unit 116 updates the mixing data to reduce the amount of water mixed in during mixing, or to increase the amount of water-reducing agent mixed in. After that, the mixing process for the materials for the next batch can be automatically executed. Alternatively, the feedback information may be incorporated not only into the mixing process, but also into the formulas used to plan the selection and proportion of materials. In other words, the feedback information may be directly reflected in the selection and proportion of materials in the default mixing process. This allows feedback information to be automatically reflected in the mixing process for the next batch, resulting in more efficient processing.

[0062] Furthermore, while this embodiment assumes the mixing of concrete materials, it may also apply to the mixing of mortar materials or rubber materials. In other words, if additional steps occur during the mixing of materials and it is effective to provide recommendations for that mixing process, the mixing state determination device according to this embodiment can be similarly applied.

[0063] The instructions shown in the processing procedure described in the above-described embodiment can be executed based on a software program. A general-purpose computer system can also obtain the same effect as the identification device described above by pre-storing this program on a recording medium and reading the stored program. Furthermore, the recording medium in this embodiment is not limited to a medium independent of the computer or embedded system, but also includes a recording medium on which a program transmitted via a LAN, the Internet, etc., has been downloaded and stored or temporarily stored. [Explanation of symbols]

[0064] 1…Mixing state determination system, 10…Mixing state determination device, 11…Processing circuit, 12…Storage unit, 13…Communication interface, 20…Camera, 21…Mixer, 22…Control device, 23…Person in charge terminal, 24…Display device, 40, 50…Machine learning model, 41…Captured image, 42, 44, 72…Determination result, 43, 74…Target image, 45…First trained model, 51…Classification information, 52…Production data, 53…Operation data, 54…Additional process information, 55…Second trained model, 56…Recommendation, 60…Feedback information, 70…Display screen, 71…State classification, 73…Recommendation, 111…Acquisition unit, 112…Estimation unit, 113…Decision unit, 114…Recommendation generation unit, 115…Display control unit, 116…Update unit

Claims

1. An acquisition unit that acquires an image or video of the material in a mixed state to be judged, An estimation unit takes the target image or target video as input to a first trained model that has been trained to take an image or video of the material in a mixed state as input and estimate a determination result regarding the mixed state of the material, and obtains a determination result regarding the target image or target video. A recommendation generation unit generates recommendations regarding the mixing process based on the aforementioned determination results, A kneading state determination device equipped with the following:

2. The acquisition unit acquires feedback information from the user regarding the properties of the material in its previous mixed state. The kneading state determination device according to claim 1, further comprising an update unit that updates the recommendation in accordance with the feedback information.

3. The system further comprises a determination unit that determines whether or not an additional process is necessary for the mixing process based on the aforementioned determination result. The kneading state determination apparatus according to claim 1, wherein the recommendation generation unit further generates recommendations relating to the additional steps.

4. The kneading state determination device according to claim 1, wherein the recommendation generation unit takes production data relating to the type and amount of the material, operation data of the equipment that performs the kneading of the material, the judgment result, and information relating to the desired kneaded state as input data, and inputs at least one of the production data, the operation data, the judgment result relating to the target image, and the information relating to the desired kneaded state to a second trained model which has been learned using information of the additional process as correct answer data, in order to obtain the recommendation.

5. The acquisition unit acquires feedback information from the user, which is information regarding the properties of the material in its previous mixed state, and which is a relative evaluation based on the user's perception. The kneading state determination device according to claim 4, further comprising an update unit that updates the parameters of the second trained model to estimate additional steps to achieve the desired state based on the relative evaluation.

6. The kneading state determination device according to any one of claims 1 to 5, wherein the recommendation includes information on additional materials, the time until the additional materials are added, and the kneading time.

7. The kneading state determination device according to claim 1, further comprising a display control unit for displaying the determination result and the recommendation.

8. The acquisition method acquires an image of the material being mixed, which is the target image for judgment. The estimation means inputs the target image to a first trained model that has been trained to take an image of the mixing state of the material and estimate a determination result regarding the mixing state of the material, obtains a determination result regarding the target image, The recommendation generation means generates recommendations regarding the refining process based on the determination result. Method for determining the mixing state.

9. Computers, A means for acquiring an image of the material being mixed, which is the target image to be judged, An estimation means for inputting the target image and obtaining a determination result regarding the target image to a first trained model that has been trained to take an image of the mixing state of the material as input and estimate a determination result regarding the mixing state of the material; A mixing state determination program that functions as a recommendation generation means for generating recommendations regarding mixing based on the aforementioned determination results.

Images