Aggregate determination device and aggregate determination method

Abstract

To provide an aggregate determination device and an aggregate determination method that can determine whether an aggregate introduced to a conveyor matches aggregate types input to a control panel.SOLUTION: An aggregate determination device 50 comprises a control panel 51, a camera 53, and an information processing unit 56. The camera 53 photographs an aggregate 9 conveyed by a reception conveyor 20. An operator inputs, to the control panel 51, information on the aggregate type of the aggregate 9 introduced to the reception conveyor 20. The information processing unit 56 inputs an image of the aggregate 9 obtained through the photographing performed by the camera 53 to a learning model generated through machine learning. Consequently, a result of estimation of the aggregate type is output from the learning model. The information processing unit 56 determines whether the information on the aggregate type input to the control panel 51 and the estimation result output from the learning model match each other.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an aggregate determination device and an aggregate determination method for determining the suitability of aggregates input to a conveyor.

Background Art

[0002] Conventionally, concrete plants are provided with a plurality of silos for storing aggregates that are raw materials for concrete. Aggregates transported by truck from a quarry are conveyed by a conveyor within the concrete plant and stored in designated silos for each aggregate type. A conventional concrete plant is described, for example, in Patent Document 1.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Aggregates are classified into a plurality of aggregate types according to characteristics such as particle size, color, and place of origin. In a concrete plant, a silo for storage is determined for each aggregate type. When an aggregate is input from a truck to a conveyor, the driver of the truck or a predetermined worker inputs the aggregate type to an operation panel. Thereby, the conveyance destination of the aggregate by the conveyor is set to a silo corresponding to the aggregate type.

[0005]

[0006] The present invention has been made in view of such circumstances, and an object thereof is to provide an aggregate determination device and an aggregate determination method capable of determining whether the aggregate input to the conveyor matches the aggregate type input to the operation panel.

Means for Solving the Problems

[0007] In order to solve the above problems, a first invention of the present application is an aggregate determination device for determining the suitability of an aggregate input to a conveyor, comprising: a camera for photographing the aggregate conveyed by the conveyor; an operation panel to which information on the aggregate type is input; and an information processing unit communicably connected to the camera and the operation panel, wherein the information processing unit inputs an image of the aggregate obtained by photographing with the camera into a learning model generated by machine learning, and outputs an estimated result of the aggregate type from the learning model, and a determination unit for determining whether the aggregate type input to the operation panel matches the estimated result.

[0008] A second invention of the present application is the aggregate determination device of the first invention, wherein the camera photographs a moving image of the aggregate conveyed by the conveyor, the moving image includes a plurality of frame images having blurring in the conveying direction of the aggregate, and the information processing unit inputs the frame images included in the moving image photographed by the camera into the learning model.

[0009] A third invention of the present application is the aggregate determination device of the second invention, wherein the information processing unit further has a learning unit for generating the learning model by performing machine learning using, as teacher data, a plurality of frame images included in a moving image of a sample aggregate prepared for learning and the known aggregate type of the sample aggregate.

[0010] A fourth invention of the present application is the aggregate determination device of the third invention, wherein the learning unit performs the machine learning after performing image processing other than rotation on the frame image.

[0011] The fifth invention of the present application is the aggregate determination device of the third invention or the fourth invention, wherein the operation panel can input correct / incorrect information indicating whether the determination result by the determination unit is correct, and the learning unit uses the image of the aggregate obtained from the camera, the estimation result, and the correct / incorrect information as teacher data to perform additional machine learning to update the learning model.

[0012] The sixth invention of the present application is the aggregate determination device of any one of the first to fifth inventions, further comprising a light source that irradiates light toward the shooting range of the camera.

[0013] The seventh invention of the present application is the aggregate determination device of the sixth invention, further comprising a hood that covers a part of the upper surface of the conveyor, and the camera and the light source are arranged inside the hood.

[0014] The eighth invention of the present application is the aggregate determination device of any one of the first to seventh inventions, further having a result output unit that outputs the determination result of the determination unit.

[0015] The ninth invention of the present application is the aggregate determination device of any one of the first to eighth inventions, further comprising a moisture sensor that measures the moisture content on the surface of the aggregate conveyed by the conveyor, and the information processing unit has a first learning model generated by performing the machine learning on aggregates with a surface moisture content less than a predetermined value and a second learning model generated by performing the machine learning on aggregates with a surface moisture content greater than or equal to the predetermined value. When the moisture content measured by the moisture sensor is less than the predetermined value, the estimation unit inputs the image of the aggregate taken by the camera into the first learning model and outputs an estimation result of the aggregate type from the first learning model. When the moisture content measured by the moisture sensor is greater than or equal to the predetermined value, the estimation unit inputs the image of the aggregate obtained from the camera into the second learning model and outputs an estimation result of the aggregate type from the second learning model.

[0016] The 10th invention of the present application is an aggregate determination device according to any one of the 1st to 9th inventions, wherein the estimation unit inputs an odd number of images obtained from the camera into the learning model, and outputs an odd number of estimation results corresponding to each of the odd number of images from the learning model. The determination unit compares the aggregate type input to the operation panel with the odd number of estimation results, and makes a determination based on a majority vote of the comparison results.

[0017] The 11th invention of the present application is an aggregate determination device according to any one of the 1st to 10th inventions, wherein the information processing unit further includes a control unit that restricts the conveyance of the aggregate to the silo when the determination unit determines that the aggregate type input to the operation panel does not match the estimation result.

[0018] The 12th invention of the present application is an aggregate determination method for determining the suitability of an aggregate input to a conveyor, comprising: a) a step of inputting information on the aggregate type of the aggregate input to the conveyor into an operation panel; b) a step of photographing the aggregate conveyed by the conveyor; c) a step of inputting the image of the aggregate obtained by the photographing in step b) into a learning model generated by machine learning, and outputting an estimation result of the aggregate type from the learning model; and d) a step of determining whether or not the aggregate type input in step a) matches the estimation result.

Effect of the Invention

[0019] According to the 1st to 12th inventions of the present application, it is possible to determine whether or not the aggregate input to the conveyor matches the aggregate type input to the operation panel.

[0020] In particular, according to the second invention of the present application, a moving image of the aggregate is captured, and a frame image having blur in the conveying direction is input into the learning model. The camera for capturing the aggregate is not a high-speed camera with a high frame rate capable of acquiring a frame image without blur in the conveying direction or a high-performance camera with a large number of pixels, but an inexpensive and easily available camera is sufficient. Thereby, the cost of the camera can be reduced. Also, by using an inexpensive and easily available camera, the replacement cost when the camera fails can be suppressed. Therefore, an aggregate determination device excellent in maintainability can be realized.

[0021] In particular, according to the third invention of the present application, a large number of frame images included in the moving image are used as learning images. For this reason, a large number of aggregate images necessary for generating the learning model can be acquired in a relatively short time. Also, a learning model having sufficient estimation accuracy can be created with a frame image having blur. Also, compared with the case of preparing a large number of fine images, the storage capacity of the aggregate determination device can be reduced. That is, a learning model with good estimation accuracy can be generated with an inexpensive device configuration.

[0022] In particular, according to the fifth invention of the present application, the learning model can be updated by performing additional machine learning. Thereby, the estimation accuracy of the aggregate type by the learning model can be maintained at a certain level. Also, by accumulating the aggregate images, estimation results, and correct / incorrect information, additional machine learning can be performed quickly.

[0023] In particular, according to the sixth invention of the present application, by suppressing the variation in brightness, the aggregate type can be estimated more accurately.

[0024] In particular, according to the seventh invention of the present application, by suppressing the influence of external ambient light, the aggregate type can be estimated more accurately.

[0025] In particular, according to the eighth invention of the present application, the determination result of the determination unit can be recognized by the user.

[0026] In particular, according to the ninth invention of the present application, different learning models are used depending on whether the moisture content on the surface of the aggregate is less than a predetermined value or not less than the predetermined value. Thereby, the aggregate type can be estimated with higher accuracy.

[0027] In particular, according to the tenth invention of the present application, a more accurate estimation result can be obtained based on an odd number of images.

[0028] In particular, according to the eleventh invention of the present application, when there is a possibility that inappropriate aggregate is input, the conveyance of the aggregate to the silo can be restricted.

Brief Description of the Drawings

[0029]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Embodiments for Carrying Out the Invention

[0030] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[0031] <1. First Embodiment> <1-1. Configuration of Aggregate Conveyor> FIG. 1 is a diagram showing the configuration of an aggregate conveyor 1 including an aggregate determination device 50. This aggregate conveyor 1 is a device that receives aggregate 9 as a raw material in a concrete plant and conveys the aggregate 9 to a plurality of silos 40. As shown in FIG. 1, the aggregate conveyor 1 includes a receiving hopper 10, a receiving conveyor 20, a shuttle conveyor 30, four silos 40, and an aggregate determination device 50.

[0032] The receiving hopper 10 is disposed above the conveying start position of the receiving conveyor 20. The receiving hopper 10 is a funnel-shaped hopper whose inner diameter gradually decreases downward. The aggregate 9 is transported from a quarry by a truck 90 and is introduced into the upper opening of the receiving hopper 10. The introduced aggregate 9 is temporarily stored in the receiving hopper 10 and is supplied from the lower opening of the receiving hopper 10 to the upper surface of the receiving conveyor 20.

[0033] The receiving conveyor 20 is a mechanism that conveys the aggregate 9 introduced into the receiving hopper 10 obliquely upward. The receiving conveyor 20 has a starting pulley 21, an ending pulley 22, and an annular belt 23 spanned between these pulleys 21 and 22. The starting pulley 21 is located below the receiving hopper 10. The ending pulley 22 is located above the shuttle conveyor 30 described later. The ending pulley 22 is disposed at a position obliquely above and away from the starting pulley 21.

[0034] At least one of the starting pulley 21 and the ending pulley 22 rotates by the power of a motor (not shown). As a result, the belt 23 rotates between the starting pulley 21 and the ending pulley 22 in the direction of the arrow in Fig. 1. The moving speed of the belt 23 is adjusted to an arbitrary speed of, for example, 50 m / min or more. The aggregate 9 fed into the receiving hopper 10 is supplied to the upper surface of the belt 23 near the starting pulley 21. Then, the aggregate 9 placed on the belt 23 moves obliquely upward from the position below the receiving hopper 10 toward the ending pulley 22 as the belt 23 rotates.

[0035] Note that the belt 23 of the receiving conveyor 20 moves continuously at a constant speed. For this reason, the aggregate 9 drawn out from the lower part of the receiving hopper 10 is conveyed while being placed on the upper surface of the belt 23 at a substantially constant height. When the aggregate 9 is conveyed to the position of the ending pulley 22, the aggregate 9 falls onto the shuttle conveyor 30 as the belt 23 reverses.

[0036] The shuttle conveyor 30 is a mechanism for distributing the aggregate 9 conveyed from the receiving conveyor 20 to the four silos 40. The shuttle conveyor 30 is located below the ending pulley 22 of the receiving conveyor 20. The shuttle conveyor 30 has a first pulley 31, a second pulley 32, and an annular belt 33 stretched between these pulleys 31, 32.

[0037] At least one of the first pulley 31 and the second pulley 32 rotates by the power of a motor (not shown). As a result, the belt 33 rotates between the first pulley 31 and the second pulley 32. The aggregate 9 that has fallen from the receiving conveyor 20 and is placed on the belt 33 of the shuttle conveyor 30 is conveyed to any one of the four silos 40 by the rotation of the belt 33.

[0038] Further, the shuttle conveyor 30 is movable along a horizontal plane by a drive mechanism (not shown). As a result, the conveying destination of the aggregate 9 by the shuttle conveyor 30 can be switched to a designated silo 40 among the four silos 40.

[0039] The four silos 40 are storage tanks for the aggregates 9. The aggregates 9 are classified into a plurality of aggregate types according to characteristics such as particle size, color, and place of origin. In this aggregate conveying device 1, a silo 40 to be stored is determined for each aggregate type. In the example of FIG. 1, the aggregates 9 of four aggregate types A to D are separately stored in the four silos 40. Note that the number of silos 40 provided in the aggregate conveying device 1 is not limited to four.

[0040] <Regarding the aggregate determination device> The aggregate determination device 50 is a device for determining the suitability of the aggregates 9 fed into the receiving conveyor 20. The aggregate determination device 50 photographs the aggregates 9 conveyed by the receiving conveyor 20 and estimates the aggregate type based on the obtained image. Then, the aggregate determination device 50 determines whether the estimated result of the aggregate type matches a specified aggregate type. As shown in FIG. 1, the aggregate determination device 50 includes an operation panel 51, a display unit 52, a camera 53, a light source 54, a hood 55, and an information processing unit 56.

[0041] The operation panel 51 is arranged in the vicinity of the receiving hopper 10. The operation panel 51 is electrically connected to the information processing unit 56. The operation panel 51 has a plurality of keys 511, 512. An operator (the driver of the truck 90 or a worker of the concrete plant) presses a key 511 indicating the aggregate type of the aggregates 9 fed into the receiving hopper 10. Thereby, the information of the aggregate type is input from the operation panel 51 to the information processing unit 56. Further, when the information of the aggregate type is input, the shuttle conveyor 30 changes its position so that the silo 40 of the said aggregate type becomes the conveyance destination.

[0042] Also, after the aggregates 9 are fed into the receiving hopper 10, the operator presses a key 512 for instructing the start of conveyance. Then, the operations of the receiving conveyor 20 and the shuttle conveyor 30 are started. Thereby, the fed aggregates 9 are conveyed to the specified silo 40 by the receiving conveyor 20 and the shuttle conveyor 30.

[0043] Note that the operation panel 51 may be configured by a general keyboard, mouse, or touch panel.

[0044] The display unit 52 is arranged near the receiving hopper 10 together with the operation panel 51. The display unit 52 is electrically connected to the information processing unit 56. For example, a liquid crystal display is used for the display unit 52. The display unit 52 displays various information regarding the aggregate determination device 50. For example, the display unit 52 displays the determination result output from the determination unit 63 (to be described later) of the information processing unit 56. That is, the display unit 52 is an example of the "result output unit" in the present invention.

[0045] The camera 53 is an imaging device that photographs the aggregate 9 conveyed by the receiving conveyor 20. The camera 53 is arranged so as to face the upper surface of the belt 23 of the receiving conveyor 20 on the downstream side of the conveying path from the receiving hopper 10. The camera 53 of the present embodiment photographs a moving image of the aggregate 9 conveyed by the receiving conveyor 20. After the operation of the receiving conveyor 20 is started and a predetermined time has elapsed and the leading aggregate 9 passes below the camera 53, the camera 53 starts photographing the moving image. Then, the camera 53 inputs the obtained moving image to the information processing unit 56.

[0046] Note that a sensor for detecting the aggregate 9 may be provided in the middle of the receiving conveyor 20, and the photographing of the moving image by the camera 53 may be started based on the detection signal of the sensor.

[0047] The light source 54 is a part for irradiating light toward the photographing range of the camera 53. For example, an LED lamp is used for the light source 54. However, the light source 54 may be a fluorescent lamp or an incandescent lamp. The light source 54 is always lit when the camera 53 performs photographing.

[0048] The hood 55 is a box-shaped member that covers a part of the upper surface of the receiving conveyor 20. The hood 55 is formed of a light-shielding material. The hood 55 is disposed in the middle of the conveying path of the aggregate 9 by the receiving conveyor 20. The hood 55 has an inlet opening 551 and an outlet opening 552. The aggregate 9 conveyed by the receiving conveyor 20 is carried into the hood 55 through the inlet opening 551 and carried out of the hood 55 through the outlet opening 552.

[0049] The camera 53 and the light source 54 are disposed inside the hood 55. External ambient light is shielded by the hood 55. Thereby, the influence of ambient light during shooting by the camera 53 is suppressed. Also, inside the hood 55, by irradiating light from the light source 54, the aggregate 9 can be illuminated with a certain brightness. Therefore, even when the amount of solar radiation changes due to the season or weather, the aggregate 9 can be photographed with a certain brightness. Also, the change in the shadow of the aggregate 9 due to the position of the sun can be suppressed.

[0050] The information processing unit 56 is a device that performs the learning process and the determination process described later. The information processing unit 56 is composed of a computer having a processor 561 such as a CPU or GPU, a memory 562 such as a RAM, and a storage unit 563 such as a hard disk drive or SSD. A computer program for executing the learning process and the determination process described later is installed in the storage unit 563. Also, the information processing unit 56 is communicably connected to the operation panel 51, the display unit 52, the camera 53, the receiving conveyor 20, and the shuttle conveyor 30 described above.

[0051] FIG. 2 is a block diagram conceptually showing the functions realized in the information processing unit 56. As shown in FIG. 2, the information processing unit 56 has a learning unit 61, an estimation unit 62, a determination unit 63, and a control unit 64. The functions of these units are realized by the computer as the information processing unit 56 operating according to the computer program stored in the storage unit 563.

[0052] The learning unit 61 is a processing unit that generates a learning model M for estimating the aggregate type based on the image of the aggregate 9. The learning unit 61 performs machine learning using the image of the sample aggregate 9 (hereinafter referred to as the "learning image") prepared for learning and the known aggregate type of the sample aggregate 9 as teacher data. As the machine learning algorithm, a convolutional neural network, which is a type of deep learning, is suitable. In the experiments conducted by the present inventors, when a convolutional neural network having a pooling layer for performing blurring processing on the image was adopted in the learning unit 61, a learning model M having sufficient estimation accuracy could be generated even for an image having blur in the conveying direction.

[0053] However, the machine learning algorithm used by the learning unit 61 is not limited to the convolutional neural network, and may be other supervised machine learning algorithms such as neural networks, linear regression, decision trees, and support vector regression.

[0054] The learning unit 61 learns the relationship between the images of a large number of sample aggregates 9 and the aggregate types of the sample aggregates 9. As a result, a learning model M capable of outputting an estimation result of the aggregate type corresponding to the image of the aggregate 9 is generated. The learning unit 61 provides the generated learning model M to the estimation unit 62. The details of the learning process will be described later.

[0055] The estimation unit 62 is a processing unit that estimates the aggregate type from the image of the aggregate 9 using the above learning model M. The estimation unit 62 cuts out an image from the moving image D captured by the camera 53 and inputs it to the learning model M. Then, the learning model M outputs an estimation result E of the aggregate type based on the input image of the aggregate 9. The estimation unit 62 sends the estimation result E output from the learning model M to the determination unit 63.

[0056] The determination unit 63 acquires the information I of the aggregate type input to the operation panel 51. Further, the determination unit 63 acquires the estimation result E of the aggregate type by the estimation unit 62. Then, the determination unit 63 determines whether or not the information I of the aggregate type input to the operation panel 51 matches the estimation result E of the aggregate type by the estimation unit 62. Also, the determination unit 63 outputs the determination result R to the display unit 52. Thereby, on the display unit 52, the determination result R as to whether or not the aggregate type of the aggregate 9 conveyed by the receiving conveyor 20 matches the aggregate type specified by the operation panel 51 is displayed.

[0057] The control unit 64 controls the operation of the receiving conveyor 20 according to the determination result R of the determination unit 63. When the determination unit 63 determines that the information I of the aggregate type input to the operation panel 51 matches the estimation result E of the estimation unit 62, the control unit 64 continues the operation of the receiving conveyor 20. When the determination unit 63 determines that the information I of the aggregate type input to the operation panel 51 does not match the estimation result E of the estimation unit 62, the control unit 64 stops the operation of the receiving conveyor 20.

[0058] <1-3. Regarding the learning process> Subsequently, the learning process executed in the above-described aggregate determination device 50 will be described. FIG. 3 is a flowchart showing the flow of the learning process. This learning process is executed in advance before the determination process of the aggregate 9 described later.

[0059] When performing the learning process, first, a sample aggregate 9 for learning is prepared (step S11). For example, when it is desired to learn four aggregate types A to D, sample aggregates 9 of the four aggregate types are prepared.

[0060] The operator feeds the prepared sample aggregates 9 one by one into the receiving conveyor 20 and operates the receiving conveyor 20. Then, while irradiating light from the light source 54, the camera 53 takes a moving image D of the sample aggregate 9 (step S12). When the shooting of the sample aggregate 9 of one aggregate type is completed, the operator feeds the sample aggregate 9 of the next aggregate type into the receiving conveyor 20. Then, the receiving conveyor 20 is operated, and the moving image D of the sample aggregate 9 is taken in the same manner as above. In this way, the moving images D of the sample aggregates 9 of all the prepared aggregate types are sequentially taken.

[0061] The moving image D taken by the camera 53 is composed of a large number of frame images. For example, when shooting is performed for 5 minutes with a camera 53 having a frame rate of 30 frames / second, the moving image D obtained by the shooting is composed of 9000 frame images. In step S12, for all the prepared aggregate types, such a moving image D (an aggregate of frame images) is taken. Then, the moving image D obtained by the shooting is transmitted from the camera 53 to the information processing unit 56.

[0062] Inside the information processing unit 56, a large number of frame images included in the moving image D are stored in the storage unit 563 as learning images (step S13). As described above, the moving image D includes a large number of frame images. Therefore, the learning unit 61 can prepare a large number of learning images. FIG. 4 is a diagram showing an example of a frame image included in the moving image D. As shown in FIG. 4, due to the relationship between the frame rate of the camera 53 and the moving speed of the belt 23, the frame image is an image having blurring in the conveying direction.

[0063] The learning unit 61 classifies a large number of learning images into teacher images and validation images (step S14). For example, when 36000 learning images are obtained by shooting 4 types of sample aggregates 9 for 5 minutes each, 27000 of the 36000 learning images, which is 3 / 4 of them, are used as teacher images, and the remaining 9000 learning images, which is 1 / 4 of them, are used as validation images. However, the ratio of the teacher images to the validation images does not necessarily have to be 3:1.

[0064] The learning unit 61 uses a set of a teacher image and known aggregate types of the sample aggregate 9 shown in the teacher image as teacher data. Further, the learning unit 61 uses a set of a validation image and known aggregate types of the sample aggregate 9 shown in the validation image as validation data.

[0065] The learning unit 61 performs image processing such as blurring, horizontal flipping, and vertical flipping on the teacher image (step S15). By changing the teacher image through image processing, the range of images to be learned is expanded. Thereby, the accuracy of machine learning can be improved. However, it is desirable that the image processing does not include image rotation processing. As described above, the teacher image is an image having blur in the conveyance direction. If rotation processing is performed, the direction of this blur changes, resulting in a decrease in the accuracy of machine learning. Note that among a large number of teacher images, there may be images on which no image processing is performed.

[0066] The learning unit 61 performs machine learning using a large number of teacher data after image processing (step S16). That is, the machine learning algorithm inputs a large number of teacher images into the learning model M one by one. Then, the parameters of the learning model M are adjusted so that the estimated result of the aggregate type output from the learning model M approaches the aggregate type of the input teacher image. The machine learning algorithm repeats such parameter adjustment processing for the number of prepared teacher data.

[0067] Thereafter, the machine learning algorithm uses a plurality of validation images to confirm the estimation accuracy of the aggregate type by the learning model M (step S17). Specifically, a validation image is input to the learning model M, and it is confirmed whether or not the estimated result of the aggregate type output from the learning model M matches the aggregate type of the input validation image. When the estimation accuracy of the aggregate type reaches a predetermined level (for example, a correct answer rate of 95% or more), the machine learning algorithm ends the learning process. Thereby, one learning model M capable of outputting an estimated result of the aggregate type based on the image of the aggregate 9 is generated.

[0068] In addition, when the estimation accuracy of the aggregate type does not reach the desired level, the processes of steps S12 to S17 described above may be repeated again. Further, after generating the learning model M, an image of a sample aggregate 9 different from the above-described teacher image and validation image is input to the learning model M, and it may be further confirmed whether the estimated result of the aggregate type output from the learning model M matches the aggregate type of the sample aggregate 9 shown in the image.

[0069] Even for the same type of aggregate 9, the surface color differs depending on the dry state, wet state, difference in the crushing location, etc. Therefore, in step S12 described above, for each type of aggregate, videos D of the sample aggregate 9 in the dry state, videos D of the sample aggregate 9 in the wet state, and videos D of the sample aggregate 9 with different crushing locations may be obtained by taking multiple shots.

[0070] Also, among the numerous learning images input to the learning model M, images of the sample aggregate 9 in the dry state, images of the sample aggregate 9 in the wet state, and images of the sample aggregate 9 with different crushing locations may be included. For example, when using 9000 learning images for one type of aggregate, a plurality of images of the sample aggregate 9 in the dry state, wet state, and with different crushing locations may be mixed among the 9000 images. Thereby, the learning unit 61 can learn various states such as the dry state, wet state, and difference in crushing location for each type of aggregate. Therefore, the estimation accuracy of the aggregate type output from the learning model M can be further improved.

[0071] <1-4. Regarding the determination process of aggregates> Subsequently, a process of determining the suitability of the aggregate 9 input to the receiving conveyor 20 using the above learning model M when actually receiving the aggregate 9 will be described. FIG. 5 is a flowchart showing the flow of the determination process.

[0072] When receiving the aggregate 9, first, the aggregate 9 loaded on the truck 90 is put into the receiving hopper 10 (step S21). Subsequently, the operator inputs the information I of the aggregate type of the input aggregate 9 into the operation panel 51 (step S22). For example, by pressing the key 511 of the operation panel 51, the information I of the aggregate type is input. After that, the operator presses the key 512 for instructing the start of conveyance on the operation panel 51. Thereby, the operation of the receiving conveyor 20 is started (step S23).

[0073] The aggregate 9 supplied from the receiving hopper 10 to the upper surface of the belt 23 is conveyed obliquely upward toward the end pulley 22 as the belt 23 rotates. And after the operation of the receiving conveyor 20 is started, when a predetermined time has elapsed and the aggregate 9 passes below the camera 53, the camera 53 captures a moving image D of the aggregate 9 (step S24). Then, the camera 53 transmits the obtained moving image D to the information processing unit 56.

[0074] Inside the information processing unit 56, the estimation unit 62 cuts out one image (frame image) from the above moving image D (step S25). Then, the estimation unit 62 inputs the cut-out image into the learning model M. Then, the learning model M outputs an estimation result E of the aggregate type based on the input image of the aggregate 9 (step S26). The estimation unit 62 sends the estimation result E output from the learning model M to the determination unit 63.

[0075] The determination unit 63 acquires the information I of the aggregate type input to the operation panel 51 in step S22 above. Also, the determination unit 63 acquires the estimation result E output from the learning model M in step S26 above. Then, the determination unit 63 determines whether or not the information I of the aggregate type input to the operation panel 51 and the estimation result E of the aggregate type by the estimation unit 62 match (step S27).

[0076] The determination unit 63 outputs the determination result R to the display unit 52. As a result, the determination result R indicating whether or not the type of the aggregate 9 conveyed by the receiving conveyor 20 matches the type of the aggregate specified by the operation panel 51 is displayed on the display unit 52 (step S28). The operator can recognize the determination result R by checking the display unit 52.

[0077] The control unit 64 checks whether the determination result R of the determination unit 63 in step S27 above is either "match" or "mismatch" (step S29). Then, when the determination result R indicates that the information I of the aggregate type input to the operation panel 51 matches the estimation result E of the estimation unit 62 (step S29: Yes), the control unit 64 continues the operation of the receiving conveyor 20. On the other hand, when the determination result R indicates that the information I of the aggregate type input to the operation panel 51 does not match the estimation result E of the estimation unit 62 (step S29: No), the control unit 64 stops the operation of the receiving conveyor 20 (step S30).

[0078] When the determination result R is "mismatch", there is a possibility that the aggregate 9 of an inappropriate aggregate type has been put into the receiving hopper 10, or that the information I of the aggregate type has been incorrectly input to the operation panel 51. In either case, the aggregate 9 on the receiving hopper 10 may be conveyed to the wrong silo 40. In step S30 above, in such a case, the operation of the receiving conveyor 20 is emergently stopped. This stops the conveyance of the aggregate 9 and prevents the aggregate 9 from being conveyed to the wrong silo 40.

[0079] As described above, this aggregate determination device 50 photographs the aggregate 9 conveyed by the receiving conveyor 20, and uses the obtained image and the learning model M generated in advance by machine learning to estimate the type of the aggregate 9 during conveyance. Then, the aggregate determination device 50 determines whether or not the estimation result E of the aggregate type matches the information I of the aggregate type input to the operation panel 51. Thereby, it is possible to detect that the aggregate 9 may be conveyed to the wrong silo 40.

[0080] In particular, in the aggregate determination device 50 of the present embodiment, the aggregate 9 is photographed without stopping the receiving conveyor 20. Therefore, the image input to the learning model M is not a high-definition image but an image with blur. The camera 53 for photographing the aggregate 9 can use an inexpensive camera that is easy to obtain, rather than a high-speed camera with a high frame rate capable of acquiring a frame image without blur in the conveying direction or a high-performance camera with a large number of pixels. Thereby, the cost of the camera 53 can be reduced. Also, by using an inexpensive camera 53 that is easy to obtain, the replacement cost when the camera 53 fails can be suppressed. Therefore, an aggregate determination device 50 with excellent maintainability can be realized.

[0081] Also, in the aggregate discrimination device 50 of the present embodiment, a convolutional neural network that performs image blurring processing is adopted for generating the learning model M in the learning unit 61. Therefore, the image input to the learning model M does not necessarily have to be a high-definition image. That is, even an image with blur in the conveying direction can obtain a learning model M with high estimation accuracy. Therefore, a learning model M with high estimation accuracy can be obtained while using an inexpensive camera 53 instead of a high-performance camera. The camera 53 may be, for example, a web camera with a frame rate of about 30 frames per second. Also, by suppressing the frame rate and the number of pixels, the storage capacity of the aggregate determination device 50 can be reduced. That is, an aggregate determination device with sufficient determination accuracy can be realized while having an inexpensive device configuration.

[0082] Incidentally, when the frame rate of the camera 53 is 30 frames per second, if the moving speed of the belt 23 of the receiving conveyor 20 is 50 m / min (the moving distance per frame is 2.78 cm / frame) or more, a frame image having blur in the conveying direction can be acquired. Further, when the moving speed of the belt 23 exceeds 120 m / min (the moving distance per frame is 6.67 cm / frame), the frame image obtained by shooting becomes blurred to the extent that the outer shape of the aggregate cannot be discriminated. Therefore, it is advisable to adjust the relationship between the frame rate of the camera 53 and the moving speed of the belt 23 so that the moving distance per frame is in the range of 2.78 cm / frame or more and 6.67 cm / frame or less.

[0083] Further, in the aggregate determination device 50 of the present embodiment, imaging is performed by the camera 53 while irradiating light from the light source 54 inside the hood 55. Thereby, the variation in the brightness of the image input to the learning model M can be suppressed. Therefore, the aggregate type can be estimated with higher accuracy.

[0084] <2. Second Embodiment (Moisture Sensor)> Subsequently, a second embodiment of the present invention will be described. FIG. 6 is a diagram showing the configuration of the aggregate conveying device 1 according to the second embodiment.

[0085] This embodiment is different from the above-described first embodiment in that the aggregate determination device 50 includes a moisture sensor 57. The moisture sensor 57 is a sensor that measures the moisture content on the surface of the aggregate 9 conveyed by the receiving conveyor 20. The moisture sensor 57 is disposed at a position above the receiving conveyor 20 and upstream of the camera 53. Further, the moisture sensor 57 is disposed inside the hood 55 together with the camera 53 and the light source 54.

[0086] For the moisture sensor 57, for example, a near-infrared moisture meter is used. The near-infrared moisture meter irradiates near-infrared rays toward the aggregate 9 and measures the moisture content of the aggregate 9 by measuring the absorbance on the surface of the aggregate 9 based on the amount of reflected light. However, the moisture sensor 57 may be a moisture meter of other types. The moisture sensor 57 is communicably connected to the information processing unit 56. The measurement result of the moisture sensor 57 is transmitted to the information processing unit 56.

[0087] FIG. 7 is a flowchart showing the flow of the learning process in the second embodiment. As shown in FIG. 7, in this second embodiment, after preparing the sample aggregate 9, first, for the sample aggregate 9 in a dry state, the same processing as steps S12 to S17 described above is performed (steps S12A to S17A). Thereby, a first learning model M1 capable of outputting an estimation result E of the aggregate type based on the image of the aggregate 9 in the dry state is generated. Next, for the sample aggregate 9 in a wet state, the same processing as steps S12 to S17 described above is performed (steps S12B to S17B). Thereby, a second learning model M2 capable of outputting an estimation result E of the aggregate type based on the image of the aggregate 9 in the wet state is generated.

[0088] Note that the "dry state" refers to a state where the moisture content on the surface of the aggregate 9 measured by the above moisture sensor 57 is less than a predetermined value. Also, the "wet state" refers to a state where the moisture content on the surface of the aggregate 9 measured by the above moisture sensor 57 is equal to or more than a predetermined value.

[0089] FIG. 8 is a flowchart showing the flow of the determination process in the second embodiment. As shown in FIG. 8, in this second embodiment, similar to the first embodiment, the processes of steps S21 to S25 are performed to acquire an image of the aggregate 9. However, immediately before the aggregate 9 is photographed by the camera 53, the moisture content on the surface of the aggregate 9 is measured by the moisture sensor 57 (step S24A).

[0090] After that, the estimation unit 62 determines whether the aggregate 9 conveyed by the receiving conveyor 20 is in a dry state or a wet state (step S31). If the amount of moisture measured by the moisture sensor 57 is less than the predetermined value described above, the estimation unit 62 determines that the aggregate 9 is in a dry state (step S31: Yes). If the amount of moisture measured by the moisture sensor 57 is greater than or equal to the predetermined value described above, the estimation unit 62 determines that the aggregate 9 is in a wet state (step S31: No).

[0091] When the aggregate 9 is in a dry state, the estimation unit 62 selects the first learning model M1 (step S32). Also, when the aggregate 9 is in a wet state, the estimation unit 62 selects the second learning model M2 (step S33).

[0092] When the first learning model M1 is selected, the estimation unit 62 inputs the image of the aggregate 9 into the first learning model M1. Then, the first learning model M1 outputs an estimation result E of the aggregate type based on the input image (step S26). Also, when the second learning model M2 is selected, the estimation unit 62 inputs the image of the aggregate 9 into the second learning model M2. Then, the second learning model M2 outputs an estimation result E of the aggregate type based on the input image (step S26).

[0093] After that, the aggregate determination device 50 performs the processes of steps S27 to S30 in the same manner as in the first embodiment.

[0094] As described above, in this second embodiment, the aggregate determination device 50 includes the moisture sensor 57. When the aggregate 9 is in a dry state, the first learning model M1 with parameters adjusted for the dry state is used. Also, when the aggregate 9 is in a wet state, the second learning model M2 with parameters adjusted for the wet state is used. By using different learning models M1 and M2 for the dry state and the wet state in this way, the aggregate type can be estimated more accurately.

[0095] <3. Third Embodiment (Majority Voting)> Next, a third embodiment of the present invention will be described. FIG. 9 is a flowchart showing the flow of the determination process in the third embodiment. As shown in FIG. 9, in this third embodiment, similar to the first embodiment, the processes of steps S21 to S24 are performed to obtain the moving image D of the aggregate 9.

[0096] The estimation unit 62 cuts out odd-numbered (odd numbers of 3 or more) images from the above moving image D (step S25). Then, the estimation unit 62 inputs the cut-out odd-numbered images into the learning model M respectively. Then, the learning model M outputs an odd number of estimation results E corresponding to each of the input odd-numbered images of the aggregate 9 (step S26). The estimation unit 62 sends the odd number of estimation results E output from the learning model M to the determination unit 63.

[0097] Subsequently, the determination unit 63 compares whether each of the above odd-numbered estimation results E matches the information I of the aggregate type input to the operation panel 51. Then, the determination unit 63 makes a final determination by a majority vote of the comparison results (step S27). That is, when the number of "matches" is more than the number of "mismatches" among the odd-numbered comparison results, the determination unit 63 sets the final determination result R to "match". Also, when the number of "mismatches" is more than the number of "matches" among the odd-numbered comparison results, the determination unit 63 sets the final determination result R to "mismatch".

[0098] After that, the aggregate determination device 50 performs the processes of steps S28 to S30 in the same manner as in the first embodiment.

[0099] As described above, in this third embodiment, an odd number of estimation results E are output from the learning model M based on the odd-numbered images of the aggregate 9. Then, based on the odd number of estimation results E, it is determined by a majority vote whether the aggregate 9 input to the receiving conveyor 20 matches the aggregate type input to the operation panel 51. Thereby, a more accurate determination result R can be obtained.

[0100] Note that the aggregate determination device 50 may have three or more odd-numbered cameras 53. Then, one image may be cut out from each of the videos D captured by the odd-numbered cameras 53 to obtain an odd number of images.

[0101] <4. Fourth Embodiment (Additional Learning)> Next, a fourth embodiment of the present invention will be described. In this fourth embodiment, while performing the above-described determination process using the once-generated learning model M, additional machine learning is appropriately performed, which is different from the first embodiment above. FIG. 10 is a flowchart showing the flow of processing when performing additional machine learning.

[0102] In the example of FIG. 10, after the determination result R is displayed on the display unit 52, the operator determines whether the displayed determination result R is correct. Then, correct / incorrect information indicating whether the determination result R is correct is input to the operation panel 51 (step S34).

[0103] For example, when the determination result R of "discrepancy" is displayed on the display unit 52, the operator checks whether the aggregate type of the aggregate 9 input to the receiving conveyor 20 and the aggregate type information I input to the operation panel 51 are indeed discrepant. And if they are indeed discrepant, the operator inputs to the operation panel 51 that the determination result R by the determination unit 63 is correct. Also, when the aggregate type of the aggregate 9 input to the receiving conveyor 20 and the aggregate type information I input to the operation panel 51 match, the operator inputs to the operation panel 51 that the determination result R of the determination unit 63 is incorrect.

[0104] The above-described correct / incorrect information input to the operation panel 51 is transmitted to the information processing unit 56. The information processing unit 56 accumulates a set of the image cut out in step S25, the estimation result in step S26, and the correct / incorrect information input in step S34 in the storage unit 563 as teacher data for additional learning (step S35). The information processing unit 56 accumulates such teacher data each time the aggregate 9 is received by the receiving conveyor 20.

[0105] Note that in step S35, not only the image (frame image) cut out in step S25 but also a moving image D for a predetermined time (for example, 5 minutes) including the image may be included in the teacher data for additional learning. By doing so, a large number of frame images included in the moving image D can be used as images for additional learning. Therefore, a large amount of teacher data can be accumulated in a short time.

[0106] Thereafter, the learning unit 61 of the information processing unit 56 performs additional machine learning using the accumulated teacher data at a predetermined timing (step S36). The timing for performing the additional machine learning may be, for example, when the ratio of the above-described correct / error information being "error" becomes higher than a predetermined threshold. The learning unit 61 updates or reconstructs the learning model M using only the teacher data for additional learning or the combination of the initial teacher data and the teacher data for additional learning. Thereby, the estimation accuracy of the aggregate type by the learning model M can be maintained at a predetermined level.

[0107] Also, when measuring the moisture content on the surface of the aggregate 9 with the moisture sensor 57 as in the above-described second embodiment, the measured moisture content may be included in the teacher data for additional learning.

[0108] <5. Modification Example> As described above, the first to fourth embodiments of the present invention have been described, but the present invention is not limited to the above-described embodiments.

[0109] In the above-described embodiment, the learning unit 61 has performed image processing such as blurring, horizontal flipping, and vertical flipping on the frame image. However, the frame image may be used as it is as a learning image without performing image processing.

[0110] Also, in the above embodiment, the camera 53 captured the moving image D of the aggregate 9 conveyed by the receiving conveyor 20. However, the camera 53 may capture a still image of the aggregate 9 conveyed by the receiving conveyor 20. Even when capturing a still image instead of the moving image D, an image including blurring can be obtained by capturing without stopping the conveyance of the aggregate 9. That is, the camera 53 may obtain, as a learning image, an image including blurring by capturing a still image of the conveyed aggregate 9, not limited to the moving image D.

[0111] Also, in the above embodiment, when the determination result R by the determination unit 63 was "discrepancy", the receiving conveyor 20 was stopped. However, for example, a cut gate may be provided at the lower part of the receiving hopper 10, and when the determination result R is "discrepancy", the cut gate may be closed. Also, a surge bin capable of temporarily storing the aggregate 9 may be provided in the conveyance path, and when the determination result R is "discrepancy", the aggregate 9 may be conveyed to the surge bin. Also, a gate may be provided somewhere in the conveyance path to the silo 40, and when the determination result R is "discrepancy", the gate may be closed. That is, when the determination result R is "discrepancy", the control unit 64 may restrict the conveyance of the aggregate 9 to the silo 40 by some method.

[0112] Also, in the above embodiment, the receiving conveyor 20 was inclined. However, the receiving conveyor 20 may not be inclined. That is, the receiving conveyor 20 may convey the aggregate 9 in the horizontal direction.

[0113] Also, in the above embodiment, the computer as the information processing unit 56 was arranged inside the concrete plant. However, part or all of the functions of the information processing unit 56 may be arranged outside the concrete plant. For example, the computer as the information processing unit 56 may be arranged at a remote location and be made communicable with the operation panel 51, the display unit 52, and the camera 53 via the Internet.

[0114] Also, in the above-described embodiment, the determination result R of the determination unit 63 was displayed on the display unit 52. However, when the determination result R of "discrepancy" is output from the determination unit 63, a warning lamp may be lit or a warning buzzer may be sounded. That is, the "result output unit" in the present invention may be a warning lamp, a warning buzzer, or the like.

[0115] Also, in the above-described embodiment, an example of determining the aggregate type in the receiving conveyor 20 of the concrete plant has been described. However, the present invention can also be used in a quarry that ships and supplies aggregate 9 to the concrete plant. That is, the aggregate determination device of the present invention may determine the aggregate type in a conveyor installed at a quarry.

[0116] Also, the respective elements appearing in the above-described embodiment and modification examples may be appropriately combined within a range where no contradiction occurs.

Industrial Applicability

[0117] The present invention can be used to determine the aggregate type in a concrete plant or a quarry.

Explanation of Signs

[0118] 1 Aggregate conveying device 9 Aggregate 10 Receiving hopper 20 Receiving conveyor 30 Shuttle conveyor 40 Silo 50 Aggregate determination device 51 Operation panel 52 Display unit 53 Camera 54 Light source 55 Hood 56 Information processing unit 57 Moisture sensor 61 Learning unit 62 Estimation unit 63 Determination unit 64 Control unit 90 Truck D Video E Estimation result I Aggregate type information M Learning model M1 First learning model M2 Second learning model R Judgment result

Claims

1. An aggregate determination device for determining the suitability of aggregates fed onto a conveyor, comprising: a camera for photographing the aggregates conveyed by the conveyor; an operation panel into which aggregate type information is input; an information processing unit communicably connected to the camera and the operation panel; wherein the information processing unit has an estimation unit that inputs an image of the aggregate obtained by photographing with the camera into a learning model generated by machine learning and outputs an estimated result of the aggregate type from the learning model; a determination unit that determines whether or not the aggregate type input to the operation panel matches the estimated result; and the camera obtains an image having blur in the conveying direction of the aggregate by photographing the aggregate conveyed by the conveyor; the learning model is generated by performing machine learning using, as teacher data, an image obtained by photographing sample aggregates with the camera without rotating the image. An aggregate determination device.

2. The aggregate determination device according to claim 1, wherein the camera photographs a moving image of the aggregate conveyed by the conveyor; the moving image includes a plurality of frame images having blur in the conveying direction of the aggregate; and the information processing unit inputs the frame images included in the moving image photographed by the camera into the learning model. An aggregate determination device.

3. The aggregate determination device according to claim 2, wherein the information processing unit further has a learning unit that generates the learning model by performing machine learning using, as teacher data, a plurality of frame images included in a moving image of sample aggregates prepared for learning and the known aggregate type of the sample aggregates. An aggregate determination device.

4. The aggregate determination device according to claim 3, wherein the learning unit performs the machine learning after performing image processing other than rotation on the frame images. An aggregate determination device.

5. The aggregate determination device according to claim 3 or claim 4, wherein the operation panel can input correct / error information indicating whether or not the determination result by the determination unit is correct; and the learning unit updates the learning model by performing additional machine learning using, as teacher data, an image of the aggregate obtained from the camera, the estimated result, and the correct / error information. An aggregate determination device.

6. The aggregate determination device according to any one of claims 1 to 5, further comprising a light source that irradiates light toward the photographing range of the camera. An aggregate determination device.

7. The aggregate determination device according to claim 6, wherein a hood that covers a part of the upper surface of the conveyor is further provided, the camera and the light source are arranged in the hood, the aggregate determination device.

8. The aggregate determination device according to any one of claims 1 to 7, wherein a result output unit that outputs the determination result of the determination unit is further provided, the aggregate determination device.

9. The aggregate determination device according to any one of claims 1 to 8, wherein a moisture sensor that measures the moisture content on the surface of the aggregate conveyed by the conveyor is further provided, the information processing unit a first learning model generated by performing the machine learning on aggregates with a moisture content on the surface less than a predetermined value, and a second learning model generated by performing the machine learning on aggregates with a moisture content on the surface greater than or equal to the predetermined value, has, the estimation unit when the moisture content measured by the moisture sensor is less than the predetermined value, an image of the aggregate taken by the camera is input to the first learning model, and an estimation result of the aggregate type is output from the first learning model, when the moisture content measured by the moisture sensor is greater than or equal to the predetermined value, an image of the aggregate obtained from the camera is input to the second learning model, and an estimation result of the aggregate type is output from the second learning model, the aggregate determination device.

10. The aggregate determination device according to any one of claims 1 to 9, wherein the estimation unit inputs an odd number of images obtained from the camera to the learning model, and outputs an odd number of estimation results corresponding to each of the odd number of images from the learning model, the determination unit compares the aggregate type input to the operation panel with the odd number of estimation results, and makes a determination by a majority vote of the comparison results, the aggregate determination device.

11. The aggregate determination device according to any one of claims 1 to 10, wherein the information processing unit a control unit that restricts the conveyance of the aggregate to the silo when the determination unit determines that the aggregate type input to the operation panel does not match the estimation result is further provided, the aggregate determination device.

12. An aggregate determination method for determining the suitability of the aggregate input to the conveyor, comprising: a) a step of inputting information on the aggregate type of the aggregate input to the conveyor to an operation panel; b) a step of photographing the aggregate conveyed by the conveyor; c) Inputting an image of the aggregate obtained by the photographing in step b) into the learning model generated by machine learning, and outputting an estimation result of the aggregate type from the learning model; d) Determining whether or not the aggregate type input in step a) matches the estimation result; which comprises: In step b), the camera obtains an image having blur in the conveyance direction of the aggregate by photographing the aggregate conveyed by the conveyor; The learning model is generated by performing machine learning using, as teacher data, an image obtained by photographing sample aggregates with the camera without rotating the image. An aggregate determination method.

Images