DETECTION DEVICE, DETECTION METHOD, DETECTION PROGRAM, AND DETECTION SYSTEM
The detection system uses image analysis and machine learning to identify bridging in hoppers by generating movement data from time-series images, ensuring early detection and preventing incinerator issues.
Patent Information
- Application Number
- JP2021070778
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-04-20
- Publication Date
- 2024-11-07
- Estimated Expiration
- 2041-04-20
AI Technical Summary
Existing hopper systems struggle with accurate detection of bridging due to the irregular shapes and movements of garbage, leading to delayed detection and potential incinerator shutdowns.
A detection system that analyzes time-series images from above the hopper to generate movement data, using image processing and machine learning to identify the unique movement patterns associated with bridging, enabling early detection.
The system allows for stable and accurate detection of bridging, reducing the time lag between occurrence and detection, thereby preventing incinerator malfunctions.
Smart Images

Figure 0007578952000001 
Figure 0007578952000002 
Figure 0007578952000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a detection device for detecting the occurrence of a bridge in a hopper. [Background technology]
[0002] A typical waste incineration facility is equipped with a funnel-shaped device called a hopper. Waste stored in a storage facility called a waste pit is picked up by a crane or other device and thrown into the hopper, from which it is sent to an incinerator for incineration. In such incineration facilities, waste thrown into the hopper can sometimes become clogged in the hopper in the form of a bridge, a phenomenon known as bridging.
[0003] When a bridge occurs, the waste above the bridge remains in the same position, so the height of the waste accumulated in the hopper (waste level) remains high. Generally, a drop in the waste level is used as a trigger for dumping waste into the hopper, so when a bridge occurs, no new waste is dumped into the hopper. On the other hand, the waste below the bridge is sent to the incinerator and incinerated one by one, so if the bridge is left unattended, the amount of waste in the incinerator will decrease, and if it is left unattended further, there will be no waste to burn in the incinerator.
[0004] When the amount of waste in an incinerator decreases or runs out, the temperature inside the incinerator drops, making it necessary to heat it with a burner or the like, and in some cases, it may become necessary to shut down the incinerator. For example, Patent Document 1 below is an example of a technology for avoiding such a situation. Patent Document 1 describes a method for detecting a bridge based on a hopper level that is periodically measured. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] Japanese Patent Publication No. 10-238735 Summary of the Invention [Problem to be solved by the invention]
[0006] However, since garbage contains various objects and has a variable shape, and some garbage may jump up or get caught on the top of the hopper when it is put into the hopper, it is difficult to measure the hopper level stably and accurately. For this reason, when detecting a bridge based on the hopper level, it is necessary to set a detection threshold value that does not cause frequent false detections, which can cause a large time lag between the occurrence of a bridge and its detection. Note that this problem is not limited to hoppers installed in garbage incinerators, but is a common problem that occurs in hoppers for any purpose where a bridge may occur.
[0007] One aspect of the present invention has been made in consideration of the above-mentioned problems, and an object of the present invention is to provide a detection device etc. that can detect the occurrence of a bridge at an early stage. [Means for solving the problem]
[0008] In order to solve the above problems, a detection device according to one embodiment of the present invention comprises a movement data generation unit that generates movement data indicating the movement state of an object during a period during which a plurality of images were taken of a hopper containing an object from a time series of images taken from above, and a detection unit that detects the occurrence of a bridge in the hopper based on the movement data generated by the movement data generation unit.
[0009] In addition, in order to solve the above problems, a detection method according to one embodiment of the present invention is a bridge detection method executed by one or more information processing devices, and includes a movement data generation step of generating movement data indicating the movement state of an object during a period in which a plurality of images were taken of a hopper containing an object from a time series of images taken from above the hopper, and a detection step of detecting the occurrence of a bridge in the hopper based on the movement data generated in the movement data generation step.
[0010] In addition, in order to solve the above problems, a detection device according to one embodiment of the present invention includes a movement data generation unit that generates movement data indicating the movement state of an object during a period in which a plurality of images were taken from above a hopper containing the object, from a plurality of time-series images of the object, and a detection unit that detects the occurrence of a bridge in the hopper based on an output value obtained by inputting the movement data into a trained model that has learned the relationship between the movement state of the object and whether or not a bridge occurs when the object is in that moving state.
[0011] In addition, in order to solve the above problems, a detection system according to one embodiment of the present invention includes an imaging device that images a hopper containing an object from above, a detection device that detects the occurrence of a bridge in the hopper based on movement data that is generated from a plurality of images in a time series captured by the imaging device and indicates the movement state of the object during a period when the plurality of images were captured, and an alarm device that alerts the occurrence of a bridge when the detection device detects the occurrence of a bridge. Effect of the Invention
[0012] According to one aspect of the present invention, the occurrence of bridging can be detected at an early stage. [Brief description of the drawings]
[0013] [Figure 1] 1 is a block diagram showing an example of a configuration of a main part of a detection device according to a first embodiment of the present invention. [Diagram 2] 1 is a diagram illustrating an example of the configuration of a detection system according to a first embodiment of the present invention. [Diagram 3] 11A and 11B are diagrams illustrating an example of a method for determining the presence or absence of dust. [Figure 4] FIG. 11 is a diagram illustrating an example of a method for generating movement data. [Diagram 5] 11A and 11B are diagrams illustrating a method for detecting the occurrence of a bridge. [Figure 6] 4 is a flowchart illustrating an example of a process executed by the detection device. [Figure 7]FIG. 11 is a block diagram showing an example of a configuration of a main part of a detection device according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0014] [Embodiment 1] (System Configuration) The configuration of the detection system 100 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of the detection system 100. Fig. 2 shows an example in which the detection system 100 is applied to bridge detection in a hopper A installed in a waste incineration facility. Note that the detection system 100 is not limited to hopper A, but can be applied to any hopper for any purpose in which a bridge may occur. The object contained in hopper A is waste G, but the detection system 100 can also detect bridges in hoppers that contain any object other than waste G.
[0015] The waste incineration facility shown in FIG. 2 is equipped with a waste pit B for storing waste G and an incinerator C for incinerating the waste G. The waste G in the waste pit B is dumped into a hopper A by a crane. The upper part of the hopper A is formed with an inclined surface A1 that is inclined so as to expand outward, and the waste G slides down this inclined surface A1 and enters the inside of the hopper A, where it is temporarily stored. The waste G in the hopper A is then sequentially sent from the opening A2 at the bottom to the incinerator C for incineration. In the example of FIG. 2, the vicinity of the opening A2 is a cavity where no waste G exists. This is because the waste is clogged in the hopper A in a bridging manner, that is, a bridge is formed. In a normal operating state where no bridge is formed, the waste G is also clogged near the opening A2.
[0016] As described above, detection system 100 is a system that detects the occurrence of a bridge in hopper A, and includes detection device 1, photographing device 2, and notification device 3. The photographing device 2 is disposed above hopper A and photographs hopper A from above at predetermined time intervals. In addition, detection device 1 acquires each image photographed by photographing device 2 and detects the occurrence of a bridge in hopper A using these images. Then, notification device 3 notifies a user of detection system 100 of the occurrence of a bridge when detection device 1 detects the occurrence of a bridge.
[0017] 2 shows an example in which the occurrence of a bridge is notified by voice, i.e., the notification device 3 is an audio output device, but is not limited to this example. Any notification method may be used as long as it can make the target person aware that a bridge has occurred, and the notification device 3 may be a device corresponding to the notification method. For example, the notification device 3 may be an alarm lamp, a display device, etc.
[0018] More specifically, the photographing device 2 photographs a plurality of images in time series of the hopper A. Then, the detection device 1 detects the occurrence of a bridge in the hopper A based on movement data that is generated from the plurality of images in time series photographed by the photographing device 2 and indicates the movement state of the garbage G during the period when the plurality of images were photographed.
[0019] Here, the inventors of the present invention have found, by closely examining images of various hoppers taken from above, that images taken when a bridge has occurred have features different from those taken when a bridge has not occurred. More specifically, they have found that there is a difference in the state of movement of the object shown in the image between when a bridge has occurred and when the hoppers are operating normally.
[0020] The detection device 1 detects the occurrence of a bridge based on this knowledge. As described above, there is a difference in the movement state of the dust G captured in the image when a bridge occurs and during normal operation. For this reason, the occurrence of a bridge can be detected based on movement data that is generated from multiple images in time series and indicates the movement state of the dust G during the period when the multiple images were taken. The details of this detection method will be described later.
[0021] In addition, since the movement state specific to the occurrence of a bridge can be observed in the image immediately after the occurrence of the bridge, the above-mentioned configuration allows for early detection of the occurrence of a bridge. Furthermore, the above-mentioned configuration allows for stable and highly accurate detection compared to bridge detection based on the hopper level. This is because, while it is difficult to stably and accurately measure the height of an irregular object using a hopper level meter, the movement state specific to the occurrence of a bridge can be stably observed in the image taken from above the hopper A.
[0022] As described above, the detection system 100 includes the photographing device 2 that photographs from above the hopper A that stores garbage G, the detection device 1 that detects the occurrence of a bridge in the hopper A based on movement data that is generated from a plurality of chronological images taken by the photographing device 2 and indicates the movement state of the garbage G during the period when the plurality of images were taken, and the notification device 3 that notifies the occurrence of a bridge when the detection device 1 detects the occurrence of a bridge. With this configuration, the occurrence of a bridge can be detected and notified at an early stage.
[0023] The location of the detection device 1 is not particularly limited, and may be, for example, located in a waste incineration facility or a monitoring center that monitors multiple waste incineration facilities. Although Fig. 2 shows an example in which one detection device 1 is used to monitor and detect the occurrence of bridging in one hopper A, it is also possible to use one detection device 1 to monitor and detect the occurrence of bridging in multiple hoppers. In this case, for example, the detection device 1 may be a cloud server.
[0024] Furthermore, the process performed when the occurrence of a bridge is detected is not limited to the above example. For example, if a bridge removal device for removing bridges is provided in a waste incineration facility, the detection device 1 may operate the bridge removal device when it detects the occurrence of a bridge to automatically remove the bridge. The bridge removal device may be any device capable of removing bridges, and may be, for example, a device equipped with a pusher that pushes down waste stuck in a hopper.
[0025] (Configuration of the detection device) The configuration of the detection device 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the configuration of the main parts of the detection device 1. As shown in the figure, the detection device 1 includes a control unit 10 that controls each part of the detection device 1, and a storage unit 11 that stores various data used by the detection device 1. The detection device 1 also includes a communication unit 12 that allows the detection device 1 to communicate with other devices, an input unit 13 that accepts input of various data to the detection device 1, and an output unit 14 that allows the detection device 1 to output various data.
[0026] The control unit 10 includes a data acquisition unit 101, a determination unit 102, a movement data generation unit 103, a detection unit 104, and a notification control unit 105. The storage unit 11 has an image DB 111 stored therein.
[0027] The data acquisition unit 101 acquires a plurality of images in a time series obtained by photographing the hopper A from above. As described above, in the detection system 100, the photographing device 2 captures a plurality of images in a time series obtained by photographing the hopper A from above, so the data acquisition unit 101 only needs to acquire the images photographed by the photographing device 2. The data acquisition unit 101 may acquire images from the photographing device 2 by communication via the communication unit 12, or may acquire images inputted via the input unit 13. The data acquisition unit 101 records the acquired images in the image DB 111. The photographing device 2 may be one that captures moving images. In this case, the data acquisition unit 101 only needs to acquire a plurality of images in a time series (for example, frame images at predetermined intervals) from the moving images photographed by the photographing device 2.
[0028] The determination unit 102 determines whether or not dust is present in a target area that is the subject of analysis for detecting the occurrence of a bridge in the image acquired by the data acquisition unit 101. The method of determining whether dust is present in the target area will be described in detail below with reference to FIG.
[0029] The movement data generation unit 103 generates movement data indicating the movement state of garbage during a period when a plurality of images were taken, from a plurality of images in time series acquired by the data acquisition unit 101 and recorded in the image DB 111. The method of generating the movement data will be described in detail later with reference to FIG.
[0030] The detection unit 104 detects the occurrence of a bridge in the hopper A based on the movement data generated by the movement data generation unit 103. The method of detecting the occurrence of a bridge will be described in detail later with reference to FIG.
[0031] The notification control unit 105 notifies the occurrence of a bridge when the detection unit 104 detects the occurrence of a bridge. Specifically, when the detection unit 104 detects the occurrence of a bridge, the notification control unit 105 notifies the occurrence of a bridge by causing the notification device 3 to output a sound indicating the occurrence of a bridge. Note that the method of notifying the occurrence of a bridge is not particularly limited as long as it can make the target person recognize that a bridge has occurred. For example, when the detection device 1 is placed near the target person, the notification control unit 105 may cause the output unit 14 to output information indicating that a bridge has occurred. Also, for example, the notification control unit 105 may perform the notification by a method such as sending a message to a terminal device carried by the target person.
[0032] As described above, the detection device 1 includes a movement data generating unit 103 that generates movement data indicating the state of movement of garbage during a period during which a plurality of images were taken from above the hopper A in a time series, and a detection unit 104 that detects the occurrence of a bridge in the hopper A based on the movement data generated by the movement data generating unit 103. With this configuration, the occurrence of a bridge can be detected at an early stage.
[0033] In addition, although details will be described later, movement data can be generated with just two images taken at different times. Therefore, there is no need to perform machine learning or the like, and operation of the detection device 1 can be started immediately in the waste incineration facility. In addition, even if the installation location or shooting angle of the imaging device 2 is changed slightly, detection can be performed without any particular influence. Furthermore, although it is expected that dirt will adhere to the lens of the imaging device 2 installed in the waste incineration facility, since the adhered dirt does not move, the influence of such dirt on the generation of movement data is also limited. In other words, the detection system 100 including the detection device 1 has the advantage of being easy to introduce and capable of stable operation.
[0034] (Determining whether or not garbage is present) The determination of the presence or absence of garbage by the determination unit 102 will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of a method for determining the presence or absence of garbage. Image D shown in Fig. 3 is an image taken from above the hopper. Image D shows garbage sliding down the inclined surface of the hopper, but the garbage does not cover the entire inclined surface. Specifically, garbage is seen in a downstream area d2 of the inclined surface of the hopper, but no garbage is seen in an upstream area d1.
[0035] If the movement data generating unit 103 generates movement data for an area that does not include dust, such as area d1, the amount of movement indicated by the movement data may be zero or close to zero. When the amount of movement is zero or close to zero, there is a possibility that a bridge may be erroneously detected as having occurred, even though there is actually no dust and no bridge has occurred.
[0036] In order to avoid the above-mentioned erroneous detection, the detection device 1 includes a determination unit 102. The determination unit 102 determines whether or not dirt is present in the target area of the image. Then, the movement data generation unit 103 generates movement data indicating the amount of movement between images in which the determination unit 102 has determined that dirt is present in the target area. This prevents images in which no object is present in the target area from being used to detect the bridge, thereby preventing the above-mentioned erroneous detection. Note that, although details will be described later, it is preferable to set the target area on the inclined surface of the hopper, more specifically, on the downstream inclined surface as in area d2.
[0037] The presence or absence of garbage may be determined using the captured image, or may be determined after applying a predetermined image processing to the captured image to make it easier to determine the presence or absence of garbage. An example of the image processing is the Canny method. The Canny method is an algorithm for edge detection in an image. Image D1 shown in FIG. 3 is an image generated by detecting edges from image D using the Canny method. In general, the appearance of garbage is much more complicated than the surface shape of a hopper. For this reason, in the example of FIG. 3, many edges are detected in area d2 where garbage is captured, compared to area d1 where garbage is not captured.
[0038] Therefore, the judgment unit 102 may apply the Canny method to detect edges in an image of the hopper, and judge an image in which the number of edges detected in the target area is equal to or greater than a threshold to be an image containing dust, and judge an image in which the number of edges detected in the target area is less than the threshold to be an image not containing dust.
[0039] Of course, an edge detection method other than the Canny method may be applied. Also, the presence or absence of dust can be determined by a method other than edge detection. For example, the determination unit 102 may determine the presence or absence of dust by analyzing at least one of the luminance value and the RGB value of each pixel constituting the image. Also, for example, the determination unit 102 may determine the presence or absence of dust by a trained model (e.g., a neural network model) that has been trained to be able to determine the presence or absence of dust.
[0040] (How movement data is generated) A method for generating movement data by the movement data generating unit 103 will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of a method for generating movement data. More specifically, Fig. 4 shows an example of generating movement data F, which indicates the movement state of garbage between an image E1 taken on the inclined surface of the hopper at time t and an image E2 taken on the inclined surface of the hopper at time t+Δt, by an image correlation method. In Fig. 4, the y direction is the direction in which garbage descends, that is, the direction from the upstream side to the downstream side of the inclined surface of the hopper (see inclined surface A1 in Fig. 2), and the x direction is the direction perpendicular to the y direction on the inclined surface.
[0041] The image correlation method is a technique in which the amount of movement of each particle in two time-series images is calculated using image processing, and the calculated amount of movement is expressed as a vector. For example, in image E1, dust enclosed in a dashed line frame is captured at position e1, but in image E2, it is captured at position e2, which is lower than position e1. In the movement data F, the movement state of this dust is represented as vector f1, which indicates the displacement from position e1 to position e2. Vectors are also calculated similarly for other positions.
[0042] In this way, the movement data F shows the state of garbage movement at each position in images E1 and E2. Specifically, it can be seen from the movement data F that the garbage at each position in images E1 and E2 is moving in the y direction (the downstream side of the hopper's inclined surface, i.e., the downward direction). Note that in the movement data F, vectors whose y direction component value is equal to or greater than the threshold value are shown with solid arrows, and vectors whose y direction component value is less than the threshold value are shown with dashed arrows. This also shows that there are positions where the garbage is moving fast and slow.
[0043] As described above, the movement data generation unit 103 may generate movement data indicating the amount of movement of an image element appearing at each of a plurality of predetermined positions set on the inclined surface appearing in images E1 and E2 in the downward direction between images E1 and E2.
[0044] On the inclined surface of the hopper, the garbage slides down in roughly the same downward direction, so the inclined surface of the hopper is an ideal location for determining whether the garbage is descending smoothly or is stalled. Therefore, with the above-mentioned configuration, movement data can be generated that accurately indicates whether the garbage is descending smoothly or is stalled. Then, by using this movement data to detect the occurrence of a bridge, highly reliable detection can be performed.
[0045] The movement data may be generated for image portions other than the inclined surface. In this case, the detection unit 104 may detect the bridge using the movement data within a target area set for the portion of the movement data generated by the movement data generation unit 103 that includes the inclined surface.
[0046] Of course, the target area may be any area where the direction of movement of the waste is generally the same, and may be an area other than the inclined surface. For example, the target area may be an area (usually extending vertically) on the inner surface of the hopper that is connected to the inclined surface downstream of the inclined surface, or an area including both the inclined surface and other parts.
[0047] The movement data is not limited to the above examples as long as it indicates the state of movement of the garbage. For example, the movement data may be the speed at which the garbage moves. Furthermore, the movement data may be a difference image showing the difference between two images taken at different times.
[0048] (Method of detecting bridge occurrence) The method of detecting the occurrence of a bridge by the detection unit 104 will be described with reference to FIG. 5. FIG. 5 is a diagram for explaining the method of detecting the occurrence of a bridge. More specifically, FIG. 5 shows an example of detecting the occurrence of a bridge using movement data H generated from a plurality of time-series images of a hopper photographed from above. Note that the movement data H is generated using the image correlation method described above, and is data indicating the movement vectors of image elements appearing at each of a plurality of predetermined positions set in the image of the hopper. Also, in FIG. 5, a target area in the movement data H that is the subject of analysis for detecting the presence or absence of the occurrence of a bridge is indicated by a frame h1.
[0049] Image J shown in FIG. 5 is an enlarged view of the target area in an image of a hopper. In addition, image J plots each position for which a movement vector has been calculated. For example, the movement vector for position j1 is calculated as (-1, 3). As shown in the figure, the positions for which a movement vector is calculated are set uniformly throughout image J. More specifically, a plurality of positions are arranged in a line perpendicular to the downward direction of the garbage, i.e., in the x direction in FIG. 5, and the line is arranged in a plurality of lines along the downward direction of the garbage, i.e., the y direction in FIG. 5.
[0050] When determining whether or not a bridge has occurred, the detection unit 104 first determines whether or not the amount of dust movement is within a normal range for each of a plurality of positions aligned in a row in the x direction.
[0051] For example, if the y value of the movement vector, i.e., the amount of movement of the dust in the downward direction, is D, the normal range of D may be defined as D≧m1, where m1 is the lower limit of the normal range. Note that the amount of movement D may be a negative value at some positions. This indicates that the dust is moving in the opposite direction to the downward direction at that position. Since this type of movement does not normally occur, positions where the amount of movement D is a negative value may be deemed undeterminable.
[0052] For example, the detection unit 104 may evaluate each position in three stages according to the following conditional expressions (1) to (3). D≧m1 (1) m1>D>0 (2) D≦0 (3) A position that satisfies the condition of conditional formula (1) can be said to be in a normal movement state where the dust is smoothly descending. On the other hand, a position that satisfies the condition of conditional formula (2) can be said to correspond to an abnormal movement state that is specific to when the dust movement amount is small and a bridge occurs. Therefore, for example, the detection unit 104 may set the evaluation value to 1 (a value indicating no abnormality) for a position that satisfies the condition of the above conditional formula (1), and may set the evaluation value to 0 (a value indicating an abnormality) for a position that satisfies the condition of the above conditional formula (2). Furthermore, the detection unit 104 may set the evaluation value to -1 (a value indicating that determination is impossible) for a position that satisfies the condition of the above conditional formula (3).
[0053] For example, when m1=1, position j1 in image J in FIG. 5 satisfies the condition of the above conditional formula (1) because the y value of the movement vector is 3. Therefore, in this case, the evaluation value of position j1 is 1. Evaluation values are calculated similarly for the other positions.
[0054] Next, the detection unit 104 judges whether or not the dust is moving normally for each row of positions lined up in the x direction, based on the result of judging whether or not the amount of movement for each position is within the normal range. For example, when the detection unit 104 calculates an evaluation value of 1, 0, or -1 for each position as described above, the detection unit 104 may evaluate each row in three stages according to the following conditional expressions (4) to (6). Note that Sum(D) is the sum of the evaluation values of the amount of movement for the positions lined up in a row in the x direction. Sum(D)>0 (4) Sum(D)=0 (5) Sum(D)<0 (6) A row that satisfies the condition of conditional formula (4) can be said to be in a normal movement state where the garbage is moving downward smoothly as a whole. On the other hand, a row that satisfies the condition of conditional formula (5) can be said to correspond to an abnormal movement state specific to when the amount of garbage moving as a whole is small and bridging occurs. For this reason, the detection unit 104 may set an evaluation value of 1 for a row that satisfies the condition of conditional formula (4) above, and an evaluation value of 0 for a row that satisfies the condition of conditional formula (5) above. Furthermore, the detection unit 104 may set an evaluation value of -1 for a row that satisfies the condition of conditional formula (6) above.
[0055] For example, column K1 in image J in FIG. 5 includes a total of five positions including position j1. Therefore, for column K1, Sum(D), which is the sum of the evaluation values of the movement amounts calculated for each of these five positions, is calculated, and an evaluation value for column K1 is calculated based on the calculated Sum(D) using conditional expressions (4) to (6). Evaluation values are calculated similarly for the other columns. In FIG. 5, the evaluation values for each column are shown in column L. In this example, the evaluation values for all columns are 1, that is, normal.
[0056] Finally, the detection unit 104 generates an overall judgment result based on the judgment results of each column described above, and detects the occurrence of a bridge based on the judgment result. For example, if the evaluation value of each column calculated as described above is J and the sum of the evaluation values of all columns is Sum(J), the detection unit 104 may generate an overall judgment result using the following conditional expressions (7) to (9), and detect the occurrence of a bridge based on the overall judgment result. Note that m2 is a predetermined threshold value. Sum(J)≧m2 (7) Sum(J) <m2 (8) -1∈J (9) When the condition of conditional formula (7) is met, it can be said that the dust is moving smoothly downwards overall and is in a normal movement state, so the detection unit 104 sets the overall judgment result to 1 and judges that no bridges have occurred. On the other hand, when the condition of conditional formula (8) is met, it can be said that the overall amount of dust moving is small and corresponds to an abnormal movement state specific to when a bridge has occurred, so the detection unit 104 sets the overall judgment result to 0 and judges that a bridge has occurred. Note that the condition of conditional formula (9) is met when it is impossible to judge for at least one column. In this case, the detection unit 104 sets the overall judgment result to -1 and judges that it is impossible to judge whether a bridge has occurred or not.
[0057] As described above, the detection unit 104 may (i) determine whether or not the amount of movement is within a normal range for each of a number of positions arranged in a row in the x direction, (ii) determine whether or not the debris is moving normally in the row from the result of the determination in (i), and (iii) detect the occurrence of a bridge based on the result of the determination in (ii).
[0058] Generally, garbage is composed of multiple objects. Therefore, garbage components aligned in a line perpendicular to the downward direction slide down the hopper while generally maintaining that line, so it is effective to detect a specific movement state when a bridge occurs along the line perpendicular to the downward direction. However, the movement amount of garbage components may vary depending on the position on the hopper. For this reason, for example, even if the garbage is smoothly moving down as a whole, it is expected that the movement amount at some positions may be outside the normal range.
[0059] Therefore, the above-mentioned configuration employs a configuration in which whether or not the dust is moving normally in a line is determined based on the result of determining whether the amount of movement at each position in the line is within a normal range. As a result, even if there is variation in the amount of movement at each position lined up in the x direction, it is possible to appropriately determine whether the dust is moving normally or not, and to accurately detect the occurrence of a bridge based on the determination result.
[0060] As described above, the detection unit 104 may perform the determinations (i) and (ii) for each of the multiple rows and detect the occurrence of a bridge in the hopper based on the determination results for each row. According to this configuration, the occurrence of a bridge is detected based on the determination results of whether or not the waste is moving normally in each row, so that the occurrence of a bridge can be detected with high accuracy even if there is variation in the determination results for each row.
[0061] Note that the above determination method is merely one example, and various methods can be applied as a method for detecting the occurrence of a bridge based on movement data. For example, the detection unit 104 may identify the amount of movement at each position in the target area from the movement data, compare the sum or average of the identified amounts of movement with a predetermined threshold, and determine that it is normal if it is equal to or greater than the threshold, and that it is abnormal, i.e., that a bridge has occurred, if it is less than the threshold.
[0062] Also, for example, the detection unit 104 may detect the occurrence of a bridge using only the movement data of each position lined up in a row in the x direction (for example, each position included in row K1 in FIG. 5). In this case, for example, the detection unit 104 may detect the occurrence of a bridge from the evaluation result according to the above-mentioned conditional expressions (4) to (6).
[0063] Furthermore, the various thresholds used to detect the occurrence of a bridge, including the above-mentioned threshold m1, may be fixed values or variable values. For example, when the speed at which waste is sent from the hopper to the incinerator (dust feeding speed) is fast, the speed at which the waste moves in the upper part of the hopper also becomes fast. For this reason, threshold m1 may be changed in conjunction with the dust feeding speed. Specifically, the threshold m1 may be set to a larger value as the dust feeding speed increases. This makes it possible to detect the occurrence of a bridge when the movement of the waste in the upper part of the hopper becomes slow despite the fast dust feeding speed.
[0064] (Processing flow) A process flow (detection method) executed by the detection device 1 will be described with reference to Fig. 6. Fig. 6 is a flowchart showing an example of the process executed by the detection device 1. Note that the process in Fig. 6 is performed, for example, every time the imaging device 2 captures a new image.
[0065] In S1, the data acquisition unit 101 acquires time-series images of the hopper taken from above and records them in the image DB 111. As described above, these images are taken by the photographing device 2 (see FIG. 2), so the data acquisition unit 101 only needs to acquire the images taken by the photographing device 2.
[0066] In S2, the determination unit 102 determines whether or not dust is present in the target area in the image acquired in S1. The method for determining whether or not dust is present in the target area is as described with reference to Fig. 3, and therefore the description will not be repeated here. If the determination in S2 is YES, the process proceeds to S3, and if the determination in S2 is NO, the process in Fig. 6 ends.
[0067] In S3 (movement data generation step), the movement data generation unit 103 generates movement data indicating the movement state of the garbage during the period when the images were taken from the image acquired in S1 and the image taken before that image. For example, when an image taken at time t+Δt is acquired in S1, the movement data generation unit 103 reads the image taken at time t from the image DB 111. Then, the movement data generation unit 103 generates movement data indicating the movement state of the garbage during the period from time t to time t+Δt. The method of generating the movement data is as described with reference to FIG. 4, and therefore will not be repeated here.
[0068] In S4, the detection unit 104 uses the movement data generated in S3 to determine whether or not the amount of movement is within a normal range for each position in the target area set in the image acquired in S1. Next, in S5, the detection unit 104 determines whether or not the movement state in each row in which the positions are arranged in a direction perpendicular to the downward direction of the dust (x direction in FIG. 5) is normal based on the determination result of S4. Then, in S6, the detection unit 104 generates a comprehensive determination result based on the determination result of each row in S5. Note that the determination method of S4 to S6 is as described with reference to FIG. 5, and detailed description will not be repeated here.
[0069] In S7 (detection step), the detection unit 104 detects the occurrence of a bridge based on the overall judgment result of S6. For example, if the overall judgment result is expressed as one of the values 1 (no abnormality), 0 (abnormality), and -1 (unable to judge), the detection unit 104 judges that a bridge has not occurred (NO in S7) if the overall judgment result of S6 is 1. In this case, the processing in FIG. 6 ends. On the other hand, if the overall judgment result of S6 is 0, the detection unit 104 judges that a bridge has occurred (YES in S7). That is, in this case, the detection unit 104 detects the occurrence of a bridge. After this, the processing proceeds to S8.
[0070] In addition, the process to be performed when the determination is impossible may be determined in advance. For example, when the determination is impossible, the detection unit 104 may end the process of FIG. 6, or may proceed to S8 to notify the user that the determination is impossible.
[0071] In S8, the notification control unit 105 notifies the occurrence of a bridge. Specifically, the notification control unit 105 notifies the occurrence of a bridge by causing the notification device 3 to output a sound indicating the occurrence of a bridge. If a bridge removal device for eliminating bridges is provided in the waste incineration facility, in S8, the notification control unit 105 may operate the bridge removal device to eliminate the bridge, in addition to or instead of issuing the notification.
[0072] As described above, the detection method according to this embodiment includes a movement data generation step (S3) for generating movement data indicating the movement state of the garbage during a period during which a plurality of images were taken from above a hopper that contains the garbage, and a detection step (S7) for detecting the occurrence of a bridge based on the movement data generated in the movement data generation step. According to this detection method, the occurrence of a bridge can be detected at an early stage.
[0073] As described at the beginning of the first embodiment, the detection system 100 is not limited to hoppers at waste incineration facilities, but can be applied to any hopper for which a bridge may occur, and the objects contained in the hopper are not limited to garbage. In other words, the "garbage" described in the above-mentioned embodiment can be read as any "object."
[0074] [Embodiment 2] Other embodiments of the present invention will be described below. For ease of explanation, the same reference numerals are given to members having the same functions as those described in the above embodiment, and the description thereof will not be repeated.
[0075] (Configuration of the detection device) The configuration of a detection device 1A according to this embodiment will be described with reference to Fig. 7. Fig. 7 is a block diagram showing an example of a configuration of a main part of the detection device 1A. The detection device 1A is different from the detection device 1 shown in Fig. 1 in that the detection unit 104 is replaced with a detection unit 104A and that a trained model 112 is stored in the storage unit 11.
[0076] The detection unit 104A detects the occurrence of a bridge in the hopper based on an output value obtained by inputting the movement data generated by the movement data generation unit 103 into the trained model 112 that has learned the relationship between the movement state of the object and the presence or absence of a bridge occurring when the object is in the moving state. Note that the movement data is generated by the movement data generation unit 103 as in the first embodiment. The object may be garbage or may be something other than garbage.
[0077] As described above, the detection device 1A comprises a movement data generation unit 103 that generates movement data indicating the movement state of an object during a period when multiple images were taken from above a hopper containing the object, based on multiple time-series images of the object, and a detection unit 104A that detects the occurrence of a bridge in the hopper based on an output value obtained by inputting the movement data into a trained model 112 that has learned the relationship between the movement state of the object and the presence or absence of a bridge when the object is in that moving state.
[0078] The above configuration is based on the knowledge that there is a difference in the amount of movement of an object captured in an image when a bridge occurs and during normal operation, similar to the detection device 1 of embodiment 1. As described above, there is a difference in the state of movement of an object captured in an image when a bridge occurs and during normal operation, so the above configuration of detection device 1A makes it possible to detect the occurrence of a bridge.
[0079] In addition, since a movement state specific to the occurrence of a bridge can be observed in an image immediately after the occurrence of the bridge, the detection device 1A can detect the occurrence of a bridge at an early stage, similar to the detection device 1 of embodiment 1. Furthermore, the detection device 1A can achieve stable and highly accurate detection compared to bridge detection based on the hopper level, similar to the detection device 1 of embodiment 1.
[0080] (About trained models) As described above, the trained model 112 is generated by learning the relationship between the movement state of an object and the presence or absence of a bridge occurring when the object is in that movement state.
[0081] When generating training data to be used to generate the trained model 112, first, a plurality of time-series images of the hopper taken from above when a bridge occurs, and a plurality of time-series images of the hopper operating normally taken from above are collected.
[0082] Next, movement data is generated from the images collected as described above. Specifically, the above images are paired in chronological order, and movement data is generated for each pair. This process is performed for each of the images captured when a bridge occurs and the images captured under normal conditions. As a result, movement data when a bridge occurs and movement data under normal conditions are generated. Note that the movement data may be any data that indicates the movement state of the object. For example, the movement data may be generated by an image correlation method, as in the first embodiment.
[0083] Then, the movement data when a bridge occurs is associated with correct answer data indicating that an abnormality exists, i.e., that a bridge has occurred, and used as teacher data, whereas the movement data when normal is associated with correct answer data indicating that the movement is normal, i.e., that a bridge has not occurred, and used as teacher data.
[0084] By performing machine learning using these teacher data, a trained model 112 that classifies input movement data into movement data in normal times and movement data in abnormal times is generated. Note that the machine learning algorithm may be any algorithm that can generate a trained model 112 that can classify input movement data into movement data in normal times and movement data in abnormal times. For example, a convolutional neural network or the like that has high image classification accuracy is preferable, but is not limited to this example.
[0085] Note that feature amounts extracted from movement data may be used as training data for the trained model 112. Such feature amounts also indicate the amount of movement of an object, and are a type of movement data. Furthermore, the training data for the trained model 112 may include data other than movement data. For example, the number of edges calculated by the Canny method from an image of a hopper taken from above, the RGB values of the image, etc. may be included in the training data.
[0086] (Processing flow) The flow of the process executed by the detection device 1A is the same as that in Fig. 6. However, the processes of S4 to S6 in Fig. 6 are replaced with a process of inputting movement data to the trained model 112 and obtaining an output value. Also, in S7 (detection step), the occurrence of a bridge is detected based on the output value of the trained model 112.
[0087] That is, the detection method according to this embodiment includes a movement data generating step of generating movement data indicating the movement state of an object during a period during which a plurality of images were taken from above a hopper storing the object, from a plurality of time-series images of the hopper, and a detection step of detecting the occurrence of a bridge in the hopper based on an output value obtained by inputting the movement data into trained model 112 that has trained the relationship between the movement state of the object and the presence or absence of a bridge when the object is in the moving state. According to this detection method, the occurrence of a bridge can be detected at an early stage.
[0088] [Modifications] The execution entity of each process described in each of the above-mentioned embodiments is arbitrary and is not limited to the above-mentioned examples. In other words, as long as the execution entity of each process described in each of the above-mentioned embodiments can be performed, the devices constituting the detection system 100 can be appropriately changed.
[0089] For example, in the detection system 100 in Fig. 1, the detection device 1 and the notification device 3 are separate devices, but the detection device 1 may also be configured to include the notification device 3. Conversely, the functions of the detection device 1 may be distributed among multiple information processing devices (computers). For example, movement data may be generated by an information processing device on the cloud, and this movement data may be transmitted to an information processing device installed in a waste incineration facility, and the information processing device may detect a bridge.
[0090] [Software implementation example] The functions of the detection devices 1 and 1A (hereinafter referred to as the "devices") can be realized by a program for causing a computer to function as the device, and a program (detection program) for causing a computer to function as each control block of the device (particularly each part included in the control unit 10).
[0091] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program to realize each function described in each of the above embodiments.
[0092] The program may be non-transitory and may be recorded in one or more computer-readable recording media. The recording media may or may not be included in the device. In the latter case, the program may be provided to the device via any wired or wireless transmission medium.
[0093] In addition, some or all of the functions of each of the control blocks can be realized by a logic circuit. For example, an integrated circuit in which a logic circuit that functions as each of the control blocks is formed is also included in the scope of the present invention. In addition, the functions of each of the control blocks can be realized by, for example, a quantum computer.
[0094] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]
[0095] 100 Detection System 1. Detection device 102 Judgment section 103 Movement Data Generation Unit 104 Detection unit 2. Imaging Device 3. Notification device 1A Detection device 104A Detection unit
Claims
1. a movement data generating unit that generates movement data indicating a movement state of the object during a period when a plurality of images were taken from above a hopper storing the object from a time series of the plurality of images; a detection unit that detects the occurrence of a bridge in the hopper based on the movement data generated by the movement data generation unit, the movement data generating unit generates the movement data indicating an amount of movement of an image element shown in each of a plurality of predetermined positions set on the inclined surface of the hopper shown in the image in a downward direction between the images; The detection unit is (1) determining whether the amount of movement is within a normal range for each of a plurality of positions arranged in a row in a direction perpendicular to the descending direction; (2) determining whether the object is moving normally in the line based on the result of the determination in (1); (3) A detection device that detects the occurrence of a bridge in the hopper based on the determination result of (2) above.
2. a plurality of rows of the positions arranged in a direction perpendicular to the descending direction are arranged along the descending direction, 2. The detection device according to claim 1, wherein the detection unit performs the determinations (1) and (2) for each of the plurality of rows, and detects the occurrence of a bridge in the hopper based on the determination results for each row.
3. a determination unit that determines whether or not the object is captured in a target area in the image, the target area being an area in which the position is set; The detection device according to claim 1 , wherein the movement data generation unit generates the movement data indicating an amount of movement between the images in which the determination unit has determined that the object appears in the object region.
4. A method for detecting a bridge executed by one or more information processing devices, comprising: a movement data generating step of generating movement data indicating a movement state of the object during a period when a plurality of images were taken from above a hopper storing the object, based on the plurality of images taken in time series; a detection step of detecting the occurrence of a bridge in the hopper based on the movement data generated in the movement data generation step, In the movement data generating step, for an image element appearing at each of a plurality of predetermined positions set on the inclined surface of the hopper appearing in the images, the movement data is generated which indicates an amount of movement of the image element in a downward direction between the images; In the detection step, (1) determining whether the amount of movement is within a normal range for each of a plurality of positions arranged in a row in a direction perpendicular to the descending direction; (2) determining whether the object is moving normally in the line based on the result of the determination in (1); (3) A detection method for detecting the occurrence of a bridge in the hopper based on the determination result of (2).
5. A detection program for causing a computer to function as the detection device according to claim 1, the detection program causing a computer to function as the movement data generating unit and the detection unit.
6. an imaging device for imaging a hopper that contains an object from above; a detection device that detects the occurrence of a bridge in the hopper based on movement data that is generated from a plurality of images in time series captured by the photographing device and indicates a movement state of the object during a period in which the plurality of images were captured; and an alarm device that notifies the occurrence of a bridge when the detection device detects the occurrence of a bridge, the movement data is data indicating, for an image element shown in each of a plurality of predetermined positions set on the inclined surface of the hopper shown in the image, an amount of movement of the image element in a downward direction between the images; The detection device includes: (1) determining whether the amount of movement is within a normal range for each of a plurality of positions arranged in a row in a direction perpendicular to the descending direction; (2) determining whether the object is moving normally in the line based on the result of the determination in (1); (3) A detection system that detects the occurrence of a bridge in the hopper based on the determination result of (2).
Citation Information
Patent Citations
Vibration-proof mechanism of high-speed rotary driving apparatus
JP1983017242A
Refuse supply device
JP1998078208A
Refuse incinerating furnace, method and apparatus for detecting hopper bridge
JP1998238735A
Refuse incineration device
JP1998288327A
Combustion furnace pipe clogging monitoring device and combustion furnace pipe clogging monitoring method
JP2016035337A
