Detection device, detection method, detection program, and detection system

Abstract

In early detection, bridging occurs. The detection device (1) includes a movement data generation unit (103) that generates movement data indicating a movement state of an object during a period in which a plurality of images are captured, based on the plurality of images captured in time series from above a hopper that stores the object, and a detection unit (104) that detects the occurrence of bridging based on the movement data.

Description

Technical Field

[0001] This invention relates to a detection device for detecting bridging in a hopper, etc. Background Technology

[0002] Typical waste incineration facilities include funnel-shaped devices called hoppers. Waste stored in a storage facility called a waste pit is lifted by cranes and fed into the hopper, and then sent from the hopper into the incinerator for combustion. In such incineration facilities, the waste fed into the hopper may form bridging structures that cause blockages; this phenomenon is called bridging.

[0003] When bridging occurs, the waste above the bridging remains in the same position, maintaining a high level of waste accumulated in the hopper. Normally, a decrease in the waste level triggers the addition of waste to the hopper; therefore, if bridging occurs, no new waste can be added. On the other hand, the waste below the bridging is fed into the incinerator for sequential combustion. If bridging is left unattended, the amount of waste in the incinerator decreases, and further neglect will result in a state where there is no waste being burned in the incinerator.

[0004] If the amount of waste in the incinerator decreases or there is no waste, the temperature inside the incinerator will drop, requiring heating using burners or the like, and sometimes it may be necessary to shut down the incinerator. As a technology to avoid such a situation, patent document 1 can be cited as an example. Patent document 1 describes a method for detecting bridging based on periodically measured hopper levels. Existing technical documents Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 10-238735 Summary of the Invention (a) Technical problems to be solved

[0006] However, because waste contains various objects and has an irregular shape, and sometimes some waste jumps up or gets stuck in the upper part of the hopper when it is being fed in, it is difficult to reliably and accurately measure the hopper level. Therefore, when detecting bridging based on hopper level, a detection threshold must be set to avoid frequent false detections, resulting in a significant time lag between the occurrence of bridging and its detection. Furthermore, this problem is not limited to hoppers installed in waste incinerators but is a common issue for hoppers used for any purpose where bridging may occur.

[0007] One aspect of the present invention addresses the aforementioned problems, and its object is to provide a detection device or the like capable of detecting bridging at an early stage. (II) Technical Solution

[0008] To solve the above problems, a detection apparatus according to one aspect of the present invention includes: a movement data generation unit that generates movement data representing the movement state of the object during the period of taking multiple images obtained by taking pictures of a hopper containing an object from above; and a detection unit that detects the occurrence of bridging in the hopper based on the movement data generated by the movement data generation unit.

[0009] In addition, to solve the above problems, one aspect of the present invention provides a bridging detection method executed by one or more information processing devices, comprising: a movement data generation step, generating movement data representing the movement state of the object during the period of taking the multiple images based on a series of images obtained by taking pictures of a hopper containing an object from above; and a detection step, detecting the occurrence of bridging in the hopper based on the movement data generated in the movement data generation step.

[0010] In addition, to solve the above-mentioned problems, a detection device according to one aspect of the present invention includes: a movement data generation unit that generates movement data representing the movement state of the object during the period of taking multiple images of a hopper containing an object, obtained by taking multiple images in a time sequence from above; and a detection unit that detects the occurrence of bridging in the hopper based on the output value obtained by inputting the movement data into a learned model, wherein the learned model has learned the relationship between the movement state of the object and whether bridging has occurred when the object is in that movement state.

[0011] In addition, to solve the above problems, a detection system according to one aspect of the present invention includes: a photographic device for photographing a hopper containing an object from above; a detection device for detecting the occurrence of bridging in the hopper based on movement data, the movement data being generated based on a series of images captured by the photographic device in a time sequence, representing the movement state of the object during the period of capturing the series of images; and a notification device for notifying that bridging has occurred when the detection device detects that bridging has occurred. (III) Beneficial Effects

[0012] According to one aspect of the present invention, bridging can be detected at an early stage. Attached Figure Description

[0013] Figure 1 This is a block diagram illustrating an example of the main structure of the detection device according to Embodiment 1 of the present invention. Figure 2 This is a diagram illustrating a structural example of the detection system according to Embodiment 1 of the present invention. Figure 3 This is a diagram illustrating an example of a method for determining whether or not garbage is present. Figure 4 This is a diagram illustrating an example of a method for generating mobile data. Figure 5 This diagram illustrates the detection method for detecting bridging. Figure 6 This is a flowchart illustrating an example of the processing performed by the aforementioned detection device. Figure 7 This is a block diagram illustrating an example of the main structure of the detection device according to Embodiment 2 of the present invention. Detailed Implementation

[0014] [Implementation Method 1] (System Architecture) based on Figure 2 The structure of the detection system 100 of this embodiment will be described. Figure 2 This is a diagram showing a structural example of the detection system 100. Figure 2 An example of applying the detection system 100 to bridging detection in a hopper A installed in a waste incineration facility is shown. Furthermore, the detection system 100 is not limited to hopper A but can be applied to hoppers for any purpose where bridging may occur. The object contained in hopper A is waste G, but the detection system 100 can detect bridging in hoppers containing any object other than waste G.

[0015] exist Figure 2 The waste incineration facility shown includes a waste pit B for storing waste G and an incinerator C for incinerating the waste G. Waste G from the waste pit B is fed into a hopper A via a crane. An inclined surface A1, extending outwards, is formed at the top of the hopper A. Waste G slides down this inclined surface A1 and enters the interior of the hopper A, where it is temporarily contained. Furthermore, the waste G from the hopper A is sequentially fed into the incinerator C through an opening A2 at its lower part for incineration. Additionally, in… Figure 2 In the example, the area near opening A2 becomes a void where there is no waste G. This is because the waste forms a bridging blockage in hopper A, which is caused by bridging. Under normal operating conditions where bridging does not occur, waste G will also accumulate near opening A2.

[0016] As described above, the detection system 100 is a system for detecting bridging in hopper A, and includes a detection device 1, an imaging device 2, and a notification device 3. The imaging device 2 is positioned above hopper A and takes images of hopper A from above at predetermined intervals. Furthermore, the detection device 1 acquires the images taken by the imaging device 2 and uses these images to detect bridging in hopper A. In addition, the notification device 3 notifies the user of the detection system 100 that bridging has occurred when the detection device 1 detects it.

[0017] also, Figure 2An example of bridging occurring using sound notification is shown, specifically an example where the notification device 3 is a sound output device, but this is not an isolated case. The notification method can be any method that enables the recipient to recognize that bridging has occurred, and the notification device 3 can be any device corresponding to the notification method. For example, the notification device 3 could also be an alarm light or a display device.

[0018] More specifically, the imaging device 2 captures multiple images of hopper A in sequence. Furthermore, the detection device 1 detects bridging in hopper A based on movement data generated from the multiple images captured by the imaging device 2, representing the movement state of the waste G during the period in which the multiple images are captured.

[0019] Here, the inventors of this invention, through careful observation of images obtained by taking pictures of various hoppers from above, discovered that images taken during bridging exhibit different characteristics compared to images taken when bridging does not occur. More specifically, they discovered that the movement state of objects reflected in the images differs between during bridging and during normal operation.

[0020] The detection device 1 detects bridging based on this insight. As mentioned above, the movement state of the debris G reflected in the image differs when bridging occurs compared to normal operation. Therefore, bridging can be detected based on movement data generated from multiple images in a time sequence, representing the movement state of the debris G during the capture of multiple images. Furthermore, details of this detection method will be described later.

[0021] Furthermore, since the movement characteristic of bridging during bridging can be observed in images immediately after bridging occurs, the aforementioned structure allows for early detection of bridging. Moreover, according to this structure, detection can be performed more stably and with higher accuracy compared to bridging detection based on hopper levels. This is because it is difficult to stably and accurately measure the height of amorphous objects using a hopper level, whereas the movement characteristic of bridging during bridging can be stably observed in images obtained by photographing hopper A from above.

[0022] As described above, the detection system 100 includes a detection device 1, an imaging device 2, and a notification device 3. The imaging device 2 takes images of the hopper A containing waste G from above. The detection device 1 detects bridging in the hopper A based on movement data generated from multiple sequential images taken by the imaging device 2, representing the movement state of the waste G during the period of taking these images. The notification device 3 notifies the system that bridging has occurred when the detection device 1 detects it. With this structure, bridging can be detected and notified at an early stage.

[0023] Furthermore, there are no particular limitations on the location where the detection device 1 can be installed. For example, it can be installed in a waste incineration facility or in a monitoring center that monitors multiple waste incineration facilities. Additionally, Figure 2 An example is shown where one detection device 1 monitors and detects bridging in one hopper A. However, one detection device 1 can also monitor and detect bridging in multiple hoppers. In this case, the detection device 1 can be, for example, a cloud server.

[0024] Furthermore, the handling procedures performed when bridging is detected are not limited to the examples described above. For instance, when a bridging removal device is installed in a waste incineration facility to eliminate bridging, the detection device 1 activates the bridging removal device when bridging is detected, automatically eliminating the bridging. The bridging removal device can be any device capable of eliminating bridging, such as a device equipped with a pressing component that presses down the waste accumulated in the hopper.

[0025] (Structure of the detection device) based on Figure 1 The structure of the detection device 1 will be described. Figure 1 This is a block diagram showing an example of the main structure of the detection device 1. As shown, the detection device 1 includes: a control unit 10 that controls all parts of the detection device 1 in general, and a storage unit 11 that stores various data used by the detection device 1. In addition, the detection device 1 includes: a communication unit 12 for communicating with other devices; an input unit 13 for receiving various data input to the detection device 1; and an output unit 14 for outputting various data from the detection device 1.

[0026] Additionally, the control unit 10 includes a data acquisition unit 101, a determination unit 102, a motion data generation unit 103, a detection unit 104, and a notification control unit 105. Furthermore, the storage unit 11 stores an image DB111.

[0027] The data acquisition unit 101 acquires multiple time-series images obtained by photographing the hopper A from above. As described above, in the detection system 100, the imaging device 2 captures multiple time-series images obtained by photographing the hopper A from above; therefore, the data acquisition unit 101 only needs to acquire the images captured by the imaging device 2. Furthermore, the data acquisition unit 101 can acquire images from the imaging device 2 via communication via the communication unit 12, or it can acquire images input via the input unit 13. The data acquisition unit 101 records the acquired images in the image DB111. Alternatively, the imaging device 2 can also be a device for capturing moving images. In this case, the data acquisition unit 101 only needs to acquire multiple time-series images (e.g., frame images at predetermined time intervals) from the moving images captured by the imaging device 2.

[0028] The determination unit 102 determines whether the object region in the image obtained by the data acquisition unit 101, which is the parsing object used to detect whether bridging has occurred, contains garbage. The method for determining garbage in the object region will be explained later. Figure 3 To elaborate further.

[0029] The mobile data generation unit 103 generates mobile data representing the movement status of trash during the period when multiple images are captured, based on multiple images acquired and recorded in the image DB111 by the data acquisition unit 101. The method for generating mobile data will be discussed later. Figure 4 To elaborate further.

[0030] The detection unit 104 detects bridging in hopper A based on the movement data generated by the movement data generation unit 103. The detection method for detecting bridging will be discussed later. Figure 5 To elaborate further.

[0031] When the detection unit 104 detects bridging, the notification control unit 105 notifies that bridging has occurred. Specifically, when the detection unit 104 detects bridging, the notification control unit 105 causes the notification device 3 to output an audible signal indicating that bridging has occurred. Furthermore, the notification method for bridging is not particularly limited as long as it enables the recipient to recognize that bridging has occurred. For example, when the detection device 1 is located near the recipient, the notification control unit 105 can cause the output unit 14 to output information indicating that bridging has occurred. Alternatively, the notification control unit 105 can send a message to the recipient's terminal device to provide notification.

[0032] As described above, the detection device 1 includes a movement data generation unit 103 and a detection unit 104. The movement data generation unit 103 generates movement data representing the movement state of the waste during the period in which the multiple images are captured, based on a series of images obtained from above the hopper A. The detection unit 104 detects bridging in the hopper A based on the movement data generated by the movement data generation unit 103. With this structure, bridging can be detected at an early stage.

[0033] Furthermore, motion data can be generated using only two images taken at different times, as detailed later. Therefore, the use of detection device 1 in waste incineration facilities can begin immediately without the need for machine learning or similar techniques. Additionally, detection can be performed without being affected by slight changes in the installation location or shooting angle of the imaging device 2. Moreover, even if stains adhere to the lens of the imaging device 2 installed in the waste incineration facility, the impact of such stains on the generation of motion data is limited because the stains do not move. In other words, the detection system 100 including detection device 1 has the advantages of easy implementation and stable operation.

[0034] (Regarding the determination of whether or not there is garbage) use Figure 3 The determination of the presence or absence of garbage by the determination unit 102 is explained. Figure 3 This is a diagram illustrating an example of a method for determining whether or not garbage is present. Figure 3 Image D shown is an image obtained by taking a picture of the hopper from above. Image D shows waste sliding down the inclined surface of the hopper, but not the entire inclined surface. Specifically, waste is shown in region d2 on the downstream side of the inclined surface of the hopper, but not in region d1 on the upstream side.

[0035] Here, if the motion data generation unit 103 generates motion data for an area such as area d1 where no garbage is reflected, the amount of motion represented by the motion data may be zero or close to zero. Furthermore, when the amount of motion is zero or close to zero, even if there is actually no garbage and no bridging has occurred, it is possible to falsely detect that bridging has occurred.

[0036] To avoid false detections like the one described above, the detection device 1 includes a determination unit 102. The determination unit 102 determines whether the object area in the image contains garbage. Furthermore, a movement data generation unit 103 generates movement data, which represents the amount of movement between images determined by the determination unit 102 to contain garbage in the object area. Therefore, images where the object area does not contain the object will not be used for bridging detection, thus preventing false detections like the one described above. Furthermore, it is preferable that the object area is located on the inclined surface of the hopper; more specifically, it is located on the downstream inclined surface, as in region d2, as will be described later.

[0037] The presence of litter can be determined using captured images, or by performing specific image processing on the captured images to easily determine whether litter is present, and then making a judgment based on this. For example, the Canny method can be used for image edge detection. The Canny method is an algorithm used for edge detection in images. Figure 3Image D1 shown is generated by edge detection of image D using the Canny method. The appearance of waste is often much more complex than the surface shape of the hopper. Therefore, in Figure 3 In the example, compared to region d1 where no trash was reflected, a large number of edges were detected in region d2 where trash was reflected.

[0038] Therefore, the determination unit 102 can use the Canny method to perform edge detection on the image obtained by photographing the hopper, and determine the image with the number of edges detected in the object area that is above a threshold as an image showing garbage, and determine the image with the number of edges detected in the object area that is less than a threshold as an image not showing garbage.

[0039] Of course, edge detection methods other than the Canny method can also be applied. Furthermore, the presence or absence of garbage can be determined using methods other than edge detection. For example, the determination unit 102 can determine the presence or absence of garbage by analyzing at least one of the brightness value and RGB value of each pixel constituting the image. Alternatively, the determination unit 102 can determine the presence or absence of garbage using a learned model (e.g., a neural network model) that has been learned in a way that can determine the presence or absence of garbage.

[0040] (Methods for generating mobile data) use Figure 4 The method for generating mobile data by the mobile data generation unit 103 is explained. Figure 4 This is a diagram illustrating an example of a method for generating mobile data. More specifically, Figure 4 The example shown illustrates the generation of motion data F using image correlation, which represents the movement state of waste between image E1, captured at time t on the inclined surface of the hopper, and image E2, captured at time t+Δt on the inclined surface of the hopper. Furthermore, in Figure 4 In the diagram, the y-direction is the direction in which the waste descends, that is, from the inclined surface of the hopper (refer to...). Figure 2 The direction from the upstream side of the inclined surface A1 to the downstream side, the x direction is the direction perpendicular to the y direction on the inclined surface.

[0041] Image correlation is a method that, for two time-series images, calculates the movement of each particle reflected in each image through image processing and represents the calculated movement as a vector. For example, in image E1, the trash indicated by the dashed bounding box is reflected at position e1, but in image E2, it is reflected at position e2, which is lower than position e1. The movement state of this trash is represented in the movement data F as a vector f1 representing the displacement from position e1 to position e2. Vectors are calculated similarly for other positions.

[0042] Thus, the movement data F represents the movement state of the waste at each location in images E1 and E2. Specifically, according to the movement data F, the waste at each location in images E1 and E2 moves in the y-direction (downstream of the hopper's inclined surface, i.e., the downward direction). Furthermore, in the movement data F, vectors with y-direction component values ​​above a threshold are represented by solid arrows, while vectors with values ​​below the threshold are represented by dashed arrows. This indicates that there are locations where the waste moves faster and slower.

[0043] As described above, the motion data generation unit 103 can generate motion data representing the amount of motion of an image element that is reflected in a plurality of predetermined positions in the image E1 and E2 along a downward direction. The plurality of predetermined positions are set on an inclined surface reflected in the images E1 and E2.

[0044] Since the waste slides down the inclined surface of the hopper in approximately the same downward direction, the inclined surface is a suitable location for determining whether the waste is falling smoothly or has stopped falling. Therefore, based on the above structure, accurate movement data indicating whether the waste is falling smoothly or has stopped falling can be generated. Furthermore, by using this movement data to detect bridging, highly reliable detection can be performed.

[0045] Furthermore, the motion data can also be used to generate image portions that reflect areas other than the existing inclined surface. In this case, the detection unit 104 only needs to use the motion data within the object area defined in the portion reflecting the inclined surface from the motion data generated by the motion data generation unit 103 to detect the bridge.

[0046] Of course, the target area can be any area where the movement of waste is in roughly the same direction, or it can be any area outside the inclined surface. For example, the target area can be the area on the inner surface of the hopper that is connected to the inclined surface on the downstream side (usually extending in the vertical direction), or it can be the area that includes both the inclined surface and other parts.

[0047] Furthermore, motion data can be any data representing the movement status of the garbage, and is not limited to the examples above. For example, the speed at which the garbage moves can be used as motion data. Alternatively, a difference image representing the difference between two images taken at different times can also be used as motion data.

[0048] (Detection methods for detecting bridging) use Figure 5 The detection method for the occurrence of bridging in the detection section 104 is explained. Figure 5 This diagram illustrates the detection method for detecting bridging. In more detail, Figure 5 An example is shown of detecting bridging using motion data H generated from multiple time-series images obtained from an overhead camera capturing the hopper. Furthermore, the motion data H is generated using the image correlation method described above, representing data of motion vectors of image features respectively mapped to multiple predetermined locations, which are defined in the images obtained from capturing the hopper. Additionally, in Figure 5 In the moving data H, the object region is represented by a bounding box h1. This object region is the parsing object used to detect whether bridging has occurred.

[0049] Figure 5 Image J shown is an enlarged version of the object area in an image obtained by photographing the hopper. Furthermore, the positions where the calculated movement vectors are calculated are depicted in image J. For example, the movement vector calculated at position j1 ​​is (-1, 3). As shown, the positions of the calculated movement vectors are uniformly set throughout image J. More specifically, multiple positions are located in a direction perpendicular to the direction of waste descent, i.e. Figure 5 They are arranged in a column along the x-direction, and this column is along the direction of descent of the garbage, i.e. Figure 5 There are multiple arrangements in the y-direction.

[0050] When determining whether bridging has occurred, the detection unit 104 first checks multiple locations arranged in a row along the x-direction to determine whether the amount of garbage movement is within the normal range.

[0051] For example, when setting the y-value of the movement vector, i.e., the amount of movement along the downward direction of the trash, as D, the normal range of D can be set to D ≥ m1. m1 is the lower limit of the normal range. Furthermore, depending on the location, sometimes the movement amount D is negative. This indicates that at that location, the trash is moving in the opposite direction to its descent. Since such movement is not usually observed, locations where the movement amount D is negative can be set to "cannot be determined".

[0052] For example, the detection unit 104 can use the following conditional formulas (1) to (3) to evaluate each position in three stages. D≥m1 (1) m1>D>0 (2) D≤0 (3) It can be said that the location satisfying condition (1) is a normal movement state where the waste descends smoothly. On the other hand, it can be said that the location satisfying condition (2) is an abnormal movement state characterized by a small amount of waste movement and bridging. Therefore, the detection unit 104 can, for example, set the evaluation value to 1 (indicating no abnormality) for the location satisfying condition (1) and set the evaluation value to 0 (indicating an abnormality) for the location satisfying condition (2). In addition, the detection unit 104 can set the evaluation value to -1 (indicating a value that cannot be determined) for the location satisfying condition (3).

[0053] For example, when m1 = 1, Figure 5 Position j1 ​​in image J satisfies condition (1) above because the y-value of the movement vector is 3. Therefore, in this case, the evaluation value of position j1 ​​is 1. The evaluation values ​​are calculated similarly for other positions.

[0054] Next, the detection unit 104 determines whether the waste has moved normally based on the judgment results of whether the amount of movement for each position is within the normal range, according to the columns of positions arranged along the x-direction. For example, if an evaluation value of 1, 0, or -1 is calculated for each position as described above, the detection unit 104 can evaluate each column in three stages using the following conditional formulas (4) to (6). Furthermore, Sum(D) is the sum of the evaluation values ​​of the amount of movement for positions arranged in a column along the x-direction. Sum(D)>0 (4) Sum(D) = 0 (5) Sum(D) < 0 (6) It can be said that the column satisfying condition (4) is in a normal movement state where the waste descends smoothly as a whole. On the other hand, it can be said that the column satisfying condition (5) is in an abnormal movement state where the overall amount of waste movement is small and bridging has occurred. Therefore, the detection unit 104 can set the evaluation value to 1 for the column satisfying condition (4) and set the evaluation value to 0 for the column satisfying condition (5). In addition, the detection unit 104 can set the evaluation value to -1 for the column satisfying condition (6).

[0055] For example, in Figure 5 Column K1 in image J contains position j1, totaling 5 positions. Therefore, Sum(D) is calculated for column K1, which is the sum of the evaluation values ​​of the movement calculated for each of these 5 positions. Based on the calculated Sum(D), the evaluation value of column K1 is calculated using conditional expressions (4) to (6). The evaluation values ​​are also calculated for the other columns in the same way. Figure 5The evaluation values ​​for each column are displayed in column L. In this example, all columns have an evaluation value of 1, which is considered normal.

[0056] Finally, the detection unit 104 generates a comprehensive judgment result based on the judgment results of each column, and detects bridging based on this judgment result. For example, if the evaluation value of each column calculated as described above is set to J, and the sum of the evaluation values ​​of all columns is set to Sum(J), the detection unit 104 can generate a comprehensive judgment result using the following conditional expressions (7) to (9), and detect bridging based on this comprehensive judgment result. In addition, m2 is a preset threshold. Sum(J)≥m2 (7) Sum(J) < m2 (8) -1∈J(9) When condition (7) is met, it can be said that the overall movement of the waste is in a normal state of smooth descent, and therefore the detection unit 104 sets the overall judgment result to 1, determining that bridging has not occurred. On the other hand, when condition (8) is met, it can be said that the overall movement of the waste is small and it is an abnormal movement state characteristic of when bridging has occurred, and therefore the detection unit 104 sets the overall judgment result to 0, determining that bridging has occurred. Furthermore, the condition (9) is met when at least one column cannot be determined. In this case, the detection unit 104 only needs to set the overall judgment result to -1, indicating that bridging cannot be determined.

[0057] As described above, the detection unit 104 can (i) determine whether the amount of movement is within the normal range for multiple positions arranged in a column along the x direction, (ii) determine whether the garbage in the column is moving normally based on the determination result of (i), and (iii) detect the occurrence of bridging based on the determination result of (ii).

[0058] Typically, waste consists of multiple components. Therefore, the components of waste arranged in a column perpendicular to the direction of descent will slide down the hopper in a manner that roughly maintains that column's position. Thus, detecting the movement characteristic of bridging events by analyzing columns perpendicular to the direction of descent is effective. However, sometimes the amount of movement of the waste components can deviate depending on their position on the hopper. Therefore, for example, it can be envisioned that even when the waste descends smoothly overall, some locations may experience movement outside the normal range.

[0059] Therefore, based on the above structure, the following structure is adopted: Based on the determination result of whether the movement amount of each position arranged in a column is within the normal range, it is determined whether the waste in that column is moving normally. Thus, even if there is a deviation in the movement amount of each position arranged along the x-direction, it is possible to appropriately determine whether the waste is moving normally, and based on this determination result, bridging can be detected with high precision.

[0060] Furthermore, as described above, the detection unit 104 can perform the determinations (i) and (ii) above for each of the multiple columns, and detect bridging in the hopper based on the determination results of each column. According to this structure, bridging is detected based on the determination results of whether the waste in each column is moving normally, thus enabling high-precision detection of bridging even if the determination results of each column deviate.

[0061] Furthermore, the above-described determination method is only one example, and various methods can be applied as a bridging detection method based on movement data. For example, the detection unit 104 can determine the amount of movement at each location within the target area based on movement data, and compare the sum or average of the determined movement amounts with a predetermined threshold. If the sum is above the threshold, it is determined to be normal; if the sum is below the threshold, it is determined to be abnormal, i.e., bridging has occurred.

[0062] Alternatively, for example, the detection unit 104 may use only the positions arranged in a column along the x-direction (e.g., Figure 5 The occurrence of bridging is detected by the movement data of each position contained in column K1. In this case, for example, the detection unit 104 can detect the occurrence of bridging based on the evaluation results based on the above-described conditional formulas (4) to (6).

[0063] Furthermore, the various thresholds including the aforementioned threshold m1 used to detect bridging can be fixed or variable values. For example, when the rate at which waste is fed from the hopper into the incinerator (feeding speed) is high, the waste at the top of the hopper also moves quickly. Therefore, the threshold m1 can be adjusted in tandem with the feeding speed. Specifically, the faster the feeding speed, the larger the threshold m1 can be set to. Thus, even when the feeding speed is high but the waste at the top of the hopper moves slowly, bridging can be detected.

[0064] (Processing flow) use Figure 6 The process flow (detection method) of the detection device 1 is explained. Figure 6 This is a flowchart illustrating an example of the processing performed by the detection device 1. Furthermore, for example, this is performed whenever the imaging device 2 captures a new image. Figure 6 The processing.

[0065] In S1, the data acquisition unit 101 acquires a timing image obtained from the overhead camera hopper and records it in image DB111. As described above, due to the camera device 2 (refer to...) Figure 2 These images are captured, so the data acquisition unit 101 only needs to acquire the images captured by the photography device 2.

[0066] In S2, the determination unit 102 determines whether the object region in the image obtained through S1 contains existing garbage. The method for determining whether the object region contains existing garbage is similar to that based on... Figure 3 As explained earlier, this will not be repeated here. If the condition in S2 is yes, proceed to S3; if the condition in S2 is no, the process ends. Figure 6 The processing.

[0067] In step S3 (mobility data generation), the mobility data generation unit 103 generates mobility data representing the movement status of trash during the period when these images were captured, based on the images obtained in step S1 and images captured before those images. For example, if an image captured at time t+Δt is obtained in step S1, the mobility data generation unit 103 reads the image captured at time t from image DB111. Furthermore, the mobility data generation unit 103 generates mobility data representing the movement status of trash during the period from time t to time t+Δt. Regarding the method for generating mobility data, it is similar to that based on... Figure 4 As explained above, this will not be repeated here.

[0068] In S4, the detection unit 104 uses the movement data generated in S3 to determine whether the amount of movement is within the normal range for each position within the target area defined in the image obtained in S1. In the following S5, based on the determination result of S4, the detection unit 104 determines whether the movement is within the normal range in the direction perpendicular to the direction of the garbage's descent. Figure 5 The detection unit 104 checks whether the movement status of each column arranged at the aforementioned positions in the x-direction is normal. Furthermore, in S6, the detection unit 104 generates a comprehensive judgment result based on the judgment results of each column in S5. Moreover, the judgment methods for S4 to S6 are similar to those based on... Figure 5 As explained, I will not repeat the details here.

[0069] In step S7 (detection step), the detection unit 104 detects the occurrence of bridging based on the comprehensive judgment result of S6. For example, if the comprehensive judgment result is represented by any value among 1 (no abnormality), 0 (abnormality), and -1 (cannot be determined), and the comprehensive judgment result of S6 is 1, then the detection unit 104 determines that bridging has not occurred (no in S7). In this case, the process ends. Figure 6The processing continues. On the other hand, if the overall determination result of S6 is 0, the detection unit 104 determines that bridging has occurred (yes in S7). That is, in this case, the detection unit 104 detects that bridging has occurred. Then, the processing proceeds to S8.

[0070] Furthermore, the handling of situations where a decision cannot be made can be predetermined. For example, the detection unit 104 can terminate the process if a decision cannot be made. Figure 6 In this case, the system can also enter S8 and notify the user that a decision cannot be made.

[0071] In S8, the notification control unit 105 notifies that bridging has occurred. Specifically, the notification control unit 105 notifies that bridging has occurred by having the notification device 3 output an audible sound indicating that bridging has occurred. Furthermore, when a bridging removal device for eliminating bridging is provided in the waste incineration facility, in S8, the notification control unit 105 may activate the bridging removal device to eliminate the bridging, either while issuing a notification or without issuing a notification.

[0072] As described above, the detection method of this embodiment includes a movement data generation step (S3) and a detection step (S7). In the movement data generation step (S3), movement data representing the movement state of the waste during the period in which the multiple images are captured is generated based on multiple time-series images of a hopper containing waste taken from above. In the detection step (S7), bridging is detected based on the movement data generated by the movement data generation step (S3). According to this detection method, bridging can be detected at an early stage.

[0073] Furthermore, as described at the beginning of Embodiment 1, the detection system 100 is not limited to the hopper of a waste incineration facility, but can be applied to any hopper that may cause bridging, and the object contained in the hopper is not limited to waste. In other words, the "waste" described in the above embodiment can be replaced with any "object".

[0074] [Implementation Method 2] Other embodiments of the present invention will be described below. Furthermore, for ease of explanation, components having the same functions as those described in the above embodiments will be labeled with the same reference numerals and will not be described repeatedly.

[0075] (Structure of the detection device) use Figure 7 The structure of the detection device 1A in this embodiment will be described. Figure 7 This is a block diagram illustrating an example of the main structural components of the detection device 1A. The detection device 1A and... Figure 1 Compared to the detection device 1 shown, the difference is that the detection unit 104 is changed to the detection unit 104A; and the learned model 112 is stored in the storage unit 11.

[0076] The detection unit 104A detects bridging in the hopper based on the output value obtained by inputting the movement data generated by the movement data generation unit 103 into the learned model 112. The learned model 112 has learned the relationship between the movement state of the object and whether bridging has occurred when the object is in that movement state. Furthermore, the movement data is generated by the movement data generation unit 103 in the same way as in Embodiment 1. Additionally, the object can be garbage or other objects besides garbage.

[0077] As described above, the detection device 1A includes a motion data generation unit 103 and a detection unit 104A. The motion data generation unit 103 generates motion data representing the motion state of the object during the period when the multiple images are captured, based on multiple time-series images obtained by taking pictures of the hopper containing the object from above. The detection unit 104A detects the occurrence of bridging in the hopper based on the output value obtained by inputting the motion data into a learned model 112, which has learned the relationship between the motion state of the object and whether bridging has occurred when the object is in that motion state.

[0078] Similar to the detection device 1 in Embodiment 1, the above structure is based on the understanding that the amount of movement of the object reflected in the image differs between when bridging occurs and during normal operation. As described above, since the movement state of the object reflected in the image differs between when bridging occurs and during normal operation, the occurrence of bridging can be detected according to the above structure provided by the detection device 1A.

[0079] Furthermore, since the movement characteristic of bridging can be observed in the image immediately after bridging occurs, detection device 1A, similar to detection device 1 in Embodiment 1, can detect bridging at an early stage. Moreover, similar to detection device 1 in Embodiment 1, detection device 1A can perform detection more stably and with higher accuracy compared to bridging detection based on hopper level.

[0080] (Regarding the learned model) As described above, the learned model 112 is generated in the following way: the relationship between the movement state of the learned object and whether bridging has occurred when the object is in that movement state.

[0081] When generating teacher data for generating the learned model 112, the following are collected first: multiple time-series images of the hopper taken from above during bridging, and multiple time-series images of the hopper taking pictures from above during normal operation.

[0082] Next, motion data is generated based on the images collected in the manner described above. Specifically, multiple images are grouped into pairs in chronological order, and motion data is generated for each pair. This processing is performed on images taken during bridging and images taken during normal operation. Thus, motion data during bridging and motion data during normal operation are generated respectively. Furthermore, the motion data only needs to be data representing the movement state of the object. For example, motion data can be generated using an image correlation method, similar to Embodiment 1.

[0083] In addition, correct data is appended to the movement data when bridging occurs, and this correct data is used as teacher data. This correct data indicates an anomaly, i.e., bridging has occurred. Conversely, correct data is appended to the movement data when bridging is normal, and this correct data is used as teacher data. This correct data indicates normal operation, i.e., no bridging has occurred.

[0084] By using this teacher data for machine learning, a learned model 112 is generated that classifies input motion data into normal motion data and abnormal motion data. Furthermore, the machine learning algorithm need only be able to generate the learned model 112, which can classify input motion data into normal motion data and abnormal motion data. For example, a convolutional neural network with high image classification accuracy is preferred, but it is not limited to this example.

[0085] Furthermore, the teacher data for the learned model 112 can also include features extracted from the movement data. Such features represent the amount of movement of the object and are also a type of movement data. Additionally, the teacher data for the learned model 112 can also include data other than movement data. For example, the teacher data can also include: the number of edges calculated using the Canny method based on an image of the hopper taken from above, or the RGB values ​​of that image, etc.

[0086] (Processing flow) The processing flow performed by detection device 1A and Figure 6 Same. However, Figure 6 The processes S4 to S6 are replaced by a process of inputting the moving data into the learned model 112 to obtain the output value. In addition, in S7 (detection step), bridging is detected based on the output value of the learned model 112.

[0087] In other words, the detection method of this embodiment includes a movement data generation step and a detection step. In the movement data generation step, movement data representing the movement state of the object during the period in which the multiple images are captured are generated based on multiple time-series images obtained by photographing a hopper containing an object from above. In the detection step, bridging in the hopper is detected based on the output value obtained by inputting the movement data into a learned model 112, which has learned the relationship between the object's movement state and whether bridging has occurred when the object is in that movement state. According to this detection method, bridging can be detected at an early stage.

[0088] [Variation Example] The entity executing each process described in the above embodiments is arbitrary and not limited to the examples above. That is, as long as the processes described in the above embodiments can be executed, the apparatus constituting the detection system 100 can be appropriately modified.

[0089] For example, in Figure 1 In the detection system 100, the detection device 1 and the notification device 3 are separate devices, but the detection device 1 may also have a structure that includes the notification device 3. Alternatively, the function of the detection device 1 may be distributed among multiple information processing devices (computers). For example, a cloud-based information processing device may generate mobile data and send it to an information processing device located in the waste incineration facility, enabling the information processing device to detect the bridging.

[0090] [Software-based implementation example] The functions of the detection devices 1 and 1A (hereinafter referred to as "devices") can be realized by a program that enables the computer to function as the device, that is, by a program (detection program) that enables the computer to function as each control block of the device (especially each part included in the control unit 10).

[0091] In this case, the aforementioned device includes a computer as hardware for executing the aforementioned program, the computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory). By utilizing the control device and the storage device to execute the aforementioned program, the functions described in the above embodiments are realized.

[0092] The above-described program can be recorded on one or more non-transitory recording media that are computer-readable. The above-described device may or may not have such a recording medium. In the latter case, the above-described program can be provided to the above-described device via any wired or wireless transmission medium.

[0093] Furthermore, some or all of the functions of the aforementioned control blocks can also be implemented using logic circuits. For example, integrated circuits that form the logic circuits that enable the functions of the aforementioned control blocks are also included within the scope of this invention. Additionally, the functions of the aforementioned control blocks can also be implemented using a quantum computer, for example.

[0094] This invention is not limited to the embodiments described above. Various modifications can be made within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of this invention. Explanation of reference numerals in the attached figures

[0095] 100 Detection System 1. Detection device 102 Judgment Department 103 Mobile Data Generation Department 104 Testing Department 2. Photographic equipment 3 Notification Device 1A Detection Device 104A Testing Department.

Claims

1. A detection device comprising: The movement data generation unit generates movement data representing the movement state of the object during the period in which the multiple images are captured, based on multiple time-series images obtained by photographing the hopper containing the object from above; and The detection unit, based on the movement data generated by the movement data generation unit, detects the occurrence of bridging in the hopper. The motion data generation unit generates motion data for image elements projected at multiple predetermined positions, wherein the multiple predetermined positions are set on the inclined surface of the hopper projected in the image, and the motion data represents the amount of movement of the image element along the downward direction between the images. Regarding the aforementioned detection unit (1) For each of the multiple positions arranged in a row in a direction perpendicular to the descent direction, determine whether the amount of movement is within the normal range. (2) Based on the determination result of (1), determine whether the object in the column has moved normally. (3) Detect the occurrence of bridging in the hopper based on the determination result of (2).

2. The detection device according to claim 1, characterized in that, Multiple columns, each consisting of a plurality of positions arranged in a direction perpendicular to the descent direction, are arranged along the descent direction. The detection section performs the determinations (1) and (2) on multiple columns respectively, and detects the occurrence of bridging in the hopper based on the determination results of each column.

3. The detection device according to claim 1 or 2, characterized in that, The image includes a determination unit that determines whether the object is reflected in the region (object region) where the specified position is located in the image. The motion data generation unit generates the motion data, which represents the amount of movement between the images that the determination unit determines are reflected in the object area.

4. A bridging detection method, performed by one or more information processing devices, comprising: The movement data generation step involves generating movement data representing the movement state of the object during the period in which the multiple images were captured, based on a series of time-series images obtained from above of a hopper containing the object; and The detection step, based on the movement data generated in the movement data generation step, detects the occurrence of bridging in the hopper. In the motion data generation step, motion data is generated for image elements that are projected at multiple predetermined locations. These predetermined locations are set on the inclined surface of the hopper projected in the image. The motion data represents the amount of movement of the image element along a descending direction between the images. In the detection step, (1) For each of the multiple positions arranged in a row in a direction perpendicular to the descent direction, determine whether the amount of movement is within the normal range. (2) Based on the determination result of (1), determine whether the object in the column has moved normally. (3) Detect the occurrence of bridging in the hopper based on the determination result of (2).

5. A detection program for enabling a computer to function as the detection device of claim 1, and for enabling the computer to function as the mobile data generation unit and the detection unit.

6. A detection system, comprising: A photographic device used to photograph the hopper containing the objects from above; A detection device detects bridging in the hopper based on motion data generated from multiple sequential images captured by a photographic device, representing the movement state of the object during the period in which the multiple images are captured; and The notification device notifies the detection device that bridging has occurred when it detects bridging. The movement data is generated for image elements projected at multiple predetermined locations, where the predetermined locations are set on the inclined surface of the hopper projected in the image. The movement data represents the amount of movement of the image element along the descending direction between the images. Regarding the aforementioned detection device (1) For each of the multiple positions arranged in a row in a direction perpendicular to the descent direction, determine whether the amount of movement is within the normal range. (2) Based on the determination result of (1), determine whether the object in the column has moved normally. (3) Detect the occurrence of bridging in the hopper based on the determination result of (2).

Images