Faller detection system, faller detection method, and faller detection program

Abstract

The faller detection system has: a first image data acquisition unit that acquires first image data from a first camera that photographs inside a storage device that stores a handled object; a second image data acquisition unit that acquires second image data from a second camera that photographs inside a platform adjacent to the storage device; a first image analysis unit that performs image analysis on the first image data to detect a person inside the storage device; a second image analysis unit that performs image analysis on the second image data to detect a person inside the platform and traces a moving route of the detected person; and a faller determination unit that determines whether there is a faller who falls from the platform to the storage device based on a combination of the analysis result of the first image data and the analysis result of the second image data.

Description

TECHNICAL FIELD

[0001] The present application relates to a faller detection system, a faller detection method, and a faller detection program that automatically detect a person who falls into a facility that stores a processed object. BACKGROUND

[0002] In waste processing-related facilities, particularly in waste incineration facilities, although the number of cases is small, several cases of a person falling into a garbage pit are reported every year.

[0003] When a person falls into a garbage pit, it is necessary to convey to a crane operator or the like, and in order to ensure the safety of the fallen person, the crane operation is temporarily stopped. However, nowadays, automatic driving technology of the crane is developed, and for example, in a case where a fallen person occurs in automatic driving of the crane (that is, when the crane operator is not present), it takes time to convey to the crane operator or the like, and a serious accident can occur.

[0004] In addition, particularly in waste incineration facilities, an accident prevention system that automatically detects a fallen person into a garbage pit and controls a crane or the like does not exist.

[0005] In other fields (particularly in the railway industry), a system that detects a fallen person who falls from a platform of a station to a track by image processing or the like is used (for example, refer to Japanese Patent No. 5386744, Japanese Patent No. 4041678). SUMMARY

[0006] In the fallen person detection system being used in the railway industry, the image of a monitoring camera that simultaneously photographs the fall cause side, that is, the platform, and the fall result side, that is, the track, is processed, and the detection of the fallen person is performed. In relation to this, in the waste incineration facility, the fall cause side, that is, the platform, and the fall result side, that is, the garbage pit, are separated by a garbage input door, and thus it is difficult to set a photographing device such as a camera at a place where the fall cause side and the fall result side can be simultaneously photographed.

[0007] In addition, in the fallen person detection system being used in the railway industry, a method of calculating the size of a falling object by processing the image of a monitoring camera and determining whether the falling object is a person from the size is sometimes used. In relation to this, in the waste incineration facility, a plurality of wastes of different sizes are input into the garbage pit from a vehicle for carrying in (a garbage truck and a light truck, etc.), and thus it is difficult to determine whether the falling object is a person only from the size of the falling object.

[0008] Further, as described above, in the case of a waste incineration facility, it is difficult to arrange a camera in a position where the camera can take an image of both the falling cause side (the platform) and the falling result side (the garbage pit), and thus, for example, a camera is arranged on the garbage pit side, and the image inside the garbage pit is processed and the fallen person is detected. However, it is possible that the waste pile collapses in the garbage pit and the waste is buried on the upper part of the fallen person, and in this case, it is difficult to detect the fallen person by processing the image inside the garbage pit.

[0009] Further, the same problem occurs not only in a facility using a pit and crane system such as an incineration facility, but also in waste treatment facilities (for example, bulky waste crushing facilities and recycling facilities) having a configuration other than the pit and crane system, such as a direct feeding to a storage device and a compactor container system.

[0010] It is desirable to provide a fallen person detection system, a fallen person detection method, and a fallen person detection program capable of automatically detecting a fallen person who falls into a storage device storing waste in a waste treatment facility such as a waste incineration facility.

[0011] A fallen person detection system according to one embodiment of the present application includes:

[0012] a first image data acquisition section that acquires first image data from a first camera that takes an image of inside a storage device storing a processed object;

[0013] a second image data acquisition section that acquires second image data from a second camera that takes an image of inside a platform adjacent to the storage device;

[0014] a first image analysis section that performs image analysis on the first image data to detect a person inside the storage device;

[0015] a second image analysis section that performs image analysis on the second image data to detect a person inside the platform and traces a moving route of the detected person; and

[0016] a fallen person determination section that determines the presence or absence of a fallen person who falls from the platform into the storage device based on a combination of the analysis result of the first image data and the analysis result of the second image data.

[0017] A fallen person detection method according to one embodiment of the present application includes:

[0018] a step of acquiring first image data from a first camera that takes an image of inside a storage device storing a processed object;

[0019] a step of acquiring second image data from a second camera that takes an image of inside a platform adjacent to the storage device;

[0020] a step of performing image analysis on the first image data to detect a person in the storage device;

[0021] a step of performing image analysis on the second image data to detect a person in the platform and tracing a moving route of the detected person; and

[0022] a step of determining whether or not a faller who falls from the platform to the storage device exists based on a combination of the analysis result of the first image data and the analysis result of the second image data. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 is a schematic diagram showing a configuration of a waste treatment facility according to one embodiment.

[0024] Figure 2 is a block diagram showing a configuration of a faller detection system according to one embodiment.

[0025] Figure 3 is a flowchart showing a first example of a faller detection method performed by a faller detection system according to one embodiment.

[0026] Figure 4A is a diagram showing an example of an analysis result of second image data taken from inside the platform.

[0027] Figure 4B is a diagram showing an example of an analysis result of first image data taken inside the garbage pit.

[0028] Figure 5 is a flowchart showing a second example of a faller detection method performed by a faller detection system according to one embodiment.

[0029] Figure 6A is a diagram showing an example of an analysis result of first image data taken inside the garbage pit.

[0030] Figure 6B is a diagram showing an example of an analysis result of second image data taken from inside the platform. DETAILED DESCRIPTION

[0031] The faller detection system according to the first aspect of the embodiment has:

[0032] a first image data acquisition section that acquires first image data from a first camera that takes an image of inside a storage device that stores a treated object;

[0033] a second image data acquisition section that acquires second image data from a second camera that takes an image of inside a platform adjacent to the storage device;

[0034] a first image analysis section that performs image analysis on the first image data to detect a person in the storage device;

[0035] a second image analysis section that performs image analysis on the second image data to detect a person within the platform and traces a movement route of the detected person; and

[0036] a faller determination section that determines whether or not there is a faller who has fallen from the platform to the storage equipment, based on a combination of the analysis result of the first image data and the analysis result of the second image data.

[0037] According to this manner, by combining the analysis result of the first image data taken from within the storage equipment and the analysis result of the second image data taken from within the platform, it is possible to automatically detect a faller who has fallen from the platform to the storage equipment. Generally, in a case where a mechanical learning model is used to detect a faller from image data, the detection accuracy of the mechanical learning model is uneven, and it is not possible to detect a faller with 100% accuracy, but by combining the analysis result of the first image data taken from within the storage equipment and the analysis result of the second image data taken from within the platform, it is possible to improve the detection accuracy of a faller. In addition, even with respect to an existing facility, by additionally providing the first camera that takes an image of the inside of the storage equipment and the second camera that takes an image of the inside of the platform, it is possible to cause the system to operate, and thus it is not necessary to significantly modify the existing facility in order to introduce the system, and it is possible to inexpensively improve the safety of the existing facility.

[0038] The faller detection system of the second aspect of the embodiment is the faller detection system of the first aspect of the embodiment, in which

[0039] the faller determination section confirms the analysis result of the second image data, and in a case where it is detected that a person has entered a predetermined area near the drop door that separates the storage equipment and the platform at the first time, confirms the analysis result of the first image data at the first time, and in a case where it is detected that a person is within the storage equipment, determines that there is a faller.

[0040] According to this manner, the analysis result of the second image data taken from within the platform is confirmed, and in a case where it is detected that a person has entered a predetermined area near the drop door (i.e., a case where it is assumed that a faller is detected in the analysis result of the second image data), the analysis result of the first image data is confirmed, and in a case where it is detected that a person is within the storage equipment (i.e., a case where it is assumed that a faller is detected in the analysis result of the first image data), it is finally determined that there is a faller, and thus it is possible to correctly detect a faller.

[0041] The faller detection system of the third aspect of the embodiment is the faller detection system of the second aspect of the embodiment, in which

[0042] The faller determination unit confirms the analysis result of the image of the region in the storage device corresponding to the position of the drop door, which is captured in the first image data in the first time, when the faller determination unit detects that the person enters the predetermined region near the drop door in the first time.

[0043] According to this manner, even for a large storage device provided with a plurality of drop doors, the analysis result of the second image data captured from inside the platform is confirmed, and in a case where it is detected that a person enters a predetermined region near a drop door (i.e., in a case where it is assumed that a faller is detected in the analysis result of the second image data), not only the analysis result of the first image data captured from inside the storage device as a whole is confirmed, but also the analysis result of the image of the region in the storage device corresponding to the position of the drop door at which the possibility of a faller is high is confirmed, whereby faller detection can be performed more correctly and in a shorter time.

[0044] The faller detection system of the fourth aspect of the embodiment is the faller detection system of the second or third aspect of the embodiment, in which

[0045] The faller determination unit extracts a difference between the first image data in the first time and the first image data in the second time after a predetermined time has elapsed from the first time, when the faller determination unit confirms the analysis result of the first image data in the first time, and determines that there is a faller in a case where the extracted difference exceeds a predetermined threshold value, and determines that there is no faller in a case where there is no difference.

[0046] According to this manner, for example, in a case where a handled object is buried in the upper portion of a faller in the storage device, it is not detected that there is a person in the storage device according to the analysis result of the first image data captured from inside the storage device (because the handled object is buried in the upper portion of the faller), but it is assumed that a faller is detected in the analysis result of the second image data, and therefore the first image data in the first time and the first image data in the second time after a predetermined time has elapsed from the first time are further compared to extract a difference therebetween. And, in a case where the difference between the first image data in the first time and the first image data in the second time exceeds a predetermined threshold value, it is considered that a pile of handled objects collapses or the like in the storage device, and it is considered that there is a possibility that the pile of handled objects that collapsed is buried in the upper portion of the faller and the faller cannot be seen, and therefore it is determined that there is a faller. Thus, even in a case where the upper portion of a faller is buried with a handled object in the storage device, a faller can be automatically detected.

[0047] The faller detection system of the fifth aspect of the embodiment is the faller detection system of the first aspect of the embodiment, in which

[0048] The faller determination unit confirms the analysis result of the first image data, and in a case where a person is detected in the storage device at the first time, confirms the analysis result of the second image data up to a predetermined time before the first time, and in a case where the person is framed out of the video within a predetermined area near the drop-in door that separates between the storage device and the platform, determines that there is a faller.

[0049] According to this manner, the analysis result of the first image data taken from within the storage device is confirmed, and in a case where a person is detected in the storage device (i.e., a case where a faller is assumed to be detected in the analysis result of the first image data), the analysis result of the second image data is confirmed, and in a case where the person is framed out of the video near the drop-in door (i.e., a case where a faller is also assumed to be detected in the analysis result of the second image data), it is finally determined that there is a faller, so that a faller can be correctly detected.

[0050] The faller detection system of the sixth aspect of the embodiment is the faller detection system of the fifth aspect of the embodiment, in which

[0051] The faller determination unit confirms the analysis result of the image of the drop-in door corresponding to the position of the area in which a person is detected in the storage device in the second image data up to a predetermined time before the first time in a case where a person is detected in the storage device at the first time.

[0052] According to this manner, even for a large platform provided with a plurality of drop-in doors, the analysis result of the first image data taken from within the storage device is confirmed, and in a case where a person is detected in the storage device (i.e., a case where a faller is assumed to be detected in the analysis result of the first image data), not only the analysis result of the entire second image data taken from within the platform is confirmed, but also the analysis result of the image of the drop-in door that is in a position where a faller is highly likely to be generated, so that a faller can be more correctly and in a shorter time detected.

[0053] The faller detection system of the seventh aspect of the embodiment is the faller detection system of the fifth or sixth aspect of the embodiment, in which

[0054] The faller determination unit determines that there is no faller in a case where the person is framed out of the video outside a predetermined area near the drop-in door when confirming the analysis result of the second image data up to a predetermined time before the first time.

[0055] According to this manner, in a case where a person is framed out of the video for reasons unrelated to a fall into the storage device (e.g., a case where a shadow of a carrying vehicle temporarily blocks, etc.) based on the second image data taken from within the platform, it is possible to prevent a false determination that there is a faller, and it is possible to improve the correctness of faller detection.

[0056] The faller detection system of the 8th embodiment of the present embodiment is the faller detection system of any one of the 5th to 7th embodiments,

[0057] The faller determination section determines that there is no faller when it is confirmed that the analysis result of the 2nd image data up to a predetermined time from the 1st time does not detect entry of a person into a predetermined area near the drop door.

[0058] According to this aspect, even in a case where a person-magnitude treated object is erroneously detected as a person (i.e., a case where a faller is erroneously assumed to be detected in the analysis result of the 1st image data), it is determined that there is no faller when it is confirmed that the analysis result of the 2nd image data taken from inside the platform does not detect entry of a person near the drop door, thereby preventing erroneous determination that there is a faller, and enabling improvement in the accuracy of faller detection.

[0059] The faller detection system of the 9th embodiment of the present embodiment is the faller detection system of any one of the 1st to 8th embodiments,

[0060] Further provided is an instruction section that, when the faller determination section determines that there is a faller, performs at least one of the following processes including:

[0061] (1) issuing an alarm;

[0062] (2) sending a control signal to a crane control device to stop a crane that stirs or carries treated objects stored in the storage device;

[0063] (3) sending a control signal to a drop door control device to close a drop door that separates the storage device from the platform;

[0064] (4) sending a control signal to the crane control device to cause the crane to act to rescue the faller;

[0065] (5) sending a control signal to a rescue equipment control device to cause rescue equipment provided in the storage device to act to rescue the faller.

[0066] According to this aspect, the faller can be rescued quickly, and the safety of the facility can be improved.

[0067] The faller detection system of the 10th embodiment of the present embodiment is the faller detection system of any one of the 1st to 9th embodiments,

[0068] The first image analysis section detects a person in the storage facility using a first detection algorithm constructed by machine learning on first training data, which is generated by giving a manual label to information on a region where a person or a human body model of a dummy person exists with respect to past image data in the storage facility, as input of new image data in the storage facility.

[0069] The faller detection system of the 11th aspect of the embodiment is the faller detection system of any one of the first to tenth aspects,

[0070] The second image analysis section detects a person in the platform using a second detection algorithm constructed by machine learning on second training data, which is generated by giving a manual label to information on a region where a person or a human body model of a dummy person exists with respect to past image data in the platform, as input of new image data in the platform.

[0071] The faller detection system of the 12th aspect of the embodiment is the faller detection system of the 11th aspect,

[0072] The second training data is generated by giving a manual label to information on a region where a person or a human body model of a dummy person exists with respect to past image data in the platform and giving a manual other label to information on a region where the incoming vehicle exists.

[0073] According to this aspect, the frequency of the person (e.g., the worker) in the platform working near the incoming vehicle is high, and there is a relationship between the position of the person and the position of the incoming vehicle, and thus, the second detection algorithm constructed by machine learning on the second training data, which is generated by giving a manual label to information on a region where a person or a human body model of a dummy person exists with respect to past image data in the platform and giving a manual other label to information on a region where the incoming vehicle exists, is used, and thus, it is possible to improve the detection accuracy of the person in the platform, and it is also possible to detect the fall of the incoming vehicle.

[0074] The faller detection system of the 13th aspect of the embodiment is the faller detection system of the 10th aspect,

[0075] The first detection algorithm includes one or two or more of a maximum likelihood classification method, a Boltzmann machine, a neural network, a support vector machine, a Bayesian network, a sparse regression, a decision tree, a statistical inference using a random forest, reinforcement learning, and deep learning.

[0076] The faller detection system of the 14th aspect of the embodiment is the faller detection system of the 11th or 12th aspect,

[0077] The second detection algorithm includes one or more of maximum likelihood classification, a postman machine, a neural network, a support vector machine, a Bayesian network, sparse regression, a decision tree, statistical inference using a random forest, reinforcement learning, and deep learning.

[0078] The faller detection system of the fifteenth aspect of the embodiment is the faller detection system of any one of the first to fourteenth aspects,

[0079] The algorithm used by the second image analysis unit for tracking the moving route of the person includes one or more of optical flow, background subtraction, a Kalman filter, a particle filter, and deep learning.

[0080] The faller detection system of the sixteenth aspect of the embodiment is the faller detection system of any one of the first to fifteenth aspects,

[0081] The first camera includes one or more of an RGB camera, a near-infrared camera, a 3D camera, and an RGB-D camera.

[0082] The faller detection system of the seventeenth aspect of the embodiment is the faller detection system of any one of the first to sixteenth aspects,

[0083] The second camera includes one or more of an RGB camera, a near-infrared camera, a 3D camera, and an RGB-D camera.

[0084] The waste treatment facility of the eighteenth aspect of the embodiment has the faller detection system of any one of the first to seventeenth aspects.

[0085] The faller detection method of the nineteenth aspect of the embodiment includes:

[0086] a step of acquiring first image data from a first camera that captures an inside of a storage device in which a treated object is stored;

[0087] a step of acquiring second image data from a second camera that captures an inside of a platform adjacent to the storage device;

[0088] a step of performing image analysis on the first image data to detect a person inside the storage device;

[0089] a step of performing image analysis on the second image data to detect a person inside the platform and track a moving route of the detected person; and

[0090] a step of determining whether a faller who has fallen from the platform to the storage device is present or absent based on a combination of a result of the analysis of the first image data and a result of the analysis of the second image data.

[0091] The storage medium of the 20th embodiment of the implementation can be read by a computer, and non-temporarily stores a faller detecting program which makes the computer execute the following steps:

[0092] a step of acquiring first image data from a first camera which photographs inside a storage device which stores a processed object;

[0093] a step of acquiring second image data from a second camera which photographs inside a platform which is adjacent to the storage device;

[0094] a step of performing image analysis on the first image data to detect a person inside the storage device;

[0095] a step of performing image analysis on the second image data to detect a person inside the platform and trace a moving route of the detected person; and

[0096] a step of determining whether there is a faller who falls from the platform to the storage device based on a combination of a result of the analysis of the first image data and a result of the analysis of the second image data.

[0097] A specific example of the implementation will be described in detail below with reference to the accompanying drawings. Furthermore, in the following description and the accompanying drawings used in the following description, the same reference numerals are used for portions which can be similarly configured, and repeated description is omitted.

[0098] (Configuration of waste treatment facility)

[0099] Figure 1 is a schematic view showing a configuration of a waste treatment facility 100 of one embodiment.

[0100] As shown in Figure 1 , the waste treatment facility 100 has a platform 21 on which a haulage vehicle (garbage truck and light truck, etc.) 22 which loads waste is parked, a garbage pit (storage device) 3 which stores waste which is dropped from the platform 21, a crane 5 which stirs and hauls the waste stored in the garbage pit 3, a hopper 4 into which the waste hauled by the crane 5 is dropped, an incinerator 1 which incinerates the waste dropped from the hopper 4, and a heat recovery boiler 2 which recovers heat from exhaust gas generated in the incinerator 1. The type of the incinerator 1 is not limited to the packed furnace shown in Figure 1 , but also includes a fluidized furnace (also called fluidized bed furnace). In addition, the configuration of the garbage pit 3 is not limited to the one-layer configuration shown in Figure 1 , but also includes a two-layer configuration which divides the garbage pit into a dropping part and a storage part. The garbage pit 3 and the platform 21 are separated by a dropping door 24. In addition, the waste treatment facility 100 is provided with a dropping door control device 20 which controls the operation of the dropping door 24, and a crane control device 30 which controls the operation of the crane 5.

[0101] The waste that is carried in the state of being loaded on the carrying vehicle 22 is dropped from the platform 21 into the garbage pit 3 via the drop door 24, and is stored in the garbage pit 3. The waste stored in the garbage pit 3 is stirred by the crane 5, and is carried by the crane 5 to the hopper 4, and is dropped into the inside of the incinerator 1 via the hopper 4, and is incinerated in the inside of the incinerator 1 to be treated.

[0102] As shown in FIG. 1, a first camera 6 that photographs the inside of the garbage pit 3, and a garbage recognition system 40 that recognizes the kind of the waste in the garbage pit 3 are provided in the waste treatment facility 100. Figure 1

[0103] The first camera 6 is disposed above the garbage pit 3, and in the illustrated example, is fixed to the track of the crane 5, and can photograph the waste stored in the garbage pit 3 from above the garbage pit 3. One or a plurality of the first cameras 6 can be provided.

[0104] The first camera 6 can be an RGB camera that outputs the shape and color image data of the waste as a photographing result, or can be a near-infrared camera that outputs the near-infrared image data of the waste as a photographing result. It can also be a 3D camera or an RGB-D camera that photographs the three-dimensional image data of the waste as a photographing result, or can be a combination of two or more of these.

[0105] The garbage recognition system 40 acquires image data (also referred to as first image data) from the first camera 6 that photographs the inside of the garbage pit 3, and performs image analysis on the first image data, and recognizes the kind of the waste stored in the garbage pit 3. For example, the garbage recognition system 40 can recognize the kind of the waste stored in the garbage pit 3 using a recognition algorithm (learning completed model) constructed by mechanically learning training data in which past image data photographed from the inside of the garbage pit 3 is labeled with the kind of the waste, with new image data in the inside of the garbage pit 3 as input.

[0106] The garbage recognition system 40 generates a graph that displays the proportion of the kind of the waste for each region as a result of recognizing the kind of the waste stored in the garbage pit 3, and transmits it to the crane control device 30. The crane control device 30 causes the crane 5 to act so as to stir the waste in the garbage pit 3 in such a manner that the proportion of the kind of the waste is equalized in the entire region, based on the graph received from the garbage recognition system 40. Thereby, automatic driving of the crane 5 becomes possible.

[0107] As the garbage recognition system 40, specifically, for example, the information processing device described in Japanese Patent No. 6731680 can be utilized.

[0108] As shown in FIG. 1, a first camera 6 that photographs the inside of the garbage pit 3, and a garbage recognition system 40 that recognizes the kind of the waste in the garbage pit 3 are provided in the waste treatment facility 100. Figure 1 ​As shown, the waste treatment facility 100 is also equipped with: a second camera 23 for filming inside the platform 21; and a faller detection system 10 for detecting fallers falling from the platform 21 into the waste pit 3.

[0109] The second camera 23 is positioned above the platform 21, and in the illustrated example, it is fixed to the wall of the platform 21 near the front of the entry door 24, enabling it to capture images of the interior of the platform 21 from near the front of the entry door 24. One or multiple second cameras 23 can be installed.

[0110] The second camera 23 can be an RGB camera that outputs the shape and color image data of the object (workers, transport vehicles 22, etc.) as the shooting result, a near-infrared camera that outputs near-infrared image data of the object as the shooting result, a 3D camera or an RGB-D camera that captures three-dimensional image data of the object as the shooting result, or a combination of two or more of these.

[0111] (The composition of a fall detection system)

[0112] Next, the structure of the fall detection system 10 for detecting fallers falling from platform 21 to garbage pit 3 will be explained. Figure 2 This is a block diagram illustrating the configuration of the fall detection system 10. The fall detection system 10 can be composed of a single computer or multiple computers that can be communicatively connected to each other.

[0113] like Figure 2 As shown, the fall detection system 10 includes a control unit 11, a storage unit 12, and a communication unit 13. Each part is communicatively connected to the others via a bus and a network.

[0114] The communication unit 13 serves as the communication interface for the first camera 6, the second camera 23, the crane control device 30, and the drop gate control device 20. The communication unit 13 transmits and receives information between each of the first camera 6, the second camera 23, the crane control device 30, and the drop gate control device 20 and the fall detection system 10.

[0115] The storage section 12 is, for example, a non-volatile data storage device such as a hard disk and a flash memory. The storage section 12 stores various data used by the control section 11. In addition, the storage section 12 stores the first detection algorithm 12a1 constructed by the first model construction section 11c1, the second detection algorithm 12a2 constructed by the second model construction section 11c2, the first image data 12b1 acquired by the first image data acquisition section 11a1, the second image data 12b2 acquired by the second image data acquisition section 11a2, the first training data 12c1 generated by the first training data generation section 11b1, and the second training data 12c2 generated by the second training data generation section 11b2, which will be described later.

[0116] The control section 11 is a control mechanism that performs various processes of the faller detection system 10. As shown in FIG. 1, the control section 11 has a first image data acquisition section 11a1, a second image data acquisition section 11a2, a first training data generation section 11b1, a second training data generation section 11b2, a first model construction section 11c1, a second model construction section 11c2, a first image analysis section 11d1, a second image analysis section 11d2, a faller determination section 11e, and an instruction section 11f. These sections can be implemented by a processor executing a prescribed program within the faller detection system 10, or can be installed by hardware. Figure 2

[0117] The first image data acquisition section 11a1 acquires first image data from the first camera 6 that photographs the inside of the garbage pit 3. The first image data can be a moving image or a continuous still image. The frame rate of the first image data need not be particularly high, and can be a normal frame rate (30 fps or so), or can be a low frame rate (5 to 10 fps or so). The metadata of the first image data includes information of the photographing date and time. The first image data 12b1 acquired by the first image data acquisition section 11a1 is stored in the storage section 12.

[0118] The second image data acquisition section 11a2 acquires second image data from the second camera 23 that photographs the inside of the platform 21. The second image data can be a moving image or a continuous still image. The frame rate of the second image data need not be particularly high, and can be a normal frame rate (30 fps or so), or can be a low frame rate (5 to 10 fps or so). The metadata of the second image data includes information of the photographing date and time. The second image data 12b2 acquired by the second image data acquisition section 11a2 is stored in the storage section 12.

[0119] ​The first training data generating section 11b1 generates first training data by labeling information on a region where a person (i.e., a faller) or a dummy of a person is present (i.e., artificially labels a region where a person or a dummy of a person is present as information) with respect to past image data taken from inside the landfill 3, based on visual observation by a skilled operator who operates the waste incineration facility 100. As an example, the first training data generating section 11b1 can generate the first training data with respect to the image data taken from inside the landfill 3 after intentionally dropping the dummy of a person into the landfill 3, or can generate the first training data with respect to image data (composite image data) obtained by synthesizing an image of a person and the past image data taken from inside the landfill 3. The first training data generating section 11b1 can also generate the first training data by labeling information on a region where a person (i.e., a faller) or a dummy of a person is present and information on a region where the carrying vehicle 22 is present (i.e., artificially labels a region where a person or a dummy of a person is present and a region where the carrying vehicle is present as information) with respect to the past image data taken from inside the landfill 3, based on visual observation by a skilled operator who operates the waste incineration facility 100. The information on a region where a person or a dummy of a person is present and the information on a region where the carrying vehicle 22 is present are labeled (i.e., artificially labeled as information) in a state of being overlaid on the image data as one layer, for example. The first training data 12c1 generated by the first training data generating section 11b1 is stored in the storage section 12.

[0120] The second training data generating section 11b2 labels information on a region where a person or a mannequin of a person is present, which is recognized by a skilled operator who operates the waste incineration facility 100 by visual observation, with respect to past image data taken from within the platform 21 (i.e., artificially labels a region where a person or a mannequin of a person is present as information), thereby generating second training data. As an example, the second training data generating section 11b2 can generate second training data with respect to image data taken from within the platform 21 after intentionally setting a mannequin of a person within the platform 21, or can generate second training data with respect to image data (composite image data) obtained by synthesizing an image of a person with past image data taken from within the platform 21. The second training data generating section 11b2 can also label information on a region where a person or a mannequin of a person is present, which is recognized by a skilled operator who operates the waste incineration facility 100 by visual observation, and information on a region where the carrying vehicle 22 is present, with respect to past image data taken from within the garbage pit 3 (i.e., artificially labels a region where a person or a mannequin of a person is present and a region where the carrying vehicle is present as information), thereby generating second training data. A person (e.g., a worker) within the platform 21 works near the carrying-in vehicle 22 with a high frequency, and there is a relationship between the position of the person and the position of the carrying-in vehicle 22, and thus the second model building 11c2 described later mechanically learns training data that artificially labels a region where a person or a mannequin of a person is present as information and artificially labels a region where the carrying-in vehicle 22 is present as other information with respect to image data taken from within the platform 21, thereby building a second detection algorithm 12a2, so that it is possible to improve the detection accuracy of a person within the platform 21 based on the second detection algorithm 12a2, and it is also possible to detect a fall of the carrying vehicle 22. The information on a region where a person or a mannequin of a person is present and the information on a region where the carrying vehicle 22 is present are labeled (i.e., artificially labeled as information) in a state where they are overlaid on image data, for example. The second training data 12c2 generated by the second training data generating section 11b2 is stored in the storage section 12.

[0121] The first model building section 11c1 builds a first detection algorithm 12a1 (a learning completed model) that detects a person (i.e., a faller) in the garbage pit 3 by inputting new image data in the garbage pit 3, by performing machine learning on the first training data 12c1 stored in the storage section 12. The first detection algorithm 12a1 can include one or two or more of a maximum likelihood classification method, a Boltzmann machine, a neural network, a support vector machine, a Bayesian network, a sparse regression, a decision tree, a statistical inference using a random forest, reinforcement learning, and deep learning. The first detection algorithm 12a1 built by the first model building section 11c1 is stored in the storage section 12.

[0122] The second model building section 11c2 builds a second detection algorithm 12a2 (a learning completed model) that detects a person in the platform 21 by inputting new image data in the platform 21, by performing machine learning on the second training data 12c2 stored in the storage section 12. The second detection algorithm 12a2 can include one or two or more of a maximum likelihood classification method, a Boltzmann machine, a neural network, a support vector machine, a Bayesian network, a sparse regression, a decision tree, a statistical inference using a random forest, reinforcement learning, and deep learning. The second detection algorithm 12a2 built by the second model building section 11c2 is stored in the storage section 12.

[0123] The first image analysis section 11d1 detects a person in the garbage pit 3 by performing image analysis on the first image data acquired by the first image data acquisition section 11a1. Specifically, for example, the first image analysis section 11d uses the first detection algorithm 12a1 (a learning completed model) built by the first model building section 11c1, inputs new image data in the garbage pit 3, and detects a person in the garbage pit 3. As a modification, the first image analysis section 11d1 can detect a person in the garbage pit 3 and a carrying vehicle 22 by performing image analysis on the first image data acquired by the first image data acquisition section 11a1. Specifically, for example, the first image analysis section 11d can use the first detection algorithm 12a1 (a learning completed model) built by the first model building section 11c1, inputs new image data in the garbage pit 3, and respectively detects a person in the garbage pit 3 and a carrying vehicle 22.

[0124] The first image analysis section 11d1 can divide the surface of the garbage pit 3 into a plurality of blocks, input new image data in the garbage pit 3 to the first detection algorithm 12a1 (a learning completed model) in block units, and acquire detection results of a person (or a person and a carrying vehicle 22) in block units. Thereby, it is possible to correctly grasp where a faller (or a fall of a person and a carrying vehicle 22) has occurred in the garbage pit 3.

[0125] The 2nd image analysis section 11d2 performs image analysis on the 2nd image data acquired by the 2nd image data acquisition section 11a2 to detect a person within the platform 21, and traces a movement route of the detected person. Specifically, for example, the 2nd image analysis section 11d uses the 2nd detection algorithm 12a2 (learned model) constructed by the 2nd model construction section 11c2, and inputs new image data within the platform 21 to detect a person within the platform 21. Next, the 2nd image analysis section 11d traces the detected person, and detects entry of the person into a predetermined area near the drop door 24. As a modification, the 2nd image analysis section 11d2 performs image analysis on the 2nd image data acquired by the 2nd image data acquisition section 11a2 to respectively detect a person and a transport vehicle 22 within the platform 21, and respectively trace movement routes of the detected person and the transport vehicle 22. Specifically, for example, the 2nd image analysis section 11d uses the 2nd detection algorithm 12a2 (learned model) constructed by the 2nd model construction section 11c2, and inputs new image data within the platform 21 to respectively detect a person and a transport vehicle 22 within the platform 21. Next, the 2nd image analysis section 11d respectively traces the detected person and the transport vehicle 22, and detects entry of the person and the transport vehicle 22 into a predetermined area near the drop door 24. The algorithm for tracing can include one or two or more of an optical flow method, a background subtraction method, a Kalman filter, a particle filter, and deep learning.

[0126] As a modification, the 2nd image analysis section 11d2 can perform image analysis on the 2nd image data acquired by the 2nd image data acquisition section 11a2 to detect a person within the platform 21, trace a movement route of the detected person, and detect, by image processing, a wearing state of safety equipment (safety belt or safety helmet) of the detected person, and determine whether there is a person who is working near the drop door 24 without wearing safety equipment. Also, in a case where it is determined that there is a person who is working near the drop door 24 without wearing safety equipment, the instruction section 11f described later can issue an alarm (warning), or can transmit a control signal to the drop door control device 20 to make the drop door 24 (in a case of being closed) unable to open or (in a case of being opened) unable to close.

[0127] The faller determination unit 11e determines the presence or absence of a faller who has fallen from the platform 21 to the garbage pit 3 on the basis of a combination of the analysis result of the first image data obtained by the first image analysis unit 11d1 and the analysis result of the second image data obtained by the second image analysis unit 11d2. Generally, in a case where a faller is detected from image data using a machine learning model, the detection accuracy of the machine learning model is uneven, and it is not possible to detect a faller with 100% accuracy, but in the present embodiment, by combining the analysis result of the first image data taken from inside the garbage pit 3 and the analysis result of the second image data taken from inside the platform 21, it is possible to improve the detection accuracy of a faller.

[0128] As an example, the faller determination unit 11e can first confirm the analysis result of the second image data, and in a case where it is detected that a person has entered a predetermined area in the vicinity of the drop opening 24 at the first time (i.e., in a case where it is assumed that a faller is detected in the analysis result of the second image data), as shown in FIG. 9, confirm the analysis result of the first image data at the first time, and in a case where it is detected that a person is present inside the garbage pit 3 (i.e., in a case where it is assumed that a faller is detected in the analysis result of the first image data), as shown in FIG. 10, finally determine that there is a faller. Thereby, it is possible to correctly detect a faller. Figure 4A Figure 4B As an example, the faller determination unit 11e can first confirm the analysis result of the second image data, and in a case where it is detected that a person has entered a predetermined area in the vicinity of the drop opening 24 at the first time (i.e., in a case where it is assumed that a faller is detected in the analysis result of the second image data), as shown in FIG. 9, confirm the analysis result of the first image data at the first time, and in a case where it is detected that a person is present inside the garbage pit 3 (i.e., in a case where it is assumed that a faller is detected in the analysis result of the first image data), as shown in FIG. 10, finally determine that there is a faller. Thereby, it is possible to correctly detect a faller.

[0129] As an example, the faller determination unit 11e can first confirm the analysis result of the second image data, and in a case where it is detected that a person has entered a predetermined area in the vicinity of the drop opening 24 at the first time (i.e., in a case where it is assumed that a faller is detected in the analysis result of the second image data), as shown in FIG. 9, confirm the analysis result of the first image data at the first time, and in a case where it is detected that a person is present inside the garbage pit 3 (i.e., in a case where it is assumed that a faller is detected in the analysis result of the first image data), as shown in FIG. 10, finally determine that there is a faller. Thereby, it is possible to correctly detect a faller. Figure 4A Figure 4B As an example, the faller determination unit 11e can first confirm the analysis result of the second image data, and in a case where it is detected that a person has entered a predetermined area in the vicinity of the drop opening 24 at the first time (i.e., in a case where it is assumed that a faller is detected in the analysis result of the second image data), as shown in FIG. 9, confirm the analysis result of the first image data at the first time, and in a case where it is detected that a person is present inside the garbage pit 3 (i.e., in a case where it is assumed that a faller is detected in the analysis result of the first image data), as shown in FIG. 10, finally determine that there is a faller. Thereby, it is possible to correctly detect a faller. Figure 4B

[0130] ​​​Further, the faller determination unit 11e can be configured to, when confirming the analysis result of the first image data in the first time, in a case where no person is detected in the garbage pit 3, compare the first image data in the first time and the first image data in a second time after a predetermined time (for example, 5 minutes) from the first time and extract a difference therebetween, in a case where the difference between the first image data in the first time and the first image data in the second time exceeds a predetermined threshold, determine that there is a faller, and in a case where there is no difference, determine that there is no faller. The reason is as follows. That is, for example, in a case where the upper portion of the faller is buried with waste in the garbage pit 3, no person is detected in the garbage pit 3 (because the upper portion of the faller is buried with waste) by the analysis result of the first image data, but the faller is assumed to be detected in the analysis result of the second image data, and therefore, the first image data in the first time and the first image data in the second time are further compared and a difference therebetween is extracted. Further, in a case where the difference between the first image data in the first time and the first image data in the second time exceeds a predetermined threshold, it is considered that a collapse of a pile of waste occurs in the garbage pit 3 or the like, and it is considered that the possibility that the pile of waste with the collapse buries the upper portion of the faller so that the faller cannot be seen, and therefore, it is determined that there is a faller. Thus, even in a case where the upper portion of the faller is buried with waste in the garbage pit 3, the faller can be automatically detected.

[0131] As a variation, the faller determination unit 11e can be configured to, first, confirm the analysis result of the first image data, in a case where a person is detected in the garbage pit 3 in the first time (i.e., a case where the faller is assumed to be detected in the analysis result of the first image data), confirm the analysis result of the second image data until a predetermined time before the first time (for example, 5 minutes before), and in a case where the person is framed from the image in a predetermined region near the drop door 24 (i.e., a case where the faller is also assumed to be detected in the analysis result of the second image data), finally determine that there is a faller. Thus, the faller can be correctly detected. Figure 6A Figure 6B As a variation, the faller determination unit 11e can be configured to, first, confirm the analysis result of the first image data, in a case where a person is detected in the garbage pit 3 in the first time (i.e., a case where the faller is assumed to be detected in the analysis result of the first image data), confirm the analysis result of the second image data until a predetermined time before the first time (for example, 5 minutes before), and in a case where the person is framed from the image in a predetermined region near the drop door 24 (i.e., a case where the faller is also assumed to be detected in the analysis result of the second image data), finally determine that there is a faller. Thus, the faller can be correctly detected.

[0132] As a variation, the faller determination unit 11e can be configured to, first, confirm the analysis result of the first image data, in a case where a person is detected in the garbage pit 3 in the first time (i.e., a case where the faller is assumed to be detected in the analysis result of the first image data), confirm the analysis result of the second image data until a predetermined time before the first time (for example, 5 minutes before), and in a case where the person is framed from the image in a predetermined region near the drop door 24 (i.e., a case where the faller is also assumed to be detected in the analysis result of the second image data), finally determine that there is a faller. Thus, the faller can be correctly detected. Figure 6A Figure 6B Figure 6A ​​​The analysis result of the image captured by the drop port (drop port B in the illustrated example) corresponding to the position of the area surrounded by the single-dot chain line indicated by reference sign B2. Thus, even for a large-scale platform 21 provided with a plurality of drop ports 24, the analysis result of the first image data captured from within the garbage pit 3 is confirmed, and in the case where a person is detected within the garbage pit 3, not only the analysis result of the second image data as a whole captured from within the platform 21 is confirmed, but also the analysis result of the image of the drop port B in a position where the possibility of a faller occurring is high is confirmed, whereby a faller can be detected more correctly and in a short time.

[0133] In addition, the faller determination unit 11e can determine that there is no faller in the case where a person is framed out of the video outside the predetermined area near the drop port 24 at the time of confirming the analysis result of the second image data with respect to the period from the first time until a predetermined time before (for example, 5 minutes before). Thus, in the case where a person is framed out of the video for reasons unrelated to a fall into the garbage pit 3 (for example, a case where the person is temporarily blocked by the shadow of the haulage vehicle 22, etc.) from the second image data captured from within the platform 21, it is possible to prevent a false determination that there is a faller, and it is possible to improve the correctness of faller detection.

[0134] In addition, the faller determination unit 11e can determine that there is no faller in the case where a person is not detected to enter the predetermined area near the drop port 24 at the time of confirming the analysis result of the second image data with respect to the period from the first time until a predetermined time before (for example, 5 minutes before). Thus, even in the case where, for example, waste having a size similar to that of a person is erroneously detected as a person from the first image data captured from within the garbage pit 3 (i.e., a case where a faller is erroneously assumed to be detected in the analysis result of the first image data), it is possible to prevent a false determination that there is a faller by determining that there is no faller in the case where a person is not detected to enter the drop port 24 at the time of confirming the analysis result of the second image data captured from within the platform 21, and it is possible to improve the correctness of faller detection.

[0135] The instruction unit 11f confirms the determination result obtained by the faller determination unit 11e, and in the case where the faller determination unit 11e determines that there is a faller, performs at least one of the following processes:

[0136] (1) sends an alarm (warning);

[0137] (2) sends a control signal to the crane control device 30 to stop the crane 5 that stirs or hauls the waste stored within the garbage pit 3;

[0138] (3) sends a control signal to the drop port control device 20 to close the drop port 24 that separates the garbage pit 3 and the platform 21;

[0139] (4) a control signal is sent to the crane control device 30 to cause the crane 5 to act to perform rescue of the faller;

[0140] (5) a control signal is sent to the rescue equipment control device (not shown) to cause the rescue equipment (not shown) provided in the garbage pit 3 to act to perform rescue of the faller.

[0141] Thus, even in a case where a faller appears in automatic driving of the crane 5 (i.e., when the crane operator is not present), prompt rescue of the faller can be performed, and the safety of the facility can be improved.

[0142] (Example 1 of faller detection method)

[0143] Next, an example 1 of a faller detection method realized based on the faller detection system 10 configured by such a configuration will be described. Figure 3 A flowchart showing the example 1 of the faller detection method.

[0144] As shown in Figure 3 , first, the first image data acquisition section 11a1 acquires first image data from the first camera 6 that photographs the inside of the garbage pit 3 (step S10). The acquired first image data 12b1 is stored in the storage section 12.

[0145] Next, the second image data acquisition section 11a2 acquires second image data from the second camera 23 that photographs the inside of the platform 21 (step S11). The acquired first image data 12b2 is stored in the storage section 12. Further, the order of the step S10 and the step S11 can be arbitrary one first, or can be simultaneous.

[0146] Next, the second image analysis section 1 Id2 performs image analysis on the second image data acquired by the second image data acquisition section 11a2 to detect a person inside the platform 21, and traces a movement route of the detected person (step S12).

[0147] The faller determination section 1 le confirms the analysis result of the second image data by the second image analysis section 1 Id2 (step S13).

[0148] In a case where no person is detected to enter the predetermined area near the drop door 24 in the first time (i.e., in a case where no faller is assumed to be detected in the analysis result of the second image data) (step S13: No), the faller determination section 1 le determines that there is no faller (step S19).

[0149] On the other hand, in a case where a person is detected to enter the predetermined area near the drop door 24 in the first time (i.e., in a case where a faller is assumed to be detected in the analysis result of the second image data) (step S13: Yes), the faller determination section 1 le determines that there is a faller (step S14). Figure 4AAs shown, in a case where it is detected that a person has entered a predetermined area near the drop door 24 in the first time (i.e., in a case where it is assumed that a faller is detected in the analysis result of the first image data) (Step S13: Yes), the first image analysis section 1 Id performs image analysis on the first image data in the first time acquired by the first image data acquisition section 11a1 to detect a person inside the garbage pit 3 (Step S14).

[0150] Also, the faller determination section 11e confirms the analysis result of the first image data in the first time by the first image analysis section 1 Id (Step S15). Here, the faller determination section 11e can be, for example, Figure 4A As shown, in a case where it is detected that a person has entered a predetermined area near the drop door 25 (drop door B in the illustrated example) in the first time, as Figure 4B As shown, the analysis result of the image taken from the area in the first image data in the first time inside the garbage pit 3 corresponding to the position of the drop door (i.e., drop door B) (the area surrounded by the single-dot chain line labeled B1 in Figure 4B Fig. 16) is confirmed.

[0151] As shown, in a case where it is detected that a person has entered a predetermined area near the drop door 25 (drop door B in the illustrated example) in the first time, as Figure 4B As shown, in a case where it is detected that a person has entered a predetermined area near the drop door 24 in the first time (i.e., in a case where it is assumed that a faller is detected in the analysis result of the first image data) (Step S13: Yes), the first image analysis section 1 Id performs image analysis on the first image data in the first time acquired by the first image data acquisition section 11a1 to detect a person inside the garbage pit 3 (Step S14).

[0152] On the other hand, in a case where a person is not detected inside the garbage pit 3 (i.e., in a case where it is not assumed that a faller is detected in the analysis result of the first image data) (Step 15: No), the faller determination section 11e extracts a difference between the first image data in the first time and the first image data in the second time after a predetermined time (e.g., 5 minutes) has passed from the first time, and compares the extracted difference with a predetermined threshold value (Step S16).

[0153] In a case where the difference between the first image data in the first time and the first image data in the second time exceeds the predetermined threshold value (Step S16: Yes), it is considered that a collapse or the like of the mountain of waste has occurred inside the garbage pit 3, and that it is possible that the faller is buried by the collapsed mountain of waste so as to be unable to be seen, and therefore the faller determination section 11e determines that there is a faller (Step S17).

[0154] On the other hand, in a case where the difference between the first image data in the first time and the first image data in the second time does not exceed the predetermined threshold value (Step S16: No), the faller determination section 11e determines that there is no faller (Step S19).

[0155] Further, in a case where the faller judging section 11e judges that there is a faller (after Step S17), the instruction section 11f issues an alarm (warning) for notifying other operators, and sends a control signal to the crane control device 30 to stop the crane 5 (Step S18).

[0156] In Step S18, the instruction section 11f can also send a control signal to the drop door control device 20 to close the drop door 24 that separates the garbage pit 3 and the platform 21, in order to prevent the faller from being rescued by preventing the waste from being dropped on the faller. In addition, the instruction section 11f can send a control signal to the crane control device 30 to cause the crane 5 to operate to rescue the faller, instead of sending a control signal to the crane control device 30 to stop the crane 5. In addition, the instruction section 11f can send a control signal to a rescue equipment control device (not shown) to cause rescue equipment (not shown) provided in the garbage pit 3 to operate to rescue the faller. Thus, even in a case where a faller occurs during automatic driving of the crane 5 (i.e., when the crane operator is not present), the faller can be quickly rescued.

[0157] (Second Example of Faller Detection Method)

[0158] Next, a second example of a faller detection method implemented by the faller detection system 10 will be described. Figure 5 is a flowchart showing the second example of the faller detection method.

[0159] As shown in Figure 5 , first, the first image data acquisition section 11a1 acquires first image data from the first camera 6 that photographs the inside of the garbage pit 3 (Step S20). The acquired first image data 12b1 is stored in the storage section 12.

[0160] Next, the second image data acquisition section 11a2 acquires second image data from the second camera 23 that photographs the inside of the platform 21 (Step S21). The acquired first image data 12b2 is stored in the storage section 12. Further, the order of Step S20 and Step S21 can be arbitrary, or they can be performed simultaneously.

[0161] Next, the first image analysis section 11d1 performs image analysis on the first image data acquired by the first image data acquisition section 11a1 to detect a person in the garbage pit 3 (Step S22).

[0162] The faller judging section 11e confirms the analysis result of the first image data obtained by the first image analysis section 11d1 (Step S23).

[0163] If no person is detected in the garbage pit 3 at the first moment (i.e., the analysis result of the first image data does not assume that a person who has fallen has been detected) (step S23: No), the person who has fallen determination unit 11e determines that there is no person who has fallen (step S29).

[0164] On the other hand, such as Figure 6A As shown, if a person is detected in the garbage pit 3 at the first moment (i.e., if a person who has fallen is detected in the analysis result of the first image data) (step S23: Yes), the second image analysis unit 11d2 performs image analysis on the second image data acquired by the second image data acquisition unit 11a2 relative to the first moment up to a predetermined time (e.g., 5 minutes ago) to detect the person in the platform 21 and track the movement route of the detected person (step S24).

[0165] Furthermore, the fall detection unit 11e confirms, in the analysis result of the second image data obtained by the second image analysis unit 11d2, whether a person has been detected entering a predetermined area near the entry gate 24 (step S25). Here, the fall detection unit 11e can be, for example... Figure 6A As shown, if a person is detected in garbage pit 3 at time 1, then... Figure 6B As shown, the area in the second image data relative to the first moment up to a predetermined time (e.g., 5 minutes ago) where a person was detected was identified in the garbage pit 3. Figure 6A The image analysis results are obtained from the input gate (in the example shown, input gate B) at the location corresponding to the area enclosed by the single-dotted line marked B2 in the attached figure.

[0166] If no person is detected entering the predetermined area near the input door 24 (i.e., the analysis result of the second image data does not assume that a faller has been detected) (step S25: No), the faller determination unit 11e determines that there is no faller (step S29).

[0167] On the other hand, such as Figure 6A As shown, if a person is detected entering a predetermined area near the input gate 24 in the analysis result of the second image data (step S25: Yes), the faller determination unit 11e confirms whether the person is out of the image frame in the predetermined area near the input gate 24 in the analysis result of the second image data (step S26).

[0168] If a person is framed out of the image within a predetermined area near the input gate 24 (i.e., the faller is also detected in the analysis result of the second image data) (step S26: Yes), the faller determination unit 11e determines that there is a faller (step S27).

[0169] On the other hand, in a case where the person is not framed from the video within the predetermined area near the drop-in door 24 (step S26: No), the faller determiner 11e determines that there is no faller (step S29).

[0170] Also, in a case where the faller determiner 11e determines that there is a faller (after step S27), the instruction unit 11f issues an alarm (warning) for notification to other operators, and transmits a control signal to the crane control device 30 to stop the crane 5 (step S28).

[0171] In step S28, the instruction unit 11f can also transmit a control signal to the drop-in door control device 20 to close the drop-in door 24 that separates the garbage pit 3 and the platform 21, in order to prevent the waste from being dropped on the faller, making rescue difficult. In addition, instead of transmitting a control signal to the crane control device 30 to stop the crane 5, the instruction unit 11f can transmit a control signal to the crane control device 30 to cause the crane 5 to operate to rescue the faller. In addition, the instruction unit 11f can transmit a control signal to a rescue equipment control device (not shown) to cause the rescue equipment (not shown) provided in the garbage pit 3 to operate to rescue the faller. Thus, even in a case where a faller occurs during automatic driving of the crane 5 (i.e., when the crane operator is not present), the faller can be quickly rescued.

[0172] However, in a waste incineration facility, the faller determination system is separated by the garbage drop-in door 24 between the fall cause side, i.e., the platform 21, and the fall result side, i.e., the garbage pit 3, and thus it is difficult to set a camera or the like at a place where the fall cause side and the fall result side can be simultaneously imaged, as in the faller determination system used in the railway industry.

[0173] In contrast, according to the present embodiment, even without setting a camera at a place where the fall cause side and the fall result side can be simultaneously imaged, by acquiring first image data from the first camera 6 that images the fall result side, i.e., the inside of the garbage pit 3, and acquiring second image data from the second camera 23 that images the fall cause side, i.e., the inside of the platform 21, the analysis result of the first image data imaged from the inside of the garbage pit 3 and the analysis result of the second image data imaged from the inside of the platform 21 are combined, and thus a faller who has fallen from the platform 21 to the garbage pit 3 can be automatically detected.

[0174] Generally, in a case where a faller is detected from image data using a machine learning model, the detection accuracy of the machine learning model is uneven, and it is not possible to detect a faller with 100% accuracy, but according to the present embodiment, by combining the analysis result of the first image data imaged from the inside of the garbage pit 3 and the analysis result of the second image data imaged from the inside of the platform 21, the detection accuracy of the faller can be improved.

[0175] In addition, according to the present embodiment, even with respect to an existing facility, it is possible to make the system function by additionally providing the first camera 6 that captures the inside of the garbage pit 3 and the second camera 23 that captures the inside of the platform 21, and thus it is not necessary to substantially reform the existing facility for introduction of the system, and it is possible to inexpensively improve the safety of the existing facility.

[0176] In addition, in the above-described embodiments, the waste treatment facility 100 in which the faller detection system 10 is provided is a facility provided with the garbage recognition system 40, and is configured in a manner in which automatic driving of the crane 5 is performed based on the recognition result of the garbage recognition system 40, but is not limited thereto, and the waste treatment facility 100 in which the faller detection system 10 is provided can also be a facility not provided with the garbage recognition system 40.

[0177] In addition, in the above-described embodiments, the faller detection system 10 is configured to determine the presence or absence of a faller based on a combination of the analysis result of the first image data captured from the inside of the garbage pit 3 and the analysis result of the second image data captured from the inside of the platform 21, but in a case in which sufficient detection accuracy can be obtained by only the analysis result of the first image data, it is also possible to determine the presence or absence of a faller based on only the analysis result of the first image data, and in a case in which sufficient detection accuracy can be obtained by only the analysis result of the second image data, it is also possible to determine the presence or absence of a faller based on only the analysis result of the second image data.

[0178] The above describes the embodiments and modified examples, but the technical scope is not limited thereto, and changes and modifications can be made within the scope of the technical idea described in the technical scope. In addition, the embodiments and modified examples can be appropriately combined within a range in which the processing contents do not contradict each other.

[0179] In addition, the faller detection system 10 of the present embodiment is constituted by one or a plurality of computers, but a program in which the faller detection system 10 is implemented in one or a plurality of computers and a storage medium in which the program is non-transitorily stored are also objects of the present application.

Claims

1. A faller detection system characterized by, has: a first image data acquisition section that acquires first image data from a first camera that photographs inside a storage device that stores a handled object; a second image data acquisition section that acquires second image data from a second camera that photographs inside a platform adjacent to the storage device; a first image analysis section that detects a person inside the storage device by performing image analysis on the first image data; a second image analysis section that detects a person inside the platform by performing image analysis on the second image data, and traces a moving route of the detected person; and a faller determination section that determines whether there is a faller who falls from the platform to the storage device based on a combination of the analysis result of the first image data and the analysis result of the second image data.

2. The faller detection system according to claim 1, wherein the faller determination section confirms the analysis result of the second image data, and in a case where it is detected in a first time that a person enters a predetermined area near a drop-in door that separates the storage device and the platform, confirms the analysis result of the first image data in the first time, and in a case where it is detected that the person is inside the storage device, determines that there is a faller.

3. The faller detection system according to claim 2, wherein the faller determination section, in a case where it is detected in the first time that the person enters the predetermined area near the drop-in door, confirms the analysis result of an image that is obtained by photographing a region inside the storage device that corresponds to a position of the drop-in door in the first image data in the first time.

4. The faller detection system according to claim 2 or 3, wherein the faller determination section, in a case where it is not detected that the person is inside the storage device when confirming the analysis result of the first image data in the first time, extracts a difference between the first image data in the first time and the first image data in a second time that is a predetermined time after the first time, and in a case where the extracted difference exceeds a predetermined threshold, determines that there is a faller, and in a case where there is no difference, determines that there is no faller.

5. The faller detection system according to claim 1, wherein the faller determination section confirms the analysis result of the first image data, and in a case where it is detected in a first time that a person is inside the storage device, confirms the analysis result of the second image data until a predetermined time before the first time, and in a case where the person is inside a predetermined area near a drop-in door that separates the storage device and the platform from the inside of the frame, determines that there is a faller.

6. The faller detection system according to claim 5, wherein the faller determination section, in a case where it is detected in the first time that the person is inside the storage device, confirms the analysis result of an image that is obtained by photographing the drop-in door at a position that corresponds to a region in which the person is detected inside the storage device in the second image data until the predetermined time before the first time.

7. The faller detection system according to claim 5 or 6, wherein ​ The faller determination unit determines that there is no faller when confirming the analysis result of the second image data with respect to the first time until the predetermined time before, in a case where a person is not framed from the video within a predetermined area near the drop door.

8. The faller detection system according to claim 5 or 6, characterized in that, The faller determination unit determines that there is no faller when confirming the analysis result of the second image data with respect to the first time until the predetermined time before, in a case where a person is not detected to enter within a predetermined area near the drop door.

9. The faller detection system according to claim 1, characterized in that, Further comprising an instruction unit that, in a case where the faller determination unit determines that there is a faller, performs at least one of the following processes, including: (1) issuing an alarm; (2) sending a control signal to a crane control device to stop a crane that stirs or carries a handled object stored in the storage equipment; (3) sending a control signal to a drop door control device to close a drop door that separates the storage equipment and the platform; (4) sending a control signal to the crane control device to cause the crane to act to rescue the faller; (5) sending a control signal to a rescue equipment control device to cause rescue equipment provided in the storage equipment to act to rescue the faller.

10. The faller detection system according to claim 1, characterized in that, The first image analysis unit detects a person within the storage equipment using a first detection algorithm constructed by mechanically learning first training data that is generated by giving an artificial label to information on an area where a person or a human body model of a simulated person exists with respect to past image data within the storage equipment, as input of new image data within the storage equipment.

11. The faller detection system according to claim 1, characterized in that, The second image analysis unit detects a person within the platform using a second detection algorithm constructed by mechanically learning second training data that is generated by giving an artificial label to information on an area where a person or a human body model of a simulated person exists with respect to past image data within the platform, as input of new image data within the platform.

12. The faller detection system according to claim 11, characterized in that, The second training data is generated by giving an artificial label to information on an area where a person or a human body model of a simulated person exists with respect to past image data within the platform, and giving an artificial other label to information on an area where a vehicle for carrying in exists.

13. The faller detection system according to claim 11, characterized in that, The first image analysis unit detects a person within the storage equipment using a first detection algorithm constructed by mechanically learning first training data that is generated by giving an artificial label to information on an area where a person or a human body model of a simulated person exists with respect to past image data within the storage equipment, as input of new image data within the storage equipment, The first detection algorithm includes one or more of maximum likelihood classification, a Boltzmann machine, a neural network, a support vector machine, a Bayesian network, sparse regression, a decision tree, statistical inference using random forests, reinforcement learning, and deep learning. The second detection algorithm includes one or more of maximum likelihood classification, a Boltzmann machine, a neural network, a support vector machine, a Bayesian network, sparse regression, a decision tree, statistical inference using random forests, reinforcement learning, and deep learning.

14. The faller detection system according to claim 1, wherein The algorithm used by the second image analysis section for tracking the moving route of the person includes one or more of optical flow, background subtraction, Kalman filter, particle filter, and deep learning.

15. The faller detection system according to claim 1, wherein The first camera includes one or more of an RGB camera, a near-infrared camera, a 3D camera, and an RGB-D camera. The second camera includes one or more of an RGB camera, a near-infrared camera, a 3D camera, and an RGB-D camera.

16. A waste processing facility, comprising: the faller detection system according to any one of claims 1 to 15.

17. A faller detection method characterized by, including: a step of acquiring first image data from a first camera that captures an inside of a storage device in which a processed object is stored; a step of acquiring second image data from a second camera that captures an inside of a platform adjacent to the storage device; a step of performing image analysis on the first image data to detect a person inside the storage device; a step of performing image analysis on the second image data to detect a person inside the platform and track a moving route of the detected person; and a step of determining whether a faller who has fallen from the platform to the storage device is present or absent based on a combination of the analysis result of the first image data and the analysis result of the second image data. a computer is caused to execute:

18. A faller detection program characterized by comprising: a step of acquiring first image data from a first camera that captures an inside of a storage device in which a processed object is stored; a step of acquiring second image data from a second camera that captures an inside of a platform adjacent to the storage device; a step of performing image analysis on the first image data to detect a person inside the storage device; a step of performing image analysis on the second image data to detect a person inside the platform and track a moving route of the detected person; and a step of determining whether a faller who has fallen from the platform to the storage device is present or absent based on a combination of the analysis result of the first image data and the analysis result of the second image data. ​ ​

Images