Faller detection system, faller detection method, and faller detection program

Abstract

A system, method, and program for detecting a person who has fallen can automatically detect a person who has fallen into storage equipment that stores waste in a waste treatment facility such as a waste incineration facility. [Solution] In a waste treatment facility 100, a fallen person detection system 10 comprises a first image data acquisition unit that acquires first image data from a first camera 6 that takes images inside a storage facility (pit 3) where the material to be treated is stored, a second image data acquisition unit that acquires second image data from a second camera 23 that takes images inside a platform 21 adjacent to the storage facility, a first image analysis unit that performs image analysis of the first image data to detect people inside the storage facility, a second image analysis unit that performs image analysis of the second image data to detect people inside the platform and track the movement of the detected people, and a fallen person determination unit that determines whether or not a person has fallen from the platform into the storage facility based on a combination of the analysis results of the first image data and the analysis results of the second image data.

Description

Technical Field

[0001] The present invention relates to a faller detection system, a faller detection method, and a faller detection program for automatically detecting a person who has fallen into a facility for storing an object to be processed.

Background Art

[0002] In facilities related to waste treatment, particularly in waste incineration facilities, although the number of cases is small, several accidents in which people fall into the garbage pit are reported every year.

[0003] When a person falls into the garbage pit, it is necessary to transmit the information to a crane operator or the like and temporarily stop the crane operation to ensure the safety of the fallen person. However, in recent years, the automatic driving technology of cranes has been developing. For example, when a fallen person occurs during automatic crane operation (i.e., when the crane operator is absent), it takes time to transmit the information to the crane operator or the like, and there is a possibility of causing a serious accident.

[0004] In particular, in waste incineration facilities, there is no accident prevention system that automatically detects a person who has fallen into the garbage pit and controls the crane.

[0005] In other fields (particularly the railway industry), a system for detecting a person who has fallen from a station platform to a track by image processing or the like is used (see, for example, Patent Documents 1 and 2).

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0007] In the faller detection system used in the railway industry, the images of a surveillance camera that simultaneously captures the platform on the fall cause side and the track on the fall result side are processed to detect fallers. On the other hand, in a waste incineration facility, since the space between the platform on the fall cause side and the garbage pit on the fall result side is partitioned by a garbage input door, it is difficult to install an imaging device such as a camera at a location where the fall cause side and the fall result side can be simultaneously photographed.

[0008] Also, in the faller detection system used in the railway industry, a method may be used in which the size of a falling object is obtained by processing the image of a surveillance camera, and it is determined whether the falling object is a person based on the size. On the other hand, in a waste incineration facility, various sizes of waste are thrown into the garbage pit from a transport vehicle (such as a packer vehicle or a light truck), so it is difficult to determine whether the falling object is a person based only on the size of the falling object.

[0009] Also, as described above, in a waste incineration facility, it is difficult to install a camera at a location where the fall cause side (platform) and the fall result side (garbage pit) can be simultaneously photographed. Therefore, for example, it is conceivable to install a camera on the garbage pit side, process the image inside the garbage pit, and detect a faller. However, inside the garbage pit, the waste pile may collapse and waste may cover the upper part of the faller. In that case, it is difficult to detect the faller by processing the image inside the garbage pit.

[0010] Note that not only facilities adopting the pit-and-crane method such as incineration facilities, but also waste treatment facilities (such as bulky waste crushing facilities and recycling facilities) with configurations such as a direct input method to a storage facility or a compactor container method other than the pit-and-crane method have the same problems.

[0011] The present invention has been made in consideration of the above points. An object of the present invention is to provide a faller detection system, a faller detection method, and a faller detection program that can automatically detect a person who has fallen into a storage facility for storing waste in a waste treatment facility such as a waste incineration facility.

Means for Solving the Problems

[0012] The faller detection system according to the first aspect of the present invention includes a first image data acquisition unit that acquires first image data from a first camera that images the inside of a storage facility in which an object to be processed is stored; a second image data acquisition unit that acquires second image data from a second camera that images the inside of a platform adjacent to the storage facility; a first image analysis unit that analyzes the first image data to detect a person inside the storage facility; a second image analysis unit that analyzes the second image data to detect a person inside the platform and tracks the movement route of the detected person; a faller determination unit that determines the presence or absence of a faller who has fallen from the platform into the storage facility based on a combination of the analysis result of the first image data and the analysis result of the second image data; and is provided with.

[0013] According to such an aspect, by combining the analysis result of the first image data obtained by imaging inside the storage facility and the analysis result of the second image data obtained by imaging inside the platform, it is possible to automatically detect a person who has fallen from the platform to the storage facility. Generally, when using a machine learning model to detect a person who has fallen from image data, the detection accuracy of the machine learning model varies, and it is impossible to detect a person who has fallen with 100% accuracy. However, by combining the analysis result of the first image data obtained by imaging inside the storage facility and the analysis result of the second image data obtained by imaging inside the platform, the detection accuracy of a person who has fallen can be improved. In addition, even for an existing facility, by simply additionally installing a first camera for imaging inside the storage facility and a second camera for imaging inside the platform, the operation of this system becomes possible. Therefore, a major renovation of the existing facility is not required when introducing this system, and it is possible to inexpensively improve the safety of the existing facility.

[0014] The fallen person detection system according to the second aspect of the present invention is the fallen person detection system according to the first aspect, wherein The fallen person determination unit checks the analysis result of the second image data, and when it is detected that a person has entered a predetermined area near the access door that separates between the storage facility and the platform at the first time, checks the analysis result of the first image data at the first time, and when a person is detected inside the storage facility, determines that there is a fallen person.

[0015] According to such an aspect, by checking the analysis result of the second image data obtained by imaging inside the platform and when it is detected that a person has entered a predetermined area near the access door (that is, when a fallen person is temporarily detected in the analysis result of the second image data), checking the analysis result of the first image data and when a person is detected inside the storage facility (that is, when a fallen person is also temporarily detected in the analysis result of the first image data), by finally determining that there is a fallen person, a fallen person can be accurately detected.

[0016] The fallen person detection system according to the third aspect of the present invention is the fallen person detection system according to the second aspect, wherein When the faller determination unit detects the entry of a person into a predetermined area near the loading door at the first time, it checks the analysis result of the image obtained by imaging the area inside the storage facility corresponding to the position of the loading door in the first image data at the first time.

[0017] According to such an aspect, even in the case of a large storage facility provided with a plurality of loading doors, instead of checking the analysis result of the second image data obtained by imaging the platform and checking the analysis result of the entire first image data obtained by imaging the inside of the storage facility when the entry of a person into a predetermined area near the loading door is detected (that is, when a faller is temporarily detected in the analysis result of the second image data), by checking the analysis result of the image of the area inside the storage facility corresponding to the position of the loading door where a faller is likely to occur, it is possible to detect a faller more accurately and in a shorter time.

[0018] The faller detection system according to the fourth aspect of the present invention is the faller detection system according to the second or third aspect, When the faller determination unit checks the analysis result of the first image data at the first time and no person is detected inside the storage facility, it extracts the difference between the first image data at the first time and the first image data at the second time after a predetermined time has elapsed from the first time. When the extracted difference exceeds a predetermined threshold value, it determines that there is a faller, and when there is no difference, it determines that there is no faller.

[0019] According to such an aspect, for example, when the object to be processed covers the upper part of the fallen person in the storage facility, in the analysis result of the first image data obtained by imaging the storage facility, (since the object to be processed covers the upper part of the fallen person) no person is detected in the storage facility. However, since the fallen person has already been temporarily detected in the analysis result of the second image data, further, the first image data at the first time is compared with the first image data at the second time after a predetermined time has elapsed from the first time, and the difference therebetween is extracted. When the difference between the first image data at the first time and the first image data at the second time exceeds a predetermined threshold value, it is considered that, for example, the pile of the object to be processed has collapsed in the storage facility, and since it is considered that the fallen object may have covered the upper part of the fallen person and the fallen person may not be visible, it is determined that there is a fallen person. Thereby, even when the object to be processed covers the upper part of the fallen person in the storage facility, it becomes possible to automatically detect the fallen person.

[0020] The fallen person detection system according to the fifth aspect of the present invention is the fallen person detection system according to the first aspect, The fallen person determination unit checks the analysis result of the first image data, and when a person is detected in the storage facility at the first time, checks the analysis result of the second image data up to a predetermined time before the first time, and determines that there is a fallen person when the person has dropped out of the video within a predetermined area near the loading door that partitions between the storage facility and the platform.

[0021] According to such an aspect, by checking the analysis result of the first image data obtained by imaging the storage facility, and when a person is detected in the storage facility (that is, when a fallen person is temporarily detected in the analysis result of the first image data), checking the analysis result of the second image data, and when the person has dropped out of the video near the loading door (that is, when a fallen person is also temporarily detected in the analysis result of the second image data), by finally determining that there is a fallen person, the fallen person can be accurately detected.

[0022] The faller detection system according to the sixth aspect of the present invention is the faller detection system according to the fifth aspect, wherein When a person is detected in the storage facility at the first time, the faller determination unit checks the analysis result of the image obtained by imaging the access door at the position corresponding to the area where the person is detected in the storage facility among the second image data up to a predetermined time before the first time.

[0023] According to such an aspect, even in a large platform provided with a plurality of access doors, instead of checking the analysis result of the first image data obtained by imaging the inside of the storage facility and checking the analysis result of the entire second image data obtained by imaging the inside of the platform when a person is detected in the storage facility (that is, when a faller is temporarily detected in the analysis result of the first image data), by checking the analysis result of the image of the access door at a position where a faller is likely to occur, it is possible to detect a faller more accurately and in a shorter time.

[0024] The faller detection system according to the seventh aspect of the present invention is the faller detection system according to the fifth or sixth aspect, wherein When the faller determination unit checks the analysis result of the second image data up to a predetermined time before the first time and a person has dropped out of the video outside a predetermined area near the access door, the faller determination unit determines that there is no faller.

[0025] According to such an aspect, it is possible to prevent an incorrect determination that there is a faller when a person has dropped out of the video for a reason unrelated to falling into the storage facility (for example, temporarily hidden behind a transport vehicle) in the second image data obtained by imaging the inside of the platform, and it is possible to improve the accuracy of faller detection.

[0026] The faller detection system according to the eighth aspect of the present invention is the faller detection system according to any one of the fifth to seventh aspects, wherein When the faller determination unit checks the analysis result of the second image data up to a time before the first time and no entry of a person into a predetermined area near the loading door is detected, it determines that there is no faller.

[0027] According to such an aspect, in the first image data obtained by imaging the inside of the storage facility, even if an object to be processed having approximately the same size as a person is erroneously detected as a person (that is, when a faller is erroneously detected in the analysis result of the first image data), when checking the analysis result of the second image data obtained by imaging the inside of the platform and no entry of a person into the area near the loading door is detected, by determining that there is no faller, it is possible to prevent an incorrect determination that there is a faller and improve the accuracy of faller detection.

[0028] The faller detection system according to the ninth aspect of the present invention is a faller detection system according to any one of the first to eighth aspects, and when the faller determination unit determines that there is a faller, (1) issues an alarm, (2) transmits a control signal to the crane control device to stop the crane that stirs or conveys the object to be processed stored in the storage facility, (3) transmits a control signal to the loading door control device to close the loading door that partitions between the storage facility and the platform, (4) transmits a control signal to the crane control device to operate the crane to rescue the faller, (5) transmits a control signal to the rescue equipment control device to operate the rescue equipment provided in the storage facility to rescue the faller, and further includes an instruction unit that performs at least one of the above processes.

[0029] According to such an aspect, rapid rescue of the faller becomes possible, and the safety of the facility can be improved.

[0030] The faller detection system according to the tenth aspect of the present invention is a faller detection system according to any one of the first to ninth aspects, and The first image analysis unit detects a person in the storage facility by using the first detection algorithm constructed by machine learning the first teacher data generated by assigning an artificial label as information to the area where a person or a dummy figure imitating a person exists in the past image data in the storage facility, with the new image data in the storage facility as the input.

[0031] The fallen person detection system according to the eleventh aspect of the present invention is the fallen person detection system according to any one of the first to tenth aspects, The second image analysis unit detects a person on the platform by using the second detection algorithm constructed by machine learning the second teacher data generated by assigning an artificial label as information to the area where a person or a dummy figure imitating a person exists in the past image data on the platform, with the new image data on the platform as the input.

[0032] The fallen person detection system according to the twelfth aspect of the present invention is the fallen person detection system according to the eleventh aspect, The second teacher data is generated by assigning an artificial label as information to the area where a person or a dummy figure imitating a person exists in the past image data on the platform and assigning another artificial label as information to the area where the loading vehicle exists.

[0033] According to such an aspect, since a person (for example, a worker) works in the vicinity of the loading vehicle frequently on the platform and there is a relationship between the position of the person and the position of the loading vehicle, by using the second detection algorithm constructed by machine learning the teacher data generated by assigning an artificial label as information to the area where a person or a dummy figure imitating a person exists in the past image data on the platform and assigning another artificial label as information to the area where the loading vehicle exists, not only can the detection accuracy of the person on the platform be improved, but also the fall of the loading vehicle can be detected.

[0034] The faller detection system according to the 13th aspect of the present invention is the faller detection system according to the 10th aspect, and the first detection algorithm includes one or more of maximum likelihood classification method, Boltzmann machine, neural network, support vector machine, Bayesian network, sparse regression, decision tree, statistical estimation using random forest, reinforcement learning, and deep learning.

[0035] The faller detection system according to the 14th aspect of the present invention is the faller detection system according to the 11th or 12th aspect, and the second detection algorithm includes one or more of maximum likelihood classification method, Boltzmann machine, neural network, support vector machine, Bayesian network, sparse regression, decision tree, statistical estimation using random forest, reinforcement learning, and deep learning.

[0036] The faller detection system according to the 15th aspect of the present invention is the faller detection system according to any one of the 1st to 14th aspects, and the algorithm used by the second image analysis unit for tracking the movement line of a person includes one or more of optical flow, background subtraction method, Kalman filter, particle filter, and deep learning.

[0037] The faller detection system according to the 16th aspect of the present invention is the faller detection system according to any one of the 1st to 15th aspects, and the first camera includes one or more of an RGB camera, a near-infrared camera, a 3D camera, or an RGB-D camera.

[0038] The faller detection system according to the 17th aspect of the present invention is the faller detection system according to any one of the 1st to 16th aspects, and the second camera includes one or more of an RGB camera, a near-infrared camera, a 3D camera, or an RGB-D camera.

[0039] The waste treatment facility according to the 18th aspect of the present invention includes the faller detection system according to any one of the 1st to 17th aspects.

[0040] The faller detection method according to the 19th aspect of the present invention is as follows. A step of acquiring first image data from a first camera that images the inside of a storage facility where the object to be processed is stored; A step of acquiring second image data from a second camera that images the inside of a platform adjacent to the storage facility; A step of analyzing the first image data to detect a person inside the storage facility; A step of analyzing the second image data to detect a person inside the platform and tracking the movement line of the detected person; A step of determining the presence or absence of a faller who falls from the platform to the storage facility based on a combination of the analysis result of the first image data and the analysis result of the second image data; including.

[0041] The faller detection program according to the 20th aspect of the present invention is as follows. To a computer, A step of acquiring first image data from a first camera that images the inside of a storage facility where the object to be processed is stored; A step of acquiring second image data from a second camera that images the inside of a platform adjacent to the storage facility; A step of analyzing the first image data to detect a person inside the storage facility; A step of analyzing the second image data to detect a person inside the platform and tracking the movement line of the detected person; A step of determining the presence or absence of a faller who falls from the platform to the storage facility based on a combination of the analysis result of the first image data and the analysis result of the second image data; to execute.

Advantages of the Invention

[0042] According to the present invention, in a waste treatment facility such as a waste incineration facility, a faller into a storage facility for storing waste can be automatically detected.

Brief Description of the Drawings

[0043]

Figure 1

Figure 2

Figure 3

Figure 4A

Figure 4B

Figure 5

Figure 6A

Figure 6B

Embodiments for Carrying Out the Invention

[0044] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description and the drawings used in the following description, for parts that can be configured identically, the same reference numerals are used and duplicate descriptions are omitted.

[0045] (Configuration of Waste Treatment Facility) FIG. 1 is a schematic diagram showing the configuration of a waste treatment facility 100 according to an embodiment.

[0046] As shown in FIG. 1, the waste treatment facility 100 includes a platform 21 where a transport vehicle (such as a packer vehicle or a light truck) 22 for loading waste stops, a garbage pit (storage facility) 3 where the waste introduced from the platform 21 is stored, a crane 5 for stirring and transporting the waste stored in the garbage pit 3, a hopper 4 into which the waste transported by the crane 5 is introduced, an incinerator 1 for incinerating the waste introduced from the hopper 4, and a waste heat boiler 2 for recovering waste heat from the exhaust gas generated in the incinerator 1. The type of the incinerator 1 is not limited to the stoker furnace as shown in FIG. 1, and also includes a fluidized furnace (also referred to as a fluidized bed furnace). Further, the structure of the garbage pit 3 is not limited to the single-stage pit as shown in FIG. 1, and also includes a two-stage pit in which the garbage pit is divided into an input section and a storage section. The space between the garbage pit 3 and the platform 21 is partitioned by an input door 24. The waste treatment facility 100 is also provided with an input door control device 20 for controlling the operation of the input door 24 and a crane control device 30 for controlling the operation of the crane 5.

[0047] The waste carried in on the transport vehicle 22 is introduced into the garbage pit 3 from the platform 21 through the input door 24 and stored in the garbage pit 3. The waste stored in the garbage pit 3 is stirred by the crane 5 and transported to the hopper 4 by the crane 5, and then introduced into the incinerator 1 through the hopper 4 and incinerated and processed inside the incinerator 1.

[0048] As shown in FIG. 1, the waste treatment facility 100 is provided with a first camera 6 for imaging the inside of the garbage pit 3 and a garbage identification system 40 for identifying the type of waste in the garbage pit 3.

[0049] The first camera 6 is disposed above the garbage pit 3 and fixed to the rail of the crane 5 in the illustrated example, and is capable of imaging the waste stored in the garbage pit 3 from above the garbage pit 3. Only one first camera 6 may be installed, or a plurality of first cameras 6 may be installed.

[0050] The first camera 6 may be an RGB camera that outputs waste shape and color image data as an imaging result, or a near-infrared camera that outputs near-infrared image data of waste as an imaging result, or a 3D camera or an RGB-D camera that captures 3D image data of waste as an imaging result, or a combination of two or more of these may also be used.

[0051] The garbage identification system 40 acquires image data (also referred to as first image data) from the first camera 6 that images the inside of the garbage pit 3, analyzes the first image data, and identifies the types of waste stored in the garbage pit 3. For example, the garbage identification system 40 may use an identification algorithm (trained model) constructed by machine learning teacher data in which the types of waste are labeled in past image data of the inside of the garbage pit 3, and input new image data of the inside of the garbage pit 3 to identify the types of waste stored in the garbage pit 3.

[0052] As an identification result of the types of waste stored in the garbage pit 3, the garbage identification system 40 generates a map that displays the ratio of the types of waste for each area and transmits it to the crane control device 30. Based on the map received from the garbage identification system 40, the crane control device 30 operates the crane 5 to stir the waste in the garbage pit 3 so that the ratio of the types of waste is equal in all areas. As a result, the automatic operation of the crane 5 becomes possible.

[0053] Specifically, as the garbage identification system 40, for example, the information processing device described in Japanese Patent No. 6731680 can be used.

[0054] As shown in FIG. 1, the waste treatment facility 100 is further provided with a second camera 23 that images the inside of the platform 21 and a faller detection system 10 that detects a person falling from the platform 21 into the garbage pit 3.

[0055] The second camera 23 is disposed above the platform 21 and fixed to the wall of the platform 21 located near the front of the loading door 24 in the illustrated example, and is capable of imaging the inside of the platform 21 from near the front of the loading door 24. Only one second camera 23 may be installed, or a plurality of second cameras 23 may be installed.

[0056] The second camera 23 may be an RGB camera that outputs shape and color image data of an object (such as a person like a worker or a transport vehicle 22) as an imaging result, or may be a near-infrared camera that outputs near-infrared image data of the object as an imaging result, or may be a 3D camera or an RGB-D camera that captures three-dimensional image data of the object as an imaging result, or may be a combination of two or more of these.

[0057] (Configuration of the faller detection system) Next, the configuration of a faller detection system 10 that detects a person who falls from the platform 21 into the garbage pit 3 will be described. FIG. 2 is a block diagram showing the configuration of the faller detection system 10. The faller detection system 10 may be configured by one computer, or may be configured by a plurality of computers connected to be communicable with each other.

[0058] As shown in FIG. 2, the faller detection system 10 includes a control unit 11, a storage unit 12, and a communication unit 13. Each unit is connected to be communicable with each other via a bus or a network.

[0059] Among these, the communication unit 13 is a communication interface for the first camera 6, the second camera 23, the crane control device 30, and the loading door control device 20. The communication unit 13 transmits and receives information between each of the first camera 6, the second camera 23, the crane control device 30, and the loading door control device 20 and the faller detection system 10.

[0060] The storage unit 12 is a non-volatile data storage such as a hard disk or a flash memory. Various data handled by the control unit 11 are stored in the storage unit 12. Further, in the storage unit 12, a first detection algorithm 12a1 constructed by a first model construction unit 11c1 described later, a second detection algorithm 12a2 constructed by a second model construction unit 11c2, first image data 12b1 acquired by a first image data acquisition unit 11a1, second image data 12b2 acquired by a second image data acquisition unit 11a2, first teacher data 12c1 generated by a first teacher data generation unit 11b1, and second teacher data 12c2 generated by a second teacher data generation unit 11b2 are stored.

[0061] The control unit 11 is a control means for performing various processes of the faller detection system 10. As shown in FIG. 2, the control unit 11 includes a first image data acquisition unit 11a1, a second image data acquisition unit 11a2, a first teacher data generation unit 11b1, a second teacher data generation unit 11b2, a first model construction unit 11c1, a second model construction unit 11c2, a first image analysis unit 11d1, a first image analysis unit 11d2, a faller determination unit 11e, and an instruction unit 11f. Each of these units may be realized by a processor in the faller detection system 10 executing a predetermined program, or may be implemented by hardware.

[0062] Among these, the first image data acquisition unit 11a1 acquires first image data from a first camera 6 that images the inside of the trash pit 3. The first image data may be a moving image or a series of still images. The frame rate of the first image data may be a general frame rate (about 30 fps), and there is no particular need for a high frame rate, and it may also be a low frame rate (about 5 to 10 fps). The metadata of the first image data includes information on the shooting date and time. The first image data 12b1 acquired by the first image data acquisition unit 11a1 is stored in the storage unit 12.

[0063] The second image data acquisition unit 11a2 acquires second image data from the second camera 23 that images the interior of the platform 21. The second image data may be a moving image or a series of still images. The frame rate of the second image data may be a general frame rate (about 30 fps), and there is no particular need for a high frame rate, and it may also be a low frame rate (about 5 to 10 fps). The metadata of the second image data includes information on the shooting date and time. The second image data 12b2 acquired by the second image data acquisition unit 11a2 is stored in the storage unit 12.

[0064] The first teacher data generation unit 11b1 labels (i.e., artificially assigns a label as information) the information of the area where a person (i.e., a fallen person) or a dummy figure imitating a person visually identified by a skilled operator who operates the waste incineration facility 100 exists in the past image data obtained by imaging the inside of the garbage pit 3, thereby generating the first teacher data. As an example, the first teacher data generation unit 11b1 may generate the first teacher data for the image data obtained by imaging the inside of the garbage pit 3 after a dummy figure imitating a person is intentionally dropped into the garbage pit 3, or may generate the first teacher data for the image data (synthetic image data) obtained by synthesizing a person's image with the past image data obtained by imaging the inside of the garbage pit 3. The first teacher data generation unit 11b1 labels (i.e., artificially assigns a label as information) the information of the area where a person (i.e., a fallen person) or a dummy figure imitating a person visually identified by a skilled operator who operates the waste incineration facility 100 exists and the information of the area where the transport vehicle 22 exists in the past image data obtained by imaging the inside of the garbage pit 3, thereby generating the first teacher data. The information of the area where a person or a dummy figure imitating a person exists and the information of the area where the transport vehicle 22 exists are labeled (i.e., an artificial label is assigned as information) in a state of being superimposed on the image data as layers, for example. The first teacher data 12c1 generated by the first teacher data generation unit 11b1 is stored in the storage unit 12.

[0065] The second teacher data generation unit 11b2 labels the information of the area where a person or a dummy figure imitating a person visually identified by a skilled operator who operates the waste incineration facility 100 exists in the past image data captured within the platform 21 (that is, an artificial label is given as information to the area where a person or a dummy figure imitating a person exists), thereby generating the second teacher data. As an example, after a dummy figure imitating a person is intentionally installed within the platform 21, the second teacher data generation unit 11b2 may generate the second teacher data for the image data captured within the platform 21, or may generate the second teacher data for the image data (synthetic image data) obtained by synthesizing a person's image with the past image data captured within the platform 21. The second teacher data generation unit 11b2 labels the information of the area where a person or a dummy figure imitating a person visually identified by a skilled operator who operates the waste incineration facility 100 exists and the information of the area where the transport vehicle 22 exists in the past image data captured within the garbage pit 3 (that is, an artificial label is given as information to the area where a person or a dummy figure imitating a person exists and the area where the transport vehicle exists), thereby generating the second teacher data. Since a person (for example, a worker) frequently works in the vicinity of the loading vehicle 22 within the platform 21 and there is a relationship between the position of the person and the position of the loading vehicle 22, the second model construction 11c2 described later gives an artificial label as information to the area where a person or a dummy figure imitating a person exists in the image data captured within the platform 21 and gives another artificial label as information to the area where the loading vehicle 22 exists, and by machine learning the teacher data, the second detection algorithm 12a2 is constructed. In addition to being able to improve the detection accuracy of a person within the platform 21 by the second detection algorithm 12a2, the fall of the transport vehicle 22 can also be detected. The information of the area where a person or a dummy figure imitating a person exists and the information of the area where the transport vehicle 22 exists are labeled (that is, an artificial label is given as information) in a state of being superimposed on the image data as layers, for example.The second teacher data 12c2 generated by the second teacher data generation unit 11b2 is stored in the storage unit 12.

[0066] The first model construction unit 11c1 constructs a first detection algorithm 12a1 (trained model) that detects a person (i.e., a fallen person) in the garbage pit 3 by using the first teacher data 12c1 stored in the storage unit 12 and inputting new image data in the garbage pit 3. The first detection algorithm 12a1 may include one or more of maximum likelihood classification, Boltzmann machine, neural network, support vector machine, Bayesian network, sparse regression, decision tree, statistical estimation using random forest, reinforcement learning, and deep learning. The first detection algorithm 12a1 constructed by the first model construction unit 11c1 is stored in the storage unit 12.

[0067] The second model construction unit 11c2 constructs a second detection algorithm 12a2 (trained model) that detects a person in the platform 21 by using the second teacher data 12c2 stored in the storage unit 12 and inputting new image data in the platform 21. The second detection algorithm 12a2 may include one or more of maximum likelihood classification, Boltzmann machine, neural network, support vector machine, Bayesian network, sparse regression, decision tree, statistical estimation using random forest, reinforcement learning, and deep learning. The second detection algorithm 12a2 constructed by the second model construction unit 11c2 is stored in the storage unit 12.

[0068] The first image analysis unit 11d1 analyzes the first image data acquired by the first image data acquisition unit 11a1 to detect a person within the garbage pit 3. Specifically, for example, the first image analysis unit 11d uses the first detection algorithm 12a1 (pre-trained model) constructed by the first model construction unit 11c1, takes the new image data within the garbage pit 3 as input, and detects the person within the garbage pit 3. As a variant, the first image analysis unit 11d1 may analyze the first image data acquired by the first image data acquisition unit 11a1 to detect both the person and the transport vehicle 22 within the garbage pit 3. Specifically, for example, the first image analysis unit 11d may use the first detection algorithm 12a1 (pre-trained model) constructed by the first model construction unit 11c1, take the new image data within the garbage pit 3 as input, and detect the person and the transport vehicle 22 within the garbage pit 3 respectively.

[0069] The first image analysis unit 11d1 divides the surface of the garbage pit 3 into a plurality of blocks, inputs the new image data within the garbage pit 3 into the first detection algorithm 12a1 (pre-trained model) in block units, and may obtain the detection results of the person (or the person and the transport vehicle 22) in block units. Thereby, it becomes possible to accurately grasp where in the garbage pit 3 the fallen person (or the fall of the fallen person and the transport vehicle 22) has occurred.

[0070] The second image analysis unit 11d2 analyzes the second image data acquired by the second image data acquisition unit 11a2 to detect a person within the platform 21 and tracks the movement route of the detected person. Specifically, for example, the second image analysis unit 11d uses the second detection algorithm 12a2 (trained model) constructed by the second model construction unit 11c2, takes the new image data within the platform 21 as input, and detects the person within the platform 21. Next, the second image analysis unit 11d tracks the detected person and detects the entry of the person into a predetermined area near the loading door 24. As a modified example, the second image analysis unit 11d2 analyzes the second image data acquired by the second image data acquisition unit 11a2 to detect the person and the transport vehicle 22 within the platform 21 respectively, and tracks the movement routes of the detected person and the transport vehicle 22 respectively. Specifically, for example, the second image analysis unit 11d uses the second detection algorithm 12a2 (trained model) constructed by the second model construction unit 11c2, takes the new image data within the platform 21 as input, and detects the person and the transport vehicle 22 within the platform 21 respectively. Next, the second image analysis unit 11d tracks the detected person and the transport vehicle 22 respectively, and detects the entry of the person and the transport vehicle 22 into a predetermined area near the loading door 24. The algorithm used for tracking may include one or more of optical flow, background difference method, Kalman filter, particle filter, and deep learning.

[0071] As a modification example, the second image analysis unit 11d2 analyzes the second image data acquired by the second image data acquisition unit 11a2 to detect a person within the platform 21, tracks the movement route of the detected person, and detects the wearing status of the safety equipment (seat belt or helmet) of the detected person by image processing, and may determine whether there is a person working near the loading door 24 without wearing the safety equipment. Then, when it is determined that there is a person working near the loading door 24 without wearing the safety equipment, the instruction unit 11f described later may issue an alarm, or may transmit a control signal to the loading door control device 20 to prevent the loading door 24 from opening (if it is closed) or to close it (if it is open).

[0072] The faller determination unit 11e determines the presence or absence of a faller who falls from the platform 21 into the garbage pit 3 based on the combination of the analysis result of the first image data by the first image analysis unit 11d1 and the analysis result of the second image data by the second image analysis unit 11d2. Generally, when using a machine learning model to detect a faller from image data, the detection accuracy of the machine learning model varies, and it is impossible to detect a faller with 100% accuracy. However, in this embodiment, by combining the analysis result of the first image data obtained by imaging the inside of the garbage pit 3 and the analysis result of the second image data obtained by imaging the inside of the platform 21, the detection accuracy of the faller can be improved.

[0073] As an example, the faller determination unit 11e first checks the analysis result of the second image data. As shown in FIG. 4A, when the entry of a person into a predetermined area near the loading door 24 is detected at the first time (that is, when a faller is temporarily detected in the analysis result of the second image data), the analysis result of the first image data at the first time is checked. As shown in FIG. 4B, when a person is detected in the garbage pit 3 (that is, when a faller is also temporarily detected in the analysis result of the first image data), it may be finally determined that there is a faller. Thereby, it becomes possible to accurately detect a faller.

[0074] As shown in FIG. 4A, when the faller determination unit 11e detects the entry of a person into a predetermined area near the loading door 25 (loading door B in the illustrated example) at the first time, as shown in FIG. 4B, it may check the analysis result of the image obtained by imaging the area within the dust pit 3 corresponding to the position of the loading door (i.e., loading door B) in the first image data at the first time (the area surrounded by the dashed-dotted line labeled B1 in FIG. 4B). Thereby, even in the large dust pit 3 provided with a plurality of loading doors 24, instead of checking the analysis result of the second image data obtained by imaging the inside of the platform 21 and checking the analysis result of the entire first image data obtained by imaging the inside of the dust pit 3 when the entry of a person into a predetermined area near the loading door B is detected, by checking the analysis result of the image of the area B1 within the dust pit 3 corresponding to the position of the loading door B where a faller is highly likely to occur, it becomes possible to detect a faller more accurately and in a shorter time.

[0075] Further, when the fallen person determination unit 11e checks the analysis result of the first image data at the first time and no person is detected in the garbage pit 3, it compares the first image data at the first time with the first image data at the second time after a predetermined time has elapsed from the first time (for example, 5 minutes later), extracts the difference between them, and determines that there is a fallen person when the difference between the first image data at the first time and the first image data at the second time exceeds a predetermined threshold, and may determine that there is no fallen person when there is no difference. The reason is as follows. That is, for example, when waste covers the upper part of the fallen person in the garbage pit 3, in the analysis result of the first image data, no person is detected in the garbage pit 3 (because waste covers the upper part of the fallen person), but in the analysis result of the second image data, the fallen person has already been temporarily detected. Therefore, further, the first image data at the first time and the first image data at the second time are compared to extract the difference between them. And when the difference between the first image data at the first time and the first image data at the second time exceeds a predetermined threshold, it is considered that the waste mountain in the garbage pit 3 has collapsed, etc., and it is considered that there is a possibility that the fallen person is not visible because the collapsed waste mountain covers the upper part of the fallen person. Therefore, it is determined that there is a fallen person. Thereby, even when waste covers the upper part of the fallen person in the garbage pit 3, it becomes possible to automatically detect the fallen person.

[0076] As a modification, first, the fallen person determination unit 11e checks the analysis result of the first image data. As shown in FIG. 6A, when a person is detected in the garbage pit 3 at the first time (that is, when a fallen person is temporarily detected in the analysis result of the first image data), it checks the analysis result of the second image data up to a predetermined time before the first time (for example, 5 minutes before). As shown in FIG. 6B, when a person has dropped out of the video in a predetermined area near the loading door 24 (that is, when a fallen person is also temporarily detected in the analysis result of the second image data), it may finally determine that there is a fallen person. Thereby, it becomes possible to accurately detect the fallen person.

[0077] As shown in FIG. 6A, when a person is detected in the garbage pit 3 at the first time, the faller determination unit 11e may check the analysis result of the image obtained by imaging the loading door (loading door B in the illustrated example) at the position corresponding to the area (the area surrounded by the dashed-dotted line labeled B2 in FIG. 6A) where a person was detected in the garbage pit 3 among the second image data up to a time (for example, 5 minutes) before the first time as shown in FIG. 6B. Thereby, even in the case of the large platform 21 provided with a plurality of loading doors 24, instead of checking the analysis result of the first image data obtained by imaging the inside of the garbage pit 3 and checking the analysis result of the entire second image data obtained by imaging the inside of the platform 21 when a person is detected in the garbage pit 3, by checking the analysis result of the image of the loading door B at the position where a faller is highly likely to occur, it becomes possible to detect a faller more accurately and in a shorter time.

[0078] Further, when the faller determination unit 11e checks the analysis result of the second image data up to a time (for example, 5 minutes) before the first time and a person has dropped out of the frame outside a predetermined area near the loading door 24, it may be determined that there is no faller. Thereby, when a person has dropped out of the frame in the second image data obtained by imaging the inside of the platform 21 for reasons unrelated to falling into the garbage pit 3 (for example, temporarily hidden behind the transport vehicle 22), it is possible to prevent an incorrect determination that there is a faller, and the accuracy of faller detection can be improved.

[0079] Further, when the faller determination unit 11e checks the analysis result of the second image data up to a time predetermined before the first time (for example, 5 minutes before), if no entry of a person into a predetermined area near the loading door 24 is detected, it may be determined that there is no faller. Thereby, in the first image data obtained by imaging the inside of the garbage pit 3, even if a waste of approximately the same size as a person is erroneously detected as a person (that is, when a faller is erroneously detected in the analysis result of the first image data), when checking the analysis result of the second image data obtained by imaging the inside of the platform 21 and no entry of a person into the vicinity of the loading door 24 is detected, by determining that there is no faller, it is possible to prevent an erroneous determination of the presence of a faller and improve the accuracy of faller detection.

[0080] The instruction unit 11f checks the determination result by the faller determination unit 11e, and when the faller determination unit 11e determines that there is a faller, (1) issues an alarm, (2) transmits a control signal to the crane control device 30 to stop the crane 5 that stirs or conveys the waste stored in the garbage pit 3, (3) transmits a control signal to the loading door control device 20 to close the loading door 24 that partitions between the garbage pit 3 and the platform 21, (4) transmits a control signal to the crane control device 30 to operate the crane 5 to rescue the faller, (5) transmits a control signal to a rescue equipment control device (not shown) to operate rescue equipment (not shown) provided in the garbage pit 3 to rescue the faller. It performs at least one of the above processes. Thereby, even when a faller occurs during the automatic operation of the crane 5 (that is, when the crane operator is absent), prompt rescue of the faller becomes possible, and the safety of the facility can be improved.

[0081] (First example of faller detection method) Next, a first example of a faller detection method by the faller detection system 10 having such a configuration will be described. FIG. 3 is a flowchart showing the first example of the faller detection method.

[0082] As shown in FIG. 3, first, the first image data acquisition unit 11a1 acquires first image data from the first camera 6 that images the inside of the garbage pit 3 (step S10). The acquired first image data 12b1 is stored in the storage unit 12.

[0083] Next, the second image data acquisition unit 11a2 acquires second image data from the second camera 23 that images the inside of the platform 21 (step S11). The acquired first image data 12b2 is stored in the storage unit 12. Note that the order of step S10 and step S11 may be either first, or they may be simultaneous.

[0084] Next, the second image analysis unit 11d2 performs image analysis on the second image data acquired by the second image data acquisition unit 11a2 to detect a person inside the platform 21 and tracks the movement route of the detected person (step S12).

[0085] The faller determination unit 11e confirms the analysis result of the second image data by the second image analysis unit 11d2 (step S13).

[0086] When the entry of a person into a predetermined area near the loading door 24 at the first time is not detected (that is, when a faller is not temporarily detected in the analysis result of the second image data) (step S13: NO), the faller determination unit 11e determines that there is no faller (step S19).

[0087] On the other hand, as shown in FIG. 4A, when the entry of a person into a predetermined area near the loading door 24 is detected at the first time (that is, when a potential fallen person is temporarily detected in the analysis result of the second image data) (step S13: YES), the first image analysis unit 11d1 analyzes the first image data at the first time acquired by the first image data acquisition unit 11a1 to detect the person in the garbage pit 3 (step S14).

[0088] Then, the fallen person determination unit 11e checks the analysis result of the first image data at the first time by the first image analysis unit 11d1 (step S15). Here, as shown in FIG. 4A, when the entry of a person into a predetermined area near the loading door 25 (loading door B in the illustrated example) is detected at the first time, as shown in FIG. 4B, the fallen person determination unit 11e may check the analysis result of the image obtained by imaging the area in the garbage pit 3 corresponding to the position of the loading door (that is, loading door B) in the first image data at the first time (the area surrounded by the dashed-dotted line labeled B1 in FIG. 4B).

[0089] As shown in FIG. 4B, when a person is detected in the garbage pit 3 (that is, when a potential fallen person is also temporarily detected in the analysis result of the first image data) (step 15: YES), the fallen person determination unit 11e determines that there is a fallen person (step S17).

[0090] On the other hand, when no person is detected in the garbage pit 3 (that is, when a potential fallen person is not temporarily detected in the analysis result of the first image data) (step 15: NO), the fallen person determination unit 11e extracts the difference between the first image data at the first time and the first image data at the second time after a predetermined time has elapsed from the first time (for example, 5 minutes later), and compares the extracted difference with a predetermined threshold value (step S16).

[0091] If the difference between the first image data at the first time and the first image data at the second time exceeds a predetermined threshold (step S16: YES), it is considered that the waste pile in the waste pit 3 has collapsed, etc., and the waste from the collapsed pile may cover the upper part of the fallen person and the fallen person may not be visible. Therefore, the fallen person determination unit 11e determines that there is a fallen person (step S17).

[0092] On the other hand, if the difference between the first image data at the first time and the first image data at the second time does not exceed the predetermined threshold (step S16: NO), the fallen person determination unit 11e determines that there is no fallen person (step S19).

[0093] And when the fallen person determination unit 11e determines that there is a fallen person (after step S17), the instruction unit 11f issues an alarm (alert) to inform other workers and transmits a control signal to the crane control device 30 to stop the crane 5 (step S18).

[0094] In step S18, the instruction unit 11f may transmit a control signal to the input door control device 20 to close the input door 24 that partitions between the waste pit 3 and the platform 21 in order to prevent waste from being thrown onto the fallen person and making rescue difficult. Also, instead of transmitting a control signal to the crane control device 30 to stop the crane 5, the instruction unit 11f may transmit a control signal to the crane control device 30 to operate the crane 5 to rescue the fallen person. Further, the instruction unit 11f may transmit a control signal to a rescue equipment control device (not shown) to operate rescue equipment (not shown) provided in the waste pit 3 to rescue the fallen person. Thereby, even when a fallen person occurs during the automatic operation of the crane 5 (that is, when the crane operator is absent), prompt rescue of the fallen person becomes possible.

[0095] (Second example of fallen person detection method) Next, a second example of the fallen person detection method by the fallen person detection system 10 will be described. FIG. 5 is a flowchart showing a second example of the fallen person detection method.

[0096] As shown in FIG. 5, first, the first image data acquisition unit 11a1 acquires first image data from the first camera 6 that images the inside of the trash pit 3 (step S20). The acquired first image data 12b1 is stored in the storage unit 12.

[0097] Next, the second image data acquisition unit 11a2 acquires second image data from the second camera 23 that images the inside of the platform 21 (step S21). The acquired first image data 12b2 is stored in the storage unit 12. Note that the order of step S20 and step S21 may be either first, or they may be simultaneous.

[0098] Next, the first image analysis unit 11d1 performs image analysis on the first image data acquired by the first image data acquisition unit 11a1 to detect a person inside the trash pit 3 (step S22).

[0099] The fallen person determination unit 11e confirms the analysis result of the first image data by the first image analysis unit 11d1 (step S23).

[0100] If no person is detected inside the trash pit 3 at the first time (that is, if no fallen person is temporarily detected in the analysis result of the first image data) (step S23: NO), the fallen person determination unit 11e determines that there is no fallen person (step S29).

[0101] On the other hand, as shown in FIG. 6A, if a person is detected inside the trash pit 3 at the first time (that is, if a fallen person is temporarily detected in the analysis result of the first image data) (step S23: YES), the second image analysis unit 11d2 performs image analysis on the second image data acquired by the second image data acquisition unit 11a2 for the second image data up to a predetermined time before the first time (for example, 5 minutes before) to detect a person inside the platform 21 and track the movement line of the detected person (step S24).

[0102] Then, the fallen person determination unit 11e checks whether a person has entered a predetermined area near the loading door 24 in the analysis result of the second image data by the second image analysis unit 11d2 (step S25). Here, as shown in FIG. 6A, when a person is detected in the garbage pit 3 at the first time, as shown in FIG. 6B, the fallen person determination unit 11e may check the analysis result of the image obtained by imaging the loading door (loading door B in the illustrated example) at the position corresponding to the area (the area surrounded by the dashed-dotted line labeled B2 in FIG. 6A) where a person was detected in the garbage pit 3 in the second image data up to a predetermined time (for example, 5 minutes) before the first time.

[0103] If no entry of a person into the predetermined area near the loading door 24 is detected (that is, if no fallen person is temporarily detected in the analysis result of the second image data) (step S25: NO), the fallen person determination unit 11e determines that there is no fallen person (step S29).

[0104] On the other hand, as shown in FIG. 6A, if the entry of a person into the predetermined area near the loading door 24 is detected in the analysis result of the second image data (step S25: YES), the fallen person determination unit 11e checks whether a person has disappeared from the video frame in the predetermined area near the loading door 24 in the analysis result of the second image data (step S26).

[0105] If a person has disappeared from the video frame in the predetermined area near the loading door 24 (that is, if a fallen person is also temporarily detected in the analysis result of the second image data) (step S26: YES), the fallen person determination unit 11e determines that there is a fallen person (step S27).

[0106] On the other hand, if a person has not disappeared from the video frame in the predetermined area near the loading door 24 (step S26: NO), the fallen person determination unit 11e determines that there is no fallen person (step S29).

[0107] When the fallen person determination unit 11e determines that there is a fallen person (after step S27), the instruction unit 11f issues an alarm to notify other workers and transmits a control signal to the crane control device 30 to stop the crane 5 (step S28).

[0108] In step S28, in order to prevent waste from being thrown onto the fallen person and making rescue difficult, the instruction unit 11f may transmit a control signal to the input door control device 20 to close the input door 24 that partitions between the garbage pit 3 and the platform 21. Also, instead of transmitting a control signal to the crane control device 30 to stop the crane 5, the instruction unit 11f may transmit a control signal to the crane control device 30 to operate the crane 5 to rescue the fallen person. Further, the instruction unit 11f may transmit a control signal to a rescue equipment control device (not shown) to operate rescue equipment (not shown) provided in the garbage pit 3 to rescue the fallen person. Thereby, even when a fallen person occurs during the automatic operation of the crane 5 (that is, when the crane operator is absent), prompt rescue of the fallen person becomes possible.

[0109] By the way, in the waste incineration facility, since the space between the platform 21 on the side where the fall originated and the garbage pit 3 on the side where the fall occurred is partitioned by the garbage input door 24, it is difficult to install an imaging device such as a camera at a location where the side where the fall originated and the side where the fall occurred can be photographed simultaneously, like the fallen person detection system used in the railway industry.

[0110] On the other hand, according to the present embodiment, without installing a camera at a location where the side where the fall originated and the side where the fall occurred can be photographed simultaneously, first image data is acquired from the first camera 6 that images the inside of the garbage pit 3 on the side where the fall occurred, and second image data is acquired from the second camera 23 that images the inside of the platform 21 on the side where the fall originated. By combining the analysis result of the first image data obtained by imaging the inside of the garbage pit 3 and the analysis result of the second image data obtained by imaging the inside of the platform 21, a fallen person falling from the platform 21 to the garbage pit 3 can be automatically detected.

[0111] Generally, when using a machine learning model to detect a fallen person from image data, the detection accuracy of the machine learning model varies, and it is impossible to detect a fallen person with 100% accuracy. However, according to this embodiment, by combining the analysis result of the first image data captured inside the garbage pit 3 and the analysis result of the second image data captured inside the platform 21, the detection accuracy of the fallen person can be improved.

[0112] Also, according to this embodiment, even for an existing facility, by simply additionally installing the first camera 6 that captures inside the garbage pit 3 and the second camera 23 that captures inside the platform 21, the operation of this system becomes possible. Therefore, a major renovation of the existing facility is not required for the introduction of this system, and it is possible to inexpensively enhance the safety of the existing facility.

[0113] In the above-described embodiment, the waste treatment facility 100 where the fallen person detection system 10 is provided is a facility where the garbage identification system 40 is provided, and the automatic operation of the crane 5 is configured to be performed based on the identification result of the garbage identification system 40. However, it is not limited to this. The waste treatment facility 100 where the fallen person detection system 10 is provided may be a facility where the garbage identification system 40 is not provided.

[0114] Also, in the above-described embodiment, the fallen person detection system 10 is configured to determine the presence or absence of a fallen person based on a combination of the analysis result of the first image data captured inside the garbage pit 3 and the analysis result of the second image data captured inside the platform 21. However, when sufficient detection accuracy can be obtained only from the analysis result of the first image data, the presence or absence of a fallen person may be determined based only on the analysis result of the first image data. Or when sufficient detection accuracy can be obtained only from the analysis result of the second image data, the presence or absence of a fallen person may be determined based only on the analysis result of the second image data.

[0115] As described above, the embodiments and modifications of the present invention have been explained by way of example. However, the scope of the present invention is not limited to these, and it is possible to make changes and modifications according to the purpose within the scope described in the claims. Also, each embodiment and modification can be appropriately combined as long as the processing contents do not conflict with each other.

[0116] Further, the faller detection system 10 according to the present embodiment can be configured by one or more computers. However, a program for realizing the faller detection system 10 on one or more computers and a recording medium on which the program is non-temporarily recorded are also the protection objects of this case.

Explanation of Signs

[0117] 1 Incinerator 2 Combustion device 3 Garbage pit 4 Hopper 5 Crane 6 First camera 10 Faller detection system 11 Control unit 11a1 First image data acquisition unit 11a2 Second image data acquisition unit 11b1 First teacher data generation unit 11b2 Second teacher data generation unit 11c1 First model construction unit 11c2 Second model construction unit 11d1 First image analysis unit 11d2 Second image analysis unit 11e Faller determination unit 11f Instruction unit 12 Storage unit 12a1 First detection algorithm 12a2 Second detection algorithm 12b1 First image data 12b2 Second image data 12c1 First teacher data 12c2 Second teacher data 13 Communication unit 20 Loading door control device 21 Platform 22 Conveyor vehicle 23 Second camera 24 Loading door 30 Crane control device 40 Garbage identification system 100 Waste treatment facility

Claims

1. A first image data acquisition unit that acquires first image data from a first camera that images the inside of a storage facility in which an object to be processed is stored; Comprising a first image analysis unit that performs image analysis on the first image data to detect a person inside the storage facility; The first image analysis unit uses a first detection algorithm constructed by machine learning of first teacher data generated by assigning an artificial label as information to an area where a person or a dummy figure imitating a person exists in past image data inside the storage facility. With the new image data inside the storage facility obtained by dividing the surface of the storage facility into a plurality of blocks as input in block units, a person inside the storage facility is detected in block units, As the plurality of blocks, including a block corresponding to the position of the loading door that partitions between the storage facility and the platform, A falling person detection system characterized by the above.

2. A first image data acquisition unit that acquires first image data from a first camera that images the inside of a storage facility in which an object to be processed is stored; Comprising a first image analysis unit that performs image analysis on the first image data to detect a person and / or a transport vehicle inside the storage facility; The first image analysis unit uses a first detection algorithm constructed by machine learning of first teacher data generated by assigning artificial labels as information to an area where a person or a dummy figure imitating a person exists and an area where a transport vehicle exists in past image data inside the storage facility. With the new image data inside the storage facility obtained by dividing the surface of the storage facility into a plurality of blocks as input in block units, a person and / or a transport vehicle inside the storage facility is detected in block units, As the plurality of blocks, including a block corresponding to the position of the loading door that partitions between the storage facility and the platform, A falling person detection system characterized by the above.

3. When the entry of a person into a predetermined area near the loading door at a first time is detected, the analysis result of the image obtained by imaging the area inside the storage facility corresponding to the position of the loading door in the first image data at the first time is confirmed. When a person is detected inside the storage facility, further comprising a falling person determination unit that determines that there is a falling person The falling person detection system according to claim 1 or 2, characterized by the above. When no person is detected in the storage facility when checking the analysis result of the first image data at the first time, the first image data at the first time is compared with the first image data at the second time after a predetermined time has elapsed from the first time, and the difference between them is extracted. When the difference between the first image data at the first time and the first image data at the second time exceeds a predetermined threshold value, it is determined that there is a fallen person, and when there is no difference, it further includes a fallen person determination unit that determines that there is no fallen person. The fallen person detection system according to claim 1 or 2, characterized in that.

5. A second image data acquisition unit that acquires second image data from a second camera that images the inside of a platform adjacent to a storage facility in which an object to be processed is stored; A second image analysis unit that performs image analysis on the second image data to detect a person inside the platform and tracks the movement line of the detected person; Based on the analysis result of the second image data, it includes a fallen person determination unit that determines the presence or absence of a fallen person who falls from the platform to the storage facility. The second image analysis unit uses a second detection algorithm constructed by machine learning of second teacher data generated by artificially assigning labels as information to areas where a person or a dummy figure imitating a person exists in past image data inside the platform. Using new image data inside the platform as input, a person inside the platform is detected. The fallen person detection system, characterized in that.

6. A second image data acquisition unit that acquires second image data from a second camera that images the inside of a platform adjacent to a storage facility in which an object to be processed is stored; A second image analysis unit that performs image analysis on the second image data to detect a person and a transport vehicle inside the platform and tracks the movement lines of the detected person and the transport vehicle; Based on the analysis result of the second image data, it includes a fallen person determination unit that determines the presence or absence of a fallen person and a fallen transport vehicle that fall from the platform to the storage facility. The second image analysis unit uses a second detection algorithm constructed by machine learning of second teacher data generated by artificially assigning labels as information to areas where a person or a dummy figure imitating a person exists and areas where a transport vehicle exists in past image data inside the platform. Using new image data inside the platform as input, a person and a transport vehicle inside the platform are detected. A falling person detection system characterized by the following.

7. The second image analysis unit respectively tracks the movement lines of the detected person and / or the transport vehicle, and detects the entry of the person and the transport vehicle into a predetermined area near the loading door that separates the storage facility and the platform. The falling person detection system according to claim 5 or 6.

8. The falling person determination unit checks the analysis result of the second image data, and when a person and / or a transport vehicle has dropped out of the video within a predetermined area near the loading door that separates the storage facility and the platform, determines that there is a falling person and / or a fall of the transport vehicle. The falling person detection system according to claim 5 or 6.

9. The first detection algorithm includes one or more of the maximum likelihood classification method, Boltzmann machine, neural network, support vector machine, Bayesian network, sparse regression, decision tree, statistical estimation using random forest, reinforcement learning, and deep learning. The falling person detection system according to any one of claims 1 to 4.

10. The second detection algorithm includes one or more of the maximum likelihood classification method, Boltzmann machine, neural network, support vector machine, Bayesian network, sparse regression, decision tree, statistical estimation using random forest, reinforcement learning, and deep learning. The falling person detection system according to any one of claims 5 to 8.

11. The algorithm used by the second image analysis unit for tracking the movement line of a person includes one or more of optical flow, background difference method, Kalman filter, particle filter, and deep learning. The falling person detection system according to any one of claims 5 to 8 and 10.

12. The first camera includes one or more of an RGB camera, a near-infrared camera, a 3D camera, or an RGB-D camera. The falling person detection system according to any one of claims 1 to 4 and 9.

13. The second camera includes one or more of an RGB camera, a near-infrared camera, a 3D camera, or an RGB-D camera. The falling person detection system according to any one of claims 5 to 8, 10, and 11.

14. A waste treatment facility equipped with the falling person detection system according to any one of claims 1 to 13.

15. A step of acquiring first image data from a first camera that images the inside of a storage facility where an object to be processed is stored; A step of performing image analysis on the first image data to detect a person inside the storage facility; A step of detecting a person inside the storage facility in block units, using a first detection algorithm constructed by machine learning on first teacher data generated by artificially attaching labels as information to areas where a person or a dummy figure imitating a person exists in past image data inside the storage facility, with new image data inside the storage facility obtained by dividing the surface of the storage facility into a plurality of blocks being input in block units, wherein the plurality of blocks includes a block corresponding to the position of an input door that partitions between the storage facility and the platform; A falling person detection method including the above.

16. A step of acquiring first image data from a first camera that images the inside of a storage facility where an object to be processed is stored; A step of performing image analysis on the first image data to detect a person and / or a transport vehicle inside the storage facility; A step of detecting a person and / or a transport vehicle inside the storage facility in block units, using a first detection algorithm constructed by machine learning on first teacher data generated by artificially attaching labels as information to areas where a person or a dummy figure imitating a person exists and areas where a transport vehicle exists in past image data inside the storage facility, with new image data inside the storage facility obtained by dividing the surface of the storage facility into a plurality of blocks being input in block units, wherein the plurality of blocks includes a block corresponding to the position of an input door that partitions between the storage facility and the platform; A falling person detection method including the above.

17. A step of acquiring second image data from a second camera that images the inside of a platform adjacent to the storage facility; A step of performing image analysis on the second image data to detect a person inside the platform and tracking the detected person; A step of determining the presence or absence of a falling person who falls from the platform to the storage facility based on the analysis result of the second image data; Using a second detection algorithm constructed by machine learning the second teacher data generated by artificially attaching a label as information to an area where a person or a dummy figure imitating a person exists in past image data within the platform, taking new image data within the platform as input, and detecting a person within the platform; A falling person detection method including the above.

18. Obtaining second image data from a second camera that images the platform adjacent to the storage facility; Performing image analysis on the second image data to detect a person and a transport vehicle within the platform and tracking the movement paths of the detected person and the transport vehicle; Based on the analysis result of the second image data, determining whether there is a falling person falling from the platform to the storage facility and whether the transport vehicle has fallen; Using a second detection algorithm constructed by machine learning the second teacher data generated by artificially attaching a label as information to an area where a person or a dummy figure imitating a person exists and an area where a transport vehicle exists in past image data within the platform, taking new image data within the platform as input, and detecting a person and a transport vehicle within the platform; A falling person detection method including the above.

19. On a computer, Obtaining first image data from a first camera that images the inside of a storage facility where an object to be processed is stored; Performing image analysis on the first image data to detect a person within the storage facility; Using a first detection algorithm constructed by machine learning the first teacher data generated by artificially attaching a label as information to an area where a person or a dummy figure imitating a person exists in past image data within the storage facility, taking new image data within the storage facility obtained by dividing the surface of the storage facility into a plurality of blocks as input in block units, and detecting a person within the storage facility in block units, wherein the plurality of blocks includes a block corresponding to the position of an input door that partitions between the storage facility and the platform; A falling person detection program for causing the above to be executed.

20. On a computer, Obtaining first image data from a first camera that images the inside of a storage facility where an object to be processed is stored; Performing image analysis on the first image data to detect a person and / or a transport vehicle within the storage facility; Using a first detection algorithm constructed by machine learning the first training data generated by assigning artificial labels as information to areas where there are people or dummy figures imitating people in past image data in the storage facility and areas where the transport vehicle exists, taking new image data in the storage facility divided into a plurality of blocks as input in block units, and detecting people and / or transport vehicles in the storage facility in block units, wherein the plurality of blocks includes a block corresponding to the position of the loading door partitioning between the storage facility and the platform, and A falling person detection program to be executed.

21. Causing a computer to acquire second image data from a second camera that images inside the platform adjacent to the storage facility; perform image analysis on the second image data to detect people inside the platform and track the movement paths of the detected people; judge whether there is a falling person who falls from the platform to the storage facility based on the analysis result of the second image data; using a second detection algorithm constructed by machine learning the second training data generated by assigning artificial labels as information to areas where there are people or dummy figures imitating people in past image data inside the platform and areas where the transport vehicle exists, taking new image data inside the platform as input, and detecting people inside the platform; A falling person detection program to be executed.

22. Causing a computer to acquire second image data from a second camera that images inside the platform adjacent to the storage facility; perform image analysis on the second image data to detect people and transport vehicles inside the platform and track the movement paths of the detected people and transport vehicles; judge whether there is a falling person and whether a transport vehicle falls from the platform to the storage facility based on the analysis result of the second image data; using a second detection algorithm constructed by machine learning the second training data generated by assigning artificial labels as information to areas where there are people or dummy figures imitating people in past image data inside the platform and areas where the transport vehicle exists, taking new image data inside the platform as input, and detecting people and transport vehicles inside the platform; A falling person detection program that causes execution.

Images