Falling person detection system, falling person detection method, and falling person detection program

Abstract

To provide a system, a method, and a program for detecting a falling person, capable of automatically detecting a person falling into a storage facility where waste is stored within a waste treatment facility.SOLUTION: A falling person detection system 10 includes: a first image data acquisition unit that acquires first image data from a first camera 6 imaging inside a storage facility (pit 3) where processed items are stored; a second image data acquisition unit that acquires second image data from a second camera 23 imaging inside a platform 21 adjacent to the storage facility; a first image analysis unit that analyzes the first image data to detect a person within the storage facility; a second image analysis unit that analyzes the second image data to detect a person within the platform and tracks a flow line of the detected person; and a falling person detection unit that determines the presence or absence of a person falling from the platform into the storage facility based on a combination of analysis results of the first image data and analysis results of the second image data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a fallen person detection system, a fallen person detection method, and a fallen person detection program that automatically detect a person who has fallen into a facility that stores materials to be processed. [Background technology]

[0002] At waste disposal facilities, particularly waste incineration facilities, although the number of incidents is small, several accidents involving people falling into garbage pits are reported each year.

[0003] If a person falls into a garbage pit, the crane operator must be notified and the crane operation must be temporarily stopped to ensure the safety of the person who fell. However, with the recent development of automated crane operation technology, if a person falls while a crane is operating automatically (i.e., when the crane operator is not present), it may take time to notify the crane operator, which could result in a serious accident.

[0004] Furthermore, particularly in waste incineration facilities, there are no accident prevention systems that automatically detect people who fall into garbage pits and control cranes.

[0005] In other fields (particularly the railway industry), systems are used that use image processing or the like to detect people who have fallen from station platforms onto the tracks (see, for example, Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent No. 5386744 [Patent Document 2] Patent No. 4041678 Summary of the Invention [Problem to be solved by the invention]

[0007] In the railway industry, fallen person detection systems process images from surveillance cameras that simultaneously capture the platform, which is the side where the fall occurred, and the tracks, which are the side where the fall occurred, to detect fallen persons.In contrast, at waste incineration facilities, the platform, which is the side where the fall occurred, and the garbage pit, which is the side where the fall occurred, are separated by a garbage input door, making it difficult to install an imaging device such as a camera in a location where it can simultaneously capture the side where the fall occurred and the side where the fall occurred.

[0008] Furthermore, in the railway industry, fallen person detection systems sometimes use a method of determining the size of a fallen object by processing images from surveillance cameras, and then determining from that size whether the fallen object is a person.However, at waste incineration facilities, waste of various sizes is dumped into the waste pit from delivery vehicles (compact trucks, light trucks, etc.), making it difficult to determine whether a fallen object is a person based on size alone.

[0009] Furthermore, as mentioned above, in waste incineration facilities, it is difficult to install a camera in a location that can simultaneously capture both the side where the fall occurred (platform) and the side where the fall resulted (garbage pit), so one option is to install a camera on the garbage pit side and process images from inside the garbage pit to detect a person who has fallen. However, there is a possibility that a pile of waste may collapse inside the garbage pit, causing waste to fall over the person who has fallen, in which case it is difficult to detect a person who has fallen by processing images from inside the garbage pit.

[0010] In addition to facilities that use the pit and crane system, such as incineration facilities, Similar issues exist in waste treatment facilities (such as bulky waste crushing facilities and recycling facilities) that use a system other than the dump and crane system, such as direct dumping into a storage facility or a compactor container system.

[0011] The present invention has been made in consideration of the above points, and an object of the present invention is to provide a fallen person detection system, a fallen person detection method, and a fallen person detection program that can automatically detect a person who has fallen into storage equipment that stores waste in a waste treatment facility such as a waste incineration facility. [Means for solving the problem]

[0012] A fallen person detection system according to a first aspect of the present invention includes: a first image data acquisition unit that acquires first image data from a first camera that captures an image of the inside of the storage facility in which the object to be processed is stored; a second image data acquisition unit that acquires second image data from a second camera that captures an image of the inside of a platform adjacent to the storage facility; a first image analysis unit that performs image analysis on the first image data to detect a person in the storage facility; a second image analysis unit that performs image analysis of the second image data to detect a person on the platform and track the movement of the detected person; a faller determination unit that determines whether or not a person has fallen from the platform into the storage facility based on a combination of the analysis results of the first image data and the analysis results of the second image data; Equipped with.

[0013] According to this embodiment, by combining the analysis results of the first image data of the image inside the storage facility and the analysis results of the second image data of the image inside the platform, it is possible to automatically detect a person who falls from the platform into the storage facility. Generally, when using a machine learning model to detect a person who has fallen from image data, the detection accuracy of the machine learning model varies, and it is impossible to detect a person who has fallen with 100% accuracy. However, by combining the analysis results of the first image data of the image inside the storage facility and the analysis results of the second image data of the image inside the platform, it is possible to improve the accuracy of detecting a person who has fallen. Furthermore, even in existing facilities, the system can be operated simply by additionally installing a first camera that images the inside of the storage facility and a second camera that images the inside of the platform. Therefore, introducing the system does not require significant modifications to the existing facility, making it possible to inexpensively improve the safety of existing facilities.

[0014] A fallen person detection system according to a second aspect of the present invention is the fallen person detection system according to the first aspect, The fallen person determination unit checks the analysis results of the second image data, and if a person is detected entering a predetermined area near the input door separating the storage facility and the platform at a first time, it checks the analysis results of the first image data at the first time and determines that a person has fallen if a person is detected within the storage facility.

[0015] According to this aspect, the analysis results of the second image data taken inside the platform are checked, and if a person is detected entering a predetermined area near the input door (i.e., if a person who has fallen is tentatively detected in the analysis results of the second image data), the analysis results of the first image data are checked, and if a person is detected within the storage facility (i.e., if a person who has fallen is also tentatively detected in the analysis results of the first image data), a final determination is made that a person has fallen, thereby making it possible to accurately detect a person who has fallen.

[0016] A fallen person detection system according to a third aspect of the present invention is the fallen person detection system according to the second aspect, When a person is detected entering a predetermined area near the deposit door at a first time, the fallen person determination unit checks the analysis results of an image captured of the area within the storage facility corresponding to the position of the deposit door from the first image data at the first time.

[0017] According to this aspect, even in the case of a large storage facility with multiple input doors, if the analysis results of the second image data taken inside the platform are checked and a person is detected entering a predetermined area near a input door (i.e., if a person who has fallen is tentatively detected in the analysis results of the second image data), rather than checking the analysis results of the entire first image data taken inside the storage facility, the person who has fallen can be detected more accurately and in a shorter time by checking the analysis results of the image of the area inside the storage facility that corresponds to the position of the input door where the person who has fallen is most likely to have occurred.

[0018] A fallen person detection system according to a fourth aspect of the present invention is the fallen person detection system according to the second or third aspect, If no person is detected in the storage facility when checking the analysis results of the first image data at the first time, the fallen person determination unit extracts the difference between the first image data at the first time and the first image data at a second time a predetermined time after the first time, and determines that a person has fallen if the extracted difference exceeds a predetermined threshold, or determines that no person has fallen if there is no difference.

[0019] According to this aspect, for example, if a processing object falls over the person who has fallen inside the storage facility, the analysis result of the first image data captured inside the storage facility will indicate that no person has been detected inside the storage facility (because the processing object is falling over the person), but because the analysis result of the second image data has already provisionally detected the person who has fallen, the system further compares the first image data at the first time with the first image data at a second time a predetermined time after the first time, and extracts the difference between them. If the difference between the first image data at the first time and the first image data at the second time exceeds a predetermined threshold, it is considered that a pile of processing objects has collapsed inside the storage facility, and it is possible that the person who has fallen is hidden by the processing object in the collapsed pile, and therefore it is determined that a person has fallen. This makes it possible to automatically detect a person who has fallen inside the storage facility, even if the person who has fallen inside the storage facility is covered by the processing object.

[0020] A fallen person detection system according to a fifth aspect of the present invention is the fallen person detection system according to the first aspect, The fallen person determination unit checks the analysis results of the first image data, and if a person is detected in the storage facility at the first time, checks the analysis results of the second image data up to a predetermined time before the first time, and determines that a person has fallen if the person is framed out of the image within a predetermined area near the input door that separates the storage facility from the platform.

[0021] According to this aspect, the analysis results of the first image data taken inside the storage facility are checked, and if a person is detected inside the storage facility (i.e., if a person who has fallen is tentatively detected in the analysis results of the first image data), the analysis results of the second image data are checked, and if a person has fallen out of the frame near the input door (i.e., if a person who has fallen is also tentatively detected in the analysis results of the second image data), a final determination is made that a person has fallen, thereby making it possible to accurately detect a person who has fallen.

[0022] A fallen person detection system according to a sixth aspect of the present invention is the fallen person detection system according to the fifth aspect, When a person is detected in the storage facility at the first time, the fallen person determination unit detects the person in the storage facility from the second image data up to a predetermined time before the first time. The analysis results of the image of the input door at the position corresponding to the area are confirmed.

[0023] According to this aspect, even in the case of a large platform with multiple input doors, the analysis results of the first image data taken inside the storage facility are checked, and if a person is detected inside the storage facility (i.e., if a person who has fallen is tentatively detected in the analysis results of the first image data), rather than checking the analysis results of the entire second image data taken inside the platform, the analysis results of the image of the input door located in a position where the person who fell is most likely to have occurred are checked, making it possible to detect the person who has fallen more accurately and in a shorter time.

[0024] A fallen person detection system according to a seventh aspect of the present invention is the fallen person detection system according to the fifth or sixth aspect, The fallen person determination unit determines that no person has fallen if, when checking the analysis results of the second image data up to a predetermined time before the first time, a person is found to have fallen out of the image outside a predetermined area near the input door.

[0025] According to this aspect, if a person is out of frame in the image in the second image data taken inside the platform for reasons unrelated to falling into the storage facility (for example, if the person is temporarily hiding behind a transport vehicle), it is possible to prevent a false determination that a person has fallen, thereby improving the accuracy of detecting a person who has fallen.

[0026] A fallen person detection system according to an eighth aspect of the present invention is a fallen person detection system according to any one of the fifth to seventh aspects, The fallen person determination unit determines that no person has fallen if, when checking the analysis results of the second image data up to a predetermined time before the first time, no person is detected entering a predetermined area near the input door.

[0027] According to this aspect, even if, for example, an object to be processed that is approximately the same size as a person is mistakenly detected as a person in the first image data taken inside the storage facility (i.e., a person who has fallen is mistakenly detected as a person in the analysis results of the first image data), if, when the analysis results of the second image data taken inside the platform are checked, no person is detected entering near the input door, it is determined that no person has fallen, thereby preventing an erroneous determination that a person has fallen and improving the accuracy of detecting people who have fallen.

[0028] A fallen person detection system according to a ninth aspect of the present invention is the fallen person detection system according to any one of the first to eighth aspects, When the faller determination unit determines that a person has fallen, (1) Issue an alarm; (2) sending a control signal to the crane control device to stop the crane that mixes or transports the material stored in the storage facility; (3) Sending a control signal to the loading door control device to close the loading door separating the storage facility and the platform; (4) Sending a control signal to the crane control device to operate the crane and rescue the fallen victim; (5) Sending a control signal to the rescue equipment control device to operate the rescue equipment installed in the storage facility and rescue the person who fell; The apparatus further includes an instruction unit that performs at least one of the processes above.

[0029] According to this aspect, it is possible to quickly rescue a person who has fallen, thereby increasing the safety of the facility.

[0030] A fallen person detection system according to a tenth aspect of the present invention is a fallen person detection system according to any one of the first to ninth aspects, The first image analysis unit uses a first detection algorithm constructed by machine learning first training data generated by assigning artificial labels as information to areas in past image data within the storage facility where people or dummy dolls resembling people are present, and detects people within the storage facility using new image data within the storage facility as input.

[0031] A fallen person detection system according to an eleventh aspect of the present invention is a fallen person detection system according to any one of the first to tenth aspects, The second image analysis unit uses a second detection algorithm constructed by machine learning second training data generated by assigning artificial labels as information to areas of past image data within the platform where people or dummy dolls resembling people exist, and detects people within the platform using new image data within the platform as input.

[0032] A fallen person detection system according to a twelfth aspect of the present invention is the fallen person detection system according to the eleventh aspect, The second training data was generated by assigning artificial labels as information to areas where people or dummy dolls resembling people exist in past image data within the platform, and by assigning another artificial label as information to areas where delivery vehicles exist.

[0033] According to this aspect, people (e.g., workers) on the platform are often working near delivery vehicles, and there is a relationship between the position of the person and the position of the delivery vehicle.Therefore, by using a second detection algorithm constructed by machine learning training data generated by assigning artificial labels as information to areas of past image data on the platform where people or dummy dolls resembling people are present, and assigning another artificial label as information to areas where delivery vehicles are present, not only can the accuracy of detecting people on the platform be improved, but it can also detect falling delivery vehicles.

[0034] A fallen person detection system according to a thirteenth aspect of the present invention is the fallen person detection system according to the tenth aspect, The first detection algorithm may include one or more of a maximum likelihood classification method, a Boltzmann machine, a neural network, a support vector machine, a Bayesian network, sparse regression, a decision tree, statistical inference using random forests, reinforcement learning, and deep learning.

[0035] A fallen person detection system according to a fourteenth aspect of the present invention is the fallen person detection system according to the eleventh or twelfth aspect, The second detection algorithm may include one or more of maximum likelihood classification, Boltzmann machines, neural networks, support vector machines, Bayesian networks, sparse regression, decision trees, statistical inference using random forests, reinforcement learning, and deep learning.

[0036] A fallen person detection system according to a fifteenth aspect of the present invention is a fallen person detection system according to any one of the first to fourteenth aspects, The algorithm used by the second image analysis unit to track the movement line of the person includes one or more of optical flow, background subtraction, Kalman filter, particle filter, and deep learning.

[0037] A fallen person detection system according to a sixteenth aspect of the present invention is a fallen person detection system according to any one of the first to fifteenth aspects, The first camera includes one or more of an RGB camera, a near-infrared camera, a 3D camera, or an RGB-D camera.

[0038] A fallen person detection system according to a seventeenth aspect of the present invention is a fallen person detection system according to any one of the first to sixteenth aspects, The second camera includes one or more of an RGB camera, a near-infrared camera, a 3D camera, or an RGB-D camera.

[0039] A waste disposal facility according to an eighteenth aspect of the present invention includes the fallen person detection system according to any one of the first to seventeenth aspects.

[0040] A method for detecting a person falling according to a nineteenth aspect of the present invention includes: acquiring first image data from a first camera that captures an image of the inside of a storage facility in which the object to be processed is stored; acquiring second image data from a second camera imaging an interior of a platform adjacent to the storage facility; performing image analysis on the first image data to detect a person in the storage facility; performing image analysis on the second image data to detect a person on the platform and track the movement of the detected person; A step of determining whether or not a person has fallen from the platform into the storage facility based on a combination of the analysis results of the first image data and the analysis results of the second image data; Includes.

[0041] A fallen person detection program according to a twentieth aspect of the present invention includes: On the computer, acquiring first image data from a first camera that captures an image of the inside of a storage facility in which the object to be processed is stored; acquiring second image data from a second camera imaging an interior of a platform adjacent to the storage facility; performing image analysis on the first image data to detect a person in the storage facility; performing image analysis on the second image data to detect a person on the platform and track the movement of the detected person; A step of determining whether or not a person has fallen from the platform into the storage facility based on a combination of the analysis results of the first image data and the analysis results of the second image data; Execute the following. [Effects of the Invention]

[0042] According to the present invention, it is possible to automatically detect a person who has fallen into storage equipment that stores waste in a waste treatment facility such as a waste incineration facility. [Brief explanation of the drawings]

[0043] [Figure 1] FIG. 1 is a schematic diagram showing the configuration of a waste treatment facility according to one embodiment. [Figure 2] FIG. 2 is a block diagram showing the configuration of a fallen person detection system according to one embodiment. [Figure 3] FIG. 3 is a flowchart showing a first example of a method for detecting a fallen person by the fallen person detection system according to one embodiment. [Figure 4A] FIG. 4A is a diagram showing an example of an analysis result of second image data obtained by capturing an image of the inside of the platform. [Figure 4B] FIG. 4B is a diagram showing an example of the analysis result of the first image data obtained by capturing an image of the inside of the garbage pit. [Figure 5] FIG. 5 is a flowchart showing a second example of a method for detecting a fallen person by the fallen person detection system according to the embodiment. [Figure 6A] FIG. 6A is a diagram showing an example of an analysis result of first image data obtained by capturing an image of the inside of a garbage pit. [Figure 6B] FIG. 6B is a diagram showing an example of the analysis result of the second image data obtained by capturing an image of the inside of the platform. DETAILED DESCRIPTION OF THE INVENTION

[0044] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description and the drawings used in the following description, parts that can be configured identically will be designated by the same reference numerals, and duplicated descriptions will be omitted.

[0045] (Configuration of waste treatment facilities) FIG. 1 is a schematic diagram showing the configuration of a waste treatment facility 100 according to one embodiment.

[0046] As shown in FIG. 1, the waste treatment facility 100 includes a platform 21 on which transport vehicles (such as compactor vehicles and light trucks) 22 carrying waste are parked; a waste pit (storage facility) 3 in which waste deposited from the platform 21 is stored; a crane 5 that mixes and transports the waste stored in the waste pit 3; a hopper 4 into which the waste transported by the crane 5 is deposited; an incinerator 1 that incinerates the waste deposited from the hopper 4; and a waste heat boiler 2 that recovers waste heat from the exhaust gas generated in the incinerator 1. The type of incinerator 1 is not limited to a stoker incinerator as shown in FIG. 1 but also includes a fluidized bed incinerator. Furthermore, the structure of the waste pit 3 is not limited to the single-tiered pit shown in FIG. 1 but also includes a two-tiered pit divided into a loading section and a storage section. The waste pit 3 and the platform 21 are separated by a loading door 24. The waste treatment facility 100 is also provided with a loading door control device 20 that controls the operation of the loading door 24 and a crane control device 30 that controls the operation of the crane 5.

[0047] Waste carried in on a transport vehicle 22 is dumped from the platform 21 through a dump door 24 into the garbage pit 3, where it is stored. The waste stored in the garbage pit 3 is stirred by a crane 5, and is also transported to a hopper 4 by the crane 5, and dumped into the incinerator 1 via the hopper 4, where it is incinerated and disposed of.

[0048] As shown in FIG. 1, the waste treatment facility 100 is provided with a first camera 6 that takes images of the inside of the waste pit 3, and a waste identification system 40 that identifies the type of waste inside the waste pit 3.

[0049] The first camera 6 is placed above the garbage pit 3, and in the illustrated example is fixed to the rails of the crane 5, so that it can capture images of the waste stored in the garbage pit 3 from above the garbage pit 3. Only one first camera 6 may be installed, or multiple first cameras 6 may be installed.

[0050] The first camera 6 may be an RGB camera that outputs shape and color image data of the waste as an imaging result, a near-infrared camera that outputs near-infrared image data of the waste as an imaging result, a 3D camera or RGB-D camera that captures three-dimensional image data of the waste as an imaging result, or a combination of two or more of these. It's okay to have one.

[0051] The waste identification system 40 acquires image data (also referred to as first image data) from the first camera 6 that captures images inside the waste pit 3, and performs image analysis of the first image data to identify the type of waste stored in the waste pit 3. For example, the waste identification system 40 may use an identification algorithm (trained model) constructed by machine learning training data in which past image data captured inside the waste pit 3 is labeled with the type of waste, to input new image data of the inside of the waste pit 3 and identify the type of waste stored in the waste pit 3.

[0052] The waste identification system 40 generates a map showing the ratio of waste types for each area as a result of identifying the types of waste stored in the waste pit 3, and transmits this to the crane control device 30. Based on the map received from the waste identification system 40, the crane control device 30 operates the crane 5 to mix the waste in the waste pit 3 so that the ratio of waste types is equal in all areas. This enables the crane 5 to be operated automatically.

[0053] Specifically, the litter identification system 40 may be, for example, an information processing device described in Japanese Patent No. 6731680.

[0054] As shown in Figure 1, the waste treatment facility 100 is further provided with a second camera 23 that takes images inside the platform 21 and a falling person detection system 10 that detects a person falling from the platform 21 into the garbage pit 3.

[0055] The second camera 23 is disposed above the platform 21, and in the illustrated example is fixed to the wall of the platform 21 located near the front of the input door 24, so that it can capture images of the inside of the platform 21 from near the front of the input door 24. Only one second camera 23 may be installed, or multiple second cameras 23 may be installed.

[0056] The second camera 23 may be an RGB camera that outputs shape and color image data of an object (such as a person such as a worker or a transport vehicle 22) as the imaging result, a near-infrared camera that outputs near-infrared image data of the object as the imaging result, a 3D camera or RGB-D camera that captures three-dimensional image data of the object as the imaging result, or a combination of two or more of these.

[0057] (Configuration of a fall detection system) Next, we will explain the configuration of the fallen person detection system 10 that detects a person who falls from the platform 21 into the garbage pit 3. Figure 2 is a block diagram showing the configuration of the fallen person detection system 10. The fallen person detection system 10 may be configured by one computer, or may be configured by multiple computers that are connected to each other so that they can communicate with each other.

[0058] 2, the fallen person detection system 10 includes a control unit 11, a storage unit 12, and a communication unit 13. Each unit is connected to each other via a bus or a network so as to be able to communicate with each other.

[0059] Of these, the communication unit 13 is a communication interface for the first camera 6, the second camera 23, the crane control device 30, and the dropping door control device 20. The communication unit 13 transmits and receives information between the first camera 6, the second camera 23, the crane control device 30, the dropping door control device 20, and the fallen person detection system 10.

[0060] The storage unit 12 is a non-volatile data storage such as a hard disk or flash memory. The memory unit 12 stores various data handled by the control unit 11. The memory unit 12 also stores a first detection algorithm 12a1 constructed by a first model construction unit 11c1 (described later), a second detection algorithm 12a2 constructed by a second model construction unit 11c2, first image data 12b1 acquired by a first image data acquisition unit 11a1, second image data 12b2 acquired by a second image data acquisition unit 11a2, first teacher data 12c1 generated by a first teacher data generation unit 11b1, and second teacher data 12c2 generated by a second teacher data generation unit 11b2.

[0061] The control unit 11 is a control means that performs various processes of the fallen person detection system 10. As shown in Fig. 2, the control unit 11 has a first image data acquisition unit 11a1, a second image data acquisition unit 11a2, a first teacher data generation unit 11b1, a second teacher data generation unit 11b2, a first model construction unit 11c1, a second model construction unit 11c2, a first image analysis unit 11d1, a first image analysis unit 11d2, a fallen person determination unit 11e, and an instruction unit 11f. Each of these units may be realized by a processor in the fallen person detection system 10 executing a predetermined program, or may be implemented in hardware.

[0062] Of these, the first image data acquisition unit 11a1 acquires first image data from the first camera 6 that captures images inside the garbage pit 3. The first image data may be a moving image or a series of still images. The frame rate of the first image data may be a general frame rate (approximately 30 fps), and does not need to be a particularly high frame rate, and may also be a low frame rate (approximately 5 to 10 fps). Metadata of the first image data includes information on the date and time of shooting. The first image data 12b1 acquired by the first image data acquisition unit 11a1 is stored in the storage unit 12.

[0063] The second image data acquisition unit 11a2 acquires second image data from the second camera 23 that captures images inside the platform 21. The second image data may be a moving image or a series of still images. The frame rate of the second image data may be a general frame rate (approximately 30 fps), and does not need to be a particularly high frame rate, and may also be a low frame rate (approximately 5 to 10 fps). Metadata of the second image data includes information on the date and time of shooting. The second image data 12b2 acquired by the second image data acquisition unit 11a2 is stored in the storage unit 12.

[0064] The first teacher data generation unit 11b1 generates the first teacher data by labeling information about an area where a person (i.e., a person who fell) or a dummy doll resembling a person is present that has been visually identified by an experienced operator who operates the waste incineration facility 100 (i.e., by artificially assigning a label as information to an area where a person or a dummy doll resembling a person is present) for past image data taken of the inside of the garbage pit 3. As an example, the first teacher data generation unit 11b1 may generate the first teacher data for image data taken of the inside of the garbage pit 3 after a dummy doll resembling a person has been intentionally dropped into the garbage pit 3, or may generate the first teacher data for image data (composite image data) in which an image of a person is composited with past image data taken of the inside of the garbage pit 3. The first teacher data generation unit 11b1 may generate the first teacher data by labeling information about areas where a person (i.e., a person who fell) or a dummy doll resembling a person visually identified by an experienced operator operating the waste incineration facility 100 is present and information about areas where the transport vehicle 22 is present (i.e., by assigning artificial labels as information to areas where a person or a dummy doll resembling a person is present and areas where the transport vehicle 22 is present) for past image data captured inside the garbage pit 3. The information about areas where a person or a dummy doll resembling a person is present and the information about areas where the transport vehicle 22 is present are labeled (i.e., artificial labels are assigned as information) while being superimposed on the image data, for example, as layers. The first teacher data 12c1 generated by the first teacher data generation unit 11b1 is stored in the memory unit 12.

[0065] The second teacher data generation unit 11b2 generates the second teacher data by labeling information about an area where a person or a dummy doll imitating a person exists that has been visually identified by an experienced operator who operates the waste incineration facility 100 (i.e., by artificially assigning a label as information to an area where a person or a dummy doll imitating a person exists) for past image data taken of the inside of the platform 21. As an example, the second teacher data generation unit 11b2 may generate the second teacher data for image data taken of the inside of the platform 21 after a dummy doll imitating a person is intentionally installed within the platform 21, or may generate the second teacher data for image data (composite image data) in which an image of a person is composited with past image data taken of the inside of the platform 21. The second teacher data generation unit 11b2 may generate the second teacher data by labeling information on areas where people or dummy dolls resembling people that have been visually identified by an experienced operator operating the waste incineration facility 100 are present and information on areas where transport vehicles 22 are present (i.e., by assigning artificial labels as information to areas where people or dummy dolls resembling people are present and areas where transport vehicles are present) for past image data taken inside the garbage pit 3. Since people (e.g., workers) frequently work near delivery vehicle 22 within platform 21 and there is a relationship between the position of the person and the position of delivery vehicle 22, second model construction 11c2, which will be described later, performs machine learning on training data in which an artificial label is assigned as information to an area where a person or a dummy doll imitating a person is present in image data captured within platform 21 and another artificial label is assigned as information to an area where delivery vehicle 22 is present, thereby constructing second detection algorithm 12a2, which not only improves the detection accuracy of people within platform 21 by second detection algorithm 12a2 but also makes it possible to detect a fall of delivery vehicle 22. Information on the area where a person or a dummy doll imitating a person is present and information on the area where delivery vehicle 22 is present are labeled (i.e., artificial labels are assigned as information) while being superimposed on the image data, for example, as layers.The second teacher data 12c2 generated by the second teacher data generating unit 11b2 is stored in the storage unit 12.

[0066] The first model construction unit 11c1 performs machine learning on the first teacher data 12c1 stored in the memory unit 12 to construct a first detection algorithm 12a1 (trained model) that uses new image data of the garbage pit 3 as input to detect a person (i.e., a person who has fallen) in the garbage pit 3. The first detection algorithm 12a1 may include one or more of a maximum likelihood classification method, a Boltzmann machine, a neural network, a support vector machine, a Bayesian network, sparse regression, a decision tree, statistical estimation using random forests, reinforcement learning, and deep learning. The first detection algorithm 12a1 constructed by the first model construction unit 11c1 is stored in the memory unit 12.

[0067] The second model construction unit 11c2 performs machine learning on the second teacher data 12c2 stored in the memory unit 12 to construct a second detection algorithm 12a2 (trained model) that detects people within the platform 21 using new image data within the platform 21 as input. The second detection algorithm 12a2 may include one or more of a maximum likelihood classification method, a Boltzmann machine, a neural network, a support vector machine, a Bayesian network, sparse regression, a decision tree, statistical estimation using random forests, reinforcement learning, and deep learning. The second detection algorithm 12a2 constructed by the second model construction unit 11c2 is stored in the memory unit 12.

[0068] The first image analysis unit 11d1 performs image analysis on the first image data acquired by the first image data acquisition unit 11a1 to detect a person in the garbage pit 3. Specifically, for example, the first image analysis unit 11d uses the first detection algorithm 12a1 (trained model) constructed by the first model construction unit 11c1 to input new image data of the inside of the garbage pit 3, A person is detected inside the garbage pit 3. As a modified example, the first image analysis unit 11d1 may perform image analysis of the first image data acquired by the first image data acquisition unit 11a1 to detect a person and a transport vehicle 22 inside the garbage pit 3. Specifically, for example, the first image analysis unit 11d may use the first detection algorithm 12a1 (trained model) constructed by the first model construction unit 11c1 as input of new image data inside the garbage pit 3 to detect a person and a transport vehicle 22 inside the garbage pit 3.

[0069] The first image analysis unit 11d1 may divide the surface of the garbage pit 3 into a plurality of blocks, input new image data of the inside of the garbage pit 3 to the first detection algorithm 12a1 (trained model) in block units, and acquire the detection results of the person (or the person and the transport vehicle 22) in block units. This makes it possible to accurately determine where in the garbage pit 3 the person who fell (or the person who fell and the transport vehicle 22 who fell) occurred.

[0070] The second image analysis unit 11d2 performs image analysis on the second image data acquired by the second image data acquisition unit 11a2 to detect people on the platform 21 and track the movement of the detected people. Specifically, for example, the second image analysis unit 11d uses the second detection algorithm 12a2 (trained model) constructed by the second model construction unit 11c2 as input new image data of the platform 21 to detect people on the platform 21. Next, the second image analysis unit 11d tracks the detected people and detects the people entering a predetermined area near the input door 24. As a modified example, the second image analysis unit 11d2 performs image analysis on the second image data acquired by the second image data acquisition unit 11a2 to detect people and transport vehicles 22 on the platform 21 and track the movement of the detected people and transport vehicles 22. Specifically, for example, the second image analysis unit 11d uses the second detection algorithm 12a2 (trained model) constructed by the second model construction unit 11c2 as input new image data of the inside of the platform 21 to detect people and transport vehicles 22 on the platform 21. Next, the second image analysis unit 11d tracks the detected people and transport vehicles 22, respectively, and detects the entry of the people and transport vehicles 22 into predetermined areas near the input door 24. The algorithm used for tracking may include one or more of optical flow, background subtraction, a Kalman filter, a particle filter, and deep learning.

[0071] As a modified example, the second image analysis unit 11d2 may perform image analysis on the second image data acquired by the second image data acquisition unit 11a2 to detect people on the platform 21, track the movement of the detected people, and detect whether the detected people are wearing safety equipment (safety belts or helmets) by image processing, and determine whether there is a person working near the input door 24 without wearing safety equipment. If it is determined that there is a person working near the input door 24 without wearing safety equipment, the instruction unit 11f, which will be described later, may issue an alarm or send a control signal to the input door control device 20 to prevent the input door 24 from opening (if closed) or to close it (if open).

[0072] The fallen person determination unit 11e determines whether or not a person has fallen from the platform 21 into the garbage pit 3, based on a combination of the analysis results of the first image data by the first image analysis unit 11d1 and the analysis results of the second image data by the second image analysis unit 11d2. Generally, when detecting a person who has fallen from image data using a machine learning model, the detection accuracy of the machine learning model varies, and it is impossible to detect a person who has fallen with 100% accuracy. However, in this embodiment, the analysis results of the first image data obtained by imaging the inside of the garbage pit 3 and the analysis results of the second image data obtained by imaging the inside of the platform 21 can be combined to improve the accuracy of detecting a person who has fallen.

[0073] As an example, the fallen person determination unit 11e first checks the analysis result of the second image data, and As shown in FIG. 4A, if a person is detected entering a predetermined area near the input door 24 at a first time (i.e., if a person who has fallen is provisionally detected in the analysis results of the second image data), the analysis results of the first image data at the first time may be checked, and if a person is detected in the garbage pit 3 as shown in FIG. 4B (i.e., if a person who has fallen is also provisionally detected in the analysis results of the first image data), a final determination may be made that a person has fallen. This makes it possible to accurately detect a person who has fallen.

[0074] As shown in Fig. 4A, when a person is detected entering a predetermined area near the input door 25 (in the illustrated example, input door B) at a first time, the fallen person determination unit 11e may check the analysis results of an image of the area in the garbage pit 3 corresponding to the position of the input door (i.e., input door B) in the first image data at the first time (the area surrounded by a dashed line marked with symbol B1 in Fig. 4B) as shown in Fig. 4B. This allows for more accurate and quicker detection of a fallen person by checking the analysis results of the second image data obtained by imaging the inside of the platform 21 and checking the analysis results of the image of area B1 in the garbage pit 3 corresponding to the position of input door B, which is likely to have been the location of the person who fell, rather than checking the analysis results of the entire first image data obtained by imaging the inside of the garbage pit 3.

[0075] Furthermore, if no person is detected in the garbage pit 3 when checking the analysis result of the first image data at the first time, the fallen person determination unit 11e may compare the first image data at the first time with the first image data at a second time a predetermined time after the first time (for example, five minutes) to extract the difference therebetween, and determine that a person has fallen if the difference between the first image data at the first time and the first image data at the second time exceeds a predetermined threshold, or determine that no person has fallen if there is no difference. The reason for this is as follows: For example, if waste falls over the person who has fallen in the garbage pit 3, the analysis result of the first image data will indicate that no person has been detected in the garbage pit 3 (because waste is over the person), but because the analysis result of the second image data has already provisionally detected the person who has fallen, the fallen person determination unit 11e further compares the first image data at the first time with the first image data at the second time to extract the difference therebetween. Then, if the difference between the first image data at the first time and the first image data at the second time exceeds a predetermined threshold, it is considered that a pile of waste has collapsed in the garbage pit 3, and it is possible that the person who fell is covered by waste from the collapsed pile and cannot be seen, so it is determined that a person has fallen. This makes it possible to automatically detect a person who has fallen in the garbage pit 3, even if waste has fallen over them.

[0076] As a variation, the fallen person determination unit 11e may first check the analysis results of the first image data, and if a person is detected in the garbage pit 3 at the first time as shown in Fig. 6A (i.e., if a person who has fallen is tentatively detected in the analysis results of the first image data), check the analysis results of the second image data up to a predetermined time before the first time (for example, 5 minutes before), and make a final determination that a person has fallen if a person has fallen out of the image frame in a predetermined area near the drop-off door 24 as shown in Fig. 6B (i.e., if a person who has fallen is also tentatively detected in the analysis results of the second image data). This makes it possible to accurately detect a person who has fallen.

[0077] As shown in Fig. 6A, when a person is detected in the garbage pit 3 at the first time, the fallen person determination unit 11e may check the analysis results of an image of the input door (in the illustrated example, input door B) at a position corresponding to the area where the person was detected in the garbage pit 3 (the area surrounded by the dashed line marked with symbol B2 in Fig. 6A) in the second image data up to a predetermined time before the first time (for example, 5 minutes before), as shown in Fig. 6B. Even in the case of a large platform 21 with multiple doors 24, by checking the analysis results of the first image data taken inside the garbage pit 3 and, if a person is detected inside the garbage pit 3, rather than checking the analysis results of the entire second image data taken inside the platform 21, it is possible to detect the person who has fallen more accurately and in a shorter time by checking the analysis results of the image of the input door B, which is located in a position where it is highly likely that the person may have fallen.

[0078] Furthermore, the fallen person determination unit 11e may determine that no person has fallen if, when checking the analysis results of the second image data up to a predetermined time before the first time (for example, 5 minutes before), a person has framed out of the image outside a predetermined area near the drop-in door 24. This makes it possible to prevent an erroneous determination that a person has fallen when a person has framed out of the image in the second image data taken inside the platform 21 for a reason unrelated to falling into the garbage pit 3 (for example, when the person has temporarily hidden behind the transport vehicle 22), and improves the accuracy of detecting a person who has fallen.

[0079] Furthermore, the fallen person determination unit 11e may determine that no person has fallen if, when checking the analysis results of the second image data up to a predetermined time before the first time (for example, 5 minutes before), no person is detected entering a predetermined area near the drop-off door 24. As a result, even if, for example, waste of approximately the same size as a person is mistakenly detected as a person in the first image data capturing the inside of the garbage pit 3 (i.e., a person who has fallen is mistakenly detected in the analysis results of the first image data), if no person is detected entering the vicinity of the drop-off door 24 when checking the analysis results of the second image data capturing the inside of the platform 21, it is possible to prevent an erroneous determination that a person has fallen, and to improve the accuracy of the detection of a person who has fallen.

[0080] The instruction unit 11f checks the determination result by the faller determination unit 11e, and when the faller determination unit 11e determines that a person has fallen, (1) Issue an alarm, (2) Sending a control signal to the crane control device 30 to stop the crane 5 that mixes or transports the waste stored in the garbage pit 3; (3) Sending a control signal to the input door control device 20 to close the input door 24 separating the garbage pit 3 and the platform 21; (4) Sending a control signal to the crane control device 30 to operate the crane 5 and rescue the fallen person; (5) Sending a control signal to a rescue equipment control device (not shown) to operate rescue equipment (not shown) installed in the garbage pit 3 and rescue the person who fell. As a result, even if someone falls while the crane 5 is automatically operating (i.e., when the crane operator is not present), the person can be rescued quickly, thereby improving the safety of the facility.

[0081] (First example of a method for detecting people falling) Next, a first example of a method for detecting a fallen person using the fallen person detection system 10 configured as described above will be described. Fig. 3 is a flowchart showing the first example of the method for detecting a fallen person.

[0082] 3, first, the first image data acquisition unit 11a1 acquires first image data from the first camera 6 capturing an image of the inside of the garbage pit 3 (step S10). The acquired first image data 12b1 is stored in the storage unit 12.

[0083] Next, the second image data acquisition unit 11a2 acquires second image data from the second camera 23 that captures images inside the platform 21 (step S11). The acquired first image data 12b2 is stored in the storage unit 12. Note that the order of steps S10 and S11 is not specified. Either one can be done first, or they can be done simultaneously.

[0084] Next, the second image analysis unit 11d2 performs image analysis on the second image data acquired by the second image data acquisition unit 11a2 to detect people on the platform 21 and track the movement of the detected people (step S12).

[0085] The faller determination unit 11e checks the analysis result of the second image data by the second image analysis unit 11d2 (step S13).

[0086] If no person is detected entering a predetermined area near the input door 24 at the first time (i.e., if a person who has fallen is not provisionally detected in the analysis results of the second image data) (step S13: NO), the person who has fallen determination unit 11e determines that no person has fallen (step S19).

[0087] On the other hand, as shown in Figure 4A, if a person is detected entering a predetermined area near the drop-in door 24 at the first time (i.e., if a person who has fallen is provisionally detected in the analysis results of the second image data) (step S13: YES), the first image analysis unit 11d1 performs image analysis of the first image data at the first time acquired by the first image data acquisition unit 11a1 to detect a person in the garbage pit 3 (step S14).

[0088] Then, the fallen person determination unit 11e confirms the analysis result of the first image data at the first time by the first image analysis unit 11d1 (step S15). Here, when a person's entry into a predetermined area near the drop door 25 (drop door B in the illustrated example) is detected at the first time as shown in Fig. 4A, the fallen person determination unit 11e may confirm the analysis result of an image captured of the area in the garbage pit 3 corresponding to the position of the drop door (i.e., drop door B) in the first image data at the first time (the area surrounded by a dashed line marked with symbol B1 in Fig. 4B), as shown in Fig. 4B.

[0089] As shown in Figure 4B, if a person is detected in the garbage pit 3 (i.e., if a person who has fallen is also provisionally detected in the analysis results of the first image data) (step 15: YES), the person who has fallen judgment unit 11e judges that a person has fallen (step S17).

[0090] On the other hand, if no person is detected in the garbage pit 3 (i.e., if a person who fell is not provisionally detected in the analysis results of the first image data) (step 15: NO), the fallen person determination unit 11e extracts the difference between the first image data at the first time and the first image data at the second time after a predetermined time has elapsed from the first time (for example, 5 minutes later), and compares the extracted difference with a predetermined threshold value (step S16).

[0091] If the difference between the first image data at the first time and the first image data at the second time exceeds a predetermined threshold (step S16: YES), it is thought that a pile of waste has collapsed within the garbage pit 3, and it is thought that the person who fell may have been covered by waste from the collapsed pile, making the person who fell invisible, so the fallen person determination unit 11e determines that a person has fallen (step S17).

[0092] On the other hand, if the difference between the first image data at the first time and the first image data at the second time does not exceed a predetermined threshold (step S16: NO), the faller determination unit 11e determines that no person has fallen (step S19).

[0093] Then, when the fallen person determination unit 11e determines that a person has fallen (after step S17), the instruction unit 11f issues an alarm to notify other workers, and A control signal is sent to the crane control device 30 to stop the crane 5 (step S18).

[0094] In step S18, the instruction unit 11f may send a control signal to the input door control device 20 to close the input door 24 separating the garbage pit 3 and the platform 21, in order to prevent waste from being thrown onto the person who has fallen and making rescue difficult. Furthermore, instead of sending a control signal to the crane control device 30 to stop the crane 5, the instruction unit 11f may send a control signal to the crane control device 30 to operate the crane 5 to rescue the person who has fallen. Furthermore, the instruction unit 11f may send a control signal to the rescue equipment control device (not shown) to operate rescue equipment (not shown) provided in the garbage pit 3 to rescue the person who has fallen. This enables the person who has fallen to be quickly rescued, even if the crane 5 is operating automatically (i.e., when the crane operator is not present).

[0095] (Second example of a method for detecting people falling) Next, we will explain a second example of the method for detecting a fallen person by the fallen person detection system 10. Fig. 5 is a flowchart showing the second example of the method for detecting a fallen person.

[0096] 5, first, the first image data acquisition unit 11a1 acquires first image data from the first camera 6 capturing an image of the inside of the garbage pit 3 (step S20). The acquired first image data 12b1 is stored in the storage unit 12.

[0097] Next, the second image data acquisition unit 11a2 acquires the second image data from the second camera 23 that captures the image inside the platform 21 (step S21). The acquired first image data 12b2 is stored in the storage unit 12. Note that the order of step S20 and step S21 does not matter, and they may be performed simultaneously.

[0098] Next, the first image analysis unit 11d1 performs image analysis on the first image data acquired by the first image data acquisition unit 11a1 to detect a person in the garbage pit 3 (step S22).

[0099] The fallen person determination unit 11e checks the analysis result of the first image data by the first image analysis unit 11d1 (step S23).

[0100] If no person is detected in the garbage pit 3 at the first time (i.e., if a person who has fallen is not tentatively detected in the analysis results of the first image data) (step S23: NO), the faller determination unit 11e determines that no person has fallen (step S29).

[0101] On the other hand, as shown in Figure 6A, if a person is detected in the garbage pit 3 at the first time (i.e., if a person who has fallen is provisionally detected in the analysis results of the first image data) (step S23: YES), the second image analysis unit 11d2 performs image analysis of the second image data acquired by the second image data acquisition unit 11a2 up to a predetermined time before the first time (for example, 5 minutes before), to detect the person on the platform 21 and track the movement of the detected person (step S24).

[0102] Then, the fallen person determination unit 11e checks whether or not a person has been detected in a predetermined area near the throw-in door 24 in the analysis result of the second image data by the second image analysis unit 11d2 (step S25). Here, when a person is detected in the garbage pit 3 at the first time as shown in FIG. 6A, the fallen person determination unit 11e analyzes an image of the throw-in door (in the illustrated example, throw-in door B) at a position corresponding to the area where the person was detected in the garbage pit 3 (the area surrounded by the dashed line marked with symbol B2 in FIG. 6A) in the second image data up to a predetermined time before the first time (for example, 5 minutes before), as shown in FIG. 6B. You may check the analysis results.

[0103] If no person is detected entering a predetermined area near the deposit door 24 (i.e., if a person who has fallen is not provisionally detected in the analysis results of the second image data) (step S25: NO), the faller determination unit 11e determines that no person has fallen (step S29).

[0104] On the other hand, as shown in Figure 6A, if the analysis results of the second image data detect that a person has entered a predetermined area near the deposit door 24 (step S25: YES), the fallen person determination unit 11e checks whether the analysis results of the second image data show that a person has framed out of the image within the predetermined area near the deposit door 24 (step S26).

[0105] If a person is out of frame in the image within a predetermined area near the deposit door 24 (i.e., if a person who has fallen is also provisionally detected in the analysis results of the second image data) (step S26: YES), the person who has fallen determination unit 11e determines that a person has fallen (step S27).

[0106] On the other hand, if no person has fallen out of the image within the predetermined area near the deposit door 24 (step S26: NO), the fallen person determination unit 11e determines that no person has fallen (step S29).

[0107] Then, if the fallen person determination unit 11e determines that a person has fallen (after step S27), the instruction unit 11f issues an alarm to notify other workers and sends a control signal to the crane control device 30 to stop the crane 5 (step S28).

[0108] In step S28, the instruction unit 11f may send a control signal to the input door control device 20 to close the input door 24 separating the garbage pit 3 and the platform 21, in order to prevent waste from being thrown onto the person who has fallen and making rescue difficult. Furthermore, instead of sending a control signal to the crane control device 30 to stop the crane 5, the instruction unit 11f may send a control signal to the crane control device 30 to operate the crane 5 to rescue the person who has fallen. Furthermore, the instruction unit 11f may send a control signal to the rescue equipment control device (not shown) to operate rescue equipment (not shown) provided in the garbage pit 3 to rescue the person who has fallen. This enables the person who has fallen to be quickly rescued, even if the crane 5 is operating automatically (i.e., when the crane operator is not present).

[0109] However, in waste incineration facilities, the platform 21, which is the side where the fall occurred, and the garbage pit 3, which is the side where the fall occurred, are separated by a garbage door 24, so it is difficult to install an imaging device such as a camera in a location where it can simultaneously photograph both the side where the fall occurred and the side where the fall occurred, as is the case with the fallen person detection systems used in the railway industry.

[0110] In contrast, according to this embodiment, even without installing a camera in a location where it can simultaneously photograph both the side where the fall occurred and the side where the fall resulted, first image data is obtained from a first camera 6 that images the inside of the garbage pit 3, which is the side where the fall resulted, and second image data is obtained from a second camera 23 that images the inside of the platform 21, which is the side where the fall occurred.By combining the analysis results of the first image data that imaged the inside of the garbage pit 3 with the analysis results of the second image data that imaged the inside of the platform 21, it is possible to automatically detect a person who falls from the platform 21 into the garbage pit 3.

[0111] Generally, when using a machine learning model to detect a person who has fallen from image data, the detection accuracy of the machine learning model varies, and it is impossible to detect a person who has fallen with 100% accuracy.However, according to this embodiment, the analysis results of the first image data taken inside the garbage pit 3 and the analysis results of the second image data taken inside the platform 21 can be combined to improve the accuracy of detecting a person who has fallen.

[0112] Furthermore, according to this embodiment, even in existing facilities, the system can be operated simply by installing the first camera 6 that takes images inside the garbage pit 3 and the second camera 23 that takes images inside the platform 21. Therefore, there is no need to make major modifications to existing facilities when introducing the system, and it is possible to increase the safety of existing facilities at low cost.

[0113] In the above-described embodiment, the waste disposal facility 100 in which the fallen person detection system 10 is installed is a facility in which the garbage identification system 40 is installed, and is configured so that the crane 5 is automatically operated based on the identification results of the garbage identification system 40, but this is not limited to this, and the waste disposal facility 100 in which the fallen person detection system 10 is installed may also be a facility in which the garbage identification system 40 is not installed.

[0114] Furthermore, in the above-described embodiment, the fallen person detection system 10 was configured to determine whether or not a person has fallen based on a combination of the analysis results of the first image data taken inside the garbage pit 3 and the analysis results of the second image data taken inside the platform 21; however, if sufficient detection accuracy can be obtained from the analysis results of the first image data alone, the presence or absence of a person who has fallen may be determined based only on the analysis results of the first image data, or if sufficient detection accuracy can be obtained from the analysis results of the second image data alone, the presence or absence of a person who has fallen may be determined based only on the analysis results of the second image data.

[0115] Although the embodiments and modifications of the present invention have been described above by way of example, the scope of the present invention is not limited to these, and modifications and variations can be made according to the purpose within the scope of the claims. Furthermore, the embodiments and modifications can be combined as appropriate within the scope of the processing content.

[0116] In addition, the fallen person detection system 10 in this embodiment can be configured by one or more computers, but the program for realizing the fallen person detection system 10 on one or more computers and the recording medium on which the program is non-temporarily recorded are also protected by this case. [Explanation of symbols]

[0117] 1. Incinerator 2 Combustion equipment 3. Garbage Pit 4 Hopper 5 Crane 6. Camera 1 10 Fall detection system 11 Control section 11a1 first image data acquisition unit 11a2 second image data acquisition unit 11b1 First teacher data generation unit 11b2 Second teacher data generation unit 11c1 First Model Construction Department 11c2 Second Model Construction Department 11d1 First Image Analysis Unit 11d2 2nd image analysis section 11e Faller Judgment Section 11f Instruction section 12 Storage section 12a1 First detection algorithm 12a2 Second detection algorithm 12b1 First image data 12b2 Second image data 12c1 First training data 12c2 Second training data 13 Communications Department 20. Input door control device 21 Platform 22 Transport vehicle 23 Second Camera 24 Deposit door 30 Crane control device 40 Waste Identification System 100 Waste treatment facilities

Claims

1. a first image data acquisition unit that acquires first image data from a first camera that captures an image of the inside of the storage facility in which the object to be processed is stored; a second image data acquisition unit that acquires second image data from a second camera that captures an image of the inside of a platform adjacent to the storage facility; a first image analysis unit that performs image analysis on the first image data to detect a person in the storage facility; a second image analysis unit that performs image analysis of the second image data to detect a person on the platform and track the movement of the detected person; a faller determination unit that determines whether or not a person has fallen from the platform into the storage facility based on a combination of the analysis results of the first image data and the analysis results of the second image data; A fall detection system comprising:

2. The fallen person detection system of claim 1, wherein the fallen person determination unit checks the analysis results of the second image data, and if a person is detected entering a predetermined area near the input door separating the storage facility and the platform at a first time, checks the analysis results of the first image data at the first time, and if a person is detected within the storage facility, determines that a person has fallen.

3. The fallen person detection system described in claim 2, wherein when a person is detected entering a predetermined area near the input door at a first time, the fallen person determination unit checks the analysis results of an image captured of the area within the storage facility corresponding to the position of the input door in the first image data at the first time.

4. The fallen person detection system of claim 2 or 3, wherein, when no person is detected in the storage facility when checking the analysis results of the first image data at the first time, the fallen person determination unit extracts the difference between the first image data at the first time and the first image data at a second time a predetermined time after the first time, and determines that a person has fallen if the extracted difference exceeds a predetermined threshold, and determines that no person has fallen if there is no difference.

5. The fallen person detection system of claim 1, wherein the fallen person determination unit checks the analysis results of the first image data, and if a person is detected in the storage facility at the first time, checks the analysis results of the second image data up to a predetermined time before the first time, and determines that a person has fallen if a person is framed out of the image within a predetermined area near the loading door separating the storage facility and the platform.

6. The fallen person detection system described in claim 5, wherein when a person is detected within the storage facility at a first time, the fallen person determination unit checks the analysis results of an image of the input door at a position corresponding to the area in the storage facility where the person was detected, in the second image data up to a predetermined time before the first time.

7. The fallen person detection system of claim 5 or 6, wherein the fallen person determination unit determines that no person has fallen if, when checking the analysis results of the second image data up to a predetermined time before the first time, a person is framed out of the image outside a predetermined area near the input door.

8. A fallen person detection system as described in any of claims 5 to 7, wherein the fallen person determination unit determines that no person has fallen if, when checking the analysis results of the second image data up to a predetermined time before the first time, no person is detected entering a predetermined area near the input door.

9. When the faller determination unit determines that a person has fallen, (1) Issue an alarm; (2) sending a control signal to the crane control device to stop the crane that mixes or transports the material stored in the storage facility; (3) Sending a control signal to the loading door control device to close the loading door separating the storage facility and the platform; (4) Sending a control signal to the crane control device to operate the crane and rescue the fallen victim; (5) Sending a control signal to a rescue equipment control device to operate rescue equipment provided in the storage facility to rescue the person who fell; 9. The fallen person detection system according to claim 1, further comprising an instruction unit that performs at least one of the above processes.

10. A fallen person detection system as described in any one of claims 1 to 9, wherein the first image analysis unit uses a first detection algorithm constructed by machine learning first training data generated by assigning artificial labels as information to areas in past image data within the storage facility where people or dummy dolls resembling people are present, and detects people within the storage facility using new image data within the storage facility as input.

11. A fallen person detection system as described in any one of claims 1 to 10, wherein the second image analysis unit uses a second detection algorithm constructed by machine learning second training data generated by assigning artificial labels as information to areas of past image data within the platform where people or dummy dolls resembling people exist, and detects people within the platform using new image data within the platform as input.

12. The second training data is generated by assigning artificial labels as information to areas of past image data within the platform where people or dummy dolls resembling people are present, and by assigning another artificial label as information to areas where incoming vehicles are present, as described in claim 11.

13. the first detection algorithm comprises one or more of a maximum likelihood classification method, a Boltzmann machine, a neural network, a support vector machine, a Bayesian network, a sparse regression, a decision tree, a statistical inference using random forests, reinforcement learning, and deep learning; The fall detection system of claim 11 or 12, which cites claim 10, wherein the second detection algorithm includes one or more of maximum likelihood classification, Boltzmann machines, neural networks, support vector machines, Bayesian networks, sparse regression, decision trees, statistical inference using random forests, reinforcement learning, and deep learning.

14. A fallen person detection system as described in any one of claims 1 to 13, wherein the algorithm used by the second image analysis unit to track the movement path of a person includes one or more of optical flow, background subtraction, Kalman filter, particle filter, and deep learning.

15. the first camera includes one or more of an RGB camera, a near-infrared camera, a 3D camera, or an RGB-D camera; The fallen person detection system according to any one of claims 1 to 14, wherein the second camera includes one or more of an RGB camera, a near-infrared camera, a 3D camera, or an RGB-D camera.

16. A waste disposal facility equipped with a fallen person detection system according to any one of claims 1 to 15.

17. acquiring first image data from a first camera that captures an image of the inside of a storage facility in which the object to be processed is stored; acquiring second image data from a second camera imaging an interior of a platform adjacent to the storage facility; performing image analysis on the first image data to detect a person in the storage facility; performing image analysis on the second image data to detect a person on the platform and track the movement of the detected person; A step of determining whether or not a person has fallen from the platform into the storage facility based on a combination of the analysis results of the first image data and the analysis results of the second image data; A method for detecting a person falling, comprising:

18. On the computer, acquiring first image data from a first camera that captures an image of the inside of a storage facility in which the object to be processed is stored; acquiring second image data from a second camera imaging an interior of a platform adjacent to the storage facility; performing image analysis on the first image data to detect a person in the storage facility; performing image analysis on the second image data to detect a person on the platform and track the movement of the detected person; A step of determining whether or not a person has fallen from the platform into the storage facility based on a combination of the analysis results of the first image data and the analysis results of the second image data; A fall detection program that executes the following.

Images