Monitoring system

Abstract

To provide a monitoring system capable of monitoring a work situation relating to a cargo handling work while suppressing an increase in cost.SOLUTION: A monitoring system includes: a fork lift 1 that is a cargo handling vehicle on which a cargo handling worker M can ride and which performs cargo handling work by an operation given by the cargo handling worker M; and drones 2 that are multiple unmanned flying objects which can autonomously fly and to which cameras 25 are respectively attached. The multiple drones 2 fly so as to move to a site where the cargo handling work is carried out and to image, by the respective cameras 25, a work situation relating to the cargo handling work.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a monitoring system capable of monitoring the status of loading and unloading work. [Background technology]

[0002] It is generally known that in a work environment such as a warehouse, cameras are installed in multiple locations to monitor the work situation for the purpose of checking events that occur during loading and unloading work (see, for example, Patent Document 1).

[0003] Patent Document 1 describes placing cameras that monitor the area around the forklift in predetermined locations, such as near the ceiling of the warehouse or at corners of the roadway, as well as on the forklift, for the purpose of determining the possibility of a collision between the forklift and a person.

[0004] However, in a configuration in which cameras are installed in predetermined locations, the larger the work environment, the more cameras are required, which increases costs.Also, in a configuration in which cameras are attached to forklifts, the more forklifts there are, the higher the introduction costs become. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2023-714 A Summary of the Invention [Problem to be solved by the invention]

[0006] The present invention has been made in consideration of the above circumstances, and an object of the present invention is to provide a monitoring system that can monitor the work status related to loading and unloading work while suppressing an increase in costs. [Means for solving the problem]

[0007] In order to solve the above problems, the monitoring system of the present invention comprises a loading vehicle that can be boarded by a loading worker and is operated by the loading worker to perform loading and unloading operations, and a plurality of unmanned aerial vehicles that are capable of autonomous flight and are equipped with cameras, and the multiple unmanned aerial vehicles fly to a location where the loading and unloading operations are performed and fly so as to capture images of the work situation related to the loading and unloading operations with the cameras.

[0008] It is also preferable that the system further includes a management device capable of communicating with multiple unmanned aerial vehicles, the management device transmitting shooting location information relating to the shooting location to the unmanned aerial vehicle, and the unmanned aerial vehicle flying to the shooting location and waiting in accordance with the shooting location information.

[0009] It is also preferable that the system further includes a management device capable of communicating with multiple unmanned aerial vehicles, the management device transmitting vehicle identification information capable of identifying the loading vehicle to the unmanned aerial vehicle, and the unmanned aerial vehicle flying to follow the loading vehicle in accordance with the vehicle identification information.

[0010] It is also preferable that the camera be configured to be switchable between a plurality of shooting modes, and configured such that the subject to be shot or the shooting angle of view is changed by switching the shooting mode.

[0011] It is also preferable that the camera is configured to be switchable between a loading location photography mode for photographing the location where the loading and unloading work is performed, a rack photography mode for photographing a rack located in front of the loading and unloading vehicle, and a worker photography mode for photographing the loading and unloading worker on board the loading and unloading vehicle. Effect of the Invention

[0012] According to the present invention, it is possible to provide a monitoring system capable of monitoring the work status relating to loading and unloading work while suppressing an increase in cost. [Brief description of the drawings]

[0013] [Figure 1]1 is a schematic diagram of a monitoring system according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a block diagram showing a schematic configuration of a monitoring system according to the embodiment. [Diagram 3] 11 is a flowchart showing a monitoring procedure according to the embodiment. [Figure 4] FIG. 13 is a block diagram showing a schematic configuration of a monitoring system according to a first modified example. [Diagram 5] FIG. 13 is a block diagram showing a schematic configuration of a monitoring system according to a second modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] A monitoring system according to an embodiment of the present invention will be described with reference to the drawings. As shown in Fig. 1, the monitoring system according to this embodiment includes a forklift 1, multiple drones 2, and a management device 3. The monitoring system monitors the status of loading and unloading work performed using the forklift 1 by using the drone 2.

[0015] The forklift 1 is a manned vehicle on which a cargo worker M can ride, and is operated by the cargo worker M to perform cargo handling operations including the loading and unloading operations of cargo N. The loading and unloading operations include the operation of placing the cargo N on a rack R, and the unloading operations include the operation of removing the cargo N from the rack R.

[0016] The drone 2 is an unmanned flying vehicle capable of autonomous flight and equipped with a camera 25, and is a multicopter with excellent attitude stability. In order to monitor the work status related to the loading and unloading work, the drone 2 flies to the location where the loading and unloading work is performed and captures the work status with the camera 25. The drone 2 transmits video data related to the video captured by the camera 25 to the management device 3.

[0017] The management device 3 is a management server that manages various information and is configured to be able to communicate with the forklift 1 and the multiple drones 2. The management device 3 transmits loading / unloading work information related to the loading / unloading work to be performed by the loading / unloading worker M to the forklift 1, and transmits shooting location information and shooting mode information, which will be described later, to the drones 2.

[0018] With reference to FIG. 2, the schematic configurations of the forklift 1, the drone 2, and the management device 3 will be described. As shown in FIG. 2, the forklift 1 includes a display unit 11 and a position information acquisition unit 12.

[0019] The display unit 11 is configured with a display that presents information to the cargo worker M, and displays the cargo handling work information received from the management device 3. The cargo worker M operates the forklift 1 by referring to the cargo handling work information displayed on the display unit 11.

[0020] The position information acquiring unit 12 acquires position information of the forklift 1, which is the vehicle itself. The position information acquiring unit 12 is configured, for example, with a sensor that detects the surrounding environment of the forklift 1, or a receiver that receives a signal from a positioning satellite. The position information acquiring unit 12 transmits the position information of the forklift 1 to the management device 3.

[0021] As shown in FIG. 2, the drone 2 includes a rotor 21, a position information acquisition unit 22, an object recognition unit 23, a flight control unit 24, a camera 25, and a video transmission unit 26.

[0022] A plurality of rotors 21 are provided concentrically at intervals from one another, and generate lift for flying the drone 2. The flight speed, flight direction, and attitude of the drone 2 are configured to be changeable by controlling the rotors 21.

[0023] The position information acquisition unit 22 acquires position information of the drone 2, which is the drone itself. The position information acquisition unit 22 is configured, for example, by a laser sensor that acquires data for SLAM (Simultaneous Localization and Mapping) using LiDAR (Light Detection and Ranging), a motion sensor that acquires data for odometry, a receiver that receives signals from positioning satellites, or a combination of these.

[0024] The object recognition unit 23 recognizes objects present around the drone 2. The object recognition unit 23 is configured, for example, with a laser sensor that acquires data for SLAM using LiDAR, or a camera that captures images of the surroundings.

[0025] The flight control unit 24 is composed of a controller that controls the flight of the drone 2. Based on the shooting location information received from the management device 3 and the position information acquired by the position information acquisition unit 22, the flight control unit 24 controls the rotor 21 so that the drone 2 flies to the shooting location where the loading and unloading work is performed. In addition, based on the position information of an obstacle, which is an object recognized by the object recognition unit 23, the flight control unit 24 controls the rotor 21 so that the drone 2 continues flying while avoiding the obstacle. In this way, the drone 2 is configured to fly to the shooting location and wait according to the shooting location information.

[0026] The camera 25 captures images of the work status related to the loading and unloading work (i.e., the operating status of the forklift 1, the status in front of the forklift 1, and the status of the loading and unloading worker M operating the forklift 1) based on the shooting mode information received from the management device 3. In this embodiment, the camera 25 is configured to be switchable between a plurality of shooting modes, namely, a loading and unloading place shooting mode, a rack shooting mode, and a worker shooting mode, and is configured to change the shooting subject or the shooting angle of view by switching between these three shooting modes.

[0027] The loading and unloading place photography mode is a photography mode for photographing a place where loading and unloading work is performed in order to photograph the operating status of the forklift 1. That is, the camera 25 in the loading and unloading place photography mode photographs the space in which the forklift 1 exists, and sets a wide angle of view including the forklift 1 and the surrounding environment of the forklift 1 as the photography angle.

[0028] The rack photography mode is a photography mode in which the rack R located in front of the forklift 1 is photographed in order to photograph the situation in front of the forklift 1. In the rack photography mode, the camera 25 photographs the space located in front of the forklift 1, and the photography angle of view is set to include the entire or part of the rack R.

[0029] The worker photography mode is a photography mode in which a loading / unloading worker M riding on the forklift 1 is photographed in order to photograph the condition of the loading / unloading worker M. In the worker photography mode, the camera 25 photographs the space in which the loading / unloading worker M exists, and sets a narrow field of view that includes the whole or part (e.g., face or hands) of the loading / unloading worker M. In other words, the field of view in the worker photography mode is set to be narrower than the field of view in the loading / unloading place photography mode.

[0030] Video transmission unit 26 is configured with a wireless communication module, and transmits the video captured by camera 25. Specifically, video transmission unit 26 wirelessly transmits data related to the video captured by camera 25 to management device 3. In this way, the video captured by camera 25 is stored in storage unit 31, which will be described later.

[0031] 2, the management device 3 includes a storage unit 31, a loading / unloading operation instruction unit 32, an information acquisition unit 33, an image capture planning unit 34, and an image capture instruction unit 35. Each of the units 31 to 35 constituting the management device 3 is configured by hardware that operates according to a predetermined program.

[0032] The storage unit 31 stores various types of information and data. Specifically, the storage unit 31 stores, for example, loading / unloading work information related to the loading / unloading work performed by the forklift 1, position information of the forklift 1, and position information of the drone 2. The loading / unloading work information is information input using an input device (not shown) connected to the management device 3, and the position information of the forklift 1 and the drone 2 is information acquired through communication with the forklift 1 and the drone 2. The storage unit 31 also stores video data received from the drone 2.

[0033] The loading / unloading work instructing unit 32 is configured to be able to communicate with the forklift 1, and instructs the loading / unloading worker M on the loading / unloading work to be performed by the forklift 1. Specifically, the loading / unloading work instructing unit 32 instructs the forklift 1 to perform the loading / unloading work by transmitting loading / unloading work information stored in the memory unit 31 to the forklift 1.

[0034] The information acquisition unit 33 acquires cargo handling work information as information for determining the shooting location and shooting mode from the storage unit 31. The cargo handling work information includes information on the location where the cargo handling work will be performed and the scheduled time.

[0035] The photography planning unit 34 determines the photography location and photography mode to be assigned to each of the multiple drones 2 based on the cargo handling work information acquired by the information acquisition unit 33. Specifically, the photography planning unit 34 determines the photography location to be photographed by the camera 25 from the multiple locations where the cargo handling work is performed based on the cargo handling work information. At this time, the photography planning unit 34 selects, for example, a location with a high priority as the photography location from multiple locations to which a priority has been given in advance and which have not been selected as a photography location. After the photography location is determined, the photography planning unit 34 determines one photography mode according to the photography location. At this time, the photography planning unit 34 selects, for example, the rack photography mode or the worker photography mode at a location where the forklift 1 performs a warehousing work, and selects the loading location photography mode or the worker photography mode at a location where the forklift 1 performs a warehousing work.

[0036] The photography instruction unit 35 is configured to be able to communicate with the multiple drones 2, and instructs the multiple drones 2 to photograph the work situation. Specifically, the photography instruction unit 35 instructs the drones 2 to photograph by transmitting, to the drones 2, photography location information related to the photography location and photography mode information related to the photography mode determined by the photography planning unit 34.

[0037] The flow of monitoring the work status related to cargo handling work will be described with reference to Fig. 3. It is assumed that the above-mentioned various information is stored in advance in the storage unit 31.

[0038] First, the information acquisition unit 33 acquires cargo handling work information from the storage unit 31 (step S1), and the photography planning unit 34 determines a photography location and a photography mode based on the cargo handling work information acquired in step S1 (step S2).

[0039] Then, the photography instruction unit 35 transmits the photography location information related to the photography location and the photography mode information related to the photography mode determined in step S2 to the multiple drones 2 (step S3), and the multiple drones 2 that have received the photography location information and the photography mode information photograph the work situation related to the loading and unloading work performed by the forklift 1 (step S4). That is, in step S4, each of the multiple drones 2 photographs with the camera 25 the place where the loading and unloading work is performed, the rack R located in front of the forklift 1, or the loading and unloading worker M aboard the forklift 1, at the photography location and in the photography mode determined in step S2.

[0040] In this manner, each of the multiple drones 2, at the assigned shooting location and shooting mode, captures the work situation related to the loading and unloading work performed by the forklift 1. The captured video of the work situation is used, for example, as evidence of an incident (e.g., an accident) that occurred during the loading and unloading work, or as training data for predicting possible incidents.

[0041] The present embodiment provides the following effects. (1) Multiple drones 2 (unmanned aerial vehicles) fly to the location where loading and unloading work is performed, and capture the work status related to the loading and unloading work with the camera 25. With this configuration, the increase in the introduction cost of the forklift 1 can be suppressed compared to a configuration in which a camera is attached to the forklift 1 (loading and unloading vehicle). In addition, since the location where the work status is captured can be changed by moving the drone 2, costs can be reduced compared to a configuration in which cameras are installed in many predetermined locations. Therefore, the work status related to the loading and unloading work can be monitored while suppressing an increase in costs. In addition, by constantly monitoring the status of the loading and unloading worker M as the work status related to the loading and unloading work, the safety awareness of the loading and unloading worker M can be improved.

[0042] (2) The management device 3 transmits information about the shooting location to the drone 2, and the drone 2 flies to the shooting location in accordance with the shooting location information and waits there. With this configuration, the waiting locations of multiple drones 2 (i.e., shooting locations where the work situation is photographed by the camera 25) can be centrally controlled by the management device 3.

[0043] (3) Camera 25 is configured to be switchable between a plurality of shooting modes, and configured to change the shooting subject or the shooting angle of view by switching the shooting mode. With this configuration, the shooting manner can be changed by switching the shooting mode of camera 25.

[0044] (4) The camera 25 is configured to be switchable among a loading / unloading location photography mode, a rack photography mode, and a worker photography mode. With this configuration, by switching between these photography modes, it is possible to capture images of the operating status of the forklift 1, the status in front of the forklift 1, and the status of the loading / unloading worker M as the work status related to the loading / unloading work.

[0045] The present invention is not limited to the above embodiment, and the above configurations can be modified. For example, the following modifications can be made, or the following modifications can be combined to make the present invention.

[0046] As shown in Fig. 4, the drone 2 may include a photography planning unit 28. The configuration of the first modified example shown in Fig. 4 will be described below. Note that the description of the same configuration as the above embodiment will be omitted.

[0047] The drone 2 of the first modified example further includes an information acquisition unit 27 and a photography planning unit 28, and is configured to be capable of communicating with other drones 2 (i.e., capable of communicating between drones 2). The information acquisition unit 27 acquires cargo handling operation information from the storage unit 31 of the management device 3 as information for determining the photography location and photography mode.

[0048] The photography planning unit 28 determines the photography location and photography mode based on the cargo handling work information acquired by the information acquisition unit 27 and the photography location information received from the other drones 2. Specifically, the photography planning unit 28, for example, excludes locations that have been assigned a priority in advance and that have not been selected as photography locations by the other drones 2 from a plurality of locations where cargo handling work is performed, and selects a location with a high priority as the photography location. After determining the photography location, the photography planning unit 28 determines one photography mode according to the photography location, similar to the photography planning unit 34. The photography location information related to the determined photography location is transmitted to the other drones 2 as information for determining the photography location.

[0049] The flight control unit 24 of this modification controls the rotor 21 so that the drone 2 flies to the photography location based on the photography location information related to the photography location determined by the photography planning unit 28 and the position information acquired by the position information acquisition unit 22. Also, the camera 25 of this modification captures images of the work situation related to the loading and unloading work based on the photography mode information related to the photography mode information determined by the photography planning unit 28.

[0050] The first modified example provides the following effects. (5) Since the drone 2 is equipped with the shooting planning unit 28, the management device 3 does not need to determine the shooting location and shooting mode, thereby reducing the burden on the management device 3 in the surveillance system.

[0051] As shown in Fig. 5, the drone 2 may include a baggage identification unit 29 that identifies the baggage N photographed by the camera 25. The configuration of the second modified example shown in Fig. 5 will be described below. Note that the description of the same configuration as the above embodiment will be omitted.

[0052] The baggage identifying unit 29 acquires baggage identification information for identifying the baggage N from the memory unit 31 of the management device 3, and identifies the baggage N being handled by the forklift 1 or the baggage N that has been handled by the forklift 1 based on the baggage identification information from the video captured by the camera 25. Then, the baggage identifying unit 29 transmits information related to the identification result of the baggage N to the management device 3.

[0053] The loading / unloading work instruction unit 32 of this modified example is configured to be able to communicate with the drone 2, and receives information related to the identification result of the baggage N from the drone 2. The loading / unloading work instruction unit 32 determines whether the forklift 1 is performing the instructed loading / unloading work based on the identification result of the baggage N, and when it determines that the forklift 1 is not performing the instructed loading / unloading work, it transmits attention calling information to the forklift 1 to call the attention of the loading / unloading worker M. Specifically, the loading / unloading work instruction unit 32 determines that the forklift 1 is not performing the instructed loading / unloading work when, for example, the baggage N being transported by the forklift 1 is not the specified baggage N, or when the place where the forklift 1 has placed the baggage N is not the specified place.

[0054] The display unit 11 of this modified example displays the attention-calling information received from the management device 3. The cargo worker M refers to the attention-calling information displayed on the display unit 11 and operates the forklift 1 so as to correct the work mistake.

[0055] The second modified example provides the following effects. (6) The management device 3 is configured to determine whether the forklift 1 is performing the instructed loading and unloading work based on the image captured by the camera 25. With this configuration, for example, when the forklift 1 is not handling the specified cargo N, the attention of the loading and unloading worker M is called, thereby making it possible to reduce operational mistakes.

[0056] In the second modified example, the drone 2 may be configured to determine whether or not the forklift 1 is performing the instructed loading and unloading work based on the image captured by the camera 25. In this case, when the drone 2 determines that the forklift 1 is not performing the instructed loading and unloading work, it is preferable that the drone 2 transmits attention-calling information to the forklift 1 without going through the management device 3.

[0057] The management device 3 may include a baggage identification unit that identifies the baggage N based on the video data received from the drone 2. With this configuration, the effect of (6) above can be obtained with a configuration similar to that of the second modified example, and the drone 2 does not need to identify the baggage N, thereby reducing the burden on the drone 2.

[0058] The management device 3 may transmit vehicle designation information capable of identifying the forklift 1 to the drone 2, and the drone 2 may fly so as to follow the forklift 1 in accordance with the vehicle designation information. In this case, the object recognition unit 23 recognizes the forklift 1 to be followed, and the flight control unit 24 controls the rotor 21 so that the drone 2 moves following the forklift 1, based on the position information of the forklift 1 recognized by the object recognition unit 23. With this configuration, the drone 2 can photograph the work situation in the vicinity of the forklift 1, compared to a configuration in which the drone 2 does not follow the forklift 1.

[0059] The drone 2 may be equipped with a storage unit that stores data related to the video captured by the camera 25. This configuration eliminates the need to transmit video data from the drone 2 to the management device 3, and reduces the amount of communication between the management device 3 and the drone 2.

[0060] The method of determining the shooting location and the shooting mode may be changed as appropriate. For example, the shooting mode may be determined, and then a shooting location suitable for the shooting mode may be selected. Also, the management device 3 may be configured to determine the shooting location, and the drone 2 may be configured to determine the shooting mode.

[0061] The loading vehicle on which the loading worker M rides may be a loading vehicle other than the forklift 1. That is, the drone 2 may capture, for example, a transport cart, a towing vehicle, or the working situation related to the loading and unloading work performed by the towing vehicle with the camera 25. [Explanation of symbols]

[0062] 1. Forklift (cargo handling vehicle) 2. Drones (unmanned aerial vehicles) 3 Management device 25 Camera M. Loader R Rack

Claims

1. A loading vehicle that can be boarded by a loading worker and that is operated by the loading worker to perform loading and unloading work; A plurality of unmanned aerial vehicles capable of autonomous flight and equipped with cameras; The plurality of unmanned aerial vehicles fly to a location where the loading and unloading work is to be performed, and photograph the working conditions of the loading and unloading work with the camera. A monitoring system comprising:

2. Further comprising a management device capable of communicating with a plurality of the unmanned aerial vehicles; The management device transmits shooting location information relating to the shooting location to the unmanned aerial vehicle, The unmanned aerial vehicle flies to the photographing location according to the photographing location information and waits there.

2. The monitoring system according to claim 1 .

3. Further comprising a management device capable of communicating with a plurality of the unmanned aerial vehicles; The management device transmits vehicle identification information capable of identifying the loading vehicle to the unmanned aerial vehicle, The unmanned aerial vehicle flies so as to follow the loading vehicle in accordance with the vehicle identification information.

2. The monitoring system according to claim 1 .

4. The camera is configured to be switchable between a plurality of shooting modes, and configured to change a shooting subject or a shooting angle of view by switching the shooting mode.

4. The monitoring system according to claim 1, wherein the monitoring system comprises: a first detecting means for detecting a first signal;

5. The camera is configured to be switchable between a loading location photographing mode for photographing the location where the loading and unloading work is performed, a rack photographing mode for photographing a rack located in front of the loading and unloading vehicle, and a worker photographing mode for photographing the loading and unloading worker aboard the loading and unloading vehicle.

4. The monitoring system according to claim 1, wherein the monitoring system comprises: a first detecting means for detecting a first signal;

Images