Monitoring system

Abstract

To provide a monitoring system capable of monitoring a work situation relating to a cargo handling work while suppressing an increase in cost.SOLUTION: A monitoring system includes: a fork lift 1 that is a cargo handling vehicle on which a cargo handling worker M can ride and which performs cargo handling work by an operation given by the cargo handling worker M; and drones 2 that are multiple unmanned flying objects which can autonomously fly and to which cameras 25 are respectively attached. The multiple drones 2 fly so as to move to a site where the cargo handling work is carried out and to image, by the respective cameras 25, a work situation relating to the cargo handling work.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a monitoring system capable of monitoring work status related to cargo handling work.

Background Art

[0002] It is generally known that cameras for monitoring work status are installed in multiple locations for the purpose of, for example, confirming events that occurred during cargo handling work in a work environment such as a warehouse (see, for example, Patent Document 1).

[0003] Patent Document 1 describes that, for the purpose of determining the possibility of a collision between a forklift and a person, a camera for monitoring the surroundings of the forklift is arranged at predetermined locations such as near the ceiling of a warehouse or a corner of a travel path, and is also arranged on the forklift.

[0004] However, in a configuration in which cameras are installed at predetermined locations, the larger the work environment is, the more cameras are required, which causes a problem of increased cost. Further, in a configuration in which a camera is attached to a forklift, there is a problem that the introduction cost increases as the number of forklifts increases.

Prior Art Literature

Patent Literature

[0005]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0006] The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a monitoring system capable of monitoring work status related to cargo handling work while suppressing an increase in cost.

Means for Solving the Problem

[0007] To solve the above problems, the monitoring system of the present invention comprises a cargo handling vehicle on which a cargo handling worker can ride and which is operated by the cargo handling worker to perform cargo handling operations, and a plurality of unmanned aerial vehicles capable of autonomous flight and equipped with cameras, wherein the plurality of unmanned aerial vehicles move to the location where the cargo handling operations are performed and fly to photograph the work status related to the cargo handling operations with the cameras.

[0008] Furthermore, it is preferable to further include a management device capable of communicating with multiple unmanned aerial vehicles, wherein the management device transmits shooting location information relating to the shooting location to the unmanned aerial vehicles, and the unmanned aerial vehicles fly to the shooting location according to the shooting location information and wait there.

[0009] Furthermore, it is preferable to further include a management device capable of communicating with multiple of the aforementioned unmanned aerial vehicles, wherein the management device transmits vehicle identification information that can identify the cargo handling vehicle to the unmanned aerial vehicle, and the unmanned aerial vehicle flies in accordance with the vehicle identification information to follow the cargo handling vehicle.

[0010] Furthermore, it is preferable that the camera is configured to be switchable between multiple shooting modes, and that the subject being photographed or the angle of view being photographed changes when the shooting mode is switched.

[0011] Furthermore, it is preferable that the camera be configured to be switchable between a cargo handling location shooting mode for shooting the location where the cargo handling work is performed, a rack shooting mode for shooting the rack located in front of the cargo handling vehicle, and a worker shooting mode for shooting the cargo handling workers riding in the cargo handling vehicle. [Effects of the Invention]

[0012] According to the present invention, it is possible to provide a monitoring system that can monitor the work status related to cargo handling operations while suppressing cost increases. [Brief explanation of the drawing]

[0013] [Figure 1]This is an overview diagram of the monitoring system related to Embodiment 1. [Figure 2] This block diagram shows a schematic configuration of the monitoring system according to the same embodiment. [Figure 3] This flowchart shows the monitoring flow according to the same embodiment. [Figure 4] This is a block diagram showing the schematic configuration of the monitoring system relating to the first modified example. [Figure 5] This is a block diagram showing the schematic configuration of the monitoring system related to the second modified example. [Modes for carrying out the invention]

[0014] A monitoring system according to one embodiment of the present invention will be described with reference to the drawings. As shown in Figure 1, the monitoring system according to this embodiment comprises a forklift 1, a plurality of drones 2, and a management device 3. The monitoring system uses the drones 2 to monitor the work status related to cargo handling operations performed using the forklift 1.

[0015] Forklift 1 is a manned vehicle that can accommodate a cargo handling worker M, and is operated by the cargo handling worker M to perform cargo handling operations, including receiving and shipping cargo N. Receiving operations include placing cargo N on rack R, and shipping operations include removing cargo N from rack R.

[0016] Drone 2 is an autonomous, unmanned aerial vehicle equipped with a camera 25, and is a multi-rotor with excellent attitude stability. Drone 2 flies to the location where cargo handling operations are performed in order to monitor the work status related to cargo handling operations, and photographs the work status with the camera 25. Drone 2 transmits the video data related to the images captured by the camera 25 to the management device 3.

[0017] The management device 3 is a management server configured to be able to communicate with the forklift 1 and the plurality of drones 2, and manages various types of information. The management device 3 transmits cargo handling work information related to the cargo handling work to be performed by a cargo handling worker M to the forklift 1, and transmits photographing location information and photographing mode information, which will be described later, to the drone 2.

[0018] With reference to FIG. 2, a schematic configuration of the forklift 1, the drone 2, and the management device 3 will be described. As shown in FIG. 2, the forklift 1 includes a display unit 11 and a position information acquisition unit 12.

[0019] The display unit 11 is configured by a display that presents information to the cargo handling worker M, and displays the cargo handling work information received from the management device 3. The cargo handling worker M operates the forklift 1 with reference to the cargo handling work information displayed on the display unit 11.

[0020] The position information acquisition unit 12 acquires position information of the forklift 1, which is the own vehicle. The position information acquisition unit 12 is configured by, for example, a sensor that detects the surrounding environment of the forklift 1 or a receiver that receives a signal from a positioning satellite. The position information acquisition unit 12 transmits the position information of the forklift 1 to the management device 3.

[0021] Further, as shown in FIG. 2, the drone 2 includes rotary wings 21, a position information acquisition unit 22, an object recognition unit 23, a flight control unit 24, a camera 25, and a video transmission unit 26.

[0022] A plurality of the rotary wings 21 are provided on a concentric circle at intervals from each other, and generate lift for the drone 2 to fly. The flight speed, flight direction, and attitude of the drone 2 are configured to be changeable by controlling the rotary wings 21.

[0023] The position information acquisition unit 22 acquires the position information of the drone 2, which is the drone itself. The position information acquisition unit 22 is composed of, for example, a laser sensor that acquires SLAM (Simultaneous Localization and Mapping) data using LiDAR (Light Detection and Ranging), a motion sensor that acquires odometry data, a receiver that receives signals from positioning satellites, or a combination of these.

[0024] The object recognition unit 23 recognizes objects present around the drone 2. The object recognition unit 23 is composed of, for example, a laser sensor that acquires SLAM data using LiDAR, or a camera that takes pictures of the surroundings.

[0025] The flight control unit 24 consists of a controller that controls the flight of the drone 2. Based on the shooting location information received from the management device 3 and the location information acquired by the location information acquisition unit 22, the flight control unit 24 controls the rotor blades 21 so that the drone 2 flies to the shooting location, which is the location where cargo handling operations are performed. The flight control unit 24 also controls the rotor blades 21 so that the drone 2 avoids obstacles and continues flying, based on the location information of obstacles recognized by the object recognition unit 23. In this way, the drone 2 is configured to fly to the shooting location and wait there according to the shooting location information.

[0026] Camera 25 captures images of the work situation related to cargo handling operations (i.e., the operation status of the forklift 1, the situation in front of the forklift 1, and the situation of the cargo handling worker M operating the forklift 1) based on the shooting mode information received from the management device 3. In this embodiment, camera 25 is configured to be switchable between multiple shooting modes: a cargo handling area shooting mode, a rack shooting mode, and a worker shooting mode. By switching between these three shooting modes, the object being photographed or the shooting angle of view changes.

[0027] The loading / unloading area shooting mode is a shooting mode that photographs the area where loading / unloading operations are performed in order to photograph the operation of the forklift 1. In other words, the camera 25 in the loading / unloading area shooting mode targets the space in which the forklift 1 is located, and its field of view is wide enough to include the forklift 1 and its surrounding environment.

[0028] The rack shooting mode is a shooting mode that photographs the rack R located in front of the forklift 1 in order to photograph the situation in front of the forklift 1. The camera 25 in rack shooting mode targets the space located in front of the forklift 1 and the field of view is set to include all or part of the rack R.

[0029] The worker shooting mode is a shooting mode that photographs worker M, who is riding on forklift 1, in order to photograph the situation of worker M. The camera 25 in worker shooting mode targets the space in which worker M is located, and the shooting angle is a narrow angle that includes all or part of worker M (e.g., face or hands). In other words, the shooting angle in worker shooting mode is set to be narrower than the shooting angle in loading area shooting mode.

[0030] The video transmission unit 26 is composed of a wireless communication module and transmits video captured by the camera 25. Specifically, the video transmission unit 26 wirelessly transmits data related to the video captured by the camera 25 to the management device 3. The video captured by the camera 25 is then stored in the storage unit 31, which will be described later.

[0031] Furthermore, as shown in Figure 2, the management device 3 includes a storage unit 31, a cargo handling operation instruction unit 32, an information acquisition unit 33, a shooting planning unit 34, and a shooting instruction unit 35. Each of the units 31 to 35 constituting the management device 3 is composed of hardware that operates according to a predetermined program.

[0032] The storage unit 31 stores various types of information and data. Specifically, the storage unit 31 stores, for example, cargo handling operation information related to cargo handling operations performed by the forklift 1, location information of the forklift 1, and location information of the drone 2. The cargo handling operation information is information input using an input device (not shown) connected to the management device 3, and the location information of the forklift 1 and drone 2 is information obtained through communication with the forklift 1 and drone 2. The storage unit 31 also stores video data received from the drone 2.

[0033] The cargo handling instruction unit 32 is configured to communicate with the forklift 1 and instructs the cargo handling operator M on the cargo handling operations that the forklift 1 should perform. Specifically, the cargo handling instruction unit 32 instructs the cargo handling operations by transmitting cargo handling operation information stored in the storage unit 31 to the forklift 1.

[0034] The information acquisition unit 33 acquires cargo handling operation information from the storage unit 31 as information for determining the shooting location and shooting mode. The cargo handling operation information includes information about the location and scheduled time when the cargo handling operation will be performed.

[0035] The shooting planning unit 34 determines the shooting locations and shooting modes to be assigned to each of the multiple drones 2 based on the cargo handling operation information acquired by the information acquisition unit 33. Specifically, the shooting planning unit 34 determines the shooting locations to be photographed by the camera 25 from among multiple locations where cargo handling operations are performed, based on the cargo handling operation information. At this time, the shooting planning unit 34 selects the locations with the highest priority as shooting locations from among multiple locations that have been assigned a priority in advance and have not yet been selected as shooting locations. After determining the shooting locations, the shooting planning unit 34 determines a shooting mode corresponding to the shooting location. At this time, the shooting planning unit 34 selects, for example, the rack shooting mode or the worker shooting mode at the location where the forklift 1 performs receiving operations, and the cargo handling location shooting mode or the worker shooting mode at the location where the forklift 1 performs outbound operations.

[0036] The shooting instruction unit 35 is configured to communicate with multiple drones 2 and instructs the multiple drones 2 to take pictures of the work situation. Specifically, the shooting instruction unit 35 instructs the drones 2 to take pictures by transmitting shooting location information related to the shooting location determined by the shooting plan unit 34 and shooting mode information related to the shooting mode.

[0037] Referring to Figure 3, the process for monitoring the work status related to cargo handling operations will be explained. It is assumed that the various types of information mentioned above are pre-stored in the memory unit 31.

[0038] First, the information acquisition unit 33 acquires cargo handling operation information from the storage unit 31 (step S1), and the shooting planning unit 34 determines the shooting location and shooting mode based on the cargo handling operation information acquired in step S1 (step S2).

[0039] Then, the shooting instruction unit 35 transmits shooting location information related to the shooting location determined in step S2 and shooting mode information related to the shooting mode to multiple drones 2 (step S3), and the multiple drones 2 that receive the shooting location information and shooting mode information take pictures of the work status related to the cargo handling work performed by the forklift 1 (step S4). In other words, in step S4, each of the multiple drones 2 takes pictures with the camera 25 of the location where the cargo handling work is performed, the rack R located in front of the forklift 1, or the cargo handling worker M riding on the forklift 1, at the shooting location and shooting mode determined in step S2.

[0040] As described above, each of the multiple drones 2 films the work situation related to the cargo handling operations performed by the forklift 1 at its assigned filming location and in its filming mode. The filmed footage of the work situation is used, for example, as evidence of events that occurred during cargo handling operations (e.g., accidents), or as training data to predict events that may occur.

[0041] In this embodiment, the following effects can be obtained. (1) Multiple drones 2 (unmanned aerial vehicles) move to the location where cargo handling operations are performed and fly to photograph the work status related to cargo handling with cameras 25. With this configuration, the increase in the introduction cost of forklift 1 (cargo handling vehicle) can be reduced compared to a configuration in which cameras are attached to the forklift 1. In addition, since the location in which the work status is photographed can be changed by moving the drones 2, costs can be reduced compared to a configuration in which cameras are installed in many predetermined locations. Therefore, it is possible to monitor the work status related to cargo handling while keeping costs down. Furthermore, by constantly monitoring the status of cargo handling workers M as part of the work status related to cargo handling, it is possible to improve the safety awareness of cargo handling workers M.

[0042] (2) The control device 3 transmits location information relating to the shooting location to the drone 2, and the drone 2 flies to the shooting location according to the location information and waits there. With this configuration, the control device 3 can centrally control the waiting locations of multiple drones 2 (i.e., shooting locations where the work situation is photographed by the camera 25).

[0043] (3) The camera 25 is configured to be switchable between multiple shooting modes, and the shooting subject or shooting angle of view changes when the shooting mode is switched. With this configuration, the shooting method can be changed by switching the shooting mode of the camera 25.

[0044] (4) The camera 25 is configured to be switchable between a loading / unloading area shooting mode, a rack shooting mode, and a worker shooting mode. With this configuration, by switching between these shooting modes, it is possible to photograph the working conditions related to loading / unloading operations, including the operation of the forklift 1, the situation in front of the forklift 1, and the situation of the loading / unloading worker M.

[0045] The present invention is not limited to the embodiments described above, and the above configuration can be modified. For example, it can be implemented with the following modifications, or a combination of the following modifications can be used.

[0046] As shown in Figure 4, the drone 2 may be equipped with a shooting planning unit 28. The configuration of the first modified example shown in Figure 4 will be described below. Configurations similar to those of the above embodiment will not be described.

[0047] The first modified drone 2 further includes an information acquisition unit 27 and a shooting planning unit 28, and is configured to be able to communicate with other drones 2 (i.e., communicate with other drones 2). The information acquisition unit 27 acquires cargo handling operation information from the storage unit 31 of the management device 3 as information for determining the shooting location and shooting mode.

[0048] The shooting plan unit 28 determines the shooting location and shooting mode based on the cargo handling operation information acquired by the information acquisition unit 27 and the shooting location information received from other drones 2. Specifically, the shooting plan unit 28, for example, selects a location with a high priority from among multiple locations where cargo handling operations are performed and which have been assigned priorities in advance, by excluding locations that have not been selected as shooting locations by other drones 2. After determining the shooting location, the shooting plan unit 28 determines a shooting mode corresponding to the shooting location, similar to the shooting plan unit 34. The shooting location information related to the determined shooting location is transmitted to other drones 2 as information for determining the shooting location.

[0049] In this modified version, the flight control unit 24 controls the rotor blades 21 so that the drone 2 flies to the shooting location based on the shooting location information defined by the shooting planning unit 28 and the location information acquired by the location information acquisition unit 22. In addition, the camera 25 in this modified version photographs the work status related to cargo handling operations based on the shooting mode information defined by the shooting planning unit 28.

[0050] The following effects can be obtained in the first modified example. (5) Since the drone 2 is equipped with a shooting planning unit 28, the management device 3 does not need to determine the shooting location and shooting mode, thereby reducing the burden on the management device 3 in the monitoring system.

[0051] As shown in Figure 5, the drone 2 may be equipped with a luggage identification unit 29 that identifies luggage N captured by the camera 25. The configuration of the second modified example shown in Figure 5 will be described below. Configurations similar to those of the above embodiment will not be described.

[0052] The cargo identification unit 29 obtains cargo identification information for identifying cargo N from the storage unit 31 of the management device 3, and identifies cargo N being handled by the forklift 1, or cargo N that the forklift 1 has handled, from the video captured by the camera 25 based on the cargo identification information. The cargo identification unit 29 then transmits information related to the identification result of cargo N to the management device 3.

[0053] In this modified example, the cargo handling instruction unit 32 is configured to communicate with the drone 2 and receives information from the drone 2 regarding the identification result of the cargo N. Based on the identification result of the cargo N, the cargo handling instruction unit 32 determines whether the forklift 1 is performing the instructed cargo handling operation, and when it determines that the forklift 1 is not performing the instructed cargo handling operation, it transmits warning information to the cargo handling operator M to the forklift 1 to alert them. Specifically, for example, the cargo handling instruction unit 32 determines that the forklift 1 is not performing the instructed cargo handling operation if the cargo N being transported by the forklift 1 is not the designated cargo N, or if the location where the forklift 1 has placed the cargo N is not the designated location.

[0054] In this modified example, the display unit 11 displays warning information received from the control device 3. The cargo handling worker M operates the forklift 1 in a way that corrects any work errors by referring to the warning information displayed on the display unit 11.

[0055] The following effects can be obtained in the second modified example. (6) The control device 3 is configured to determine whether or not the forklift 1 is performing the instructed cargo handling operation based on the video footage captured by the camera 25. With this configuration, for example, if the forklift 1 is not handling the designated cargo N, the device can alert the cargo handling worker M, thereby reducing work errors.

[0056] In the second modified example, the drone 2 may be configured to determine whether or not the forklift 1 is performing the instructed cargo handling operation based on the video footage captured by the camera 25. In this case, it is preferable that when the drone 2 determines that the forklift 1 is not performing the instructed cargo handling operation, it transmits warning information to the forklift 1 without going through the management device 3.

[0057] The management device 3 may include a package identification unit that identifies the package N based on video data received from the drone 2. With this configuration, the effects of (6) above can be obtained with the same configuration as the second modified example, and the drone 2 does not need to identify the package N, thereby reducing the burden on the drone 2.

[0058] The management device 3 transmits vehicle designation information that allows for the identification of the forklift 1 to the drone 2, and the drone 2 may fly to follow the forklift 1 according to the vehicle designation information. In this case, the object recognition unit 23 recognizes the forklift 1 that is to be followed, and the flight control unit 24 controls the rotor blades 21 so that the drone 2 moves in accordance with the forklift 1 based on the position information of the forklift 1 recognized by the object recognition unit 23. With this configuration, compared to a configuration in which the drone 2 does not follow the forklift 1, it is possible to photograph the work situation in the vicinity of the forklift 1.

[0059] Drone 2 may be equipped with a storage unit that stores data related to the video footage captured by camera 25. With this configuration, it becomes unnecessary to transmit video data from drone 2 to management device 3, and the amount of communication between management device 3 and drone 2 can be reduced.

[0060] The method for determining the shooting location and shooting mode may be changed as appropriate. For example, the shooting mode may be determined first, and then a shooting location suitable for that mode may be selected. Alternatively, the system may be configured so that the shooting location is determined by the control device 3 and the shooting mode is determined by the drone 2.

[0061] The cargo handling vehicle on which cargo handling worker M is riding may be a cargo handling vehicle other than forklift 1. That is, drone 2 may, for example, photograph a transport cart, a towing vehicle, or the work status related to cargo handling operations performed by a towing vehicle with camera 25. [Explanation of Symbols]

[0062] 1. Forklift (material handling vehicle) 2. Drones (unmanned aerial vehicles) 3 Management device 25 Cameras M Cargo handling worker R Rack

Claims

1. A cargo handling vehicle that can accommodate a cargo handling worker and is operated by the said cargo handling worker to perform cargo handling operations, Multiple unmanned aircraft capable of autonomous flight and equipped with cameras, It comprises a control device capable of communicating with multiple of the aforementioned unmanned aerial vehicles, The camera is configured to be switchable between multiple shooting modes, and the subject being photographed or the field of view being photographed changes when the shooting mode is switched. The management device includes a shooting planning unit that determines a predetermined shooting location from a plurality of locations based on cargo handling information including information relating to the location where the cargo handling work is performed, and determines a specific shooting mode corresponding to the shooting location from a plurality of shooting modes. Multiple unmanned aircraft move to the shooting location, which is the location where the cargo handling work is performed, based on the shooting location information related to the shooting location determined by the shooting planning unit, and fly to photograph the work status related to the cargo handling work with the camera, and the camera attached to the unmanned aircraft photographs the work status in the specific shooting mode based on the shooting mode information related to the specific shooting mode determined by the shooting planning unit. A monitoring system characterized by the following features.

2. The unmanned aircraft flies to the shooting location and waits there according to the shooting location information. The monitoring system according to claim 1.

3. The management device transmits vehicle identification information that can identify the cargo handling vehicle to the unmanned aircraft, The unmanned aerial vehicle flies in accordance with the vehicle identification information to follow the cargo handling vehicle. The monitoring system according to claim 1.

4. A cargo handling vehicle on which a cargo handling worker can ride and which is operated by the cargo handling worker to perform cargo handling work, It comprises multiple unmanned aerial vehicles capable of autonomous flight and equipped with cameras, The camera is configured to be switchable between multiple shooting modes, and the subject being photographed or the field of view being photographed changes when the shooting mode is switched. The aforementioned unmanned aerial vehicle includes a shooting planning unit that determines a predetermined shooting location from a plurality of locations based on cargo handling information including information relating to the location where the cargo handling operation is performed, and determines a specific shooting mode corresponding to the shooting location from a plurality of shooting modes. Multiple unmanned aircraft move to the shooting location, which is the location where the cargo handling work is performed, based on the shooting location information related to the shooting location determined by the shooting planning unit, and fly to photograph the work status related to the cargo handling work with the camera, and the camera attached to the unmanned aircraft photographs the work status in the specific shooting mode based on the shooting mode information related to the specific shooting mode determined by the shooting planning unit. A monitoring system characterized by the following features.

5. The camera is configured to be switchable between a cargo handling location shooting mode for photographing the area where the cargo handling work is performed, a rack shooting mode for photographing the racks located in front of the cargo handling vehicle, and a worker shooting mode for photographing the cargo handling workers riding in the cargo handling vehicle. A monitoring system according to any one of claims 1 to 4.

Images