Conveying system

Abstract

To prevent an operator who operates a manned conveyance vehicle from falling asleep by using an unmanned flying body, in a conveyance system including the manned conveyance vehicle and the unmanned flying body.SOLUTION: An unmanned flying body 2 includes: an imaging part 25 for imaging a face of an operator O who is operating a manned conveyance vehicle 1; and a warning part 26 for giving a warning to the operator O who is operating the manned conveyance vehicle 1 by using a drowsiness determination part 35 for determining the drowsiness degree of the operator O based on the face image of the operator O by the imaging part 25, and an age determination part 36 for determining the age of the operator O based on the face image of the operator O by the imaging part 25. The age determination part 36 includes: a learning model generation part 51 for performing machine learning from teacher data 56 collected in a collection part 50, and for generating and storing a learning model by the machine learning; an acquisition part 55 for acquiring feature data of the face of the operator O at present time by the predetermined time; and a prediction part 52 for acquiring the age from the learning model.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present invention relates to a transportation system including a manned transporter and an unmanned aerial vehicle.

Background Art

[0002] A manned transporter (e.g., a forklift) used in facilities such as factories and warehouses is configured to travel and operate when an operator boards and operates it. Further, the forklift is configured to perform a loading / unloading operation for loading and unloading goods using forks.

[0003] And there is known a transportation system configured to guide a manned transporter operated by an operator using an unmanned aerial vehicle capable of hovering in the air (see Patent Document 1, etc.).

[0004] In the transportation system, the unmanned aerial vehicle includes a projector that projects a guidance image onto the road surface. The guidance image, for example, shows an arrow indicating a specific direction and is projected onto the road surface in front of the manned transporter. Thereby, an operator operating the manned transporter is configured to be guided to the loading / unloading position by checking the guidance image.

[0005] By the way, in the conventional transportation system, there was a problem that it was impossible to prevent an operator operating the manned transporter from falling asleep.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] Therefore, the problem that the present invention aims to solve is to provide a transport system that includes a manned transport vehicle and an unmanned aerial vehicle, which can prevent the operator operating the manned transport vehicle from falling asleep using the unmanned aerial vehicle. [Means for solving the problem]

[0008] To solve the above problems, the present invention provides a transport system comprising a manned transport vehicle and an unmanned aerial vehicle, wherein the unmanned aerial vehicle includes a camera unit that photographs the face of an operator operating the manned transport vehicle, a sleepiness determination unit that determines the degree of sleepiness of the operator based on the face image of the operator taken by the camera unit, and an age determination unit that determines the age of the operator based on the face image of the operator taken by the camera unit, and an alarm unit that alerts the operator operating the manned transport vehicle based on the degree of sleepiness and age. The age determination unit includes a collection unit that collects feature data related to the characteristics of a human face and training data based on the relationship between the human face and age; a learning model generation unit that performs machine learning on the training data collected by the collection unit and generates and stores a learning model through machine learning; an acquisition unit that uses an imaging unit to acquire the current face feature data of the operator at predetermined intervals; a prediction unit that inputs the current face feature data of the operator acquired from the acquisition unit into the learning model generated by the learning model generation unit to obtain the age from the learning model; and a decision unit that determines whether to alert the operator based on the age obtained by the prediction unit.

[0009] Preferably, the sleepiness detection unit includes: a collection unit that collects feature data relating to the characteristics of a human face and training data based on the relationship between the characteristics of a human face and the degree of sleepiness related to a human face; a learning model generation unit that performs machine learning on the training data collected by the collection unit and generates and stores a learning model by machine learning; an acquisition unit that uses an imaging unit to acquire the current facial feature data of the operator at predetermined intervals; a prediction unit that inputs the current facial feature data of the operator acquired from the acquisition unit into the learning model generated by the learning model generation unit to obtain the degree of sleepiness from the learning model; and a decision unit that determines whether to issue an alarm to the operator based on the degree of sleepiness obtained by the prediction unit.

[0010] Preferably, the alarm unit sounds an alarm not only to the operator but also to those around the manned transport vehicle when the level of drowsiness is above a predetermined level and the age is below a predetermined age.

[0011] Furthermore, it is desirable that the alarm unit emits sound, light, and / or a combination thereof as an alarm.

[0012] Furthermore, the unmanned aerial vehicle may be equipped with a projection unit that projects images onto the walkway, and the alarm unit may project an alarm image from the projection unit to alert the area around the manned transport vehicle.

[0013] Furthermore, the unmanned aerial vehicle may be configured to travel along a taxiway used to guide the movement of a manned transport vehicle. [Effects of the Invention]

[0014] The transport system according to the present invention is a transport system comprising a manned transport vehicle and an unmanned aerial vehicle, wherein the unmanned aerial vehicle can effectively prevent the operator of the manned transport vehicle from falling asleep. [Brief explanation of the drawing]

[0015] [Figure 1] A perspective view showing the transport system. [Figure 2] A side view showing the transport system. [Figure 3]Plan view showing a conveying system. [Figure 4] Block diagram showing a conveying system. [Figure 5] Block diagram showing the configuration of a drowsiness determination unit. [Figure 6] Block diagram showing the configuration of an age determination unit. [Figure 7] Flowchart showing the control procedure for preventing drowsy driving.

Mode for Carrying Out the Invention

[0016] Hereinafter, embodiments of a conveying system according to the present invention will be described based on the drawings.

[0017] Based on FIGS. 1 to 6, the configuration of the conveying system will be described.

[0018] As shown in FIGS. 1 to 4, the conveying system S includes a manned transport vehicle 1 on which an operator O rides and operates. The manned transport vehicle 1 is configured to travel and operate when the operator O rides and operates it. In the present embodiment, the manned transport vehicle 1 is a counterbalanced forklift, and is configured to be able to travel the vehicle body and raise and lower the fork when the operator O rides and operates it.

[0019] The conveying system S includes a plurality of shelves R installed in facilities such as factories and warehouses. The shelf R has a plurality of stepped portions in the height direction, and is configured to be able to store the load L at a predetermined position of the stepped portion. The manned transport vehicle 1 performs loading and unloading by placing and picking up the load L at a predetermined position of the shelf R. The shelves R are arranged at intervals of a predetermined width so that the manned transport vehicle 1 can travel and perform loading and unloading, and a passage P is formed between each shelf R (FIGS. 1 and 3).

[0020] The conveying system S includes an unmanned flying body 2 that can be stopped in the air. The unmanned flying body 2 is called a drone, and is configured to fly to a predetermined in-air stop position by the rotation of rotors provided at the tip sides of a plurality of arms, and to be able to hover at the predetermined in-air stop position.

[0021] The conveying system S includes a management device 3 for controlling the unmanned aircraft 2 (Fig. 4). The management device 3 includes a storage unit 30. The storage unit 30 stores a map M composed of shelves R and passages P installed in the facility, packages L arranged in the facility, and the like.

[0022] Furthermore, the storage unit 30 stores a handling task T performed by the manned carrier vehicle 1 as a handling schedule J. That is, the handling schedule J includes a plurality of handling tasks T such as a task T1 of picking up a package L from a predetermined location on a predetermined shelf R, a task T2 of placing the package L at a predetermined location on a predetermined shelf R, a task T3 of placing the package L at a shipping location, and a task T4 of picking up the package L from a receiving location, which are set in a predetermined order. The handling task T includes position information of the package L and handling (picking up or placing) information for the package L.

[0023] The management device 3 includes a handling instruction unit 34 configured to display the handling task T of the handling schedule J transmitted from the storage unit 30 on a display unit 11 provided in the driver's seat of the manned carrier vehicle 1.

[0024] The display unit 11 is, for example, composed of a touch panel display. The handling instruction unit 34 displays the handling task T that the manned carrier vehicle 1 should perform on the display unit 11. The operator O operates the manned carrier vehicle 1 to perform handling according to the handling task T displayed on the display unit 11. When the handling task T is completed, the operator O presses an end button displayed on the display unit 11, and an end signal is transmitted to the handling instruction unit 34. The handling instruction unit 34 is configured to display the next handling task T that the manned carrier vehicle 1 should perform on the display unit 11 when it receives the end signal.

[0025] The manned carrier vehicle 1 includes a position detection unit 10. The position detection unit 10 is composed of a laser sensor, a GPS sensor, an electromagnetic induction sensor, or the like. The position detection unit 10 is configured to detect the vehicle position D1 of the manned carrier vehicle 1.

[0026] The management device 3 includes a guideway generation unit 31. Based on the vehicle position D1 information of the manned transport vehicle 1 transmitted from the position detection unit 10, the facility map M transmitted from the storage unit 30, and the cargo handling task T of the cargo handling schedule J transmitted from the storage unit 30, the guideway generation unit 31 generates a virtual guideway 4 between the vehicle position D1 and the cargo handling position D2 of the manned transport vehicle 1. The cargo handling position D2 is the position on the passage P where the manned transport vehicle 1 picks up and places cargo in the cargo handling task T (Figure 3).

[0027] As shown in Figure 3, the guideway generation unit 31 is configured to generate a virtual guideway 4 that connects, for example, a vehicle position D1 and a cargo handling position D2 on a passage P. The guideway 4 is set, for example, to minimize the travel distance of the manned transport vehicle 1. In this embodiment, as shown in Figure 3, the guideway 4 is composed of a first straight section 41, a bend 40, and a second straight section 42.

[0028] The unmanned aerial vehicle 2 is equipped with a position detection unit 20. The position detection unit 20 consists of a GPS sensor, a gyro sensor, an ultrasonic sensor, a laser sensor, a barometric pressure sensor, a compass, an accelerometer, etc., and can detect the position of the unmanned aerial vehicle 2.

[0029] The unmanned aerial vehicle 2 is equipped with a flight control unit 21. The flight control unit 21 is configured to control the rotation of the rotor blades. Based on the detection results of the position detection unit 20 and the control of the flight control unit 21, the unmanned aerial vehicle 2 can fly to a predetermined aerial stopping position on the taxiway 4 and hover at the aerial stopping position.

[0030] The position determination unit 32 is further configured to determine the aerial hovering position for the unmanned aerial vehicle 2 on the taxiway 4. The position determination unit 32 is configured to hover at a position on the taxiway 4 that is a certain distance away from the vehicle position D1 of the manned transport vehicle 1 in the forward direction of the manned transport vehicle 1. Therefore, as the manned transport vehicle 1 approaches the unmanned aerial vehicle 2, the unmanned aerial vehicle 2 moves away from the manned transport vehicle 1, so that the distance between the manned transport vehicle 1 and the unmanned aerial vehicle 2 remains constant.

[0031] The unmanned aerial vehicle 2 is equipped with a memory unit 22. The memory unit 22 stores guidance images 200. The guidance images 200 consist of, for example, arrows for guiding the manned transport vehicle 1 to the loading / unloading position D2, and the direction of the arrows is configured to differ depending on the loading / unloading position D2 (Figures 1 and 3).

[0032] The unmanned aerial vehicle 2 is equipped with a projection unit 23. The projection unit 23 is composed of, for example, a projector, and can project guidance images 200 stored in the memory unit 22 onto the corridor P of the facility (Figures 1 to 3).

[0033] The control device 3 includes a projection instruction unit 33. The projection instruction unit 33 is configured to determine the guidance image 200 to be projected onto the road surface P according to the taxiway 4 from the taxiway generation unit 31 and the position of the unmanned aircraft 2 determined by the position detection unit 20, and to send a projection instruction to the projection unit 23 of the unmanned aircraft 2.

[0034] Operator O can visually observe the guidance image 200 projected onto the passage P, drive the manned transport vehicle 1 along the guidance image 200 to the loading / unloading position D2, and operate the manned transport vehicle 1 according to the loading / unloading task T displayed on the display unit 11 to perform loading / unloading operations on the cargo L.

[0035] The unmanned aerial vehicle 2 is equipped with an imaging unit 25 having a CCD image sensor, a CMOS image sensor, etc. The imaging unit 25 is configured to continuously photograph the face of the operator O who is driving the manned transport vehicle 1, by driving the imaging direction so that it is always facing the operator O, based on the position of the manned transport vehicle 1 from the position detection unit 10 and the position of the unmanned aerial vehicle 2 from the position detection unit 20.

[0036] Furthermore, the control device 3 includes a drowsiness determination unit 35. The drowsiness determination unit 35 is configured to determine the degree of drowsiness of operator O (for example, levels 1 to 5) based on the facial image data of operator O acquired by the imaging unit 25.

[0037] If we adopt NEDO's evaluation method, drowsiness level 1 is "doesn't seem sleepy at all," drowsiness level 2 is "seems a little sleepy," drowsiness level 3 is "seems sleepy," drowsiness level 4 is "seems quite sleepy," and drowsiness level 5 is "seems very sleepy."

[0038] As shown in Figure 5, the drowsiness detection unit 35 includes a collection unit 40 for collecting training data 46. The training data 46 includes feature data D relating to the characteristics of a human face. In this embodiment, the feature data D relating to various human facial features includes eye opening C1, number of eyelid openings and closings C2, gaze movement C3, mouth opening C4, head up and down movement C5, and number of yawns C6.

[0039] The drowsiness detection unit 35 includes a learning model generation unit 41 that performs machine learning on the training data 46 collected by the collection unit 40 and generates and stores a learning model through machine learning. In this embodiment, the learning model generation unit 41 performs supervised learning. In supervised learning, a large amount of training data 46, that is, pairs of input data ID and output data OD, is input to the learning model generation unit 41.

[0040] The input data ID includes eye opening degree C1, number of eyelid openings and closings C2, gaze movement C3, mouth opening degree C4, head up and down movement C5, and number of yawns C6. The output data OD represents the degree of sleepiness. The input data ID is evaluated to indicate the sleepiness level of operator O, and a numerical parameter (level) from 1 to 5 is set.

[0041] A high numerical parameter (level) indicating the degree of drowsiness, that is, a high degree of drowsiness in operator O, is determined to occur when, for example, (1) the degree of eye opening C1 is lower than the pre-stored degree of eye opening, (2) the number of times the eyelids are opened and closed C2 is less than the pre-stored number of times the eyelids are opened and closed, (3) the amount of gaze movement C3 is less than the amount of gaze movement pre-stored, (4) the degree of mouth opening C4 is higher than the pre-stored degree of mouth opening, (5) the up-and-down movement of the head C5 is greater than the pre-stored up-and-down movement of the head, or (6) the number of yawns C6 is greater than the pre-stored number of yawns. The drowsiness determination unit 35 determines the degree of drowsiness in operator O based on the numerical parameters (1) to (6) above.

[0042] The degree of drowsiness may be set using any of the following numerical parameters: (1) eye opening degree C1, (2) number of eyelid openings and closings C2, (3) amount of gaze movement C3, (4) mouth opening degree C4, (5) head up and down movement C5, or (6) number of yawns C6, or it may be set using numerical parameters weighted by a weighting coefficient.

[0043] In fact, the degree of drowsiness of operator O is often easily discernible through the state and movements of operator O's eyes, eyelids, mouth, etc. Therefore, it can be inferred that there is a certain correlation or relationship between (1) the degree of eye opening C1, (2) the number of times the eyelids are opened and closed C2, (3) the amount of gaze movement C3, (4) the degree of mouth opening C4, (5) the up and down movement of the head C5, and (6) the number of times yawning C6, and the degree of drowsiness of operator O.

[0044] The learning model generation unit 41 uses a general machine learning algorithm such as a neural network. The learning model generation unit 41 performs machine learning using the correlated input data ID and output data OD as training data 46 to generate a model (learning model) that estimates output from input, that is, a model that outputs the degree of sleepiness when input data ID is input.

[0045] The drowsiness detection unit 35 includes an acquisition unit 45 that acquires the current input data ID at predetermined intervals. As described above, the input data ID consists of (1) eye opening degree C1, (2) number of eyelid opening and closing cycles C2, (3) gaze movement amount C3, (4) mouth opening degree C4, (5) head up and down movement C5, and (6) number of yawns C6. The acquisition unit 45 is configured to acquire the input data ID based on the face image data of operator O obtained by the imaging unit 25 using known image analysis techniques. The input data ID is acquired at predetermined intervals (e.g., every second).

[0046] The drowsiness determination unit 35 includes a prediction unit 42 that predicts numerical parameters indicating the degree of drowsiness of operator O by applying the learning model generated by the learning model generation unit 41 to the current input data ID acquired from the acquisition unit 45.

[0047] The drowsiness detection unit 35 includes a decision unit 43, which determines whether to issue an alarm to operator O based on the output data OD predicted by the prediction unit 42, and outputs a decision signal to the alarm unit 26.

[0048] The drowsiness determination unit 35 determines that there is no drowsiness at level 1, and that there is drowsiness at levels 2 to 5.

[0049] Furthermore, the management device 3 includes an age determination unit 36. The age determination unit 36 ​​is configured to determine the age of operator O based on the facial image data of operator O acquired by the imaging unit 25.

[0050] As shown in Figure 6, the age determination unit 36 ​​includes a collection unit 50 for collecting training data 56. The training data 56 includes feature data D relating to the characteristics of a human face. In this embodiment, the feature data D relating to various human facial characteristics are facial skin E1, the relationship between the position and size of the eyes E2, the relationship between the position and size of the nose E3, the relationship between the position and size of the mouth E4, and skin color E5.

[0051] The age determination unit 36 ​​includes a learning model generation unit 51 that performs machine learning on the training data 56 collected by the collection unit 50 and generates and stores a learning model through machine learning. In this embodiment, the learning model generation unit 51 performs supervised learning. In supervised learning, a large amount of training data 56, that is, pairs of input data ID and output data OD, is input to the learning model generation unit 51.

[0052] The input data ID includes facial skin type E1, eye position and size relationship E2, nose position and size relationship E3, mouth position and size relationship E4, and skin color E5. The output data OD is age. The input data ID is evaluated as the age of operator O, and the numerical parameter for age is set.

[0053] The numerical parameter for age is high, meaning that operator O is judged to be older. Examples of cases where this is determined include: (1) the number of wrinkles on the facial skin E1 is greater than the number of wrinkles previously stored; (2) the degree to which the relationship between the position and size of the eyes E2 differs from the previously stored relationship between the position and size; (3) the degree to which the relationship between the position and size of the nose E3 differs from the previously stored relationship between the position and size; (4) the degree to which the relationship between the position and size of the mouth E4 differs from the previously stored relationship between the position and size; and (5) the skin color E5 is darker than the previously stored skin color. The drowsiness determination unit 36 ​​determines operator O's age based on the numerical parameters (1) to (5) above.

[0054] Age may be set using any of the following numerical parameters: (1) facial skin E1, (2) relationship between eye position and size E2, (3) relationship between nose position and size E3, (4) relationship between mouth position and size E4, or (5) skin color E5, or it may be set using numerical parameters weighted by a weighting coefficient.

[0055] In practice, the age of operator O can often be easily determined by the condition of operator O's facial skin, eyes, nose, mouth, and skin. Therefore, it can be inferred that there is a certain correlation or relationship between (1) facial skin E1, (2) the relationship between the position and size of the eyes E2, (3) the relationship between the position and size of the nose E3, (4) the relationship between the position and size of the mouth E4, and (5) skin color E5, and operator O's age.

[0056] The learning model generation unit 51 uses a general machine learning algorithm such as a neural network. The learning model generation unit 51 performs machine learning using the correlated input data ID and output data OD as training data 56 to generate a model (learning model) that estimates output from input, that is, a model that outputs the degree of sleepiness when input data ID is input.

[0057] The age determination unit 36 ​​includes an acquisition unit 55 that acquires the current input data ID at predetermined intervals. As described above, the input data ID consists of (1) facial skin E1, (2) the relationship between the position and size of the eyes E2, (3) the relationship between the position and size of the nose E3, (4) the relationship between the position and size of the mouth E4, and (5) skin color E5. The acquisition unit 55 is configured to acquire the input data ID based on the facial image data of operator O obtained by the imaging unit 25 using known image analysis techniques. The input data ID is acquired at predetermined intervals (e.g., every second).

[0058] The age determination unit 36 ​​includes a prediction unit 52 that predicts numerical parameters indicating the age of operator O by applying the learning model generated by the learning model generation unit 51 to the current input data ID acquired from the acquisition unit 55.

[0059] The age determination unit 36 ​​includes a determination unit 53, which determines whether to issue an alarm to operator O based on the output data OD predicted by the prediction unit 52, and outputs a determination signal to the alarm unit 26.

[0060] The unmanned aerial vehicle 2 is equipped with an alarm unit 26. The alarm unit 26 is configured to alert the operator O and workers around the manned transport vehicle 1 based on decisions made by the drowsiness determination unit 35 and the age determination unit 36.

[0061] The alarm unit 26 may consist of an acoustic device and be configured to emit a sound as an alarm. The volume of the sound may be adjusted according to the degree of drowsiness and age. Furthermore, the alarm unit 26 may consist of a lighting device and be configured to emit light as an alarm. The intensity of the light may be adjusted according to the degree of drowsiness and age. Furthermore, the alarm unit 26 may consist of a water discharge device and be configured to emit water as an alarm. The strength of the water discharge may be adjusted according to the degree of drowsiness and age.

[0062] Based on Figure 7, the control procedure for preventing drowsiness will be explained. Note that, to avoid repetition, parts already explained will be omitted.

[0063] The drowsiness detection unit 35 and the age detection unit 36 ​​collect training data 46 and 56 using the collection units 40 and 50 (step S1). Then, the learning model generation units 41 and 51 perform machine learning on the training data 46 and 56 collected by the collection units 40 and 50 in step S1, and generate and store a learning model through machine learning (step S2).

[0064] The drowsiness determination unit 35 and the age determination unit 36 ​​acquire the image data of operator O's face captured by the imaging unit 25 at predetermined intervals using the acquisition units 45 and 55 as the current input data ID (step S3).

[0065] The drowsiness determination unit 35 predicts the degree of drowsiness by operator O by applying the learning model generated in step S2 to the current input data ID acquired in step S3 using the prediction unit 42 (step S4).

[0066] Furthermore, the age determination unit 36 ​​predicts the age of operator O by applying the learning model generated in step S2 by the prediction unit 52 to the current input data ID obtained in step S3 (step S5).

[0067] The drowsiness determination unit 35 determines the degree of drowsiness of operator O based on the output data OD predicted in step S4 by the determination unit 43, and determines whether or not drowsiness is present (step S6). If the drowsiness determination unit 35 determines that the degree of drowsiness is level 1, it is determined that there is no drowsiness, and if the drowsiness determination unit 35 determines that the degree of drowsiness is levels 2 to 5, it is determined that there is drowsiness.

[0068] If the determination unit 43 determines that operator O is not sleepy (level 1), the process returns to step S3, where the imaging unit 25 of the unmanned aerial vehicle 2 photographs the face of operator O of the manned transport vehicle 1, and the acquisition unit 45 acquires the face image data. If the determination unit 43 determines that operator O is sleepy (levels 2-5), it determines which level of sleepiness from 2 to 5 it is (step S6).

[0069] If the drowsiness determination unit 35 determines that the drowsiness level is 2 or 3 (step S7), and the age determination unit 36 ​​determines that operator O is 40 years of age or older (step S8), the determination units 43 and 53 issue an alarm only to operator O, because the drowsiness level is low and the sleep depth is shallow due to the high age, and there is a high probability that operator O will wake up if an alarm is issued to operator O (step S9).

[0070] Then, in the case of drowsiness levels 4 and 5 (step S7), operator O is determined to be in a deep slumber, and since there is little chance that operator O will wake up even if an alarm is issued to operator O, the decision unit 43 also issues an alarm to workers around the manned transport vehicle 1 in addition to operator O (step S10).

[0071] Furthermore, even if the drowsiness determination unit 35 determines that the drowsiness level is 2 or 3 (step S7), and the age determination unit 36 ​​determines that operator O is under 40 years old (step S8), although the drowsiness level is low, the depth of sleep is deep because of the young age, so there is little chance that operator O will wake up even if an alarm is issued to operator O. Therefore, the decision unit 53 issues an alarm not only to operator O but also to workers around the manned transport vehicle 1 (step S10).

[0072] If the alarm unit 26 consists of an acoustic device, the acoustic device is equipped with a directional sound generating unit, and when alarming only the operator O, the sound is emitted locally towards the operator O, and when alarming not only the operator O but also workers around the manned transport vehicle 1, the sound is emitted over a wide area towards the operator O.

[0073] Furthermore, if the alarm unit 26 consists of a lighting device, the lighting device is equipped with a directional light-emitting unit. When alarming only the operator O, light is emitted locally towards the operator O. When alarming not only the operator O but also workers around the manned transport vehicle 1, light is emitted over a wide area towards the operator O. The lighting device may also alarm by flashing light.

[0074] Furthermore, if the alarm unit 26 consists of a water spraying device, the water spraying device is equipped with a directional water jet. When an alarm is to be issued only to operator O, water is sprayed locally towards operator O. When an alarm is to be issued to workers around the manned transport vehicle 1 in addition to operator O, water is sprayed over a wide area towards operator O.

[0075] Furthermore, the alarm unit 26 may use the projection unit 25 to display a large alarm image (for example, an alarm message such as "The operator of this forklift is asleep! Please be careful!") in the passageway P around the manned transport vehicle 1, thereby alerting workers around the manned transport vehicle 1 that the operator O operating the manned transport vehicle 1 is asleep.

[0076] Then, the drowsiness prevention control in steps S3 to S10 is repeatedly performed to sequentially determine and alert whether operator O is falling asleep.

[0077] Although preferred embodiments of the present invention have been described above, the configuration of the present invention is not limited to these embodiments. For example, it can be modified as follows.

[0078] In the above embodiment, the unmanned aerial vehicle 2 is configured to travel along a taxiway 4 for guiding the movement of the manned transport vehicle 1. However, it may also be configured to photograph the operator O of the manned transport vehicle 1 at any time and along any route without traveling along the taxiway 4.

[0079] In the above embodiment, the drowsiness determination unit 35 may be configured to determine the degree of drowsiness based on the angle of operator O's head, facial expression, changes in eyelid opening and closing, etc. Alternatively, the drowsiness determination unit 35 may be configured to store a facial image of operator O when not drowsy, and to determine the degree of drowsiness by comparing this stored facial image of operator O with the facial image of operator O acquired by the imaging unit 25.

[0080] Furthermore, the age determination unit 36 ​​may be configured using known image determination techniques such as biological features, local binary patterns, scale-invariant feature transformations, and deep learning.

[0081] In the above embodiment, the alarm unit 26 is composed of an acoustic device, a lighting device, or a water spraying device, but these may be selectively combined, or all of them may be provided in the unmanned aerial vehicle 2.

[0082] In the above embodiment, the unmanned aerial vehicle 2 is configured to project a guidance image 200 onto the passage P so that the taxiway 4 can be recognized visually by the operator O. However, it may also be equipped with a sound generating unit (not shown) that emits sounds such as voice, buzzer, or chime, so that the taxiway 4 can be recognized auditorily by the operator O. The sound generating unit is configured to emit voices such as, for example, "Turn left 15m ahead," "Your destination is 30m ahead," or "There is an obstacle ahead. Please be careful."

[0083] The effects of the present invention will be explained.

[0084] In a transport system S comprising a manned transport vehicle 1 and an unmanned aerial vehicle 2, the unmanned aerial vehicle 2 includes a camera unit 25 that photographs the face of operator O operating the manned transport vehicle 1, a sleepiness determination unit 35 that determines the degree of sleepiness of operator O based on the face image of operator O taken by the camera unit 25, and an age determination unit 36 ​​that determines the age of operator O based on the face image of operator O taken by the camera unit 25, and an alarm unit 26 that alerts operator O operating the manned transport vehicle 1 based on the degree of sleepiness and age. Furthermore, the age determination unit 36 ​​includes a collection unit 50 that collects feature data related to the characteristics of a human face and training data 56 based on the relationship between the human face and age; a learning model generation unit 51 that performs machine learning on the training data 56 collected by the collection unit 50 and generates and stores a learning model through machine learning; an acquisition unit 55 that uses the imaging unit 25 to acquire the current facial feature data of operator O at predetermined intervals; a prediction unit 52 that inputs the current facial feature data of operator O acquired from the acquisition unit 55 into the learning model generated by the learning model generation unit 51 to acquire the age from the learning model; and a decision unit 53 that determines whether to issue an alarm to operator O based on the age acquired by the prediction unit 52.

[0085] Therefore, by equipping the unmanned aerial vehicle 2 with a camera unit 25 and taking photographs from outside the manned transport vehicle 1, it is not necessary to equip each manned transport vehicle 1 with a camera unit 25, and the configuration for photographing the operator O can be minimized. In addition, since the operator O does not know when the unmanned aerial vehicle 2 will fly in, it is expected that the operator O will operate the manned transport vehicle 1 with a sense of tension. Furthermore, the drowsiness detection unit 35 determines the degree of drowsiness of the operator O, and the age detection unit 36 ​​determines the age of the operator O, so that if the operator O falls asleep, an alarm can be issued to the operator O to wake them up. Moreover, by using a learning model generated by machine learning, the age of the operator O can be determined more accurately.

[0086] Furthermore, the drowsiness determination unit 35 includes a collection unit 40 that collects feature data related to the characteristics of a human face and training data 46 based on the relationship between the characteristics of a human face and the degree of drowsiness related to a human face; a learning model generation unit 41 that performs machine learning on the training data 46 collected by the collection unit 40 and generates and stores a learning model by machine learning; an acquisition unit 45 that uses an imaging unit 25 to acquire the current facial feature data of operator O at predetermined intervals; a prediction unit 42 that inputs the current facial feature data of operator O acquired from the acquisition unit into the learning model generated by the learning model generation unit 41 to acquire the degree of drowsiness from the learning model; and a decision unit 43 that determines whether to issue an alarm to operator O based on the degree of drowsiness acquired by the prediction unit 42.

[0087] Therefore, by using a learning model generated by machine learning, it is possible to more accurately determine the degree of sleepiness of operator O.

[0088] Furthermore, it is desirable that the alarm unit 26 alerts the area around the manned transport vehicle 1 in addition to the operator O when the level of drowsiness exceeds a predetermined level.

[0089] If operator O's level of drowsiness exceeds a predetermined level, operator O may not wake up even if an alarm is issued. Therefore, by issuing an alarm to workers around the manned transport vehicle 1, it is possible to prevent workers from colliding with the manned transport vehicle 1 operated by the sleeping operator O.

[0090] Furthermore, the alarm unit 26 is designed to emit sound, light, and / or a combination thereof as an alarm. In this way, the alarm unit 26 can reliably and quickly wake up the sleeping operator O by emitting sound, light, and water.

[0091] Furthermore, the unmanned aerial vehicle 2 may be equipped with a projection unit 23 that projects an image onto the passage P, and the alarm unit 26 may project an alarm image from the projection unit 23 to alert the area around the manned transport vehicle 1. In this way, the projection unit 23 projects an alarm image, which directly and effectively alerts workers around the manned transport vehicle 1 that the operator O of the manned transport vehicle 1 is dozing off.

[0092] Furthermore, the unmanned aerial vehicle 2 is configured to travel along a taxiway 4 that guides the manned transport vehicle 1.

[0093] Therefore, the unmanned aerial vehicle 2 guiding the manned transport vehicle 1 can sequentially and accurately capture images of operator O's face. As a result, the drowsiness detection unit 35 determines the degree of operator O's drowsiness, and the age detection unit 36 ​​determines operator O's age. This allows for a quick and appropriate warning to be issued to operator O if they fall asleep, waking them up. [Explanation of Symbols]

[0094] S Conveyor System O Operator 1 Manned guided vehicle 2 Unmanned aircraft 4 Taxiway 23 Projection section 25 Photography Department 26 Alarm section 35 Sleepiness detection unit 36 Age determination section 40, 50 Collection Department 41, 51 Learning Model Generation Unit 42,52 Prediction section 43, 53 Decision Section 45,55 Acquisition Department 46,56 Training data ID Input Data OD output data

Claims

1. In a transport system equipped with a manned transport vehicle and an unmanned aerial vehicle, The aforementioned unmanned aircraft, A camera unit that photographs the face of the operator operating the aforementioned manned transport vehicle, The system includes a sleepiness determination unit that determines the degree of sleepiness of the operator based on the operator's facial image taken by the camera unit, and an age determination unit that determines the operator's age based on the operator's facial image taken by the camera unit, and an alarm unit that alerts the operator operating the manned transport vehicle based on the degree of sleepiness and the age. The age determination unit is, A collection unit that collects feature data related to the characteristics of human faces and training data based on the relationship between the human face and age, A learning model generation unit performs machine learning on the training data collected by the collection unit, and generates and stores a learning model using the machine learning. The aforementioned imaging unit acquires the facial feature data of the operator at the present time at predetermined intervals, and the acquisition unit A prediction unit obtains the age from the learning model by inputting the current face feature data of the operator obtained from the acquisition unit into the learning model generated by the learning model generation unit, The system includes a determination unit that determines whether to alert the operator based on the age obtained by the prediction unit. A transport system characterized by the following features.

2. The aforementioned drowsiness detection unit, A data collection unit collects feature data related to the characteristics of human faces and training data based on the relationship between the degree of sleepiness of the said human faces, A learning model generation unit performs machine learning on the training data collected by the collection unit, and generates and stores a learning model using the machine learning. The aforementioned imaging unit acquires the facial feature data of the operator at the present time at predetermined intervals, and the acquisition unit A prediction unit obtains the degree of drowsiness from the learning model by inputting the current facial feature data of the operator obtained from the acquisition unit into the learning model generated by the learning model generation unit, The system includes a determination unit that determines whether to alert the operator based on the degree of drowsiness acquired by the prediction unit. The transport system according to feature 1.

3. The alarm unit, when the level of drowsiness is above a predetermined level and the age is below a predetermined age, will issue an alarm not only to the operator but also to the area around the manned transport vehicle. The transport system according to feature 1.

4. The alarm unit emits sound, light, and water, or a combination thereof, as the alarm. The transport system according to feature 1.

5. The aforementioned unmanned aerial vehicle is equipped with a projection unit that projects images onto a passageway, The alarm unit is configured to project an alarm image from the projection unit to sound an alarm around the manned transport vehicle. The transport system according to feature 3.

6. The unmanned aerial vehicle is configured to travel along a taxiway for guiding the manned transport vehicle. The transport system according to feature 1.

Images