Notification device and control device

Abstract

To provide technology that can notify an appropriate worker or an on-site station.SOLUTION: Upon receiving a signal indicating the occurrence of an abnormal condition of a moving vehicle that can be moved by unmanned operation, a notification device identifies a notification target that satisfies predetermined specific conditions from among specific targets including at least one of a worker and an on-site station, and notifies the identified notification target of instructions for dealing with the abnormal condition of the moving vehicle.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a notification device and a control device. [Background technology]

[0002] 2. Description of the Related Art In a manufacturing process of a moving body, a technique for running the moving body in an unmanned manner, as described in Patent Document 1, for example, is known. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2017-538619 Summary of the Invention [Problem to be solved by the invention]

[0004] When a malfunction occurs during the movement of a moving object by unmanned operation and the moving object stops, workers may be called to respond. When issuing a notice to instruct all workers and all stations where workers are waiting to respond, the notice may be sent to workers or stations that are not suitable for responding, such as workers who are located far from the stopped moving object or stations where no workers are waiting. [Means for solving the problem]

[0005] The present disclosure has been made to solve the above-mentioned problems, and can be realized in the following forms.

[0006] (1) According to a first aspect of the present disclosure, there is provided a notification device that receives a signal indicating the occurrence of an abnormal state of a mobile body that can move by unmanned operation, identifies a notification target that satisfies a predetermined specific condition from specific targets including at least one of a worker and a guardhouse, and notifies the identified notification target of an instruction to deal with the abnormal state of the mobile body. According to this type of notification device, instructions for dealing with the abnormal state of the mobile body are notified to a notification target that satisfies a specific condition, so that notification can be sent to an appropriate worker or station. (2) In the notification device of the above aspect, the countermeasure does not have to include controlling the moving object by unmanned driving. According to this type of notification device, for example, it is possible to instruct an operator to move the mobile object in which the abnormality has occurred by manned operation. (3) In the notification device of the above aspect, the notification target may be notified of a moving object location indicating a location of the moving object. According to the notification device of this embodiment, it is possible to notify the notification target of the location of the mobile object. (4) In the notification device of the above form, the specific condition may include that the distance between the moving body position and the location of each of the identified targets is equal to or less than a predetermined first threshold distance, and the specific targets whose distance between the moving body position and the location is equal to or less than the first threshold distance may be identified. According to this type of notification device, it is possible to notify a notification target located close to the mobile body. (5) In the notification device of the above aspect, the specific condition may include a status of a task of the specific target, and the specific target may be identified as a task status for which the action can be taken. According to this type of notification device, for example, a specific target who is available can be identified, and instructions for dealing with the abnormal condition of the mobile body can be notified to the target who is likely to be able to deal with the abnormal condition of the mobile body. (6) In the notification device of the above form, the specific condition may include not performing any work that is more urgent than responding to the abnormal condition, and the specific target that is not performing the more urgent work may be identified. According to this type of notification device, it is possible to notify a notification target who is not performing a highly urgent task. (7) In the notification device of the above aspect, the specific condition may include a manufacturing process in which the moving object is located, and may specify the specific target associated with the manufacturing process. According to the notification device of this embodiment, for example, it is possible to notify a notification target belonging to a manufacturing process in which a mobile object is located. (8) In the notification device of the above aspect, the specific condition may include a cause of the occurrence of the abnormal state, and may identify the specific target that can deal with the cause. According to the notification device of this embodiment, it is possible to notify a notification target who can deal with the cause. (9) In the notification device of the above aspect, upon receipt of the completion of the handling, the notification device may notify the notification target that the handling has been completed. According to this form of notification device, the notification subject who has been notified of instructions to deal with an abnormal condition of a moving body can be prevented from trying to deal with the abnormal condition of a moving body that has already been dealt with. (10) The control device may, when receiving from the notification device of the above form information that there is no notification target that satisfies the specific condition, perform control to evacuate the moving body by unmanned operation. According to the control device of this embodiment, even if an operator is unable to directly deal with an abnormal state of the moving body, the moving body can be evacuated. (11) In the above-mentioned notification device, the unmanned operation includes a manual mode in which the operator operates an input device provided at a location different from the mobile body, and an automatic mode in which the operator does not operate the input device, and the abnormal condition occurs in the automatic mode, and the response may be to operate the input device. According to this type of notification device, instructions for operating the input device are notified to notification targets who satisfy specific conditions, so that the notification can be sent to an appropriate worker or station. (12) In the notification device of the above embodiment, the notification device may further include a memory unit that stores an installation position of the input device, and the specific condition may include a distance between the installation position and a location position of each of the identified targets being equal to or less than a predetermined second threshold distance, and the specific targets whose distance between the installation position and the location position is equal to or less than the second threshold distance may be identified. According to the notification device of this embodiment, it is possible to notify a notification target that is close to the input device. (13) In the notification device of the above aspect, the specific condition may include a task status of the specific target, and the specific target may be identified as a task status in which the input device can be operated. According to the notification device of this aspect, it is possible to notify an instruction for unmanned driving to a notification subject who is highly likely to be able to perform unmanned driving using an input device. (14) In the notification device of the above embodiment, the specific condition may include not performing any work that is more urgent than responding to the abnormal condition, and the specific target that is not performing the more urgent work may be identified. According to this type of notification device, it is possible to notify a notification target who is not performing a highly urgent task. (15) In the notification device of the above form, when there is no notification target that satisfies the specific condition, at least a portion of the specific targets may be notified of instructions for a response to the abnormal state of the moving body that does not include control of the moving body by unmanned driving. According to the notification device of this embodiment, even if unmanned operation using an input device is not possible, it is possible to deal with an abnormal state of the moving object. (16) The control device may, when receiving from the notification device of the above form information that the notification target that satisfies the specific condition does not exist, perform control to evacuate the moving body in the automatic mode. According to the control device of this embodiment, even when unmanned operation by the input device is not possible, the moving body can be evacuated.

[0007] The present disclosure can be realized in various forms, such as a remote control system, a mobile body control device, a remote automatic driving method, a method for manufacturing a mobile body, a control method for a control device, a control method for a remote control system, a control method for a mobile body, a computer program for realizing these control methods, and a non-transitory recording medium on which the computer program is recorded. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is an explanatory diagram showing a configuration of a remote control system. [Diagram 2] FIG. 1 is a schematic diagram showing an overview of a notification device. [Diagram 3] FIG. 1 is a schematic diagram showing an overview of a moving body. [Figure 4] 11 is a flowchart showing an example of a method for running a moving object. [Diagram 5] 13 is a flowchart illustrating an example of a notification process. [Figure 6] 10 is a flowchart showing an example of a notification process in the second embodiment. [Figure 7] 13 is a flowchart showing an example of a notification process in the third embodiment. [Figure 8] FIG. 13 is a schematic diagram showing an overview of a notification device according to a fourth embodiment. [Figure 9] 13 is a flowchart showing an example of a notification process in the fourth embodiment. [Figure 10] 13 is a flowchart showing an example of a notification process in the fifth embodiment. [Figure 11] 13 is a flowchart illustrating an example of a method for running a moving object according to another embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] A. First embodiment: 1 is an explanatory diagram showing the configuration of a remote control system 10 in the first embodiment. The remote control system 10 is used to move a moving body 200 in an unmanned manner in a factory where the moving body 200 is manufactured.

[0010] "Unmanned driving" means driving without the driver's operation. Driving operation means at least one of the following operations: "running", "turning", and "stopping". Unmanned driving is realized by automatic or manual remote control using a device installed outside the vehicle, or by autonomous control of the vehicle. A vehicle that is running by unmanned driving may have a passenger who does not perform driving operations on board. Passengers who do not perform driving operations include, for example, a person who simply sits in the seat of the vehicle, or a person who performs work other than driving operations, such as assembly, inspection, and operation of switches, while riding in the vehicle. Note that driving by a passenger's driving operation is sometimes called "manned driving".

[0011] In this embodiment, the unmanned operation includes a manual mode and an automatic mode. The manual mode is a mode performed by an operator WO using an input device 121, which will be described later. The automatic mode is a mode that does not depend on an operator WO using an input device 121.

[0012] The remote control system 10 includes a control device 100, a notification device 110, a moving object 200, and a detection device 300.

[0013] The control device 100 is a device that remotely controls the moving object 200 during the manufacturing process of the moving object 200. The control device 100 includes a remote manual control device 120 and a remote automatic control device 130. In this embodiment, the remote manual control device 120 and the remote automatic control device 130 are integrally configured. The control device 100 is provided at a location different from the moving object 200. The control device 100 and the notification device 110 are capable of communicating with each other.

[0014] The notification device 110 is a device that receives a signal indicating the occurrence of an abnormal state of the moving body 200 operated by remote control using the remote manual control device 120 or the remote automatic control device 130, and notifies the notification target identified from the specific target 400. In this embodiment, the abnormal state of the moving body 200 means that the moving body 200 has stopped abnormally. An abnormal stop means an unintended stop of the moving body 200 that is remotely operated. In this embodiment, the specific target 400 includes multiple workers WO and multiple guardhouses OF. The guardhouses OF are places where the workers WO take a break or wait. The notification device 110 notifies the workers WO via a terminal device T1 owned by the workers WO, and notifies the guardhouses OF via a terminal device T2 installed in the guardhouses OF.

[0015] 2 is a schematic diagram showing an overview of notification device 110. Notification device 110 is made up of a computer configured with a central processing unit (CPU), RAM, ROM, etc., and realizes the functions of identification unit 113 and notification unit 114 by CPU 111 executing a program pre-installed in storage unit 112 of notification device 110. However, some or all of the functions of these units may be realized by hardware circuits.

[0016] In response to the occurrence of an abnormal state of the mobile object 200 operated by remote control, the identification unit 113 identifies a notification target that satisfies a predetermined specific condition from among the specific targets 400. The specific condition includes, for example, a condition related to the position of the specific target 400, a work status of the specific target 400, a manufacturing process in which the mobile object 200 is located, and a cause of the occurrence of the abnormal state of the mobile object 200. Details of the specific condition will be described later. The identification unit 113 identifies a specific target 400 that satisfies the specific condition and is close to the mobile object 200, a specific target 400 that is a work status in which the mobile object 200 can be dealt with, a specific target 400 that is associated with a manufacturing process in which the mobile object 200 is located, and a specific target 400 that can handle the cause of the occurrence of the abnormal state of the mobile object 200.

[0017] The notification unit 114 notifies the notification target identified by the identification unit 113 of an instruction to deal with the abnormal state of the moving body 200. The deal with the stop of the moving body 200 is an operation that does not include remote operation. In this embodiment, the deal with the abnormal state of the moving body 200 is to recover the moving body 200, and to move the moving body 200 by manned operation.

[0018] The remote manual control device 120 (see FIG. 1) is a device that remotely controls the moving body 200 by the operation of the operator WO. That is, it is a device that performs unmanned driving in a manual mode. More specifically, the remote manual control device 120 remotely controls the moving body 200 by a travel control signal generated by the operation of the input device 121 by the operator WO. The input device 121 includes a steering wheel, an accelerator pedal, a brake pedal, and the like (not shown). Note that the moving body 200 employs a steer-by-wire system, a throttle-by-wire system, and a brake-by-wire system. In the remote manual control device 120, the input device 121 and a control unit (not shown) that generates a control command value may be provided separately. The travel control signal includes, for example, the travel route (path), position, travel speed, acceleration, and wheel steering angle of the moving body 200.

[0019] The remote automatic control device 130 is a device that remotely controls the moving body 200 without the need for operation by the worker WO. That is, it is a device that performs unmanned operation in an automatic mode. More specifically, the remote automatic control device 130 generates a control command value and remotely controls the moving body 200 without the operation of the input device 121 by the worker WO.

[0020] In this embodiment, the moving body 200 is a vehicle. More specifically, the moving body 200 is an electric vehicle (BEV: Battery Electric Vehicle). Note that the moving body 200 is not limited to an electric vehicle, and may be, for example, a gasoline-powered vehicle, a hybrid vehicle, or a fuel cell vehicle. The moving body 200 is not limited to a vehicle, and may be, for example, an electric vertical take-off and landing aircraft (a so-called flying car). When the moving body is other than a vehicle, the expression "running" in this disclosure can be appropriately replaced with "moving".

[0021] 3 is a schematic diagram showing an overview of the moving body 200. The moving body 200 includes a battery 210 which is a power source for the moving body 200, a driving device 220 for driving the moving body 200, a steering device 230 for changing the traveling direction of the moving body 200, a braking device 240 for decelerating the moving body 200, a moving body control device 250 for controlling each part of the moving body 200 according to an operation from outside the moving body 200, a communication device 260 for communicating with the control device 100 by wireless communication, an external sensor 270 capable of detecting the surroundings of the moving body 200, and an internal sensor 280 capable of acquiring information indicating the state of the moving body 200.

[0022] In this embodiment, the drive device 220 includes a traveling motor and a drive wheel rotated by the traveling motor. The mobile body control device 250 is configured with two or more ECUs (Electronic Control Units). When a driver is on board the mobile body 200, the mobile body control device 250 can control each part of the mobile body 200 according to the operation of the driver, thereby causing the mobile body 200 to travel. Regardless of whether a driver is on board the mobile body 200 or not, the mobile body control device 250 can control each part of the mobile body 200 according to a control command from the control device 100, thereby causing the mobile body 200 to travel.

[0023] The communication device 260 communicates with the outside of the mobile object 200, for example, by CAN (Fontroller Area Network) communication. The CAN communication is a communication standard that allows multi-directional transmission and reception. The communication device 260 also communicates with the control device 100 connected to the network NT, for example, via an access point (not shown) in a factory.

[0024] The external sensor 270 is a sensor that can detect targets existing around the moving body 200. The external sensor 270 includes, for example, an in-vehicle camera, a radar, a lidar, and the like.

[0025] The internal sensor 280 is a sensor that can detect various physical quantities necessary for controlling the operation of the moving body 200. The internal sensor 280 includes, for example, a current sensor, a voltage sensor, a battery temperature sensor, a wheel speed sensor, a steering angle sensor, a GNSS (Global Navigation Satellite System) receiver that receives GNSS information, and the like.

[0026] The detection device 300 is a device that detects the moving body 200, the surroundings of the moving body 200, and the worker WO. In this embodiment, the detection device 300 is a camera. The detection device 300 is installed in a factory. The detection device 300 communicates with the control device 100 connected to the network 26, for example, via an access point (not shown) in the factory. In this embodiment, the detection device 300 uses the detected information to identify the position, shape, etc. of the moving body 200, the position, shape, etc. of an object present in the surrounding area of ​​the moving body 200, and the position and movement, etc. of the worker WO. The detection device 300 transmits the identified information to the control device 100.

[0027] FIG. 4 is a flowchart showing a method for causing the moving body 200 to travel by the remote control system 10 of this embodiment. In step S110, the control device 100 acquires position information of the moving body 200 using a detection result output from the detection device 300 located outside the moving body 200. In this embodiment, the position information of the moving body 200 includes X, Y, and Z coordinates in a global coordinate system of the factory. The detection device 300 is a camera installed in the factory, and an image is output from the detection device 300 as a detection result. The position and orientation of the detection device 300 are adjusted in advance. The control device 100 detects the moving body 200 from an image acquired from the detection device 300, and acquires position information of the moving body 200 in the factory from the position of the moving body 200 in the image. The control device 100 can detect the moving body 200 from an image, for example, using a machine learning model that has been trained in advance to detect the moving body 200 from an image.

[0028] In step S120, the control device 100 determines a target position to which the mobile object 200 should next head. In this embodiment, the target position is represented by X, Y, and Z coordinates in the global coordinate system of the factory. The control device 100 prestores a reference route, which is a route along which the mobile object 200 should travel. The route is represented by nodes indicating a starting point, nodes indicating passing points, nodes indicating a destination, and links connecting the nodes. The control device 100 uses the position information of the mobile object 200 and the reference route to determine a target position to which the mobile object 200 should next head. The control device 100 determines a target position on the reference route that is ahead of the current location of the mobile object 200.

[0029] In step S130, the control device 100 generates a travel control signal for causing the moving body 200 to travel toward the determined target position. In this embodiment, the travel control signal includes the acceleration and steering angle of the moving body 200 as parameters. The control device 100 calculates the travel speed of the moving body 200 from the transition of the position of the moving body 200, and compares the calculated travel speed with a predetermined target speed of the moving body 200. When the travel speed is lower than the target speed, the control device 100 determines the acceleration so that the moving body 200 accelerates, and when the travel speed is higher than the target speed, the control device 100 determines the acceleration so that the moving body 200 decelerates. When the moving body 200 is located on the reference path, the control device 100 determines the steering angle and acceleration so that the moving body 200 does not deviate from the reference path, and when the moving body 200 is not located on the reference path, in other words, when the moving body 200 deviates from the reference path, the control device 100 determines the steering angle and acceleration so that the moving body 200 returns to the reference path.

[0030] In step S140, the control device 100 transmits the generated driving control signal to the moving object 200. The control device 100 repeats, at a predetermined cycle, obtaining position information of the moving object 200, determining a target position, generating a driving control signal, and transmitting the driving control signal.

[0031] In step S150, the mobile body control device 250 mounted on the mobile body 200 receives a traveling control signal from the control device 100. In step S160, the mobile body control device 250 controls the driving device 220, the steering device 230, and the braking device 240 using the received traveling control signal, thereby causing the mobile body 200 to travel at the acceleration and steering angle represented by the traveling control signal. The mobile body control device 250 repeats receiving the traveling control signal and controlling the driving device 220, the steering device 230, and the braking device 240 at a predetermined cycle. According to the remote operation system 10 of this embodiment, the mobile body 200 can be caused to travel by remote control, and therefore the mobile body 200 can be moved without using transportation equipment such as a crane or a conveyor.

[0032] The position and orientation of the moving body 200 are estimated using a captured image acquired by a detection device 300 provided at a location different from the moving body 200. The position of the moving body 200 can be acquired, for example, by calculating the coordinates of the positioning point of the moving body 200 in an image coordinate system using the outer shape of the moving body 200 detected from the captured image, and converting the calculated coordinates into coordinates in a global coordinate system. The orientation of the moving body 200 can be estimated based on the direction of the movement vector of the moving body 200 calculated from the position change of the feature point of the moving body 200 between frames of the captured image, using, for example, an optical flow method. The orientation of the moving body 200 may be calculated, for example, using the output result of a yaw rate sensor or the like mounted on the moving body 200.

[0033] The outer shape of the moving object 200 included in the captured image can be detected, for example, by inputting the captured image into a detection model that utilizes artificial intelligence. As the detection model, for example, a trained machine learning model that has been trained to realize either semantic segmentation or instance segmentation can be used. As this machine learning model, for example, a convolutional neural network (hereinafter, CNN) trained by supervised learning to which a training dataset is input can be used. The training dataset has, for example, a plurality of training images including the moving object 200, and a correct answer label indicating whether each region in the training image is a region indicating the moving object 200 or a region indicating something other than the moving object 200. During training of the CNN, it is preferable that the parameters of the CNN are updated by backpropagation (error backpropagation method) so as to reduce an error between the output result by the detection model and the correct answer label.

[0034] FIG. 5 is a flowchart showing an example of the notification process. The notification process is a process for notifying the notification target identified by the notification device 110 of an instruction to deal with the abnormal state of the moving body 200. In this embodiment, this process is a process that is performed when the moving body 200 operated by remote control stops unintentionally. The notification device 110 executes the notification process, for example, when it receives a signal indicating that the moving body 200 has stopped unintentionally from the moving body 200, or when it receives a signal indicating that the moving body 200 has stopped unintentionally from the control device 100 or the detection device 300. In this embodiment, the response to the abnormal state is a process that does not include the control of the moving body 200 by unmanned driving.

[0035] In step S210, the notification device 110 identifies a notification target. More specifically, the notification device 110 identifies a notification target that satisfies a first identifying condition from the identified target 400. Whether or not the identified target 400 satisfies the first identifying condition can be determined using a detection signal from the external sensor 270 or the internal sensor 280 of the moving object 200 or a detection signal from the detection device 300. As the first identifying condition, for example, any one or more of the following conditions can be adopted.

[0036] <Condition 1> The distance between the moving body position, which is the position of the moving body 200, and the location, which is the position of the specific target 400, is equal to or less than a first threshold distance determined in advance. <Condition 2> Does not belong to break time <Condition 3> The worker WO is waiting. <Condition 4> No urgent work is being carried out <Condition 5> The person is working in the manufacturing process in which the moving object 200 is located when it stops. <Condition 6> The distance between the facility that caused the moving body 200 to stop and the location is equal to or less than a second threshold distance that is determined in advance. <Condition 7> Ability to respond to the cause of the stoppage of the moving object 200

[0037] The mobile object position in the above condition 1 is identified, for example, by the GNSS receiver and the detection device 300 of the mobile object 200. The location is identified, for example, by the GNSS receiver and the detection device 300 of the terminal device T1. In addition, information indicating the position of the station OF is stored, for example, in the storage unit 112. If this condition 1 is not satisfied, the distance between the identification target 400 and the mobile object 200 is large, and it takes time for the worker WO to reach the mobile object 200. Therefore, it is preferable that the first identification condition includes this condition 1.

[0038] If the above condition 2 is not satisfied, the worker WO is in a break time, but it is not desirable to instruct the worker WO to deal with the stop of the moving body 200 outside of working hours. Therefore, it is preferable that the first identifying condition includes this condition 2. The identifying unit 113, for example, uses the shift information of the identified target 400 stored in the memory unit 112 to determine whether the identified target 400 is in a break time.

[0039] If the above condition 3 is not satisfied, the worker WO is not present at the guardhouse OF, and therefore even if an instruction to deal with the stop of the mobile body 200 is notified, the notification may not be received. Therefore, it is preferable that the first identifying condition includes this condition 3. The identifying unit 113, for example, uses information indicating whether the worker WO has entered or left the guardhouse OF, detected by the detection device 300, to determine whether the worker WO is waiting at the guardhouse OF.

[0040] The urgent task in the above condition 4 is a task that is more urgent than a task for dealing with an abnormal state of the moving body 200, for example, a task related to safety such as an abnormality response. If this condition 4 is not satisfied, the specific target 400 is performing a task with a high urgency and is busy, so even if an instruction to deal with the stop of the moving body 200 is notified, the specific target 400 may not be able to deal with the stop of the moving body 200. Therefore, it is preferable that the first identification condition includes this condition 4. In this way, the identification unit 113 identifies the specific target 400 that is not performing a task with a high urgency. For example, the identification unit 113 uses information indicating the operation of the specific target 400 identified by the detection device 300 to determine whether the specific target 400 is performing a task with a high urgency.

[0041] When the above condition 5 is satisfied, there is a high possibility that the specific target 400 is located close to the moving body 200. In addition, there is a high possibility that the specific target 400 can eliminate the cause of the stopping of the moving body 200. Therefore, it is preferable that the first identification condition includes this condition 5. For example, the identification unit 113 uses information indicating the manufacturing process to which the specific target 400 belongs stored in the storage unit 112 to determine whether or not the specific target 400 is working in the manufacturing process in which the moving body 200 is located when it stops.

[0042] When the above condition 6 is satisfied, the identified target 400 is located close to the equipment causing the mobile body 200 to stop, and the time required to resolve the cause can be shortened. Therefore, it is preferable that the first identifying condition includes this condition 1. The equipment causing the mobile body 200 to stop is identified, for example, by using whether or not an abnormality has been detected. The identifying unit 113 calculates the distance between the position of the equipment causing the stop and the location of the identified target 400, for example, by using the position information of the equipment stored in advance in the storage unit 112.

[0043] When the above condition 7 is satisfied, the worker WO can eliminate the cause of the stop of the moving body 200. Therefore, it is preferable that the first identifying condition includes this condition 7. The identifying unit 113 determines whether or not the cause can be dealt with by using, for example, information indicating a cause that the identified target 400 can deal with, which is stored in advance in the storage unit 112.

[0044] Moreover, the above conditions 1 to 7 and other conditions may be appropriately combined to form the first specific condition. In this embodiment, the above-mentioned condition 1 is adopted.

[0045] In step S220, the notification device 110 notifies the notification target identified in step S210 of an instruction to deal with the abnormal state of the mobile object 200. More specifically, the notification device 110 notifies the terminal device T1 owned by the worker WO who is the notification target and the terminal device T2 installed in the station OF who is the notification target. The notification device 110 notifies, for example, a message saying "The mobile object 200 has stopped, so please take action." In this embodiment, the notification device 110 notifies the mobile object location, which is the location of the mobile object 200 that is the target of the action, together with the instruction to deal with the action.

[0046] According to the notification device 110 of the present embodiment described above, a notification target that satisfies a specific condition is notified of an instruction to deal with the abnormal state of the mobile object 200. Therefore, it is possible to notify an appropriate worker WO or station OF.

[0047] Furthermore, the notification unit 114 notifies the notification target of the mobile body position. Therefore, the notification target can head toward the mobile body 200 without the need to specify the position of the mobile body 200 in which an abnormal state has occurred. Furthermore, when there are multiple mobile bodies 200, the notification target can head toward the mobile body 200 in which an abnormal state has occurred without the need to specify which mobile body 200 in which an abnormal state has occurred.

[0048] The first specific condition also includes a condition that the distance between the moving object location and the current location is equal to or shorter than a first threshold distance, so that the notification unit 114 can notify notification targets that are close to the moving object 200.

[0049] B. Second embodiment: 6 is a flowchart showing an example of a notification process in the second embodiment. The notification process in the second embodiment differs from the first embodiment in that, after an instruction to deal with an abnormal state of the moving body 200 is notified, when the deal is completed, the fact that the deal has been completed is notified. The configuration of the remote control system 10 in the second embodiment is the same as that in the first embodiment.

[0050] In step S230, the notification device 110 determines whether or not the abnormal state of the moving object 200 in which the abnormal state has occurred has been dealt with. More specifically, the notification device 110 determines whether or not a signal indicating the completion of the deal from the worker WO who dealt with the stoppage of the moving object 200 has been received. If the deal with the stoppage of the moving object 200 has been completed, the notification device 110 proceeds to the process of step S240. On the other hand, if the deal with the abnormal state of the moving object 200 has not been completed, the notification device 110 returns to the process of step S230. That is, the process of step S230 is repeated until the deal with the abnormal state of the moving object 200 is completed.

[0051] In step S240, the notification device 110 notifies the notification target identified in step S210 that the handling of the abnormal state of the mobile object 200 has been completed.

[0052] According to the notification device 110 of the second embodiment described above, the notification target is notified that the countermeasure has been completed. Therefore, the notification target who has been notified of the instruction to deal with the abnormal state of the moving object 200 can be prevented from trying to deal with the abnormal state of the moving object 200 that has already been dealt with.

[0053] C. Third embodiment: 7 is a flowchart showing an example of a notification process in the third embodiment. The notification process in the third embodiment differs from the first embodiment in that, when there is no notification target, the moving body 200 is evacuated by remote control. The configuration of the remote control system 10 in the third embodiment is the same as that in the first embodiment.

[0054] In step S200, the notification device 110 determines whether or not a notification target exists. More specifically, the notification device 110 determines whether or not a specific target 400 that satisfies a first specific condition exists. If a specific target 400 that satisfies the first specific condition exists, that is, if a notification target exists, the notification device 110 proceeds to processing in step S210. On the other hand, if a specific target 400 that satisfies the first specific condition does not exist, that is, if a notification target does not exist, the notification device 110 proceeds to processing in step S225.

[0055] In step S225, the notification device 110 instructs the control device 100 to perform control to evacuate the moving object 200 by remote operation. In this embodiment, the notification device 110 instructs the remote automatic control device 130 to perform control to evacuate. Upon receiving the instruction from the notification device 110, the remote automatic control device 130 evacuates the moving object 200 by remote operation.

[0056] According to the notification device 110 of the third embodiment described above, when there is no notification target, control is performed to evacuate the moving object 200 by remote operation. Therefore, even when it is not possible to deal with the abnormal state of the moving object 200, the moving object 200 can be evacuated.

[0057] D. Fourth embodiment: 8 is a schematic diagram showing an overview of a notification device 110B in the fourth embodiment. Notification device 110B in the fourth embodiment differs from the first embodiment in that notification device 110B notifies a notification target of an instruction to remotely control moving object 200 by operating input device 121. The configuration of remote control system 10 in the fourth embodiment is the same as that in the first embodiment.

[0058] The notification device 110B is a device that notifies a notification target identified from the specific target 400 when an abnormal state occurs in the moving object 200 that is remotely operated by the remote manual control device 120 or the remote automatic control device 130.

[0059] Notification device 110B is made up of a computer including a central processing unit (CPU), RAM, ROM, etc., and realizes the functions of identification unit 113B and notification unit 114B by CPU 111B executing a program pre-installed in storage unit 112B of notification device 110B. However, some or all of the functions of these units may be realized by hardware circuits.

[0060] When the moving body 200 operated by remote control stops unintentionally, the identifying unit 113B identifies a notification target that satisfies a predetermined identifying condition from the identified target 400. The identifying condition includes, for example, a condition related to the position of the identified target 400 and a condition related to the work status of the identified target 400. The identifying condition will be described in detail later.

[0061] The notification unit 114B notifies the notification target identified by the identification unit 113B of an instruction to remotely control the moving object 200 by the input device 121.

[0062] 9 is a flowchart showing an example of the notification process. The notification process is a process for notifying a notification target identified by the notification device 110B of an instruction to remotely operate the moving object 200 by the input device 121. In this embodiment, this process is a process that is performed when the moving object 200 remotely operated by the remote automatic control device 130 stops unintentionally.

[0063] In step S310, notification device 110B identifies a notification target. More specifically, notification device 110B identifies a notification target that satisfies a predetermined second identifying condition from the identified target 400. Whether or not the identified target 400 satisfies the second identifying condition can be determined using a detection signal from the external sensor 270 or the internal sensor 280 of the moving object 200 or a detection signal from the detection device 300. As the second identifying condition, for example, any one or more of the following conditions can be adopted.

[0064] <Condition 1> The distance between the installation position of the input device 121 and the location of the specific target 400 is equal to or less than a third threshold distance that is determined in advance. <Condition 2> Does not belong to break time <Condition 3> The worker WO is waiting. <Condition 4> No urgent work is being carried out

[0065] The installation position of the input device 121 in the above condition 1 is stored in, for example, the storage unit 112B. If this condition 1 is not satisfied, the distance between the specific target 400 and the input device 121 is large, so it takes time for the worker WO to reach the input device 121. Therefore, it is preferable that the second specific condition includes this condition 1.

[0066] Moreover, the second specific condition can be a combination of the above conditions 1 to 4 and other conditions as appropriate. In this embodiment, the above-mentioned condition 1 is adopted.

[0067] In step S320, notification device 110B notifies the notification target identified in step S310 of an instruction to remotely operate moving object 200 by input device 121. More specifically, notification device 110B notifies terminal device T1 owned by worker WO who is the notification target and terminal device T2 installed in station OF who is the notification target. In this embodiment, notification device 110B notifies moving object 200 that is the target of remote operation together with an instruction to remotely operate moving object 200.

[0068] According to the notification device 110B of the present embodiment described above, a notification target that satisfies a specific condition is notified of an instruction to operate the input device 121. Therefore, it is possible to notify an appropriate worker WO or station OF.

[0069] The second specific condition also includes that the distance between the installation position and the location position is equal to or shorter than a second threshold distance. Therefore, notification unit 114B can notify a notification target that is close to input device 121.

[0070] E. Fifth embodiment: 10 is a flowchart showing an example of a notification process in the fifth embodiment. The notification process in the fifth embodiment differs from the fourth embodiment in that, when there is no notification target, an instruction is given to deal with the abnormal state of the moving object 200. The configuration of the remote control system 10 in the fifth embodiment is the same as that in the fourth embodiment.

[0071] In step S300, notification device 110B determines whether a notification target exists. More specifically, notification device 110B determines whether a specific target 400 that satisfies the second specific condition exists. If a specific target 400 that satisfies the second specific condition exists, that is, if a notification target exists, notification device 110B proceeds to processing of step S310. On the other hand, if a specific target 400 that satisfies the second specific condition does not exist, that is, if a notification target does not exist, notification device 110B proceeds to processing of step S315.

[0072] Notification device 110B identifies a target in step S315. For example, notification device 110B identifies a target that satisfies the first specific condition described above. The first specific condition is a condition different from the second specific condition.

[0073] In step S325, the notification device 110B notifies the target identified in step S315 of an instruction to deal with the abnormal state of the mobile body 200. More specifically, the notification device 110B notifies the terminal device T1 owned by the target worker WO and the terminal device T2 installed in the target station OF.

[0074] According to the notification device 110B of the fifth embodiment described above, when there is no notification target, an instruction to deal with the abnormal state of the moving object 200 is notified. Therefore, even when remote operation by the input device 121 is not possible, it is possible to deal with the abnormal state of the moving object 200.

[0075] F. Other Embodiments: (F1) In the above-described embodiment, the detection device 300 is a camera. In contrast, the detection device 300 does not have to be a camera, and may be, for example, a LiDAR. In this case, the detection result output by the detection device 300 is three-dimensional point cloud data. The control device 100 and the mobile object control device 250 may calculate the position and orientation of the mobile object by template matching using, for example, the three-dimensional point cloud data as the detection result and the reference point cloud data. For example, ICP (Iteractive Closest Point) or NDT (Normal Distributions Transform) is used as the template matching algorithm. The reference point cloud data may be generated based on, for example, three-dimensional CAD data representing the outer shape of the mobile object 200.

[0076] (F2) In the above-described embodiment, the control device 100 executes the processes from obtaining the position information of the moving body 200 to generating the traveling control signal. In contrast, at least a part of the processes from obtaining the position information of the moving body 200 to generating the traveling control signal may be executed by the moving body 200. For example, the following forms (1) to (3) may be used.

[0077] (1) The control device 100 may acquire position information of the moving body 200, determine a target position to which the moving body 200 should next head, and generate a route from the current location of the moving body 200 represented in the acquired position information to the target position. The control device 100 may generate a route to a target position between the current location and the destination, or may generate a route to the destination. The control device 100 may transmit the generated route to the moving body 200. The moving body control device 250 may generate a traveling control signal so that the moving body 200 travels on the route received from the control device 100, and may control the drive device 220, the steering device 230, and the braking device 240 using the generated traveling control signal.

[0078] (2) The control device 100 may acquire position information of the moving body 200 and transmit the acquired position information to the moving body 200. The moving body control device 250 may determine a target position to which the moving body 200 should next head, generate a route from the current location of the moving body 200 represented in the received position information to the target position, generate a traveling control signal so that the moving body 200 travels on the generated route, and control the drive device 220, the steering device 230, and the braking device 240 using the generated traveling control signal.

[0079] (3) In the above embodiments (1) and (2), the detection results output from the external sensor 270 and the internal sensor 280 may be used for at least one of generating a route and generating a travel control signal. For example, in the above embodiment (1), the control device 100 may acquire the detection results of the external sensor 270 and the internal sensor 280, and may reflect the detection results of the external sensor 270 and the internal sensor 280 in the route when generating a route. In the above embodiment (1), the moving body 200 may acquire the detection results of the external sensor 270 and the internal sensor 280, and may reflect the detection results of the external sensor 270 and the internal sensor 280 in the travel control signal when generating a travel control signal. In the above embodiment (2), the moving body 200 may acquire the detection results of the external sensor 270 and the internal sensor 280, and may reflect the detection results of the external sensor 270 and the internal sensor 280 in the route when generating a route. In the above embodiment (2), the moving body 200 may acquire the detection results of the external sensor 270 and the internal sensor 280, and may reflect the detection results of the internal sensor in the travel control signal when generating a travel control signal.

[0080] (4) In the above-described embodiment, the control device 100 automatically generates a driving control signal to be transmitted to the moving body 200. In contrast, the control device 100 may generate a driving control signal to be transmitted to the moving body 200 in accordance with the operation of an operator located outside the moving body 200. For example, the operator may operate a control device including a display for displaying an image output from the detection device 300, a steering wheel for remotely controlling the moving body 200, an accelerator pedal, a brake pedal, and a communication device for communicating with the control device 100 by wired communication or wireless communication, and the control device 100 may generate a driving control signal in accordance with the operation applied to the control device.

[0081] (F3) In the above-described embodiment, the moving body 200 may have a configuration capable of moving by unmanned driving, and may be in the form of a platform having the configuration described below, for example. Specifically, the moving body 200 may have at least the moving body control device 250, a drive device, a steering device, and a braking device in order to perform the three functions of "running", "turning", and "stopping" by unmanned driving. When the moving body 200 acquires information from the outside for unmanned driving, the moving body 200 may further have a communication device. That is, the moving body 200 capable of moving by unmanned driving may not be equipped with at least a part of interior parts such as a driver's seat and a dashboard, may not be equipped with at least a part of exterior parts such as a bumper and a fender, and may not be equipped with a body shell. In this case, the remaining parts such as the body shell may be attached to the moving body 200 before the moving body 200 is shipped from the factory, or the remaining parts such as the body shell may be attached to the moving body 200 after the moving body 200 is shipped from the factory in a state in which the remaining parts such as the body shell are not attached to the moving body 200. Each part may be attached from any direction such as the upper side, lower side, front side, rear side, right side, or left side of the moving body 200, and each part may be attached from the same direction or from different directions.

[0082] (F4) The moving body 200 may be manufactured by combining a plurality of modules. A module means a unit composed of a plurality of parts grouped according to the part or function of the moving body 200. For example, the platform of the moving body 200 may be manufactured by combining a front module that constitutes the front part of the platform, a central module that constitutes the central part of the platform, and a rear module that constitutes the rear part of the platform. The number of modules that constitute the platform is not limited to three, and may be two or less, or four or more. In addition to or instead of the parts that constitute the platform, parts that constitute parts of the moving body 200 that are different from the platform may be modularized. The various modules may include any exterior parts such as a bumper or a grille, or any interior parts such as a seat or a console. In addition, not limited to the moving body 200, any moving body may be manufactured by combining a plurality of modules. Such a module may be manufactured, for example, by joining a plurality of parts by welding or a fastener, or may be manufactured by integrally molding at least a part of the parts that constitute the module as one part by casting. The molding method for integrally molding a single component, particularly a relatively large component, is also called gigacast or megacast. For example, the front module, the center module, and the rear module may be manufactured using gigacast.

[0083] (F5) Transporting the moving body 200 by using the traveling of the moving body 200 through unmanned driving is also called "self-propelled transport". Also, a configuration for realizing the self-propelled transport is also called a "vehicle remote control autonomous traveling transport system". Also, a production method for producing the moving body 200 by using the self-propelled transport is also called "self-propelled production". In the self-propelled production, for example, at a factory where the moving body 200 is manufactured, at least a part of the transport of the moving body 200 is realized by the self-propelled transport.

[0084] (F6) In the above-described embodiment, the unmanned operation in the automatic mode is performed by the remote automatic control device 130. Not limited to this, the mobile body control device 250 mounted on the mobile body 200 may perform the unmanned operation in the automatic mode. FIG. 11 is a flowchart showing a method in which the mobile body control device 250 causes the mobile body 200 to travel in an unmanned manner. In step S410, the mobile body control device 250 mounted on the mobile body 200 acquires position information of the mobile body 200 using the detection result output from the detection device 300. In step S420, the mobile body control device 250 determines a target position to which the mobile body 200 should next head. In this embodiment, a reference route is stored in advance in the mobile body control device 250. In step S430, the mobile body control device 250 generates a travel control signal for causing the mobile body 200 to travel toward the determined target position. In step S440, the mobile body control device 250 uses the generated travel control signal to control the driving device 220, the steering device 230, and the braking device 240, thereby causing the mobile body 200 to travel at the acceleration and steering angle represented by the travel control signal. The mobile body control device 250 repeats acquisition of position information of the mobile body 200, determination of a target position, generation of a travel control signal, and control of the driving device 220, the steering device 230, and the braking device 240 at a predetermined cycle.

[0085] Note that detection results output from the external sensor 270 and the internal sensor 280 may be used for at least one of generating a route and generating a traveling control signal. For example, the moving body 200 may acquire the detection results of the external sensor 270 and the internal sensor 280, and reflect the detection results of the external sensor 270 and the internal sensor 280 in the route when generating a route. The moving body 200 may acquire the detection results of the external sensor 270 and the internal sensor 280, and reflect the detection results of the external sensor 270 and the internal sensor 280 in the traveling control signal when generating a traveling control signal.

[0086] The mobile object control device 250 may also acquire position information using the detection results of the external sensor 270 and the internal sensor 280 equipped in the mobile object 200, determine a target position to which the mobile object 200 should next move, generate a route from the current location of the mobile object 200 represented in the acquired position information to the target position, generate a travel control signal for traveling along the generated route, and control the drive device 220, the steering device 230, and the braking device 240 using the generated travel control signal. In this case, the mobile object 200 can travel without using any detection results of the detection device 300. The mobile object 200 may acquire a target arrival time and traffic congestion information from outside the mobile object 200, and reflect the target arrival time and traffic congestion information in at least one of the route and the travel control signal.

[0087] (F7) In the above-described embodiment, the abnormal state of the moving body 200 is an abnormal stop of the moving body 200. However, the abnormal state of the moving body 200 may be, for example, a failure occurring in a communication line between the control device 100 and the moving body 200, or a failure occurring in the control device 100 itself.

[0088] (F8) In the above-described embodiment, the remote manual control device 120 and the remote automatic control device 130 are integrally configured as the control device 100. However, the remote manual control device 120 and the remote automatic control device 130 may be configured separately within the control device 100. Also, the control device 100 and the notification devices 110, 110B may be integrally configured. The control device 100 may be configured to include the notification devices 110, 110B, or the notification devices 110, 110B may be configured to include the control device 100.

[0089] (F9) In the above-described embodiment, the specific target 400 includes a plurality of workers WO and a plurality of stations OF. However, the specific target 400 is not limited to this and may include at least one of the workers WO and the stations OF.

[0090] (F10) In the above-described embodiment, the response to the stoppage of the moving object 200 is to recover the moving object 200. However, the response to the stoppage of the moving object 200 is not limited to this, and may be to operate the moving object 200 to move the moving object 200 to repair the cause of the stoppage.

[0091] (F11) In the above-described embodiment, notification device 110, 110B notifies the mobile object location, which is the location of mobile object 200 that is the target of the abnormal condition countermeasure, along with the instruction to counter the abnormal condition. This is not limited to the above, and notification device 110 may notify only the instruction to counter the abnormal condition. In this case, the countermeasure to the abnormal condition is a task that includes identifying mobile object 200 in which the abnormal condition has occurred.

[0092] (F12) In the above-described third embodiment, the notification device 110 instructs the remote automatic control device 130 to perform control for evacuation when there is no notification target. Without being limited to this, the notification device 110 may instruct the remote manual control device 120 to perform control for evacuation. More specifically, the notification device 110 may notify, for example, an instruction to operate the input device 121 to a target that satisfies a second specific condition that is different from the first specific condition.

[0093] (F13) In the above-described fifth embodiment, when there is no notification target, the notification device 110B notifies the specified target of an instruction to handle the stop of the moving body 200. Without being limited to this, the notification device 110B may instruct the remote automatic control device 130 to perform control to evacuate.

[0094] The present disclosure is not limited to the above-mentioned embodiment, and can be realized in various configurations without departing from the spirit of the present disclosure. For example, the technical features in the embodiments corresponding to the technical features in each form described in the Summary of the Invention column can be appropriately replaced or combined in order to solve the above-mentioned problems or to achieve some or all of the above-mentioned effects. Furthermore, if the technical feature is not described as essential in this specification, it can be appropriately deleted. [Explanation of symbols]

[0095] 10... remote operation system, 100... control device, 110, 110B... notification device, 111, 111B... CPU, 112, 112B... memory unit, 113, 113B... identification unit, 114, 114B... notification unit, 120... remote manual control device, 130... remote automatic control device, 200... mobile body, 210... battery, 220... drive device, 230... steering device, 240... braking device, 250... mobile body control device, 260... communication device, 270... external sensor, 280... internal sensor, 300... detection device, 400... specific target, NT... network, OF... guardhouse, T1... terminal device, T2... terminal device, WO... worker

Claims

1. A notification device, comprising: A notification device that receives a signal indicating the occurrence of an abnormal condition of a mobile body that can move by unmanned operation, identifies a notification target that satisfies a predetermined specific condition from specific targets including at least one of a worker and a guardhouse, and notifies the identified notification target of instructions for dealing with the abnormal condition of the mobile body.

2. The notification device according to claim 1, A notification device, wherein the countermeasure does not include controlling the moving object by unmanned driving.

3. The notification device according to claim 2, A notification device notifies the notification target of a moving object location indicating the location of the moving object.

4. The notification device according to claim 3, the specific condition includes a distance between the moving object position and each of the specific objects being equal to or less than a first threshold distance; A notification device that identifies the specific target whose distance between the moving object location and the location is equal to or less than the first threshold distance.

5. The notification device according to claim 2 or 3, The specific condition includes a status of the task to be identified, A notification device that identifies the specific target, which is a work situation in which the action can be taken.

6. The notification device according to claim 5, the specific condition includes that no work of higher urgency than the response to the abnormal state is being performed; A notification device that identifies the specific target that is not performing the highly urgent task.

7. The notification device according to claim 2 or 3, the specific condition includes a manufacturing process in which the moving object is located, A notification device that identifies the specific object associated with the manufacturing process.

8. The notification device according to claim 2 or 3, The specific condition includes a cause of the occurrence of the abnormal state, A notification device that identifies the specific target that can respond to the cause.

9. The notification device according to claim 2 or 3, Upon receiving the completion of the handling, a notification device notifies the notification target of the completion of the handling.

10. A control device, 4. A control device that performs control to cause the moving body to move away by unmanned operation when receiving information from the notification device according to claim 2 or 3 that the notification target does not exist and that satisfies the specific condition.

11. The notification device according to claim 1, The unmanned operation is a manual mode in which the operator operates an input device provided at a location different from the movable body; an automatic mode not depending on an operation by the operator using the input device, the abnormal condition occurs in the automatic mode; The notification device, wherein the action is an operation of the input device.

12. The notification device according to claim 11, further comprising: A storage unit that stores an installation position of the input device, the specific condition includes that a distance between the installation position and a location of each of the specific targets is equal to or less than a second threshold distance determined in advance; A notification device that identifies the specific target whose installation location and location are located at a distance equal to or less than the second threshold distance.

13. The notification device according to claim 11 or 12, The specific condition includes a status of the task to be identified, A notification device that identifies the specific target, which is a task status in which the input device can be operated.

14. 14. The notification device according to claim 13, the specific condition includes that no work of higher urgency than the response to the abnormal state is being performed; A notification device that identifies the specific target that is not performing the highly urgent task.

15. The notification device according to claim 11 or 12, A notification device that, when there is no notification target that satisfies the specific condition, notifies at least a portion of the specific target of instructions for dealing with the abnormal state of the moving body, the action not including control of the moving body by unmanned driving.

16. A control device, 13. A control device that performs control to cause the moving body to evacuate in the automatic mode when a notification that the notification target that satisfies the specific condition does not exist is received from the notification device according to claim 11 or 12.

Images