Notification device and control device

Abstract

A notification device includes a processor configured to: receive a signal indicating an occurrence of an abnormal state of a moving body movable by an unattended operation, and specify a notification target satisfying a predetermined specified condition from one or more specified targets including at least one of a worker and a site; and transmitting, to the specified notification target, a notification for instructing processing of the abnormal state of the moving body.

Description

Technical Field

[0001] The present disclosure relates to a notification device and a control device. Background Art

[0002] A technology for causing a moving body to travel by unattended operation in the process of manufacturing the moving body is known, for example, as described in Japanese Patent Application Publication (Translation of PCT Application) No. 2017-538619. Summary of the invention

[0003] When a moving body stops due to a certain fault while moving by unattended operation, a worker may be requested to take action. When a notification requesting action is sent to all workers and all stations where workers are on standby, for example, the notification may be sent to workers and stations that are not suitable for taking action, such as workers at a location far from the stopped moving body and stations where no workers are on standby.

[0004] The present disclosure may be implemented as follows.

[0005] A first aspect of the present disclosure provides a notification device. The notification device includes a processor, and the processor is configured to: receive a signal indicating the occurrence of an abnormal state of a mobile body that can be moved by unattended operation, and specify a notification target that satisfies a predetermined specified condition from one or more specified targets including at least one of a worker and a site; and send a notification for instructing processing of the abnormal state of the mobile body to the specified notification target. With the notification device according to this aspect, a notification for instructing processing of an abnormal state of a moving body is sent to a notification target that satisfies a specified condition, thereby enabling the notification to be sent to appropriate workers and sites.

[0006] In the notification device according to the above aspect, the processing may not include controlling the moving body by unattended operation. With the notification device according to this aspect, it is possible to instruct, for example, a worker to move the moving body in which an abnormal state has occurred by an on-duty operation.

[0007] In the notification device according to the above aspect, the processor may be configured to notify the notification target of a mobile body position indicating a position of the mobile body. With the notification device according to this aspect, the position of the mobile body can be notified to the notification target.

[0008] In the notification device according to the above aspect, the specified condition may include that the distance between the position of the mobile body and the location of a specified target among one or more specified targets is equal to or less than a predetermined first threshold distance; and the processor may be configured to specify the distance between the position of the mobile body and the location of the specified target is equal to or less than the first threshold distance. With the notification device according to this aspect, a notification can be sent to a notification target located at a short distance from the moving object.

[0009] In the notification device according to the above aspect, the designated condition may include a work status of a designated target among the one or more designated targets; and the processor may be configured to designate the designated target whose work status allows the processing to be performed. With the notification device according to this aspect, since available designated targets are designated, notification for instructing processing of an abnormal state of a mobile body can be sent to a notification target with a high probability of being able to process the abnormal state of the mobile body, for example.

[0010] In the notification device according to the above aspect, the designation condition may include that the designated target is not performing work more urgent than processing of the abnormal state; and the processor may be configured to designate the designated target not performing the more urgent work. With the notification device according to this aspect, a notification can be sent to a notification target who is not performing urgent work.

[0011] In the notification device according to the above aspect, the designated condition may include a manufacturing process in which the moving body is in; and the processor may be configured to designate a designated target related to the manufacturing process among the one or more designated targets. With the notification device according to this aspect, it is possible to send a notification to a notification target belonging to a manufacturing process in which the mobile body is, for example.

[0012] In the notification device according to the above aspect, the designated condition may include a cause of occurrence of the abnormal state; and the processor may be configured to designate a designated target that can handle the cause among the one or more designated targets. With the notification device according to this aspect, a notification can be sent to a notification target that can handle a cause.

[0013] In the notification device according to the above aspect, the processor may be configured to notify the notification target of completion of the processing in response to the processing being completed. With the notification device according to this aspect, it is possible to suppress a notification target that has received a notification instructing to process an abnormal state of a mobile body from attempting to process the abnormal state of the mobile body for which the processing of the abnormal state has been completed.

[0014] The control device may include a processor, and the processor may be configured to, when receiving a notification that there is no notification target satisfying a specified condition from the notification device according to the above aspect, perform evacuation control of the mobile body by an unattended operation. With the control device according to this aspect, even when a worker cannot directly handle an abnormal state of the moving body, the moving body can be evacuated.

[0015] In the above-mentioned notification device, unattended operation may include a manual mode based on operations performed by a worker using an input device set at a position different from that of the moving body, and an automatic mode not based on operations performed by a worker using the input device; the abnormal state may be caused in the automatic mode; and the processing may be the operation of the input device. With the notification device according to this aspect, a notification for instructing the operation of the input device is sent to a notification target that satisfies a specified condition. Therefore, the notification can be sent to appropriate workers and sites.

[0016] According to the above aspect, the notification device may also include a storage unit, which stores the installation location of the input device; the specified condition includes that the distance between the installation location and the location of a specified target among one or more specified targets is equal to or less than a predetermined second threshold distance; and the processor may be configured to specify the installation location and the specified target whose distance between the location is equal to or less than the second threshold distance. With the notification device according to this aspect, a notification can be sent to a notification target that is close to the input device.

[0017] In the notification device according to the above aspect, the designation condition may include an operating state of a designated target among the one or more designated targets; and the processor may be configured to designate the designated target in an operating state that allows the designated target to operate the input device. With the notification device according to this aspect, a notification instructing an unattended operation can be sent to a notification target for which there is a high possibility that the unattended operation can be performed using the input device.

[0018] In the notification device according to the above aspect, the designation condition may include that the designated target is not performing work more urgent than processing of the abnormal state; and the processor may be configured to designate the designated target not performing the more urgent work. With the notification device according to this aspect, a notification can be sent to a notification target who is not performing urgent work.

[0019] In the notification device according to the above aspects, the processor can be configured to send a notification for instructing processing of the abnormal state of the mobile body to at least a part of one or more designated targets when there is no notification target that meets the specified conditions, and the processing does not include controlling the mobile body through unattended operation. With the notification device according to this aspect, even when unattended operation cannot be performed using the input device, the abnormal state of the moving body can be processed.

[0020] The control device may include a processor, and the processor may be configured to, when receiving a notification from the notification device according to the above aspect that there is no notification target satisfying a specified condition, execute evacuation control of the mobile body in an automatic mode. With the control device according to this aspect, even when unattended operation cannot be performed using the input device, the moving body can be evacuated.

[0021] The present disclosure can be implemented in various forms, including, for example, a remote operating system, a mobile body control device, a remote autonomous driving method, a method of manufacturing a mobile body, a method of controlling a control device, a method of controlling a remote operating system, a method of controlling a mobile body, a computer program for implementing these control methods, a non-temporary storage medium for storing computer programs, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described below with reference to the accompanying drawings, wherein like numerals represent like elements, and wherein: Figure 1 shows the configuration of the remote operating system; Figure 2 is a schematic diagram showing an overview of a notification device; Figure 3 is a schematic diagram showing an overview of a moving body; Figure 4 is a flowchart showing an example of a method of causing a moving body to travel; Figure 5 is a flowchart illustrating an example of notification processing; Figure 6 is a flowchart showing an example of notification processing according to the second embodiment; Figure 7 is a flowchart showing an example of notification processing according to the third embodiment; Figure 8 is a schematic diagram showing an overview of a notification device according to a fourth embodiment; Fig. 9 is a flowchart showing an example of notification processing according to the fourth embodiment; Fig.10is a flowchart showing an example of notification processing according to the fifth embodiment; and Fig.11 is a flowchart showing an example of a method of causing a moving object to travel according to another embodiment. DETAILED DESCRIPTION A. First Embodiment

[0023] Figure 1 The configuration of the remote operation system 10 according to the first embodiment is shown. The remote operation system 10 is used to move the moving body 200 by unattended operation in a factory that produces the moving body 200.

[0024] The term "unattended operation" refers to an operation that is not based on a passenger's driving operation. Driving operation refers to an operation related to at least one of "driving", "turning" and "stopping" of the vehicle. Unattended operation is achieved by automatic or manual remote control performed using a device provided outside the vehicle or by autonomous control of the vehicle. Passengers who do not perform driving operations may be on board a vehicle that is traveling by unattended operation. Examples of passengers who do not perform driving operations include persons who simply sit on a seat in the vehicle and persons who perform work different from driving operations (such as assembly, inspection, and operation of switches) while riding in the vehicle. Operations based on a passenger's driving operation are sometimes also referred to as "attended operation."

[0025] In this embodiment, unattended operation includes manual mode and automatic mode. The manual mode is a mode based on operations performed by the worker WO using the input device 121, which will be discussed later. The automatic mode is a mode not based on operations performed by the worker WO using the input device 121.

[0026] The remote operation system 10 includes a control device 100 , a notification device 110 , a moving object 200 , and a detection device 300 .

[0027] The control device 100 is a device that remotely operates the mobile body 200 during the manufacturing process of the mobile body 200. The control device 100 includes a remote manual control device 120 and a remote automatic control device 130. In the present embodiment, the remote manual control device 120 and the remote automatic control device 130 are integrally configured with each other. The control device 100 is provided at a position different from the mobile body 200. The control device 100 and the notification device 110 can communicate with each other. The remote manual control device 120 and the remote automatic control device 130 are both examples of remote operation devices.

[0028] The notification device 110 is a device that notifies the notification target specified from the designated target 400 when receiving a signal indicating the occurrence of an abnormal state of the moving body 200 operated by remote operation of the remote manual control device 120 and the remote automatic control device 130. In the present embodiment, the abnormal state of the moving body 200 refers to a state in which the moving body 200 has experienced an abnormal stop. The abnormal stop refers to an unexpected stop of the moving body 200 being remotely operated. In the present embodiment, the designated target 400 includes a plurality of workers WO and a plurality of sites OF. Each site OF is a place where the worker WO rests or is on standby. The notification device 110 sends a notification to the worker WO via the terminal device T1 of the worker WO, and sends a notification to the site OF via the terminal device T2 installed at the site OF.

[0029] Figure 2 1 is a schematic diagram showing an overview of the notification device 110. The notification device 110 is composed of a computer composed of a central processing unit (CPU), a random access memory (RAM), a read-only memory (ROM), etc., and the functions of the designation unit 113 and the notification unit 114 are realized by the CPU 111 executing a program pre-installed in the storage unit 112 of the notification device 110. However, some or all of the functions of these units can be realized by hardware circuits. The CPU 111 can be, for example, a processor.

[0030] The designation unit 113 designates a notification target satisfying a predetermined designation condition from the designated targets 400 in response to the occurrence of an abnormal state of the mobile body 200 operated by remote operation. Examples of the designated conditions include conditions regarding the location of the designated target 400, the working state of the designated target 400, the manufacturing process in which the mobile body 200 is located, and the cause of the occurrence of the abnormal state of the mobile body 200. The designated conditions will be discussed in detail later. The designation unit 113 designates a designated target 400 at a short distance from the mobile body 200, a designated target 400 in a working state that allows the designated target 400 to process the mobile body 200, a designated target 400 related to the manufacturing process in which the mobile body 200 is located, and a designated target 400 capable of taking action on the cause of the occurrence of the abnormal state of the mobile body 200, which satisfies the designated conditions.

[0031] The notification unit 114 sends a notification to the notification target specified by the specifying unit 113 to instruct processing of the abnormal state of the mobile body 200. The processing of stopping the mobile body 200 is a work that does not include remote operation. In the present embodiment, the processing of the abnormal state of the mobile body 200 includes collecting the mobile body 200 and moving the mobile body 200 by an attended operation.

[0032] Remote manual control device 120 (see Figure 1 ) is a device that remotely operates the moving body 200 according to the operation of the worker WO. That is, the remote manual control device 120 is a device that performs unattended operation in manual mode. More specifically, the remote manual control device 120 remotely operates the moving body 200 according to the driving control signal generated according to the operation of the worker WO on the input device 121. The input device 121 includes a steering wheel, an accelerator pedal, a brake pedal, etc. (not shown). The moving body 200 adopts a wire-controlled steering system, a wire-controlled throttle system, and a wire-controlled braking system. In the remote manual control device 120, the input device 121 and the control unit that generates the control command value can be arranged separately from each other. The driving control signal includes, for example, the driving route (path), position, driving speed, acceleration, and wheel steering angle of the moving body 200. The control unit can be, for example, a processor.

[0033] The remote automatic control device 130 is a device that remotely operates the moving body 200 without the operation of the worker WO. That is, the remote automatic control device 130 is a device that performs unattended operation in the automatic mode. More specifically, the remote automatic control device 130 remotely operates the moving body 200 by generating a control instruction value not according to the operation of the worker WO on the input device 121. The remote automatic control device 130 may be, for example, a processor.

[0034] In the present embodiment, the moving body 200 is a vehicle. More specifically, the moving body 200 is a pure electric vehicle (Battery Electric Vehicle, BEV). The moving body 200 is not limited to a pure electric vehicle, and may be, for example, a gasoline vehicle, a hybrid electric vehicle, or a fuel cell electric vehicle. The moving body 200 is not limited to a vehicle, and may be, for example, an electric vertical lift (a so-called flying vehicle). When the moving body is not a vehicle, the term "travel" used in the present disclosure may be replaced with "move" as appropriate.

[0035] Figure 3 2 is a schematic diagram showing an overview of the mobile body 200. The mobile body 200 includes: a battery 210 which is a power source of the mobile body 200; a driving device 220 which drives the mobile body 200; a steering device 230 which changes the forward direction of the mobile body 200; a braking device 240 which decelerates the mobile body 200; a mobile body control device 250 which controls various parts of the mobile body 200 according to an operation from outside the mobile body 200; a communication device 260 which communicates with the control device 100 through wireless communication; an external sensor 270 which can detect the surroundings of the mobile body 200; and an internal sensor 280 which can acquire information indicating the state of the mobile body 200.

[0036] In this embodiment, the driving device 220 includes a driving motor and a driving wheel rotated by the driving motor. The mobile body control device 250 is composed of two or more electronic control units. When a driver is on the mobile body 200, the mobile body control device 250 can control various parts of the mobile body 200 according to the driver's operation to make the mobile body 200 travel. The mobile body control device 250 can control various parts of the mobile body 200 according to the control command from the control device 100 to make the mobile body 200 travel, regardless of whether there is a driver on the mobile body 200.

[0037] For example, the communication device 260 communicates with the outside of the mobile body 200 through a controller area network (CAN) communication. CAN communication is a communication standard that allows multi-directional transmission and reception. For example, the communication device 260 communicates with the control device 100 connected to the network NT via an access point (not shown) in a factory.

[0038] The external sensor 270 is a sensor capable of detecting an object existing around the moving object 200. The external sensor 270 may include, for example, a vehicle-mounted camera, a radar, a laser radar, and the like.

[0039] The internal sensor 280 is a sensor capable of detecting various physical quantities required for controlling the operation of the mobile body 200. The internal sensor 280 may include, for example, a current sensor, a voltage sensor, a battery temperature sensor, a wheel speed sensor, a steering angle sensor, a Global Navigation Satellite System (GNSS) receiver that receives GNSS information, and the like.

[0040] The detection devices 300 are each a device for detecting the moving body 200, the surroundings of the moving body 200, and the worker WO. In the present embodiment, the detection device 300 is a camera. The detection device 300 is installed in a factory. For example, the detection device 300 communicates with the control device 100 connected to the network NT via an access point (not shown) in the factory. In the present embodiment, the detection device 300 uses the detected information to specify the position, shape, etc. of the moving body 200, the position, shape, etc. of objects present in the area around the moving body 200, and the position, operation, etc. of the worker WO. The detection device 300 transmits the specified information to the control device 100.

[0041] Figure 41 is a flowchart showing a method for the remote operation system 10 to make the mobile body 200 travel according to the present embodiment. In step S110, the control device 100 uses the detection result output by the detection device 300 located outside the mobile body 200 to obtain the position information of the mobile body 200. In the present embodiment, the position information of the mobile body 200 includes X, Y and Z coordinates in the global coordinate system of the factory. The detection device 300 is a camera installed in the factory, and outputs an image as a detection result. The position and orientation of the detection device 300 are pre-adjusted. The control device 100 detects the mobile body 200 from the image obtained by the detection device 300, and obtains the position information of the mobile body 200 in the factory according to the position of the mobile body 200 in the image. For example, the control device 100 can detect the mobile body 200 from the image using a machine learning model that is pre-trained to detect the mobile body 200 from the image.

[0042] In step S120, the control device 100 determines the target position to which the moving body 200 should go next. In the present embodiment, the target position is represented by X, Y, and Z coordinates in the global coordinate system of the plant. The control device 100 pre-stores a reference path as the path along which the moving body 200 should travel. The path is represented by a node indicating a departure position, a node indicating a passing point, a node indicating a destination position, and links connecting the nodes. The control device 100 uses the position information of the moving body 200 and the reference path to determine the target position to which the moving body 200 should go next. The control device 100 determines a target position on the reference path that is ahead of the current position of the moving body 200.

[0043] In step S130, the control device 100 generates a travel control signal for causing the mobile body 200 to travel toward the determined target position. In the present embodiment, the travel control signal includes the acceleration and steering angle of the mobile body 200 as parameters. The control device 100 calculates the travel speed of the mobile body 200 according to the conversion of the position of the mobile body 200, and compares the calculated travel speed with the target speed of the mobile body 200 determined in advance. The control device 100 determines the acceleration so that the mobile body 200 accelerates when the travel speed is lower than the target speed, and determines the acceleration so that the mobile body 200 decelerates when the travel speed is higher than the target speed. The control device 100 determines the steering angle and the acceleration so that the mobile body 200 does not deviate from the reference path when the mobile body 200 is located on the reference path, and determines the steering angle and the acceleration so that the mobile body 200 returns to the reference path when the mobile body 200 is not located on the reference path (in other words, when the mobile body 200 has deviated from the reference path).

[0044] In S140, the control device 100 transmits the generated travel control signal to the moving body 200. The control device 100 repeatedly acquires the position information of the moving body 200, determines the target position, generates the travel control signal, and transmits the travel control signal at a predetermined cycle.

[0045] In step S150, the mobile body control device 250 installed on the mobile body 200 receives the travel control signal from the control device 100. In step S160, the mobile body control device 250 uses the received travel control signal to control the driving device 220, the steering device 230 and the braking device 240 so that the mobile body 200 travels according to the acceleration and the steering angle indicated by the travel control signal. The mobile body control device 250 repeatedly receives the travel control signal at a predetermined cycle, and controls the driving device 220, the steering device 230 and the braking device 240. According to the remote operation system 10 of the present embodiment, the mobile body 200 can be driven by remote control, and therefore, the mobile body 200 can be moved without using a conveying device such as a crane or a conveyor.

[0046] The position of the mobile object 200 and the orientation of the mobile object 200 are estimated using a captured image acquired by the detection device 300 provided at a different position from the mobile object 200. The position of the mobile object 200 can be acquired, for example, by calculating the coordinates of the measurement point of the mobile object 200 in the image coordinate system using the outer shape of the mobile object 200 detected from the captured image, and converting the calculated coordinates into coordinates in the global coordinate system. The orientation of the mobile object 200 can be estimated, for example, using an optical flow method based on the orientation of the motion vector of the mobile object 200 calculated from the position change of the feature point of the mobile object 200 between frames of the captured image. The orientation of the mobile object 200 can be calculated, for example, using the output result from a yaw rate sensor or the like mounted on the mobile object 200.

[0047] For example, the shape of the mobile body 200 included in the captured image can be detected by inputting the captured image into a detection model using artificial intelligence. Examples of detection models include, for example, a trained machine learning model that has been trained to achieve one of semantic segmentation and instance segmentation. As a machine learning model, for example, a convolutional neural network (hereinafter referred to as "CNN") trained by supervised learning with an input of a learning data set can be used. The learning data set includes, for example, a plurality of training images including the mobile body 200 and a correct answer label indicating whether each area in the training image indicates an area of ​​the mobile body 200 or an area outside the mobile body 200. During the training of the CNN, the parameters of the CNN are preferably updated by back propagation (error back propagation method) to reduce the error between the result output from the detection model and the correct answer label.

[0048] Figure 5 is a flowchart showing an example of notification processing. The notification processing is a process in which the notification device 110 sends a notification to a specified notification target to instruct processing of an abnormal state of the mobile body 200. In the present embodiment, the processing is performed when the mobile body 200 operated by remote operation is unexpectedly stopped. For example, the notification device 110 performs the notification processing when a signal indicating that the mobile body 200 is unexpectedly stopped is received from the mobile body 200, or a signal indicating that the mobile body 200 is unexpectedly stopped is received from the control device 100 or the detection device 300. In the present embodiment, the processing of the abnormal state is a processing that does not include controlling the mobile body 200 by unattended operation.

[0049] In step S210, the notification device 110 specifies a notification target. More specifically, the notification device 110 specifies a notification target that satisfies a first specified condition from the specified targets 400. Whether the specified target 400 satisfies the first specified condition may be determined using a detection signal from the external sensor 270 or the internal sensor 280 of the mobile body 200 or a detection signal from the detection device 300. For example, one or more of the following conditions may be adopted as the first specified condition. The term "specify" used herein may be replaced with "select".

[0050] Condition 1 The distance between the moving body position, which is the position of the moving body 200 , and the existing position, which is the position of the designated target 400 , is equal to or smaller than a first threshold distance determined in advance. Condition 2 Not resting Condition 3 Worker WO is on standby. Condition 4 There is no emergency work in progress. Condition 5 The moving body 200 is working in the manufacturing process in which it is stopped. Condition 6 The distance between the device causing the moving body 200 to stop and the location is equal to or less than a predetermined second threshold distance. Condition 7 Able to handle the reason why the moving body 200 stops

[0051] The position of the moving body in the above-mentioned condition 1 is specified, for example, by a GNSS receiver or a detection device 300 of the moving body 200. The position is specified, for example, by a GNSS receiver or a detection device 300 of the terminal device T1. Information indicating the position of the station OF is stored, for example, in the storage unit 112. When the condition 1 is not satisfied, the distance between the designated target 400 and the moving body 200 is long, so it takes time for the worker WO to reach the moving body 200. Therefore, the first designated condition preferably includes the condition 1.

[0052] When the above-mentioned condition 2 is not satisfied, the worker WO is on break, and it is not desirable to instruct the worker WO to process the stop of the moving body 200 outside of working hours. Therefore, the first designation condition preferably includes condition 2. The designation unit 113 determines whether the designated target 400 is on break using, for example, the shift information of the designated target 400 stored in the storage unit 112.

[0053] When the above-mentioned condition 3 is not satisfied, the worker WO does not exist at the station OF, so even if a notification is sent to instruct processing of the stop of the moving body 200, the notification may not be received. Therefore, the first specified condition preferably includes condition 3. The specifying unit 113 determines whether the worker WO is on standby at the station OF, for example, using information indicating whether the worker WO enters or exits the station OF detected by the detection device 300.

[0054] The emergency work in the above-mentioned condition 4 is, for example, a work that is more urgent than processing the abnormal state of the mobile body 200, such as a work related to safety, such as processing of an abnormality. When condition 4 is not satisfied, the designated target 400 is performing an emergency work that cannot be interrupted, so even if a notification is sent to instruct processing of the stop of the mobile body 200, the stop of the mobile body 200 may not be processed. Therefore, the first designated condition preferably includes condition 4. This allows the designation unit 113 to designate a designated target 400 that is not performing an emergency work. The designation unit 113 determines whether the designated target 400 is performing an emergency work, for example, using information indicating the operation of the designated target 400 designated by the detection device 300.

[0055] When the above-mentioned condition 5 is satisfied, there is a high possibility that the designated target 400 is located at a position short from the moving body 200. There is also a high possibility that the designated target 400 can solve the cause of the stop of the moving body 200. Therefore, the first designation condition preferably includes the condition 5. The designation unit 113 determines whether the designated target 400 is working in the manufacturing process in which the moving body 200 is located when the moving body 200 stops, using, for example, information indicating the manufacturing process to which the designated target 400 belongs, which is stored in the storage unit 112.

[0056] When the above-mentioned condition 6 is satisfied, the designated target 400 is located at a position short from the device causing the mobile body 200 to stop, and thus the time required to solve the cause can be shortened. Therefore, the first designated condition preferably includes condition 6. The device causing the mobile body 200 to stop is designated using, for example, whether an abnormality is detected. The designation unit 113 calculates the distance between the position of the device as the cause and the position of the designated target 400 using, for example, the position information of the device pre-stored in the storage unit 112.

[0057] When the above condition 7 is satisfied, the worker WO can solve the cause of the stop of the moving body 200. Therefore, the first designation condition preferably includes condition 7. The designation unit 113 determines whether the cause can be handled using, for example, information indicating causes that the designated target 400 can handle, which is pre-stored in the storage unit 112.

[0058] The first specified condition can be set by appropriately combining the above conditions 1 to 7 and other conditions. In this embodiment, the above condition 1 is adopted.

[0059] In step S220, the notification device 110 sends a notification to the notification target specified in step S210 to instruct to process the abnormal state of the moving body 200. More specifically, the notification device 110 sends a notification to the terminal device T1 of the worker WO as the notification target and the terminal device T2 installed at the site OF as the notification target. The notification device 110 sends, for example, a message indicating "the moving body 200 has stopped, please take action." In the present embodiment, the notification device 110 sends a notification of the moving body position as the position of the moving body 200 to be processed together with an instruction to process the moving body 200.

[0060] With the notification device 110 according to the above-described embodiment, a notification for instructing processing of an abnormal state of the moving body 200 is sent to a notification target that satisfies a specified condition. Therefore, the notification can be sent to appropriate workers WO and sites OF.

[0061] The notification unit 114 notifies the notification target of the mobile body position. Therefore, the notification target can go to the mobile body 200 without taking the trouble to specify the position of the mobile body 200 in the abnormal state. When there are a plurality of mobile bodies 200, furthermore, the notification target can go to the mobile body 200 in the abnormal state without taking the trouble to specify which mobile body 200 among the mobile bodies 200 is the mobile body 200 in the abnormal state.

[0062] The first specified condition includes that the distance between the moving body position and the present position is equal to or less than the first threshold distance. Therefore, the notification unit 114 can send a notification to a notification target that is at a short distance from the moving body 200. B. Second Embodiment

[0063] Figure 6 1 is a flowchart showing an example of notification processing according to the second embodiment. The notification processing according to the second embodiment is different from the notification processing according to the first embodiment in that after the notification for instructing the processing is performed, the notification of the completion of the processing of the abnormal state of the mobile body 200 is performed. The configuration of the remote operation system 10 according to the second embodiment is the same as the configuration of the remote operation system 10 according to the first embodiment.

[0064] In step S230, the notification device 110 determines whether the processing of the abnormal state of the moving body 200 in which the abnormality has occurred is completed. More specifically, the notification device 110 determines whether a signal of the completion of the processing by the worker WO who has processed the stop of the moving body 200 is received. When the processing of the stop of the moving body 200 is completed, the notification device 110 proceeds to the processing in step S240. On the other hand, when the processing of the abnormal state of the moving body 200 is not completed, the notification device 110 returns to the processing in step S230. That is, the processing in S230 is repeatedly performed until the processing of the abnormal state of the moving body 200 is completed.

[0065] In step S240 , the notification device 110 notifies the notification target specified in step S210 that the processing of the abnormal state of the moving object 200 is completed.

[0066] With the notification device 110 according to the second embodiment described above, the notification target is notified of the completion of the process. Therefore, the notification target having received the notification instructing to process the abnormal state of the mobile body 200 can be suppressed from attempting to process the abnormal state of the mobile body 200 having completed the process of the abnormal state. C. Third Embodiment

[0067] Figure 71 is a flowchart showing an example of the notification process according to the third embodiment. The notification process according to the third embodiment is different from the notification process according to the first embodiment in that when there is no notification target, the mobile body 200 is evacuated by remote operation. The configuration of the remote operation system 10 according to the third embodiment is the same as the configuration of the remote operation system 10 according to the first embodiment.

[0068] In step S200, the notification device 110 determines whether there is a notification target. More specifically, the notification device 110 determines whether there is a designated target 400 that satisfies the first designated condition. When there is a designated target 400 that satisfies the first designated condition, that is, when there is a notification target, the notification device 110 proceeds to the processing in step S210. On the other hand, when there is no designated target 400 that satisfies the first designated condition, that is, when there is no notification target, the notification device 110 proceeds to the processing in step S225.

[0069] In step S225, the notification device 110 instructs the control device 100 to perform control to evacuate the mobile body 200 by remote operation. In the present embodiment, the notification device 110 instructs the remote automatic control device 130 to perform control to evacuate the mobile body 200. When receiving the instruction from the notification device 110, the remote automatic control device 130 evacuates the mobile body 200 by remote operation.

[0070] With the notification device 110 according to the third embodiment described above, when there is no notification target, control for evacuating the mobile body 200 by remote operation is performed. Therefore, even when the abnormal state of the mobile body 200 cannot be processed, the mobile body 200 can be evacuated. D. Fourth Embodiment

[0071] Figure 8 1 is a schematic diagram showing an overview of a notification device 110B according to a fourth embodiment. The notification device 110B according to the fourth embodiment is different from the notification device according to the first embodiment in that a notification instructing a remote operation of a mobile body 200 by an operation of an input device 121 is sent to a notification target. The configuration of the remote operation system 10 according to the fourth embodiment is the same as that of the remote operation system 10 according to the first embodiment.

[0072] The notification device 110B is a device that notifies a notification target designated from the designated target 400 in response to the occurrence of an abnormal state of the moving body 200 operated by remote operation of the remote manual control device 120 and the remote automatic control device 130 .

[0073] The notification device 110B is composed of a computer composed of a central processing unit (CPU), a random access memory (RAM), a read only memory (ROM), etc., and the functions of the designation unit 113B and the notification unit 114B are realized by the CPU 111B executing a program preinstalled in the storage unit 112B of the notification device 110B. However, some or all of the functions of these units may be realized by hardware circuits.

[0074] The designation unit 113B designates a notification target satisfying a predetermined designation condition from the designation target 400 in response to an unexpected stop of the moving body 200 operated by the remote operation. Examples of the designation condition include, for example, a condition regarding the position of the designation target 400 and a condition regarding the working state of the designation target 400. The designation condition will be discussed in detail later.

[0075] The notification unit 114B transmits a notification to instruct a remote operation of the moving body 200 using the input device 121 to the notification target specified by the specifying unit 113B.

[0076] Fig. 9 1 is a flowchart showing an example of the notification process. The notification process is a process in which the notification device 110B sends a notification to instruct the remote operation of the moving body 200 using the input device 121. In the present embodiment, this process is performed when the moving body 200 operated by the remote operation of the remote automatic control device 130 is unexpectedly stopped.

[0077] In step S310, the notification device 110B specifies a notification target. More specifically, the notification device 110B specifies a notification target that satisfies a predetermined second specified condition from the specified targets 400. Whether the specified target 400 satisfies the second specified condition may be determined using a detection signal from the external sensor 270 or the internal sensor 280 of the mobile body 200 or a detection signal from the detection device 300. For example, one or more of the following conditions may be adopted as the second specified condition.

[0078] Condition 1 The distance between the installation position of the input device 121 and the location as the location of the designated target 400 is equal to or smaller than a third threshold distance determined in advance. Condition 2 Not resting Condition 3 Worker WO is on standby. Condition 4 There is no emergency work in progress.

[0079] The installation position of the input device 121 in the above condition 1 is stored in the storage unit 112B, for example. When condition 1 is not satisfied, the distance between the designated target 400 and the input device 121 is long, so it takes time for the worker WO to reach the input device 121. Therefore, the second designated condition preferably includes condition 1.

[0080] The second specified condition can be set by appropriately combining the above conditions 1 to 4 and other conditions. In this embodiment, the above condition 1 is adopted.

[0081] In step S320, the notification device 110B sends a notification to the notification target specified in step S310 to instruct the remote operation of the moving body 200 using the input device 121. More specifically, the notification device 110B sends the notification to the terminal device T1 of the worker WO as the notification target and the terminal device T2 installed at the site OF as the notification target. In the present embodiment, the notification device 110B sends the notification of the moving body 200 to be remotely operated together with the instruction to remotely operate the moving body 200.

[0082] With the notification device 110B according to the above-described embodiment, a notification for instructing the operation of the input device 121 is sent to a notification target that satisfies a specified condition. Therefore, the notification can be sent to an appropriate worker WO and site OF.

[0083] The second specified condition includes that the distance between the installation location and the existing location is equal to or less than the second threshold distance. Therefore, the notification unit 114B can send a notification to a notification target that is at a short distance from the input device 121. E. Fifth Embodiment

[0084] Fig.10 1 is a flowchart showing an example of notification processing according to the fifth embodiment. The notification processing according to the fifth embodiment is different from the notification processing according to the fourth embodiment in that, when there is no notification target, an instruction to process the abnormal state of the mobile body 200 is sent. The configuration of the remote operation system 10 according to the fifth embodiment is the same as the configuration of the remote operation system 10 according to the fourth embodiment.

[0085] In step S300, the notification device 110B determines whether there is a notification target. More specifically, the notification device 110B determines whether there is a designated target 400 that satisfies the second designated condition. When there is a designated target 400 that satisfies the second designated condition, that is, when there is a notification target, the notification device 110B proceeds to the processing in step S310. On the other hand, when there is no designated target 400 that satisfies the second designated condition, that is, when there is no notification target, the notification device 110B proceeds to the processing in step S315.

[0086] In step S315, the notification device 110B specifies a target. For example, the notification device 110B specifies a target that satisfies the first specifying condition described above. The first specifying condition is different from the second specifying condition.

[0087] In step S325, the notification device 110B sends a notification to the target specified in step S315 to instruct processing of the abnormal state of the moving body 200. More specifically, the notification device 110B sends a notification to the terminal device T1 of the target worker WO and the terminal device T2 installed at the target site OF.

[0088] With the notification device 110B according to the fifth embodiment described above, when there is no notification target, notification is made to instruct processing of the abnormal state of the mobile body 200. Therefore, even when remote operation cannot be performed using the input device 121, the abnormal state of the mobile body 200 can be processed. F. Other Embodiments

[0089] (F1) In the above-mentioned embodiments, the detection device 300 is a camera. However, the detection device 300 may not be a camera, and may be, for example, a LiDAR. In this case, the detection device 300 outputs three-dimensional point cloud data as a detection result. The control device 100 and the mobile body control device 250 may, for example, calculate the position and orientation of the mobile body by using the three-dimensional point cloud data as the detection result and the reference point cloud data for template matching. For example, iterative closest point (ICP) and normal distribution transform (NDT) are used as algorithms for template matching. The reference point cloud data may be generated, for example, based on three-dimensional computer-aided design (CAD) data representing the outer shape of the mobile body 200.

[0090] (F2) In the above-described embodiment, the control device 100 performs the processing from acquiring the position information of the moving body 200 to generating the travel control signal. In contrast, the moving body 200 may perform at least a part of the processing from acquiring the position information of the moving body 200 to generating the travel control signal. For example, the following aspects (1) to (3) may be used:

[0091] (1) The control device 100 can obtain the position information of the moving body 200, determine the target position to which the moving body 200 should go next, and generate a path from the current position of the moving body 200 represented by the obtained position information to the target position. The control device 100 can generate a path to the target position between the current position and the destination position, or can also generate a path to the destination position. The control device 100 can transmit the generated path to the moving body 200. The moving body control device 250 can generate a travel control signal for causing the moving body 200 to travel on the path received from the control device 100, and use the generated travel control signal to control the drive device 220, the steering device 230, and the brake device 240.

[0092] (2) The control device 100 can obtain the position information of the moving body 200 and transmit the obtained position information to the moving body 200. The moving body control device 250 can determine the target position that the moving body 200 should go to next, generate a path from the current position of the moving body 200 indicated by the received position information to the target position, generate a travel control signal for causing the moving body 200 to travel on the generated path, and control the driving device 220, the steering device 230, and the braking device 240 using the generated travel control signal.

[0093] (3) In the above aspects (1) and (2), the detection results output from the external sensor 270 or the internal sensor 280 can be used for at least one of the generation of the path and the generation of the travel control signal. For example, in the above aspect (1), the control device 100 can obtain the detection results from the external sensor 270 or the internal sensor 280, and reflect the detection results from the external sensor 270 or the internal sensor 280 in the path when generating the path. In the above aspect (1), the mobile body 200 can obtain the detection results from the external sensor 270 or the internal sensor 280, and reflect the detection results from the external sensor 270 or the internal sensor 280 in the travel control signal when generating the travel control signal. In the above aspect (2), the mobile body 200 can obtain the detection results from the external sensor 270 or the internal sensor 280, and reflect the detection results from the external sensor 270 or the internal sensor 280 in the path when generating the path. In the above aspect (2), the moving body 200 may obtain the detection result from the external sensor 270 or the internal sensor 280 and reflect the detection result from the external sensor 270 or the internal sensor 280 in the traveling control signal when generating the traveling control signal.

[0094] (4) In the above-described embodiment, the control device 100 automatically generates the travel control signal to be transmitted to the mobile body 200. Instead, the control device 100 may generate the travel control signal to be transmitted to the mobile body 200 according to the operation of an operator located outside the mobile body 200. For example, the operator may operate a manipulation device including a display that displays an image output from the detection device 300, a steering wheel, an accelerator pedal, and a brake pedal for remotely operating the mobile body 200, and a communication device that communicates with the control device 100 through wired communication or wireless communication, and the control device 100 may generate the travel control signal according to the operation performed on the manipulation device.

[0095] (F3) In the above embodiment, the mobile body 200 only needs to include components that enable it to move by unattended operation, and the mobile body 200 can be in the form of a platform including the following components, for example. Specifically, in order to realize the three functions of "driving", "turning" and "stopping" by unattended operation, the mobile body 200 only needs to include at least a mobile body control device 250, a drive device, a steering device and a braking device. In order for the mobile body 200 to obtain information for unattended operation from the outside, the mobile body 200 only needs to include a communication device. That is, at least a portion of internal components such as a driver's seat or an instrument panel may not be installed on the mobile body 200 that can be moved by unattended operation, at least a portion of external components such as a bumper or a fender may not be installed on the mobile body 200, or the body shell may not be installed on the mobile body 200. In this case, the remaining components such as the vehicle body shell may be mounted to the mobile body 200 before the mobile body 200 is shipped from the factory, or the remaining components such as the vehicle body shell may be mounted to the mobile body 200 after the mobile body 200 is shipped from the factory in a state where the remaining components such as the vehicle body shell are not mounted on the mobile body 200. These components may be mounted to the mobile body 200 from any direction (including the upper side, the lower side, the front side, the rear side, the right side, and the left side), and may be mounted from the same direction or from different directions.

[0096] (F4) The mobile body 200 can be manufactured by combining a plurality of modules. Each module refers to a unit composed of a plurality of components grouped according to the area or function of the mobile body 200. For example, the platform of the mobile body 200 can be manufactured by combining a front module constituting the front part of the platform, a central module constituting the central part of the platform, and a rear module constituting the rear part of the platform. The number of modules constituting the platform is not limited to 3, but can be less than 2 or more than 4. In addition to or instead of the components constituting the platform, the module may include components constituting a portion of the mobile body 200 that is different from the platform. Various modules may include any external component (such as a bumper or grille) or any internal component (such as a seat or console). Not only the mobile body 200, but also any aspect of the mobile body can be manufactured by combining a plurality of modules. For example, such a module can be manufactured by welding, joining a plurality of components using a fixture, etc., or can be manufactured by casting at least a portion of the components constituting the module to be integrally molded as a single component. The method of integrally molding a single component (especially a relatively large component) is called gigacasting or megacasting. For example, the front module, the central module, and the rear module may be manufactured by integral casting.

[0097] (F5) Transporting the moving body 200 by driving the moving body 200 with unattended operation is called "self-propelled transportation". The configuration for realizing self-propelled transportation is called "vehicle remote control autonomous driving transportation system". The method of producing the moving body 200 using self-propelled transportation is called "self-propelled production". In self-propelled production, at least a part of the transportation of the moving body 200 in the factory that manufactures the moving body 200 is realized by, for example, self-propelled transportation.

[0098] (F6) In the above-described embodiment, the unattended operation in the automatic mode is performed by the remote automatic control device 130. Without being limited thereto, the unattended operation in the automatic mode may be performed by the moving body control device 250 installed in the moving body 200. Fig.114 is a flowchart showing a method in which the mobile body control device 250 causes the mobile body 200 to travel by unattended operation. In step S410, the mobile body control device 250 installed on the mobile body 200 uses the detection result output from the detection device 300 to obtain the position information of the mobile body 200. In step S420, the mobile body control device 250 determines the target position to which the mobile body 200 should go next. In this embodiment, the reference path is pre-stored in the mobile body control device 250. In step S430, the mobile body control device 250 generates a travel control signal for causing the mobile body 200 to travel toward the determined target position. In step S440, the mobile body control device 250 uses the generated travel control signal to control the drive device 220, the steering device 230 and the brake device 240 so that the mobile body 200 travels according to the acceleration and steering angle indicated by the travel control signal. The mobile body control device 250 repeatedly acquires position information of the mobile body 200 at a predetermined cycle, determines a target position, generates a travel control signal, and controls the driving device 220 , the steering device 230 , and the braking device 240 .

[0099] The detection result output from the external sensor 270 or the internal sensor 280 may be used for at least one of the generation of the route and the generation of the travel control signal. For example, the mobile body 200 may obtain the detection result from the external sensor 270 or the internal sensor 280, and reflect the detection result from the external sensor 270 or the internal sensor 280 in the route when generating the route. The mobile body 200 may obtain the detection result from the external sensor 270 or the internal sensor 280, and reflect the detection result from the external sensor 270 or the internal sensor 280 in the travel control signal when generating the travel control signal.

[0100] The mobile body control device 250 can obtain position information using the detection results from the external sensor 270 or the internal sensor 280 of the mobile body 200, determine the target position to which the mobile body 200 should go next, generate a path from the current position of the mobile body 200 indicated by the acquired position information to the target position, generate a travel control signal for traveling along the generated path, and use the generated travel control signal to control the drive device 220, the steering device 230, and the brake device 240. In this case, the mobile body 200 can travel without using the detection results from the detection device 300 at all. The mobile body 200 can obtain the target arrival time and the traffic congestion information from outside the mobile body 200, and reflect the target arrival time or the traffic congestion information in at least one of the path and the travel control signal.

[0101] (F7) In the above embodiment, the abnormal state of the moving body 200 corresponds to an abnormal stop of the moving body 200. Without being limited thereto, the abnormal state of the moving body 200 may correspond to, for example, a failure in a communication line between the control device 100 and the moving body 200 or a failure in the control device 100 itself.

[0102] (F8) In the above-described embodiment, the remote manual control device 120 and the remote automatic control device 130 are integrally configured with each other as the control device 100. Without being limited thereto, the remote manual control device 120 and the remote automatic control device 130 may be separately configured in the control device 100. The control device 100 and the notification device 110, 110B may be integrally configured with each other. The control device 100 may be configured to include the notification device 110, 110B, or the notification device 110, 110B may be configured to include the control device 100.

[0103] (F9) In the above-described embodiment, the designated target 400 includes a plurality of workers WO and a plurality of sites OF. Without being limited thereto, the designated target 400 may include at least one of the workers WO and the sites OF.

[0104] (F10) In the above-described embodiment, the process of stopping the moving body 200 is to collect the moving body 200. Without being limited thereto, the process of stopping the moving body 200 may be to repair the cause of the stopping of the moving body 200 and to move the moving body 200.

[0105] (F11) In the above-described embodiment, the notification device 110, 110B sends a notification of the position of the moving body 200 as the position of the moving body 200 in the abnormal state to be processed together with an instruction to process the abnormal state. Without being limited to this, the notification device 110 may send only an instruction to process the abnormal state. In this case, the processing of the abnormal state is a work including designation of the moving body 200 in the abnormal state.

[0106] (F12) In the above embodiment, when there is no notification target, the notification device 110 instructs the remote automatic control device 130 to perform control for evacuating the mobile body 200. Without being limited to this, the notification device 110 may instruct the remote manual control device 120 to perform control for evacuating the mobile body 200. More specifically, the notification device 110 may send a notification for instructing the operation of the input device 121 to a target that satisfies a second specified condition different from the first specified condition, for example.

[0107] (F13) In the fifth embodiment, when there is no notification target, the notification device 110B transmits a notification to the designated target to instruct the stop process of the moving body 200. Without being limited to this, the notification device 110B may instruct the remote automatic control device 130 to execute control to evacuate the moving body 200.

[0108] The present disclosure is not limited to the above-mentioned embodiments, and can be implemented by various configurations without departing from the spirit of the present disclosure. For example, the technical features in the various embodiments corresponding to the technical features of the various aspects described in the summary of the invention can be appropriately replaced or combined to solve the above-mentioned problems, or to achieve some or all of the above-mentioned effects. When a technical feature is not described as being necessary in this article, such a feature can be omitted as appropriate.

Claims

1. A notification device, characterized in that The invention comprises a processor, wherein the processor is configured to: receiving a signal indicating occurrence of an abnormal state of a movable body movable by unattended operation, and specifying a notification target satisfying a predetermined specified condition from one or more specified targets including at least one of a worker and a station; as well as A notification for instructing to handle the abnormal state of the moving body is transmitted to the designated notification target.

2. The notification device according to claim 1, characterized in that The processing does not include controlling the moving object by the unattended operation.

3. The notification device according to claim 2, characterized in that: The processor is configured to notify the notification target of a moving body position indicating a position of the moving body.

4. The notification device according to claim 3, characterized in that: The designated condition includes that the distance between the position of the mobile body and the position of the designated target among the one or more designated targets is equal to or less than a predetermined first threshold distance; and The processor is configured to designate a designated target whose distance between the position of the moving object and the current position is equal to or smaller than the first threshold distance.

5. The notification device according to claim 2 or 3, characterized in that: The specified condition includes a working state of a specified target among the one or more specified targets; and The processor is configured to specify that its operating state allows the specified target of the process to be performed.

6. The notification device according to claim 5, characterized in that: The designated condition includes that the designated target is not performing work that is more urgent than processing of the abnormal state; and The processor is configured to designate a designated target that is not performing more urgent work.

7. The notification device according to claim 2 or 3, characterized in that: The specified conditions include the manufacturing process in which the moving body is in; and The processor is configured to specify a specified target related to the manufacturing process among the one or more specified targets.

8. The notification device according to claim 2 or 3, characterized in that: The specified condition includes the cause of the abnormal state; and The processor is configured to designate a designated target among the one or more designated targets that is capable of processing the cause.

9. The notification device according to claim 2 or 3, characterized in that: The processor is configured to notify the notification target of completion of the process in response to the process being completed.

10. A control device, characterized in that A processor is included, the processor being configured to, when receiving a notification from the notification device according to claim 2 or 3 that there is no notification target satisfying the specified condition, execute evacuation control of the mobile body by unattended operation.

11. The notification device according to claim 1, characterized in that: The unattended operation includes: a manual mode based on an operation performed by the worker using an input device provided at a position different from that of the moving body, and an automatic mode that is not based on operations performed by said worker using said input device; The abnormal state is caused in the automatic mode; and The processing is the operation of the input device.

12. The notification device according to claim 11, characterized in that Also included is a storage unit, the storage unit stores the installation position of the input device, wherein: The designated condition includes that the distance between the installation location and the location of a designated target among the one or more designated targets is equal to or less than a predetermined second threshold distance; and The processor is configured to designate a designated target where the distance between the installation location and the located location is equal to or less than the second threshold distance.

13. The notification device according to claim 11 or 12, characterized in that: The specified condition includes a working state of a specified target among the one or more specified targets; and The processor is configured to designate a designated target in an operating state that allows the designated target to operate the input device.

14. The notification device according to claim 13, characterized in that: The designated condition includes that the designated target is not performing work that is more urgent than processing of the abnormal state; and The processor is configured to designate a designated target that is not performing more urgent work.

15. The notification device according to claim 11 or 12, characterized in that: The processor is configured to, when there is no notification target that satisfies the specified condition, send a notification to at least a part of the one or more specified targets to instruct them to process the abnormal state of the mobile body, and the processing does not include controlling the mobile body through the unattended operation.

16. A control device, characterized in that A processor is included, the processor being configured to, when receiving a notification from the notification device according to claim 11 or 12 that there is no notification target satisfying the specified condition, execute evacuation control of the mobile body in the automatic mode.

Images