Vehicle management system

Abstract

To provide a vehicle management system capable of properly managing multiple self-propelled transport vehicles. [Solution] The vehicle management system according to the present disclosure is a vehicle management system comprising a plurality of vehicles that can be moved by unmanned operation and a management device that manages the operation of the plurality of vehicles, wherein each vehicle has an external sensor that detects obstacles in its surroundings, and the management device comprises a communication unit that receives the detection results of the external sensor and a control unit that controls the operation of the plurality of vehicles moving in the first area based on the detection results of the external sensor and an image captured by a first surveillance camera that photographs the first area, and the control unit controls the operation of the plurality of vehicles moving in the first area based on the image captured by the first surveillance camera when a malfunction is detected in each of the external sensors of the plurality of vehicles moving in the first area.

Description

Technical Field

[0001] The present disclosure relates to a vehicle management system.

Background Art

[0002] In recent years, development of a management system for remotely managing the operations of a plurality of vehicles (i.e., self-propelled transport vehicles) capable of moving by autonomous driving has been underway. For example, Patent Document 1 discloses an apparatus for remotely controlling a moving body.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a management system for remotely managing the operations of a plurality of self-propelled transport vehicles, appropriate management of each self-propelled transport vehicle is required assuming a case where a camera or the like mounted on each self-propelled transport vehicle is temporarily malfunctioning due to the influence of disturbances.

[0005] The present disclosure has been made in view of the above background, and an object thereof is to provide a vehicle management system capable of appropriately managing a plurality of self-propelled transport vehicles.

Means for Solving the Problems

[0006] The vehicle management system according to this disclosure comprises a plurality of vehicles that can be moved by unmanned operation, and a management device that manages the operation of the plurality of vehicles, wherein each of the vehicles has an external sensor that detects surrounding obstacles, and the management device comprises a communication unit that receives the detection results of the external sensor, and a control unit that controls the operation of the plurality of vehicles moving in the first area based on the detection results of the external sensor and images captured by a first surveillance camera that photographs the first area, and the control unit controls the operation of the plurality of vehicles moving in the first area based on images captured by the first surveillance camera when a malfunction is detected in each of the external sensors of the plurality of vehicles moving in the first area. The vehicle management system according to this disclosure, when it detects a malfunction in each of the external sensors of the plurality of vehicles moving in the first area, determines that it is not a failure of the external sensor but a temporary malfunction due to external disturbances such as strong winds or rain, and continues to move the plurality of vehicles moving in the first area based on images captured by a first surveillance camera that photographs the first area without stopping their operation. This improves the productivity of the vehicles and reduces costs. In other words, the vehicle management system described in this disclosure can enable the proper management of multiple vehicles. [Effects of the Invention]

[0007] This disclosure provides a vehicle management system that enables the proper management of multiple self-propelled transport vehicles. [Brief explanation of the drawing]

[0008] [Figure 1] This is a schematic diagram showing a part of the vehicle management system according to Embodiment 1. [Figure 2] This is a block diagram showing the control system of the vehicle management system according to Embodiment 1. [Figure 3] This is a block diagram showing the control system of the vehicle management system according to Embodiment 2. [Figure 4] This is a schematic diagram showing a part of the vehicle management system according to Embodiment 2. [Figure 5]This is a diagram illustrating the vehicle's driving control. [Figure 6] This is a control block diagram illustrating example 1 of the driving control system. [Figure 7] This is a flowchart to explain example 1 of the driving control system. [Figure 8] This is a control block diagram illustrating example 2 of the driving control system. [Figure 9] This is a flowchart to explain example 2 of the driving control system. [Modes for carrying out the invention]

[0009] The following describes specific embodiments to which the present invention is applied, with reference to the drawings. However, the present invention is not limited to the following embodiments. Also, for clarity of explanation, the following description and drawings have been simplified as appropriate.

[0010] <Embodiment 1> Figure 1 is a schematic diagram showing a part of the vehicle management system 50 according to Embodiment 1. The vehicle management system 50 is applied, for example, in a vehicle manufacturing plant that manufactures vehicles 100. In the example in Figure 1, the vehicle management system 50 monitors and manages multiple vehicles 100 moving in outdoor area A1 and indoor area A2, respectively. Note that Figure 1 shows an XY Cartesian coordinate system for illustrative purposes.

[0011] As shown in Figure 1, the vehicle management system 50 comprises a server 200, surveillance cameras 321 and 322, and a plurality of vehicles 100. Each vehicle 100 is, for example, a self-propelled vehicle capable of moving under its own power during the manufacturing process. In other words, each vehicle 100 is, for example, a vehicle capable of moving unmanned during the manufacturing process.

[0012] Each vehicle 100 is a vehicle in its pre-completion state. Each vehicle 100 moves in a convoy along a predetermined route (path), and is manufactured into a finished product by having workers (not shown) or robots (not shown) perform predetermined tasks in each manufacturing process area. These predetermined tasks include assembling parts, operating switches, welding, and inspection.

[0013] Each vehicle 100 is also equipped with external sensors. These external sensors include, for example, an onboard camera that photographs the area around the vehicle, radar that detects obstacles around the vehicle, and LiDAR (Light Detection and Ranging) that detects obstacles around the vehicle. Each vehicle 100 uses these external sensors to detect the vehicle 100 in front of (or behind) the vehicle during platooning, and to detect other obstacles around the vehicle. For example, each vehicle 100 uses an onboard camera, which is one of the external sensors, to photograph the vehicle 100 in front of (or behind) the vehicle during platooning. Each vehicle 100 has a communication function and transmits the detection results of the external sensors (e.g., data such as captured images) to the server 200 via the network 500.

[0014] Surveillance camera 321 captures an outdoor area A1 where multiple vehicles 100 move sequentially. In the example in Figure 1, surveillance camera 321 consists of three cameras. Surveillance camera 322 captures an indoor area A2 where multiple vehicles 100 move sequentially. In the example in Figure 1, surveillance camera 322 consists of three cameras.

[0015] Server 200 monitors and manages a plurality of vehicles 100 moving in Area A1 based on the captured images of surveillance camera 321 that captures Area A1 and the detection results such as the captured images received from the respective external sensors of the plurality of vehicles 100 moving in Area A1. Also, server 200 monitors and manages a plurality of vehicles 100 moving in Area A2 based on the captured images of surveillance camera 322 that captures Area A2 and the detection results such as the captured images received from the respective external sensors of the plurality of vehicles 100 moving in Area A2. For example, server 200 monitors whether each vehicle 100 is moving on the planned route or not based on the acquired captured images and the like, and monitors whether the work is being carried out on each vehicle 100 as planned or not. Also, server 200 controls the movement of each vehicle 100 while estimating the position of each vehicle 100 based on the acquired captured images and the like. Server 200 is a vehicle management device and is also referred to as a vehicle management system by itself.

[0016] Subsequently, the control system of vehicle management system 50 will be described using FIG. 2. FIG. 2 is a block diagram showing the control system of vehicle management system 50.

[0017] As shown in FIG. 2, server 200 includes at least a communication device 205, an analysis unit 207, an analysis unit 208, and a remote control unit 210. Each vehicle 100 includes a vehicle control device 110, an actuator group 120, a communication device 130, and an external sensor 140. The external sensor 140 has three types of sensors: an in-vehicle camera 141, a radar 142, and a LiDAR 143. However, the external sensor 140 is not limited to three types of sensors and may have less than three types of sensors or four or more types of sensors. Note that server 200 is not limited to being physically configured by a single device and may be configured by a plurality of distributed devices. For example, analysis units 207 and 208 may be physically configured by a single device or may be configured by separate devices.

[0018] In server 200, communication device 205 communicates with surveillance cameras 321, 322 and each vehicle 100 via network 500. For example, communication device 205 receives data such as captured images from surveillance cameras 321, 322 and each vehicle 100, and transmits information regarding vehicle control to each vehicle 100.

[0019] Analysis unit 207 analyzes each of the captured images of surveillance camera 321 that captures area A1 and the captured image of surveillance camera 322 that captures area A2. Specifically, communication device 205 receives data such as captured images from each of surveillance camera 321 that captures area A1 and surveillance camera 322 that captures area A2. Then, by analyzing each of the received captured image of surveillance camera 321 and the received captured image of surveillance camera 322, for example, the outer shape of each vehicle 100 shown in the captured image and the surrounding environment of each vehicle 100 are identified. Thereby, it is possible to identify the position and orientation of each vehicle 100 in area A1 and the driving state of each vehicle 100 in area A1. Also, it is possible to identify the position and orientation of each vehicle 100 in area A2 and the driving state of each vehicle 100 in area A2.

[0020] The analysis unit 208 analyzes the detection results of the external sensors 140 mounted on each vehicle 100 moving in area A1, and the detection results of the external sensors 140 mounted on each vehicle 100 moving in area A2. Specifically, the communication device 205 receives the detection results (data such as captured images) of the external sensors 140 mounted on each vehicle 100 moving in area A1, and the detection results (data such as captured images) of the external sensors 140 mounted on each vehicle 100 moving in area A2. The analysis unit 208 then analyzes the received detection results to identify, for example, the external shape of the vehicle in front of (or behind) each vehicle 100, and the surrounding environment of each vehicle 100. This makes it possible to identify the distance between each vehicle 100 and the vehicle in front of (or behind) it in area A1, the position and orientation of each vehicle 100 in area A1, and the driving state of each vehicle 100 in area A1. Furthermore, it is possible to determine the distance between each vehicle 100 in area A2 and the vehicle in front of (or behind), the position and orientation of each vehicle 100 in area A2, and the driving status of each vehicle 100 in area A2.

[0021] The monitoring unit 209 monitors each vehicle 100 moving through areas A1 and A2 based on the analysis results from the analysis units 207 and 208. For example, the monitoring unit 209 monitors whether each vehicle 100 is moving while maintaining a predetermined distance between vehicles, whether each vehicle 100 is moving along the planned route, and whether each vehicle 100 is performing the planned work.

[0022] The remote control unit 210 remotely controls each vehicle 100 by transmitting vehicle control information to each vehicle 100 based on the monitoring results from the monitoring unit 209 (including the location information of each vehicle 100). Specifically, the communication device 205 transmits vehicle control information instructed by the remote control unit 210 to each vehicle 100. In each vehicle 100, the communication device 130 receives vehicle control information from the server 200, and the vehicle control device 110 drives the vehicle by controlling it with the actuator group 120 according to the received vehicle control information.

[0023] In this context, each vehicle 100, especially while moving through the outdoor area A1, is susceptible to external disturbances such as strong winds and rain. Specifically, the external sensors 140 (at least one of the on-board camera 141, radar 142, and LiDAR 143) mounted on each vehicle 100 while moving through area A1 may temporarily malfunction due to the effects of these disturbances. For example, the on-board camera 141 mounted on each vehicle 100 while moving through area A1 may temporarily malfunction if its view is obstructed by rain or if it is violently shaken by strong winds.

[0024] Therefore, if the monitoring unit 209 detects a malfunction in each of the external sensors 140 of multiple vehicles 100 moving in area A1 (preferably when it detects a malfunction in all of the external sensors 140 of multiple vehicles 100 moving in area A1), it determines that the malfunction is not due to a failure of the external sensors 140, but rather to a temporary malfunction caused by external disturbances such as strong winds or rain. The monitoring unit 209 may have a function to detect a malfunction in each of the external sensors 140 of multiple vehicles 100 moving in area A1, or it may acquire a signal indicating a malfunction in the external sensor 140 from each vehicle 100.

[0025] In this case, the remote control unit 210 remotely controls the multiple vehicles 100 moving in area A1 to continue moving, based on the images captured by the surveillance camera 321 (more specifically, the results of the analysis of the captured images), without using the detection results of the external sensors 140 of each vehicle 100 moving in area A1. As a result, the vehicle management system 50 improves the productivity of the vehicles and reduces costs. In other words, the vehicle management system 50 can achieve proper management of the multiple vehicles 100. At this time, the remote control unit 210 may reduce the travel speed of the multiple vehicles 100 traveling in area A1, or increase the distance between the multiple vehicles 100 traveling in area A1.

[0026] Furthermore, the monitoring unit 209 may determine that it is possible to continue moving each vehicle 100 in area A1 if it detects a malfunction in fewer than a predetermined number of sensors (for example, fewer than two types) among the three types of sensors included in the external sensors 140 of each vehicle 100 moving in area A1: the onboard camera 141, radar 142, and LiDAR 143. In this case, the remote control unit 210 remotely controls the multiple vehicles 100 moving in area A1 to continue moving based on the detection results of the sensors among the three types of sensors included in the external sensors 140 of each vehicle 100 that have not been detected as malfunctioning, and the images captured by the monitoring camera 321. At this time, the remote control unit 210 may reduce the travel speed of the multiple vehicles 100 traveling in area A1, or increase the distance between the multiple vehicles 100 traveling in area A1.

[0027] On the other hand, if the monitoring unit 209 detects a malfunction in a predetermined number of sensors (for example, two or more) among the three types of sensors included in the external sensors 140 of each vehicle 100 moving in area A1—namely, the onboard camera 141, radar 142, and LiDAR 143—the monitoring unit 209 may determine that it is impossible to continue moving each vehicle 100 in area A1. In this case, the remote control unit 210 remotely controls the multiple vehicles 100 moving in area A1 to stop their movement. The criteria for determining whether it is possible to continue moving each vehicle 100 in area A1, as well as the driving speed and distance between vehicles, may be arbitrarily determined according to the environment of area A1. For example, if workers are permitted to enter area A1, the criteria for determining whether it is possible to continue moving each vehicle 100 in area A1 may be set strictly, taking safety into consideration.

[0028] Thus, when the vehicle management system 50 according to this disclosure detects a malfunction in the external sensor 140 of each of the multiple vehicles 100 moving in area A1, it determines that the malfunction is not due to a failure of the external sensor 140, but rather to a temporary malfunction caused by external disturbances such as strong winds or rain. At this time, the vehicle management system 50 according to this disclosure continues to move the multiple vehicles 100 moving in area A1 without stopping their operation, based on the images captured by the surveillance camera 321 that photographs area A1. This improves the productivity of the vehicles and reduces costs. In other words, the vehicle management system 50 according to this disclosure can achieve proper management of multiple vehicles 100.

[0029] In this disclosure, the monitoring unit 209 has provided an example in which it determines that the malfunction of each of the external sensors 140 of multiple vehicles 100 moving in area A1 is not due to a failure of the external sensors 140, but rather to a temporary malfunction caused by external disturbances such as strong winds or rain. However, the disclosure is not limited to this example. For example, even if the monitoring unit 209 detects a malfunction of each of the external sensors 140 of multiple vehicles 100 moving in a predetermined area other than area A1 (including area A2) where external disturbances are likely to occur, it may also determine that the malfunction is not due to a failure of the external sensors 140, but rather to a temporary malfunction caused by external disturbances such as strong winds or rain. In this case, the remote control unit 210 does not use the detection results of each of the external sensors 140 of the multiple vehicles 100 moving in the predetermined area, but instead remotely controls the multiple vehicles 100 moving in the predetermined area based on images captured by a surveillance camera photographing the predetermined area (more specifically, the results of analyzing the captured images) to continue moving the multiple vehicles 100 moving in the predetermined area. The criteria for determining whether or not it is possible to continue moving each vehicle 100 may be set for each area.

[0030] Similarly, the monitoring unit 209 may determine that it is possible to continue moving each vehicle 100 moving within a predetermined area if it detects a malfunction in fewer than a predetermined number of sensors (for example, fewer than two types) among the three types of sensors included in the external sensor 140 of each vehicle 100 moving within a predetermined area: the onboard camera 141, radar 142, and LiDAR 143. In this case, the remote control unit 210 remotely controls the multiple vehicles 100 moving within the predetermined area to continue moving based on the detection results of the sensors among the three types of sensors included in the external sensor 140 of each vehicle 100 that have not been detected as malfunctioning, and the images captured by the monitoring camera 321. At this time, the remote control unit 210 may reduce the travel speed of the multiple vehicles 100 traveling within the predetermined area, or increase the distance between the multiple vehicles 100 traveling within the predetermined area.

[0031] On the other hand, if the monitoring unit 209 detects a malfunction in a predetermined number of sensors (for example, two or more) among the three types of sensors included in the external sensors 140 of each vehicle 100 moving within a predetermined area—namely, the onboard camera 141, radar 142, and LiDAR 143—the monitoring unit 209 may determine that it is impossible to continue moving each vehicle 100 within the predetermined area. In this case, the remote control unit 210 remotely controls the multiple vehicles 100 moving within the predetermined area to stop their movement. The criteria for determining whether it is possible to continue moving each vehicle 100 within the predetermined area, as well as the driving speed and distance between vehicles within the predetermined area, may be arbitrarily determined according to the environment of the predetermined area. For example, if workers are permitted to enter the predetermined area, the criteria for determining whether it is possible to continue moving each vehicle 100 within the predetermined area may be set strictly, taking safety into consideration.

[0032] Furthermore, the vehicle management system 50 may also include a database that stores multiple combinations of the operating status of the external sensor 140 installed on the vehicle 100, the area in which the vehicle 100 moves, and the control content for the operation of the vehicle 100. In this case, the remote control unit 210 extracts the control content for the operation of each vehicle 100 from the database according to the operating status of the external sensor 140 installed on each vehicle 100 moving in area A1, and remotely controls the operation of each vehicle 100 moving in area A1 according to the extracted control content. The operating status of the external sensor 140 refers, for example, to information regarding the presence or absence of malfunctions in one or more types of sensors included in the external sensor 140.

[0033] <Embodiment 2> Figure 3 is a block diagram showing the control system of the vehicle management system 50 according to Embodiment 2. As shown in Figure 3, the server 200 provided in the vehicle management system 50 according to Embodiment 2 further includes an output unit 211. The other configurations of the vehicle management system 50 according to Embodiment 2 are the same as those of the vehicle management system 50 according to Embodiment 1, so their description is omitted.

[0034] Figure 4 is a schematic diagram showing a part of the vehicle management system 50 according to Embodiment 2. For example, if the monitoring unit 209 detects a malfunction in the external sensor 140 of some of the multiple vehicles 100 moving in area A1 (one vehicle in the example of Figure 4), it determines that there is a high possibility of a failure of the external sensor 140, rather than a temporary malfunction due to external disturbances such as strong winds or rain. Note that the malfunction of the external sensor 140 also includes the malfunction of some of the multiple types of sensors included in the external sensor 140. In this case, the output unit 211 outputs information indicating that the detected external sensor 140 may be at risk of failure. The output unit 211 outputs the information indicating that the detected external sensor 140 may be at risk of failure via a speaker, displays it on a monitor, or notifies a worker in area A1 of their mobile terminal. This allows the worker to repair the vehicle 100 equipped with the potentially faulty external sensor 140 or guide it to an evacuation site. Furthermore, the remote control unit 210 may remotely control the vehicle 100 equipped with an external sensor 140 that is judged to be highly likely to malfunction, to bring it to an emergency stop, evacuate it to a safe place, or have it travel along an alternative route.

[0035] Thus, when the vehicle management system 50 according to this disclosure detects a malfunction in some of the external sensors 140 of multiple vehicles 100 moving in area A1, it determines that there is a high probability of a failure of the external sensors 140, rather than a temporary malfunction due to external disturbances such as strong winds or rain. At this time, the vehicle management system 50 according to this disclosure outputs information to the external sensor 140 in which the malfunction was detected that there is a possibility of failure. As a result, workers can repair the vehicle 100 equipped with the potentially malfunctioning external sensor 140 or guide it to an evacuation site.

[0036] In the present disclosure, when the monitoring unit 209 detects a malfunction of some of the external sensors 140 of a plurality of vehicles 100 moving in area A1 and determines that it is not a temporary malfunction due to disturbances such as strong winds or rain but that there is a high possibility of a malfunction of the external sensor 140, this has been described as an example, but it is not limited thereto. For example, when the monitoring unit 209 detects a malfunction of some of the external sensors 140 of a plurality of vehicles 100 moving in a predetermined area (including area A2) other than area A1, it may also determine that it is not a temporary malfunction due to disturbances such as strong winds or rain but that there is a high possibility of a malfunction of the external sensor 140. In this case, the output unit 211 outputs information indicating that there is a possibility of a malfunction in the detected external sensor 140. Further, the remote control unit 210 may remotely control the vehicle 100 equipped with the external sensor 140 determined to have a high possibility of malfunction to cause an emergency stop, evacuate to an evacuation location, or travel along a detour route. Note that the criterion for determining whether the external sensor 140 is malfunctioning may be set for each area.

[0037] Hereinafter, in a system 50 related to the manufacture of a vehicle including the vehicle management system according to the present disclosure, an example of travel control for controlling the travel of the vehicle 100 will be described.

[0038] <A. Travel Control Example 1> FIG. 5 is a conceptual diagram showing the configuration of the system 50 in travel control example 1. The system 50 includes one or more vehicles 100 as moving bodies, a server 200, and one or more external sensors 300.

[0039] When the moving body is other than a vehicle, the expressions "vehicle" and "car" in the present disclosure can be appropriately replaced with "moving body", and the expression "travel" can be appropriately replaced with "movement".

[0040] Vehicle 100 is configured to operate autonomously. "Autonomous operation" means operation without the operation of a passenger. Operation refers to operations related to at least one of the following: "going," "turning," or "stopping" of vehicle 100. Autonomous operation is achieved by automatic or manual remote control using a device located outside vehicle 100, or by autonomous control of vehicle 100. Vehicle 100 operating autonomously may have passengers on board who do not perform operation. Passengers who do not perform operation include, for example, people simply sitting in the seats of vehicle 100, or people performing tasks other than operation, such as assembly, inspection, or operating switches, while on board vehicle 100. Operation by a passenger is sometimes called "manned operation."

[0041] In this specification, "remote control" includes "fully remote control," in which all operations of the vehicle 100 are completely determined from outside the vehicle 100, and "partial remote control," in which some operations of the vehicle 100 are determined from outside the vehicle 100. Furthermore, "autonomous control" includes "fully autonomous control," in which the vehicle 100 autonomously controls its own operations without receiving any information from external devices, and "partial autonomous control," in which the vehicle 100 autonomously controls its own operations using information received from external devices.

[0042] In this embodiment, system 50 is used in a factory FC where vehicle 100 is manufactured. The reference coordinate system of the factory FC is the global coordinate system GC. That is, any position within the factory FC is represented by X, Y, Z coordinates in the global coordinate system GC. The factory FC comprises a first location PL1 and a second location PL2. The first location PL1 and the second location PL2 are connected by a track TR on which vehicle 100 can travel. Multiple external sensors 300 are installed along the track TR in the factory FC. The position of each external sensor 300 in the factory FC is pre-adjusted. Vehicle 100 moves from the first location PL1 to the second location PL2 via the track TR by unmanned operation.

[0043] Figure 6 is a block diagram showing the configuration of system 50. The vehicle 100 includes a vehicle control device 110 for controlling various parts of the vehicle 100, an actuator group 120 including one or more actuators driven under the control of the vehicle control device 110, and a communication device 130 for communicating wirelessly with external devices such as a server 200. The actuator group 120 includes actuators for a drive system to accelerate the vehicle 100, actuators for a steering system to change the direction of travel of the vehicle 100, and actuators for a braking system to decelerate the vehicle 100.

[0044] The vehicle control device 110 is composed of a computer comprising a processor 111, a memory 112, an input / output interface 113, and an internal bus 114. The processor 111, the memory 112, and the input / output interface 113 are connected via the internal bus 114 to enable bidirectional communication. The input / output interface 113 is connected to an actuator group 120 and a communication device 130. The processor 111 implements various functions, including those of a vehicle control unit 115, by executing a program PG1 stored in the memory 112.

[0045] The vehicle control unit 115 drives the vehicle 100 by controlling the actuator group 120. The vehicle control unit 115 can drive the vehicle 100 by controlling the actuator group 120 using the driving control signal received from the server 200. The driving control signal is a control signal for driving the vehicle 100. In this embodiment, the driving control signal includes the acceleration and steering angle of the vehicle 100 as parameters. In other embodiments, the driving control signal may include the speed of the vehicle 100 as a parameter instead of, or in addition to, the acceleration of the vehicle 100.

[0046] The server 200 is composed of a computer comprising a processor 201, memory 202, an input / output interface 203, and an internal bus 204. The processor 201, memory 202, and input / output interface 203 are connected via the internal bus 204 to enable bidirectional communication. A communication device 205 for communicating with various external devices of the server 200 is connected to the input / output interface 203. The communication device 205 can communicate with the vehicle 100 via wireless communication and can communicate with each external sensor 300 via wired or wireless communication. The processor 201 implements various functions, including those of a remote control unit 210, by executing a program PG2 stored in memory 202.

[0047] The remote control unit 210 acquires detection results from sensors, generates a driving control signal to control the actuator group 120 of the vehicle 100 using the detection results, and transmits the driving control signal to the vehicle 100, thereby driving the vehicle 100 by remote control. In addition to the driving control signal, the remote control unit 210 may also generate and output control signals to control various auxiliary equipment and actuators that operate various devices such as wipers, power windows, and lamps, which are provided on the vehicle 100. In other words, the remote control unit 210 may operate these various devices and auxiliary equipment by remote control. For example, the remote control unit 210 may include the functions of the analysis unit 207, 208 and the monitoring unit 209, which are shown separately from the remote control unit 210 in Figure 2.

[0048] The external sensor 300 is a sensor located outside the vehicle 100. In this embodiment, the external sensor 300 is a sensor that captures the vehicle 100 from outside the vehicle 100. The external sensor 300 is equipped with a communication device (not shown) and can communicate with other devices such as the server 200 via wired or wireless communication. The external sensor 300 includes the functions of the surveillance cameras 321 and 322 shown in Figure 2.

[0049] Specifically, the external sensor 300 is comprised of a camera. The camera, acting as the external sensor 300, captures an image including the vehicle 100 and outputs the captured image as the detection result.

[0050] Figure 7 is a flowchart showing the processing procedure for vehicle 100's driving control in an example of driving control. In the processing procedure shown in Figure 7, the processor 201 of the server 200 functions as a remote control unit 210 by executing program PG2. The processor 111 of the vehicle 100 functions as a vehicle control unit 115 by executing program PG1.

[0051] In step S110, the processor 201 of the server 200 acquires vehicle position information of the vehicle 100 using the detection results output from the external sensor 300. The vehicle position information is the position information that forms the basis for generating the driving control signal. In this embodiment, the vehicle position information includes the position and orientation of the vehicle 100 in the global coordinate system GC of the factory FC. Specifically, in step S110, the processor 201 acquires vehicle position information using the captured image acquired from the camera, which is the external sensor 300.

[0052] In detail, in step S110, the processor 201 detects the outline of the vehicle 100 from the captured image, calculates the coordinates of the vehicle 100's positioning point in the coordinate system of the captured image, i.e., the local coordinate system, and obtains the position of the vehicle 100 by converting the calculated coordinates to coordinates in the global coordinate system GC. The outline of the vehicle 100 included in the captured image can be detected, for example, by inputting the captured image into a detection model DM that utilizes artificial intelligence. The detection model DM is prepared, for example, within or outside the system 50 and pre-stored in the memory 202 of the server 200. Examples of the detection model DM include a pre-trained machine learning model that has been trained to implement either semantic segmentation or instance segmentation. As this machine learning model, for example, a convolutional neural network (CNN) trained by supervised learning using a training dataset can be used. The training dataset includes, for example, multiple training images containing vehicle 100, and labels indicating whether each region in the training images represents vehicle 100 or something other than vehicle 100. During CNN training, it is preferable to update the CNN parameters using backpropagation to reduce the error between the output result of the detection model DM and the labels. Furthermore, the processor 201 can obtain the orientation of vehicle 100 by, for example, using the optical flow method, estimating it based on the direction of the vehicle 100's movement vector calculated from the positional changes of the vehicle 100's feature points between frames of the captured images.

[0053] In step S120, the processor 201 of the server 200 determines the next target location that the vehicle 100 should head to. In this embodiment, the target location is represented by X, Y, Z coordinates in the global coordinate system GC. The memory 202 of the server 200 pre-stores a reference route RR, which is the path that the vehicle 100 should travel. The route is represented by a node indicating the starting point, nodes indicating waypoints, a node indicating the destination, and links connecting each node. The processor 201 uses the vehicle position information and the reference route RR to determine the next target location that the vehicle 100 should head to. The processor 201 determines the target location on the reference route RR beyond the vehicle 100's current location.

[0054] In step S130, the processor 201 of the server 200 generates a driving control signal to drive the vehicle 100 toward the determined target position. The processor 201 calculates the vehicle's speed from the change in the vehicle's position and compares the calculated speed with the target speed. Overall, the processor 201 determines the acceleration so that the vehicle 100 accelerates if the speed is lower than the target speed, and determines the acceleration so that the vehicle 100 decelerates if the speed is higher than the target speed. Furthermore, if the vehicle 100 is located on the reference path RR, the processor 201 determines the steering angle and acceleration so that the vehicle 100 does not deviate from the reference path RR, and if the vehicle 100 is not located on the reference path RR, in other words, if the vehicle 100 has deviated from the reference path RR, the processor 201 determines the steering angle and acceleration so that the vehicle 100 returns to the reference path RR.

[0055] In step S140, the processor 201 of the server 200 transmits the generated driving control signal to the vehicle 100. The processor 201 repeats the process of acquiring the position of the vehicle 100, determining the target position, generating the driving control signal, and transmitting the driving control signal at predetermined intervals.

[0056] In step S150, the processor 111 of the vehicle 100 receives a driving control signal transmitted from the server 200. In step S160, the processor 111 of the vehicle 100 controls the actuator group 120 using the received driving control signal, thereby driving the vehicle 100 at the acceleration and steering angle represented by the driving control signal. The processor 111 repeats the reception of the driving control signal and the control of the actuator group 120 at a predetermined cycle. According to the system 50 in this example, the vehicle 100 can be driven by remote control, and the vehicle 100 can be moved without using conveying equipment such as a crane or a conveyor.

[0057] <B:Driving Control Example 2> FIG. 8 is an explanatory diagram showing a schematic configuration of the system 50v in Driving Control Example 2. In this example, the system 50v is different from Driving Control Example 1 in that it does not include the server 200. Also, the vehicle 100v in the configuration can travel by autonomous control of the vehicle 100v. For other configurations, they are the same as above unless otherwise specified.

[0058] In this example, the processor 111v of the vehicle control device 110v functions as the vehicle control unit 115v by executing the program PG1 stored in the memory 112v. The vehicle control unit 115v acquires the output result from the sensor, generates a driving control signal using the output result, and outputs the generated driving control signal to operate the actuator group 120, thereby enabling the vehicle 100v to travel by autonomous control. In this example, in addition to the program PG1, a detection model DM and a reference route RR are stored in the memory 112v in advance.

[0059] FIG. 9 is a flowchart showing the processing procedure of the driving control of the vehicle 100v in Example 2. In the processing procedure of FIG. 9, the processor 111v of the vehicle 100v functions as the vehicle control unit 115v by executing the program PG1.

[0060] In step S210, the processor 111v of the vehicle control device 110v acquires vehicle position information using the detection result output from the camera, which is an external sensor 300. In step S220, the processor 111v determines the target position to which the vehicle 100v should next go. In step S230, the processor 111v generates a driving control signal to drive the vehicle 100v toward the determined target position. In step S240, the processor 111v controls the actuator group 120 using the generated driving control signal to drive the vehicle 100v according to the parameters expressed in the driving control signal. The processor 111v repeats the acquisition of vehicle position information, determination of the target position, generation of the driving control signal, and control of the actuators at a predetermined cycle. According to the system 50v in this example, the vehicle 100v can be driven by autonomous control of the vehicle 100v without remote control of the vehicle 100v by the server 200.

[0061] YY: Other examples of driving control (YY1) In the above example, the external sensor 300 is a camera. However, the external sensor 300 does not have to be a camera; for example, it could be a LiDAR (Light Detection And Ranging). In this case, the detection result output by the external sensor 300 may be 3D point cloud data representing the vehicle 100. In this case, the server 200 and the vehicle 100 may acquire vehicle position information by template matching using the 3D point cloud data as the detection result and pre-prepared reference point cloud data.

[0062] In (YY2) Driving control example 1, the server 200 performs the processing from acquiring vehicle position information to generating driving control signals. In contrast, the vehicle 100 may perform at least a part of the processing from acquiring vehicle position information to generating driving control signals. For example, the following forms (1) to (3) may be used.

[0063] (1) The server 200 may acquire vehicle location information, determine the next target location that vehicle 100 should head to, and generate a route from the vehicle 100's current location, as shown in the acquired vehicle location information, to the target location. The server 200 may generate a route to the target location between the current location and the destination, or it may generate a route to the destination. The server 200 may transmit the generated route to vehicle 100. Vehicle 100 may generate a driving control signal so that vehicle 100 travels along the route received from the server 200, and may use the generated driving control signal to control the actuator group 120.

[0064] (2) The server 200 may acquire vehicle location information and transmit the acquired vehicle location information to the vehicle 100. The vehicle 100 may determine the next target location to which the vehicle 100 should go, generate a route from the vehicle 100's current location shown in the received vehicle location information to the target location, generate a driving control signal so that the vehicle 100 travels along the generated route, and control the actuator group 120 using the generated driving control signal.

[0065] (3) In the embodiments of (1) and (2) above, the vehicle 100 is equipped with internal sensors, and the detection results output from the internal sensors may be used in at least one of the generation of a route and the generation of a driving control signal. The internal sensors are sensors mounted on the vehicle 100 and correspond to the external sensor 140 shown in Figure 2. The internal sensors may include, for example, sensors that detect the motion state of the vehicle 100, sensors that detect the operating state of each part of the vehicle 100, and sensors that detect the environment around the vehicle 100. Specifically, the internal sensors may include, for example, cameras, LiDAR, millimeter-wave radar, ultrasonic sensors, GPS sensors, acceleration sensors, gyroscopes, etc. For example, in the embodiment of (1) above, the server 200 may acquire the detection results of the internal sensors and reflect the detection results of the internal sensors in the route when generating a route. In the embodiment of (1) above, the vehicle 100 may acquire the detection results of the internal sensors and reflect the detection results of the internal sensors in the driving control signal when generating a driving control signal. In the embodiment of (2) above, the vehicle 100 may acquire the detection results of the internal sensors and reflect the detection results of the internal sensors in the route when generating the route. In the embodiment of (2) above, the vehicle 100 may acquire the detection results of the internal sensors and reflect the detection results of the internal sensors in the driving control signal when generating the driving control signal.

[0066] (YY3) In the driving control example 2, the vehicle 100v is equipped with an internal sensor, and the detection result output from the internal sensor may be used in at least one of the generation of the route and the generation of the driving control signal. For example, the vehicle 100v may acquire the detection result from the internal sensor and reflect the detection result from the internal sensor in the route when generating the route. The vehicle 100v may acquire the detection result from the internal sensor and reflect the detection result from the internal sensor in the driving control signal when generating the driving control signal.

[0067] (YY4) In driving control example 2, vehicle 100v acquires vehicle position information using the detection results of the external sensor 300. Alternatively, vehicle 100v may be equipped with an internal sensor, which may acquire vehicle position information using the detection results of the internal sensor, determine the next target location to which vehicle 100v should go, generate a route from vehicle 100v's current location to the target location as shown in the acquired vehicle position information, generate a driving control signal for driving along the generated route, and control the actuator group 120 using the generated driving control signal. In this case, vehicle 100v can drive without using the detection results of the external sensor 300 at all. Vehicle 100v may also acquire target arrival time and congestion information from outside vehicle 100v and reflect the target arrival time and congestion information in at least one of the route and the driving control signal. Furthermore, all the functional configurations of system 50v may be provided in vehicle 100v. That is, the processing realized by system 50v in this disclosure may be realized by vehicle 100v alone.

[0068] (YY5) In the driving control example 1, the server 200 automatically generates a driving control signal to be transmitted to the vehicle 100. Alternatively, the server 200 may generate a driving control signal to be transmitted to the vehicle 100 in accordance with the operation of an external operator located outside the vehicle 100. For example, the external operator may operate a control device that includes a display for displaying captured images output from an external sensor 300, a steering wheel for remotely controlling the vehicle 100, an accelerator pedal, a brake pedal, and a communication device for communicating with the server 200 via wired or wireless communication, and the server 200 may generate a driving control signal in accordance with the operation applied to the control device.

[0069] (YY6) In each of the above driving control examples, the vehicle 100 only needs to have a configuration that allows it to move by unmanned operation, and may be in the form of a platform having the configuration described below. Specifically, in order for the vehicle 100 to perform the three functions of "driving," "turning," and "stopping" by unmanned operation, it only needs to be equipped with at least a vehicle control device 110 and an actuator group 120. When the vehicle 100 acquires information from the outside for unmanned operation, the vehicle 100 may further be equipped with a communication device 130. That is, the vehicle 100 that can move by unmanned operation does not need to have at least some of the interior parts such as the driver's seat and dashboard installed, at least some of the exterior parts such as the bumper and fender installed, and does not need to have a body shell installed. In this case, the remaining parts such as the body shell may be attached to the vehicle 100 before it is shipped from the factory FC, or the remaining parts such as the body shell may be attached to the vehicle 100 after it has been shipped from the factory FC, while the remaining parts such as the body shell are not attached to the vehicle 100. Each part may be attached from any direction, such as the top, bottom, front, rear, right, or left side of the vehicle 100, and each part may be attached from the same direction or from different directions. The positioning of the platform can also be determined in the same way as the vehicle 100 in the first embodiment.

[0070] (YY7) Vehicle 100 may be manufactured by combining multiple modules. A module means a unit composed of multiple parts grouped together according to the part or function of the vehicle 100. For example, the platform of vehicle 100 may be manufactured by combining a front module that constitutes the front part of the platform, a central module that constitutes the central part of the platform, and a rear module that constitutes the rear part of the platform. The number of modules that constitute the platform is not limited to three, but may be two or fewer, or four or more. In addition to, or instead of, the parts that constitute the platform may be modularized, as well as parts that constitute parts of the vehicle 100 that are different from the platform. Various modules may also include any exterior parts such as bumpers and grilles, or any interior parts such as seats and consoles. Furthermore, not limited to vehicle 100, any type of mobile body may be manufactured by combining multiple modules. Such modules may be manufactured, for example, by joining multiple parts by welding or fasteners, or by integrally molding at least a part of the parts that constitute the module as a single part by casting. A molding technique for integrally molding a single component, especially a relatively large component, is also called gigacast or megacast. For example, the front module, central module, and rear module mentioned above may be manufactured using gigacast.

[0071] (YY8) Transporting vehicle 100 using the unmanned operation of the vehicle 100 is also called "autonomous transport." The configuration for realizing autonomous transport is also called a "vehicle remote control autonomous driving transport system." Furthermore, a production method that uses autonomous transport to produce vehicle 100 is also called "autonomous production." In autonomous production, for example, at a factory fuel cell (FC) that manufactures vehicle 100, at least a portion of the transport of vehicle 100 is realized by autonomous transport.

[0072] (YY9) In each of the above driving control examples, some or all of the functions and processes implemented in software may be implemented in hardware. Also, some or all of the functions and processes implemented in hardware may be implemented in software. As hardware for implementing the various functions in each of the above embodiments, various circuits such as integrated circuits and discrete circuits may be used.

[0073] Furthermore, this disclosure can be realized by having a CPU (Central Processing Unit) execute a computer program to perform some or all of the processing in the external sensor 300, vehicle 100, server 200, etc. as described above.

[0074] The program described above includes, when loaded into a computer, a set of instructions (or software code) for causing the computer to perform one or more of the functions described in the embodiments. The program may be stored in a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include RAM (Random-Access Memory), ROM (Read-Only Memory), flash memory, SSD (Solid-State Drive), or other memory technologies, CD-ROM, DVD (Digital Versatile Disc), Blu-ray® disc, or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage, or other magnetic storage devices. The program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include temporary computer-readable medium or a communication medium that includes electrically, optically, acoustically, or otherwise propagating signals.

[0075] Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. Various modifications to the structure and details of the present disclosure can be made as can be understood by those skilled in the art within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate. [Explanation of symbols]

[0076] 50 Vehicle Management Systems 100 vehicles 110 Vehicle control system 111 processors 112 memory 113 Input / Output Interfaces 114 Internal bus 115 Vehicle Control Unit 120 Actuator Group 130 Communication equipment 140 External Sensors 141 In-car camera 142 Radar 143 LiDAR 200 servers 201 Processor 202 memory 203 Input / Output Interfaces 204 Internal Bus 205 Communication equipment 207 Analysis Department 208 Analysis Department 209 Monitoring Department 210 Remote Control Unit 211 Output section 300 External Sensors 321 surveillance cameras 322 surveillance cameras 500 Networks

Claims

1. Multiple vehicles capable of moving by unmanned operation, A management device for managing the operation of the aforementioned multiple vehicles, A vehicle management system equipped with, Each of the aforementioned vehicles is It has an external sensor that detects surrounding obstacles, The aforementioned control device is A communication unit that receives the detection result of the external sensor, A control unit controls the operation of the plurality of vehicles moving through the first area based on the detection results of the external sensor and the images captured by the first surveillance camera that photographs the first area. Equipped with, If the control unit detects a malfunction in the external sensor of each of the multiple vehicles moving in the first area, it controls the operation of the multiple vehicles moving in the first area based on the image captured by the first surveillance camera. Vehicle management system.

2. The aforementioned external sensor is A car-mounted camera that films the surroundings, A radar that detects surrounding obstacles, LiDAR (Light Detection and Ranging) detects surrounding obstacles, It has multiple types of sensors, including The vehicle management system according to claim 1.

3. The control unit, If a malfunction is detected in fewer than a predetermined number of sensors among the multiple types of sensors included in the external sensor provided on each of the vehicles moving through the first area, the operation of the multiple vehicles moving through the first area is controlled based on the detection results of the sensors among the multiple types of sensors included in the external sensor provided on each of the vehicles that have not been found to be malfunctioning, and the images captured by the first surveillance camera. If a malfunction is detected in one or more of the multiple types of sensors included in the external sensor provided on each of the vehicles moving through the first area, the movement of the multiple vehicles moving through the first area will be stopped. The vehicle management system according to claim 2.

4. The system further includes a database that stores multiple combinations of the operating status of the external sensors installed on the vehicle, the area in which the vehicle moves, and the control content of the vehicle's operation. The control unit extracts control content for the operation of each vehicle moving in the first area from the database, according to the operating status of the external sensors provided on each vehicle. The vehicle management system according to claim 1.

5. If the control unit detects a malfunction in the external sensor of each of the multiple vehicles moving in the first area, it will reduce the travel speed of the multiple vehicles moving in the first area, or increase the distance between the multiple vehicles moving in the first area. The vehicle management system according to claim 1.

Images