Display control device and display device

Abstract

A display control device (8) comprises a built-in detection unit (821) or an image quality detection unit (822), and a display control (823), wherein the display control (823) is configured to display a built-in camera image on a display surface (45) in one case from a case in which a built-in camera (16) is mounted in a moving object and a case in which the image quality is equal to or greater than a criterion, and to display an external camera image on the display surface (45) in one case from a case in which the built-in camera (16) is not mounted in the moving object and a case in which the image quality is less than the criterion.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention

[0001] The present disclosure relates to a display control device and a display device. 2. Description of the related art

[0002] From the related prior art, an autonomous remote-controlled vehicle is known (unexamined published Japanese patent application (translation of PCT application) No. 2017-538619 (JP 2017-538619 A)). SUMMARY OF THE INVENTION

[0003] In a case where a moving object, such as a vehicle, is stopped during a period in which autonomous driving control is performed, the moving object may be remotely controlled by an operator operating an operator input device provided at a location different from the moving object. In this case, a technique is desired in which the operator can visually recognize an image indicating an environmental situation of the moving object to operate the operator input device.

[0004] The present disclosure may be implemented in the following forms.

[0005] A first aspect of the present disclosure relates to a display control device. In the display control device configured to control a display device used to move a moving object configured to move via unmanned traveling, the moving object is configured to receive a traveling control signal defining a traveling operation of the moving object and generated in response to an operation performed by an operator on an operator input device provided at a location different from the moving object, and to travel in a remote manual traveling mode using the received traveling control signal.During a period of control of the remote manual drive mode, the drive control signal is generated in response to the operation performed by the operator on the operator input device using an image captured by a camera, and is displayed on a display screen of the display device provided at a location different from the moving object. The camera is at least one of a built-in camera mounted in the moving object and an external camera configured to image the moving object from the outside. The display control device includes at least one of a built-in detection unit and an image quality detection unit, and a display controller. The built-in detection unit is configured to detect built-in information indicating whether or not the built-in camera is mounted in the moving object.The image quality acquisition unit is configured to acquire image quality information indicating an image quality of a built-in camera image acquired by the built-in camera. The display controller is configured to display the image on the display screen. The display controller is configured to display the built-in camera image on the display screen in at least one of the cases where the built-in camera is mounted in the moving object and in a case where the image quality is equal to or greater than a predetermined criterion, and to display an external camera image, which is the acquired image of the external camera provided at a location different from the moving object, on the display screen in at least one of the cases where the built-in camera is not mounted in the moving object and in a case where the image quality is less than the criterion.According to the first aspect of the present disclosure, the display control device can display at least one of the built-in camera images and the external camera image on the display screen in accordance with the built-in information and the image quality information. That is, in a case where the operator operates the operator input device to remotely control the moving object, the display control device can provide the operator with an image showing an environmental situation of the moving object.

[0006] According to the first aspect of the present disclosure, the image quality of the built-in camera image can be determined by at least one of the following: the resolution of the built-in camera image, the distortion of the built-in camera image, and the detection range of the built-in camera. According to the first aspect of the present disclosure, the image quality of the built-in camera image can be determined by at least one of the following criteria: the resolution of the built-in camera image, the distortion of the built-in camera image, and the detection range of the built-in camera that records the built-in camera image.

[0007] In the first aspect of the present disclosure, the display controller may be configured to determine that the image quality is equal to or greater than the criterion in at least one of a first case, a second case, and a third case, the first case being when the resolution of the in-camera image is equal to or greater than a first threshold, the second case being when the distortion of the in-camera image is less than a predetermined second threshold, and the third case being when the detection range of the in-camera is equal to or greater than a predetermined third threshold.

[0008] In the first aspect of the present disclosure, the moving object may be further configured to travel in an automated driving mode using the generated driving control signal without responding to the operation performed by the operator on the operator input device, the display controller may be configured to display at least one of the built-in camera image and the external camera image on the display screen in accordance with the built-in information and the image quality information, at least in one of the cases where the automated driving mode is switched to the remote manual driving mode and in a case where the switch from the automated driving mode to the remote manual driving mode is predicted.According to the first aspect of the present disclosure, when the automated driving mode is switched to the remote manual driving mode, the display control device can display at least one of the built-in camera images and the external camera image on the display screen according to the built-in information and the image quality information. Thus, the display control device can provide the operator with the image indicating the surrounding situation of the moving object during the period during which the remote manual driving mode control is executed.According to the first aspect of the present disclosure, in a case where the switch from the automated driving mode to the remote manual driving mode is predicted, the display control device can display at least one of the built-in camera image and the external camera image on the display screen according to the installation information and the image quality information. In this way, the display control device can provide the operator with the image indicating the surrounding situation of the moving object before the automated driving mode is switched to the remote manual driving mode. Therefore, the display control device can enable the operator to start the operation of the operator input device in a timely manner by allowing the operator to visually recognize the image displayed on the display screen when the automated driving mode is switched to the remote manual driving mode.

[0009] In the first aspect of the present disclosure, the display controller may be configured to display at least one of the built-in camera image and the external camera image on the display screen according to the installation information and the image quality information even during a period during which the automated driving mode control is executed.

[0010] According to the first aspect of the present disclosure, the display controller may be configured to display the external camera image in addition to the built-in camera image on the display screen when the display controller displays the built-in camera image on the display screen. According to the first aspect of the present disclosure, in a case where the built-in camera image is displayed on the display screen, the display control device may display the external camera image in addition to the built-in camera image on the display screen. In this way, the display control device can provide the external camera image to the operator in a situation of a region that cannot be visually recognized with the built-in camera image.

[0011] A second aspect of the present disclosure relates to a display device. In the display device configured to move a moving object to move via unmanned traveling, the moving object is configured to receive a traveling control signal defining a traveling operation of the moving object and generated in response to an operation performed by an operator on an operator input device provided at a location different from the moving object, and to travel in a remote manual traveling mode using the received traveling control signal. The display device includes a display screen provided at a location different from the moving object and on which a captured image from a camera is displayed.During a period of control of the remote manual driving mode, the driving control signal is generated in response to the operator's operation on the operator input device using the image displayed on the display screen. The camera is at least one of a built-in camera mounted in the moving object and an external camera configured to image the moving object from the outside.A built-in camera image captured by the built-in camera is displayed on the display screen in at least one of the cases where the built-in camera is mounted in the moving object and in a case where the image quality of the built-in camera image is equal to or greater than a predetermined criterion, and an external camera image, which is the captured image from the external camera provided at a location different from the moving object, is displayed on the display screen in at least one of the cases where the built-in camera is not mounted in the moving object and a case where the image quality is lower than the criterion.According to the second aspect of the present disclosure, the display device can display the built-in camera image and / or the external camera image on the display screen, depending on whether the built-in camera is mounted in the moving object or not and the image quality of the built-in camera image. That is, in a case where the operator operates the operator input device to remotely control the moving object, the display device can provide the operator with the image showing the surrounding situation of the moving object.

[0012] The present disclosure may be implemented in various forms other than the display control device and the display device described above. For example, the present disclosure may be implemented in the form of a display system including a display control device and a display device, a method for manufacturing a display control device and a display device, a method for controlling a display control device and a display device, a computer program implementing the method for controlling the display control device and the display device, a non-transitory recording medium recording the computer program, and the like. BRIEF DESCRIPTION OF THE CHARACTERS

[0013] Features, advantages and technical and industrial significance of exemplary embodiments of the invention are described below with reference to the accompanying figures, in which like symbols denote like elements and in which: Fig. 1 is a diagram showing the configuration of a driving system in a first embodiment; Fig. 2 is a diagram showing the configuration of an input system; Fig. 3 is a diagram showing the configuration of an information processing system; Fig. 4 is a flowchart illustrating a method for controlling operation of a vehicle in an automated driving mode; Fig. 5 is a flowchart illustrating a method for controlling a display screen in a remote manual drive mode; Fig. 6 is a flowchart illustrating a method for operating the vehicle in the remote manual drive mode; Fig. 7 is a diagram showing a configuration of a driving system in a second embodiment; and Fig. 8 is a flowchart illustrating a method for controlling operation of a vehicle in an autonomous driving mode. DETAILED DESCRIPTION OF THE EMBODIMENTS A. First Embodiment:

[0014] Fig. 1 is a diagram showing the configuration of a driving system 1 in a first embodiment. The driving system 1 is a system that allows a moving object to be moved without steering by an occupant boarding the moving object. The driving system 1 includes one or more moving objects, an external camera 90, an input system 2 including an operator input device 3 and a display device 4, and an information processing system 5 including a manual remote control device 6, an automatic remote control device 7, and a display control device 8.

[0015] The "moving object" is an object that can be moved, for example, a vehicle 10 or an electric vertical takeoff and landing aircraft (so-called flying car). In the present embodiment, the moving object is the vehicle 10. The vehicle 10 may be a vehicle that moves using wheels or a vehicle that moves using a crawler, and is, for example, a passenger car, a truck, a bus, a two-wheeled vehicle, a four-wheeled vehicle, a tank, and a construction vehicle. The vehicle 10 includes a battery electric vehicle (BEV), a gasoline-powered vehicle, a hybrid electric vehicle, and a fuel cell-powered vehicle.Furthermore, in a case where the moving object is other than the vehicle 10, the term “vehicle” and “car” in the present disclosure may be replaced with “moving object,” and the term “driving” may be replaced with “moving,” as appropriate.

[0016] The vehicle 10 includes a driving device 11, a steering device 12, a braking device 13, a communication device 14, and a vehicle control device 15. The driving device 11 accelerates the vehicle 10. The steering device 12 changes a traveling direction of the vehicle 10. The braking device 13 decelerates the vehicle 10. The communication device 14 communicates with an external device through wireless communication or the like. The external device is another device, such as the manual remote control device 6, and another vehicle 10. The communication device 14 is, for example, a wireless communication device. The vehicle control device 15 controls an operation of the vehicle 10.

[0017] The vehicle control device 15 includes an input and output interface 159, a storage unit 157, and a central processing unit (CPU) 150. The input and output interface 159 is used for communication with various devices installed in the vehicle 10, the host vehicle. The storage unit 157 of the vehicle control device 15 stores various types of information, including various programs P15 for controlling an operation of the vehicle control device 15. The CPU 150 of the vehicle control device 15 functions as an operation controller 153 and a vehicle transmission unit 154 by loading various programs P15 stored in the storage unit 157. The operation controller 153 performs driving control of the vehicle 10 by controlling an operation of an actuator that changes the acceleration and deceleration and a steering angle of the vehicle 10.For example, "driving control" refers to various controls such as adjusting the acceleration, speed, steering angle, and the like of the vehicle 10 to drive the actuator that performs the three functions of "driving," "turning," and "stopping" of the vehicle 10. In the present embodiment, the actuator includes a driving device actuator 11, a steering device actuator 12, and a braking device actuator 13.

[0018] The vehicle transmission unit 154 transmits a later-described built-in camera image to the display control device 8 in response to a request from the display control device 8.

[0019] The configuration of the vehicle 10 is not limited to the configuration described above. The vehicle 10 may also include a built-in camera 16 mounted in the vehicle 10. In a case where the vehicle 10 includes a built-in camera 16, the built-in camera 16 is mounted on a front side of the vehicle 10 so that the built-in camera 16 can image a landscape in front of the vehicle 10.

[0020] The vehicle 10 can travel via unmanned driving. "Unmanned driving" means driving that does not depend on the occupant's driving operation. Driving operation is an operation that involves at least one of "driving," "turning," and "stopping" of the vehicle 10. Unmanned driving is realized through automatic or manual remote control using a device external to the vehicle 10 or through autonomous control of the vehicle 10. The occupant who does not perform the driving operation can board the vehicle 10 traveling via unmanned driving. Examples of the occupant who does not perform the driving operation include a person who simply sits in a seat of the vehicle 10 and a person who performs work other than driving operation, such as assembly, inspection, or operating switches, while boarding the vehicle 10.In addition, driving with the occupant's driving operation can be referred to as "manned driving".

[0021] In the present embodiment, the vehicle 10 travels in a remote manual driving mode or an automated driving mode within a factory where the vehicle 10 is manufactured. In the remote manual driving mode, the vehicle 10 receives a driving control signal (hereinafter referred to as a manual control signal) that defines a driving operation of the vehicle 10 and is generated externally in response to an operation performed by an operator on the operator input device 3.

[0022] Therefore, the vehicle 10 travels using the received manual control signal. In the automated driving mode, the vehicle 10 receives the generated driving control signal (hereinafter, an automatic control signal) without responding to the external operation performed by the operator. Therefore, the vehicle 10 travels using the received automatic control signal. For example, if the vehicle 10 is unexpectedly stopped during a period during which the automatic driving mode control is being executed even though the vehicle 10 receives the automatic control signal, the driving mode of the vehicle 10 is switched from the automatic driving mode to the remote manual driving mode. The vehicle 10 can travel to a different location from the factory.

[0023] The external camera 90 is a camera provided at a location different from the vehicle 10. The installation position and orientation of the external camera 90, as well as the number of installed external cameras 90, are determined by considering a detection range RG (angle of view) of the external camera 90. In the present embodiment, the external camera 90 is a factory-installed camera. The external camera 90 images the detection range RG including the vehicle 10 from the outside of the vehicle 10. Therefore, the external camera 90 outputs the image (hereinafter referred to as the external camera image) as the detection result.

[0024] Fig. 2 is a diagram showing a configuration of the input system 2. The input system 2 is a system that allows an operator Op to remotely control the driving operation of the vehicle 10 from a location other than the vehicle 10. As described above, the input system 2 includes the operator input device 3 and the display device 4.

[0025] The operator input device 3 is an input device for the operator Op to remotely control the driving operation of the vehicle 10. The operator input device 3 is provided at a different location of the vehicle 10. The operator input device 3 is communicatively connected to the manual remote control device 6. The operator input device 3 includes one or more input operation units 30 operated by the operator Op. In the present embodiment, the operator input device 3 includes a steering wheel 31, an accelerator pedal 32, and a brake pedal 33 as input operation units 30. The steering wheel 31 changes the traveling direction of the vehicle 10 via remote control. The accelerator pedal 32 accelerates the vehicle 10 via remote control. The brake pedal 33 decelerates the vehicle 10 via remote control. The configuration of the operator input device 3 is not limited to the configuration described above.The operator input device 3 may include, for example, a push-button switch or a lever switch as the actuating unit 30.

[0026] The display device 4 is used for driving the vehicle 10. Specifically, the display device 4 allows the operator Op to visually recognize the surroundings of the vehicle 10 as a control target when the operator Op operates the operator input device 3. The display device 4 is provided at a different location from the vehicle 10. The display device 4 is communicatively connected to the display control device 8. The display device 4 includes a display screen 45. The display screen 45 displays the images captured by the cameras 16, 90, which include at least a surroundings of the vehicle 10 as a control target within the detection range RG. The display device 4 is located at a position where the operator Op can visually recognize the display screen 45.The operator Op operates the operator input device 3 using the image displayed on the display screen 45 in a period during which the control of the remote manual travel mode is executed.

[0027] Fig. 3 is a diagram showing a configuration of the information processing system 5. As described above, the information processing system 5 includes the manual remote control device 6, the automatic remote control device 7, and the display control device 8. The function of the information processing system 5 is realized, for example, by a server.

[0028] The manual remote control device 6 generates the manual control signal and transmits the generated manual control signal to the vehicle 10 as the control target. The manual remote control device 6 includes a communication unit 61, a storage unit 63, and a CPU 62. The communication unit 61, the storage unit 63, and the CPU 62 in the manual remote control device 6 are connected to each other, for example, via an internal bus.

[0029] The communication unit 61 of the manual remote control device 6 is communicatively connected to a device other than the manual remote control device 6 and the manual remote control device 6. The communication unit 61 of the manual remote control device 6 is, for example, a wireless communication device.

[0030] The storage unit 63 of the manual remote control device 6 stores various types of information including various programs P6 for controlling the operation of the manual remote control device 6. The storage unit 63 of the manual remote control device 6 includes, for example, a random access memory (RAM), a read-only memory (ROM), and a hard disk drive (HDD).

[0031] The CPU 62 of the manual remote control device 6 functions as the manual signal generation unit 621 and the manual transmission unit 622 by loading various programs P6 stored in the storage unit 63. The manual signal generation unit 621 generates the manual control signal. In the present embodiment, the manual control signal includes the acceleration and steering angle of the vehicle 10 as parameters. The manual transmission unit 622 transmits the manual control signal to the vehicle 10 as a control target. Therefore, the vehicle control device 15 mounted in the vehicle 10 receives the manual control signal from the manual remote control device 6 and drives the actuator using the received manual control signal, so that the vehicle 10 travels in response to the operation performed by the operator Op on the operator input device 3.

[0032] The automatic remote control device 7 generates the automatic control signal and transmits the generated automatic control signal to the vehicle 10 as the control target. The automatic remote control device 7 includes a communication unit 71, a storage unit 73, and a CPU 72. The communication unit 71, the storage unit 73, and the CPU 72 in the automatic remote control device 7 are connected to each other, for example, via an internal bus.

[0033] The communication unit 71 of the automatic remote control device 7 is communicatively connected to a device other than the automatic remote control device 7 and the automatic remote control device 7. The communication unit 71 of the automatic remote control device 7 is, for example, a wireless communication device.

[0034] The storage unit 73 of the automatic remote control device 7 stores various types of information, including various programs P7 for controlling the operation of the automatic remote control device 7, a detection model Md, and an ideal path Ip. The details of the detection model Md and the ideal path Ip will be described later. The storage unit 63 of the automatic remote control device 7 includes, for example, a RAM, a ROM, and a hard disk drive (HDD).

[0035] The CPU 72 of the automatic remote control device 7 functions as a calculation unit 721, an automatic signal generation unit 722, and an automatic transmission unit 723 by loading various programs P7 stored in the storage unit 73.

[0036] The calculation unit 721 calculates a position of the vehicle 10. In the present embodiment, the calculation unit 721 calculates the position of the vehicle 10 using the external camera image output from the external camera 90 as an external sensor Se provided at a location different from the vehicle 10. The calculation unit 721 calculates the coordinates of a position point 10e of the vehicle 10 in an image coordinate system using, for example, an appearance of the vehicle 10 obtained from the external camera image. Then, the calculation unit 721 calculates the position of the vehicle 10 by converting the calculated coordinates into coordinates in a global coordinate system. In the present embodiment, the position information of the vehicle 10 includes the X, Y, and Z coordinates in the global coordinate system of the factory.

[0037] The appearance of the vehicle 10 in the external camera image can be recognized, for example, by inputting the external camera image to the detection model Md using artificial intelligence. Examples of the detection model Md include a trained machine learning model trained to perform semantic segmentation or instance segmentation. For example, a convolutional neural network (CNN) trained by supervised learning using a training dataset can be used as the detection model Md. The training dataset contains, for example, a plurality of training images containing the vehicle 10 and a correct answer label indicating whether each region in the training image is a region indicating the vehicle 10 or a region indicating a region other than the vehicle 10.When training the CNN, the parameters of the CNN are preferably updated by backpropagation (error backpropagation method) to reduce an error between the output result of the detection model Md and the correct answer label.

[0038] The calculation unit 721 may also calculate the orientation of the vehicle 10. The orientation of the vehicle 10 may be calculated, for example, based on an orientation of a motion vector of the vehicle 10 calculated from a position change of a feature point of the vehicle 10 between frames of the external camera image using an optical flow method. The orientation of the vehicle 10 may be calculated, for example, using an output result of a yaw rate sensor or the like installed in the vehicle 10. The calculation unit 721 may be a single function of a device other than the automatic remote control device 7, e.g., the external camera 90. In this case, the automatic remote control device 7 includes an acquisition unit that acquires the position data of the vehicle 10 instead of the calculation unit 721.

[0039] The automatic signal generation unit 722 generates the automatic control signal. The automatic signal generation unit 722 first determines a target position to which the vehicle 10 should next travel to generate the automatic control signal. In the present embodiment, the target position is represented by the X, Y, and Z coordinates in the factory's global coordinate system. The automatic signal generation unit 722 determines the target position to which the vehicle 10 should next travel using the position information of the vehicle 10 and the ideal path Ip stored in advance in the storage unit 73 of the automatic remote control device 7. The ideal path Ip is a path along which the vehicle 10 should travel.The path is represented by a node indicating a departure point, a node indicating a transit point, a node indicating a destination, and a connection between the respective nodes. The automatic signal generation unit 722 determines the target position Ip on the ideal path ahead of a current position of the vehicle 10. The automatic signal generation unit 722 generates the automatic control signal to cause the vehicle 10 to travel toward the specified target position. The automatic signal generation unit 722 calculates a traveling speed of the vehicle 10 from the transition of the position of the vehicle 10 and compares the calculated traveling speed with a predetermined target speed of the vehicle 10.The automatic signal generation unit 722 decides the acceleration to accelerate the vehicle 10 when the traveling speed is lower than the target speed, and decides the acceleration to decelerate the vehicle 10 when the traveling speed is higher than the target speed. The automatic signal generation unit 722 determines the steering angle so that the vehicle 10 does not deviate from the ideal path Ip when the vehicle 10 is on the ideal path Ip, and determines the steering angle so that the vehicle 10 returns to the ideal path Ip when the vehicle 10 is not on the ideal path Ip, in other words, when the vehicle 10 deviates from the ideal path Ip.

[0040] In this way, the automatic signal generating unit 722 generates the driving control signal containing the acceleration and the steering angle of the vehicle 10 as parameters.

[0041] The automatic transmission unit 723 transmits the automatic control signal to the vehicle 10 as the control target. Therefore, the vehicle control device 15 receives the automatic control signal from the remote automatic control device 7 and controls the actuator using the automatic control signal, so that the vehicle 10 is controlled autonomously without responding to the operation performed by the operator Op on the operator input device 3.

[0042] During the period during which the control of the automated driving mode is executed, the automatic remote control device 7 repeatedly performs the acquisition of the position information of the vehicle 10, the decision of the target position, the generation of the automatic control signal, and the transmission of the automatic control signal in a predetermined cycle. The vehicle control device 15 repeatedly performs the reception of the automatic control signal and the control of the actuator in a predetermined cycle.

[0043] The display control device 8 controls the display contents displayed on the display screen 45. The display control device 8 includes a communication unit 81, a storage unit 83, and a CPU 82. The communication unit 81, the storage unit 83, and the CPU 82 in the display control device 8 are connected to each other, for example, via an internal bus.

[0044] The communication unit 81 of the display control device 8 connects a device other than the display control device 8 and the display control device 8 in a communicable manner. The communication unit 81 of the display control device 8 is, for example, a wireless communication device.

[0045] The storage unit 83 of the display control device 8 stores various types of information including various programs P8 for controlling the operation of the display control device 8. The storage unit 83 of the display control device 8 includes, for example, a RAM, a ROM, and a hard disk drive (HDD).

[0046] The CPU 82 of the display control device 8 functions as a mounting detection unit 821, an image quality detection unit 822, and a display controller 823 by loading various programs P8 stored in the storage unit 83.

[0047] The installation detection unit 821 detects the installation information. The installation information is information indicating whether the built-in camera 16 is mounted as an inspection target in the vehicle 10 or not.

[0048] The image quality acquisition unit 822 acquires image quality information. The image quality information is information indicating the image quality of an image (hereinafter referred to as an in-camera image) captured by the in-camera 16 mounted in the vehicle 10 as the inspection target. The image quality of the in-camera image is determined by at least one of the following criteria: resolution of the in-camera image, distortion of the in-camera image, and detection range (angle of view) of the in-camera 16 mounted in the vehicle 10 as the inspection target. The resolution of the in-camera image is calculated by, for example, specifying a model number of the in-camera 16 mounted in the vehicle 10 as the inspection target and the like, and by acquiring information (hereinafter referred to as performance information) indicating the performance of the in-camera 16. The resolution of the in-camera image can be determined based on the number of pixels, the image size, or the like.of the actually recorded built-in camera image. The distortion of the built-in camera image is calculated, for example, by analyzing the actually recorded built-in camera image. The detection range of the built-in camera 16 mounted in the vehicle 10 as the inspection target is calculated, for example, by specifying the model number or similar of the built-in camera 16 mounted in the vehicle 10 as the inspection target and recording the performance information.

[0049] The image quality of the built-in camera image in the image quality information means the image quality when the display device 4 receives the built-in camera image. When the built-in camera 16 mounted in the vehicle 10 as the inspection target has a processing function, the image quality of the built-in camera image in the image quality information means an image quality in an image (hereinafter, processed image) obtained by processing raw data (hereinafter, original image) captured by the built-in camera 16 mounted in the vehicle 10 as the inspection target. The processing function is a function for processing the original image to generate the processed image.The processing function includes, for example, a correction function that corrects distortion of the original image, a noise removal function that removes noise from the original image, and a resolution adjustment function that adjusts the resolution of the original image. When the in-camera image is transmitted from the in-camera 16 to the display device 4 via the display control device 8, and the display control device 8 has the processing function, the image quality of the in-camera image in the image quality information means the image quality of the processed image. The method for evaluating the image quality of the in-camera image is not limited to the method described above. The image quality of the in-camera image can also be evaluated using an evaluation index other than the noise value of the in-camera image.

[0050] The display controller 823 displays at least one of the built-in camera image and the external camera image on the display screen 45. In the present embodiment, the display controller 823 displays at least one of the built-in camera image and the external camera image on the display screen 45 in accordance with the installation information and the image quality information when the driving mode of the vehicle 10 is switched from the automated driving mode to the remote manual driving mode.

[0051] The display controller 823 displays the built-in camera image on the display screen 45 in at least one of the cases where the built-in camera 16 is mounted in the vehicle 10 as a control target, and in a case where the image quality of the built-in camera image is equal to or greater than a predetermined criterion. On the other hand, the display controller 823 displays the external camera image on the display screen 45 in at least one of the cases where the built-in camera 16 is not mounted in the vehicle 10 as a control target, and in a case where the image quality of the built-in camera image is below the criterion.

[0052] For example, in a case where the image quality of the in-camera image is determined by the resolution of the in-camera image, the display controller 823 determines that the image quality of the in-camera image is equal to or greater than the criterion in a case where the resolution of the in-camera image is equal to or greater than a predetermined resolution threshold. For example, in a case where the image quality of the in-camera image is determined by the resolution of the in-camera image, the distortion of the in-camera image, and the detection range of the in-camera 16, the display controller 823 determines that the image quality of the in-camera image is equal to or greater than the criterion in the following case. In this case, the display controller 823 determines that the image quality of the in-camera image is equal to or greater than the criterion in at least one of a first case, a second case, and a third case.The first case is a case where the resolution of the built-in camera image is equal to or greater than the resolution threshold. The second case is a case where the distortion of the built-in camera image is less than a predetermined distortion threshold. The third case is a case where the detection range of the built-in camera 16 is equal to or greater than a predetermined range threshold.

[0053] In the present embodiment, when the in-vehicle camera 16 is mounted in the vehicle 10 as an inspection target and the image quality of the in-vehicle camera image is equal to or higher than the criterion, the display controller 823 displays only the in-vehicle camera image on the display screen 45. On the other hand, when the in-vehicle camera 16 is mounted in the vehicle 10 as an inspection target and the image quality of the in-vehicle camera image is lower than the criterion, the display controller 823 displays the in-vehicle camera image and the external camera image on the display screen 45. When the in-vehicle camera 16 is not mounted in the vehicle 10 as an inspection target, the display controller 823 displays the external camera image on the display screen 45.

[0054] Fig. 4 is a flowchart illustrating an example of an operation control method of the vehicle 10 during the period during which the control of the automated driving mode is executed. Fig. The operation control method shown in Fig. 4 is repeatedly executed at predetermined time intervals, for example, from a time when the vehicle 10 as a control target starts traveling in the automated driving mode to a time when the driving mode of the vehicle 10 is switched from the automated driving mode to the remote manual driving mode.

[0055] In step S101, the calculation unit 721 of the automatic remote control device 7 sends a request signal for acquiring the external camera image to the external camera 90. The external camera 90, which has received the request signal, sends the external camera image to the automatic remote control device 7 in step S102. In step S103, the calculation unit 721 of the automatic remote control device 7 calculates the position of the control target vehicle 10 using the external camera image. In step S104, the automatic signal generation unit 722 decides the target position to which the control target vehicle 10 should next travel using the position information of the vehicle 10 and the ideal path Ip. In step S105, the automatic signal generation unit 722 generates an automatic control signal to cause the vehicle 10 as the control target to travel toward the set target position.In step S106, the automatic transmission unit 723 transmits the automatic control signal to the vehicle 10 as the control target. In step S107, the operation controller 153 of the vehicle control device 15 mounted on the vehicle 10 controls the actuator using the received automatic control signal to drive the vehicle 10 at the acceleration and steering angle represented by the automatic control signal.

[0056] Fig. 5 is a flowchart illustrating a method for display control of the display screen 45 during the period during which the remote manual driving mode control is executed. Fig. The display control method shown in Fig. 5 is executed, for example, when the driving mode of the vehicle 10 is switched from the automatic driving mode to the remote-controlled manual driving mode.

[0057] In step S201, the installation acquisition unit 821 of the display control device 8 acquires the installation information. In a case where the in-vehicle camera 16 is mounted in the vehicle 10 as an inspection target (step S202: Yes), the display controller 823 executes step S203. In step S203, the display controller 823 sends a request signal for acquiring the in-vehicle camera image to the vehicle 10 as an inspection target. When the vehicle 10 receives the request signal, in step S204, the vehicle transmission unit 154 of the vehicle control device 15 mounted in the vehicle 10 acquires the in-vehicle camera image and transmits the acquired in-vehicle camera image to the display control device 8. In step S205, the image quality acquisition unit 822 of the display control device 8 acquires the image quality information.In the case where the image quality of the built-in camera image is equal to or greater than the criterion (step S206: Yes), the display controller 823 transmits the built-in camera image to the display device 4 in step S207. In a case where the built-in camera 16 is not mounted in the vehicle 10 (step S202: No) and in a case where the image quality of the built-in camera image is below the criterion (step S206: No), the display controller 823 transmits the request signal for acquiring the external camera image to the external camera 90 in step S208. The external camera 90, which has received the request signal, transmits the external camera image to the display control device 8 in step S209.In a case where the built-in camera 16 is mounted in the vehicle 10 as an inspection target (step S202: Yes) and the image quality of the built-in camera image is below the criterion (step S206: No), the display controller 823 of the display control device 8 executes step S210. In step S210, the display controller 823 transmits the built-in camera image and the external camera image to the display device 4. In the case where the built-in camera 16 is not mounted in the vehicle 10 (step S202: No), the display controller 823 executes step S211. In step S211, the display control device 8 transmits the external camera image to the display device 4. In step S212, the display device 4 displays the received image on the display screen 45.

[0058] Fig. 6 is a flowchart illustrating an operation control method of the vehicle 10 during the period during which the remote manual drive mode control is executed. Fig. The operation control method shown in Fig. 6 is repeatedly executed at predetermined time intervals, for example, from a time when the vehicle 10 as a control target starts to travel in the remote manual travel mode to a time when the travel mode of the vehicle 10 is switched from the remote manual travel mode to the automated travel mode.

[0059] In step S301, the operator input device 3 receives the operator's input Op. In step S302, the operator input device 3 transmits an operation amount of each input operation unit 30 to the manual remote control device 6. In step S303, the manual signal generation unit 621 of the manual remote control device 6 generates the manual control signal. In step S304, the manual transmission unit 622 transmits the manual control signal as a control target to the vehicle 10. In step S305, the operation controller 153 of the vehicle control device 15 mounted in the vehicle 10 controls the actuator using the received manual control signal to move the vehicle 10 with the acceleration and steering angle represented by the manual control signal.

[0060] According to the first embodiment, the display control device 8 can display the image captured by the cameras 16, 90, which includes at least the surroundings of the vehicle 10 to be controlled within the detection range RG, on the display screen 45 of the display device 4 located at the position visually recognizable by the operator Op. That is, in a case where the vehicle 10 is remotely controlled via the operator Op operating the operator input device 3, the display control device 8 can provide the operator Op with the image indicating the surrounding situation of the vehicle 10. Therefore, the operator Op can operate the operator input device 3 while visually recognizing the image displayed on the display screen 45.

[0061] According to the first embodiment, by acquiring the installation information, the display control device 8 can recognize whether or not the in-vehicle camera 16 is mounted in the vehicle 10 as a control target. Therefore, the display control device 8 can display the in-vehicle camera image on the display screen 45 by acquiring the in-vehicle camera image from the control target vehicle 10 in a case where the in-vehicle camera 16 is mounted in the control target vehicle 10. The scenery displayed by the in-vehicle camera image acquired by the in-vehicle camera 16 mounted on the front of the vehicle 10 is close to a scenery of the front of the vehicle 10 as a control target viewed from the driver's seat of the vehicle 10.Therefore, by displaying the built-in camera image on the display screen 45, the input system 2 can recreate a situation close to a situation in which a driver who enters the vehicle 10 as a control target operates the steering wheel or the like mounted in the vehicle 10. In this way, in a case where the vehicle 10 is remotely controlled by the operator Op operating the operator input device 3, the display control device 8 can easily enable the operator Op to recognize an object or the like present around the vehicle 10. That is, the display control device 8 can create a state in which the operator Op can easily operate the operator input device 3. Therefore, the possibility of the vehicle 10 colliding with another object or deviating from the path on which the vehicle 10 should travel due to the operator Op's operation error can be reduced.

[0062] According to the first embodiment, in a case where the built-in camera 16 is not mounted as a control target in the vehicle 10, the display control device 8 can display the external camera image on the display screen 45 by acquiring the external camera image from the external camera 90. Therefore, even if the built-in camera 16 is not mounted in the vehicle 10, the operator Op can operate the operator input device 3 using the external camera image.

[0063] According to the first embodiment, the display control device 8 can acquire the image quality information and compare the image quality information with the predetermined criterion to determine whether or not the image quality of the in-vehicle camera image is equal to or greater than the criterion. Therefore, the display control device 8 can display the in-vehicle camera image on the display screen 45 by acquiring the in-vehicle camera image of the vehicle 10 as a control target in a case where the image quality of the in-vehicle camera image is equal to or greater than the criterion. In this way, the display control device 8 can display the image with assured visibility on the display screen 45.Therefore, in a case where the vehicle 10 is remotely controlled by the operator Op operating the operator input device 3, the display control device 8 can easily enable the operator Op to recognize the object or the like present around the vehicle 10. Therefore, the possibility of the vehicle 10 colliding with another object or deviating from the path on which the vehicle 10 should travel due to the operation error of the operator Op can be further reduced.

[0064] According to the first embodiment, even in a case where the in-vehicle camera 16 is mounted as an inspection target in the vehicle 10, the display control device 8 can display the external camera image on the display screen 45 in addition to the in-vehicle camera image when the image quality of the in-vehicle camera image is below the criterion. In this way, the display control device 8 can display the display screen 45 in a state where a part that is difficult to visually recognize in the in-vehicle camera image is supplemented by the external camera image. Therefore, the operator Op can operate the operator input device 3 while visually recognizing both the in-vehicle camera image and the external camera image, so as to supplement the part that is difficult to visually recognize in the in-vehicle camera image with the visually recognized external camera image.

[0065] According to the first embodiment, the image quality of the built-in camera image can be determined by at least one of the following: the resolution of the built-in camera image, the distortion of the built-in camera image, and the detection range of the built-in camera 16 that captures the built-in camera image. Therefore, the image quality of the built-in camera image can be easily evaluated.

[0066] According to the first embodiment, in a case where the image quality of the in-camera image is determined by the resolution and distortion of the in-camera image, the display control device 8 can display the in-camera image and the external camera image on the display screen 45 when the image quality of the in-camera image is below the criterion. In this way, even in a case where the in-camera image is unclear or distorted and difficult to visually recognize, the display control device 8 can enable the operator Op to operate the operator input device 3 by displaying the external camera image on the display screen 45. Thus, a situation in which the operator Op cannot operate the operator input device 3 due to the image quality of the in-camera image can be avoided.Furthermore, in a case where the image quality of the in-vehicle camera image is determined by the detection range of the in-vehicle camera 16 that captures the in-vehicle camera image, the display control device 8 can display the in-vehicle camera image and the external camera image on the display screen 45 if the image quality of the in-vehicle camera image is below the criterion. In this way, a situation can be avoided in which the operation of the operator input device 3 is hindered or an object other than the vehicle 10 as a control target might collide with the vehicle 10 because the detection range of the in-vehicle camera 16 is narrower than a desired detection range.

[0067] According to the first embodiment, when the automated driving mode is switched to the remote manual driving mode, the display control device 8 can display at least one of the built-in camera images and the external camera image on the display screen 45 in accordance with the installation information and the image quality information. In this way, the display control device 8 can provide the operator Op with the image indicating the surrounding situation of the vehicle 10 during the period during which the remote manual driving mode control is executed.

[0068] According to the first embodiment, the driving system 1 can move the vehicle 10 within the factory via remote control. Therefore, the driving system 1 can move the vehicle 10 within the factory without using transportation means such as a crane or a conveyor belt. B. Second embodiment:

[0069] Fig. 7 is a diagram showing the configuration of a traveling system 1a in a second embodiment. The traveling system 1a includes a vehicle 10a as one or more moving objects, the external camera 90, the input system 2, and an information processing system 5a. The traveling system 1a according to the present embodiment differs from the first embodiment in that the automatic remote control device 7 is not provided. That is, the information processing system 5a includes the manual remote control device 6 and the display control device 8 without including the automatic remote control device 7. Therefore, a vehicle control device 15a mounted in the vehicle 10a decides a target position to which the vehicle 10a should next travel and generates the automatic control signal that causes the vehicle 10a to travel to this target position.That is, the vehicle 10a has an autonomous driving mode in which the vehicle 10a autonomously executes the decision of the target position and the generation of the automatic control signal, as an automated driving mode in which the vehicle 10a travels using the generated automatic control signal without responding to the operation of the operator Op.

[0070] The vehicle control device 15a includes an input and output interface 159, a storage unit 157a, and a CPU 150a. The storage unit 157a of the vehicle control device 15a stores various types of information, including various programs P15a for controlling the operation of the vehicle control device 15a, the detection model Md, and the ideal path Ip. The CPU 150a of the vehicle control device 15a functions as the calculation unit 151, the automatic signal generation unit 152, the operation controller 153, and the vehicle transmission unit 154 by loading various programs P15a stored in the storage unit 157a. The calculation unit 151 calculates a position of the vehicle 10a using the external camera image, in the same manner as the calculation unit 1521 in the first embodiment.The automatic signal generation unit 152 determines the target position to which the vehicle 10a should next travel, using the calculated position information of the vehicle 10a and the ideal path Ip stored in advance in the storage unit 157a of the vehicle control device 15a. Then, the automatic signal generation unit 152 generates the automatic control signal to cause the vehicle 10a to travel toward the specified target position. The actuation controller 153 controls the actuator using the generated automatic control signal to move the vehicle 10a at the acceleration and steering angle represented by the automatic control signal.During a period of autonomous driving mode control, the vehicle control device 15a repeatedly performs the acquisition of position information of the vehicle 10a, the decision of the target position, the generation of the automatic control signal, and the control of the actuator in a predetermined cycle. The same components as in the first embodiment are designated by the same reference numerals.

[0071] Fig. Fig. 8 is a flowchart illustrating an example of a method for controlling the operation of the vehicle 10a during a period during which the control of the autonomous driving mode is executed. Fig.The operation control method shown in Fig. 8 is repeatedly executed at predetermined time intervals, for example, from a time when the vehicle 10a as a control target starts to travel in the autonomous travel mode to a time when the travel mode of the vehicle 10a is switched from the autonomous travel mode to the remote manual travel mode.

[0072] In step S401, the calculation unit 151 of the vehicle control device 15a mounted in the vehicle 10a sends the request signal for acquiring the external camera image to the external camera 90. The external camera 90, having received the request signal, sends the external camera image to the vehicle 10a in step S402. In step S403, the calculation unit 151 of the vehicle control device 15a mounted in the vehicle 10a calculates the position of the vehicle 10a as a control target using the external camera image. In step S404, the automatic signal generation unit 152 decides which target position the vehicle 10a should next head for using the position information of the vehicle 10a and the ideal path Ip. In step S405, the automatic signal generation unit 152 generates the automatic control signal to cause the vehicle 10a to travel toward the set target position.In step S406, the actuation controller 153 controls the actuator using the automatic control signal to move the vehicle 10a with the acceleration and steering angle represented by the automatic control signal.

[0073] According to the second embodiment, the driving system 1a can drive the vehicle 10a independently. Therefore, the driving system 1a can move the vehicle 10a without using transportation means such as a crane or a conveyor belt.

[0074] The calculation unit 151 may be a single function of a device other than the automatic remote control device 7, e.g., the external camera 90. In this case, the vehicle control device 15a of the vehicle 10a includes a detection unit that detects the position data of the vehicle 10a instead of the calculation unit 151. C. Other Embodiments:C-1. Other Embodiment 1:

[0075] The display control device 8 may include the in-vehicle detection unit 821 without the image quality detection unit 822. In this case, the display controller 823 displays the in-vehicle camera image on the display screen 45 regardless of whether the image quality of the in-vehicle camera image is equal to or greater than the criterion when the in-vehicle camera 16 is mounted in the vehicles 10, 10a as inspection targets. The display control device 8 may include the image quality detection unit 822 without the in-vehicle detection unit 821. In this case, the image quality detection unit 822 determines that the in-vehicle camera 16 is mounted in the vehicles 10, 10a as inspection targets in a case where the image quality information can be detected. If the image quality of the in-vehicle camera image is equal to or greater than the criterion, the display controller 823 displays the in-vehicle camera image on the display screen 45.In a case where the image quality detection unit 822 can detect the image quality information and the image quality of the built-in camera image is lower than the criterion, the display controller 823 displays the built-in camera image and the external camera image on the display screen 45. Also in this form, the display controller 823 can display the built-in camera image and the external camera image on the display screen 45 in accordance with one of the installation information and the image quality information. C-2. Other embodiment 2:

[0076] The display controller 823 may display at least one of the built-in camera images and the external camera image on the display screen 45 in accordance with the installation information and the image quality information in a case where the transition from the automated driving mode to the remote manual driving mode is predicted. For example, the display controller 823 may display at least one of the built-in camera images and the external camera image on the display screen 45 in response to the prediction that the automated driving mode will be switched to the remote manual driving mode. In such a form, the display control device 8 may display at least one of the built-in camera images and the external camera image on the display screen 45 in accordance with the installation information and the image quality information in a case where the transition from the automated driving mode to the remote manual driving mode is predicted.In this way, the display controller 823 can display at least one of the built-in camera images and the external camera image on the display screen 45 before switching the automated driving mode to the remote manual driving mode. Therefore, in a case where the automated driving mode is switched to the remote manual driving mode, the operator Op can visually recognize the image displayed on the display screen 45 and promptly start operating the operator input device 3. In a case where the vehicles 10, 10a are unexpectedly stopped during the period while the automated driving mode control is being executed and the automated driving mode is switched to the remote manual driving mode, the time during which the vehicles 10, 10a are unexpectedly stopped can be minimized. C-3. Other embodiment 3:

[0077] The display controller 823 may display at least one of the built-in camera images and the external camera image on the display screen 45 in accordance with the installation information and the image quality information in a case other than a case where the automated driving mode is switched to the remote manual driving mode and a case where the switch from the automated driving mode to the remote manual driving mode is predicted. For example, the display controller 823 may display at least one of the built-in camera images and the external camera image on the display screen 45 in accordance with the installation information and the image quality information even during the period during which the automated driving mode control is executed.In such a form, in a case where it is necessary to switch from the automated driving mode to the remote manual driving mode, the operator Op can visually recognize the image displayed on the display screen 45 and operate the operator input device 3 more quickly. Furthermore, the operator Op can appropriately check a driving situation of the vehicles 10, 10a during the period in which the automated driving mode control is executed. C-4. Other embodiment 4:

[0078] The driving system 1 can detect an area outside the detection range of the built-in camera 16 in the surroundings of the vehicles 10, 10a via the external camera 90. In a case where the display controller 823 displays the built-in camera image on the display screen 45, the display controller 823 can also display the external camera image on the display screen 45 in addition to the built-in camera image. That is, in a case where the image quality of the built-in camera image is equal to or greater than the criterion, or in a case where the built-in camera image is uniformly displayed on the display screen 45, regardless of whether the image quality of the built-in camera image is equal to or greater than the criterion, the display controller 823 can display the built-in camera image and the external camera image on the display screen 45.In such a form, the operator Op can visually recognize the location of the area that cannot be visually recognized with the built-in camera image using the external camera image. Therefore, a blind spot occurring in the surroundings of the vehicles 10, 10a can be reduced in a case where the operator Op operates the operator input device 3 using the image displayed on the display screen 45. C-5. Other Embodiment 5:

[0079] The vehicles 10, 10a may include a plurality of built-in cameras 16. In this case, each of the built-in cameras 16 is mounted on the vehicles 10, 10a, for example, with the detection ranges of the adjacent built-in cameras 16 overlapping less. The display control device 8 displays the built-in camera images captured by the built-in cameras 16, respectively, on the display screen 45. In such a form, the possibility of the blind spot occurring in the surroundings of the vehicles 10, 10a can be reduced in a case where the operator Op operates the operator input device 3 using the image displayed on the display screen 45, compared to a case where one built-in camera 16 is provided in the vehicles 10, 10a. If four built-in cameras 16 are provided in the vehicles 10, 10a, the built-in cameras 16 can be mounted, for example, on the front, the rear, the right side and the left side of the vehicle 10, 10a.The landscape captured by the built-in camera 16 mounted on the rear of the vehicle 10, 10a is close to a landscape at the rear of the vehicle 10, 10a as viewed from the driver's seat of the vehicle 10, 10a. The landscape captured by the built-in camera 16 on the right side of the vehicle 10, 10a is close to a landscape of the right side of the vehicle 10, 10a as the control targets as viewed from the driver's seat of the vehicle 10, 10a. The landscape captured by the built-in camera 16 mounted on the left side of the vehicle 10, 10a is close to a landscape of a left side of the vehicle 10, 10a as the control targets as viewed from the driver's seat of the vehicle 10, 10a.Therefore, the built-in camera images taken by the built-in cameras 16 on the front, rear, right side, and left side of the vehicle 10, 10a are displayed on the display screen 45, so that the scenery can be brought closer to the scenery seen from the driver's seat. C-6. Other embodiment 6:

[0080] The driving system 1 can capture the surroundings of the vehicles 10, 10a via the external cameras 90. The display controller 823 can display the external camera images captured by the external cameras 90 on the display screen 45. In this form, the display controller 823 can display, on the display screen 45, the external camera images in which the surroundings of the vehicles 10, 10a are captured as control targets from different directions. Therefore, a blind spot occurring in the surroundings of the vehicles 10, 10a can be reduced in a case where the operator Op operates the operator input device 3 using the image displayed on the display screen 45. C-7. Other embodiment 7:

[0081] The display control device 8 may further include a communication information acquisition unit that acquires communication information. The communication information is information indicating whether or not the vehicles 10, 10a as control targets can transmit the in-car camera image to the display control device 8. In this case, if the vehicles 10, 10a as control targets can transmit the in-car camera image to the display control device 8, the display control device 8 acquires the image quality information or displays the in-car camera image on the display screen 45. On the other hand, if the vehicles 10, 10a as control targets cannot transmit the in-car camera image to the display control device 8, the display control device 8 displays the external camera image on the display screen 45.In such a form, in a case where the display controller 823 cannot transmit the built-in camera image to the display control device 8 due to a communication load or the like, the display controller 823 can display the external camera image on the display screen 45. C-8. Other embodiment 8:

[0082] At least one of the manual remote control device 6, the automatic remote control device 7, and the display control device 8 can be configured integrally. Furthermore, each of the manual remote control device 6, the automatic remote control device 7, and the display control device 8 can be implemented through cloud computing using, for example, one or more computers. In such a form, the configurations of the manual remote control device 6, the automatic remote control device 7, and the display control device 8 can be changed as needed. C-9. Other embodiment 9:

[0083] In the first embodiment, the processing from the acquisition of the position information of the vehicle 10 to the generation of the travel control signal is performed by the remote automatic control device 7. Meanwhile, the vehicle 10 may perform at least part of the processing from the acquisition of the position information of the vehicle 10 to the generation of the automatic control signal. For example, the following forms (1) to (3) can be used. (1) The automatic remote control device 7 can acquire the position information of the vehicle 10, determine the target position to which the vehicle 10 should travel next, and generate a path from the current position of the vehicle 10, represented by the acquired position information, to the target position. The automatic remote control device 7 can generate a path to the target position between the current position and the target location or a path to the target location. The automatic remote control device 7 can transmit the generated path to the vehicle 10. The vehicle 10 can generate the automatic control signal to make the vehicle 10 travel along the path received from the automatic remote control device 7 and control the actuator using the generated automatic control signal. (2) The automatic remote control device 7 can acquire the position information of the vehicle 10 and transmit the acquired position information to the vehicle 10. The vehicle 10 can determine the target position to which the vehicle 10 should travel next, generate the path from the current position of the vehicle 10 represented by the received position information to the target position, generate the automatic control signal to cause the vehicle 10 to travel the generated path, and control the actuator using the generated automatic control signal. (3) In forms (1) and (2), an internal sensor Si may be installed in the vehicle 10, and a detection result output from the internal sensor Si may be used for at least one of generating the path and generating the automatic control signal. Examples of the internal sensor Si include the built-in camera 16, a mounted LiDAR, a millimeter-wave radar, an ultrasonic sensor, a GPS sensor, an acceleration sensor, and a gyro sensor. In form (1), for example, the automatic remote control device 7 may detect the detection result of the internal sensor Si and reflect the detection result of the internal sensor Si into the path when generating the path. In form (1), the vehicle 10 may detect the detection result of the internal sensor Si and reflect the detection result of the internal sensor Si into the automatic control signal when generating the automatic control signal.In form (2), the vehicle 10 can detect the detection result of the internal sensor Si and reflect the detection result of the internal sensor Si into the path when the path is generated. In form (2), the vehicle 10 can detect the detection result of the internal sensor Si and reflect the detection result of the internal sensor Si into the automatic control signal when the automatic control signal is generated. (4) In the second embodiment, the internal sensor Si may be mounted in the vehicle 10a, and the detection result output by the internal sensor Si may be used for path generation and / or automatic control signal generation. For example, the vehicle 10a may detect the detection result of the internal sensor Si and incorporate the detection result of the internal sensor Si into the path when the path is generated. The vehicle 10a may detect the detection result of the internal sensor Si and incorporate the detection result of the internal sensor Si into the automatic control signal when the automatic control signal is generated. (5) In the second embodiment, the vehicle 10a detects the position information of the vehicle 10a using the detection result of the external sensor Se. Meanwhile, the internal sensor Si may be installed in the vehicle 10a, and the vehicle 10a can detect the position information using the detection result of the internal sensor Si, determine the target position to which the vehicle 10a should travel next, generate the path from the current position of the vehicle 10a represented by the detected position information to the target position, generate the automatic control signal to make the vehicle 10a travel along the generated path, and control the actuator using the generated automatic control signal. In this case, the vehicle 10a cannot travel at all using the detection result of the external sensor Se.The vehicle 10a can acquire a destination arrival time or traffic jam information from outside the vehicle 10a and reflect the destination arrival time or traffic jam information in at least one of the path and the automatic control signal. Furthermore, the functional configurations of the driving systems 1, 1a in the vehicles 10, 10a can be predictable. That is, the processing implemented by the driving systems 1, 1a, such as the driving control of the vehicles 10, 10a and the display control of the display device 4 described in the present disclosure, can be performed exclusively by the vehicles 10, 10a. C-10. Other embodiment 10:

[0084] The calculation unit 151, 721 can calculate the position and orientation of the vehicles 10, 10a using the sensor information acquired by the sensor of a different type from the external camera 90. For example, the calculation unit 151, 721 can calculate the position and orientation of the vehicles 10, 10a using measurement point group data representing the appearance of the vehicles 10, 10a in three dimensions, wherein the measurement point group data is acquired by a Light Detection and Ranging (LiDAR) (hereinafter referred to as external LiDAR) as an external sensor Se. In this case, the calculation units 151, 721 calculate the position and orientation of the vehicles 10, 10a by, for example, comparing the measurement point group data with the point group data as a reference. The point group data as a reference is reference data that serves as a template when comparing with the measurement point group data.The point group data for the reference is, for example, virtual three-dimensional point group data generated based on three-dimensional CAD data representing the appearance of the vehicles 10, 10a. An algorithm for the comparison via the calculation units 151, 721 includes, for example, iterative point proximity (ICP) and normal distribution transformation (NDT). The calculation units 151, 721 can calculate the position and orientation of the vehicles 10, 10a using the measurement point group data obtained by detecting the vehicles 10, 10a as calculation targets via a LiDAR mounted on other vehicles 10, 10a other than the vehicles 10, 10a. C-11. Other embodiment 11:

[0085] The manual signal generation unit 621 can obtain the manual control signal by performing processing in which an operation amount corresponding to each input operation unit 30 of the operator input device 3, that is, the operation amount corresponding to each element, such as the acceleration and steering angle, that define the driving operation of the vehicles 10, 10a, is uniformly multiplied by a predetermined constant. Specifically, in the case where the manual control signal includes the acceleration and steering angle of the vehicles 10, 10a as parameters, the manual signal generation unit 621 uniformly multiplies the operation amount of the steering wheel 31, the accelerator pedal 32, and the brake pedal 33 of the operator input device 3 by the predetermined constant.Then, the manual signal generation unit 621 generates the manual control signal such that the acceleration of the vehicles 10, 10a becomes a value obtained by multiplying the operation amounts of the accelerator pedal 32 and the brake pedal 33 of the operator input device 3 by the constant. The manual signal generation unit 621 generates the manual control signal such that the steering angle of the vehicles 10, 10a becomes a value obtained by multiplying the operation amount of the steering wheel 31 of the operator input device 3 by the constant. In such a form, if an upper limit value and a lower limit value of an output value of the operator input device 3 and an upper limit value and a lower limit value of an output value of the actuator, such asthe driving device 11 of the vehicles 10, 10a, are different from each other, the manual control signal corresponding to a possible output range of the actuator of the vehicles 10, 10a is generated on a receiving side. C-12. Other embodiment 12:

[0086] The manual signal generation unit 621 may provide, for each element defining the driving operation of the vehicle 10, 10a, an upper limit value for the operation amount corresponding to each input operation unit 30 of the operator input device 3, that is, the operation amount corresponding to each element, such as the acceleration and the steering angle, that defines the driving operation of the vehicle 10, 10a. For example, the manual signal generation unit 621 performs the following processing in a case where the manual control signal includes the acceleration and the steering angle of the vehicles 10, 10a as parameters. In this case, where the operation amount of the accelerator pedal 32 of the operator input device 3 is equal to or greater than a first upper limit value, the manual signal generation unit 621 generates the manual control signal such that the acceleration of the vehicles 10, 10a becomes the first upper limit value.In a case where the operation amount of the steering wheel 31 of the operator input device 3 is equal to or greater than a second upper limit value, the manual signal generation unit 621 generates the manual control signal such that the steering angle of the vehicles 10, 10a becomes the second upper limit value. By providing the upper limit value for each element defining the driving operation of the vehicle 10, 10a, the driving operation of the vehicle 10, 10a can be prevented from changing rapidly, for example, by rapidly accelerating or rapidly turning the vehicle 10, 10a. C-13. Other embodiment 13:

[0087] In each of the embodiments described above, the vehicles 10, 10a only need to have a configuration capable of moving via unmanned driving, and may have a platform form having, for example, a configuration described below. Specifically, the vehicles 10, 10a only need to have at least the vehicle control devices 15, 15a, the driving device 11, the steering device 12, and the braking device 13 to enable the three functions of "driving," "turning," and "stopping" via unmanned driving. In a case where the vehicles 10, 10a obtain the information for unmanned driving from the outside, the vehicles 10, 10a only need to include the communication device 14. That is, the vehicles 10, 10a that can be moved via unmanned driving do not need to be equipped with at least some of the internal components, such asa driver's seat and a dashboard, do not have to be equipped with at least some of the exterior components, such as a bumper and an exterior mirror, and do not have to be equipped with the body. In this case, the remaining components, such as the body, can be installed in the vehicle 10, 10a while the vehicle 10, 10a is delivered from the factory, or the remaining components, such as the body, can be installed in the vehicle 10, 10a after the vehicle 10, 10a has been delivered from the factory in a state in which the remaining components, such as the body, are not installed in the vehicle 10, 10a. The respective components can be viewed from any direction, such asThe platform can be equipped on the top, bottom, front, rear, right, or left side of the vehicle 10, 10a, and can be equipped from the same direction or from different directions. The decision regarding the position of the platform can be made in the same way as for the vehicles 10, 10a in each of the embodiments described above. C-14. Other embodiment 14:

[0088] The vehicles 10, 10a can be manufactured by combining a plurality of modules. The module is a unit consisting of a plurality of components assembled according to the parts or functions of the vehicle 10, 10a. The platforms of the vehicles 10, 10a can be manufactured, for example, by combining a front module that forms a front part of the platform, a middle module that forms a middle part of the platform, and a rear module that forms a rear part of the platform. Furthermore, the number of modules that make up the platform is not limited to three, but can also be two or fewer or four or more. In addition to or instead of the components that make up the platform, the components that make up the parts of the vehicles 10, 10a other than the platform can be modularized.Furthermore, the various modules may include any exterior component, such as a bumper or a grille, or any interior component, such as the seat or a console. Furthermore, in addition to the vehicles 10, 10a, a moving object of any aspect may be manufactured by combining the modules. Such a module may be manufactured, for example, by joining the components together by welding or fastening, or it may be manufactured by integrally molding at least a portion of the components constituting the module as one component by casting. A casting method in which a component, especially a relatively large component, is integrally molded is also called giga-casting or mega-casting. For example, the front module, the center module, and the rear module may be manufactured using giga-casting. C-15. Other embodiment 15:

[0089] The transportation of the vehicles 10, 10a using unmanned driving is also referred to as "autonomous transportation." A configuration for implementing autonomous transportation is also referred to as a "vehicle-remotely controlled autonomous driving transportation system." A production method for manufacturing the vehicles 10, 10a using autonomous transportation is also referred to as "autonomous production." In autonomous production, for example, at least part of the transportation of the vehicles 10, 10a is realized by autonomous transportation in the factory where the vehicles 10, 10a are manufactured.

[0090] The present disclosure is not limited to the embodiments described above and can be implemented with various configurations without departing from the spirit of the present disclosure. For example, the technical features of the embodiments that correspond in any form to the technical features described in the SUMMARY OF THE INVENTION section can be replaced or combined to achieve some or all of the above-mentioned objects. In cases where the technical features are not described as necessary in this specification, the features can be deleted as needed. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] JP 2017-538619

[0002] JP 2017-538619 A

[0002]

Claims

[1] A display control device (8) configured to control a display device (4) used to move a moving object configured to move by unmanned driving, wherein: the moving object is configured to receive a driving control signal specifying a driving operation of the moving object and to drive in a manual remote driving mode using the received driving control signal; in a period during which control of the manual remote driving mode is executed, the driving control signal is generated in response to an operation performed by an operator on an operator input device (3) using an image captured by a camera and displayed on a display screen (45) of the display device (4) provided at a location other than the moving object; the camera is at least one of a built-in camera (16) installed in the moving object or an external camera (90) configured to image the moving object from an outside; the operator input device (3) is provided at a different location than the moving object; the display control device (8) comprises: at least one of an installation detection unit (821) configured to detect installation information indicating whether or not the built-in camera (16) is installed in the moving object, and an image quality detection unit (822) configured to detect image quality information indicating an image quality of a built-in camera image, which is the image captured by the built-in camera; and a display controller (823) configured to display the image on the display screen (45); and the display controller (823) is configured to display the built-in camera image on the display screen (45) in at least one of a case where the built-in camera (16) is installed in the moving object and a case where the image quality is higher than or equal to a predetermined criterion, and to display an external camera image, which is the captured image by the external camera (90) provided at a location other than the moving object, on the display screen (45) in at least one of a case where the built-in camera (16) is not installed in the moving object and a case where the image quality is lower than the criterion. [2] Display control device (8) according to claim 1, wherein the image quality is determined by a resolution of the built-in camera image and / or a distortion of the built-in camera image and / or a detection range of the built-in camera (16). [3] The display control device (8) according to claim 2, wherein the display controller (823) is configured to determine that the image quality is higher than or equal to the criterion in at least one of a first case, a second case, and a third case, the first case being when the resolution of the built-in camera image is higher than or equal to a first threshold, the second case being when the distortion of the built-in camera image is less than a predetermined second threshold, and the third case being when the detection range of the built-in camera (16) is higher than or equal to a predetermined third threshold. [4] The display control device (8) according to claim 1, wherein the moving object is further configured to drive in an automated driving mode using the driving control signal generated without responding to the operation performed by the operator on the operator input device (3); and the display controller (823) is configured to display at least one of the built-in camera image and the external camera image on the display screen (45) in accordance with the built-in information and the image quality information in at least one of a case where the automated driving mode is switched to the manual remote driving mode and a case where the switch from the automated driving mode to the manual remote driving mode is predicted. [5] The display control device (8) according to claim 4, wherein the display controller (823) is configured to display at least one of the built-in camera image and the external camera image on the display screen (45) in accordance with the installation information and the image quality information even during a period during which control of the automated driving mode is executed. [6] The display control device (8) according to claim 1, wherein the display controller (823) is configured to display the external camera image on the display screen (45) in addition to the built-in camera image when the display controller (823) displays the built-in camera image on the display screen (45). [7] A display device (4) used to move a moving object arranged to move by unmanned driving, wherein: the moving object is configured to receive a driving control signal specifying a driving operation of the moving object and to drive in a manual remote driving mode using the received driving control signal; the display device (4) comprises a display image surface (45) which is provided at a different location than the moving object and on which an image captured by a camera is displayed; in a period during which control of the manual remote drive mode is executed, the drive control signal is generated in response to an operation performed by an operator on an operator input device (3) using the image displayed on the display screen (45); the camera is at least one of a built-in camera (16) installed in the moving object or an external camera (90) configured to image the moving object from an outside; and the operator input device (3) is provided at a different location than the moving object; a built-in camera image, which is the image captured by the built-in camera (16), is displayed on the display image surface (45) in at least one of a case where the built-in camera (16) is installed in the moving object and a case where an image quality of the built-in camera image is higher than or equal to a predetermined criterion, and an external camera image, which is the image captured by the external camera (90) provided at a location different from the moving object, is displayed on the display image surface (45) in at least one of a case where the built-in camera (16) is not installed in the moving object and a case where the image quality is lower than the criterion.

Images