Display control device and display device

Abstract

A display control device including one of a mounting acquisition unit and an image quality acquisition unit, and a display controller configured to display a mounting camera image on a display screen in one of a case where the mounting camera is mounted on the moving body and a case where the image quality is equal to or greater than a standard; and an external camera image is displayed on the display screen in one of a case where the mobile body is not equipped with the camera and a case where the image quality is less than a standard.

Description

Technical Field

[0001] The present disclosure relates to a display control device and a display device. Background Art

[0002] In the related art, a vehicle that is autonomously driven by remote control is known (Japanese Unexamined Patent Application Publication (Translation of PCT Application) No. 2017-538619 (JP 2017-538619A)). Summary of the invention

[0003] In a case where a moving body such as a vehicle stops during a period during which control of autonomous driving is performed, the moving body can be remotely controlled by an operator who operates an operator input device provided at a location different from the moving body. In this case, a technology is desired in which the operator can visually recognize an image indicating the surrounding situation of the moving body so as to operate the operator input device.

[0004] The present disclosure can be implemented in the following forms.

[0005] A first aspect of the present disclosure relates to a display control device. In a display control device configured to control a display device for moving a mobile body configured to move via unmanned driving, the mobile body is configured to receive a travel control signal generated in response to an operation performed by an operator on an operator input device set at a location different from the mobile body and defining a travel operation of the mobile body, and to travel in a remote manual driving mode by using the received travel control signal. During a time period during which control of the remote manual driving mode is performed, the travel control signal is generated in response to an operation performed by the operator on the operator input device using an image acquired by a camera and displayed on a display screen of a display device set at a location different from the mobile body. The camera is at least one of an onboard camera mounted in the mobile body or an external camera configured to photograph the mobile body from the outside. The display control device includes at least one of an onboard acquisition unit or an image quality acquisition unit, and a display controller. The onboard acquisition unit is configured to acquire onboard information indicating whether an onboard camera is mounted in the mobile body. The image quality acquisition unit is configured to acquire image quality information indicating the image quality of an onboard camera image as an image acquired by the onboard camera. The display controller is configured to display the image on the display screen. The display controller is configured to display an onboard camera image on a display screen in at least one of a case where an onboard camera is mounted on the mobile body and a case where the image quality is equal to or greater than a predetermined standard; and to display an external camera image, which is an image acquired by an external camera set at a location different from the mobile body, on a display screen in at least one of a case where an onboard camera is not mounted on the mobile body and a case where the image quality is less than the standard. According to the first aspect of the present disclosure, the display control device is capable of displaying at least one of the onboard camera image and the external camera image on a display screen based on the mounting information and the image quality information. That is, when an operator operates an operator input device to remotely control the mobile body, the display control device is capable of providing an image showing the surrounding situation of the mobile body to the operator.

[0006] In a first aspect of the present disclosure, image quality may be determined by at least one of a resolution of an onboard camera image, a distortion of the onboard camera image, and a detection range of the onboard camera. According to a first aspect of the present disclosure, image quality of an onboard camera image may be determined by at least one of a resolution of an onboard camera image, a distortion of the onboard camera image, and a detection range of the onboard camera that acquires the onboard camera image.

[0007] In the first aspect of the present disclosure, the display controller can be configured to determine that the image quality is equal to or greater than a standard in at least any one of a first case, a second case, and a third case, the first case being a case where the resolution of the image of the onboard camera is equal to or greater than a first threshold, the second case being a case where the distortion of the image of the onboard camera is less than a predetermined second threshold; and the third case being a case where the detection range of the onboard camera is equal to or greater than a predetermined third threshold.

[0008] In the first aspect of the present disclosure, the mobile body may be further configured to travel in the automatic driving mode by using a driving control signal generated not in response to an operation performed by an operator on an operator input device; and the display controller may be configured to display at least one of an onboard camera image and an external camera image on a display screen according to the onboard information and the image quality information in at least any one of a case where the automatic driving mode is switched to the remote manual driving mode and a case where a switch from the automatic driving mode to the remote manual driving mode is predicted. According to the first aspect of the present disclosure, in the case where the automatic driving mode is switched to the remote manual driving mode, the display control device may display at least one of the onboard camera image and the external camera image on the display screen according to the onboard information and the image quality information. In this way, the display control device may provide an image indicating the surrounding situation of the mobile body to the operator during a time period during which control of the remote manual driving mode is performed. According to the first aspect of the present disclosure, in the case where a switch from the automatic driving mode to the remote manual driving mode is predicted, the display control device is capable of displaying at least one of the onboard camera image and the external camera image on the display screen according to the onboard information and the image quality information. In this way, the display control device can provide the operator with an image indicating the surrounding situation of the mobile body before the automatic driving mode is switched to the remote manual driving mode. Therefore, in the case where the automatic driving mode is switched to the remote manual driving mode, the display control device can allow the operator to quickly start the operation of the operator input device by allowing the operator to visually recognize the image displayed on the display screen.

[0009] In the first aspect of the present disclosure, the display controller can be configured to display at least one of the onboard camera image and the external camera image on the display screen based on the onboard information and the image quality information, even during a time period during which control of the autonomous driving mode is performed.

[0010] In the first aspect of the present disclosure, the display controller may be configured to display an external camera image on the display screen in addition to displaying the onboard camera image when the display controller displays the onboard camera image on the display screen. According to the first aspect of the present disclosure, in the case where the onboard camera image is displayed on the display screen, the display control device may display the external camera image on the display screen in addition to displaying the onboard camera image. In this way, the display control device can provide the operator with an external camera image that can be visually identified when a certain area cannot be visually identified using the onboard camera image.

[0011] A second aspect of the present disclosure relates to a display device. In a display device for moving a mobile body configured to move via unmanned driving, the mobile body is configured to receive a travel control signal generated in response to an operation performed by an operator on an operator input device set at a location different from the mobile body and defining a travel operation of the mobile body, and travel in a remote manual driving mode by using the received travel control signal. The display device includes a display screen set at a location different from the mobile body and on which an image acquired by a camera is displayed. During a time period during which control of the remote manual driving mode is performed, the travel control signal is generated in response to an operation performed by an operator on the operator input device using an image displayed on the display screen. The camera is at least one of an onboard camera mounted in the mobile body or an external camera configured to photograph the mobile body from the outside. In at least one of a case where an onboard camera is mounted in the mobile body and a case where the image quality of the onboard camera image as an image acquired by the onboard camera is equal to or greater than a predetermined standard, the onboard camera image is displayed on the display screen; and in at least one of a case where an onboard camera is not mounted in the mobile body and a case where the image quality is less than the standard, an external camera image as an image acquired by an external camera set at a location different from the mobile body is displayed on the display screen. According to the second aspect of the present disclosure, the display device can display at least one of the onboard camera image and the external camera image on the display screen, depending on whether the onboard camera is mounted in the mobile body and the image quality of the onboard camera image. That is, when the operator operates the operator input device to remotely control the mobile body, the display device can provide the operator with an image showing the surrounding situation of the mobile body.

[0012] The present disclosure may be implemented in various forms other than the above-mentioned display control device and display device. For example, the present disclosure may be implemented in the following forms: a display system including a display control device and a display device, a method for manufacturing a display control device and a display device, a method for controlling a display control device and a display device, a computer program for implementing the method for controlling a display control device and a display device, and a non-transitory recording medium recording the computer program, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described below with reference to the accompanying drawings, wherein like numerals represent like elements, and wherein: Figure 1 is a diagram showing the configuration of a traveling system in a first embodiment; Figure 2 is a diagram showing a configuration of an input system; Figure 3 is a diagram showing a configuration of an information processing system; Figure 4 is a flow chart showing an operation control method of a vehicle in an automatic driving mode; Figure 5 is a flow chart showing a display control method of a display screen in a remote manual driving mode; Figure 6 is a flowchart showing an operation control method of a vehicle in a remote manual driving mode. Figure 7 is a diagram showing a configuration of a traveling system in a second embodiment; and Figure 8 is a flow chart showing an operation control method of a vehicle in an autonomous driving mode. DETAILED DESCRIPTION A. First embodiment:

[0014] Figure 1 1 is a diagram showing the configuration of a travel system 1 in the first embodiment. The travel system 1 is a system for causing a mobile body to travel without being manipulated by a passenger riding in the mobile body. The travel system 1 includes one or more mobile bodies, an external camera 90, an input system 2 including an operator input device 3 and a display device 4, and an information processing system 5 including a remote manual control device 6, a remote automatic control device 7, and a display control device 8.

[0015] "Mobile body" refers to an object that can move, for example, a vehicle 10 or an electric vertical lift (a so-called flying car). In the present embodiment, the mobile body is a vehicle 10. The vehicle 10 may be a vehicle that runs on wheels or a vehicle that runs on tracks, such as a passenger car, a truck, a bus, a two-wheeled vehicle, a four-wheeled vehicle, a tank, or a construction vehicle. The vehicle 10 includes a pure electric vehicle (Battery Electric Vehicle, BEV), a gasoline vehicle, a hybrid vehicle, and a fuel cell electric vehicle. In addition, in the case where the mobile body is not the vehicle 10, the expressions "vehicle" and "car" in the present disclosure may be replaced with "mobile body" as appropriate, and the expression "running" may be replaced with "moving" as appropriate.

[0016] The vehicle 10 includes a driving device 11, a steering device 12, a braking device 13, a communication device 14, and a vehicle control device 15. The driving device 11 accelerates the vehicle 10. The steering device 12 changes the driving direction of the vehicle 10. The braking device 13 decelerates the vehicle 10. The communication device 14 communicates with an external device through wireless communication or the like. The external device is other devices such as the remote manual control device 6, and other vehicles 10. The communication device 14 is, for example, a wireless communication device. The vehicle control device 15 controls the operation of the vehicle 10.

[0017] The vehicle control device 15 includes an input and output interface 159, a storage unit 157, and a central processing unit (CPU) 150. The input and output interface 159 is used to communicate with various devices mounted in the vehicle 10 as the host vehicle. The storage unit 157 of the vehicle control device 15 stores various information including various programs P15 for controlling the operation of the vehicle control device 15. The CPU 150 of the vehicle control device 15 operates as an operation controller 153 and a vehicle transmission unit 154 by loading various programs P15 stored in the storage unit 157. The operation controller 153 performs driving control of the vehicle 10 by controlling the operation of the actuator that changes the acceleration and deceleration and the steering angle of the vehicle 10. "Driving control" refers to, for example, various controls for driving the actuator that exhibits the three functions of "driving", "turning" and "stopping" of the vehicle 10, such as adjustment of the acceleration, speed, steering angle, etc. of the vehicle 10. In the present embodiment, the actuators include an actuator of the drive device 11, an actuator of the steering device 12, and an actuator of the brake device 13. The vehicle transmission unit 154 transmits an onboard camera image described later to the display control device 8 in response to a request from the display control device 8.

[0018] The configuration of the vehicle 10 is not limited to the above configuration. The vehicle 10 may further include an onboard camera 16 as a camera mounted in the vehicle 10. In the case where the vehicle 10 includes one onboard camera 16, the onboard camera 16 is mounted on the front side of the vehicle 10 so that the onboard camera 16 can capture the scenery in front of the vehicle 10.

[0019] The vehicle 10 is capable of traveling via unmanned driving. "Unmanned driving" refers to driving that does not rely on the driving operation of the occupants. Driving operation refers to an operation related to at least one of "driving", "turning" and "stopping" of the vehicle 10. Unmanned driving is achieved by automatic or manual remote control using a device provided outside the vehicle 10 or by autonomous control of the vehicle 10. Occupants who do not perform driving operations may be carried in the vehicle 10 that travels via unmanned driving. Examples of occupants who do not perform driving operations include persons who simply sit in a seat of the vehicle 10, and persons who perform work different from driving operations (such as assembly, inspection, or operating switches) while riding in the vehicle 10. In addition, driving performed using the driving operation of the occupants may be referred to as "manned driving".

[0020] In the present embodiment, the vehicle 10 is driven in any one of the remote manual driving mode and the automatic driving mode in the factory where the vehicle 10 is manufactured. In the remote manual driving mode, the vehicle 10 receives from the outside a driving control signal (hereinafter referred to as a manual control signal) that defines the driving operation of the vehicle 10 and is generated in response to the operation performed by the operator on the operator input device 3. Therefore, the vehicle 10 drives by using the received manual control signal. In the automatic driving mode, the vehicle 10 receives from the outside a driving control signal (hereinafter referred to as an automatic control signal) that is generated without responding to the operation performed by the operator. Therefore, the vehicle 10 drives by using the received automatic control signal. For example, during a time period during which the control of the automatic driving mode is performed, when the vehicle 10 stops unexpectedly, even if the vehicle 10 receives an automatic control signal, the driving mode of the vehicle 10 is switched from the automatic driving mode to the remote manual driving mode. The vehicle 10 can be driven in a location different from the factory.

[0021] The external camera 90 is a camera installed at a location different from the vehicle 10. The installation position and orientation of the external camera 90 and the number of external cameras 90 installed are determined in consideration of the detection range RG (viewing angle) of the external camera 90. In the present embodiment, the external camera 90 is a camera installed in a factory. The external camera 90 captures the detection range RG including the vehicle 10 from the outside of the vehicle 10. Therefore, the external camera 90 outputs an image (hereinafter referred to as an external camera image) as a detection result.

[0022] Figure 2 2 is a diagram showing a configuration of the input system 2. The input system 2 is a system for the operator Op to remotely control the travel operation of the vehicle 10 from a location different from the vehicle 10. As described above, the input system 2 includes the operator input device 3 and the display device 4.

[0023] The operator input device 3 is an input device for the operator Op to remotely control the driving operation of the vehicle 10. The operator input device 3 is set at a location different from the vehicle 10. The operator input device 3 is connected to the remote manual control device 6 in a communicable manner. The operator input device 3 includes one or more input operation units 30 operated by the operator Op. In the present embodiment, the operator input device 3 includes a steering wheel 31, an accelerator pedal 32, and a brake pedal 33 as the input operation unit 30. The steering wheel 31 changes the driving direction of the vehicle 10 via remote control. The accelerator pedal 32 accelerates the vehicle 10 via remote control. The brake pedal 33 decelerates the vehicle 10 via remote control. The configuration of the operator input device 3 is not limited to the above configuration. The operator input device 3 may include, for example, a push button switch or a joystick switch as the input operation unit 30.

[0024] The display device 4 is used for driving of the vehicle 10. Specifically, when the operator Op operates the operator input device 3, the display device 4 is used for the operator Op to visually recognize the surrounding situation of the vehicle 10 as a control target. The display device 4 is set at a location different from the vehicle 10. The display device 4 is connected to the display control device 8 in a communicable manner. The display device 4 includes a display screen 45. The display screen 45 displays an image acquired by the camera 16, 90 including at least the surrounding area of ​​the vehicle 10 as a control target in the detection range RG. The display device 4 is set at a position where the operator Op can visually recognize the display screen 45. During the period of time during which the control of the remote manual driving mode is performed, the operator Op uses the image displayed on the display screen 45 to operate the operator input device 3.

[0025] Figure 3 is a diagram showing a configuration of the information processing system 5. As described above, the information processing system 5 includes the remote manual control device 6, the remote automatic control device 7, and the display control device 8. The functions of the information processing system 5 are realized by, for example, a server.

[0026] The remote manual control device 6 generates a manual control signal and transmits the generated manual control signal to the vehicle 10 as a control target. The remote manual control device 6 includes a communication unit 61, a storage unit 63, and a CPU 62. The communication unit 61, the storage unit 63, and the CPU 62 in the remote manual control device 6 are connected to each other via an internal bus, for example.

[0027] The communication unit 61 of the remote manual control device 6 connects the remote manual control device 6 to other devices other than the remote manual control device 6 in a communicable manner. The communication unit 61 of the remote manual control device 6 is, for example, a wireless communication device.

[0028] The storage unit 63 of the remote manual control device 6 stores various information including various programs P6 for controlling the operation of the remote manual control device 6. The storage unit 63 of the remote manual control device 6 includes, for example, a random-access memory (RAM), a read-only memory (ROM), and a hard disk drive (HDD).

[0029] The CPU 62 of the remote manual control device 6 operates as a manual signal generation unit 621 and a manual transmission unit 622 by loading various programs P6 stored in the storage unit 63. The manual signal generation unit 621 generates a manual control signal. In the present embodiment, the manual control signal includes the acceleration and the steering angle of the vehicle 10 as parameters. The manual transmission unit 622 transmits the manual control signal to the vehicle 10 as a control target. Therefore, the vehicle control device 15 mounted in the vehicle 10 receives the manual control signal from the remote manual control device 6, and drives the actuator by using the received manual control signal, so that the vehicle 10 travels in response to the operation of the operator input device 3 performed by the operator Op.

[0030] The remote automatic control device 7 generates an automatic control signal and transmits the generated automatic control signal to the vehicle 10 as a control target. The remote automatic control device 7 includes a communication unit 71, a storage unit 73, and a CPU 72. The communication unit 71, the storage unit 73, and the CPU 72 in the remote automatic control device 7 are connected to each other via an internal bus, for example.

[0031] The communication unit 71 of the remote automatic control device 7 connects the remote automatic control device 7 to other devices other than the remote automatic control device 7 in a communicable manner. The communication unit 71 of the remote automatic control device 7 is, for example, a wireless communication device.

[0032] The storage unit 73 of the remote automatic control device 7 stores various information including various programs P7, a detection model Md, and an ideal path Ip for controlling the operation of the remote automatic control device 7. The details of the detection model Md and the ideal path Ip will be described later. The storage unit 63 of the remote automatic control device 7 includes, for example, a RAM, a ROM, and a hard disk drive (HDD).

[0033] The CPU 72 of the remote automatic control device 7 operates as a calculation unit 721 , an automatic signal generation unit 722 , and an automatic transmission unit 723 by loading various programs P7 stored in the storage unit 73 .

[0034] The calculation unit 721 calculates the position of the vehicle 10. In the present embodiment, the calculation unit 721 calculates the position of the vehicle 10 by using an external camera image output from an external camera 90 as an external sensor Se provided at a location different from the vehicle 10. The calculation unit 721 calculates the coordinates of the positioning point 10e of the vehicle 10 in the image coordinate system by using, for example, the appearance shape of the vehicle 10 detected from the external camera image. Then, the calculation unit 721 calculates the position of the vehicle 10 by converting the calculated coordinates into coordinates in the global coordinate system. In the present embodiment, the position information of the vehicle 10 includes X, Y, and Z coordinates in the global coordinate system of the factory.

[0035] For example, the appearance shape of the vehicle 10 included in the external camera image can be detected by inputting the external camera image into the detection model Md using artificial intelligence. Examples of the detection model Md include a trained machine learning model that has been trained to achieve any semantic segmentation and instance segmentation. As the detection model Md, for example, a convolutional neural network (CNN) that has been trained by supervised learning using a training data set can be used. The training data set includes, for example, a plurality of training images including the vehicle 10 and a correct answer label indicating whether each area in the training image indicates an area of ​​the vehicle 10 or an area outside the vehicle 10. When the CNN is trained, the parameters of the CNN are preferably updated by back propagation (error back propagation method) to reduce the error between the output result of the detection model Md and the correct answer label.

[0036] The calculation unit 721 can also calculate the orientation of the vehicle 10. The orientation of the vehicle 10 can be calculated by using an optical flow method based on, for example, the orientation of a motion vector of the vehicle 10 calculated from a position change of a feature point of the vehicle 10 between frames of an external camera image. For example, the orientation of the vehicle 10 can be calculated by using an output result of a yaw rate sensor or the like mounted in the vehicle 10. The calculation unit 721 may be a single function of a device other than the remote automatic control device 7 such as the external camera 90. In this case, the remote automatic control device 7 includes an acquisition unit that acquires the position information of the vehicle 10, but does not include the calculation unit 721.

[0037] The automatic signal generating unit 722 generates an automatic control signal. The automatic signal generating unit 722 first determines the target position to which the vehicle 10 should go next to generate the automatic control signal. In the present embodiment, the target position is represented by X, Y, and Z coordinates in the global coordinate system of the plant. The automatic signal generating unit 722 determines the target position to which the vehicle 10 should go next by using the position information of the vehicle 10 and the ideal path Ip pre-stored in the storage unit 73 of the remote automatic control device 7. The ideal path Ip is the path on which the vehicle 10 should travel. The path is represented by a node indicating a starting point, a node indicating a passing point, a node indicating a destination, and a link connecting each node. The automatic signal generating unit 722 determines the target position on the ideal path Ip ahead of the current position of the vehicle 10. The automatic signal generating unit 722 generates an automatic control signal for causing the vehicle 10 to travel toward the determined target position. The automatic signal generating unit 722 calculates the driving speed of the vehicle 10 according to the transition of the position of the vehicle 10, and compares the calculated driving speed with the predetermined target speed of the vehicle 10. The automatic signal generating unit 722 determines the acceleration so that the vehicle 10 accelerates when the travel speed is lower than the target speed, and determines the acceleration so that the vehicle 10 decelerates when the travel speed is higher than the target speed. The automatic signal generating unit 722 determines the steering angle so that the vehicle 10 does not deviate from the ideal path Ip when the vehicle 10 is on the ideal path Ip, and determines the steering angle so that the vehicle 10 returns to the ideal path Ip when the vehicle 10 is not on the ideal path Ip (in other words, when the vehicle 10 deviates from the ideal path Ip). In this way, the automatic signal generating unit 722 generates a travel control signal including the acceleration and steering angle of the vehicle 10 as parameters.

[0038] The automatic transmission unit 723 transmits the automatic control signal to the vehicle 10 as the control target. Therefore, the vehicle control device 15 receives the automatic control signal from the remote automatic control device 7, and drives the actuator by using the automatic control signal so that the vehicle 10 drives autonomously without responding to the operation of the operator input device 3 by the operator Op.

[0039] During the period of time during which the control of the automatic driving mode is performed, the remote automatic control device 7 repeatedly performs the acquisition of the position information of the vehicle 10, the determination of the target position, the generation of the automatic control signal, and the transmission of the automatic control signal at a predetermined cycle. The vehicle control device 15 repeatedly performs the reception of the automatic control signal and the control of the actuator at a predetermined cycle.

[0040] The display control device 8 controls display contents displayed on the display screen 45. The display control device 8 includes a communication unit 81, a storage unit 83, and a CPU 82. The communication unit 81, the storage unit 83, and the CPU 82 in the display control device 8 are connected to each other via, for example, an internal bus.

[0041] The communication unit 81 of the display control device 8 connects the display control device 8 to other devices other than the display control device 8 in a communicable manner. The communication unit 81 of the display control device 8 is, for example, a wireless communication device.

[0042] The storage unit 83 of the display control device 8 stores various information including various programs P8 for controlling the operation of the display control device 8. The storage unit 83 of the display control device 8 includes, for example, a RAM, a ROM, and a hard disk drive (HDD).

[0043] The CPU 82 of the display control device 8 operates as a loading acquisition unit 821 , an image quality acquisition unit 822 , and a display controller 823 by loading various programs P8 stored in the storage unit 83 .

[0044] The mounting acquisition unit 821 acquires the mounting information. The mounting information is information indicating whether the mounting camera 16 is mounted in the vehicle 10 as the control target.

[0045] The image quality acquisition unit 822 acquires image quality information. The image quality information is information indicating the image quality of an image acquired by the onboard camera 16 mounted in the vehicle 10 as the control target (hereinafter referred to as the onboard camera image). The image quality of the onboard camera image is determined according to at least one of the resolution of the onboard camera image, the distortion of the onboard camera image, and the detection range (angle of view) of the onboard camera 16 mounted in the vehicle 10 as the control target. The resolution of the onboard camera image is calculated, for example, by specifying the model of the onboard camera 16 mounted in the vehicle 10 as the control target, and acquiring information indicating the performance of the onboard camera 16 (hereinafter referred to as the performance information). The resolution of the onboard camera image can be calculated based on the number of pixels, image size, etc. of the actually acquired onboard camera image. The distortion of the onboard camera image is calculated, for example, by analyzing the actually acquired onboard camera image. The detection range of the onboard camera 16 mounted in the vehicle 10 as the control target is calculated, for example, by specifying the model of the onboard camera 16 mounted in the vehicle 10 as the control target, and acquiring the performance information.

[0046] The image quality of the mounted camera image in the image quality information refers to the image quality when the display device 4 receives the mounted camera image. When the mounted camera 16 mounted in the vehicle 10 as the control target has a processing function, the image quality of the mounted camera image in the image quality information refers to the image quality of the image (hereinafter referred to as the processed image) obtained by processing the original data (hereinafter referred to as the original image) acquired by the mounted camera 16 mounted in the vehicle 10 as the control target. The processing function is a function of processing the original image to generate a processed image. The processing function is, for example, a correction function for correcting the distortion of the original image, a noise removal function for removing the noise of the original image, and a resolution adjustment function for adjusting the resolution of the original image. In the case where the mounted camera image is transmitted from the mounted camera 16 to the display device 4 via the display control device 8, when the display control device 8 has a processing function, the image quality of the mounted camera image in the image quality information refers to the image quality of the processed image. The method for evaluating the image quality of the mounted camera image is not limited to the above method. Among them, the image quality of the mounted camera image can be evaluated by an evaluation index other than the noise amount of the mounted camera image.

[0047] The display controller 823 displays at least one of the onboard camera image and the external camera image on the display screen 45. In the present embodiment, when the driving mode of the vehicle 10 is switched from the automatic driving mode to the remote manual driving mode, the display controller 823 displays at least one of the onboard camera image and the external camera image on the display screen 45 based on the onboard information and the image quality information.

[0048] The display controller 823 displays the onboard camera image on the display screen 45 in at least one of a case where the onboard camera 16 is mounted in the vehicle 10 as the control target and a case where the image quality of the onboard camera image is equal to or greater than a predetermined standard. On the other hand, the display controller 823 displays the external camera image on the display screen 45 in at least one of a case where the onboard camera 16 is not mounted in the vehicle 10 as the control target and a case where the image quality of the onboard camera image is less than the standard.

[0049] For example, in a case where the image quality of the onboard camera image is determined by the resolution of the onboard camera image, the display controller 823 determines that the image quality of the onboard camera image is equal to or greater than the standard when the resolution of the onboard camera image is equal to or greater than a predetermined resolution threshold. For example, in a case where the image quality of the onboard camera image is determined by the resolution of the onboard camera image, the distortion of the onboard camera image, and the detection range of the onboard camera 16, the display controller 823 determines that the image quality of the onboard camera image is equal to or greater than the standard in the following case. In this case, the display controller 823 determines that the image quality of the onboard camera image is equal to or greater than the standard in at least any one of the first case, the second case, and the third case. The first case is a case where the resolution of the onboard camera image is equal to or greater than the resolution threshold. The second case is a case where the distortion of the onboard camera image is less than a predetermined distortion threshold. The third case is a case where the detection range of the onboard camera 16 is equal to or greater than a predetermined range threshold.

[0050] In the present embodiment, when the onboard camera 16 is mounted in the vehicle 10 as the control target, and the image quality of the onboard camera image is equal to or greater than the standard, the display controller 823 displays only the onboard camera image on the display screen 45. On the other hand, when the onboard camera 16 is mounted in the vehicle 10 as the control target, and the image quality of the onboard camera image is less than the standard, the display controller 823 displays the onboard camera image and the external camera image on the display screen 45. When the onboard camera 16 is not mounted in the vehicle 10 as the control target, the display controller 823 displays the external camera image on the display screen 45.

[0051] Figure 4 : is a flowchart showing an example of an operation control method of the vehicle 10 during a period during which control of the automatic driving mode is executed. Figure 4 The operation control method shown in , for example, is repeatedly executed at predetermined time intervals from the time point when the vehicle 10 as the control target starts traveling in the automatic driving mode to the time point when the driving mode of the vehicle 10 is switched from the automatic driving mode to the remote manual driving mode.

[0052] In step S101, the calculation unit 721 of the remote automatic control device 7 transmits a request signal for acquiring an external camera image to the external camera 90. In step S102, the external camera 90 having received the request signal transmits the external camera image to the remote automatic control device 7. In step S103, the calculation unit 721 of the remote automatic control device 7 calculates the position of the control target vehicle 10 by using the external camera image. In step S104, the automatic signal generation unit 722 determines the target position to which the control target vehicle 10 should go next by using the position information of the vehicle 10 and the ideal path Ip. In step S105, the automatic signal generation unit 722 generates an automatic control signal for causing the vehicle 10 as the control target to travel toward the determined target position. In step S106, the automatic transmission unit 723 transmits the automatic control signal to the vehicle 10 as the control target. In step S107 , the operation controller 153 of the vehicle control device 15 mounted in the vehicle 10 controls the actuator by using the received automatic control signal so that the vehicle 10 travels at the acceleration and the steering angle indicated by the automatic control signal.

[0053] Figure 5 : is a flowchart showing a display control method of the display screen 45 during a period during which control of the remote manual driving mode is performed. Figure 5 The display control method shown in is executed, for example, when the driving mode of the vehicle 10 is switched from the automatic driving mode to the remote manual driving mode.

[0054] In step S201, the mounted acquisition unit 821 of the display control device 8 acquires the mounted information. In the case where the mounted camera 16 is mounted in the vehicle 10 as the control target (step S202: Yes), the display controller 823 executes step S203. In step S203, the display controller 823 transmits a request signal for acquiring the mounted camera image to the vehicle 10 as the control target. In the case where the vehicle 10 receives the request signal, in step S204, the vehicle transmission unit 154 of the vehicle control device 15 mounted in the vehicle 10 acquires the mounted camera image and transmits the acquired mounted camera image to the display control device 8. In step S205, the image quality acquisition unit 822 of the display control device 8 acquires the image quality information. In the case where the image quality of the mounted camera image is equal to or greater than the standard (step S206: Yes), in step S207, the display controller 823 transmits the mounted camera image to the display device 4. On the other hand, in the case where the onboard camera 16 is not mounted in the vehicle 10 (step S202: No), and in the case where the image quality of the onboard camera image is less than the standard (step S206: No), in step S208, the display controller 823 transmits a request signal for acquiring the external camera image to the external camera 90. In step S209, the external camera 90 having received the request signal transmits the external camera image to the display control device 8. In the case where the onboard camera 16 is mounted in the vehicle 10 as the control target (step S202: Yes), and the image quality of the onboard camera image is less than the standard (step S206: No), the display controller 823 of the display control device 8 executes step S210. In step S210, the display controller 823 transmits the onboard camera image and the external camera image to the display device 4. In the case where the onboard camera 16 is not mounted in the vehicle 10 (step S202: No), the display controller 823 executes step S211. In step S211, the display control device 8 transmits the external camera image to the display device 4. In step S212 , the display device 4 displays the received image on the display screen 45 .

[0055] Figure 6 : is a flowchart showing an operation control method of the vehicle 10 during a period during which control of the remote manual driving mode is performed. Figure 6 The operation control method shown in is, for example, repeatedly executed at predetermined time intervals from the time point when the vehicle 10 as the control target starts traveling in the remote manual driving mode to the time point when the driving mode of the vehicle 10 is switched from the remote manual driving mode to the automatic driving mode.

[0056] In step S301, the operator input device 3 receives the input of the operator Op. In step S302, the operator input device 3 transmits the operation amount of each input operation unit 30 to the remote manual control device 6. In step S303, the manual signal generation unit 621 of the remote manual control device 6 generates a manual control signal. In step S304, the manual transmission unit 622 transmits the manual control signal to the vehicle 10 as the control target. In step S305, the operation controller 153 of the vehicle control device 15 mounted in the vehicle 10 controls the actuator by using the received manual control signal so that the vehicle 10 travels at the acceleration and steering angle indicated by the manual control signal.

[0057] According to the first embodiment, the display control device 8 can display an image including at least the surrounding area of ​​the control target vehicle 10 within the detection range RG acquired by the camera 16, 90 on the display screen 45 of the display device 4 provided at a position that the operator Op can visually recognize. That is, in the case where the vehicle 10 is remotely controlled by operating the operator input device 3 via the operator Op, the display control device 8 can provide the operator Op with an image indicating the surrounding situation of the vehicle 10. Therefore, the operator Op can operate the operator input device 3 while visually recognizing the image displayed on the display screen 45.

[0058] According to the first embodiment, the display control device 8 can recognize whether the mounted camera 16 is mounted in the vehicle 10 as the control target by acquiring the mounted information. Therefore, the display control device 8 can display the mounted camera image on the display screen 45 by acquiring the mounted camera image from the control target vehicle 10 in the case where the mounted camera 16 is mounted in the control target vehicle 10. The scenery represented by the mounted camera image acquired by the mounted camera 16 attached to the front side of the vehicle 10 is close to the scenery of the front side of the vehicle 10 as the control target viewed from the driver's seat of the vehicle 10. Therefore, by displaying the mounted camera image on the display screen 45, the input system 2 can reproduce a situation close to the situation where the driver riding in the vehicle 10 as the control target operates the steering wheel and the like mounted in the vehicle 10. In this way, in the case where the vehicle 10 is remotely controlled by the operator Op who operates the operator input device 3, the display control device 8 can easily allow the operator Op to recognize objects and the like existing around the vehicle 10. That is, the display control device 8 can create a state in which the operator Op can easily operate the operator input device 3. Therefore, the possibility that the vehicle 10 collides with other objects or deviates from the path on which the vehicle 10 should travel due to an operational error by the operator Op can be reduced.

[0059] According to the first embodiment, in a case where the onboard camera 16 is not mounted in the vehicle 10 as the control target, the display control device 8 can display the external camera image on the display screen 45 by acquiring the external camera image from the external camera 90. Therefore, even in a case where the onboard camera 16 is not mounted in the vehicle 10, the operator Op can operate the operator input device 3 by using the external camera image.

[0060] According to the first embodiment, the display control device 8 can acquire image quality information and compare the image quality information with a predetermined standard to determine whether the image quality of the mounted camera image is equal to or greater than the standard. Therefore, in the case where the image quality of the mounted camera image is equal to or greater than the standard, the display control device 8 can display the mounted camera image on the display screen 45 by acquiring the mounted camera image from the vehicle 10 as the control target. In this way, the display control device 8 can display an image with ensured visibility on the display screen 45. Therefore, in the case where the vehicle 10 is remotely controlled by the operator Op who operates the operator input device 3, the display control device 8 can easily allow the operator Op to recognize objects and the like existing around the vehicle 10. Therefore, the possibility of the vehicle 10 colliding with other objects or deviating from the path on which the vehicle 10 should travel due to an operational error of the operator Op can be further reduced.

[0061] According to the first embodiment, even in the case where the onboard camera 16 is mounted in the vehicle 10 as the control target, the display control device 8 can display the external camera image on the display screen 45 in addition to displaying the onboard camera image when the image quality of the onboard camera image is less than the standard. In this way, the display control device 8 can display the display screen 45 in a state where the portion of the onboard camera image that is difficult to visually recognize is supplemented by the external camera image. Therefore, the operator Op can visually recognize both the onboard camera image and the external camera image to supplement the portion of the onboard camera image that is difficult to visually recognize with the visually recognized external camera image, while operating the operator input device 3.

[0062] According to the first embodiment, the image quality of the mounted camera image can be determined by at least one of the resolution of the mounted camera image, the distortion of the mounted camera image, and the detection range of the mounted camera 16 that acquires the mounted camera image. Therefore, the image quality of the mounted camera image can be easily evaluated.

[0063] According to the first embodiment, in a case where the image quality of the onboard camera image is determined by the resolution and distortion of the onboard camera image, the display control device 8 can display the onboard camera image and the external camera image on the display screen 45 when the image quality of the onboard camera image is less than the standard. In this way, even in a case where the onboard camera image is unclear or distorted and difficult to be visually recognized, the display control device 8 can allow the operator Op to operate the operator input device 3 by displaying the external camera image on the display screen 45. Therefore, it is possible to avoid a situation where the operator Op cannot operate the operator input device 3 due to the image quality of the onboard camera image. Further, in a case where the image quality of the onboard camera image is determined by the detection range of the onboard camera 16 that acquires the onboard camera image, the display control device 8 can display the onboard camera image and the external camera image on the display screen 45 when the image quality of the onboard camera image is less than the standard. In this way, it is possible to avoid a situation where the operation of the operator input device 3 is hindered or an object other than the vehicle 10 as a control target is likely to collide with the vehicle 10 due to the fact that the detection range of the onboard camera 16 is less than the desired detection range.

[0064] According to the first embodiment, in the case where the automatic driving mode is switched to the remote manual driving mode, the display control device 8 can display at least one of the onboard camera image and the external camera image on the display screen 45 based on the onboard information and the image quality information. In this way, the display control device 8 can provide the operator Op with an image indicating the surrounding situation of the vehicle 10 during the time period during which the control of the remote manual driving mode is performed.

[0065] According to the first embodiment, the travel system 1 can make the vehicle 10 travel in the factory via remote control. Therefore, the travel system 1 can move the vehicle 10 in the factory without using a transport device such as a crane or a belt conveyor. B. Second embodiment:

[0066] Figure 7is a diagram showing the configuration of the travel system 1a in the second embodiment. The travel system 1a includes a vehicle 10a as one or more moving bodies, an external camera 90, an input system 2, and an information processing system 5a. The travel system 1a according to the present embodiment is different from the first embodiment in that the remote automatic control device 7 is not provided. That is, the information processing system 5a includes a remote manual control device 6 and a display control device 8, but does not include the remote automatic control device 7. Therefore, the vehicle control device 15a mounted in the vehicle 10a determines the target position to which the vehicle 10a should go next, and generates an automatic control signal for causing the vehicle 10a to travel toward the determined target position. That is, the vehicle 10a has an autonomous driving mode in which the vehicle 10a autonomously performs the determination of the target position and the generation of the automatic control signal, as an automatic driving mode in which the vehicle 10a travels by using an automatic control signal generated without responding to an operation performed by the operator Op.

[0067] The vehicle control device 15a includes an input and output interface 159, a storage unit 157a, and a CPU 150a. The storage unit 157a of the vehicle control device 15a stores various information including various programs P15a, a detection model Md, and an ideal path Ip for controlling the vehicle control device 15a. The CPU 150a of the vehicle control device 15a operates as a calculation unit 151, an automatic signal generation unit 152, an operation controller 153, and a vehicle transmission unit 154 by loading various programs P15a stored in the storage unit 157a. The calculation unit 151 calculates the position of the vehicle 10a by using an external camera image in the same manner as the calculation unit 721 in the first embodiment. The automatic signal generation unit 152 determines the target position to which the vehicle 10a should go next by using the calculated position information of the vehicle 10a and the ideal path Ip pre-stored in the storage unit 157a of the vehicle control device 15a. Then, the automatic signal generation unit 722 generates an automatic control signal for causing the vehicle 10a to travel toward the determined target position. The operation controller 153 controls the actuator by using the generated automatic control signal so that the vehicle 10a travels at the acceleration and steering angle indicated by the automatic control signal. During the period of time during which the control of the autonomous driving mode is performed, the vehicle control device 15a repeatedly performs the acquisition of the position information of the vehicle 10a, the determination of the target position, the generation of the automatic control signal, and the control of the actuator at a predetermined cycle. The same components as those in the first embodiment are denoted by the same reference numerals.

[0068] Figure 8 : is a flowchart showing an example of an operation control method of the vehicle 10a during a period during which control of the autonomous driving mode is executed. Figure 8The operation control method shown in , for example, is repeatedly executed at predetermined time intervals from a point in time when the vehicle 10a as a control target starts traveling in the autonomous driving mode to a point in time when the driving mode of the vehicle 10a is switched from the autonomous driving mode to the remote manual driving mode.

[0069] In step S401, the calculation unit 151 of the vehicle control device 15a mounted in the vehicle 10a transmits a request signal for acquiring an external camera image to the external camera 90. In step S402, the external camera 90 having received the request signal transmits the external camera image to the vehicle 10a. In step S403, the calculation unit 151 of the vehicle control device 15a mounted in the vehicle 10a calculates the position of the vehicle 10a as a control target by using the external camera image. In step S404, the automatic signal generation unit 152 determines the target position to which the vehicle 10a as a control target should go next by using the position information of the vehicle 10a and the ideal path Ip. In step S405, the automatic signal generation unit 152 generates an automatic control signal for causing the vehicle 10a to travel toward the determined target position. In step S406, the operation controller 153 controls the actuator by using the automatic control signal so that the vehicle 10a travels at the acceleration and steering angle indicated by the automatic control signal.

[0070] According to the second embodiment, the travel system 1a can autonomously drive the vehicle 10a. Therefore, the travel system 1a can move the vehicle 10a without using a transport device such as a crane or a belt conveyor.

[0071] The calculation unit 151 may be a single function of other devices other than the remote automatic control device 7 such as the external camera 90. In this case, the vehicle control device 15a of the vehicle 10a includes an acquisition unit that acquires position information of the vehicle 10a, but does not include the calculation unit 151. C. Other embodiments: C-1. Other Example 1:

[0072] The display control device 8 may include the onboard acquisition unit 821, but not the image quality acquisition unit 822. In this case, when the onboard camera 16 is mounted in the vehicle 10, 10a as the control target, the display controller 823 displays the onboard camera image on the display screen 45 regardless of whether the image quality of the onboard camera image is greater than or equal to the standard. The display control device 8 may include the image quality acquisition unit 822, but not the onboard acquisition unit 821. In this case, the image quality acquisition unit 822 determines that the onboard camera 16 is mounted in the vehicle 10, 10a as the control target when the image quality information can be acquired. Then, when the image quality of the onboard camera image is equal to or greater than the standard, the display controller 823 displays the onboard camera image on the display screen 45. When the image quality acquisition unit 822 can acquire the image quality information and the image quality of the onboard camera image is less than the standard, the display controller 823 displays the onboard camera image and the external camera image on the display screen 45. Even in such a form, the display controller 823 can display at least one of the mounted camera image and the external camera image on the display screen 45 according to any one of the mounted information and the image quality information. C-2. Other Example 2:

[0073] In the case where the switch from the automatic driving mode to the remote manual driving mode is predicted, the display controller 823 may display at least one of the mounted camera image and the external camera image on the display screen 45 based on the mounted information and the image quality information. For example, the display controller 823 may display at least one of the mounted camera image and the external camera image on the display screen 45 in response to the prediction that the automatic driving mode is switched to the remote manual driving mode. In this form, in the case where the switch from the automatic driving mode to the remote manual driving mode is predicted, the display control device 8 can display at least one of the mounted camera image and the external camera image on the display screen 45 based on the mounted information and the image quality information. In this way, the display controller 823 can display at least one of the mounted camera image and the external camera image on the display screen 45 before the automatic driving mode is switched to the remote manual driving mode. Therefore, in the case where the automatic driving mode is switched to the remote manual driving mode, the operator Op can visually recognize the image displayed on the display screen 45 and quickly start the operation of the operator input device 3. In the case where the vehicle 10 , 10 a stops unexpectedly during a period during which control of the automatic driving mode is executed, and the automatic driving mode is switched to the remote manual driving mode, the time during which the vehicle 10 , 10 a stops unexpectedly can be minimized. C-3. Other Embodiment 3:

[0074] In cases other than the case where the automatic driving mode is predicted to be switched to the remote manual driving mode and the case where the switching from the automatic driving mode to the remote manual driving mode is predicted, the display controller 823 may display at least one of the mounted camera image and the external camera image on the display screen 45 based on the mounted information and the image quality information. For example, even in a time period during which the control of the automatic driving mode is performed, the display controller 823 may display at least one of the mounted camera image and the external camera image on the display screen 45 based on the mounted information and the image quality information. In such a form, in a case where it is necessary to switch from the automatic driving mode to the remote manual driving mode, the operator Op can visually recognize the image displayed on the display screen 45 and operate the operator input device 3 more quickly. Further, the operator Op can appropriately check the driving condition of the vehicle 10, 10a during the time period during which the control of the automatic driving mode is performed. C-4. Other Example 4:

[0075] The travel system 1 can detect an area outside the detection range of the onboard camera 16 in the surrounding area of ​​the vehicle 10, 10a via the external camera 90. Then, in the case where the display controller 823 displays the onboard camera image on the display screen 45, the display controller 823 can display the external camera image on the display screen 45 in addition to displaying the onboard camera image. That is, in the case where the image quality of the onboard camera image is equal to or greater than the standard, or in the case where the onboard camera image is displayed on the display screen 45 regardless of whether the image quality of the onboard camera image is equal to or greater than the standard, the display controller 823 can display the onboard camera image and the external camera image on the display screen 45. In this form, the operator Op can visually recognize the area that cannot be visually recognized using the onboard camera image by using the external camera image. Therefore, in the case where the operator Op operates the operator input device 3 by using the image displayed on the display screen 45, the blind spot occurring in the surrounding area of ​​the vehicle 10, 10a can be reduced. C-5. Other Embodiment 5:

[0076] The vehicle 10, 10a may include a plurality of mounted cameras 16. In this case, for example, each mounted camera 16 is attached to the vehicle 10, 10a in such a manner that the overlap between the detection ranges of adjacent mounted cameras 16 is small. The display control device 8 displays the mounted camera images respectively acquired by the mounted cameras 16 on the display screen 45. In this form, compared with a case where one mounted camera 16 is provided in the vehicle 10, 10a, when the operator Op operates the operator input device 3 by using the image displayed on the display screen 45, the possibility of a blind spot occurring in the surrounding area of ​​the vehicle 10, 10a can be reduced. In addition, in the case where four mounted cameras 16 are provided in the vehicle 10, 10a, the mounted cameras 16 may be attached to the front side, the rear side, the right side, and the left side of the vehicle 10, 10a, respectively. The scenery represented by the mounted camera image acquired by the mounted camera 16 attached to the rear side of the vehicle 10, 10a is close to the scenery of the rear side of the vehicle 10, 10a as the control target viewed from the driver's seat of the vehicle 10, 10a. The scenery represented by the onboard camera image acquired by the onboard camera 16 attached to the right side of the vehicle 10, 10a is close to the scenery of the right side of the vehicle 10, 10a as the control target as viewed from the driver's seat of the vehicle 10, 10a. The scenery represented by the onboard camera image acquired by the onboard camera 16 attached to the left side of the vehicle 10, 10a is close to the scenery of the left side of the vehicle 10, 10a as the control target as viewed from the driver's seat of the vehicle 10, 10a. Therefore, the onboard camera images acquired by the onboard cameras 16 attached to the front side, rear side, right side and left side of the vehicle 10, 10a are displayed on the display screen 45, thereby making it possible to make the scenery closer to the scenery viewed from the driver's seat. C-6. Other Example 6:

[0077] The travel system 1 can detect the surrounding area of ​​the vehicle 10, 10a via the external camera 90. The display controller 823 can display the external camera images respectively acquired by the external camera 90 on the display screen 45. In this form, the display controller 823 can display the external camera images detecting the surrounding area of ​​the vehicle 10, 10a as the control target from different directions on the display screen 45. Therefore, in the case where the operator Op operates the operator input device 3 by using the image displayed on the display screen 45, the blind spot occurring in the surrounding area of ​​the vehicle 10, 10a can be reduced. C-7. Other Example 7:

[0078] The display control device 8 may further include a communication information acquisition unit that acquires communication information. The communication information is information indicating whether the vehicle 10, 10a as the control target is able to transmit the onboard camera image to the display control device 8. In this case, when the vehicle 10, 10a as the control target is able to transmit the onboard camera image to the display control device 8, the display control device 8 acquires image quality information or displays the onboard camera image on the display screen 45. On the other hand, when the vehicle 10, 10a as the control target is unable to transmit the onboard camera image to the display control device 8, the display control device 8 displays the external camera image on the display screen 45. In this form, when the display controller 823 is unable to transmit the onboard camera image to the display control device 8 due to reasons such as communication load, the display controller 823 can display the external camera image on the display screen 45. C-8. Other Example 8:

[0079] At least any one of the remote manual control device 6, the remote automatic control device 7, and the display control device 8 may be configured integrally. In addition, each unit of the remote manual control device 6, the remote automatic control device 7, and the display control device 8 may be implemented by cloud computing in which, for example, one or more computers are used. In such a form, the configuration of the remote manual control device 6, the remote automatic control device 7, and the display control device 8 may be appropriately changed. C-9. Other Examples 9:

[0080] In the first embodiment, the process from the acquisition of the position information of the vehicle 10 to the generation of the travel control signal is performed by the remote automatic control device 7. Meanwhile, the vehicle 10 may perform at least a part of the process from the acquisition of the position information of the vehicle 10 to the generation of the automatic control signal. For example, the following forms (1) to (3) may be used.

[0081] (1) The remote automatic control device 7 can acquire the position information of the vehicle 10, determine the target position to which the vehicle 10 should go next, and generate a path from the current position of the vehicle 10 indicated by the acquired position information to the target position. The remote automatic control device 7 can generate a path to the target position between the current position and the destination, or can also generate a path to the destination. The remote automatic control device 7 can transmit the generated path to the vehicle 10. The vehicle 10 can generate an automatic control signal for causing the vehicle 10 to travel on the path received from the remote automatic control device 7, and control the actuator by using the generated automatic control signal.

[0082] (2) The remote automatic control device 7 can acquire the position information of the vehicle 10 and transmit the acquired position information to the vehicle 10. The vehicle 10 can determine the target position to which the vehicle 10 should go next, generate a path from the current position of the vehicle 10 indicated by the received position information to the target position, generate an automatic control signal for causing the vehicle 10 to travel on the generated path, and control the actuator by using the generated automatic control signal.

[0083] (3) In forms (1) and (2), an internal sensor Si may be mounted in the vehicle 10, and the detection result output from the internal sensor Si may be used for at least one of the generation of the path and the generation of the automatic control signal. Examples of the internal sensor Si include a mounted camera 16, a mounted LiDAR, a millimeter wave radar, an ultrasonic sensor, a GPS sensor, an acceleration sensor, and a gyroscope sensor. For example, in form (1), the remote automatic control device 7 may obtain the detection result of the internal sensor Si, and reflect the detection result of the internal sensor Si in the path when generating the path. In form (1), the vehicle 10 may obtain the detection result of the internal sensor Si, and reflect the detection result of the internal sensor Si in the automatic control signal when generating the automatic control signal. In form (2), the vehicle 10 may obtain the detection result of the internal sensor Si, and reflect the detection result of the internal sensor Si in the path when generating the path. In form (2), the vehicle 10 may obtain the detection result of the internal sensor Si, and reflect the detection result of the internal sensor Si in the path when generating the automatic control signal.

[0084] (4) In the second embodiment, the vehicle 10a may be equipped with an internal sensor Si, and the detection result outputted from the internal sensor Si may be used for at least one of the generation of the route and the generation of the automatic control signal. For example, the vehicle 10a may obtain the detection result of the internal sensor Si, and reflect the detection result of the internal sensor Si in the route when generating the route. The vehicle 10a may obtain the detection result of the internal sensor Si, and reflect the detection result of the internal sensor Si in the automatic control signal when generating the automatic control signal.

[0085] (5) In the second embodiment, the vehicle 10a acquires the position information of the vehicle 10a by using the detection result of the external sensor Se. At the same time, the vehicle 10a may be equipped with an internal sensor Si, and the vehicle 10a may acquire the position information by using the detection result of the internal sensor Si, determine the target position that the vehicle 10a should go to next, generate a path from the current position of the vehicle 10a represented by the acquired position information to the target position, generate an automatic control signal for causing the vehicle 10a to travel on the generated path, and control the actuator by using the generated automatic control signal. In this case, the vehicle 10a can travel without using the detection result of the external sensor Se at all. The vehicle 10a can acquire the target arrival time or traffic congestion information from outside the vehicle 10a, and reflect the target arrival time or traffic congestion information in at least one of the path and the automatic control signal. In addition, the functional configuration of the travel system 1, 1a can be provided in the vehicle 10, 10a. That is, the processing implemented by the travel system 1 , 1 a such as travel control of the vehicle 10 , 10 a , display control of the display device 4 described in the present disclosure may be implemented only by the vehicle 10 , 10 a . C-10. Other Examples 10:

[0086] The calculation unit 151, 721 can calculate the position and orientation of the vehicle 10, 10a by using sensor information obtained by a different type of sensor from the external camera 90. The calculation unit 151, 721 can calculate the position and orientation of the vehicle 10, 10a by using, for example, measurement point group data that three-dimensionally represents the appearance shape of the vehicle 10, 10a, obtained by a light detection and ranging (Light Detection And Ranging, LiDAR) (hereinafter referred to as external LiDAR) as an external sensor Se. In this case, the calculation unit 151, 721 calculates the position and orientation of the vehicle 10, 10a by, for example, matching the measurement point group data with reference point group data. The reference point group data is reference data used as a template in the process of matching with the measurement point group data. The reference point group data is, for example, virtual three-dimensional point group data generated based on three-dimensional CAD data representing the appearance shape of the vehicle 10, 10a. The matching algorithm performed by the calculation unit 151, 721 includes, for example, iterative closest point (ICP) and normal distribution transform (NDT). The calculation unit 151, 721 can calculate the position and orientation of the vehicle 10, 10a by using the measurement point group data obtained by detecting the vehicle 10, 10a as the calculation target through the LiDAR mounted in other vehicles 10, 10a different from the vehicle 10, 10a as the calculation target. C-11. Other Examples 11:

[0087] The manual signal generating unit 621 can obtain the manual control signal by performing a process of multiplying the operation amount corresponding to each input operation unit 30 of the operator input device 3, that is, the operation amount corresponding to each element (such as acceleration and steering angle that define the driving operation of the vehicle 10, 10a) by a predetermined constant. Specifically, in the case where the manual control signal includes the acceleration and steering angle of the vehicle 10, 10a as parameters, the manual signal generating unit 621 multiplies the operation amount of each of the steering wheel 31, the accelerator pedal 32, and the brake pedal 33 of the operator input device 3 by a predetermined constant. Then, the manual signal generating unit 621 generates the manual control signal so that the acceleration of the vehicle 10, 10a becomes a value obtained by multiplying the operation amount of the accelerator pedal 32 and the brake pedal 33 of the operator input device 3 by the constant. The manual signal generating unit 621 generates the manual control signal so that the steering angle of the vehicle 10, 10a becomes a value obtained by multiplying the operation amount of the steering wheel 31 of the operator input device 3 by the constant. In such a form, when the upper limit value and the lower limit value of the output value of the operator input device 3 are different from the upper limit value and the lower limit value of the output value of the actuator of the drive device 11 of the vehicle 10, 10a, etc., a manual control signal corresponding to the possible output range of the actuator of the vehicle 10, 10a on the receiving side can be generated. C-12. Other Examples 12:

[0088] The manual signal generation unit 621 can provide an upper limit value of each element defining the driving operation of the vehicle 10, 10a for the operation amount corresponding to each input operation unit 30 of the operator input device 3, that is, the operation amount corresponding to each element (such as the acceleration and the steering angle that define the driving operation of the vehicle 10, 10a). For example, the manual signal generation unit 621 performs the following processing when the manual control signal includes the acceleration and the steering angle of the vehicle 10, 10a as parameters: In this case, when the operation amount of the accelerator pedal 32 of the operator input device 3 is equal to or greater than the first upper limit value, the manual signal generation unit 621 generates a manual control signal so that the acceleration of the vehicle 10, 10a becomes the first upper limit value. When the operation amount of the steering wheel 31 of the operator input device 3 is equal to or greater than the second upper limit value, the manual signal generation unit 621 generates a manual control signal so that the steering angle of the vehicle 10, 10a becomes the second upper limit value. In this manner, by providing the upper limit value for each element that defines the running operation of the vehicle 10, 10a, a rapid change in the running operation of the vehicle 10, 10a, such as a rapid acceleration or a rapid turn of the vehicle 10, 10a, can be avoided. C-13. Other Examples 13:

[0089] In each of the above-mentioned embodiments, the vehicle 10, 10a only needs to have a configuration capable of moving via unmanned driving, and, for example, can have the form of a platform having the configuration described below. Specifically, in order to exhibit the three functions of "driving", "turning" and "stopping" via unmanned driving, the vehicle 10, 10a only needs to include at least a vehicle control device 15, 15a, a drive device 11, a steering device 12 and a braking device 13. In the case where the vehicle 10, 10a obtains information from the outside for unmanned driving, the vehicle 10, 10a only needs to include a communication device 14. That is, the vehicle 10, 10a capable of moving via unmanned driving does not need to be equipped with at least a part of the internal components such as a driver's seat and an instrument panel, nor does it need to be equipped with at least a part of the external components such as a bumper and a rearview mirror, and it does not need to be equipped with a body shell. In this case, the remaining components (such as the body shell) can be assembled into the vehicle 10, 10a when the vehicle 10, 10a is shipped from the factory, or the remaining components (such as the body shell) can be assembled into the vehicle 10, 10a after the vehicle 10, 10a is shipped from the factory without being assembled therein. The respective components can be assembled from any direction of the vehicle 10, 10a, such as the upper side, the lower side, the front side, the rear side, the right side, or the left side, and can also be assembled from the same direction or different directions. As for the form of the platform, the position can be determined in the same manner as the vehicle 10, 10a in each of the above-mentioned embodiments. C-14. Other Examples 14:

[0090] The vehicle 10, 10a can be manufactured by combining a plurality of modules. A module refers to a unit composed of a plurality of components assembled depending on the part or function of the vehicle 10, 10a. For example, the platform of the vehicle 10, 10a can be manufactured by combining a front module constituting the front of the platform, a center module constituting the center of the platform, and a rear module constituting the rear of the platform. In addition, the number of modules constituting the platform is not limited to 3, and can be less than 2 or more than 4. In addition to or instead of the components constituting the platform, the components constituting the parts different from the platform in the vehicle 10, 10a can be modularized. In addition, the various modules can include any external components (such as a bumper or grille) or any internal components (such as a seat or a console). In addition, in addition to the vehicle 10, 10a, any aspect of the mobile body can also be manufactured by combining the modules. Such a module can be manufactured, for example, by joining the components via welding or fasteners, or can be manufactured by molding at least a portion of the components constituting the module into one component in one piece by casting. The molding method of integrally molding a component, especially a relatively large component, is also called giga casting or mega casting. For example, the front module, the center module and the rear module can be manufactured by using giga casting. C-15. Other Examples 15:

[0091] The transportation of the vehicle 10, 10a by using the autonomous driving to make the vehicle 10, 10a travel is also referred to as "autonomous transportation". The configuration for realizing the autonomous transportation is also referred to as "vehicle remote-controlled autonomous driving transportation system". The production method of producing the vehicle 10, 10a by using the autonomous transportation is also referred to as "autonomous production". In the autonomous production, for example, at least a part of the transportation of the vehicle 10, 10a is realized by autonomous transportation in the factory where the vehicle 10, 10a is manufactured.

[0092] The present disclosure is not limited to the above-described embodiments, and can be implemented in various configurations without departing from the spirit of the present disclosure. For example, the technical features in the embodiments corresponding to the technical features of each form described in the Summary of the Invention section may be appropriately replaced or combined to solve some or all of the above-described purposes, or to achieve some or all of the above-described effects. Where technical features are not described as necessary in this specification, these features may be deleted as appropriate.

Claims

1. A display control device configured to control a display device for moving a mobile body configured to move via unmanned driving, characterized in that: The mobile body is configured to receive a travel control signal defining a travel operation of the mobile body, and travel in a remote manual driving mode by using the received travel control signal; during a time period during which control of the remote manual driving mode is executed, the travel control signal is generated in response to an operation of an operator input device by an operator using an image acquired by a camera and displayed on a display screen of the display device provided at a location different from the moving body; The camera is at least one of an onboard camera mounted in the mobile body or an external camera configured to photograph the mobile body from the outside; The operator input device is disposed at a location different from that of the moving body; The display control device comprises: at least one of an onboard acquisition unit configured to acquire onboard information indicating whether the onboard camera is onboard the mobile body and an image quality acquisition unit configured to acquire image quality information indicating an image quality of an onboard camera image that is an image acquired by the onboard camera; as well as A display controller configured to display an image on the display screen; and The display controller is configured to display the onboard camera image on the display screen in at least one of a case where the onboard camera is mounted on the mobile body and a case where the image quality is equal to or greater than a predetermined standard, and to display an external camera image on the display screen as an image acquired by the external camera set at a location different from the mobile body in at least one of a case where the onboard camera is not mounted on the mobile body and a case where the image quality is less than the standard.

2. The display control device according to claim 1, characterized in that: The image quality is determined by at least one of a resolution of the onboard camera image, a distortion of the onboard camera image, and a detection range of the onboard camera.

3. The display control device according to claim 2, characterized in that: The display controller is configured to determine that the image quality is equal to or greater than the standard in at least any one of a first case, a second case and a third case, wherein the first case is a case where the resolution of the onboard camera image is equal to or greater than a first threshold, the second case is a case where the distortion of the onboard camera image is less than a predetermined second threshold; and the third case is a case where the detection range of the onboard camera is equal to or greater than a predetermined third threshold.

4. The display control device according to claim 1, characterized in that: The mobile body is further configured to travel in the automatic driving mode by using the travel control signal generated not in response to an operation of the operator input device by the operator; and The display controller is configured to display at least one of the onboard camera image and the external camera image on the display screen based on the onboard information and the image quality information in at least any one of a case where the autonomous driving mode is switched to the remote manual driving mode and a case where a switch from the autonomous driving mode to the remote manual driving mode is predicted.

5. The display control device according to claim 4, characterized in that: The display controller is configured to display at least one of the onboard camera image and the external camera image on the display screen based on the onboard information and the image quality information even during a time period during which control of the autonomous driving mode is performed.

6. The display control device according to claim 1, characterized in that: The display controller is configured to display the external camera image on the display screen in addition to displaying the onboard camera image when the display controller displays the onboard camera image on the display screen.

7. A display device for moving a moving body configured to move via unmanned driving, characterized in that: The mobile body is configured to receive a travel control signal defining a travel operation of the mobile body, and travel in a remote manual driving mode by using the received travel control signal; The display device includes a display screen that is provided at a location different from the moving body and on which an image acquired by a camera is displayed; During a period during which control of the remote manual driving mode is executed, the travel control signal is generated in response to an operation of an operator input device by an operator using an image displayed on the display screen; The camera is at least one of an onboard camera mounted in the mobile body or an external camera configured to photograph the mobile body from the outside; and The operator input device is disposed at a location different from that of the moving body; In at least one of a case where the onboard camera is mounted on the mobile body and a case where the image quality of the onboard camera image, which is an image acquired by the onboard camera, is equal to or greater than a predetermined standard, the onboard camera image is displayed on the display screen; and in at least one of a case where the onboard camera is not mounted on the mobile body and a case where the image quality is less than the standard, an external camera image, which is an image acquired by the external camera set at a location different from the mobile body, is displayed on the display screen.

Images