Remote Control Device

Abstract

To provide an appropriate image to a user who drives a target vehicle remotely.SOLUTION: A remote driving apparatus (10) drives a target vehicle (20) remotely. The remote driving apparatus includes: image output means (112) which provides a plurality of images captured by a plurality of cameras (21L, 21C, 21R) mounted on the target vehicle to a user who drives the target vehicle remotely; and speed detection means (11) which detects speed of the target vehicle. The image output means reduces, when the speed of the target vehicle is high, the amount of overlap of multiple imaging ranges corresponding to the images, respectively, compared with the case where the speed of the target vehicle is low.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to the technical field of a remote driving device that remotely drives a target vehicle. [Background technology]

[0002] As a technology used in this type of device, for example, a technology has been proposed in which the angle of view of the vehicle-mounted camera is increased when the vehicle speed is relatively fast and the angle of view of the vehicle-mounted camera is reduced when the vehicle speed is relatively slow (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2006-264574 A Summary of the Invention [Problem to be solved by the invention]

[0004] The technology described in Patent Document 1 has room for improvement.

[0005] The present invention has been made in consideration of the above circumstances, and has an objective of providing a remote driving device that can provide appropriate images to a user remotely driving a target vehicle. [Means for solving the problem]

[0006] A remote driving device according to one aspect of the present invention is a remote driving device for remotely driving a target vehicle, comprising: an image output means for providing a plurality of images captured by a plurality of cameras mounted on the target vehicle to a user who remotely drives the target vehicle; and a speed detection means for detecting the speed of the target vehicle. the plurality of images include a first image including a first range ahead in a traveling direction of the target vehicle, a second image including a second range to the right of the traveling direction of the target vehicle and adjacent to the first range, and a third image including a third range to the left of the traveling direction of the target vehicle and adjacent to the first range, When the speed of the target vehicle is high, the image output means outputs the following compared to when the speed of the target vehicle is low: the amount of overlap between the first range and the second range, and the amount of overlap between the first range and the third range; The aim is to reduce the [Brief description of the drawings]

[0007] [Figure 1] 1 is a block diagram showing a configuration of a remote driving system according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of an imaging range of a camera of a vehicle according to an embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an example of a display according to an embodiment. [Figure 4] FIG. 11 is a diagram showing another example of the imaging range of the camera of the vehicle according to the embodiment. [Diagram 5] 13 is an example of a map that defines the relationship between speed and overlap amount. [Figure 6] 4 is a flowchart showing the operation of the remote driving device according to the embodiment. [Figure 7] FIG. 4 is a diagram illustrating an example of image processing according to the embodiment. [Figure 8] FIG. 13 is a diagram illustrating an example of a method for displaying an image. [Figure 9] FIG. 1 is a block diagram showing a configuration of a computer according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] An embodiment of a remote driving device will be described with reference to Fig. 1 to Fig. 8. A remote driving device 10 according to the embodiment constitutes a part of a remote driving system 1. In Fig. 1, the remote driving system 1 includes the remote driving device 10 and a vehicle 20 remotely driven by the remote driving device 10.

[0009] The remote driving device 10 is modeled after, for example, a vehicle cockpit. The remote driving device 10 is provided with a steering wheel, an accelerator pedal, and a brake pedal (none of which are shown) that are operated by a user who uses the remote driving device 10 to remotely drive a target vehicle (here, vehicle 20).

[0010] The remote driving device 10 is configured with a control device 11, a communication unit 12, a steering sensor 13, an accelerator pedal sensor 14, a brake pedal sensor 15, a display 16, and a speaker 17. The steering sensor 13 detects the amount of operation of the steering wheel. The accelerator pedal sensor 14 detects the amount of operation of the accelerator pedal. The brake pedal sensor 15 detects the amount of operation of the brake pedal.

[0011] The vehicle 20 is configured to include cameras 21L, 21C, and 21R, a microphone 22 (hereinafter, referred to as “mic 22”), a speed sensor 23, an acceleration sensor 24, a yaw rate sensor 25, and a communication unit .

[0012] The microphone 22 detects sounds around the vehicle 20 (hereinafter referred to as "external environmental sounds"). The speed sensor 23 detects the speed of the vehicle 20. The acceleration sensor 24 detects the acceleration of the vehicle 20. The yaw rate sensor 25 detects the yaw rate of the vehicle 20. The external environmental sounds detected by the microphone 22, the speed detected by the speed sensor 23, the acceleration detected by the acceleration sensor 24, and the yaw rate detected by the yaw rate sensor 25 are transmitted to the remote driving device 10 via the communication unit 26.

[0013] Camera 21C is attached to vehicle 20 so as to be able to capture an image of the area in front of vehicle 20. Camera 21L is attached to vehicle 20 so as to be able to capture an image of the left side area of ​​vehicle 20. Camera 21R is attached to vehicle 20 so as to be able to capture an image of the right side area of ​​vehicle 20. Images captured by cameras 21L, 21C, and 21R are transmitted to remote driving device 10 via communication unit 26.

[0014] 2, the range IRL1 corresponds to an example of the imaging range of the camera 21L, the range IRC1 corresponds to an example of the imaging range of the camera 21C, and the range IRR1 corresponds to an example of the imaging range of the camera 21R. In addition to the cameras 21L, 21C, and 21R, the vehicle 20 may be equipped with other cameras, such as a camera capable of imaging an area behind the vehicle 20.

[0015] The control device 11 of the remote driving device 10 has a vehicle control unit 111, a display control unit 112, and a sound control unit 113 as logic blocks logically realized therein or as processing circuits physically realized therein.

[0016] The vehicle control unit 111 acquires the speed, acceleration, and yaw rate transmitted from the vehicle 20 via the communication unit 12. The vehicle control unit 111 calculates a command value for controlling the vehicle 20 based on the acquired speed, acceleration, and yaw rate, and the operation amount detected by the steering sensor 13, the operation amount detected by the accelerator pedal sensor 14, and the operation amount detected by the brake pedal sensor 15. The vehicle control unit 111 transmits the calculated command value to the vehicle 20 via the communication unit 12.

[0017] The display control unit 112 acquires images captured by the cameras 21L, 21C, and 21R via the communication unit 12. Here, the display 16 has display areas DRL, DRC, and DRR, for example, as shown in Fig. 3. The display control unit 112 displays the image captured by the camera 21L in the display area DRL, displays the image captured by the camera 21C in the display area DRC, and displays the image captured by the camera 21R in the display area DRR.

[0018] Each of the display regions DRL, DRC, and DRR may be realized as a partial region in one display. Each of the display regions DRL, DRC, and DRR may be realized by a plurality of (e.g., three) displays. Furthermore, the display control unit 112 may combine the images captured by the cameras 21L, 21C, and 21R, and then display the combined image on the display 16, instead of individually displaying the images captured by the cameras 21L, 21C, and 21R.

[0019] The sound control unit 113 acquires external environmental sounds transmitted from the vehicle 20 via the communication unit 12. The sound control unit 113 controls the speaker 17 so as to emit the acquired external environmental sounds.

[0020] Incidentally, when the imaging ranges of the cameras 21L, 21C, and 21R of the vehicle 20 are the ranges IRL1, IRC1, and IRR1 shown in Fig. 2, respectively, the following phenomenon may occur: Since the ranges IRL1 and IRC1 partially overlap, the same object may be displayed in the display areas DRL and DRC of the display 16. Similarly, since the ranges IRC1 and IRR1 partially overlap, the same object may be displayed in the display areas DRC and DRR of the display 16.

[0021] If an object is located relatively close to the vehicle 20, for example, only a part of the object is displayed in both the display areas DRL and DRC. Therefore, there is almost no possibility that a user remotely driving the vehicle 20 as the target vehicle will mistakenly recognize the object displayed in the display area DRL and the object displayed in the display area DRC as separate objects.

[0022] In contrast, an object that is relatively far from the vehicle 20 is displayed in its entirety, for example, in both the display areas DRL and DRC. Here, the area (or the gaze point) to which the user (in other words, the driver) remotely driving the vehicle 20 as the target vehicle directs his / her attention changes according to the speed of the vehicle 20.

[0023] Specifically, when the speed of the vehicle 20 is relatively low, the user often focuses his or her attention on an area around the vehicle 20 that is close to the vehicle 20. When the speed of the vehicle 20 is relatively low, even if an object located relatively far from the vehicle 20 is entirely displayed in both the display areas DRL and DRC, the effect of this on the user's operation is relatively small.

[0024] On the other hand, when the speed of the vehicle 20 is relatively high, the user often focuses his / her attention on an area around the vehicle 20 that is relatively far from the vehicle 20. When the speed of the vehicle 20 is relatively high, if an object that is relatively far from the vehicle 20 is entirely displayed in both the display areas DRL and DRC, this may have a relatively large effect on the operation of the user.

[0025] The display control unit 112 according to the present embodiment is configured to change the amount of overlap of the imaging ranges of the cameras 21L, 21C, and 21R in accordance with the speed of the vehicle 20 as the target vehicle. Specifically, when the speed of the vehicle 20 is relatively low, the display control unit 112 relatively increases the amount of overlap of the imaging ranges of the cameras 21L, 21C, and 21R, for example, as shown in Fig. 2. On the other hand, when the speed of the vehicle 20 is relatively high, the display control unit 112 relatively decreases the amount of overlap of the imaging ranges of the cameras 21L, 21C, and 21R, for example, as shown in Fig. 4.

[0026] In FIG. 4, range IRL2 corresponds to another example of the imaging range of camera 21L, range IRC2 corresponds to another example of the imaging range of camera 21C, and range IRR2 corresponds to another example of the imaging range of camera 21R.

[0027] In FIG. 4, the ranges IRL2, IRC2, and IRR2 do not overlap with each other. However, among the area in front of the vehicle 20, a part of the area near the vehicle 20 is not included in any of the ranges IRL2, IRC2, and IRR2 (i.e., a blind spot occurs). As described above, when the speed of the vehicle 20 is relatively high, the user remotely driving the vehicle 20 as the target vehicle often focuses attention on an area around the vehicle 20 that is relatively far from the vehicle 20. Therefore, even if the part of the area is not included in any of the ranges IRL2, IRC2, and IRR2, it is considered that there is little or no influence on the operation of the user.

[0028] The display control unit 112 may change the overlap amount related to the imaging ranges of the cameras 21L, 21C, and 21R according to a map that defines the relationship between the speed and the overlap amount, for example, as shown in Fig. 5. The map that defines the relationship between the speed and the overlap amount is not limited to the map shown in Fig. 5, and may be, for example, a map in which the overlap amount changes stepwise according to the change in speed. Alternatively, the display control unit 112 may obtain the overlap amount from a relational expression that defines the relationship between the speed and the overlap amount, instead of a map.

[0029] The operation of the remote driving device 10 configured as described above will be described with reference to the flowchart of Fig. 6. In Fig. 6, the control device 11 of the remote driving device 10 acquires the speed of the vehicle 20 as the target vehicle via the communication unit 12 (step S101). The display control unit 112 of the control device 11 determines the overlap amount of the imaging ranges of the cameras 21L, 21C, and 21R according to the speed acquired in the processing of step S101 (step S102). Next, the display control unit 112 performs image processing based on the overlap amount determined in the processing of step S102 (step S103).

[0030] The image processing of step S103 will be specifically described with reference to Fig. 7 and Fig. 8. In Fig. 7, the image captured by camera 21L is image 211L, the image captured by camera 21C is image 211C, and the image captured by camera 21R is image 211R. Display control unit 112 determines the angle of view for each of cameras 21L, 21C, and 21R based on the amount of overlap determined in the processing of step S102.

[0031] As shown in FIG. 7, the display control unit 112 detects a virtual viewpoint P L The horizontal angle from the camera 21L is the angle of view θ L In the same manner, the display control unit 112 extracts (i.e., trims) an image portion 212L corresponding to the virtual viewpoint P C The horizontal angle from the camera 21C is the angle of view θ C The display control unit 112 extracts an image portion 212C from the image 211C, where the virtual viewpoint P R The horizontal angle from the camera 21R is the angle of view θ R An image portion 212R in which

[0032] The display control unit 112 displays the extracted image portions 212L, 212C, and 212R on the display 16. The sizes of the extracted image portions 212L, 212C, and 212R may be smaller than the sizes of the corresponding display regions DRL, DRC, and DRR of the display 16. In this case, the display control unit 112 may display the extracted image portions 212L, 212C, and 212R in parts of the corresponding display regions DRL, DRC, and DRR, for example, as shown in FIG.

[0033] Alternatively, the display control unit 112 may horizontally enlarge the extracted image portions 212L, 212C, and 212R to fit the sizes of the corresponding display areas DRL, DRC, and DRR, and then display the enlarged image portions 212L, 212C, and 212R.

[0034] (Technical effect) The area to which the user remotely driving the vehicle 20 as the target vehicle pays attention changes according to the speed of the vehicle 20. When the speed of the vehicle 20 is relatively high, the user often pays attention to an area around the vehicle 20 that is relatively far from the vehicle 20. In this case, the display control unit 112 relatively reduces the overlapping amount of the imaging ranges of the cameras 21L, 21C, and 21R. With this configuration, it is possible to prevent the entire object located relatively far from the vehicle 20 from being displayed in both of the two areas of the display 16. As a result, it is possible to prevent the user, who is paying attention to an area relatively far from the vehicle 20 due to the relatively high speed of the vehicle 20, from erroneously recognizing the object located relatively far from the vehicle 20.

[0035] On the other hand, when the speed of the vehicle 20 is relatively low, the user often focuses his / her attention on the area around the vehicle 20 near the vehicle 20. In this case, the display control unit 112 relatively increases the overlap amount of the imaging ranges of the cameras 21L, 21C, and 21R. This configuration can reduce blind spots. As a result, when the speed of the vehicle 20 is relatively low and the user focuses his / her attention on the area near the vehicle 20, he / she can appropriately recognize objects near the vehicle 20.

[0036] In this way, the remote driving device 10 can provide an appropriate image to the user who remotely drives the target vehicle.

[0037] <Modification> Instead of or in addition to trimming the images captured by each of the cameras 21L, 21C, and 21R, the display control unit 112 may change the amount of overlap of the imaging ranges by mechanically changing the imaging range of at least one of the cameras 21L, 21C, and 21R. Here, examples of a method for mechanically changing the imaging range include a method for changing the focal length of the camera and a method for changing the orientation of the camera.

[0038] <Computer Program> An embodiment relating to a computer program will be described with reference to Fig. 9. Fig. 9 is a block diagram showing the configuration of a computer according to the embodiment.

[0039] 9, a computer 50 constitutes a part of the remote operation device. The computer 50 is configured with a CPU (Central Processing Unit) 51, a RAM 52, a HDD (Hard Disk Drive) 53, and an I / O 54. The CPU 51, the RAM 52, the HDD 53, and the I / O 54 are connected to each other by a bus 55. A computer program 531 according to this embodiment is stored in advance in the HDD 53. The I / O 54 may constitute the above-mentioned communication unit 12.

[0040] The processing of the CPU 51 by the computer program 531 will be described. The CPU 51 acquires the speed, acceleration, and yaw rate transmitted from the vehicle 20 via the I / O 54. The CPU 51 acquires images captured by the cameras 21L, 21C, and 21R via the I / O 54. The CPU 51 acquires external environmental sounds transmitted from the vehicle 20 via the I / O 54.

[0041] The CPU 51 changes the amount of overlap of the imaging ranges of the cameras 21L, 21C, and 21R in accordance with the speed of the vehicle 20. The CPU 51 may determine the amount of overlap of the imaging ranges of the cameras 21L, 21C, and 21R in accordance with a map as shown in Fig. 5, for example. The map as shown in Fig. 5 may be stored in the HDD 53.

[0042] The CPU 51 may, for example, trim the images captured by the cameras 21L, 21C, and 21R so as to achieve the determined amount of overlap. Then, the CPU 51 controls a display (not shown) via the I / O 54 so as to display the trimmed images. Note that the CPU 51 may, for example, mechanically change the imaging range of at least one of the cameras 21L, 21C, and 21R instead of or in addition to trimming the images captured by the cameras 21L, 21C, and 21R so as to achieve the determined amount of overlap.

[0043] The computer program 531 may be stored in the HDD 53 by the computer 50 reading the computer program 531 from a recording medium, such as an optical disk, such as a CD-ROM (Compact Disc Read Only Memory), or a USB (Universal Serial Bus) memory, that stores the computer program 531. Alternatively, the computer 50 may store the computer program 531 in the HDD 53 by downloading the computer program 531 via a network, such as the Internet.

[0044] According to the computer program 531, it is possible to provide an appropriate image to a user who remotely drives a target vehicle, similarly to the remote driving device 10 in the above-mentioned embodiment. According to the computer program 531, it is possible to relatively easily realize the remote driving device 10 in the above-mentioned embodiment.

[0045] Various aspects of the invention derived from the above-described embodiment and modifications will be described below.

[0046] A remote driving device according to one aspect of the invention is a remote driving device for remotely driving a target vehicle, and includes an image output means for providing a plurality of images captured by a plurality of cameras mounted on the target vehicle to a user remotely driving the target vehicle, and a speed detection means for detecting the speed of the target vehicle, and the image output means reduces the amount of overlap of a plurality of image capture ranges corresponding to the plurality of images when the speed of the target vehicle is high, compared to when the speed of the target vehicle is low. In the above-mentioned embodiment, the "display control unit 112" corresponds to an example of the "image output means", and the "control device 11" corresponds to an example of the "speed detection means".

[0047] In the remote driving device, the image output means may perform a trimming process on at least one of the multiple images so that when the speed of the target vehicle is high, the amount of overlap between the multiple imaging ranges corresponding to the multiple images is smaller than when the speed of the target vehicle is low.

[0048] In the remote driving device, the image output means may enlarge the cropped image in the horizontal direction in accordance with the horizontal size of an area in which the cropped image is displayed.

[0049] In the remote driving device, the image output means may change the imaging range of at least one of the multiple cameras so that the amount of overlap between the multiple imaging ranges corresponding to the multiple images is smaller when the speed of the target vehicle is high compared to when the speed of the target vehicle is low.

[0050] An image display method according to one embodiment of the present invention is an image display method in a remote driving device that remotely drives a target vehicle, and includes an image output process that provides a plurality of images captured by a plurality of cameras mounted on the target vehicle to a user who remotely drives the target vehicle, and a speed detection process that detects the speed of the target vehicle, wherein in the image output process, when the speed of the target vehicle is high, the amount of overlap of a plurality of imaging ranges corresponding to each of the plurality of images is made smaller than when the speed of the target vehicle is low.

[0051] In the image output process, when the speed of the target vehicle is high, a trimming process may be applied to at least one of the multiple images so that the amount of overlap between the multiple imaging ranges corresponding to the multiple images is smaller than when the speed of the target vehicle is low.

[0052] In the image outputting step, the cropped image may be enlarged in the horizontal direction in accordance with a horizontal size of an area in which the cropped image is displayed.

[0053] The image output process may change the imaging range of at least one of the multiple cameras so that the amount of overlap between the multiple imaging ranges corresponding to the multiple images is smaller when the speed of the target vehicle is high compared to when the speed of the target vehicle is low.

[0054] A computer program according to one embodiment of the present invention causes a computer of a remote driving device that remotely drives a target vehicle to function as an image output means that provides a plurality of images taken by a plurality of cameras mounted on the target vehicle to a user who remotely drives the target vehicle, and a speed detection means that detects the speed of the target vehicle, and when the speed of the target vehicle is high, the image output means reduces the amount of overlap of a plurality of imaging ranges corresponding to each of the plurality of images when the speed of the target vehicle is high compared to when the speed of the target vehicle is low.

[0055] The present invention is not limited to the above-described embodiments, but can be modified as appropriate within the scope that does not contradict the gist or concept of the invention that can be read from the claims and the entire specification, and remote operating devices with such modifications are also included in the technical scope of the present invention. [Explanation of symbols]

[0056] 1... remote driving system, 10... remote driving device, 11... control device, 12, 26... communication unit, 13... steering sensor, 14... accelerator pedal sensor, 15... brake pedal sensor, 16... display, 17... speaker, 20... vehicle, 21L, 21C, 21R... camera, 22... microphone, 23... speed sensor, 24... acceleration sensor, 25... yaw rate sensor, 111... vehicle control unit, 112... display control unit, 113... sound control unit

Claims

1. A remote driving device for remotely driving a target vehicle, an image output unit for providing a plurality of images captured by a plurality of cameras mounted on the target vehicle to a user who remotely drives the target vehicle; a speed detection means for detecting a speed of the target vehicle; Equipped with the plurality of images include a first image including a first range ahead of the target vehicle in a traveling direction, a second image including a second range to the right of the target vehicle in a traveling direction and adjacent to the first range, and a third image including a third range to the left of the target vehicle in a traveling direction and adjacent to the first range, When the speed of the target vehicle is high, the image output means reduces an overlap amount between the first range and the second range and an overlap amount between the first range and the third range, compared to when the speed of the target vehicle is low. A remote driving device characterized by:

2. The remote driving device according to claim 1, characterized in that the image output means performs a trimming process on at least one of the first image, the second image, and the third image so that the amount of overlap between the first range and the second range, and the amount of overlap between the first range and the third range, is smaller when the speed of the target vehicle is high than when the speed of the target vehicle is low.

3. 3. The remote driving device according to claim 2, wherein the image output means enlarges the cropped image in a horizontal direction in accordance with a horizontal size of an area in which the cropped image is displayed.

4. The remote driving device according to claim 1, characterized in that the image output means changes the imaging range of at least one of the multiple cameras so that the amount of overlap between the first range and the second range, and the amount of overlap between the first range and the third range, is smaller when the speed of the target vehicle is high than when the speed of the target vehicle is low.

Images