Remote Control Device

Abstract

To bring speed perceived by a user who remotely drives an object vehicle close to actual speed of the object vehicle.SOLUTION: A remote driving device (10) remotely drives an object vehicle (20). The remote driving device comprises generation means (113) for generating synthetic sound information including at least one of external environmental sound which is sound in the surroundings of the object vehicle collected by a microphone (22), and virtual travel sound artificially representing travel sound of the object vehicle. The generation means makes ratio of the external environmental sound included in the synthetic sound information larger in comparison with the case that the speed of the object vehicle is low when the speed of the object vehicle is high.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to the technical field of a remote driving device that remotely drives a target vehicle. [Background technology]

[0002] As this type of device, for example, a device has been proposed that transmits images from a plurality of cameras mounted on a vehicle to a control center and displays the images on a monitor at the control center (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 10-145777 Summary of the Invention [Problem to be solved by the invention]

[0004] A user who drives the target vehicle remotely from outside the vehicle is not in the target vehicle. For this reason, there is a technical problem that a discrepancy may occur between the speed that the user perceives from an image (video) displayed on a monitor, for example, and the actual speed of the target vehicle.

[0005] The present invention has been made in consideration of, for example, the above-mentioned problems, and has an objective of providing a remote driving device that can bring the speed perceived by a user remotely driving a target vehicle closer to the actual speed of the target vehicle. [Means for solving the problem]

[0006] A remote driving device according to one aspect of the present invention is a remote driving device for remotely driving a target vehicle, and includes an external environmental sound that is a sound around the target vehicle collected by a microphone, and a virtual driving sound that simulates the driving sound of the target vehicle. AndThe device is equipped with a generation means for generating synthetic sound information including the external environmental sound, and when the speed of the target vehicle is high, the generation means increases the proportion of the external environmental sound included in the synthetic sound information compared to when the speed of the target vehicle is low. [Brief description of the drawings]

[0007] [Figure 1] 1 is a block diagram showing a configuration of a remote operation system according to a first embodiment. [Diagram 2] 3 is a diagram showing an example of an imaging range of a camera of a vehicle according to the first embodiment; FIG. [Diagram 3] FIG. 2 is a diagram illustrating an example of a display according to the first embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of a distributor according to the first embodiment. [Diagram 5] 13 is an example of a map that defines a ratio between external environmental sounds and virtual driving sounds. [Figure 6] 4 is a flowchart showing the operation of the remote operation device according to the first embodiment. [Figure 7] 13 is another example of a map that defines the ratio between external environmental sounds and virtual driving sounds. [Figure 8] 10 is a flowchart showing the operation of the remote operation device according to the second embodiment. [Figure 9] FIG. 1 is a block diagram showing a configuration of a computer according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] First Embodiment A first embodiment of a remote driving device will be described with reference to Fig. 1 to Fig. 6. A remote driving device 10 according to the first embodiment constitutes a part of a remote driving system 1. In Fig. 1, the remote driving system 1 includes the remote driving device 10 and a vehicle 20 that is remotely driven by the remote driving device 10.

[0009] The remote driving device 10 is modeled after, for example, a vehicle cockpit. The remote driving device 10 is provided with a steering wheel, an accelerator pedal, and a brake pedal (none of which are shown) that are operated by a user who uses the remote driving device 10 to remotely drive a target vehicle (here, vehicle 20).

[0010] The remote driving device 10 is configured with a control device 11, a communication unit 12, a steering sensor 13, an accelerator pedal sensor 14, a brake pedal sensor 15, a display 16, and a speaker 17. The steering sensor 13 detects the amount of operation of the steering wheel. The accelerator pedal sensor 14 detects the amount of operation of the accelerator pedal. The brake pedal sensor 15 detects the amount of operation of the brake pedal.

[0011] The vehicle 20 is configured to include cameras 21L, 21C, and 21R, a microphone 22 (hereinafter, referred to as “mic 22”), a speed sensor 23, an acceleration sensor 24, a yaw rate sensor 25, and a communication unit .

[0012] The microphone 22 detects sounds around the vehicle 20 (hereinafter referred to as "external environmental sounds"). The speed sensor 23 detects the speed of the vehicle 20. The acceleration sensor 24 detects the acceleration of the vehicle 20. The yaw rate sensor 25 detects the yaw rate of the vehicle 20. The external environmental sounds detected by the microphone 22, the speed detected by the speed sensor 23, the acceleration detected by the acceleration sensor 24, and the yaw rate detected by the yaw rate sensor 25 are transmitted to the remote driving device 10 via the communication unit 26.

[0013] Camera 21C is attached to vehicle 20 so as to be able to capture an image of the area in front of vehicle 20. Camera 21L is attached to vehicle 20 so as to be able to capture an image of the left side area of ​​vehicle 20. Camera 21R is attached to vehicle 20 so as to be able to capture an image of the right side area of ​​vehicle 20. Images captured by cameras 21L, 21C, and 21R are transmitted to remote driving device 10 via communication unit 26.

[0014] 2, the range IRL corresponds to the imaging range of the camera 21L, the range IRC corresponds to the imaging range of the camera 21C, and the range IRR corresponds to the imaging range of the camera 21R. In addition to the cameras 21L, 21C, and 21R, the vehicle 20 may be equipped with other cameras, such as a camera capable of imaging an area behind the vehicle 20.

[0015] The control device 11 of the remote driving device 10 has a vehicle control unit 111, a display control unit 112, and a sound control unit 113 as logic blocks logically realized therein or as processing circuits physically realized therein.

[0016] The vehicle control unit 111 acquires the speed, acceleration, and yaw rate transmitted from the vehicle 20 via the communication unit 12. The vehicle control unit 111 calculates a command value for controlling the vehicle 20 based on the acquired speed, acceleration, and yaw rate, and the operation amount detected by the steering sensor 13, the operation amount detected by the accelerator pedal sensor 14, and the operation amount detected by the brake pedal sensor 15. The vehicle control unit 111 transmits the calculated command value to the vehicle 20 via the communication unit 12.

[0017] The display control unit 112 acquires images captured by the cameras 21L, 21C, and 21R via the communication unit 12. Here, the display 16 has display areas DRL, DRC, and DRR, for example, as shown in Fig. 3. The display control unit 112 displays the image captured by the camera 21L in the display area DRL, displays the image captured by the camera 21C in the display area DRC, and displays the image captured by the camera 21R in the display area DRR.

[0018] Each of the display regions DRL, DRC, and DRR may be realized as a partial region in one display. Each of the display regions DRL, DRC, and DRR may be realized by a plurality of (e.g., three) displays. Furthermore, the display control unit 112 may combine the images captured by the cameras 21L, 21C, and 21R, and then display the combined image on the display 16, instead of individually displaying the images captured by the cameras 21L, 21C, and 21R.

[0019] The sound control unit 113 acquires external environmental sounds transmitted from the vehicle 20 via the communication unit 12. The sound control unit 113 generates virtual running sounds that simulate the running sounds of the vehicle 20. The sound control unit 113 generates synthetic sound information that includes at least one of the external environmental sounds and the virtual running sounds. The sound control unit 113 controls the speaker 17 so that a sound indicated by the generated synthetic sound information is emitted.

[0020] Here, the sound control unit 113 may have a distributor 1131 (see FIG. 4). The synthetic sound information may be generated, for example, by inputting an external environmental sound and a virtual running sound to the distributor 1131 and synthesizing the two by the distributor 1131. Note that various existing aspects can be applied to the method of generating the virtual running sound, and therefore detailed explanations thereof will be omitted.

[0021] In particular, the sound control unit 113 changes the ratio of the external environmental sound and the virtual running sound included in the synthetic sound information according to the speed of the vehicle 20. The sound control unit 113 may change the ratio of the external environmental sound and the virtual running sound included in the synthetic sound information according to a map such as that shown in FIG. 5. When the speed of the vehicle 20 is relatively low, the synthetic sound information may include only the virtual running sound (i.e., the ratio of the virtual running sound may be 100%). When the speed of the vehicle 20 is relatively high, the synthetic sound information may include only the external environmental sound (i.e., the ratio of the external environmental sound may be 100%).

[0022] The operation of the remote driving device 10 configured as described above will be described with reference to the flowchart in Fig. 6. In Fig. 6, the control device 11 of the remote driving device 10 acquires the speed of the vehicle 20 as the target vehicle via the communication unit 12 (step S101). The sound control unit 113 of the control device 11 determines the ratio of the external environmental sound and the virtual driving sound included in the synthetic sound information according to the speed acquired in the processing of step S101 (step S102).

[0023] In the process of step S102, the sound control unit 113 may determine the ratio of the external environmental sound and the virtual running sound according to a map such as that shown in Fig. 5. Alternatively, when the speed of the target vehicle is lower than a threshold, the sound control unit 113 may increase the ratio of the virtual running sound included in the synthetic sound information to a greater ratio than the ratio of the external environmental sound. When the speed of the target vehicle is higher than a threshold, the sound control unit 113 may increase the ratio of the external environmental sound included in the synthetic sound information to a greater ratio than the ratio of the virtual running sound.

[0024] Thereafter, the sound control unit 113 generates synthetic sound information from the external environmental sound and the virtual running sound so that the ratio is the ratio determined in the processing of step S102. Then, the sound control unit 113 controls the speaker 17 so that the sound indicated by the generated synthetic sound information is emitted (step S103).

[0025] (Technical effect) A user who remotely drives a target vehicle (here, vehicle 20) using the remote driving device 10 cannot feel, for example, vibrations and accelerations caused by the behavior of the vehicle 20. To enhance the sense of realism of the user who is remotely driving, external environmental sounds are emitted, for example, from the speaker 17. However, when the speed of the vehicle 20 is relatively low, for example, vibration sounds and friction sounds of the tires of the vehicle 20 are relatively small. For this reason, even if external environmental sounds are emitted from the speaker 17, the effect of improving the sense of realism is not as great as expected.

[0026] Therefore, in the remote driving device 10, when the speed of the vehicle 20 is relatively low, synthetic sound information is generated in which the proportion of virtual running sounds is higher than when the speed of the vehicle 20 is relatively high. Specifically, when the speed of the vehicle 20 is relatively low, synthetic sound information is generated in which the proportion of virtual running sounds is higher than that of external environmental sounds. On the other hand, when the speed of the vehicle 20 is relatively high, synthetic sound information is generated in which the proportion of external environmental sounds is higher than that of virtual running sounds. This is because, for example, vibration sounds and friction sounds of the tires of the vehicle 20 are sufficiently large when the speed of the vehicle 20 is relatively high.

[0027] According to the remote driving device 10, the speed perceived by the user remotely driving the target vehicle can be made closer to the actual speed of the target vehicle.

[0028] Among the functions of the sound control unit 113, the function of generating synthetic sound information from external environmental sounds and virtual driving sounds may be provided in the target vehicle (for example, the vehicle 20) to be remotely driven.

[0029] <Second embodiment> A second embodiment of the remote operation device will be described with reference to Fig. 7 and Fig. 8. The second embodiment is similar to the above-mentioned first embodiment except for a part of the operation of the sound control unit 113. Therefore, the second embodiment will be described mainly with respect to the differences from the first embodiment, and the description overlapping with the first embodiment will be omitted as appropriate.

[0030] The sound control unit 113 (see FIG. 1) acquires external environmental sound transmitted from the vehicle 20 (i.e., the target vehicle of remote driving) via the communication unit 12. The sound control unit 113 determines whether or not the acquired external environmental sound includes a siren sound emitted from an emergency vehicle. For example, the sound control unit 113 may perform a predetermined frequency analysis on the external environmental sound. Then, when the result of the frequency analysis includes a frequency specific to a siren sound emitted from an emergency vehicle, the sound control unit 113 may determine that the external environmental sound includes a siren sound emitted from an emergency vehicle.

[0031] When the external environmental sound includes a siren sound emitted from an emergency vehicle, the sound control unit 113 increases the ratio of the external environmental sound included in the synthetic sound information to the ratio of the virtual running sound, regardless of the speed of the vehicle 20. Specifically, the sound control unit 113 may determine the ratio of the external environmental sound and the virtual running sound according to a map such as that shown in FIG.

[0032] As a result, a sound represented by the synthesized sound information containing a relatively large amount of external environmental sounds is emitted from the speaker 17. With this configuration, it is possible to make a user who is remotely driving the target vehicle 20 using the remote driving device 10 aware of the presence of an emergency vehicle around the vehicle 20. If the user is aware of the presence of the emergency vehicle, he or she can, for example, move the vehicle 20 to the shoulder of the road.

[0033] If the external environmental sound does not include a siren sound emitted from an emergency vehicle, the sound control unit 113 may perform the same operation as in the first embodiment described above. That is, the sound control unit 113 may determine the ratio of the external environmental sound and the virtual driving sound according to a map such as that shown in FIG.

[0034] The operation of sound control unit 113 according to this embodiment will be described with reference to the flowchart in Fig. 8. Note that steps S101, S102, and S103 in Fig. 8 correspond to steps S101, S102, and S103 in Fig. 6.

[0035] 8, after the process of step S101, the sound control unit 113 determines whether or not an emergency vehicle is present around the vehicle 20 as the target vehicle of remote driving (step S201). Specifically, the sound control unit 113 determines whether or not the external environmental sound transmitted from the vehicle 20 includes a siren sound emitted from the emergency vehicle.

[0036] If it is determined in the processing of step S201 that an emergency vehicle is present around vehicle 20 (step S201: Yes), the sound control unit 113 determines the proportion of external environmental sounds and virtual driving sounds included in the synthetic sound information according to the speed obtained in the processing of step S101 so that the proportion of external environmental sounds included in the synthetic sound information is greater than the proportion of virtual driving sounds (step S202).

[0037] Thereafter, the sound control unit 113 generates synthetic sound information from the external environmental sound and the virtual running sound so that the ratio is the ratio determined in the processing of step S202. Then, the sound control unit 113 controls the speaker 17 so that the sound indicated by the generated synthetic sound information is emitted (step S103).

[0038] In the process of step S201, if it is determined that there is no emergency vehicle around the vehicle 20 (step S201: No), the sound control unit 113 performs the process of step S102.

[0039] (Technical effect) According to the remote driving device 10, the speed perceived by the user remotely driving the target vehicle can be made to approach the actual speed of the target vehicle, while allowing the user to perceive the presence of an emergency vehicle.

[0040] <Computer Program> An embodiment relating to a computer program will be described with reference to Fig. 9. Fig. 9 is a block diagram showing the configuration of a computer according to the embodiment.

[0041] 9, a computer 50 constitutes a part of the remote operation device. The computer 50 is configured with a CPU (Central Processing Unit) 51, a RAM 52, a HDD (Hard Disk Drive) 53, and an I / O 54. The CPU 51, the RAM 52, the HDD 53, and the I / O 54 are connected to each other by a bus 55. A computer program 531 according to this embodiment is stored in advance in the HDD 53. The I / O 54 may constitute the above-mentioned communication unit 12.

[0042] The processing of the CPU 51 by the computer program 531 will be described. The CPU 51 acquires the speed, acceleration, and yaw rate transmitted from the vehicle 20 via the I / O 54. The CPU 51 acquires images captured by the cameras 21L, 21C, and 21R via the I / O 54. The CPU 51 acquires external environmental sounds transmitted from the vehicle 20 via the I / O 54.

[0043] The CPU 51 generates a virtual running sound that simulates the running sound of the vehicle 20. The CPU 51 generates synthetic sound information that includes at least one of an external environmental sound and a virtual running sound. The CPU 51 controls a speaker (not shown) via the I / O 54 so that a sound indicated by the generated synthetic sound information is emitted.

[0044] Here, the CPU 51 changes the ratio of the external environmental sound and the virtual running sound included in the synthetic sound information according to the speed of the vehicle 20. The CPU 51 may change the ratio of the external environmental sound and the virtual running sound included in the synthetic sound information according to a map such as that shown in Fig. 5. The map such as that shown in Fig. 5 may be stored in the HDD 53.

[0045] The CPU 51 may determine whether or not the external environmental sound includes a siren sound emitted from an emergency vehicle (i.e., whether or not an emergency vehicle is present around the vehicle 20). If the external environmental sound includes a siren sound emitted from an emergency vehicle, the CPU 51 may increase the proportion of the external environmental sound included in the synthetic sound information to be greater than the proportion of the virtual running sound, regardless of the speed of the vehicle 20. In this case, the CPU 51 may determine the proportion of the external environmental sound and the virtual running sound according to a map such as that shown in FIG. 7, for example. The map such as that shown in FIG. 7 may be stored in the HDD 53.

[0046] The computer program 531 may be stored in the HDD 53 by the computer 50 reading the computer program 531 from a recording medium, such as an optical disk, such as a CD-ROM (Compact Disc Read Only Memory), or a USB (Universal Serial Bus) memory, that stores the computer program 531. Alternatively, the computer 50 may store the computer program 531 in the HDD 53 by downloading the computer program 531 via a network, such as the Internet.

[0047] According to the computer program 531, like the remote driving device 10 in the above-mentioned first embodiment, the speed perceived by the user remotely driving the target vehicle can be made to approach the actual speed of the target vehicle. In addition, according to the computer program 531, like the remote driving device 10 in the above-mentioned second embodiment, the presence of an emergency vehicle can be made to be perceived by the user remotely driving the target vehicle. According to the computer program 531, the remote driving device 10 in the above-mentioned first and second embodiments can be realized relatively easily.

[0048] Various aspects of the invention derived from the above-described embodiment will be described below.

[0049] A remote driving device according to one aspect of the invention is a remote driving device that remotely drives a target vehicle, and includes a generating means for generating synthetic sound information including at least one of an external environmental sound, which is a sound around the target vehicle collected by a microphone, and a virtual running sound that simulates the running sound of the target vehicle, and the generating means increases the proportion of the external environmental sound included in the synthetic sound information when the speed of the target vehicle is high compared to when the speed of the target vehicle is low. In the above-mentioned embodiment, the "sound control unit 113" corresponds to an example of the "generating means".

[0050] The remote driving device may include a detection means for detecting an emergency vehicle present around the target vehicle, and when the detection means detects an emergency vehicle, the generation means may increase the proportion of the external environmental sound contained in the synthetic sound information to be greater than the proportion of the virtual driving sound contained in the synthetic sound information. In the above-mentioned embodiment, the "sound control unit 113" corresponds to an example of the "detection means".

[0051] A remote driving method according to one embodiment of the invention is a remote driving method in a remote driving device that remotely drives a target vehicle, and includes a generation step of generating synthetic sound information including at least one of external environmental sound, which is sound around the target vehicle collected by a microphone, and virtual driving sound that simulates the driving sound of the target vehicle, and in the generation step, when the speed of the target vehicle is high, the proportion of the external environmental sound included in the synthetic sound information is increased compared to when the speed of the target vehicle is low.

[0052] The remote driving method may include a detection step of detecting an emergency vehicle present around the target vehicle prior to the generation step, and in the generation step, if an emergency vehicle is detected by the detection step, the proportion of the external environmental sound contained in the synthetic sound information may be made larger than the proportion of the virtual driving sound contained in the synthetic sound information.

[0053] A computer program according to one embodiment of the invention causes a computer of a remote driving device that remotely drives a target vehicle to function as a generation means that generates synthetic sound information including at least one of external environmental sounds, which are sounds around the target vehicle collected by a microphone, and virtual driving sounds that simulate the driving sounds of the target vehicle, and that increases the proportion of the external environmental sounds included in the synthetic sound information when the speed of the target vehicle is high compared to when the speed of the target vehicle is low.

[0054] The computer program may further cause the computer to function as a detection means for detecting emergency vehicles present around the target vehicle, and when an emergency vehicle is detected by the detection means, the generation means may increase the proportion of the external environmental sound contained in the synthetic sound information to be greater than the proportion of the virtual running sound contained in the synthetic sound information.

[0055] The present invention is not limited to the above-described embodiments, but can be modified as appropriate within the scope that does not contradict the gist or concept of the invention that can be read from the claims and the entire specification, and remote operating devices with such modifications are also included in the technical scope of the present invention. [Explanation of symbols]

[0056] 1... remote driving system, 10... remote driving device, 11... control device, 12, 26... communication unit, 13... steering sensor, 14... accelerator pedal sensor, 15... brake pedal sensor, 16... display, 17... speaker, 20... vehicle, 21L, 21C, 21R... camera, 22... microphone, 23... speed sensor, 24... acceleration sensor, 25... yaw rate sensor, 111... vehicle control unit, 112... display control unit, 113... sound control unit

Claims

1. A remote driving device for remotely driving a target vehicle, A generating unit generates synthetic sound information including an external environmental sound, which is a sound around the target vehicle collected by a microphone, and a virtual running sound, which simulates a running sound of the target vehicle, The generating means increases a ratio of the external environmental sound included in the synthetic sound information when the speed of the target vehicle is high, compared to when the speed of the target vehicle is low. A remote driving device characterized by:

2. a detection means for detecting an emergency vehicle existing around the target vehicle; When an emergency vehicle is detected by the detection means, the generation means increases a ratio of the external environmental sound included in the synthetic sound information to a ratio of the virtual running sound included in the synthetic sound information.

2. The remote driving device according to claim 1.

Images