Remote Operation System

Abstract

To obtain a remote driving system that in remotely operating and driving a vehicle from a cockpit outside the vehicle, can inhibit sound around the vehicle from being delayed and transmitted to the cockpit.SOLUTION: A remote driving system 10 comprises: a vehicle 12 including a sound collection unit 14 for collecting sound around the vehicle, a control unit 16 for converting the sound collected by the sound collection unit 14 to predetermined flag information, and a transmission unit 18 for transmitting the flag information; and a cockpit 20 that includes a reception unit 22 for receiving the flag information and a notification unit 24 for performing notification corresponding to a content of the received flag information and makes it possible to drive the vehicle 12 by remote operation from the outside of the vehicle.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a remote driving system. [Background technology]

[0002] A remote image monitoring device that transmits a main image for live monitoring and a detailed sub-image that supplements the main image, as well as audio, sensor data, and the like as auxiliary information, has been known for some time (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-118019 A Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, there has been a demand for remote driving technology that allows a vehicle to be remotely operated from a cockpit installed outside the vehicle. In such remote driving, it is necessary to transmit the raw sounds generated around the vehicle to the cockpit. However, the amount of information (sound data) from the raw sounds actually generated is large, and there is a possibility that communication delays will occur when transmitting the sound data to the cockpit.

[0005] In other words, when a vehicle is driven by remote control, the sound around the vehicle that is transmitted to the cockpit may be delayed (sound out of sync) from the timing of the sound that is actually generated. Therefore, in the above-mentioned remote image monitoring device, the sound data is compressed before being transmitted, but this is not sufficient to prevent the occurrence of sound out of sync, and there is room for improvement in terms of preventing the occurrence of sound out of sync.

[0006] Therefore, an object of the present invention is to provide a remote driving system that can prevent sounds around the vehicle from being displaced and transmitted to the cockpit when the vehicle is remotely operated from a cockpit outside the vehicle. [Means for solving the problem]

[0007] In order to achieve the above object, a remote driving system according to a first aspect of the present invention includes a vehicle having a sound collection unit that collects sounds around the vehicle, a control unit that converts the sounds collected by the sound collection unit into predetermined flag information, and a transmission unit that transmits the flag information, and a cockpit that has a receiving unit that receives the flag information and a notification unit that issues a notification corresponding to the content of the received flag information, and that can remotely drive the vehicle from outside the vehicle. The notification unit stores various sampled sound data and vibration data, and the notification unit selects a sound and vibration corresponding to the content from the sound data and vibration data and notifies the pilot in the cockpit. do.

[0008] According to the invention described in claim 1, when a sound around the vehicle is picked up by the sound pickup unit of the vehicle, the sound is converted into predetermined flag information by the control unit of the vehicle, and the flag information is transmitted by the transmission unit of the vehicle. The flag information transmitted from the transmission unit of the vehicle is then received by the receiving unit of the cockpit. In other words, the information communicated between the vehicle and the cockpit is flag information, and is not information (sound data) of the actual sound picked up by the sound pickup unit.

[0009] Here, the amount of flag information is smaller than that of sound data, and communication delays are less likely to occur. Also, In the cockpit notification area teeth, Various pre-sampled sound data and vibration data record Therefore, The notification unit in the cockpit notifies the pilot in accordance with the contents of the flag information received by the receiving unit in the cockpit. In other words, the pilot in the cockpit receives a sampling sound corresponding to the flag information. and vibration This prevents the sound around the vehicle from being transmitted to the driver's seat with a delay when the vehicle is being driven remotely from the driver's seat outside the vehicle. Effect of the Invention

[0010] As described above, according to the present invention, when a vehicle is driven by remotely operating the vehicle from a cockpit outside the vehicle, sounds around the vehicle can be prevented from being displaced and transmitted to the cockpit. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a configuration of a remote driving system according to an embodiment of the present invention; [Diagram 2] 4 is a flowchart showing a vehicle-side processing process of the remote driving system according to the embodiment. [Diagram 3] 4 is a flowchart showing the processing steps on the remote cockpit side of the remote driving system according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. As shown in Fig. 1, a remote driving system 10 according to this embodiment includes a vehicle 12 that actually runs on a roadway, and a remote cockpit 20 as a cockpit for a driver (operator) who remotely controls and drives the vehicle 12 from outside the vehicle.

[0013] The vehicle 12 has a microphone 14 as a sound collection unit that collects sounds around the vehicle, a control unit 16 that converts sound information (sound data) collected by the microphone 14 into predetermined flag information, and a communication unit 18 equipped with a transmission unit that transmits the flag information to a remote cockpit 20 via an antenna 19. The communication unit 18 also has a receiving unit that receives information transmitted from the remote cockpit 20 (e.g., information for driving the vehicle 12).

[0014] The microphone 14 is desirably configured to be able to recognize the source of sound, such as the front, rear, left and right of the vehicle 12, and there are no restrictions on the number of microphones and the locations where they are installed in the vehicle 12. The control unit 16 provided in the vehicle 12 is configured to filter the collected sound and analyze and identify it using AI or the like.

[0015] The remote cockpit 20 has a communication unit 22 equipped with a receiving unit that receives flag information from the communication unit 18 via an antenna 21. The communication unit 22 also has a transmitting unit that transmits information (e.g., information for driving the vehicle 12) to the vehicle 12. The remote cockpit 20 also has a control unit 24 as a notification unit that issues a predetermined notification corresponding to the content of the received flag information.

[0016] Various sound data, vibration data, and the like sampled when the vehicle 12 actually travels on the roadway are pre-recorded in the control unit 24. The control unit 24 then analyzes the flag information received by the communication unit 22, selects an appropriate sound or vibration according to the state (driving situation), and notifies the driver of the selected sound or vibration.

[0017] That is, the control unit 24 is electrically connected to a speaker 26 in the remote cockpit 20 as a notification means for notifying the driver who is actually remotely operating the vehicle 12 by sound (generating a sound), and a vibration device 28 as a notification means for notifying the driver by vibration (generating a vibration) depending on the state obtained from the flag information.

[0018] The speakers 26 are configured to notify the driver's hearing, and are preferably configured to reproduce the direction of sound, such as front, rear, left, right, etc., and there are no restrictions on the number of speakers and the installation locations in the remote cockpit 20. Therefore, for example, at least a portion of the speakers 26 may be configured as stereophonic speakers.

[0019] The vibration device 28 is provided on a seat or steering wheel (not shown) in the remote cockpit 20, and is configured to notify the driver's tactile sense by vibrating the seat or steering wheel. Note that the vibration device 28 is preferably configured to be capable of changing the location to be vibrated and the strength of the vibration.

[0020] The control unit 24 is also electrically connected to a display unit 30. The display unit 30 is a display device associated with the remote cockpit 20, and the display device is provided with a display that shows camera images of the surroundings of the vehicle 12 to be remotely operated, various meters of the vehicle 12, and the like. In other words, the vehicle 12 is provided with a camera, and is configured so that the image captured by the camera can be visually notified to the driver by a notification means such as a display.

[0021] Next, the operation of the remote operation system 10 according to this embodiment configured as above will be described with reference to FIGS.

[0022] When driving the vehicle 12 by remote control from the remote cockpit 20, it is necessary to grasp sounds generated around the vehicle 12, such as the approach of an emergency vehicle or a railroad crossing. For this reason, as shown in Fig. 2, first, on the vehicle 12 side, the microphone 14 provided on the vehicle 12 picks up sounds generated around the vehicle 12 that is actually traveling (step S11).

[0023] When the microphone 14 picks up sounds generated around the vehicle 12, the control unit 16 filters the picked up sounds to eliminate unnecessary noise (step S12), and analyzes and identifies the sounds using AI or the like (step S13). For example, sounds generated by emergency vehicles, preceding vehicles, following vehicles, oncoming vehicles, railroad crossings, pedestrians, bicycles, construction works, etc. are analyzed and identified.

[0024] Then, the control unit 16 judges whether or not the sound needs to be transmitted to the remote cockpit 20 based on the analysis and identification results of the sound (step S14). If the sound does not need to be transmitted, the process returns to step S11, and if the sound needs to be transmitted, the analysis and identification results are converted into predetermined flag information.

[0025] The predetermined flag information includes the types of the above-mentioned various vehicles, pedestrians, bicycles, etc., their directions, estimated relative distances, and states such as stopped or approaching. Then, the communication unit 18 transmits the flag information to the remote cockpit 20 via the antenna 19 (step S15). The above steps are repeated on the vehicle 12 side.

[0026] 3, in the remote cockpit 20, the communication unit 22 waits for the reception of flag information via the antenna 21 (step S21). That is, the control unit 24 determines whether or not the flag information has been received (step S22), and if the flag information has not been received, the process returns to step S21.

[0027] On the other hand, when flag information is received, the control unit 24 analyzes the received flag information (step S23) and judges whether or not to notify the driver (step S24). If it is not necessary to notify the driver, the process returns to step S21. If it is necessary to notify the driver, the process selects the most appropriate notification means according to the driving situation (step S25), and notifies the driver by the notification means (step S26). The above steps are repeated on the remote cockpit 20 side.

[0028] The notification means includes, first, a display device (such as a display) of the display unit 30 that visually notifies the driver. The display unit 30 can notify the driver of the situation by, for example, displaying camera images of emergency vehicles, preceding vehicles, following vehicles, oncoming vehicles, railroad crossings, pedestrians, bicycles, construction works, etc. around the vehicle 12 on the display, and displaying their identification marks, estimated relative distances to them, and states such as stopped or approaching, etc., together with the indications of various meters.

[0029] The notification means includes the speaker 26 that notifies the driver's hearing and the vibration device 28 that notifies the driver's tactile sense. That is, the notification means includes the speaker 26 that generates a sound toward the driver and the vibration device 28 that vibrates at least one of the seat on which the driver sits and the steering wheel held by the driver according to the state obtained from the flag information.

[0030] Here, various sound data, vibration data, etc. sampled in advance are recorded in the control unit 24. Therefore, the sound to be generated from the speaker 26 includes real sounds sampled in advance, voice guidance, alert sounds, etc., and is selected from among them. In other words, the driver (pilot) in the remote cockpit 20 is notified of sampled sounds, etc. corresponding to flag information.

[0031] If at least a part of the speaker 26 is a stereophonic speaker, for example, a realistic sound (siren) of an emergency vehicle can be generated (reproduced) together with the direction. Even if the speaker 26 does not include a stereophonic speaker, for example, the approach of an emergency vehicle can be communicated together with the direction by voice alone.

[0032] As described above, it is desirable that vibration device 28 be capable of changing the location of vibration and the strength of vibration depending on the state obtained from the flag information. For example, when another vehicle is approaching from the right side of the vehicle, the right side of the seat or the right side of the steering wheel can be vibrated, and the vibration can be gradually increased as the other vehicle approaches, thereby notifying the driver of the situation.

[0033] In this way, according to the remote driving system 10 of this embodiment, the driver of the remote cockpit 20 is given an optimal notification according to the contents of the flag information received by the communication unit 22. Moreover, the amount of data to be communicated for flag information is smaller than that of sound data, and communication delays are less likely to occur. Therefore, when the vehicle 12 is remotely operated and driven from the remote cockpit 20 outside the vehicle, it is possible to suppress the sound around the vehicle from being transmitted to the remote cockpit 20 with a delay (sound lag).

[0034] More specifically, if raw sound data is always transmitted from the vehicle 12 to the remote cockpit 20, the amount of communication data will increase, which may cause sound delays and affect communication costs. In particular, if the direction of sound generation is to be realistically reproduced using raw sound data, the number of audio channels will increase, which will increase the amount of sound data to be transmitted.

[0035] In addition, the remote cockpit 20 side, which generates raw sound based on the received sound data, also requires more environmental equipment such as multiple speakers, etc. Furthermore, when the traveling speed of the vehicle 12 increases, road noise, wind noise, etc. become louder, making it difficult to hear necessary sounds or causing unnecessary sound data to be transmitted.

[0036] In contrast, in the remote driving system 10 according to this embodiment, the information transmitted from the vehicle 12 to the remote cockpit 20 is only flag information, not the actual raw sound data picked up by the microphone 14, so the amount of communication data can be reduced. Therefore, in the remote driving system 10 according to this embodiment, there is no risk of sound lag due to communication delays, and there is no impact on communication costs.

[0037] The remote driving system 10 according to the present embodiment has been described above with reference to the drawings, but the remote driving system 10 according to the present embodiment is not limited to the one shown in the drawings, and can be appropriately modified in design without departing from the scope of the present invention. For example, the vibration device 28 does not have to be provided on the remote cockpit 20 side. In addition, the sources of sound generated around the vehicle 12 that is actually traveling are not limited to those described above. [Explanation of symbols]

[0038] 10 Remote Operation System 12 Vehicles 14 Microphone (sound pickup section) 16 Control section 18 Communication unit (transmission unit) 20 Remote cockpit (pilot's seat) 22 Communication unit (receiving unit) 24 Control section (notification section)

Claims

[Claim 1] A vehicle having a sound collection unit that collects sounds around the vehicle, a control unit that converts the sounds collected by the sound collection unit into predetermined flag information, and a transmission unit that transmits the flag information; a cockpit that has a receiving unit that receives the flag information and a notification unit that issues a notification corresponding to the content of the received flag information, and that can remotely drive the vehicle from outside the vehicle; Equipped with The notification unit stores various sampled sound data and vibration data, The notification unit is a remote driving system that selects sounds and vibrations corresponding to the content from the sound data and vibration data and notifies the pilot in the cockpit.

Images