Remote Control Device

Abstract

To provide advertisement information to a remote operator in a start standby time of a vehicle during a remote operation.SOLUTION: A remote operation device determines whether or not the state of a vehicle is a temporary stop state based on travel environment data of the vehicle during the remote operation of the vehicle by a remote operator. When it is determined that the state of the vehicle is not the temporary stop state, the remote operation device outputs display image data generated by using peripheral image data of the vehicle to a display device. When it is determined that the state of the vehicle is the temporary stop state, the remote operation device outputs display image data generated by using the peripheral image data of the vehicle and advertisement image data to the display device.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an apparatus for remotely driving a vehicle. [Background technology]

[0002] JP 2004-295360 A discloses a device for remotely driving a vehicle. This conventional device is provided in a control center that communicates with the vehicle. This conventional device also includes a control device, a display, a driver's seat, various sensors, etc. Images from a camera of the vehicle are output to the display. The driver's seat has a configuration similar to that of a driver's seat in an actual car. A remote operator sitting in the driver's seat operates input devices such as a steering wheel, an accelerator pedal, and a brake pedal while looking at an image from an in-vehicle camera output to the display. Various sensors detect the amount of operation of the input devices by the remote operator. The control device calculates the control amount of the vehicle's running devices such as the steering device, the drive device, and the braking device based on this amount of operation. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2004-295360 A Summary of the Invention [Problem to be solved by the invention]

[0004] During remote driving, the remote operator may wait for the vehicle to start. For example, while the nearest traffic light is showing a red light, the remote operator stops the vehicle and waits for it to start. Another example is when waiting for a train to pass at a railroad crossing. Another example is when the vehicle is stuck in traffic. Therefore, it is desirable to develop a technology that provides useful information to the remote operator during such waiting times.

[0005] One object of the present invention is to provide a technique for providing advertising information to a remote operator during a waiting period before starting a vehicle that is being remotely driven. [Means for solving the problem]

[0006] The present invention is a remote driving device for remotely driving a vehicle, and has the following features. The remote operation device is A data processing device including a memory in which various data are stored and a processor for processing the various data; A communication device for communicating with the vehicle; a display device that provides a remote operator with display image data generated based on the various data; Equipped with. The various data include surrounding image data of the vehicle, driving environment data of the vehicle, and advertisement image data. The processor determines whether a condition for temporarily stopping the vehicle is met based on the driving environment data during remote driving of the vehicle by the remote operator. When it is determined that the temporary stop condition is not satisfied, the processor outputs the display image data generated using the surrounding image data to the display device. When the processor determines that the pause condition is satisfied, it outputs an advertising start notification to the display device, and after outputting the advertising start notification, outputs the display image data generated using the surrounding image data and the advertising image data to the display device. The processor determines whether or not an advertisement end condition is satisfied when the display image data generated using the surrounding image data and the advertisement image data is output to the display device. When the processor determines that the advertisement end condition is satisfied, it outputs an advertisement end notification to the display device, and after outputting the advertisement end notification, outputs the display image data generated using the surrounding image data to the display device. The various data further includes interior image data of the vehicle. The advertising image data is created in accordance with the attributes of customers targeted by the advertising provider. The subjects of transportation by remote driving of the vehicle include the family and relatives of the remote operator. The processor estimates attributes of the object to be transported by analyzing the indoor image data, and narrows down the advertising image data to be output to the display device based on the attributes of the object to be transported. Effect of the Invention

[0007] According to the present invention, when it is determined that the vehicle is in a temporary stop state, display image data generated using the surrounding image data and the advertising image data is output to the display device. Therefore, it is possible to provide advertising information to the remote operator during the waiting time for the vehicle to start. Advertising information that matches the attributes, tastes, etc. of the remote operator can be useful information for the remote operator. When a specific object is transported by remote driving, advertising information that matches the attributes of the transport object, the destination of the transport, etc. can be useful information for the remote operator. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of application of remote driving by a remote driving device according to an embodiment of the present invention. FIG. [Diagram 2] FIG. 1 is a diagram illustrating features of an embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an example of the configuration of a vehicle. [Figure 4] FIG. 2 is a diagram illustrating an example of the configuration of a remote driving device. [Diagram 5] 5 is a flowchart showing an example of remote driving processing by a data processing device of the remote driving device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, a remote operation device according to an embodiment of the present invention will be described with reference to the drawings. In each drawing, the same or corresponding parts are denoted by the same reference numerals, and the description thereof will be simplified or omitted.

[0010] 1. Application Examples FIG. 1 is a diagram for explaining an application example of remote driving by a remote driving device according to an embodiment. In the example shown in FIG. 1, the remote driving device 1 is installed at a first point P1 (specifically, the home HM of a remote operator OP). However, the remote driving device 1 does not have to be installed at the first point P1. In other words, the remote driving device 1 may be installed at a point different from the first point P1. The remote driving device 1 communicates with a vehicle 2. The communication between the remote driving device 1 and the vehicle 2 is performed via a base station 3.

[0011] In the example shown in Fig. 1, a vehicle 2 is remotely driven by a remote driving device 1. In this remote driving, the vehicle 2 travels back and forth between a first point P1 (specifically, a parking lot at a home HM) and a second point P2 (specifically, a parking lot at a facility FC such as a school, cram school, or hospital). The vehicle 2 travels back and forth to pick up or drop off a transportation target OB to the second point P2. Examples of the transportation target OB include a family member or relative of the remote operator OP.

[0012] To remotely drive the vehicle 2, the remote operator OP starts up the remote driving device 1. At the time of this start-up, the remote driving device 1 authenticates the remote operator OP. If the authentication conditions are met, the remote operator OP is able to remotely drive the vehicle 2. During remote driving, the remote operator OP operates input devices such as the steering wheel, accelerator pedal, and brake pedal while looking at the display device 11 of the remote driving device 1.

[0013] During remote driving, surrounding image data of the vehicle 2 is transmitted from the vehicle 2 to the remote driving device 1. The remote driving device 1 generates display image data using this surrounding image data and outputs this to the display device 11. Examples of surrounding images include a front center image, a left front image, a right front image, a left side image, a right side image, and a rear image of the vehicle 2. The display image is a composite of these images, etc. The display image is typically a moving image, but may also be a still image.

[0014] 2. Features of the embodiment Fig. 2 is a diagram for explaining the features of the embodiment. Fig. 2 shows three display devices 11a to 11c as an example of the display device 11. Display image data DIMa is displayed on the display device 11a. Display image data DIMb is displayed on the display device 11b, and display image data DIMc is displayed on the display device 11c. In the example shown in Fig. 2(I), the display image data DIMa is data of a front center image ICF, the display image data DIMb is data of a left front image ILF, and the display image data DIMc is data of a right front image IRF.

[0015] The left and right side image data and the rear image data are output to a display device other than the display devices 11a to 11c. However, these data may be combined with any image data output to the display devices 11a to 11c and output to any of the display devices 11a to 11c. For example, the rear image data may be combined with the front center image ICF data and output to the display device 11a. The left side image data may be combined with the left front image ILF data and output to the display device 11b. The right side image data may be combined with the right front image IRF data and output to the display device 11c.

[0016] During remote driving, in addition to surrounding image data, driving environment data is transmitted from the vehicle 2 to the remote driving device 1. The driving environment data is data related to the driving environment of the vehicle 2. The driving environment data includes data related to the driving conditions of the vehicle 2, such as vehicle speed data, acceleration data, and steering angle data. The driving environment data also includes data related to objects around the vehicle 2 recognized by the vehicle 2. The driving environment data further includes position data of the vehicle 2. The driving environment data further includes VICS (Vehicle Information and Communication System, registered trademark) data such as traffic congestion information and accident information.

[0017] The remote driving device 1 judges whether the vehicle 2 is in a temporary stop state based on the driving environment data. For example, if the vehicle speed is equal to or less than a predetermined value and the amount of operation of the accelerator pedal by the remote operator OP is equal to or less than a predetermined value, the vehicle is judged to be in a temporary stop state. In another example, if the vehicle speed is equal to or less than a predetermined value and a red light is recognized in front of the vehicle 2, the vehicle is judged to be in a temporary stop state. In yet another example, if the vehicle speed is equal to or less than a predetermined value and a railroad crossing in operation is recognized in front of the vehicle 2, the vehicle is judged to be in a temporary stop state. In yet another example, if the vehicle speed is equal to or less than a predetermined value and it is estimated from the position data and traffic information of the vehicle 2 that the vehicle 2 is caught in a traffic jam, the vehicle is judged to be in a temporary stop state.

[0018] When it is determined that the vehicle 2 is in a temporarily stopped state, the remote driving device 1 generates display image data by combining the surrounding image data with the advertising image data. FIG. 2(II) shows an example of display image data DIMa to DIMc generated after it is determined that the vehicle 2 is in a temporarily stopped state. In the example shown in FIG. 2(II), the display image data DIMa is data of the front center image ICF, and the display image data DIMc is data of the right front image IRF. Up to this point, it is the same as the example shown in FIG. 2(I). On the other hand, the display image data DIMb is data obtained by combining the left front image ILF and the advertising image IAD.

[0019] In FIG. 2(II), the advertising image IAD is superimposed on a partial area of ​​the left front image ILF. However, the advertising image IAD may be superimposed on the entire area of ​​the left front image ILF. The advertising image IAD may be combined with the front center image ICF and displayed on the display device 11a, or may be combined with the right front image IRF and displayed on the display device 11b. The advertising image IAD may be combined with the left front image ILF and at least one of the front center image ICF and the right front image IRF. In this case, the display image data DIM may be simultaneously displayed on the display device 11c and at least one of the display devices 11a and 11b.

[0020] While the remote driving device 1 is outputting display image data generated using the advertisement image data from the display device 11, the remote driving device 1 determines whether the state of the vehicle 2 is ready to start based on the driving environment data. For example, when a green light is recognized ahead of the vehicle 2, it is determined that the vehicle is ready to start. In another example, when the start of a vehicle preceding the vehicle 2 is recognized, it is determined that the vehicle 2 is ready to start. In yet another example, when the end of operation of a railroad crossing ahead of the vehicle 2 is recognized, it is determined that the vehicle is ready to start.

[0021] When it is determined that the state of the vehicle 2 is ready to start, the remote driving device 1 ends the generation of display image data using the advertisement image data, and resumes the generation of display image data using the surrounding image data. Fig. 2(III) shows an example of display image data DIMa-DIMc generated after it is determined that the state of the vehicle 2 is ready to start. In the example shown in Fig. 2(III), data similar to the display image data DIMa-DIMc shown in Fig. 2(I) is output to the display devices 11a-11c, respectively.

[0022] In this way, according to the feature of the embodiment, when it is determined that the state of the vehicle 2 is in a temporary stop state, the display image data DIM generated using the surrounding image data and the advertisement image data is output to the display device 11. Therefore, it is possible to provide advertisement information to the remote operator OP during the waiting time for the vehicle 2 to start moving.

[0023] A configuration example of the remote operation device 1 for realizing the features of the above-described embodiment will be described below.

[0024] 3.Remote control device 3-1. Example of a remotely operated vehicle configuration Fig. 3 is a block diagram showing an example of the configuration of the vehicle 2 shown in Fig. 1. The example of the configuration shown in Fig. 3 includes a camera 21, a microphone 22, a sensor group 23, a map database (DB) 24, a communication device 25, a traveling device 26, and a data processing device 27.

[0025] A plurality of cameras 21 are provided inside and outside the vehicle 2. The cameras 21 include an exterior camera that captures images of the surroundings of the vehicle 2. Examples of surrounding images captured by the exterior camera include a front center image, a right front image, a left front image, a right side image, a left side image, and a rear image. The exterior camera transmits these image data to the data processing device 27 as "surrounding image data IMG." The cameras 21 also include an interior camera that captures images of the interior of the vehicle 2. The interior camera transmits data of interior images to the data processing device 27.

[0026] A plurality of microphones 22 are provided on the exterior side of the vehicle 2. Each of the microphones 22 has directionality and records ambient sounds of the vehicle 2. Examples of ambient sounds include right front sound, left front sound, right rear sound, and left rear sound. Each of the microphones 22 transmits the recorded sound data to the data processing device 27 as "ambient sound data SUD."

[0027] The sensor group 23 includes a status sensor that detects the status of the vehicle 2. Examples of the status sensor include a vehicle speed sensor, an acceleration sensor, a yaw rate sensor, and a steering angle sensor. The sensor group 23 also includes a position sensor that acquires position data of the vehicle 2. Examples of the position sensor include a Global Navigation Satellite System (GNSS) sensor. The status sensor transmits the status data of the vehicle 2 to the data processing device 27, and the position sensor transmits the position data to the data processing device 27. The sensor group 23 further includes a recognition sensor other than the camera 21. The recognition sensor recognizes (detects) targets around the vehicle 2 by using radio waves or light. Examples of the recognition sensor include a millimeter wave radar and a laser imaging detection and ranging (LIDAR). The recognition sensor transmits recognition data of the targets to the data processing device 27.

[0028] The status data of the vehicle 2, the position data of the vehicle 2, and the recognition data of surrounding objects recognized by the camera 21 or a recognition sensor other than the camera 21 are included in the "driving environment data ENV".

[0029] Map data is stored in the map DB 24. Examples of the map data include data on road positions, data on road shapes (e.g., types of curves and straight lines), and data on the positions of intersections and structures. The map DB 24 is formed in an on-board storage device (e.g., a non-volatile recording medium such as a hard disk or a flash memory). The map DB 24 may be formed in a computer (e.g., an external server) capable of communicating with the vehicle 2.

[0030] The communication device 25 performs wireless communication with the base station 3. Examples of communication standards for this wireless communication include mobile communication standards such as 4G, LTE, and 5G. The connection destinations of the communication device 25 include at least the remote driving device 1. In communication with the remote driving device 1, the communication device 25 transmits various data received from the data processing device 27 to the remote driving device 1.

[0031] The traveling device 26 accelerates, decelerates, and steers the vehicle 2. The traveling device 26 includes, for example, a drive device, a steering device, and a brake device. The drive device drives the tires of the vehicle 2. The steering device steers the tires of the vehicle 2. The brake device applies a braking force to the vehicle 2. The acceleration of the vehicle 2 is performed by controlling the drive device. The deceleration of the vehicle 2 is performed by controlling the brake device. If the drive device is a motor, the braking of the vehicle 2 may be performed using regenerative braking controlled by the motor. The steering of the vehicle 2 is performed by controlling the steering device.

[0032] The data processing device 27 is a computer for processing various data acquired by the vehicle 2. The data processing device 27 includes at least one processor 27a and at least one memory 27b. The processor 27a includes a CPU (Central Processing Unit). The memory 27b is a volatile memory such as a DDR memory, and expands various programs used by the processor 27a and temporarily stores various data. Various data acquired by the vehicle 2 is stored in the memory 27b. The various data include surrounding image data IMG, surrounding sound data SUD, and driving environment data ENV. When the vehicle 2 is remotely driven, the various data also includes control data for the driving device 26 received from the remote driving device 1.

[0033] The processor 27a executes programs stored in the memory 27b to perform various controls related to the traveling of the vehicle 2. When the vehicle 2 is remotely driven, the processor 27a also encodes various data stored in the memory 27b and outputs the encoded data to the communication device 25. The various data include ambient image data IMG, ambient sound data SUD, and traveling environment data ENV. These data may be compressed during the encoding process. The processor 27a further controls the traveling device 26 based on the control data stored in the memory 27b.

[0034] 3-2. Example of remote operation device configuration Fig. 4 is a block diagram showing a configuration example of the remote driving device 1 shown in Fig. 1. The configuration example shown in Fig. 4 includes a display device 11, a sound reproducing device 12, an input device 13, a map database (DB) 14, an advertisement database (DB) 15, a communication device 16, and a data processing device 17.

[0035] The display device 11 outputs display image data DIM generated based on the surrounding image data IMG. Examples of the display device 11 include a liquid crystal display (LCD) and an organic light emitting diode (OLED) display. The display device 11 includes three display devices 11 that respectively display data of a front center image, a left front image, and a right front image of the vehicle 2. The display device 11 may be wearable glasses. The wearable glasses are glasses-type or goggle-type terminals worn by a remote operator OP. When the display device 11 is wearable glasses, the front center image, the left front image, and the right front image are synthesized and displayed on the wearable glasses as a single image.

[0036] The sound reproducing device 12 reproduces the reproduced sound data PSU generated based on the ambient sound data SUD. The ambient sound data SUD is recorded sound data in each directional direction of the microphone 22. The reproduced sound data PSU may be data after processing such as wave field synthesis. The reproduced sound data PSU may be data from which sounds other than the vehicle sound and the environmental sound recognized by the analysis of the ambient sound data SUD are cut. The vehicle sound to be recognized is a sound generated with steering, acceleration, or deceleration. Examples of sounds generated with steering include the sound of the tires turning and the friction sound between the tires and the road surface. Examples of sounds generated with acceleration include the rotation sound of the drive motor. Examples of sounds generated with deceleration include the rotation sound of the drive motor and the friction sound between the tires and the road surface. Examples of environmental sounds to be recognized include warning sounds issued by railroad crossings and traffic signals, warning sounds issued by emergency vehicles, and warning sounds issued by vehicles around the vehicle 2.

[0037] The input device 13 is a device operated by a remote operator OP. The input device 13 includes driving input devices for controlling the driving of the vehicle 2, such as an accelerator pedal, a brake pedal, and a steering wheel. The input device 13 also includes auxiliary input devices for assisting the driving of the vehicle 2, such as a turn signal switch, a light switch, and a wiper switch. The input device 13 detects the amount of operation of the driving input devices by the remote operator OP or the operation status of the auxiliary input devices, and transmits the same to the data processing device 27.

[0038] Map data is stored in the map DB 14. Examples of the map data include the same map data as the example of the map data stored in the map DB 24 described with reference to FIG.

[0039] The advertising DB15 stores data on advertising image IAD and advertising sound SAD. The advertising image IAD and advertising sound SAD are stored for each piece of advertising information. The advertising information is created according to the attributes (age group, gender, etc.) of customers targeted by the advertising provider. The advertising image IAD and advertising sound SAD are also read out for each piece of advertising information. Data constituting any piece of advertising information is read out from the advertising DB15.

[0040] Here, when the remote operator OP performs remote driving, authentication of the remote operator OP may be required. In this case, identification data of the remote operator OP is separately input to the data processing device 17. In the case of the application example described in FIG. 1, current position data of the remote operator OP and the vehicle 2 and destination data of the vehicle 2 are specified. Furthermore, when data of an interior image of the vehicle 2 is analyzed, attributes of the object OB to be transported by the vehicle 2 can also be estimated. When data is read from the advertisement DB 15, the read data may be narrowed down based on these linking data.

[0041] The communication device 16 performs wireless communication with the base station 3. Examples of communication standards for this wireless communication include mobile communication standards such as 4G, LTE, and 5G. The connection destination of the communication device 16 includes at least the vehicle 2. In communication with the vehicle 2, the communication device 16 transmits various data (e.g., data for controlling the driving device 26) received from the data processing device 17 to the vehicle 2. The connection destination of the communication device 16 may include a VICS (registered trademark) data server. In this case, the communication device 16 acquires VICS (registered trademark) data around the current location of the vehicle 2 from the server.

[0042] The data processing device 17 is a computer that processes various data related to the remote driving of the vehicle 2. The data processing device 17 includes at least one processor 17a and at least one memory 17b. The basic configuration of the data processing device 17 is the same as that of the data processing device 27 described in FIG. 3. The memory 17b stores various programs and various data used by the processor 17a. The various data include ambient image data IMG, ambient sound data SUD, and driving environment data ENV.

[0043] When remote driving the vehicle 2, the processor 17a decodes various data received from the vehicle 2, generates display image data DIS based on the ambient image data IMG, and outputs the display image data DIS to the display device 11. The processor 17a also generates playback sound data PSU based on the ambient sound data SUD, and outputs the playback sound data PSU to the sound playback device 12. If the various data received from the vehicle 2 is compressed, the information is expanded in the decoding process.

[0044] The processor 17a further generates control data for the vehicle 2 based on the amount of operation of the driving input device received from the input device 13, and outputs the control data to the communication device 16. The processor 17a still further generates operation data for various driving assistance devices of the vehicle 2 based on the operation status of the auxiliary input device received from the input device 13, and outputs the operation data to the communication device 16.

[0045] 3-3. Example of processing by data processing device Fig. 5 is a flowchart showing an example of remote driving processing by the data processing device 17 (processor 17a) The routine shown in Fig. 5 is repeatedly executed at a predetermined period after the remote operator OP starts remote driving the vehicle 2.

[0046] 5, first, it is determined whether or not the temporary stop condition is satisfied (step S11). In the process of step S11, for example, it is determined whether or not any one of the temporary stop conditions C1 to C4 is satisfied based on the traveling environment data ENV. (C1) The vehicle speed is below a predetermined value, and the amount of operation of the accelerator pedal by the remote operator OP is below a predetermined value. (C2) The vehicle speed is below a predetermined value and a red light is detected in front of vehicle 2. (C3) The vehicle speed is below a certain value, and an active railroad crossing is detected ahead of vehicle 2. (C4) The vehicle speed is below a predetermined value, and it is estimated that vehicle 2 is caught in a traffic jam based on the position data and traffic information of vehicle 2.

[0047] If the determination result in step S11 is negative, the process of step S16 (described later) is performed. If the determination result in step S11 is positive, an advertisement start notification is performed (step S12). In the process of step S12, for example, icon image data of "advertising starting" is combined with the surrounding image data IMG and output to the display device 11. In another example, audio data of "advertising starting" is output to the sound reproduction device 12.

[0048] Following the process of step S12, an advertising image IAD and an advertising sound SAD are provided (step S13). In the process of step S13, for example, data is narrowed down based on attributes of the remote operator OP or the target OB. Next, the data of the advertising image IAD read out from the advertising DB 15 is synthesized with predetermined surrounding image data IMG. Then, display image data DIS, which is the image data after the synthesis, is output to the display device 11 which outputs the predetermined surrounding image data IMG. In the process of step S13, the data of the advertising sound SAD read out from the advertising DB 15 is also output to the sound reproduction device 12.

[0049] Following the process of step S13, it is determined whether or not the advertisement termination condition is satisfied (step S14). In the process of step S14, for example, it is determined whether or not any one of the advertisement termination conditions C5 to C8 is satisfied based on the driving environment data ENV (recognition data). (C5) A green light was detected ahead of vehicle 2. (C6) The departure of the vehicle ahead of vehicle 2 was recognized. (C7) When the end of operation of the railroad crossing in front of vehicle 2 is recognized (C8) An operation to start vehicle 2 by remote operator OP (e.g., depressing the accelerator pedal) is detected.

[0050] The process of step S14 is repeatedly executed until a positive determination result is obtained. If the determination result of step S14 is positive, an advertisement end notification is issued (step S15). The advertisement end notification is output to the display device 11, for example, by combining icon image data of "The advertisement will end soon. Please resume driving" with the surrounding image data IMG. In another example, audio data of "The advertisement will end soon. Please resume driving" is output to the sound reproduction device 12. In still another example, a signal for vibrating the steering wheel, the driver's seat, etc. is output.

[0051] In the process of step S16, display image data DIM is generated using the surrounding image data IMG, and is output to the display device 11 that outputs this surrounding image data IMG. In the process of step S16, playback sound data PSU is also generated using the surrounding sound data SUD, and is output to the sound playback device 12. When generating the playback sound data PSU, processing such as wave field synthesis of the surrounding sound data SUD may be performed, and sounds other than vehicle sounds and environmental sounds recognized by analysis of the surrounding sound data SUD may be cut. [Explanation of symbols]

[0052] 1 Remote control device 2 Vehicles 11,11a,11b,11c Display device 15 Advertising Database 16,25 Communication equipment 17,27 Data processing device 17a,27a processor 17b,27b Memory FC Facilities HM Home OB Transport Target OP Remote Operator DIM, DIMa, DIMb, DIMc Display image data ENV Driving environment data IAD advertising image ICF front center image ILF Left Front Image IRF right front image IMG Surrounding image data PSU playback sound SUD ambient sound data

Claims

[Claim 1] A remote driving device for remotely driving a vehicle, A data processing device including a memory in which various data are stored and a processor for processing the various data; A communication device for communicating with the vehicle; a display device that provides a remote operator with display image data generated based on the various data; Equipped with The various data include surrounding image data of the vehicle, driving environment data of the vehicle, and advertisement image data, The processor, determining whether a temporary stop condition for the vehicle is satisfied based on the driving environment data during remote driving of the vehicle by the remote operator; When it is determined that the temporary stop condition is not satisfied, the display image data generated using the surrounding image data is output to the display device; When it is determined that the pause condition is satisfied, an advertisement start notification is output to the display device; After outputting the advertisement start notification, outputting the display image data generated by using the surrounding image data and the advertisement image data to the display device; The processor further comprises: determining whether or not an advertisement end condition is satisfied when the display image data generated using the surrounding image data and the advertisement image data is output to the display device; When it is determined that the advertisement termination condition is satisfied, an advertisement termination notice is output to the display device; After outputting the advertisement end notification, outputting the display image data generated using the surrounding image data to the display device; The various data further includes interior image data of the vehicle, The advertising image data is created in accordance with the attributes of customers targeted by the advertising provider, The transportation targets by remote driving of the vehicle include the family and relatives of the remote operator, The processor further comprises: estimating attributes of the transportation object by analyzing the indoor image data; The advertisement image data to be output to the display device is narrowed down based on the attributes of the transportation object. A remote driving device characterized by:

Images