Remote travel control device and remote operation device

Abstract

To provide a remote travel control device capable of displaying information which a use desires to know on a control screen.SOLUTION: When remote travel control is being executed to allow a vehicle to travel to a target space based on a travel instruction from a remote operation device operated by a user on an outside of the vehicle, a remote travel control device allows a display device of the remote operation device to display a control screen including information related to the remote travel control, so as to change contents to be displayed on the control screen in response to a progress of the remote travel control.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a remote driving control device that performs remote driving control to drive a vehicle to a target space based on driving instructions from a remote control device operated by a user outside the vehicle, and to a remote control device that a user outside the vehicle operates to drive the vehicle to the target space. [Background technology]

[0002] Conventionally, remote driving control devices that execute remote driving control have been known. For example, a remote driving control device described in Patent Document 1 (hereinafter referred to as the "conventional device") executes remote parking control (remote parking control) as remote driving control, in which a user outside the vehicle operates a remote control device to park the vehicle in a target space. When executing remote parking control, the conventional device causes the remote control device to display a remote parking screen as a control screen. The remote parking screen includes an image that includes a blind spot located on the opposite side of the vehicle from the user and is aligned with the user's line of sight as they look toward the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2021-160499 Summary of the Invention

[0004] The approach possibility of approaching an obstacle changes depending on the progress of the remote traveling control. Information about the remote traveling control that the user wants to know (hereinafter referred to as "request information") changes depending on the approach possibility. In other words, the request information changes depending on the progress of the remote traveling control.

[0005] For example, in remote parking control, the approach probability in the parking stage when the vehicle actually parks in the parking space is higher than the approach probability in the start stage when the remote parking control starts. For example, when the approach probability is high, the user tends to want to know the positional relationship between the vehicle and the obstacle, and when the approach probability is low, the user tends to want to know the progress of the remote driving control.

[0006] If the request information when the approach possibility is high and the request information when the approach possibility is low are displayed on the control screen at the same time, the control screen becomes cluttered, and the user is likely to find it bothersome to look at the control screen.If only the request information when the approach possibility is high is displayed on the control screen, the user cannot know the request information when the approach possibility is low.

[0007] The present invention has been made to address the above-mentioned problems. That is, one of the objects of the present invention is to provide a remote driving control device that can display information that the user wants to know on the control screen.

[0008] The remote driving control device of the present invention (hereinafter referred to as the "device of the present invention") executes remote driving control to drive the vehicle to the target space based on driving instructions from a remote control device (20) operated by a user outside the vehicle (steps 600 to 695, steps 700 to 795), If the remote driving control is being performed (step 605 "Yes", step 705 "Yes"), a control screen (200, 300) including information related to the remote driving control is displayed on the display device (66) of the remote operation device (steps 640, 650, 740, and 750). The content displayed on the control screen is changed according to the progress of the remote driving control (steps 640, 650, 740, 750, 815, and 820). It is structured as follows.

[0009] According to the device of the present invention, the content displayed on the control screen changes depending on the progress of remote driving control. This makes it possible for the device to display on the display device a control screen including user-requested information that changes depending on the progress of remote driving control. Therefore, it is possible to provide a remote driving control device that can display the information that the user wants to know on the control screen while avoiding the control screen becoming cluttered.

[0010] According to one aspect of the device of the present invention, a route to the target space on which the vehicle travels is divided into a first section in which the vehicle is less likely to approach an obstacle and a second section in which the vehicle is more likely to approach the obstacle, with a switching position as a dividing point; The remote travel control device includes: If the vehicle is traveling in the first section (step 635 "No", step 630 "No" in FIG. 7), the first control screen is displayed on the display device as the control screen (step 640, step 720), If the vehicle is traveling in the second section (step 630 "No", step 635 "No" in FIG. 7), a second control screen having content different from that of the first control screen is displayed as the control screen on the display device (step 650, step 715). It is structured as follows.

[0011] According to this aspect, when the vehicle is traveling in a first section where the accessibility is low, the first control screen is displayed, and when the vehicle is traveling in a second section where the accessibility is high, the second control screen is displayed, thereby making it possible to change the control screen depending on the accessibility.

[0012] In the above aspect, The remote travel control device includes: If the vehicle is traveling in the first section (step 635 "No", step 630 "No" in FIG. 7), progress information (210, 220) indicating the progress of the remote traveling control is displayed on the first control screen (200) (step 815), If the vehicle is traveling in the second section (step 630 "No", step 635 "No" in FIG. 7), alarm information (305a, 305b) that issues an alarm about the obstacle whose distance to the vehicle is less than or equal to a threshold distance is displayed on the second control screen (300) (step 820). It is structured as follows.

[0013] A user tends to want to know the progress of remote driving control when the approach probability is low, and tends to want to know the position of an obstacle when the approach probability is high. According to this aspect, progress information is displayed when the vehicle is traveling in the first section, and warning information is displayed when the vehicle is traveling in the second section. As a result, the information that the user wants to know is displayed according to the approach probability, so the user can know the information that he or she wants to know.

[0014] In the above aspect, The remote travel control device is configured to display, on the first control screen as the progress information, at least one of route information (210) that divides the route to the target space into routes that have been traveled and routes that have not yet been traveled, and remaining distance information (220) that indicates the remaining distance to the target space.

[0015] When the vehicle is traveling in the first section, the user can know at least one of the route information and the remaining distance information, and can know the progress of the remote traveling control.

[0016] In the above aspect, The remote traveling control device is configured to display the warning information (305a, 305b) on the second control screen so as to be superimposed on an overhead image of the periphery of the vehicle.

[0017] When the vehicle is traveling in the second section, the user can know the positions of obstacles around the vehicle.

[0018] In the above aspect, The remote travel control device includes: Until the vehicle reaches the switching position (step 635 "No"), movement control for moving the vehicle to the vicinity of the target space is executed as the remote travel control (step 625), After the vehicle reaches the switching position (step 630 "No"), parking control for parking the vehicle in the target space is executed as the remote driving control (step 625), If the movement control is being executed, it is determined that the vehicle is traveling in the first section (step 640); If the parking control is being executed, it is determined that the vehicle is traveling in the second section (step 650). It is structured as follows.

[0019] Generally, the approach possibility tends to be low when movement control is being executed, and high when parking control is being executed. According to this aspect, when movement control is being executed, it is determined that the vehicle is traveling in the first section, and when parking control is being executed, it is determined that the vehicle is traveling in the second section. This allows the device to accurately determine the approach possibility based on the progress of remote travel control.

[0020] In the above aspect, The remote traveling control device is configured to, when executing the remote traveling control, travel the vehicle so that a vehicle speed representing a speed of the vehicle does not exceed an upper limit vehicle speed (step 625); The upper limit vehicle speed of the parking control is preset to a value lower than the upper limit vehicle speed of the movement control (step 655).

[0021] According to this aspect, the upper speed limit for parking control is set to a value lower than the upper speed limit for movement control. This allows the vehicle to be stopped immediately in the parking control when the possibility of the vehicle coming into contact with an obstacle increases. Furthermore, the movement time of the vehicle in the movement control can be shortened.

[0022] In the above aspect, The remote travel control device includes: From the start of the remote traveling control until the vehicle reaches the switching position (step 635 "No" in FIG. 7), the remote traveling control is executed as the parking exit control to cause the vehicle to exit the parking space (step 625 in FIG. 7), After the vehicle reaches the switching position (step 630 "No" in FIG. 7), a movement control for moving the vehicle to the target space is executed as the remote travel control (step 625), When the leaving control is being executed, it is determined that the vehicle is traveling in the second section (step 715), If the movement control is being executed, it is determined that the vehicle is traveling in the first section (step 720); It is structured as follows.

[0023] Generally, the possibility of approach tends to be low when movement control is being executed, and high when departure control is being executed. According to this aspect, when movement control is being executed, it is determined that the vehicle is traveling in the first section, and when departure control is being executed, it is determined that the vehicle is traveling in the second section. This allows the device to accurately determine the possibility of approach based on the progress of remote travel control.

[0024] In the above aspect, The remote traveling control device is configured to, when executing the remote traveling control, travel the vehicle so that a vehicle speed representing a speed of the vehicle does not exceed an upper limit vehicle speed (step 625 in FIG. 7 ); The upper limit vehicle speed for the leaving control is preset to a value lower than the upper limit vehicle speed for the movement control (step 710).

[0025] According to this aspect, the upper speed limit for the pull-out control is set to a value lower than the upper speed limit for the movement control. This allows the vehicle to be stopped immediately in the pull-out control if the possibility of the vehicle coming into contact with an obstacle increases. Furthermore, the movement time of the vehicle in the movement control can be shortened.

[0026] The remote control device (20) of the present invention transmits driving instructions to the vehicle when operated by a user outside the vehicle (440, 465). The vehicle executes remote driving control to drive to the target space in accordance with the driving instruction (steps 600 to 695, steps 700 to 795). The remote control device is When the vehicle is performing the remote driving control, a control screen (200, 300) including information about the remote driving control is displayed on a display device of the remote operation device; The content displayed on the control screen is changed according to the progress of the remote travel control (steps 815 and 820). It is structured as follows.

[0027] According to the remote control device of the present invention, the content displayed on the control screen changes according to the progress of remote driving control, thereby enabling the remote control device to display on the display device a control screen including user request information that changes according to the progress of remote driving control. [Brief explanation of the drawings]

[0028] [Figure 1] 1 is a schematic system configuration diagram of a vehicle control system according to an embodiment of the present invention; [Figure 2] 1. FIG. 4 is an explanatory diagram of a progress screen displayed on the display device of the remote control device shown in FIG. [Figure 3] 1. FIG. 4 is an explanatory diagram of an obstacle screen displayed on the display device of the remote control device shown in FIG. [Figure 4] FIG. 3 is a sequence diagram illustrating an example of operation of the vehicle control system according to the embodiment of the present invention. [Figure 5] 2 is a flowchart of a start determination routine executed by a CPU of the ECU shown in FIG. 1. [Figure 6] 2 is a flowchart of a remote parking control routine executed by a CPU of the ECU shown in FIG. 1. [Figure 7] 2 is a flowchart of a remote retrieval control routine executed by a CPU of the ECU shown in FIG. 1. [Figure 8] 2 is a flowchart of a screen display routine executed by a CPU of the control unit shown in FIG. 1. DETAILED DESCRIPTION OF THE INVENTION

[0029] 1, the vehicle control system according to this embodiment includes a remote driving control device 10 applied to a vehicle VA and a remote operation device 20. The remote driving control device 10 and the remote operation device 20 are connected to each other so as to be able to communicate with each other via a network NW.

[0030] The remote traveling control device 10 includes the components shown in FIG. 1. In this specification, "ECU 30" is an electronic control device that includes a microcomputer as its main component. The ECU 30 is also referred to as a control unit, a controller, and a computer. The microcomputer includes a CPU (processor), a ROM, a RAM, an interface, and the like. The functions realized by the ECU 30 may be realized by multiple ECUs.

[0031] The front camera 32 captures a forward image by capturing an image of the scenery in front of the vehicle VA. The rear camera 34 captures a rearward image by capturing an image of the scenery behind the vehicle VA. The sonar 36 acquires sonar data relating to the positions of objects present in the vicinity of the vehicle VA relative to the vehicle VA. The ECU 30 acquires a forward image and a rearward image from the front camera 32 and the rearward camera 34, respectively, and acquires sonar data from the sonar 36. The ECU 30 recognizes objects located in the vicinity of the vehicle VA based on the front image, rearward image, and sonar data.

[0032] The wheel speed sensor 38 generates one pulse signal each time a wheel of the vehicle VA rotates a predetermined angle. The ECU 30 measures the number of pulse signals generated per unit time by the wheel speed sensor 38 and obtains the vehicle speed Vs, which indicates the speed of the vehicle VA, based on the number of measured pulse signals.

[0033] The GNSS (Global Navigation Satellite System) receiver 40 receives signals from multiple artificial satellites and identifies the current position (latitude and longitude) of the vehicle VA based on the received signals. The parking lot map data storage unit 42 stores map data of parking lots. The communication interface (I / F) 44 is an interface for connecting the remote traveling control device 10 to the network NW.

[0034] The power train actuator 46 changes the driving force generated by a drive device (e.g., an internal combustion engine and / or an electric motor) of the vehicle VA. The brake actuator 48 changes the braking force applied to the vehicle VA. The steering motor 50 is incorporated into a steering mechanism 52. The steering mechanism 52 is a mechanism for steering the steered wheels in response to the operation of the steering wheel. In response to a command from the ECU 30, the steering motor 50 causes the steering mechanism 52 to generate an automatic steering torque for changing the steering angle of the steered wheels.

[0035] The remote control device 20 is a device that can be operated by a user even when the user is outside the vehicle, and is, for example, a smartphone. The remote control device 20 includes the components shown in FIG.

[0036] The control unit 60 includes a CPU (processor), ROM, RAM, an interface, etc. The GNSS receiver 62 and the communication I / F 64 are respectively the same as the GNSS receiver 40 and the communication I / F 44. A description of these will be omitted.

[0037] The display device 66 is a touch panel display that allows the user to input to the remote control device 20 by touching the display device 66. If the display device 66 is not a touch panel display, the remote control device 20 is provided with an input device (not shown).

[0038] (Overview of operation) When a user outside the vehicle operates the remote operation device 20, the remote driving control device 10 executes "remote driving control for automatically driving the vehicle VA to a target space" in accordance with driving instructions transmitted by the remote operation device 20. For example, Smart Summon and Reverse Summon are known as remote driving controls.

[0039] Smart Summon is a type of remote parking control. With Smart Summon, the vehicle VA automatically travels from the parking space in which it is parked to a target space (the current position of the remote control device 20 identified by the GNSS receiver 62 or a designated position designated by the user). Reverse Summon is a type of remote parking control. With Reverse Summon, when the user gets off at the entrance to the parking lot or the like and operates the remote control device 20, the vehicle VA parks in the parking space (target space) designated by the user.

[0040] While remote driving control is being performed, the remote operation device 20 displays a control screen on the display device 66. The control screen includes an operation area (see reference numeral 215 in FIG. 2 and reference numeral 310 in FIG. 3). When the user performs a predetermined operation on the operation area, the remote operation device 20 transmits a driving instruction to the remote driving control device 10. Upon receiving the driving instruction, the remote driving control device 10 causes the vehicle VA to drive automatically. During automatic driving, the remote driving control device 10 controls the power train actuator 46, the brake actuator 48, and the steering motor 50 so that the vehicle VA drives along a route to the target space. Furthermore, the remote driving control device 10 recognizes objects based on a forward image (and / or a rearward image) and sonar data, and controls the power train actuator 46, the brake actuator 48, and the steering motor 50 to avoid contact with the objects.

[0041] The "probability of the vehicle VA approaching an obstacle" changes depending on the progress of the remote driving control. In remote leaving control, the approach probability is high in leaving control, in which the vehicle VA actually leaves the parking space, and low in movement control, in which the vehicle VA moves to the target space after leaving. In remote parking control, the approach probability is low in movement control, in which the vehicle VA moves to the vicinity of the target space, and high in parking control, in which the vehicle VA actually parks in the target space. The section with low approach probability is called the first section, and the section with high approach probability is called the second section. The position where the control switches in this way is called the switching position. The route to the target space is divided into the first section and the second section, with the switching position as the dividing point.

[0042] When the approachability changes, the information that the user wants to know also changes. More specifically, when the approachability is low, the user tends to want to know the progress of the remote driving control, and when the approachability is high, the user tends to want to know the position of the obstacle.

[0043] For this reason, in this embodiment, the remote traveling control device 10 changes the content displayed on the control screen according to the progress of the remote traveling control. Specifically, when the vehicle VA is traveling in the first section, the progress screen (first control screen) 200 shown in FIG. 2 is displayed as the control screen, and when the vehicle VA is traveling in the second section, the obstacle screen (second control screen) 300 shown in FIG. 3 is displayed as the control screen. The progress screen 200 displays at least "progress information indicating the progress of the remote traveling control," and the obstacle screen 300 displays "alarm information that issues an alert for an obstacle whose distance D to the vehicle VA is equal to or less than the first threshold distance D1th." According to this embodiment, the control screen displays the information that the user wants to know.

[0044] <Progress screen 200> The progress screen 200 will be described with reference to Fig. 2. The progress screen 200 includes a surrounding conditions display area 205, a route progress display area 210, an operation area 215, a remaining distance display area 220, and a vehicle speed display area 225.

[0045] The surrounding situation display area 205 displays the traveling direction of the vehicle VA, the driving area of ​​the vehicle VA, and objects existing around the vehicle VA. A traveled route 210a along which the vehicle VA has traveled and an untraveled route 210b along which the vehicle VA has not yet traveled are displayed in the route progress display area 210. The user can know the progress of the remote travel control by knowing the traveled route 210a and the untraveled route 210b. The operation area 215 is an area where the user performs a predetermined operation to cause the remote control device 20 to transmit a driving instruction. The remaining distance to the target space is displayed in the remaining distance display area 220. By knowing the remaining distance, the user can know the progress of the remote travel control. The vehicle speed display area 225 displays the vehicle speed Vs.

[0046] <Obstacle Screen 300> The obstacle screen 300 will be described with reference to FIG. At least the above warning information is displayed on the obstacle screen 300. The obstacle screen 300 includes a near-distance situation display area 305, an operation area 310, a traveling direction image display area 315, an obstacle distance display area 320, and a shift position display area 325.

[0047] When there is no obstacle whose distance D is equal to or less than the first threshold distance D1th, the near-distance situation display area 305 displays an overhead image of the scenery around the vehicle VA as seen from directly above the vehicle VA. When there is an obstacle whose distance D is equal to or less than the first threshold distance D1th, the near-distance situation display area 305 displays the above-mentioned warning information superimposed on the overhead image. More specifically, for an obstacle whose distance D is greater than a second threshold distance D2th that is smaller than the first threshold distance D1th and equal to or less than the first threshold distance D1th, a first warning display element 305a is displayed in the direction of the obstacle. For an obstacle whose distance D is equal to or less than the second threshold distance D2th, a second warning display element 305b is displayed in the direction of the obstacle. The second warning display element 305b is displayed in a manner that attracts the user's attention more than the first warning display element 305a.

[0048] Operation area 310 is the same as operation area 215, and therefore its description will be omitted.

[0049] An image of the traveling direction of the vehicle VA is displayed in the traveling direction image display area 315. When the shift position is either the forward position or the neutral position, a forward image is displayed in the traveling direction image display area 315, and when the shift position is the reverse position, a rearward image is displayed.

[0050] In the obstacle distance display area 320, the distance D of the nearest obstacle to the vehicle VA is displayed. The shift position display area 325 displays the current shift position.

[0051] <Example of operation> An example of the operation of the vehicle control system according to this embodiment will be described with reference to FIG. The user operates the remote operation device 20 to start the remote driving control app (405), and selects whether to perform remote leaving control or remote parking control (410). In this example, it is assumed that remote parking control is performed. The user then specifies a target space if necessary and operates a start button (not shown) displayed on the display device 66 (415), and the remote operation device 20 transmits a start instruction to the remote driving control device 10 (420). The start instruction transmitted when remote leaving control is performed is referred to as a "leaving start instruction," and the start instruction transmitted when remote parking control is performed is referred to as a "parking start instruction." The start instruction includes data capable of identifying the target space.

[0052] When the remote traveling control device 10 receives the start command, it sets a route to the target space by referring to the parking lot map data. Furthermore, the remote traveling control device 10 transmits a display command to the remote operation device 20 every time a predetermined time has elapsed since receiving the start signal (425a, 425b, 455a, 455b).

[0053] When remote parking control is performed, the vehicle VA will be traveling in the first section immediately after the start of remote parking control, so the progress screen 200 is displayed. For this reason, the remote traveling control device 10 transmits a display instruction including progress data necessary for displaying the progress screen 200. The progress data includes the following data. - Driving area data showing the driving area of ​​the vehicle VA Object data indicating the position and type of object identified by the front and rear images Travel direction data indicating the travel direction of the vehicle VA, which is determined based on the shift position and the steering angle of the vehicle VA Route data representing the route set when remote driving control was started Position history data representing the history of the current position of the vehicle VA as determined by the GNSS receiver 40 Remaining distance data showing remaining distance Vehicle speed data representing vehicle speed Vs

[0054] When remote leaving-parking control is performed, the vehicle VA will be traveling in the second section immediately after the start of remote leaving-parking control, and therefore the obstacle screen 300 will be displayed. For this reason, the remote traveling control device 10 transmits a display instruction including obstacle data necessary for displaying the obstacle screen 300. The obstacle data includes the following data. Bird's-eye view Position data representing the distance D and position of an obstacle whose distance D is equal to or less than a first threshold distance D1th A forward image that represents the forward direction of the vehicle VA, out of the forward and rearward images. Distance data representing the minimum distance D Shift position data that indicates the shift position

[0055] The remote traveling control device 10 is equipped with a left camera and a right camera (not shown). The left camera captures a left image by photographing the scenery on the left side of the vehicle VA. The right camera captures a left image by photographing the scenery on the left side of the vehicle VA. The remote traveling control device 10 generates an overhead image based on the front image, rear image, left image, and right image.

[0056] If there is no obstacle whose distance D is equal to or less than the first threshold distance D1th, the obstacle data does not include position data or distance data. The first threshold distance D1th is set to a value smaller than the distance at which the sonar 36 can detect an object. The remote traveling control device 10 determines whether there is an obstacle whose distance D is equal to or less than the first threshold distance D1th based on the sonar data.

[0057] When the remote operation device 20 receives a display instruction including the progress data, it displays the progress screen 200 on the display device 66 (430). When the user performs a predetermined operation in the operation area 215 of the progress screen 200 (435), the remote operation device 20 transmits a driving instruction (440).

[0058] As long as the remote driving control device 10 receives a driving instruction, it executes automatic driving control to automatically drive the vehicle VA along the route (445). In remote parking control, the remote driving control device 10 executes movement control as automatic driving control to move the vehicle VA to the vicinity of the target space before the vehicle VA reaches the switching position. The switching position is the position where movement control switches to parking control in remote parking control, and the position where exit control switches to movement control in remote leaving control. The switching position is set when the route to the target space is set.

[0059] Each time a predetermined time elapses after the vehicle VA reaches the switching position (450), the remote driving control device 10 transmits a display instruction including the obstacle data to the remote operation device 20 (455a, 455b). When the remote operation device 20 receives a display instruction including the obstacle data, it ends the display of the progress screen 200 and displays the obstacle screen 300 (460). When the user performs a predetermined operation in the operation area 310 of the obstacle screen 300, the remote operation device 20 transmits a driving instruction to the remote driving control device 10 (465). When the remote driving control device 10 receives the driving instruction, it executes parking control as automatic driving control to actually park the vehicle VA in the target space (470).

[0060] When the vehicle VA reaches the target space (475), the remote traveling control device 10 transmits a completion instruction to the remote operation device 20 (480). When the remote operation device 20 receives the completion instruction, it ends the display of the obstacle screen 300 and displays a completion screen (not shown) to inform the user that the vehicle VA has reached the target space (485).

[0061] As described above, when remote parking control is executed as remote traveling control, the progress screen 200 is displayed as the control screen on the display device 66 of the remote operation device 20 until the vehicle VA reaches the switching position, and after the vehicle VA reaches the switching position, the obstacle screen 300 is displayed as the control screen. This allows the user to know the information they want to know according to the progress of the remote traveling control by looking at the control screen.

[0062] (Specific operation) The CPU of the ECU 30 of the remote traveling control device 10 executes the routines shown in the flowcharts of Figures 5 to 7 every time a predetermined time elapses. Hereinafter, the CPU of the ECU 30 will be referred to as the "first CPU."

[0063] <Start determination routine> When an appropriate time arrives, the first CPU starts processing from step 500 in FIG. 5, and determines in step 505 whether the first execution flag Xexe1 is "0" and the second execution flag Xexe2 is "0".

[0064] The first execution flag Xexe1 is set to "1" when remote parking control is started, and is set to "0" when remote parking control is ended. The second execution flag Xexe2 is set to "1" when remote leaving control is started, and is set to "0" when remote leaving control is ended. Furthermore, the first execution flag Xexe1 and the second execution flag Xexe2 are set to "0" in an initial routine. The initial routine is executed by the first CPU when an ignition key switch (not shown) of the vehicle VA is changed from the OFF position to the ON position.

[0065] If the first execution flag Xexe1 is "0" and the second execution flag Xexe2 is "0", the first CPU determines "Yes" in step 505, and the process proceeds to step 510. In step 510, the CPU determines whether the remote traveling control device 10 has received a parking start command.

[0066] If the remote traveling control device 10 receives a parking start command, the first CPU determines "Yes" in step 510 and executes steps 515 to 525. Step 515: The first CPU sets the first execution flag Xexe1 to “1”. Step 520: The first CPU sets the arrival flag Xrea to "0". The arrival flag Xrea is set to "1" when the vehicle VA reaches the switching position, and is set to "0" when the remote travel control starts. The arrival flag Xrea is set to "0" in the initial routine. Step 525: The first CPU sets a route to the target space. Thereafter, the process proceeds to step 595, where the first CPU temporarily ends this routine.

[0067] If the remote traveling control device 10 has not received a parking start command when the process proceeds to step 510, the first CPU determines "No" in step 510, and the process proceeds to step 530. In step 530, the first CPU determines whether the remote traveling control device 10 has received a leaving-parking start command.

[0068] If the remote traveling control device 10 receives the instruction to start leaving the garage, the first CPU determines "Yes" in step 530, and the process proceeds to step 535. In step 535, the first CPU sets the second execution flag Xexe2 to "1." Thereafter, the first CPU executes step 520 and step 525, and the process proceeds to step 595, where the first CPU temporarily ends this routine.

[0069] If the remote traveling control device 10 has not received the instruction to start leaving the garage, the first CPU determines "No" in step 530, the process proceeds to step 595, and the first CPU temporarily ends this routine.

[0070] <Remote parking control routine> 6, and determines whether the first execution flag Xexe1 is "1" in step 605. If the first execution flag Xexe1 is "0," the first CPU determines "No" in step 605, and the process proceeds to step 695, where the first CPU temporarily ends this routine.

[0071] If the first execution flag Xexe1 is "1", the first CPU determines "Yes" in step 605, and the process proceeds to step 610. In step 610, the first CPU determines whether the remote traveling control device 10 has received a traveling instruction.

[0072] If the remote traveling control device 10 has received a traveling instruction, the first CPU determines "Yes" in step 610, and the process proceeds to step 615. In step 615, the first CPU determines whether the arrival flag Xrea is "0" or not.

[0073] If the arrival flag Xrea is "0", the first CPU determines "Yes" in step 615 and executes steps 620 to 630. Step 620: The first CPU sets the vehicle speed limit Vlmt to the first vehicle speed Vs1. Step 625: The first CPU obtains a target acceleration Gtgt and a target steering angle θtgt such that the vehicle speed Vs does not exceed the limit vehicle speed Vlmt and the vehicle VA travels along the route. In the remote parking control, if the arrival flag Xrea is "0", movement control is executed, and if the arrival flag Xrea is "1", parking control is executed. The first CPU controls the power train actuator 46 and the brake actuator 48 so that the acceleration G coincides with the target acceleration Gtgt. Furthermore, the first CPU controls the steering motor 50 so that the steering angle θ coincides with the target steering angle θtgt. Step 630: The first CPU determines whether the arrival flag Xrea is "0".

[0074] If the arrival flag Xrea is "0", the first CPU determines "Yes" in step 630, and the process proceeds to step 635. In step 635, the first CPU determines whether the vehicle VA has reached the switching position.

[0075] If the vehicle VA has not yet reached the switching position, the first CPU determines "No" in step 635, and the process proceeds to step 640. In step 640, the first CPU transmits a display instruction including the progress data to the remote operation device 20. Thereafter, the process proceeds to step 695, and the first CPU temporarily ends this routine.

[0076] If the vehicle VA has reached the switching position when the process proceeds to step 635, the first CPU determines "Yes" in step 635 and executes steps 645 and 650. Step 645: The first CPU sets the arrival flag Xrea to “1”. Step 650: The first CPU transmits a display instruction including the obstacle data to the remote operation device 20. Thereafter, the process proceeds to step 695, where the first CPU temporarily ends this routine.

[0077] If the process proceeds to step 615 after the arrival flag Xrea is set to "1," the first CPU determines "No" in step 615, and the process proceeds to step 655. In step 655, the first CPU sets the vehicle speed limit Vlmt to the second vehicle speed Vs2. The second vehicle speed Vs2 is set to a value lower than the first vehicle speed Vs1. Thereafter, the process proceeds to step 625, and the first CPU acquires the target acceleration Gtgt and the target steering angle θtgt. As described above, when the arrival flag Xrea is "1," parking control is executed, and therefore the vehicle speed limit Vlmt (Vs2) for parking control is lower than the vehicle speed limit Vlmt (Vs1) for movement control. Thereafter, the process proceeds to step 630. Since the arrival flag Xrea is set to "1," the first CPU determines "No" in step 630, and the process proceeds to step 660.

[0078] In step 660, the first CPU determines whether the vehicle VA has reached the target space. If the vehicle VA has not yet reached the target space, the first CPU determines "No" in step 660 and executes step 650. Thereafter, the process proceeds to step 695, and the first CPU temporarily ends this routine.

[0079] If the vehicle VA has reached the target space when the process proceeds to step 660, the first CPU determines "Yes" in step 660 and executes steps 665 to 675.

[0080] Step 665: The first CPU sets the first execution flag Xexe1 to “0”. Step 670: The first CPU sets the arrival flag Xrea to "0". Step 675: The first CPU sends a completion instruction to the remote operation device 20. Thereafter, the process proceeds to step 695, where the first CPU temporarily ends this routine.

[0081] If the remote traveling control device 10 has not received a traveling instruction when the process proceeds to step 610, the first CPU determines "No" in step 610, and the process proceeds to step 680. In step 680, the first CPU determines whether the non-reception period T during which the remote traveling control device 10 has not received a traveling instruction is equal to or greater than the threshold time Tth.

[0082] If the no-reception period T is less than the threshold time Tth, the first CPU determines "No" in step 680, and the process proceeds to step 615. If the no-reception period T is equal to or greater than the threshold time Tth, the first CPU determines "Yes" in step 680, and the process proceeds to step 665.

[0083] <Remote Retrieval Control Routine> Among the processes of the remote leaving control routine shown in FIG. 7, the same processes as those of the remote parking control routine shown in FIG. 6 are given the same reference numerals, and the description thereof will be omitted. 7, and determines whether the second execution flag Xexe2 is "1" in step 705. If the second execution flag Xexe2 is "0," the first CPU determines "No" in step 705, and the process proceeds to step 795, where the first CPU temporarily ends this routine.

[0084] If the second execution flag Xexe2 is "1" ("Yes" in step 705), the remote traveling control device 10 receives a traveling instruction ("Yes" in step 610 in FIG. 7), and the arrival flag Xrea is "0" ("Yes" in step 615 in FIG. 7), the process proceeds to step 710. In step 710, the first CPU sets the vehicle speed limit Vlmt to the second vehicle speed Vs2. Thereafter, the first CPU executes steps 625 and 630 in FIG. 7. Note that in the remote leaving control, leaving control is executed when the arrival flag Xrea is "0", and movement control is executed when the arrival flag Xrea is "1".

[0085] If the arrival flag Xrea is "0" ("Yes" in step 630 in FIG. 7) and the vehicle VA has not yet reached the switching position ("No" in step 635 in FIG. 7), the process proceeds to step 715. In step 715, the first CPU transmits a display instruction including obstacle data to the remote control device 20. Thereafter, the process proceeds to step 795, and the first CPU temporarily ends this routine.

[0086] If the vehicle VA has reached the switching position ("Yes" in step 635 in FIG. 7), the first CPU executes step 645 and step 720 in FIG. Step 720: The first CPU transmits a display instruction including the progress data to the remote operation device 20. Thereafter, the process proceeds to step 795, where the first CPU temporarily ends this routine.

[0087] If the arrival flag Xrea is "1" when the process proceeds to step 615 in FIG. 7 ("No" in step 615 in FIG. 7), the process proceeds to step 725. In step 725, the first CPU sets the vehicle speed limit Vlmt to the first vehicle speed Vs1. Thereafter, the process proceeds to step 625 in FIG. 7. As described above, when the arrival flag Xrea is "1", movement control is executed, so the vehicle speed limit Vlmt (Vs2) for leaving control is lower than the vehicle speed limit Vlmt (Vs1) for movement control.

[0088] If the arrival flag Xrea is "1" when the process proceeds to step 630 in Fig. 7 ("Yes" in step 630 in Fig. 7), the process proceeds to step 660 in Fig. 7. If the vehicle VA has not yet arrived at the target space ("No" in step 660 in Fig. 7), the first CPU executes step 720. Thereafter, the process proceeds to step 795, and the first CPU temporarily ends this routine.

[0089] If the vehicle VA has reached the target space when the process proceeds to step 660 in FIG. 7 (“Yes” in step 660 in FIG. 7), the process proceeds to step 730, where the first CPU sets the second execution flag Xexe2 to “0.” Thereafter, the first CPU executes steps 670 and 675 in FIG. 7, the process proceeds to step 795, and the first CPU temporarily ends this routine.

[0090] When the process proceeds to step 610, if the remote traveling control device 10 has not received a traveling instruction ("No" in step 610 in FIG. 7), and if the non-reception period T is less than the threshold time Tth ("No" in step 680 in FIG. 7), the process proceeds to step 615 in FIG. 7. On the other hand, if the non-reception period T is equal to or greater than the threshold time Tth ("Yes" in step 680 in FIG. 7), the process proceeds to step 730.

[0091] <Screen display routine> The CPU of the control unit 60 of the remote control device 20 executes the routine shown in the flowchart of Fig. 8 every time a predetermined time elapses. Hereinafter, the CPU of the control unit 60 will be referred to as the "second CPU."

[0092] When an appropriate time arrives, the second CPU starts the process from step 800 in FIG. 8, and determines in step 805 whether the remote control device 20 has received a display instruction.

[0093] If the remote operation device 20 has received a display instruction, the second CPU determines "Yes" in step 805, and the process proceeds to step 810. In step 810, the second CPU determines whether the received display instruction includes progress data.

[0094] If the display instruction includes progress data, the second CPU determines "Yes" in step 810, and the process proceeds to step 815. In step 815, the second CPU causes the progress screen 200 to be displayed on the display device 66.

[0095] In detail, the second CPU generates an image to be displayed in the surrounding situation display area 205 of the progress screen 200 based on the travel area data, object data, and traveling direction data included in the progress data. Note that object images according to the type of object are pre-stored in the control unit 60, and the second CPU displays the object image according to the type of object in the surrounding situation display area 205. This eliminates the need for the remote traveling control device 10 to transmit object images with a large data capacity, thereby reducing the data capacity of communication between the remote traveling control device 10 and the remote operation device 20.

[0096] Furthermore, the second CPU identifies a traveled route 210a and an untraveled route 210b based on the route data and the position history data, and displays the traveled route 210a and the untraveled route 210b in the route progress display area 210. Furthermore, the second CPU displays the remaining distance in a remaining distance display area 220 based on the remaining distance data. Furthermore, the second CPU displays the vehicle speed Vs in a vehicle speed display area 225 based on the vehicle speed data.

[0097] Thereafter, the process proceeds to step 895, where the second CPU temporarily ends this routine.

[0098] If the display instruction does not include progress data (i.e., if the display instruction includes obstacle data), the second CPU determines "No" in step 810, and the process proceeds to step 820. In step 820, the second CPU causes the obstacle screen 300 to be displayed on the display device 66. Thereafter, the process proceeds to step 895, and the second CPU temporarily ends this routine.

[0099] In detail, the second CPU displays the overhead image included in the obstacle data in the close-distance situation display area 305 of the obstacle screen 300. If the obstacle data includes position data, the second CPU displays warning information superimposed on the overhead image toward the position of the obstacle represented by the position data. Specifically, if the distance D represented by the position data is greater than the second threshold distance D2th and equal to or less than the first threshold distance D1th, the second CPU displays the first warning display element 305a toward the obstacle, and if the distance D represented by the position data is equal to or less than the second threshold distance D2th, the second CPU displays the second warning display element 305b toward the obstacle.

[0100] Furthermore, the second CPU displays a traveling direction image included in the obstacle data in a traveling direction image display area 315. Furthermore, if the obstacle data includes distance data, the second CPU displays the minimum distance D represented by the distance data in an obstacle distance display area 320. Furthermore, the second CPU displays the shift position represented by the shift position data included in the obstacle data in a shift position display area 325.

[0101] If the remote control device 20 has not received a display instruction, the second CPU determines "No" in step 805, the process proceeds to step 895, and the second CPU temporarily ends this routine.

[0102] As described above, according to this embodiment, when remote traveling control is being executed, the progress screen 200 is displayed when the vehicle VA is traveling in a section where accessibility is low, and the obstacle screen 300 is displayed when the vehicle VA is traveling in a section where accessibility is high. This makes it possible to display a control screen that includes user request information that changes according to the progress of remote traveling control.

[0103] (First Modification) In the above embodiment, when the remote traveling control device 10 transmits a display instruction including obstacle data, it determines whether or not an obstacle exists whose distance D is less than the first threshold distance D1th based on sonar data, but it may also determine whether or not an obstacle exists whose distance D is less than the first threshold distance D1th based on the forward image and the rearward image.

[0104] (Second Modification) In the above embodiment, when transmitting a display instruction including obstacle data, the remote traveling control device 10 determines whether or not there is an obstacle whose distance D is equal to or less than the first threshold distance D1th, and transmits the obstacle data including position data indicating the distance D and the position of the obstacle whose distance D is equal to or less than the first threshold distance D1th, but is not limited to this. For example, the remote traveling control device 10 may transmit the obstacle data including sonar data, and the remote operation device 20 may determine whether or not there is an obstacle whose distance D is equal to or less than the first threshold distance D1th.

[0105] The remote driving control device 10 is applicable to vehicles such as engine vehicles, hybrid vehicles, plug-in hybrid vehicles, fuel cell vehicles, and electric vehicles. Furthermore, remote driving control is a type of automatic driving. [Explanation of symbols]

[0106] 10...remote driving control device, 20...remote operation device, 32...front camera, 34...rear camera, 36...sonar, 66...display device

Claims

1. A remote driving control device that performs remote driving control to drive a vehicle to a target space based on a driving instruction from a remote control device operated by a user outside the vehicle, The remote travel control device includes: When the remote traveling control is being executed, a control screen including information about the remote traveling control is displayed on a display device of the remote operation device; changing the content displayed on the control screen according to the progress of the remote travel control; A remote driving control device configured as above.

2. The remote travel control device according to claim 1, a route to the target space on which the vehicle travels is divided into a first section in which the vehicle is less likely to approach an obstacle and a second section in which the vehicle is more likely to approach the obstacle, with a switching position as a dividing point; The remote travel control device includes: When the vehicle is traveling in the first section, a first control screen is displayed on the display device as the control screen; When the vehicle is traveling in the second section, a second control screen having content different from that of the first control screen is displayed on the display device as the control screen. A remote driving control device configured as above.

3. The remote travel control device according to claim 2, The remote travel control device includes: When the vehicle is traveling in the first section, progress information indicating the progress of the remote traveling control is displayed on the first control screen; When the vehicle is traveling in the second section, warning information for issuing a warning about the obstacle whose distance to the vehicle is equal to or shorter than a threshold distance is displayed on the second control screen. A remote driving control device configured as above.

4. The remote travel control device according to claim 3, The remote traveling control device is configured to display, as the progress information, at least one of route information that divides a route to the target space into a route that has been traveled and a route that has not been traveled, and remaining distance information that indicates a remaining distance to the target space, on the first control screen. Remote driving control device.

5. The remote travel control device according to claim 3, The remote traveling control device is configured to display the warning information on the second control screen by superimposing it on an overhead image of the periphery of the vehicle. Remote driving control device.

6. The remote travel control device according to claim 2, The remote travel control device includes: executing, as the remote travel control, movement control for moving the vehicle to the vicinity of the target space until the vehicle reaches the switching position; After the vehicle reaches the switching position, parking control for parking the vehicle in the target space is executed as the remote traveling control, When the movement control is being executed, it is determined that the vehicle is traveling in the first section; When the parking control is being executed, it is determined that the vehicle is traveling in the second section. A remote driving control device configured as above.

7. The remote travel control device according to claim 6, the remote traveling control device is configured to, when executing the remote traveling control, cause the vehicle to travel so that a vehicle speed representing a speed of the vehicle does not exceed an upper limit vehicle speed; The upper limit vehicle speed of the parking control is preset to a value lower than the upper limit vehicle speed of the movement control. Remote driving control device.

8. The remote travel control device according to claim 2, The remote travel control device includes: From the start of the remote traveling control until the vehicle reaches the switching position, a parking exit control for causing the vehicle to exit from the parking space is executed as the remote traveling control, After the vehicle reaches the switching position, a movement control for moving the vehicle to the target space is executed as the remote traveling control; When the leaving control is being executed, it is determined that the vehicle is traveling in the second section, When the movement control is being executed, it is determined that the vehicle is traveling in the first section. A remote driving control device configured as above.

9. The remote control device according to claim 8, the remote traveling control device is configured to, when executing the remote traveling control, cause the vehicle to travel so that a vehicle speed representing a speed of the vehicle does not exceed an upper limit vehicle speed; The upper limit vehicle speed of the leaving control is preset to a value lower than the upper limit vehicle speed of the movement control. Remote driving control device.

10. A remote control device that transmits a driving instruction to a vehicle when operated by a user outside the vehicle, the vehicle executes remote driving control to drive to the target space in accordance with the driving instruction; The remote control device is When the vehicle is performing the remote driving control, a control screen including information about the remote driving control is displayed on a display device of the remote operation device; changing the content displayed on the control screen according to the progress of the remote travel control; A remote control device configured as follows.

Images