Remote travel control device and remote operation device

Abstract

The invention relates to a remote travel control device and a remote operation device. Provided is a remote travel control device capable of displaying information that a user wants to know on a control screen. The remote travel control device displays a control screen including information relating to the remote travel control on a display device of a remote operation device when the remote travel control for causing the vehicle to travel to a target space on the basis of a travel instruction from the remote operation device operated by a user outside the vehicle is being executed. The content displayed on the control screen is changed in accordance with the progress of the remote travel control.

Description

Technical Field

[0001] The present invention relates to a remote driving control device that performs remote driving control of a vehicle to a target space based on driving instructions from a remote operating device operated by a user outside the vehicle, and a remote operating device operated by a user outside the vehicle to drive the vehicle to the target space. Background Technology

[0002] A remote driving control device for performing remote driving control is known in the past. For example, the remote driving control device described in Patent Document 1 (hereinafter referred to as the "conventional device") performs remote parking control by having a user outside the vehicle operate the remote operating device to park the vehicle in a target space. When performing remote parking control, the conventional device displays a remote parking screen as a control screen on the remote operating device. The remote parking screen includes the blind spot position relative to the vehicle on the opposite side of the user and includes an image along the line of sight from the user's viewpoint of the vehicle.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2021-160499

[0006] The probability of approaching an obstacle changes as remote driving control progresses. The information the user wants to know related to remote driving control (hereinafter referred to as "requested information") changes based on the probability of approach. In other words, the requested information changes as remote driving control progresses.

[0007] For example, in remote parking control, the probability of approaching the vehicle during the parking phase, when it is actually parked in the parking space, is higher than the probability of approaching it during the initial phase of remote parking control. For instance, when the probability of approaching is high, the user tends to want to know the positional relationship between the vehicle and obstacles; when the probability of approaching is low, the user tends to want to know the progress of remote driving control.

[0008] When both high-probability and low-probability request messages are displayed on the control panel simultaneously, the screen becomes cluttered, increasing the likelihood that users will find it cumbersome to navigate. Conversely, if only high-probability request messages are displayed, users will be unaware of low-probability request messages. Summary of the Invention

[0009] This invention was made to address the aforementioned problems. Specifically, one objective of this invention is to provide a remote driving control device that displays information desired by the user on a control screen.

[0010] The remote driving control device of the present invention (hereinafter referred to as "the device of the present invention") performs remote driving control (steps 600 to 695, steps 700 to 795) to drive a vehicle to a target space based on a driving instruction from a remote operating device (20) operated by a user outside the vehicle. The remote driving control device is configured to: display a control screen (200, 300) containing information related to the remote driving control on the display device (66) of the remote operating device when the remote driving control is being performed (step 605 is "yes", step 705 is "yes") (steps 640, 650, 715, 720); and change the content displayed on the control screen according to the progress of the remote driving control (steps 640, 650, 715, 720, 815, 820).

[0011] According to the device of the present invention, the content displayed on the control screen changes according to the progress of remote driving control. Therefore, the device of the present invention enables the display of a control screen containing user request information that changes according to the progress of remote driving control on a display device. Thus, a remote driving control device can be provided that avoids a cluttered control screen and displays the information the user wants to know on the control screen.

[0012] According to one aspect of the device of the present invention, the route traveled by the vehicle to the target space is divided into a first interval with a low probability of approaching an obstacle and a second interval with a high probability of approaching the obstacle, using the switching position as a dividing point. The remote driving control device is configured to: when the vehicle is traveling in the first interval (step 635 is "No"), Figure 7 If step 630 is "No", the first control screen is displayed on the display device as the control screen (steps 640 and 720); and if the vehicle is traveling in the second section (if step 630 is "No"), the first control screen is displayed on the display device (steps 640 and 720); and if the vehicle is traveling in the second section (if step 630 is "No"), Figure 7 If step 635 is "No", a second control screen with content different from the first control screen will be displayed on the display device as the control screen (steps 650 and 715).

[0013] According to this solution, when the vehicle is approaching a first section with a low probability of approaching, a first control screen is displayed; when the vehicle is approaching a second section with a high probability of approaching, a second control screen is displayed. Thus, the control screen can be changed based on the probability of approach.

[0014] In the above scheme, the remote driving control device is configured to: when the vehicle is traveling in the first section (step 635 is "No"), Figure 7 If step 630 is "No", progress information (210, 220) indicating the progress of the remote driving control is displayed on the first control screen (200) (step 815); and if the vehicle is traveling in the second section (if step 630 is "No"), the progress information (210, 220) indicating the progress of the remote driving control is displayed on the first control screen (200) (step 815); and if the vehicle is traveling in the second section (if step 630 is "No"), Figure 7 If step 635 is "No", alarm information (305a, 305b) that alerts the vehicle to obstacles whose distance from the vehicle is less than or equal to a threshold distance is displayed on the second control screen (300) (step 820).

[0015] When the probability of proximity is low, users tend to want to know the progress of remote driving control; when the probability of proximity is high, users tend to want to know the location of obstacles. According to this solution, progress information is displayed when the vehicle is traveling in the first zone, and alarm information is displayed when the vehicle is traveling in the second zone. Thus, by displaying the information the user wants to know based on the probability of proximity, the user can obtain the information they desire.

[0016] In the above scheme, the remote driving control device is configured to display at least one of route information (210) and remaining distance information (220) as the progress information on the first control screen, wherein the route information (210) is information obtained by dividing the route to the target space into the route that has been traveled and the route that has not been traveled, and the remaining distance information (220) represents the remaining distance to the target space.

[0017] Users can know at least one of the route information and the remaining distance information while the vehicle is traveling in the first section, thus being able to know the progress of remote driving control.

[0018] In the above scheme, the remote driving control device is configured to display the alarm information (305a, 305b) overlaid with an overhead view of the vehicle's surroundings on the second control screen.

[0019] Users can be aware of the location of obstacles around the vehicle while it is traveling in the second zone.

[0020] In the above scheme, the remote driving control device is configured to: until the vehicle reaches the switching position (step 635 is "No"), execute movement control to move the vehicle to the vicinity of the target space as the remote driving control (step 625); after the vehicle reaches the switching position (step 630 is "No"), execute parking control to park the vehicle in the target space as the remote driving control (step 625); while the movement control is being executed, determine that the vehicle is driving in the first interval (step 640); and while the parking control is being executed, determine that the vehicle is driving in the second interval (step 650).

[0021] Generally, the probability of approaching tends to decrease when motion control is in effect, and tends to increase when parking control is in effect. According to this scheme, when motion control is in effect, the vehicle is determined to be traveling in the first zone; when parking control is in effect, the vehicle is determined to be traveling in the second zone. Therefore, this device can accurately determine the probability of approaching based on the progress of remote driving control.

[0022] In the above scheme, the remote driving control device is configured to: when the remote driving control is being executed, cause the vehicle to travel at a speed not exceeding the upper limit speed (step 625), and the upper limit speed of the parking control is preset to a value lower than the upper limit speed of the movement control (step 655).

[0023] According to this scheme, the upper speed limit for parking control is set lower than the upper speed limit for motion control. Therefore, in parking control, the vehicle can be stopped immediately when the probability of contact with an obstacle increases. Furthermore, the vehicle's movement time in motion control can be shortened.

[0024] In the above scheme, the remote driving control device is configured to operate from the start of the remote driving control until the vehicle reaches the switching position. Figure 7 If step 635 is "No", then execute the exit control to make the vehicle leave the parking space as the remote driving control. Figure 7 Step 625); after the vehicle arrives at the switching position ( Figure 7 If step 630 is "No", then execute the movement control to move the vehicle to the target space as the remote driving control. Figure 7Step 625); while the outbound control is being executed, it is determined that the vehicle is traveling in the second section (step 715); and while the movement control is being executed, it is determined that the vehicle is traveling in the first section (step 720).

[0025] Generally, the probability of approaching tends to decrease when movement control is in progress, and tends to increase when exit control is in progress. According to this scheme, when movement control is in progress, the vehicle is determined to be traveling in the first zone; when exit control is in progress, the vehicle is determined to be traveling in the second zone. Therefore, this device can accurately determine the probability of approaching based on the progress of remote driving control.

[0026] In the above scheme, the remote driving control device is configured to: when performing the remote driving control, cause the vehicle to travel at a speed not exceeding the upper limit of the vehicle speed, representing the vehicle's speed. Figure 7 In step 625), the upper limit speed of the outbound control is preset to a value lower than the upper limit speed of the movement control (step 710).

[0027] According to this scheme, the upper limit speed for outbound control is set to a value lower than the upper limit speed for movement control. Therefore, in outbound control, the vehicle can be stopped immediately when the probability of contact with an obstacle increases. Furthermore, the vehicle's movement time in movement control can be shortened.

[0028] The remote operation device (20) of the present invention sends driving instructions to the vehicle (440, 465) when operated by a user outside the vehicle.

[0029] The vehicle performs remote driving control to travel to the target space according to the driving instructions (steps 600 to 695, steps 700 to 795).

[0030] The remote operating device is configured to: when the vehicle is performing the remote driving control, display a control screen (200, 300) containing information related to the remote driving control on the display device of the remote operating device, and change the content displayed on the control screen according to the progress of the remote driving control (steps 815, 820).

[0031] According to the remote operation device of the present invention, the content displayed on the control screen is changed according to the progress of remote driving control. Thus, the remote operation device enables the display of a control screen containing user request information that changes according to the progress of remote driving control on a display device. Attached Figure Description

[0032] Figure 1 This is a schematic system configuration diagram of a vehicle control system according to an embodiment of the present invention.

[0033] Figure 2 It is displayed in Figure 1 An explanatory diagram of the progress screen of the remote operation device shown.

[0034] Figure 3 It is displayed in Figure 1 An explanatory diagram of the obstacle screen on the display device of the remote operation device shown.

[0035] Figure 4 This is a sequence diagram illustrating an operational example of a vehicle control system according to an embodiment of the present invention.

[0036] Figure 5 yes Figure 1 The flowchart shown illustrates the start determination routine executed by the CPU (Central Processing Unit) of the ECU (Electronic Control Unit).

[0037] Figure 6 yes Figure 1 The flowchart shown is of the remote parking control routine executed by the CPU of the ECU.

[0038] Figure 7 yes Figure 1 The flowchart shown is of the remote outbound control routine executed by the CPU of the ECU.

[0039] Figure 8 yes Figure 1 The flowchart shown is of the screen display routine executed by the CPU of the control unit.

[0040] Explanation of reference numerals in the attached figures:

[0041] 10: Remote driving control device; 20: Remote operation device; 32: Front camera; 34: Rear camera; 36: Sonar; 66: Display device. Detailed Implementation

[0042] like Figure 1 As shown, the vehicle control system of this embodiment includes a "remote driving control device 10 for a vehicle (VA)" and a remote operation device 20. The remote driving control device 10 and the remote operation device 20 are connected in a manner that enables communication via a network (NW).

[0043] The remote driving control device 10 is equipped with Figure 1The components shown are as follows. In this specification, "ECU30" is an electronic control device with a microcomputer as its main component. ECU30 is also referred to as a control unit, controller, or computer. The microcomputer includes a CPU (processor), ROM (Read Only Memory), RAM (Random Access Memory), and interfaces. The functions implemented by ECU30 can also be implemented by multiple ECUs.

[0044] The front camera 32 acquires a frontal image by photographing the scenery in front of the vehicle VA. The rear camera 34 acquires a rearward image by photographing the scenery behind the vehicle VA. The sonar 36 acquires sonar data related to the position of objects existing around the vehicle VA relative to the vehicle VA. The ECU 30 acquires the frontal and rearward images from the front camera 32 and the rear camera 34, respectively, and acquires sonar data from the sonar 36. It should be noted that the ECU 30 identifies objects located around the vehicle VA based on the frontal image, the rearward image, and the sonar data.

[0045] The wheel speed sensor 38 generates a pulse signal every time the wheels of vehicle VA rotate a predetermined angle. The ECU 30 measures the number of pulse signals generated by the wheel speed sensor 38 per unit time and obtains the vehicle speed Vs, which represents the speed of vehicle VA, based on the measured number of pulse signals.

[0046] The GNSS (Global Navigation Satellite System) receiver 40 receives signals from multiple artificial satellites and determines the current location (latitude and longitude) of the vehicle VA based on the received signals. The parking lot map data storage unit 42 stores parking lot map data. The communication interface (I / F) 44 is an interface for connecting the remote driving control device 10 to the network NW.

[0047] The power transmission actuator 46 modifies the driving force generated by the drive unit (e.g., internal combustion engine and / or electric motor) of the vehicle VA. The brake actuator 48 modifies the braking force applied to the vehicle VA. The steering motor 50 is assembled into the steering mechanism 52. The steering mechanism 52 is a mechanism for turning the steering wheels according to the operation of the steering wheel. The steering motor 50, according to the instruction from the ECU 30, causes the steering mechanism 52 to generate an automatic steering torque for changing the steering wheel angle.

[0048] The remote control device 20 is a device that can be operated even when the user is outside the vehicle; an example is a smartphone. The remote control device 20 has... Figure 1 The constituent elements shown.

[0049] The control unit 60 includes a CPU (processor), ROM, RAM, and interfaces. The GNSS receiver 62 and communication I / F 64 are identical to those of the GNSS receiver 40 and communication I / F 44, respectively. Their descriptions are omitted.

[0050] The display device 66 is a touch panel type display that allows a user to input to the remote operation device 20 by touching the display device 66. If the display device 66 is not a touch panel type, the remote operation device 20 has an input device (not shown).

[0051] (Job Summary)

[0052] When a user outside the vehicle operates the remote operation device 20, the remote driving control device 10 executes the driving instruction sent by the remote operation device 20 to "automatically drive the vehicle VA to the target space". For example, as a remote driving control, smart summon and reverse summon are known.

[0053] Smart Summon is a type of remote parking control. In Smart Summon, the vehicle VA automatically moves from its parking space to the target space (the current location of the remote control device 20 determined by the GNSS receiver 62 or a user-specified location). Reverse Summon is a type of remote parking control. In Reverse Summon, when the user gets out of the car at the parking lot entrance and operates the remote control device 20, the vehicle VA parks in the user-specified parking space (target space).

[0054] During remote driving control, the remote operating device 20 displays a control screen on the display device 66. The control screen includes an operating area (see reference). Figure 2 Figure reference numerals 215 and Figure 3 (Ref. 310 in the accompanying drawings.) When the user performs a specified operation on the operating area, the remote operating device 20 sends a driving instruction to the remote driving control device 10. Upon receiving the driving instruction, the remote driving control device 10 enables the vehicle VA to drive automatically. It should be noted that during automatic driving, the remote driving control device 10 controls the power transmission actuator 46, the brake actuator 48, and the steering motor 50 to make the vehicle VA travel along a route to the target space. Moreover, the remote driving control device 10 identifies objects based on the front image (and / or rear image) and sonar data, and controls the power transmission actuator 46, the brake actuator 48, and the steering motor 50 to avoid contact with objects.

[0055] The "approach probability of vehicle VA approaching an obstacle" changes according to the progress of remote driving control. In remote exit control, the approach probability is high when vehicle VA actually exits the parking space, and low when vehicle VA finishes exiting and moves towards the target space. In remote parking control, the approach probability is low when vehicle VA moves to the vicinity of the target space, and high when vehicle VA actually parks in the target space. The interval with low approach probability is called the first interval, and the interval with high approach probability is called the second interval. The position where such control is switched is called the switching position. The route to the target space is divided into the first interval and the second interval by the switching position.

[0056] When the probability of proximity changes, the information a user wants to know also changes. More specifically, when the probability of proximity is low, users tend to want to know the progress of remote driving control; when the probability of proximity is high, users tend to want to know the location of obstacles.

[0057] Therefore, in this embodiment, the remote driving control device 10 changes the content displayed on the control screen according to the progress of remote driving control. Specifically, when the vehicle VA is traveling in the first section, the screen displays... Figure 2 The progress screen (first control screen) 200 shown serves as the control screen, displayed when vehicle VA is traveling in the second section. Figure 3 The obstacle screen (second control screen) 300 shown is used as the control screen. The progress screen 200 displays at least "progress information indicating the progress of remote driving control," and the obstacle screen 300 displays "alarm information warning of obstacles whose distance D from the vehicle VA is less than or equal to a first threshold distance D1th." According to this embodiment, the control screen displays information that the user wants to know.

[0058] <Progress Screen 200>

[0059] Reference Figure 2 The progress screen 200 is described below. The progress screen 200 includes a surrounding conditions display area 205, a route progress display area 210, an operation area 215, a remaining distance display area 220, and a vehicle speed display area 225.

[0060] The surrounding conditions display area 205 displays the direction of travel of vehicle VA, the driving area of ​​vehicle VA, and the objects existing around vehicle VA.

[0061] The route progress display area 210 displays the route 210a that the vehicle VA has traveled and the route 210b that the vehicle VA has not yet traveled. Users can know the progress of remote driving control by knowing the route 210a that has been traveled and the route 210b that has not yet been traveled.

[0062] The operating area 215 is the area where the user performs prescribed operations to send driving instructions from the remote operating device 20.

[0063] The remaining distance to the target space is displayed in the remaining distance display area 220. Users can track the progress of remote driving control by knowing the remaining distance. The vehicle speed Vs is displayed in the speed display area 225.

[0064] <Obstacle Screen 300>

[0065] Reference Figure 3 The obstacle image 300 is explained.

[0066] The aforementioned alarm information shall be displayed at least in the obstacle display 300. The obstacle display 300 includes a proximity status display area 305, an operation area 310, a travel direction image display area 315, an obstacle distance display area 320, and a gear shift position display area 325.

[0067] In the absence of obstacles at a distance D less than or equal to a first threshold distance D1th, an overhead view of the surrounding scenery from directly above the vehicle VA is displayed in the close-up status display area 305. In the presence of obstacles at a distance D less than or equal to the first threshold distance D1th, the aforementioned alarm information is displayed in the close-up status display area 305, overlapping the overhead view. More specifically, for obstacles at a distance D greater than a second threshold distance D2th (smaller than the first threshold distance D1th) but less than or equal to the first threshold distance D1th, a first alarm display element 305a is displayed in the direction of the obstacle. For obstacles at a distance D less than or equal to the second threshold distance D2th, a second alarm display element 305b is displayed in the direction of the obstacle. It should be noted that the second alarm display element 305b is displayed in a manner that is more likely to draw the user's attention than the first alarm display element 305a.

[0068] Operation area 310 is the same as operation area 215 described above, therefore, its description is omitted.

[0069] The image of the vehicle VA's direction of travel is displayed in the travel direction image display area 315. When the shift position is either the forward position or the neutral position, the forward image is displayed in the travel direction image display area 315; when the shift position is the reverse position, the rear image is displayed in the travel direction image display area 315.

[0070] The distance D of the obstacle closest to vehicle VA is displayed in obstacle distance display area 320.

[0071] The current shift position is displayed in the shift position display area 325.

[0072] <Work Example>

[0073] Reference Figure 4 An example of the operation of the vehicle control system of this embodiment will be described.

[0074] When the user operates the remote operating device 20 to start the remote driving control application (405), they select whether to perform remote exit control or remote parking control (410). In this example, remote parking control is selected. Then, if needed, the user specifies the target space, and when the user operates the start button (not shown) displayed on the display device 66 (415), the remote operating device 20 sends a start instruction to the remote driving control device 10 (420). The start instruction sent in the case of remote exit control is called the "exit start instruction," and the start instruction sent in the case of remote parking control is called the "parking start instruction." The start instruction contains data that determines the target space.

[0075] Upon receiving a start instruction, the remote driving control device 10 sets a route to the target space by referring to parking lot map data. Furthermore, after receiving the start instruction, the remote driving control device 10 sends a display instruction to the remote operation device 20 (425a, 425b, 455a, 455b) every time a predetermined time has elapsed.

[0076] In the case of remote parking control, vehicle VA travels in the first interval immediately after the start of remote parking control, thus displaying progress screen 200. Therefore, the remote driving control device 10 sends a display instruction containing the progress data required for displaying progress screen 200. The progress data includes the following data.

[0077] • Data representing the driving area of ​​vehicle VA

[0078] • Object data representing the position and type of an object determined based on the front and rear images.

[0079] • This represents the travel direction data, determined based on the shift position and the vehicle's (VA) steering angle.

[0080] • This indicates the route data set at the start of remote driving control.

[0081] • Location history data representing the current location of the vehicle VA as determined by the GNSS receiver 40.

[0082] • Remaining distance data representing the remaining distance

[0083] • Vehicle speed data representing vehicle speed Vs

[0084] In the case of remote exit control, vehicle VA travels in the second section immediately after the start of remote exit control, thus displaying obstacle screen 300. Therefore, the remote driving control device 10 sends a display instruction containing obstacle data required for displaying obstacle screen 300. The obstacle data includes the following data.

[0085] • Aerial view

[0086] • This represents the location data of the distance D and position of obstacles whose distance D is less than or equal to the first threshold distance D1th.

[0087] • Images representing the direction of travel of vehicle VA in the front and rear images.

[0088] • Represents the distance data of the minimum distance D

[0089] • Shift position data indicating the shift position

[0090] The remote driving control device 10 includes a left camera and a right camera (not shown). The left camera captures a left image by photographing the scenery to the left of the vehicle VA. The right camera captures a left image by photographing the scenery to the right of the vehicle VA. The remote driving control device 10 generates an overhead view based on the front image, rear image, left image, and right image.

[0091] If no obstacle exists at a distance D less than or equal to a first threshold distance D1th, the obstacle data does not include position or distance data. The first threshold distance D1th is set to a value smaller than the distance at which the sonar 36 can detect the object. The remote driving control device 10 determines the presence of an obstacle at a distance D less than or equal to the first threshold distance D1th based on the sonar data.

[0092] When the remote operating device 20 receives a display instruction containing the aforementioned progress data, it displays the progress screen 200 on the display device 66 (430). When the user performs a specified operation on the operation area 215 of the progress screen 200 (435), the remote operating device 20 sends a driving instruction (440).

[0093] Upon receiving a driving instruction, the remote driving control unit 10 executes automatic driving control (445) to automatically drive the vehicle VA along the route. In remote parking control, the remote driving control unit 10 performs movement control to move the vehicle VA to the vicinity of the target space as automatic driving control before the vehicle VA reaches the switching position. The switching position is the position where the vehicle switches from movement control to parking control in remote parking control, and the position where the vehicle switches from exit control to movement control in remote exit control. The switching position is set when the route to the target space is set.

[0094] After the vehicle VA reaches the switching position at time point (450), each time a predetermined time elapses, the remote driving control device 10 sends a display instruction containing the aforementioned obstacle data to the remote operation device 20 (455a, 455b). Upon receiving the display instruction containing obstacle data, the remote operation device 20 ends the display of the progress screen 200 and displays the obstacle screen 300 (460). When the user performs a predetermined operation on the operation area 310 of the obstacle screen 300, the remote operation device 20 sends a driving instruction to the remote driving control device 10 (465). Upon receiving the driving instruction, the remote driving control device 10 performs parking control to actually park the vehicle VA in the target space as automatic driving control (470).

[0095] When the vehicle VA reaches the target space (475), the remote driving control device 10 sends a completion instruction to the remote operation device 20 (480). Upon receiving the completion instruction, the remote operation device 20 ends the display of the obstacle screen 300 and displays a completion screen (not shown) to notify the user that the vehicle VA has reached the target space (485).

[0096] In the above manner, when remote parking control is used as remote driving control, a progress screen 200 is displayed on the display device 66 of the remote operating device 20 as a control screen until the vehicle VA reaches the switching position. After the vehicle VA reaches the switching position, an obstacle screen 300 is displayed on the display device 66 of the remote operating device 20 as a control screen. Thus, the user can observe the control screen and obtain the desired information based on the progress of the remote driving control.

[0097] (Specific tasks)

[0098] The CPU of the ECU30 of the remote driving control device 10 executes the command every time a predetermined time has elapsed. Figures 5 to 7 The example is illustrated in the flowchart. The CPU of ECU30 will be referred to as the "first CPU" below.

[0099] <Start the decision routine>

[0100] When the appropriate time arrives, the first CPU... Figure 5 The process begins at step 500, and in step 505, it is determined whether the first execution flag Xexe1 is "0" and the second execution flag Xexe2 is "0".

[0101] When remote parking control begins, the first execution flag Xexe1 is set to "1", and when remote parking control ends, the first execution flag Xexe1 is set to "0". When remote exit control begins, the second execution flag Xexe2 is set to "1", and when remote exit control ends, the second execution flag Xexe2 is set to "0". Furthermore, in the initialization routine, both the first execution flag Xexe1 and the second execution flag Xexe2 are set to "0". The initialization routine is executed by the first CPU when the ignition key switch (not shown) of the vehicle VA is changed from the off position to the on position.

[0102] If both the first execution flag Xexe1 and the second execution flag Xexe2 are "0", the first CPU determines "yes" in step 505 and proceeds to step 510. In step 510, the CPU determines whether the remote driving control device 10 has received a parking start instruction.

[0103] When the remote driving control device 10 receives a parking start instruction, the first CPU determines "yes" in step 510 and executes steps 515 to 525.

[0104] Step 515: The first CPU sets the first execution flag Xexe1 to "1".

[0105] Step 520: The first CPU sets the arrival flag Xrea to "0".

[0106] When the vehicle VA arrives at the switching position, the arrival flag Xrea is set to "1". When remote driving control is initiated, the arrival flag Xrea is set to "0". The arrival flag Xrea is set to "0" in the initialization routine.

[0107] Step 525: The first CPU sets the route up to the target space.

[0108] Then, the process proceeds to step 595, and the first CPU temporarily terminates this routine.

[0109] If the remote driving control device 10 does not receive a parking start instruction when proceeding to step 510, the first CPU determines "no" in step 510 and proceeds to step 530. In step 530, the first CPU determines whether the remote driving control device 10 has received a parking start instruction.

[0110] When the remote driving control device 10 receives a departure start instruction, the first CPU determines "yes" in step 530 and proceeds to step 535. In step 535, the first CPU sets the second execution flag Xexe2 to "1". Afterwards, the first CPU executes steps 520 and 525, proceeds to step 595, and the first CPU temporarily terminates this routine.

[0111] If the remote driving control device 10 does not receive a departure start instruction, the first CPU determines "no" in step 530, proceeds to step 595, and the first CPU temporarily terminates this routine.

[0112] <Remote Parking Control Routine>

[0113] When the appropriate time arrives, the first CPU... Figure 6 The process begins at step 600. In step 605, it is determined whether the first execution flag Xexe1 is "1". If the first execution flag Xexe1 is "0", the first CPU determines "no" in step 605, and the process proceeds to step 695, whereby the first CPU temporarily terminates the current routine.

[0114] If the first execution flag Xexe1 is "1", the first CPU determines "yes" in step 605 and proceeds to step 610. In step 610, the first CPU determines whether the remote driving control device 10 has received a driving instruction.

[0115] When the remote driving control device 10 receives a driving instruction, the first CPU determines "yes" in step 610 and proceeds to step 615. In step 615, the first CPU determines whether the arrival flag Xrea is "0".

[0116] If the arrival flag Xrea is "0", the first CPU determines "yes" in step 615 and executes steps 620 to 630.

[0117] Step 620: The first CPU sets the vehicle speed limit Vlmt to the first vehicle speed Vs1.

[0118] Step 625: The first CPU obtains the target acceleration Gtgt and the target rudder angle θtgt, such that the vehicle speed Vs does not exceed the limit vehicle speed Vlmt and the vehicle VA is traveling along the route.

[0119] It should be noted that in remote parking control, movement control is executed when the arrival sign Xrea is "0", and parking control is executed when the arrival sign Xrea is "1".

[0120] The first CPU controls the power transmission actuator 46 and the brake actuator 48 in a manner that makes the acceleration G match the target acceleration Gtgt. Furthermore, the first CPU controls the steering motor 50 in a manner that makes the rudder angle θ match the target rudder angle θtgt.

[0121] Step 630: The first CPU determines whether the arrival flag Xrea is "0".

[0122] If the arrival flag Xrea is "0", the first CPU determines "yes" in step 630 and proceeds to step 635. In step 635, the first CPU determines whether the vehicle VA has arrived at the switching position.

[0123] If the vehicle VA has not yet reached the switching position, the first CPU determines "No" in step 635 and proceeds to step 640. In step 640, the first CPU sends a display indication containing progress data to the remote operating device 20. Afterwards, the process proceeds to step 695, and the first CPU temporarily terminates this routine.

[0124] If the vehicle VA has reached the switching position when entering step 635, the first CPU determines "yes" in step 635 and executes steps 645 and 650.

[0125] Step 645: The first CPU sets the arrival flag Xrea to "1".

[0126] Step 650: The first CPU sends a display instruction containing obstacle data to the remote operating device 20.

[0127] After that, the process proceeds to step 695, and the first CPU temporarily terminates this routine.

[0128] If the process proceeds to step 615 after the arrival marker Xrea is set to "1", the first CPU determines "No" in step 615 and proceeds to step 655. In step 655, the first CPU sets the speed limit Vlmt to the second speed Vs2. The second speed Vs2 is set to a value lower than the first speed Vs1. Then, the process proceeds to step 625, where the first CPU acquires the target acceleration Gtgt and the target steering angle θtgt. As described above, parking control is performed when the arrival marker Xrea is "1", therefore, the speed limit Vlmt (Vs2) for parking control is lower than the speed limit Vlmt (Vs1) for movement control. Then, the process proceeds to step 630. Since the arrival marker Xrea is set to "1", the first CPU determines "No" in step 630 and proceeds to step 660.

[0129] In step 660, the first CPU determines whether the vehicle VA has reached the target space. If the vehicle VA has not yet reached the target space, the first CPU determines "no" in step 660 and executes step 650. Afterwards, the process proceeds to step 695, and the first CPU temporarily terminates this routine.

[0130] When the vehicle VA arrives at the target space during step 660, the first CPU determines "yes" in step 660 and executes steps 665 to 675.

[0131] Step 665: The first CPU sets the first execution flag Xexe1 to "0".

[0132] Step 670: The first CPU sets the arrival flag Xrea to "0".

[0133] Step 675: The first CPU sends a completion instruction to the remote operating device 20.

[0134] After that, the process proceeds to step 695, and the first CPU temporarily terminates this routine.

[0135] If the remote driving control device 10 does not receive a driving instruction when proceeding to step 610, the first CPU determines "no" in step 610 and proceeds to step 680. In step 680, the first CPU determines whether the non-reception period T during which the remote driving control device 10 does not receive a driving instruction is greater than or equal to the threshold time Tth.

[0136] If the non-reception period T is less than the threshold time Tth, the first CPU determines "No" in step 680 and proceeds to step 615. If the non-reception period T is greater than or equal to the threshold time Tth, the first CPU determines "Yes" in step 680 and proceeds to step 665.

[0137] <Remote Outbound Control Routine>

[0138] right Figure 7 The processing of the remote outbound control routine shown is related to... Figure 6 The remote parking control routine shown is processed by the same reference numerals as those in the accompanying drawings, and its description is omitted.

[0139] When the appropriate time arrives, the first CPU... Figure 7 The process begins at step 700. In step 705, it is determined whether the second execution flag Xexe2 is "1". If the second execution flag Xexe2 is "0", the first CPU determines "no" in step 705, and the process proceeds to step 795, whereby the first CPU temporarily terminates the current routine.

[0140] When the second execution flag Xexe2 is "1" (step 705 is "Yes"), the remote driving control device 10 receives the driving instruction ( Figure 7 Step 610 is "Yes", and the Xrea flag is "0" when it is reached ( Figure 7 If step 615 is "Yes", the process proceeds to step 710. In step 710, the first CPU sets the vehicle speed limit Vlmt to the second vehicle speed Vs2. Afterwards, the first CPU executes... Figure 7 Steps 625 and 630. It should be noted that in remote outbound control, outbound control is performed when the arrival flag Xrea is "0", and movement control is performed when the arrival flag Xrea is "1".

[0141] When the arrival marker Xrea is "0" ( Figure 7 If step 630 is "Yes", and vehicle VA has not yet reached the switching position ( Figure 7 If step 635 is "No", the process proceeds to step 715. In step 715, the first CPU sends a display instruction containing obstacle data to the remote operating device 20. Afterward, the process proceeds to step 795, and the first CPU temporarily terminates this routine.

[0142] When vehicle VA arrives at the switching position ( Figure 7 Step 635 is "Yes"), the first CPU executes Figure 7 Steps 645 and 720.

[0143] Step 720: The first CPU sends a display instruction containing progress data to the remote operating device 20.

[0144] Then, the process proceeds to step 795, and the first CPU temporarily terminates this routine.

[0145] In processing entry Figure 7 In step 615, when the flag Xrea is "1" ( Figure 7 If step 615 is "No", the process proceeds to step 725. In step 725, the first CPU sets the vehicle speed limit Vlmt to the first vehicle speed Vs1. Afterwards, the process continues... Figure 7 Step 625. As described above, movement control is performed when the arrival flag Xrea is "1". Therefore, the speed limit Vlmt (Vs2) for exit control is lower than the speed limit Vlmt (Vs1) for movement control.

[0146] In processing entry Figure 7 In step 630, when the flag Xrea is "1" ( Figure 7 Step 630 is "No"), proceed with processing Figure 7 Step 660. Before the vehicle VA has reached the target space ( Figure 7 If step 660 is "No", the first CPU executes step 720. Afterwards, processing proceeds to step 795, and the first CPU temporarily terminates this routine.

[0147] In processing entry Figure 7 When the vehicle VA reaches the target space in step 660 ( Figure 7 If step 660 is "yes", the process proceeds to step 730, where the first CPU sets the second execution flag Xexe2 to "0".

[0148] Then, the first CPU executes. Figure 7 After steps 670 and 675, the process proceeds to step 795, where the first CPU temporarily terminates this routine.

[0149] If the remote driving control device 10 does not receive a driving instruction when processing step 610 ( Figure 7 Step 610 is "No"), if the period T during which no data is received is less than the threshold time Tth ( Figure 7 If step 680 is "No", then the process proceeds to... Figure 7 Step 615. On the other hand, in the case where the non-receiving period T is greater than or equal to the threshold time Tth ( Figure 7 If step 680 is "Yes", proceed to step 730.

[0150] <Screen Display Example>

[0151] The CPU of the control unit 60 of the remote operating device 20 executes the command every time a predetermined time has elapsed. Figure 8 The routine is illustrated in the flowchart. Hereinafter, the CPU of the control unit 60 will be referred to as the "second CPU".

[0152] When the appropriate time arrives, the second CPU will... Figure 8 The process begins at step 800. In step 805, it is determined whether the remote operating device 20 has received a display instruction.

[0153] When the remote operating device 20 receives a display instruction, the second CPU determines "yes" in step 805 and proceeds to step 810. In step 810, the second CPU determines whether the received display instruction contains progress data.

[0154] If the displayed indication includes progress data, the second CPU determines "yes" in step 810 and proceeds to step 815. In step 815, the second CPU displays the progress screen 200 on the display device 66.

[0155] In detail, the second CPU generates an image displayed in the surrounding situation display area 205 of the progress screen 200 based on the driving area data, object data, and driving direction data included in the progress data. It should be noted that the control unit 60 pre-stores object images corresponding to the types of objects, and the second CPU displays these object images in the surrounding situation display area 205. Therefore, the remote driving control device 10 does not need to send object images with large data volumes, thus reducing the data volume of communication between the remote driving control device 10 and the remote operation device 20.

[0156] Furthermore, the second CPU determines the traveled route 210a and the untraveled route 210b based on route data and location history data, and displays the traveled route 210a and the untraveled route 210b in the route progress display area 210. Also, the second CPU displays the remaining distance in the remaining distance display area 220 based on the remaining distance data. Finally, the second CPU displays the vehicle speed Vs in the vehicle speed display area 225 based on the vehicle speed data.

[0157] After that, the process proceeds to step 895, and the second CPU temporarily terminates this routine.

[0158] If the display indication does not contain progress data (i.e., if the display indication contains obstacle data), the second CPU determines "No" in step 810 and proceeds to step 820. In step 820, the second CPU displays the obstacle image 300 on the display device 66. Afterwards, the process proceeds to step 895, and the second CPU temporarily terminates this routine.

[0159] In detail, the second CPU displays the overhead view included in the obstacle data in the close-up status display area 305 of the obstacle screen 300. When the obstacle data includes location data, the second CPU displays the alarm information overlaid with the overhead view, facing the obstacle as indicated by the location data. Specifically, when the distance D indicated by the location data is greater than a second threshold distance D2th and less than or equal to a first threshold distance D1th, the second CPU displays a first alarm display element 305a facing the obstacle; when the distance D indicated by the location data is less than or equal to the second threshold distance D2th, it displays a second alarm display element 305b facing the obstacle.

[0160] Furthermore, the second CPU displays the travel direction image included in the obstacle data in the travel direction image display area 315. Also, if the obstacle data includes distance data, the second CPU displays the minimum distance D represented by the distance data in the obstacle distance display area 320. Furthermore, the second CPU displays the shift position represented by the shift position data included in the obstacle data in the shift position display area 325.

[0161] If the remote operating device 20 does not receive a display instruction, the second CPU determines "no" in step 805, proceeds to step 895, and the second CPU temporarily terminates this routine.

[0162] Based on the above, according to this embodiment, when remote driving control is being performed, a progress screen 200 is displayed when the vehicle VA is approaching a low-probability zone, and an obstacle screen 300 is displayed when the vehicle VA is approaching a high-probability zone. Therefore, a control screen containing user request information that changes according to the progress of remote driving control can be displayed.

[0163] (First variation)

[0164] In the above embodiments, when the remote driving control device 10 sends a display instruction containing obstacle data, it determines whether there is an obstacle with a distance D less than or equal to a first threshold distance D1th based on sonar data. However, it can also determine whether there is an obstacle with a distance D less than or equal to a first threshold distance D1th based on the front image and the rear image.

[0165] (Second variation)

[0166] In the above embodiment, when the remote driving control device 10 sends a display instruction containing obstacle data, it determines whether there is an obstacle with a distance D less than or equal to a first threshold distance D1th, and sends the distance D and position data of the obstacle with a distance D less than or equal to the first threshold distance D1th in the obstacle data, but it is not limited to this. For example, the remote driving control device 10 may also send sonar data in the obstacle data, and the remote operation device 20 may determine whether there is an obstacle with a distance D less than or equal to the first threshold distance D1th.

[0167] The remote driving control device 10 can be applied to vehicles such as engine vehicles, hybrid vehicles, plug-in hybrid vehicles, fuel cell vehicles, and electric vehicles. Moreover, remote driving control is a type of autonomous driving.

Claims

1. A remote driving control device, which executes remote driving control of a vehicle to a target space based on driving instructions from a user operating a remote control device outside the vehicle, wherein, The remote driving control device is configured as follows: While the remote driving control is being executed, a control screen containing information related to the remote driving control is displayed on the display device of the remote operating device; as well as The content displayed on the control screen is changed according to the progress of the remote driving control.

2. The remote driving control device according to claim 1, wherein, The route taken by the vehicle to the target space is divided into a first interval with a low probability of approaching an obstacle and a second interval with a high probability of approaching the obstacle, using the change of location as the dividing point. The remote driving control device is configured as follows: When the vehicle is traveling in the first section, the first control screen is displayed on the display device as the control screen; as well as When the vehicle is traveling in the second section, a second control screen with content different from the first control screen is displayed on the display device as the control screen.

3. The remote driving control device according to claim 2, wherein, The remote driving control device is configured as follows: When the vehicle is traveling in the first section, progress information indicating the progress of the remote driving control is displayed on the first control screen; as well as When the vehicle is traveling in the second section, an alarm message is displayed on the second control screen to warn of obstacles whose distance from the vehicle is less than or equal to a threshold distance.

4. The remote driving control device according to claim 3, wherein, The remote driving control device is configured to display at least one of route information and remaining distance information as the progress information on the first control screen, wherein the route information is obtained by dividing the route to the target space into a route that has been traveled and a route that has not been traveled, and the remaining distance information represents the remaining distance to the target space.

5. The remote driving control device according to claim 3, wherein, The remote driving control device is configured to display the alarm information overlaid with an overhead view of the vehicle's surroundings in the second control screen.

6. The remote driving control device according to claim 2, wherein, The remote driving control device is configured as follows: Until the vehicle reaches the switching position, motion control is performed to move the vehicle to the vicinity of the target space as the remote driving control; as well as After the vehicle arrives at the switching position, parking control is performed to park the vehicle in the target space as the remote driving control. While the motion control is being executed, it is determined that the vehicle is traveling in the first section; as well as While the parking control is being executed, it is determined that the vehicle is traveling in the second section.

7. The remote driving control device according to claim 6, wherein, The remote driving control device is configured to, while performing the remote driving control, cause the vehicle to travel at a speed not exceeding the upper limit, representing the vehicle's speed. The maximum speed limit for parking control is preset to a value lower than the maximum speed limit for motion control.

8. The remote driving control device according to claim 2, wherein, The remote driving control device is configured as follows: From the start of the remote driving control until the vehicle reaches the switching position, the remote driving control is performed to make the vehicle leave the parking space as an exit control. After the vehicle reaches the switching position, motion control is performed to move the vehicle to the target space as the remote driving control; If the outbound control is being executed, it is determined that the vehicle is traveling in the second section; as well as While the motion control is being executed, it is determined that the vehicle is traveling in the first section.

9. The remote driving control device according to claim 8, wherein, The remote driving control device is configured to, while performing the remote driving control, cause the vehicle to travel at a speed not exceeding the upper limit, representing the vehicle's speed. The upper limit speed of the outbound control is preset to be lower than the upper limit speed of the movement control.

10. A remote operating device that sends driving instructions to the vehicle when operated by a user located outside the vehicle, wherein, The vehicle performs remote driving control to travel to the target space according to the driving instructions. The remote operating device is configured to: When the vehicle is performing the remote driving control, a control screen containing information related to the remote driving control will be displayed on the display device of the remote operation device; as well as The content displayed on the control screen is changed according to the progress of the remote driving control.

Images