Vehicle control system

Abstract

This improves the safety of autonomous driving by allowing the operator to monitor the driving status of the vehicle through remote control. [Solution] The vehicle control device is a vehicle control device capable of automatic driving, and the vehicle comprises a camera unit that photographs the direction of travel of the vehicle, and a control unit that communicates with the vehicle and acquires operation instruction information from a terminal that enables remote operation by an operator, and controls the vehicle based on the operation instruction information, wherein the control unit displays the photographed image information of the direction of travel of the vehicle, which has been photographed by the camera unit, on the terminal, and when it recognizes a designated stop position based on the operation instruction information in the photographed image information, it displays the designated stop position in the photographed image information.

Description

Technical Field

[0001] The present disclosure relates to a control device for a vehicle.

Background Art

[0002] Conventionally, as one of the special vehicles equipped with a working section where an operator performs work, a garbage truck (dust collection truck) equipped with a garbage inlet as the working section is known. In the garbage truck, an operator performs the work of throwing garbage into the garbage inlet. On the other hand, in Patent Document 1, on the condition that there is no person (operator) near the garbage inlet, by increasing the operating speed of the movable device that pushes the garbage into the storage box, it is proposed to improve the work efficiency while ensuring safety.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, running a vehicle having a working section as described above without a driver by autonomous driving has been studied and tested. If such driverless running is realized, since it is not necessary for an operator to repeatedly board and alight for driving the vehicle and working on the working section, it is expected that the work efficiency will be improved.

[0005]

[0006] <00000···>The purpose of this disclosure is to address the above-mentioned challenges and to improve the safety of autonomous driving by enabling operators to understand the driving status of vehicles operating autonomously through remote control. [Means for solving the problem]

[0007] This disclosure is made to solve at least some of the aforementioned problems and can be implemented in the following embodiments or applications.

[0008] (1) The vehicle control device according to this application example is a vehicle control device capable of automatic driving, wherein the vehicle comprises a camera unit that photographs the direction of travel of the vehicle, and a control unit that communicates with the vehicle and acquires operation instruction information from a terminal that enables remote operation by an operator, and controls the vehicle based on the operation instruction information, wherein the control unit displays the photographed image information of the direction of travel of the vehicle, which has been photographed by the camera unit, on the terminal, and when it recognizes a designated stop position based on the operation instruction information in the photographed image information, it displays the designated stop position in the photographed image information.

[0009] In this application example, the operator can check blind spots by displaying the vehicle's direction of travel on the operator's terminal. Additionally, by displaying the designated stopping position within the captured image information, the operator can recognize the vehicle's stopping position. As a result, according to this application example, the operator can understand the driving status of the vehicle being driven automatically via remote control, thereby improving the safety of autonomous driving.

[0010] (2) In the vehicle control device according to the application example of (1) above, the vehicle has a location information acquisition unit that acquires location information of the vehicle, and a surrounding image generation unit that generates surrounding image information of the surrounding situation including the vehicle based on the location information acquired by the location information acquisition unit, and the control unit displays the surrounding image information generated by the surrounding image generation unit on the terminal. By displaying the surrounding situation including the vehicle on the terminal in this way, the operator can grasp the surrounding situation as well. This can improve the safety of autonomous driving.

[0011] (3) In the vehicle control device according to the application example of (2) above, the operation instruction information includes a follow mode in which the vehicle travels behind the operator and a stop mode in which the vehicle automatically travels to a designated stop position and stops, and when the control unit is controlling the vehicle based on the follow mode, if it receives operation instruction information for the stop mode from the operator, it displays at least one of the captured image information and the surrounding image information on the terminal. Thus, in this application example, if the vehicle's camera cannot recognize the operator due to the operation instruction information for the stop mode, the control unit can switch the display on the terminal to image information of the vehicle's direction of travel and surrounding conditions. This allows the operator to grasp the surrounding conditions of the vehicle that is automatically traveling by remote control in more detail.

[0012] (4) In the vehicle control device according to the application example of (2) above, the control unit displays the designated stop position based on the operation instruction information within the surrounding image information. This allows the operator to easily recognize the designated stop position of the vehicle. [Brief explanation of the drawing]

[0013] [Figure 1] This is a schematic diagram of the vehicle control system according to this embodiment. [Figure 2] This block diagram shows the control configuration of the vehicle control system according to this embodiment. [Figure 3] This is an example image (1) of the operation screen displayed on the operator's terminal. [Figure 4] This is an example image (2) of the operation image displayed on the operator's terminal. [Figure 5] This flowchart shows the control procedures performed by the vehicle control device of this embodiment. [Figure 6] This is an explanatory diagram of the first stop mode. [Figure 7] This is an explanatory diagram for the second stop mode. [Figure 8] This is an explanatory diagram for the third stop mode. [Modes for carrying out the invention]

[0014] Figure 1 is a schematic diagram of the vehicle control device according to this embodiment, Figure 2 is a block diagram showing the control configuration of the vehicle control device according to this embodiment, and Figures 3 and 4 are example images of operation images displayed on the operator's terminal. The configuration of the vehicle control device of this embodiment will be described below based on these figures.

[0015] As shown in Figure 1, the vehicle control device 1 of this embodiment comprises an electric work vehicle (hereinafter referred to as vehicle 2) and a terminal 3 operated by operator A. In this embodiment, for the sake of simplicity, there is one operator A and one terminal 3 in the vehicle control device 1, but it may be equipped with multiple operators and multiple terminals.

[0016] First, let me explain the structure of vehicle 2.

[0017] Vehicle 2 is an electric vehicle having a body 10, a body 20 mounted on the body 10, a PTO unit 30 that drives the body 20, an automatic driving device 40 for automatic driving, a VCU 50, and a vehicle memory unit 51, and is a work vehicle that performs specific tasks. In this embodiment, vehicle 2 is a refuse vehicle (a so-called garbage truck) that collects garbage. Note that the control device of the vehicle according to this embodiment is not limited to refuse vehicles, but can also be applied to other work vehicles such as refrigerated trucks, dump trucks, mixer trucks, fire trucks, or crane trucks.

[0018] The vehicle body 10 includes a main frame 11, a battery 12, a driving unit 13 for traveling, wheels 14, and a cab 15.

[0019] The main frame 11 is a frame that supports, in addition to the cab 15 and the mounting body 20, other heavy objects mounted on the vehicle body 10. The battery 12 is a power supply source that supplies power necessary for driving the vehicle body 10 and driving electric auxiliary machines mounted on the vehicle 2. The driving unit 13 for traveling is a mechanism that converts the power supplied from the battery 12 into traveling power by a traveling motor and transmits it to the wheels 14. The wheels 14 are suspended on the main frame 11 and include front wheels 14F provided below the cab 15 in the vehicle height direction (Z direction in the figure) and rear wheels 14R driven by the driving unit 13 for traveling. The cab 15 is a structure including a driver's seat and the like, and is provided at the front portion of the main frame 11 in the vehicle longitudinal direction (X direction in the figure).

[0020] The mounting body 20 is a refuse working part where one or more operators A perform refuse collection work from outside the vehicle, and has a hopper part 21 and a charging part 22.

[0021] The hopper part 21 is a part that stores the input refuse. The hopper part 21 also has a mechanism for discharging the stored refuse. For example, in the case of a so-called extrusion-type discharge refuse truck, a discharge plate is provided in the hopper part 21 to discharge the refuse by moving from the front of the vehicle (the back side inside the hopper) to the rear of the vehicle. Also, for example, in the case of a so-called dump-type discharge refuse truck, a tilt mechanism for lifting the front part of the hopper part 21 upward with respect to the main frame is provided.

[0022] The input section 22 has an input opening into which garbage is fed, and has a mechanism inside that collects the garbage and transfers it into the cargo box section 21. For example, in the case of a so-called press-type garbage truck, the input section 22 has a compression plate inside, and the fed-in garbage is compressed by the compression plate and transferred into the cargo box section 21. Alternatively, for example, in the case of a so-called roll-up type garbage truck, the input section 22 has a rotating plate inside, and the fed-in garbage is collected by the rotating plate and transferred into the cargo box section 21.

[0023] The PTO unit 30 converts power supplied from the battery 12 into rotational power to drive the various mechanisms of the body 20. Specifically, the PTO unit 30 includes a body motor 31 that is rotationally driven by power from the battery 12, and a hydraulic pump 33 that is driven by the rotational power of the body motor 31 transmitted via a belt 32.

[0024] The autonomous driving equipment 40 includes a vehicle communication unit 41, multiple cameras (photography units) 42 for recognizing the situation around the vehicle, a location information acquisition unit 43 for acquiring location information of the vehicle 2, and an automatic driving control unit 44 for controlling automatic driving.

[0025] The vehicle communication unit 41 is a device capable of communicating with external devices such as the terminal 3, and in this embodiment, it is a wireless LAN transceiver.

[0026] Cameras 42 are installed on the left and right sides of the front and the left and right sides of the rear of the vehicle 2. Each camera 42 captures images (including still images and moving images) of the area around the vehicle 2. In addition, the cameras 42 may be capable of measuring the distance from the camera 42 to objects in the image, i.e., the distance from the vehicle 2, and outputting it as distance image data.

[0027] Specifically, camera 42FR (not shown) located on the front right side of the vehicle captures images of the front and right side of vehicle 2, camera 42FL located on the front left side of vehicle captures images of the front and left side of vehicle 2, and camera 42RR (not shown) located on the rear right side of vehicle and camera 42RL located on the rear left side of vehicle capture images of the rear of vehicle 2. Cameras 42FR and 42FL located on the left and right sides of the front of the vehicle are also pointed towards the cab 15, so it is possible to capture people (operator A, etc.) getting in and out of the cab 15. Cameras 42RR and 42RL located on the left and right sides of the rear of the vehicle are also pointed towards the input side of the input section 22, so it is possible to capture the process of garbage being input. Hereafter, these four cameras 42FR, 42FL, 42RR, and 42RL may be collectively referred to as cameras 42.

[0028] The location information acquisition unit 43 has the function of acquiring the location information of the vehicle 2. The location information acquisition unit 43 receives the current location information of the vehicle 2 by receiving GPS (Global Positioning System) signals from artificial satellites, for example.

[0029] The automatic driving control unit 44 is a control unit that controls the automatic driving of vehicle 2. More specifically, the automatic driving control unit 44 is a computer equipped with a central processing unit (CPU), main memory (ROM, RAM, etc.) used for storing control programs and control maps, input / output devices, timer counters, etc. The automatic driving control unit 44 is connected to various devices such as the vehicle communication unit 41, each camera 42, VCU 50, vehicle memory unit 51, PTO unit 30, etc., via the vehicle's internal communication network, CAN (Control Area Network).

[0030] The automatic driving control unit 44 can execute an automatic driving program that controls the driving drive unit 13 and other components to drive the unmanned vehicle 2 without human intervention in steering, acceleration, or braking. For example, the automatic driving control unit 44 in this embodiment has functions such as making the vehicle 2 follow an operator A walking in front of the vehicle 2 (follow mode), making the vehicle drive along a pre-registered route that patrols garbage collection points, and making the vehicle take evasive action when it detects a person or obstacle in front of it. The automatic driving control unit 44 also has a function to stop the vehicle 2 in response to a stop command from the terminal 3. The automatic driving control unit 44 will not perform follow driving if it cannot recognize the operator A that it is following, and will stop the vehicle 2 if it loses the ability to recognize operator A during follow driving.

[0031] Furthermore, the automatic driving control unit 44 includes a surrounding situation recognition unit 44a that recognizes the situation around the vehicle 2, a surrounding image generation unit 44b that generates image information of the situation around the vehicle 2, and a stop position setting unit 44c for automatically stopping the vehicle 2.

[0032] The surrounding situation recognition unit 44a has the function of recognizing objects around the vehicle from the first image information (captured image information) P1 captured by each of the cameras 42 described above. Specifically, the surrounding situation recognition unit 44a has a trained model (so-called AI: Artificial Intelligence) that has been trained to recognize objects such as people (including operator A), roads, other vehicles, buildings, garbage collection points, etc. from the first image information P1. In this embodiment, the surrounding situation recognition unit 44a recognizes a person getting in and out of the cab 15 as operator A using the cameras 42FR and 42FL in front of the vehicle, and can measure the distance from the vehicle 2 to operator A continuously or periodically.

[0033] The surrounding image generation unit 44b has the function of generating second image information (surrounding image information) P2, which includes the vehicle 2, from the location information of the vehicle 2 acquired by the location information acquisition unit 43. Specifically, the surrounding image generation unit 44b can generate second image information P2 as the current location by reflecting the location information of the vehicle 2 received by the location information acquisition unit 43 onto map information including garbage collection routes that are stored in advance in the vehicle storage unit 51, which will be described later.

[0034] The stop position setting unit 44c has the function of automatically stopping (bringing to a halt) the vehicle 2 at a stop position corresponding to the stop instruction given from terminal 3. The specific stop position will be described later.

[0035] Vehicle 2 is also equipped with a VCU (Vehicle Control Unit) 50 for controlling all aspects of the vehicle other than autonomous driving. The VCU 50 is a computer that includes a central processing unit (CPU), main memory (ROM, RAM, etc.) for storing control programs and control maps, input / output devices, timer counters, etc. The VCU 50 is connected to various devices and other control units mounted on Vehicle 2 via a vehicle communication network called CAN (Control Area Network).

[0036] The vehicle storage unit 51 is a device capable of storing various types of information, such as an SSD (Solid State Drive), HDD (Hard Disk Drive), optical disc, or USB memory. In this embodiment, the vehicle storage unit 51 stores map information including the garbage collection route that the vehicle 2, which is a garbage truck, travels. The garbage collection route includes location information of garbage collection points, information on candidate stopping points corresponding to those garbage collection points, and image information of those garbage collection points to recognize them.

[0037] Next, we will explain the configuration of terminal 3.

[0038] Terminal 3 is an information terminal that communicates with vehicle 2 and enables remote operation by operator A, and can be, for example, a smartphone, smartwatch, smart glasses, tablet, mobile phone, etc. In this embodiment, terminal 3 will be described using a smartphone owned by operator A as an example. Operator A can operate terminal 3 by holding it in their hand, or by wearing it on their arm and operating it.

[0039] Terminal 3 includes a terminal communication unit 61, a display unit 62, a terminal storage unit 63, and a terminal control unit 64.

[0040] The terminal communication unit 61 is a device capable of communicating with external devices such as the vehicle communication unit 41 of the vehicle 2, and in this embodiment, it is a wireless LAN transceiver.

[0041] The display unit 62 is, for example, a liquid crystal display or an organic EL display, and is configured to display various types of information. Furthermore, the display unit 62 is a touch panel and also serves as the operating unit for the terminal 3.

[0042] The terminal storage unit 63 is a device capable of storing various types of information, such as an SSD (Solid State Drive), HDD (Hard Disk Drive), optical disc, or USB memory. The terminal storage unit 63 in this embodiment stores information related to the three stop modes described later.

[0043] The terminal control unit 64 is a control unit that controls terminal 3. More specifically, the terminal control unit 64 is a computer equipped with a central processing unit (CPU), main memory (ROM, RAM, etc.) used for storing control programs and control maps, input / output devices, timer counters, etc. The terminal control unit 64 is communicated with the aforementioned terminal communication unit 61, display unit 62, and terminal storage unit 63.

[0044] The terminal control unit 64 is capable of executing a start instruction program in which operator A instructs vehicle 2 to start following a vehicle via terminal 3, and a stop instruction program instructing vehicle 2 to stop driving during automatic driving. In particular, the terminal control unit 64 of this embodiment includes a vehicle stop instruction unit 64a, an image information receiving unit 64b, an image display switching unit 64c, and a display information generation unit 64d.

[0045] The vehicle stop instruction unit 64a has a set of stop instruction modes that are pre-assigned to the vehicle 2, and in this embodiment, it has three stop modes with different stopping positions. The vehicle stop instruction unit 64a has the function of issuing a stop instruction to the vehicle 2 via the terminal communication unit 61, according to the stop mode selected by operator A.

[0046] Specifically, the first stop mode is a stop mode in which vehicle 2 is stopped at the moment an operation is received from operator A.

[0047] The second stop mode is a stopping mode in which vehicle 2 is stopped according to the position of operator A at the time the operation from operator A is received. Specifically, in the second stop mode, vehicle 2 is stopped so that the rear end of vehicle 2 is positioned at the position of operator A at the time the operation from operator A is received.

[0048] The third stop mode is a stop mode in which vehicle 2 is stopped at a pre-set designated stop position. More specifically, in the third stop mode, the vehicle stop instruction unit 64a acquires stop candidate location information corresponding to garbage collection points stored in the vehicle memory unit 51 and sets stop candidate locations. From these multiple stop candidate locations, the vehicle stop instruction unit 64a designates the stop candidate location recognized in front of the vehicle 2 by the surrounding situation recognition unit 44a as the designated stop position and stops the vehicle 2 there. More specifically, in the third stop mode, if there are multiple stop candidate locations recognized by the surrounding situation recognition unit 44a from among the multiple stop candidate locations, the vehicle stop instruction unit 64a designates the stop candidate location closest to vehicle 2 as the designated stop position.

[0049] The image information receiving unit 64b can communicate with the automatic driving control unit 44 of the vehicle 2 and acquire the automatic driving status of the vehicle 2. For example, the image information receiving unit 64b can receive recognition information of the surrounding conditions of the vehicle 2 recognized by the surrounding conditions recognition unit 44a as automatic driving status information. This recognition information of the surrounding conditions includes, for example, first image information P1 captured by each camera 42, recognition information of operator A, recognition information of garbage collection points, etc. Also, for example, the image information receiving unit 64b can receive generated information of the surrounding conditions including the vehicle 2 generated by the surrounding image generation unit 44b. This generated information of the surrounding conditions includes, for example, second image information P2 created by the surrounding image generation unit 44b, location information of garbage collection points, stop candidate information corresponding to the garbage collection points, etc.

[0050] The image display switching unit 64c has the function of displaying the vehicle 2's automatic driving status information received by the image information receiving unit 64b on the display unit 62. For example, when the image information receiving unit 64b receives the first image information P1 captured by each camera 42, the image display switching unit 64c causes the first image information P1 to be displayed on the display unit 62. Also, for example, when the image information receiving unit 64b receives the second image information P2 created by the surrounding image generation unit 44b, the image display switching unit 64c causes the second image information P2 to be displayed on the display unit 62. Furthermore, the image display switching unit 64c has the function of switching the vehicle 2's automatic driving status information displayed on the display unit 62. For example, when the first image information P1 is displayed on the display unit 62, if the image information receiving unit 64b receives both the first image information P1 and the second image information P2, the image display switching unit 64c switches the display unit 62 to display either the first image information P1 or the second image information P2. The image display switching unit 64c can also display both the recognition information and the generated information of the surrounding environment on the display unit 62. For example, the image display switching unit 64c can display the first image information P1 and the second image information P2 side by side on the display unit 62. Specific display examples will be described later, along with image examples from the display information generation unit 64d.

[0051] The display information generation unit 64d has the function of generating display information to be displayed on the display unit 62. For example, when the vehicle 2 is stopped, the display information generation unit 64d generates operation image information for issuing a start command to the vehicle 2 and displays it on the display unit 62. The display information generation unit 64d also has the function of generating display information on the image showing the automatic driving status information of the vehicle 2, which is displayed on the display unit 62 by the image display switching unit 64c. In this embodiment, the display information generation unit 64d generates operation image information for operator A to issue a stop command. The display information generation unit 64d generates operation image information that allows operator A, who is performing garbage collection work, to easily select a stop mode. For example, when the surrounding situation recognition unit 44a of the vehicle 2 recognizes a candidate stop location and recognizes garbage (garbage collection point) at the candidate stop location, the display information generation unit 64d generates operation image information that accepts the operation for the third stop mode. In this case, the display information generation unit 64d generates display information for the designated stop position E1 at the stop location recognized by the vehicle 2's surrounding situation recognition unit 44a as a candidate stop location. The display information for the designated stop position E1 is generated by the image display switching unit 64c so that it can be displayed overlaid on the display of the vehicle 2's automatic driving status information displayed on the display unit 62. Specific generated images will be explained based on the image examples in Figures 3 and 4.

[0052] Figure 3(a) shows the operation screen when the surrounding situation recognition unit 44a of vehicle 2 has not recognized any trash at a candidate stopping site. In the operation screen shown in this figure, two stopping modes are selectable. As shown in the figure, in this embodiment, first image information P1 captured by the cameras 42FR and 42FL in front of the vehicle is displayed on the display unit 62 of terminal 3. The display unit 62 of terminal 3 displays a first icon B1, which is an operation icon for selecting the first stopping mode, at the top center of the screen, and a second icon B2, which is an operation icon for selecting the second stopping mode, at the top left of the screen. Each icon is roughly rectangular.

[0053] The first icon B1 displays the words "STOP NOW" in two lines, indicating that vehicle 2 should be stopped immediately, while the second icon B2 displays the word "HERE," indicating that operator A should stop at their own location. The display format is not limited to these, including the third icon B3 which will be described later, but by displaying each icon with a different representation, it becomes easier to distinguish between different stopping modes, thereby suppressing errors.

[0054] Figure 3(b) shows the operation screen when the surrounding environment recognition unit 44a of vehicle 2 has recognized garbage at a potential stopping location. In addition to the operation screen shown in Figure 3(a), the operation screen in this figure displays a third icon B3 in the upper right corner of the screen, which is an operation icon for selecting the third stopping mode. In other words, in the operation screen in this figure, all three stopping modes are selectable. The third icon B3 displays a trash can mark symbolizing a garbage collection point, indicating that the vehicle will stop at a garbage collection point. In this figure, since the surrounding environment recognition unit 44a of vehicle 2 has recognized a potential stopping location, the display unit 62 of terminal 3 displays the designated stopping position E1 generated by the display information generation unit 64d.

[0055] Figure 4 shows the operation image displayed on the display unit 62 of terminal 3 when the third icon B3 on the upper right side of the screen is selected in the operation screen of Figure 3(b). At this time, the third icon B3 is displayed blinking. In this embodiment, the second image information P2 is shown on the left side of the screen of the display unit 62 of terminal 3, and the first image information P1 is shown on the right side of the screen. In this figure, since the surrounding situation recognition unit 44a of vehicle 2 has recognized a candidate stopping location, the display information generation unit 64d displays the designated stopping position E1 in the first image information P1 on the right side of the screen. Similarly, the display information generation unit 64d displays the designated stopping position E1 in the second image information P2 on the left side of the screen. Also in this figure, the second icon B2 on the upper left side of the screen is hidden on the display unit 62, and it is not accepting operations from operator A. Depending on the stopping state of vehicle 2, the display information generation unit 64d switches to the image of Figure 3(a) or (b), and the vehicle stopping instruction unit 64a starts accepting operations for each stopping mode.

[0056] Figure 5 shows a flowchart illustrating the control performed by the vehicle control device 1 of this embodiment. The vehicle's starting and stopping control performed by the vehicle's control system will be explained with reference to the explanatory diagrams for each stopping mode in Figures 6 to 8. Note that the flowchart in Figure 5 starts from the state where vehicle 2 is stopped, that is, the vehicle 2 is in the starting standby state.

[0057] First, in step S1, the terminal control unit 64 of terminal 3 determines whether or not it has received a follow-up operation from operator A. If the result of this determination is false (No), meaning there is no operation from operator A, this flowchart is returned. On the other hand, if the result of this determination is true (Yes), that is, if operator A has given a start command, the terminal control unit 64 transmits the start command information to vehicle 2 and proceeds to the next step S2.

[0058] In step S2, the automatic driving control unit 44 of vehicle 2 receives a start command information for follow-me driving from the terminal control unit 64 and starts follow-me driving.

[0059] In step S3, when follow-me driving begins, the automatic driving control unit 44 transmits the first image information P1 captured by the camera 42 to the image information receiving unit 64b. The image display switching unit 64c, upon receiving the first image information P1, displays the first image information P1 on the display unit 62. At this time, the terminal control unit 64 also displays the operation image for stop instruction generated by the display information generation unit 64d on the display unit 62, as shown in Figure 3 above.

[0060] In step S4, the terminal control unit 64 of terminal 3 determines whether or not it has received a stop command from operator A. That is, it determines whether or not operator A has selected one of the first to third stop modes. If the first stop mode is selected, the terminal control unit 64 sends a stop command according to the first stop mode to vehicle 2 and proceeds to step S5; if the second stop mode is selected, the terminal control unit 64 sends a stop command according to the second stop mode to vehicle 2 and proceeds to step S7; and if the third stop mode is selected, the terminal control unit 64 sends a stop command according to the third stop mode to vehicle 2 and proceeds to step S9.

[0061] In step S5, the automatic driving control unit 44 receives a stop instruction for the vehicle 2 in the first stop mode from the vehicle stop instruction unit 64a of the terminal control unit 64.

[0062] In step S6, the automatic driving control unit 44 stops vehicle 2. In this case, as shown in Figure 6, as soon as operator A selects the first stop mode, the automatic driving control unit 44 promptly stops vehicle 2. Specifically, when the automatic driving control unit 44 of vehicle 2 receives a stop instruction from terminal 3 in the first stop mode, the stop position setting unit 44c sets a designated stop position at a position where the vehicle can stop with a predetermined deceleration and stops the vehicle. In step S6, the automatic driving control unit 44 confirms that vehicle 2 has stopped and returns this flowchart.

[0063] In step S7, the automatic driving control unit 44 receives a stop instruction for vehicle 2 in the second stop mode from the vehicle stop instruction unit 64a of the terminal control unit 64. In this case, as shown in Figure 7, vehicle 2 drives until its rear end is at position A1 where operator A was when operator A selected the second stop mode, and then stops. Specifically, when the automatic driving control unit 44 of vehicle 2 receives a stop instruction in the second stop mode from terminal 3, the stop position setting unit 44c measures the distance to operator A (distance to position A1), sets the position where the rear end of vehicle 2 is at position A1 as the designated stop position E1, and stops the vehicle. In this case, since the surrounding situation recognition unit 44a of vehicle 2 recognizes a candidate stop location (next to operator A), the designated stop position E1 is displayed on the first image information P1 on the display unit 62 of terminal 3.

[0064] In step S8, the automatic driving control unit 44 determines whether the vehicle 2 has stopped at the designated stopping position E1. If the determination result is false (No), meaning the vehicle has not stopped at the designated stopping position E1, the automatic driving control unit 44 returns to step S8. On the other hand, if the determination result is true (Yes), meaning the vehicle has stopped at the designated stopping position E1, the automatic driving control unit 44 returns this flowchart.

[0065] In step S9, the automatic driving control unit 44 receives a stop instruction for vehicle 2 in the third stop mode from the vehicle stop instruction unit 64a of the terminal control unit 64. In this case, as shown in Figure 8, when operator A selects the third stop mode, vehicle 2 drives to the nearest stop candidate location D1 corresponding to the nearest garbage collection point C1 in front of vehicle 2 and stops. Specifically, when the automatic driving control unit 44 of vehicle 2 receives a stop instruction in the third stop mode from terminal 3, the stop position setting unit 44c sets the nearest stop candidate location D1 to vehicle 2 among the stop candidate locations D1 and D2 corresponding to the garbage collection points C1 and C2 in front of vehicle 2, which are recognized by the surrounding situation recognition unit 44a, as the designated stop position E1 and stops the vehicle.

[0066] In step S10, the automatic driving control unit 44 transmits the second image information P2 generated by the surrounding image generation unit 44b to the image information receiving unit 64b, and the image display switching unit 64c switches the display on the display unit 62 to a screen that displays the second image information P2 together with the first image information P1 (for example, Figure 4 above). In this case, since the surrounding situation recognition unit 44a of the vehicle 2 recognizes a candidate stopping location, the designated stopping position E1 is displayed on the first image information P1 and the second image information P2 displayed on the display unit 62 of the terminal 3. The image display switching unit 64c can also switch the display on the display unit 62 from the first image information P1 to only the second image information P2.

[0067] In step S11, the automatic driving control unit 44 determines whether or not the vehicle 2 has stopped at the designated stopping position E1. If the determination result is false (No), meaning the vehicle has not stopped at the designated stopping position E1, the automatic driving control unit 44 returns to step S11. On the other hand, if the determination result is true (Yes), meaning the vehicle has stopped at the designated stopping position E1, the automatic driving control unit 44 returns this flowchart.

[0068] In this embodiment, the control device 1 displays the direction of travel of the vehicle 2 on the display unit 62 of the operator A's terminal 3, allowing operator A to check areas that are blind spots. Furthermore, by displaying the designated stopping position E1 in the first image information P1 (captured image information), operator A can recognize the stopping position of the vehicle 2. As a result, according to this application example, the operator can grasp the driving status of the vehicle that is automatically driving via remote control, thereby improving the safety of automatic driving.

[0069] Furthermore, in the control device 1 of this embodiment, the vehicle 2 has a location information acquisition unit 43 that acquires location information of the vehicle 2, and a surrounding image generation unit 44b that generates a second image information P2 (surrounding image information) of the surrounding situation including the vehicle 2 based on the location information acquired by the location information acquisition unit 43. The automatic driving control unit 44 displays the second image information P2 generated by the surrounding image generation unit 44b on the display unit 62 of the terminal 3. By displaying the surrounding situation including the vehicle 2 on the terminal 3 in this way, operator A can grasp the surrounding situation as well. This improves the safety of automatic driving.

[0070] Furthermore, in the control device 1 of this embodiment, the operation instruction information includes a follow mode in which the vehicle 2 drives behind operator A, and a stop mode in which the vehicle 2 automatically drives to a designated stop position E1 and stops. When the automatic driving control unit 44 is controlling the vehicle 2 based on the follow mode, if it receives operation instruction information for the stop mode from operator A, it displays the second image information P2 together with the first image information P1 on the terminal 3. Thus, in this application example, if the vehicle 2's surrounding situation recognition unit 44a cannot recognize operator A due to the operation instruction information for the stop mode, the automatic driving control unit 44 can switch the display on the terminal 3's display unit 62 to image information of the vehicle 2's direction of travel and its surroundings. This makes it possible to grasp the surrounding situation of a vehicle that is automatically driving via remote control by an operator in more detail.

[0071] Furthermore, in the control device 1 of this embodiment, the automatic driving control unit 44 displays the designated stopping position E1 based on the operation instruction information within the second image information P2. This allows the operator to easily recognize the designated stopping position of the vehicle.

[0072] This concludes the description of embodiments of the present invention. However, the specific configurations shown in these embodiments are merely examples, and the embodiments of the present invention are not limited to those shown in these embodiments.

[0073] For example, in this embodiment, the vehicle stop instruction unit 64a can issue stop instructions using three stop modes: a first stop mode, a second stop mode, and a third stop mode. However, it may also have only two of these stop modes. In other words, it is sufficient to have any combination of stop modes among the first stop mode and the second stop mode, the second stop mode and the third stop mode, and the first stop mode and the third stop mode.

[0074] In the above embodiment, the first stop mode may also be selectable even when the vehicle 2 is traveling to the stopping position in accordance with a stop instruction in the second stop mode or the third stop mode. When the first stop mode is selected, the stopping position setting unit 44c of the vehicle 2 prioritizes the stop instruction in the first stop mode over the stop instruction in the second stop mode or the third stop mode.

[0075] Furthermore, in the above embodiment, the stop position setting unit 44c sets multiple stop candidate locations in the third stop mode based on map information of the garbage collection route which includes stop candidate location information corresponding to garbage collection points in advance. However, the map information of the garbage collection route does not necessarily have to include stop candidate location information corresponding to garbage collection points. For example, when the surrounding situation recognition unit recognizes a garbage collection point from an image captured by the camera, the stop position setting unit may set the vicinity of the recognized garbage collection point as a stop candidate location and designate the stop candidate location closest to the vehicle as the specified stop position.

[0076] Furthermore, in the above embodiment, the surrounding situation recognition unit 44a recognizes the surrounding situation of the vehicle 2 from images captured by each camera 42, but the equipment for recognizing the surrounding situation is not limited to cameras. For example, a sensor using laser light such as LiDAR or a sensor using millimeter waves such as millimeter-wave radar may be used as equipment for recognizing the surrounding situation.

[0077] Furthermore, in the above embodiment, the vehicle communication unit 41 and the terminal communication unit 61 use wireless LAN, but the communication means is not limited to this, and may also be Bluetooth®, for example. Also, the vehicle communication unit and the terminal communication unit are not limited to being directly connected to each other for communication, but may be connected to each other via an external information network such as the Internet or a dedicated network.

[0078] Furthermore, in the above embodiment, the automatic driving control unit 44 and the VCU 50 are mounted separately on the vehicle 2, but the VCU may also function as the automatic driving control unit.

[0079] Furthermore, although the terminal 3 in the above embodiment has a display unit 62 that also serves as an operating unit, it may also have other operating units such as a mouse, keyboard, or stylus.

[0080] Furthermore, in the above embodiment, map information including garbage collection routes is stored in the vehicle storage unit 51, but this information may also be stored in the terminal storage unit 63.

[0081] Furthermore, although the PTO unit 30 in the above embodiment has a mounting motor 31, a belt 32, and a hydraulic pump 33, the configuration of the PTO unit is not limited to this. For example, gears may be used instead of a belt, or a compressor may be used instead of a hydraulic pump.

[0082] Furthermore, in the above embodiment, the display information generation unit 64d generates a designated stop position E1 at a stop location recognized by the surrounding situation recognition unit 44a of the vehicle 2 as a candidate stop location. However, the designated stop position E1 may also be displayed based on the location information acquisition unit 43 from map information including garbage collection routes that is pre-stored in the vehicle memory unit 51 of the vehicle 2.

[0083] Furthermore, in steps S8 and S11 of the above embodiment, the automatic driving control unit 44 determines whether or not the vehicle 2 has stopped. However, the stopping of the vehicle 2 may be determined by obtaining information regarding the stopping of the vehicle 2 from, for example, the terminal control unit 64 of the terminal 3 or the VCU 50 of the vehicle 2, or operator A may confirm the stopping and perform a stop confirmation operation on the terminal 3.

[0084] Furthermore, in the above embodiment, the display switches to show the first image information P1 and the second image information P2 when the third stop mode is instructed by operator A. However, the timing of the image switching is not limited to operation by operator A. For example, the second image information P2 may be automatically displayed when the image display switching unit 64c can no longer recognize part or all of the designated stop position E1 on the image of the first image information P1 because the vehicle 2 is approaching the stop position. [Explanation of Symbols]

[0085] 1. Control System 2 vehicles 3 terminals 10 car bodies 11 Mainframe 12 batteries 13. Drive unit for traction 14(14F, 14R) wheels 15 Cab 20 Bodywork 21 Packing box section 22 Insertion section 30 PTO units 31. Motor for mounting 32 belts 33 Hydraulic pump 40 Autonomous driving equipment 41 Vehicle Communications Department 42 (42FL, 42FR, 42RL, 42RR) Camera 43 Location information acquisition unit 44 Automatic Driving Control Unit 44a Surrounding Situation Recognition Unit 44b Peripheral Image Generation Unit 44c Stop position setting section 50 VCU 51 Vehicle memory unit 61 Terminal Communications Section 62 Display section 63 Terminal Storage Unit 64 Terminal Control Unit 64a Vehicle Stop Indicator 64b Image information receiving unit 64c Image display switching section 64d Display information generation section Operator A B1 1st Icon B2 Second Icon B3 3rd icon C1, C2 Garbage Collection Point D1, D2 Candidate stop locations E1 Specified stop position P1 First Image Information P2 Second image information

Claims

1. A control device for a vehicle capable of autonomous driving, The aforementioned vehicle is A camera unit that photographs the direction of travel of the aforementioned vehicle, The system includes a control unit that communicates with the vehicle, acquires operation instruction information from a terminal that enables remote operation by an operator, and controls the vehicle based on the operation instruction information, The control unit is characterized by displaying captured image information of the vehicle's direction of travel, captured by the imaging unit, on the terminal, and displaying the designated stop position within the captured image information when the control unit recognizes a designated stop position based on the operation instruction information in the captured image information.

2. The aforementioned vehicle includes a location information acquisition unit that acquires the location information of the vehicle, The system includes a surrounding image generation unit that generates surrounding image information of the surrounding situation including the vehicle based on the location information acquired by the location information acquisition unit, The control unit is characterized in that it displays the surrounding image information generated by the surrounding image generation unit on the terminal, as described in claim 1.

3. The operation instruction information includes a follow mode in which the vehicle travels behind the operator, and a stop mode in which the vehicle automatically travels to a designated stop position and stops. The vehicle control device according to claim 2, characterized in that when the control unit is controlling the vehicle based on the follow mode, it receives an operation instruction information for the stop mode from the operator and displays at least one of the captured image information and the surrounding image information on the terminal.

4. The vehicle control device according to claim 2, characterized in that the control unit displays a designated stop position based on the operation instruction information within the surrounding image information.

Images