Automated driving system and display method

Abstract

To provide an automatic traveling system improved in convenience.SOLUTION: An automatic traveling system comprises an automatic-traveling control unit that allows a work vehicle to automatically move along a travel route and a display control unit that displays a predetermined image on a display unit. The display control unit allows the display unit to display a travel route image showing the travel route, a vehicle-body position image indicating a position of the work vehicle and a bird-eye image of a surrounding of the work vehicle.SELECTED DRAWING: Figure 14

Description

Technical Field

[0001] The present disclosure relates to an automatic driving system and a display method for automatically driving a work vehicle along a predetermined travel route.

Background Art

[0002] WO 2015 / 119264 discloses a system for an automatically driving work vehicle. This system includes a work vehicle and a remote control device for remotely operating the work vehicle. The work vehicle uses a satellite positioning system to measure the position of the vehicle body, and the remote control device automatically drives along a preset travel route. The work vehicle is equipped with a camera for photographing the surrounding area of the vehicle body, and the photographed video is displayed on the display of the remote control device, and it is described that obstacles can be confirmed by the image showing the front. In addition, it is described that the travel route of the work vehicle can be displayed on the display of the remote control device. [[ID=zo]]

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The preset travel route is preferably displayed prominently on the display in order to indicate the route along which the work vehicle automatically travels. In addition, the image captured by the camera is preferably displayed prominently on the display for monitoring the state of automatic driving.

[0005] <000003o>

[0005] ​​​​​However, in order to display both images on the screen, both images must be scaled down. The only way to display them side-by-side is to make the route and camera images small and difficult to see. Since the images will be displayed in separate areas, it is not possible to view both images simultaneously. Therefore, eye movement is essential.

[0006] This disclosure was made in response to these challenges, and its purpose is to improve convenience. The objective is to provide an automated driving system that achieves the desired results. [Means for solving the problem]

[0007] The automated driving system disclosed herein is An automatic driving control unit that automatically drives a work vehicle along a travel route, It comprises a display control unit that causes a predetermined image to be displayed on the display unit, The display control unit, The display unit displays a travel route image showing the travel route, a vehicle position image showing the position of the work vehicle, and an overhead view image of the area around the work vehicle.

[0008] Because the image is a bird's-eye view looking down on the work vehicle, the image of the work vehicle's surroundings and the travel route can be displayed in a single area. Therefore, compared to displaying the image of the work vehicle's surroundings and the travel route separately in different areas, both can be displayed larger, providing a more easily viewable display. In addition, the supervisor only needs to look at one display area, eliminating the need to shift their gaze, thus making monitoring easier for the supervisor. Therefore, it becomes possible to provide an autonomous driving system with improved convenience. [Brief explanation of the drawing]

[0009] [Figure 1] A diagram showing the schematic configuration of an automated driving system. [Figure 2] A block diagram showing the general configuration of an automated driving system. [Figure 3] A diagram showing a splash screen displayed on a display device. [Figure 4] A diagram showing a disclaimer screen displayed on a display device. [Figure 5] A diagram showing an initial registration screen displayed on a display device. [Figure 6] A diagram showing a top screen displayed on a display device. [Figure 7] A diagram showing a field list screen displayed on a display device. [Figure 8] A diagram showing a path list screen displayed on a display device. [Figure 9] A diagram showing a path details screen displayed on a display device. [Figure 10] A diagram showing a work preparation screen displayed on a display device. [Figure 11] A diagram showing a work screen displayed on a display device. [Figure 12] A diagram showing a notification information screen displayed on a display device. [Figure 13] A diagram showing a work screen displayed on a display device. [Figure 14] An explanatory diagram showing the relationship between a travel route, an aerial view image, and the current position of a vehicle. [Figure 15] A diagram showing a work screen displayed on a display device. [Figure 16] A diagram showing a work screen displayed on a display device.

Mode for Carrying Out the Invention

[0010] Embodiments of the present disclosure will be described with reference to the drawings.

[0011] As shown in FIG. 1, the automatic driving system includes an automatic driving work vehicle 1 that automatically drives along a predetermined travel route, and a display device 2 that can communicate with the automatic driving work vehicle 1. The system of this embodiment further includes a reference station 4 for acquiring the position information of the automatic driving work vehicle 1. The display device 2 can instruct various types of information to the automatic driving work vehicle 1.

[0012]

[0012] In Figure 1, a tractor is used as an example of the automated work vehicle 1, but in addition to tractors, rice transplanters, In addition to ride-on work vehicles such as combine harvesters, civil engineering and construction equipment, and snowplows, we also offer pedestrian-only work vehicles. Applicable.

[0013] Let's describe an example of a vehicle: an automated driving work vehicle 1. As shown in Figure 1, The work vehicle 1 is equipped with a body frame 10a. At the front of the body frame 10a, the front axle The drive case 10b is supported, and the front axle drive case 10b controls a pair of front wheels 1 0c is supported. The hood 10d is mounted on the front upper part of the body frame 10a. As shown in Figure 1, the engine 10e is located inside the hood 10d. At the rear of the hood 10d, the cabin 10f is mounted above the rear of the body frame 10a. It is shown. Inside cabin 10f, there are seats, a steering wheel, etc. (not shown). It can be done.

[0014] As shown in Figure 1, below the cabin 10f, the rear end of the vehicle frame 10a is the gear shift It is connected to the transmission case of the 10g transmission. The 10g transmission has a pair of rear components on the left and right. It supports the axle of wheel 10h and outputs power for front-wheel drive to the front axle drive case 10b. It supports the front-wheel drive axle for generating power.

[0015] As shown in Figure 2, the automated driving work vehicle 1 has a vehicle-side wireless communication unit 14, and the display device 2 is The terminal side has a wireless communication unit 21, and the base station 4 has a wireless communication unit 41 on the base station side. The communication unit 14 and the terminal-side wireless communication unit 21 communicate via wireless communication to display "Automated Driving Work Vehicle 1". The device is configured to send and receive various types of information between it and the device 2, and the vehicle-side wireless communication unit 14 and the reference Wireless communication between the station-side wireless communication unit 41 and the automated driving work vehicle 1 and the reference station 4 is transmitted. It is configured to send and receive species information. The display device 2 and the reference station 4 are connected to the automatic driving The system is configured to send and receive various types of information via the work vehicle 1. Furthermore, the terminal-side wireless communication unit... The display device 2 and the reference station 4 communicate via wireless communication between 21 and the reference station side wireless communication unit 41. It is also possible to configure the system to send and receive various types of information directly without going through the mobile work vehicle 1. The frequency band used for wireless communication between line communication units may be a common frequency band. Furthermore, they may be in different frequency bands.

[0016] The automated driving work vehicle 1 includes a positioning antenna 11 and a control unit 12 (automatic) as shown in Figure 2. Driving control unit, position information acquisition unit 13, vehicle-side wireless communication unit 14, camera 15, obstacle sensor 16 is provided, and under the control of the control unit 12, the position information acquisition unit 13 acquires its own current position information It is configured to operate autonomously while acquiring information (the current position of the autonomous work vehicle 1).

[0017] Camera 15 takes pictures of the area around the aircraft. The images taken by camera 15 are displayed on display device 2. It is configured to be displayable on Play24. In this embodiment, at least four cameras are They are arranged. In this embodiment, a first camera takes a picture of the front of the aircraft, and a second camera takes a picture of the right side of the aircraft. A second camera, a third camera capturing the left side of the aircraft, and a fourth camera capturing the rear of the aircraft are positioned. Of course, if it's possible to photograph the area around the aircraft, it's not limited to four cameras, but one omnidirectional camera can be used. Mera would also be fine.

[0018] The obstacle sensor 16 detects whether or not an obstacle exists within a predetermined distance around the aircraft. For example, it can be made up of an ultrasonic sensor or the like. The obstacle sensor 16 is located at a predetermined distance around the aircraft. If an obstacle is detected within the aircraft's range, the control unit 12 stops the aircraft from moving. It was also possible to control it in that way.

[0019] As shown in Figure 1, the positioning antenna 11 is, for example, connected to a Global Navigation Satellite System (GNSS). It is configured to receive signals from the positioning satellites 3 that make up the system. The positioning antenna 11 is For example, it is positioned on the upper surface of the roof of the cabin 10f of the autonomous work vehicle 1.

[0020] As shown in Figure 1, a positioning method using a satellite positioning system involves predetermined reference points. Based on correction information from the base station 4 installed at [location], the satellite positioning information of the automated driving work vehicle 1 (mobile station) The system allows for the application of a positioning method that corrects the reported data to determine the current position of the automated work vehicle 1. For example, DGPS (Differential GPS positioning), RTK positioning (Real-time Cinematography) Various positioning methods such as tick positioning can be applied. Therefore, it is also possible to use standalone positioning without having a base station 4.

[0021] In this embodiment, for example, RTK positioning is applied, as shown in Figures 1 and 2. In addition to equipping the autonomous driving work vehicle 1, which will be the mobile station, with a positioning antenna 11, A substation 4 is provided. The location information of the reference point where base station 4 will be installed is pre-set. It is understood. Reference station 4 is, for example, located around a field, and does not interfere with the movement of the automated work vehicle 1. It is positioned in a location (reference point) that does not become a reference point. The reference station 4 has a reference station side radio communication unit 41 and It is equipped with a base station positioning antenna 42.

[0022] In RTK positioning, a reference station 4 is installed at a reference point, and the mobile station whose location information is to be obtained is also involved. Both the positioning antenna 11 of the autonomous driving work vehicle 1 and the carrier wave phase from the positioning satellite 3 are detected. It measures satellite positioning information. Base station 4 measures satellite positioning information from positioning satellite 3. Each time, or after a set period has elapsed, supplementary data including measured satellite positioning information and reference point location information, etc. The correct information is generated and transmitted from the reference station side radio communication unit 41 to the vehicle side radio communication unit of the automated driving work vehicle 1. Correction information is being transmitted to 14. The position information acquisition unit 13 of the automated driving work vehicle 1 uses positioning A The satellite positioning information measured by the antenna 11 is corrected using the correction information transmitted from the base station 4. The system then obtains the current location information of the automated driving work vehicle 1. For example, latitude and longitude information are obtained as the current location information of the work vehicle 1.

[0023] Display device 2 is, for example, a tablet-type personal computer with a touch panel. It consists of information processing devices such as the touch panel, and various types of information can be displayed on the touch panel. By operating this, various types of information can also be input. Regarding the display device 2, the user It can be carried and used outside of the automated driving work vehicle 1, and the automated driving It can also be used by attaching it to the side of the driver's seat of work vehicle 1, etc.

[0024] As shown in Figure 2, the display device 2 includes a terminal-side wireless communication unit 21, a display control unit 22, and a path generation unit. The unit has a section 23 and a display 24 (touch panel). The display control unit 22 controls the display The system is configured to display various screens on 24 display screens. In the following description, If the display control unit 22 decides to display a predetermined screen, unless otherwise specified, the display This indicates the display to 24. The route generation unit 23 will determine the route that the automated driving work vehicle 1 will drive automatically. It is configured to generate a travel route. The display device 2 also includes a storage unit (not shown). It is equipped with a memory unit that stores various types of information, such as information registered by the user. It is being done.

[0025] To operate the automated driving work vehicle 1, the user first operates the display device 2. As a result, the route generation unit 23 generates a travel route, and information related to the generated travel route, etc. Other various information is transmitted from the display device 2 to the automated driving work vehicle 1. Then, the automated driving begins. After all the conditions for starting the automatic operation of work vehicle 1 are met, the user will use a display device. By operating 2 and instructing the automated driving work vehicle 1 to start automated driving, the automated driving operation The automated driving of the work vehicle 1 begins. During automated driving, the control unit 12 of the automated work vehicle 1 The automated driving work vehicle 1 will automatically drive along the driving path generated by the path generation unit 23. Based on the current location information of the automated driving work vehicle 1 acquired by the location information acquisition unit 13, It controls the operating status of the mobile work vehicle 1. It also controls the position information acquired by the position information acquisition unit 13. The current location information of the mobile work vehicle 1 is automatically collected in real time (for example, every few seconds). The data is transmitted from the mobile work vehicle 1 to the display device 2, and is automatically displayed on the display 24 of the display device 2. The current location of the mobile work vehicle 1 can be displayed.

[0026] Based on Figures 3 to 13 below, when the automated driving operation of the automated driving work vehicle 1 is performed, the display equipment The control of the display screen by the display control unit 22 in unit 2 will be explained. Figures 3 to 13 show: All of these images show the display screen shown on the display 24 of the display device 2.

[0027] The display control unit 22 operates after the power is turned on to the display device 2 or when an application is displayed on the display device 2. After startup, the splash screen shown in Figure 3, the disclaimer screen shown in Figure 4, and the top screen shown in Figure 6 will appear. Display them sequentially on the P screen.

[0028] The splash screen shown in Figure 3 is displayed when the power is turned on or when an application is launched. This is the screen that is displayed, with the company logo and company name etc. in the center of the screen, and the user can select... The operable operation unit 51 (see Figures 4 to 13) is not displayed. The display control unit 22 is powered by After a certain period of time has passed since insertion, the screen will change from the splash screen (Figure 3) to the disclaimer screen (Figure 4). Transition. The disclaimer screen shown in Figure 4 contains a disclaimer regarding the use of the automated driving work vehicle 1. This will be displayed.

[0029] In the disclaimer screen shown in Figure 4, the upper display area 53 at the top edge of the screen and the area below it are shown. The lower display area 54 is divided into two parts, and the upper display area 53 is for this disclaimer. The various screens displayed from this point onward (Figures 5 to 13) also share the same display screen. The upper display area 53 is a fixed display area whose displayed content does not change. The lower display area 54 can display different content by the user selecting an option on the operation unit 51. This is a non-fixed display area.

[0030] In the upper display area 53, the "To Map" and "Help" control sections 51 are displayed at the left edge. The "GNSS" and "MENU" control panel 51 is displayed on the far right, and the current status is shown in the center. The content of the screen that is currently displayed will be shown. Incidentally, the "Go to Map" operation part 51 is the control unit for displaying map information, and the "GNSS" control unit 51 is for latitude , longitude, GNSS status, number of satellites acquiring satellite positioning information, serial number of base station 4 To display the license plate number, the battery level of base station 4, and other detailed information of the satellite positioning system. This is the control unit. Also, when generating a new travel route in the route generation unit 23, "M Select the operation unit 51 of "ENU" and then, according to the displayed content, various information will be displayed. By inputting this information, the route generation unit 23 generates a travel route.

[0031] In the disclaimer screen shown in Figure 4, the user checks the box that says "I have read everything." Then, by selecting the "Agree and use" operation unit 51, the display control unit 22 will, The user transitions from the disclaimer screen shown in Figure 4 to the initial registration screen shown in Figure 5. By selecting the "Exit Application" operation section 51 on the responsibility screen, The application is terminated. The initial registration screen shown in Figure 5 is for the automated work vehicle 1. This screen is for registering various information about the automated driving work vehicle 1, such as the model, and the user Various information about the automated work vehicle 1 is registered through the operation. Incidentally, selected The frame of the selected model will be surrounded by a thick red line, making it easy to see which model you have chosen. The display control unit 22 registers various information related to the automated driving work vehicle 1. Then, the user transitions from the initial registration screen shown in Figure 5 to the top screen shown in Figure 6. Before transitioning from the initial registration screen to the top screen shown in Figure 6, information regarding the automated driving work vehicle 1 is displayed. You may also go through other initial registration screens (for example, home address registration) besides the notification screen.

[0032] If various information regarding the automated work vehicle 1 has already been registered, it will be displayed. The control unit 22, without displaying the initial registration screen shown in Figure 5, moves from the disclaimer screen shown in Figure 4. The screen will transition to the top screen shown in Figure 6.

[0033] The top screen shown in Figure 6 has multiple user-selectable operation units 51 (various eye icons in the figure). The console and map information 52 are displayed, and one of the multiple operation units 51 is Selecting an option will transition you to another screen. In the top screen shown in Figure 6, the lower display area 54 Map information 52 is displayed in the center of the screen, including the registered fields H1 to H7. Incidentally, when the system is first launched and no fields are registered, the center of map information 52 is pre-set. It is displayed in the designated position. In addition, the lower display area 54 is used for displaying the field list. Work unit 51a (icon indicated by "←" in the figure), Previous operation operation unit 51b (indicated by "Previous operation" in the figure) The icon shown in "Start work on the specified path," and other operating units 51, map information 52 It is displayed surrounding the area, excluding the top.

[0034] When the display control unit 22 displays the map information 52, the center of the map information 52 is the same as the previous display. The display is made to be in the same position as the center of the screen. In addition, the display control unit 22 will determine the current position When the control unit 51 is operated, the center of the map information 52 is the current position of the automated driving work vehicle 1. Display it in a way that makes it appear as intended.

[0035] When the display control unit 22 displays the map information 52 on the top screen shown in Figure 6, Regarding the position of the automated work vehicle 1 when it performed automated driving: The previous work position information 55 is displayed in an identifiable manner. The previous work position information 55 is marked with a predetermined mark ( The pin indicates the center of field H2 where the automated work vehicle 1 previously performed an automated run. It is displayed in the section. This allows the user to see when the automated driving work vehicle 1 last performed an automated driving operation. This makes it easy to identify which field was worked on. The previous work location 55 is shown. The location to be indicated only needs to be a location that allows identification of which field H it is; for example, When registering field H, the starting position of the work and the entry of the automated driving work vehicle 1 into field H are recorded. It is also possible to display the previous work position 55 on the mouth. Incidentally, the display control unit 22 is located in the field. This applies not only when the previous automated driving is terminated upon completion of the task, but also when the previous automated driving is terminated in the middle of field work. Even after the automated run is complete, the previous work position information 55 will be displayed at the center of the field. It's sea urchin.

[0036] In the top screen shown in Figure 6, the user operates the field list display control unit 51a (indicated by "←" in the figure). When you select the icon shown in Figure 6, the display control unit 22 will switch from the top screen shown in Figure 6 to Figure This will transition to the field list screen shown in 7.

[0037] In the field list screen shown in Figure 7, the lower display area 54 is the lower left display area 54a and the lower The right-hand display area 54b is divided into two sections, and the lower left-hand display area 54a contains the top section shown in Figure 6. Multiple operation units 51 and map information 52 that were displayed in the lower display area 54 of the screen are displayed. The lower right display area 54b displays a list of field information 56 stored in the display device 2, and Furthermore, to display a list of previously generated travel routes (paths) in a specific field... The control panel 51c for displaying the path list (the icon for "Display path list for this field" in the diagram) is displayed. In the list of field information 56, there are options for sorting by "distance," "name," and "registration order." When 51 is selected, the items can be sorted and displayed in the order in which the operation was performed. Field information at the site includes the distance from the current position of the automated work vehicle 1 to the field, and the field itself. The registration date and field location information are displayed. Additionally, a list of field information (56) is available. When displaying the information, a predetermined mark is used for the field where the automated driving vehicle 1 previously drove automatically. The previous field information 57 (pin) is displayed in the "K" section. This allows the user to see the previous field information. The map information 52 identifies which field the mobile work vehicle 1 automatically drove through. It can be recognized above (previous work location information 55), and the field can be identified from the field information list 56. Information related to this (previous field information 57) can also be recognized.

[0038] Furthermore, when the user selects a specific field on the field list screen shown in Figure 7, the display control unit... 22 changes the color of the frame of the selected field and surrounds it with a thick line (for example, with a thick red line) By enclosing the selected field, it is displayed in a way that makes it distinguishable from other fields, and the selection operation is performed. The operating section 51 is also enclosed in a thick line with a different frame color (for example, with a thick red line). By enclosing it, the selected operation unit 51 is displayed in a way that makes it distinguishable from other operation units 51.

[0039] In the field list screen shown in Figure 7, when the user selects a specific field, the travel route is When the display operation unit 51c is selected, the display control unit 22 displays the field list screen shown in Figure 7. This transitions to the path list screen shown in Figure 8.

[0040] In the path list screen shown in Figure 8, the lower left display area 54a shows an enlarged view of the selected field. 58 is displayed, and in the lower right display area 54b, previously generated path information (related to the driving route) The list of information (59) and the selected path information (information related to the driving route) are used for automatic driving operations. Path transfer operation unit 51d for transferring to work vehicle 1 (in the figure, "Start work on this path") The icon shown in "[ ]" will be displayed.

[0041] The enlarged field screen 58 displays the travel route 25 generated by the route generation unit 23. In Figure 8, for example, multiple straight travel paths 25 are arranged in a line with intervals between them in the left-right direction. It is displayed as "uni".

[0042] The list of path information 59 includes operation buttons 51 for "order by distance," "order by name," and "order by registration." When an operation is performed, the items can be displayed in the order in which the operation was performed, and the path information The report includes the distance from the current position of the automated work vehicle 1 to the starting position of the path, and the generated path. The date, starting location information of the path, etc. will be displayed. Also, in the list of path information 59, path information Path detail display operation unit 51e (shown as "Details" in the figure) for displaying detailed information The icon is displayed, and the selected path information and the selected operation unit 51 are shown. By changing the color of the frame and surrounding it with a thick line (for example, with a thick red line), other path information can be displayed. It is displayed in a way that allows it to be identified from the information and other operating parts 51.

[0043] In the path list screen shown in Figure 8, the path details are displayed when the user has selected a specific path. When the user selects operation on the operation unit 51e, the display control unit 22 displays the path list screen shown in Figure 8, and then displays the path list screen shown in Figure 9. This will transition to the path details screen shown.

[0044] In the path details screen shown in Figure 9, the lower right display area 54b is accessed from the path list screen shown in Figure 8. Only the displayed content is changed, and path details 60 is displayed. As path details 60, Field area, presence or absence of obstacles, non-working area surrounding the work area where the path (travel route) is generated ( Width of the headland, etc., base station information when generating the path (travel route), and the current communication status of the automated driving work vehicle 1. Base station information, tractor information, work equipment information attached to the autonomous driving work vehicle 1, path ( The number of trains (and other information related to the route) will be displayed.

[0045] In the path list screen shown in Figure 8 or the path details screen shown in Figure 9, the user performs the path transfer operation When the operation of the work unit 51d is performed, the display control unit 22 displays the path list screen shown in Figure 8 or the screen shown in Figure 9. From the path details screen, transition to the work preparation screen shown in Figure 10. At this time, path information (driving Information regarding the route is displayed by the terminal-side wireless communication unit 21 and the vehicle-side wireless communication unit 14. The data is transferred from the device 2 to the automated driving work vehicle 1. There, the control unit (not shown) of the display device 2 and The control unit 12 of the automated driving work vehicle 1 consists of the terminal-side wireless communication unit 21 and the vehicle-side wireless communication unit 14 The path information transmitted via the terminal-side wireless communication unit 21 and the vehicle-side wireless communication unit 14 Wireless communication checks whether the path information acquired by the automated work vehicle 1 matches. Perform the check process.

[0046] This wireless communication check process includes, for example, the terminal-side wireless communication unit 21 and the vehicle-side wireless communication unit. The total amount of path information transmitted via the signal unit 14, and the terminal-side wireless communication unit 21 and the vehicle-side wireless communication Whether or not the total amount of path information acquired by the automated driving work vehicle 1 via unit 14 matches the amount of path information acquired by the automated driving work vehicle 1. Then, a process to check whether the path information matches (CRC check process) is applied. This is possible. Also, as a wireless communication check process, instead of the CRC check process, for example Then, the individual ID information in the path information is transferred, and whether or not that ID information matches is determined. It can also check whether the path information matches, and various wireless communication check processes can be performed. This can be applied.

[0047] The display control unit 22 transmitted via the terminal-side wireless communication unit 21 and the vehicle-side wireless communication unit 14. Path information and the automated driving operation vehicle via the terminal-side wireless communication unit 21 and the vehicle-side wireless communication unit 14 After confirming that the path information obtained by 1 matches, proceed from the preparation screen shown in Figure 10. The system transitions to the work screen shown in Figure 11. This allows the display control unit 22 to communicate with the terminal-side wireless communication unit. When path information is transmitted to the automated driving work vehicle 1 via 21 and the vehicle-side wireless communication unit 14, The path information transmitted from the display device 2 matches the path information acquired by the automated driving work vehicle 1. Only when these predetermined conditions are met can the work screen shown in Figure 11 be displayed. .

[0048] The display control unit 22 receives path information transmitted from the display device 2 and information acquired by the automated driving work vehicle 1. If the predetermined condition that the path information matches is not met, the work screen shown in Figure 11 will not be displayed. Instead, it either displays an error screen or returns to the path list screen shown in Figure 8. Although the error screen is not illustrated here, it displays the error details in an identifiable manner. It would be nice to have, for example, a display screen where the content displayed differs depending on the type of error, or error Regardless of the content, a common display screen with identical display content can be used. In addition, if an error screen is displayed, you can return to the path list screen shown in Figure 8 from the error screen. I try to do that.

[0049] In the work screen shown in Figure 11, the lower display area 54 is located towards the upper end in the vertical direction. The lower upper display area 54c, the lower lower display area 54d located at the lower end, and the lower upper display area The lower intermediate display area 54e is located between the display area 54c and the lower lower display area 54d. It is divided into two display areas. And the lower upper display area 54c and the lower lower display area 54 d cannot be switched to other screens, while the lower intermediate display area 54e can be switched to other screens. That's how it is.

[0050] The lower intermediate display area 54e displays either the path list screen shown in Figure 8 or the path details screen shown in Figure 9. In the lower left display area 54a, there is an enlarged view of the field 58 and multiple control panels 51. The same display screen as above is shown. The display control unit 22 automatically displays the enlarged field screen 58. The current position 26 of the mobile work vehicle 1 is displayed, and the current position of the automated work vehicle 1 is It is designed to recognize whether it is. The display control unit 22 then displays an enlarged screen of the field 58 In this document, the currently automated driving route 25 of the automated work vehicle 1 and the next planned route are shown. The travel route 25 can also be displayed in a distinguishable manner by changing its color, etc. Furthermore, the display control unit 22 checks the travel routes 25 for which work has already been performed. The route 25, which has not yet been worked on, is displayed in a different form by changing the color, etc. It is possible.

[0051] Incidentally, the display control unit 22 displays the field expansion on the display screen of the lower intermediate display area 54e. In addition to the large 58-inch screen, it is configured to allow switching between tractor information screens and work command screens. Although not shown in the diagram, the tractor information screen displays actual vehicle speed, actual engine speed, load factor, etc. Information indicating the status of the automated work vehicle 1 is displayed. On the work command screen, vehicle speed and engine speed are displayed. The control unit 51 for changing the gin rotation speed is displayed. The display control unit 22 also displays the following: The lower intermediate display area 54e can be used as a display screen for interrupting the display of warning screens, caution screens, etc. It is composed of various conditions for starting automatic driving, such as: This screen displays conditions that are not met, and the warning screen appears during automatic driving, indicating the current driving conditions. This screen displays the reason why the automatic driving system stops if it is being maintained in a certain state. Control information related to this warning screen and alert screen is generated by the control unit 12 of the automated driving work vehicle 1. Therefore, the display device 2 receives the generated control information at the terminal-side wireless communication unit 21. By doing so, the display control unit 22 will interrupt and display a caution screen or warning screen. ru.

[0052] The lower upper display area 54c has a start operation unit 51 at its left end that can be used to instruct the start of automatic driving. f is positioned, and a stop operation unit 51g capable of instructing the automatic driving to stop is positioned at the right end. The lower display area 54d contains an operation section 5 for checking error notification information. 1h and other control units 51 are located, and the work progress rate and work by automatic driving The remaining time and other information are displayed.

[0053] The display control unit 22 will operate the start operation unit until various conditions for starting automatic driving are met. The 51f is set to an inactive state where it cannot be selected, and various conditions for starting automatic driving are set. When this condition is met, the start operation unit 51f is set to an active state, making it selectable. The control unit 22, for example, lights up the start operation unit 51f when inactive, and activates In this state, the display mode of the start operation unit 51f is changed, such as by making the start operation unit 51f blink. This allows the user to distinguish between the inactive and active states.

[0054] The display control unit 22 receives an instruction from the start operation unit 51f to start automatic driving, and then the automatic driving operation begins. When automatic driving of vehicle 1 begins, the start operation unit 51f can be instructed to temporarily suspend automatic driving. The pause operation unit (not shown) has been changed. And the pause operation unit is selected. When this happens, the display control unit 22 changes the start operation unit 51f back to the original display mode from the pause operation unit. The start operation unit 51f displays a message indicating the start of automatic driving and a message indicating the temporary suspension of automatic driving. The display control unit 22 is configured to be switchable between a display mode that instructs and a start. By changing the display format of the control unit 51f, one control unit can function as two separate control units. It can also be made to start or stop automatic driving, etc. This makes it easier for users to understand the status of autonomous driving operations.

[0055] In the display device 2, the terminal-side wireless communication unit 21 and the vehicle-side wireless communication unit 14 communicate the automatic driving operation. The current location information of the automated driving work vehicle 1 is acquired by the location information acquisition unit 13 of the work vehicle 1. It is receiving in real time. As a result, the display control unit 22 can see the current position of the automated driving work vehicle 1. Location information is acquired in real time, and the display device 2 uses the terminal-side wireless communication unit 21 and display control The unit 22 etc. functions as a position information acquisition unit capable of acquiring the position information of the automated driving work vehicle 1. It is.

[0056] In the work screen shown in Figure 11, the leftmost part of the lower display area 54d is the operation unit 51 for notification information. Since h (the icon shown in "Details" in the diagram) is located, the display control unit 22 automatically If the various conditions for starting the automatic operation of the vehicle 1 are met, the notification information operation The left side of the work unit 51h is illuminated in a specific color for completion (for example, green). Conversely, automatic driving operation If the various conditions for starting automatic driving of vehicle 1 are not met, the notification information operation unit The left side of 51h is lit in a specific color (for example, red) to indicate that it is not yet established. And then, notification information When the left side of the control unit 51h is illuminated in a specific color (for example, red) indicating that it is not active, When the user selects the notification information operation unit 51h, the display control unit 22 performs the operation shown in Figure 11. The system transitions from the work screen to the notification information screen shown in Figure 12.

[0057] In the notification information screen shown in Figure 12, the lower intermediate display area 54 in the work screen shown in Figure 11 The display content of e has been changed, and various conditions for starting the automatic driving of the automated driving work vehicle 1 have been set. Parts that are affected (steering, reverser, brakes, clutch, parking brake, sub-transmission) A status display screen 63 is shown, indicating the device, vehicle speed, and vehicle body horizontal control settings (UFO setting). The display control unit 22, in the established status display screen 63, checks for each part if the conditions are met. For the parts that are working, a specific color (for example, green) will be lit to indicate that the conditions are met. For areas that are not yet completed, a specific color (for example, red) is illuminated to indicate that the function is not yet established.

[0058] As shown in the work screens in Figures 11 and 13, in the enlarged field screen 58, the travel route 2 The deviation of the position of the automated work vehicle 1 relative to 5 is shown in Figure 13. In terms of the field, the display control unit 22 displays the travel route 25 and the automatic travel operation on the enlarged field screen 58. The current position 26 of the work vehicle 1 is displayed. The current position 26 of the automated work vehicle 1 is displayed. Depending on its position relative to road 25, the user can determine the current position of the autonomous driving work vehicle 1. It is easy to recognize how much and in which direction the position is shifted in the lateral direction of the travel path 25. This is possible. The display control unit 22, as shown in Figure 13, not only before the start of automatic driving, but also Even during automatic driving, the driving route 25 and the automatic driving work vehicle 1 are displayed on the enlarged field screen 58. The current position 26 is displayed. This allows the automated driving work vehicle 1 to move to the automated driving start position. Not only when performing positioning by moving, but also during autonomous driving, the user can use the driving path 2 It is possible to recognize how much the automated work vehicle 1 is shifted in the lateral direction relative to 5. In Figure 13, the current position 26 of the automated driving work vehicle 1, shown by the dotted line, is along the driving path 2. This indicates a state where the number is off to the right or left relative to the value of 5.

[0059] <Display overhead view> As shown in Figures 11 and 13, at least one of the following: before the start of autonomous driving and during autonomous driving. At that moment, the camera 15 of the autonomous work vehicle 1 takes pictures of the area around the vehicle. The four images of the aircraft's surroundings taken by the Mera 15 are transmitted to the display device 2 via the radio communication units 14 and 21. It is transmitted to the display device 2. The overhead image generation unit 20 in the display device 2 generates the overhead image based on the four images of the aircraft's surroundings. Next, an overhead view image 27 is generated, showing the area around the aircraft. In this embodiment, the overhead view image is generated The component 20 is provided on the display device 2, but it may also be provided on the automated driving work vehicle 1.

[0060] As shown in Figure 13, the forward portion 27a of the overhead view image 27 is a photograph of the front of the aircraft. The image used is from the first camera. Of the overhead view image 27, the right part of the aircraft 27b is the aircraft The image used is from a second camera that photographs the right side of the aircraft. Of the overhead view image 27, the left side of the aircraft Section 27c uses images taken by the third camera, which photographs the left side of the aircraft. Overhead view 2 Of the 7, the rear portion 27d of the aircraft was captured using images taken by the fourth camera that photographs the rear of the aircraft. As shown in Figures 11 and 13, the display 24 of the display device 2 shows the travel route. The images include a travel route image 25, a vehicle position image 26 showing the current location of work vehicle 1, and an overhead view image 27. And is displayed overlaid.

[0061] In this embodiment, in order to make the shooting range of the camera 15 identifiable, the shooting range of the camera is The image showing the shooting range is displayed. The shooting range image shows the shooting range of camera 15 with a solid line or This is a frame such as a dotted line. Furthermore, as shown in Figure 14, the shooting range of camera 15 can be identified. In order to display, the range travel paths 25b and 25c located within the shooting range of camera 15 and In a different manner from the out-of-range travel paths 25a and 25d, which are located outside the shooting range of camera 15. It is displayed. Specifically, it is the travel route 25a which is outside the shooting range and the work has already been completed. Areas (shown as dashed lines in the diagram) are shown in green, indicating that the work has already been completed within the shooting range. Route 25b (shown as a dashed line in the diagram) is shown in orange, and is within the shooting range and is not yet being worked on. The travel route 25c (shown as a dashed line in the diagram) is shown in yellow, and is outside the shooting range and undeveloped. The route 25d (shown as a solid line in the diagram), which is the route of the business, is shown in blue. The display method may vary in color or line type, or in duration, such as lighting or flashing. That's good too.

[0062] In this embodiment, the travel route 25 is divided into four types, and the display method is varied accordingly. However, it is sufficient to have at least two types of display modes, both inside and outside of the shooting. Also, in this embodiment, shooting The display shows a frame indicating the range, and the 25 travel routes are displayed separately by type, but at least one of them Displaying this allows you to identify the camera's field of view.

[0063] As shown in Figures 11 and 13, the position of the obstacle 28 can be set in advance. The location of obstacle 28 is displayed, but it is not immediately clear what kind of obstacle it is. In this embodiment, as shown in Figure 15, the camera 15 detects a pre-set obstacle 28. If an image is taken, the obstacle image 28 showing the captured obstacle is displayed along with the driving path image 25 and the vehicle position It is configured to be displayed overlaid on the local image 26 and the overhead image 27. Pre-set obstacles 2 The determination of whether camera 15 captured 8 is based on the fact that the position of obstacle 28 is known, and the current position of work vehicle 1 Since the shooting range of camera 15 is also known from the current position, the determination can be made using only the position coordinates. The detection results from the obstacle sensor 16 may also be included in the determination.

[0064] The overhead view image 27 in the enlarged screen 58 of the work operation screen shown in Figures 11, 13, and 15. You don't need to draw anything in the background of the field outside, and you can generate a background image based on the generated overhead image. It may be configured to display the image, or an external server may store the overall image corresponding to the field. If available, the system may be configured to display the entire image as the background. Even in such cases, the overhead view will be displayed preferentially in the foreground.

[0065] In the operation screens shown in Figures 11, 13, and 15, zooming in and out is possible. When zooming in, the entire overhead image 27 should be displayed on the display 24. It is resizable. When further resized, only a portion of the overhead image 27 is displayed on the display 24. At this magnification, the entire overhead image 27 is not visible, creating blind spots, which is preferable from a safety standpoint. This is because it cannot be said. When reducing the size, it is limited to the area that can accommodate the entire field, and the adjacent field is It doesn't display.

[0066] If an unregistered obstacle (person, object, etc.) is detected in the video feed of camera 15, The aircraft's automatic navigation is stopped, and a notification system is used to alert the user that an unregistered obstacle is present. The system may be configured in such a way. Examples of notification means include lamps and indicators installed on the aircraft. Examples include displaying information on the display 24 of the device 2 and producing sound from the display device 2.

[0067] In the operation screens shown in Figures 11, 13, and 15, by performing a switching operation... As shown in Figure 16, the overhead view image 27 and the travel route 25 are displayed in a reduced size, and other information is displayed. The system may be configured to display the information. In the example shown in Figure 16, a camera is installed on the work vehicle 1. This displays the video 29 ​​of the work machine of work vehicle 1. In the example in Figure 16, the work machine is tilling. Although it's labeled as a rice planting machine, if it were a rice transplanter, it could display images of the rice planting equipment. If it's a harvester, various sensors installed in the grain tank will display information such as the harvest yield. These are some of the things that can be done.

[0068] As described above, the automated driving system of this embodiment drives the work vehicle 1 along the driving path 25. The system includes an automatic driving control unit 12 that enables automatic driving, and a display unit 24 that displays a predetermined image. The system includes a display control unit 22 that displays the travel route 25. Image 25 and a vehicle position image showing the location of work vehicle 1 as identified using a positioning satellite system. 26 and the surroundings of work vehicle 1 generated based on images from a camera that photographs the area around work vehicle 1. The overhead view image 27 is superimposed and displayed on the display unit, which is the display screen 24.

[0069] In this way, an overhead view image 27 of the area around the work vehicle is generated based on the images of the area around the work vehicle. Therefore, images of the work vehicle's surroundings and the travel route 25 can be displayed in a single area. Therefore, compared to displaying the image of the work vehicle's surroundings and the travel path 25 separately in different areas, Since both can be displayed in large size, it becomes possible to provide an easy-to-read display. Also, the monitor can Monitoring becomes easier because you only need to look at the display area, without having to move your gaze. Therefore, it becomes possible to provide an autonomous driving system with improved convenience.

[0070] In this embodiment, the display control unit 22 further displays a shooting range image indicating the shooting range of the camera 15. The image is superimposed on the driving route image 25, the vehicle position image 26, and the overhead view image 27, and displayed on the display unit. Display on Play24.

[0071] Thus, the shooting range of camera 15 can be identified by the shooting range image, allowing for an overhead view. By looking at image 27, you can confirm that there are no abnormalities within the shooting range. Also, outside the shooting range... It may be possible to make people aware that an abnormality may be present.

[0072] In this embodiment, in the travel path image 25, the area of ​​the travel path 25 captured by the camera 15 The range of travel paths 25b and 25c located within the enclosure, and the range located outside the shooting range of camera 15. The enclosed travel routes 25a and 25d are displayed in different ways.

[0073] According to this configuration, the display method of the travel route (25a, 25b, 25c, 25d) This makes it possible to identify the shooting range of camera 15.

[0074] In this embodiment, the display control unit 22 further displays an obstacle image 28 indicating an obstacle, based on the travel time. The road image 25, the vehicle position image 26, and the overhead view image 27 are superimposed on the display unit, the display 24. The display control unit 22 can display the information when the obstacle 28 is located within the shooting range of the camera 15. At the same time, the image from camera 15 is used to display the obstacle image 28.

[0075] In this way, the observer can find out what the obstacle 28 is. If the person who pre-sets object 28 and the person who monitors the work vehicle 1 while it is in motion are different: It is effective.

[0076] The embodiments of this disclosure have been described above with reference to the drawings, but the specific configurations are as follows: It should be understood that this disclosure is not limited to the embodiments described above. This is shown not only in the description of the embodiments but also in the claims, and furthermore, it is equivalent to the claims. This includes all changes within the meaning and scope of the above. [Explanation of Symbols]

[0077] 1. Work vehicles 2 Display device 13 Location information acquisition section 12. Control Unit (Automatic Driving Control Unit) 15 Cameras 22 Display Control Unit 24 Display (Display Unit) 25. Route 26 Current location 27. Overhead view 28 Obstacles

Claims

1. An automatic driving control unit that automatically drives a work vehicle along a travel route, It comprises a display control unit that causes a predetermined image to be displayed on the display unit, The display control unit, The driving route image showing the aforementioned driving route, the vehicle position image showing the position of the work vehicle, and the overhead view image showing the area around the work vehicle are superimposed and displayed on the display unit. The display magnification of the overhead view image can be changed along with the driving route image and the vehicle position image. An automated driving system in which the range in which the overhead view image can be enlarged is limited to the range in which the entire overhead view image is displayed on the display unit.

2. The range in which the overhead view image can be reduced is limited to the range in which the entire field on which the work vehicle travels in the travel path image fits within the display unit, and adjacent fields are not displayed in the display unit, as described in claim 1 of the automatic driving system.

3. The automatic driving system according to claim 1 or 2, wherein the display control unit reduces the overhead view image when displaying an image of the work machine of the work vehicle on the display unit.

4. The display unit overlays and displays a vehicle position image showing the position of the work vehicle as it automatically travels along the travel path, a travel path image showing the travel path, and an overhead view image showing the area around the work vehicle. A display method in which the overhead view image, which is provided to be enlargeable together with the vehicle position image and the driving path image, can be enlarged to the extent that the entire overhead view image is displayed on the display unit.

Images