Display systems and work vehicles

Abstract

A display system is installed in a work vehicle provided with a vibration sensor. The display system is provided with a screen and a control device that controls the display of an image on the screen, generates an image including an input interface for a user to perform an input operation, and causes the screen to display the image. The control device changes the position of the display of the input interface on the screen on the basis of vibrations of the work vehicle detected by a vibration sensor.

Description

Technical Field

[0001] The present disclosure relates to a display system and a work vehicle.

Background Art

[0002] Techniques have been developed to improve the visibility of images displayed on a screen or a display when a vehicle vibrates or tilts due to unevenness of the road surface or acceleration / deceleration of the vehicle.

[0003] Patent Document 1 discloses a head-up display system that corrects the display position of an object to be displayed based on angular velocity information in two axial directions acquired from a gyroscope. Patent Document 2 discloses a head-up display device that enables an appropriate superimposition of a virtual image on an actual landscape according to the driving situation of a vehicle. Patent Document 3 discloses a car navigation system that enlarges and displays a display area within a certain range according to the vibration of a user's finger.

[0004] The techniques disclosed in Patent Documents 1 to 3 can all suppress the displacement of an object accompanying vehicle vibration by shifting the object in the image in the direction opposite to the direction of the displacement of the vibration (that is, in the direction canceling the displacement of the vibration).

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0006] This improves the usability of the input interface displayed on the screen when the work vehicle vibrates or tilts. [Means for solving the problem]

[0007] A display system according to one aspect of the present disclosure is a display system mounted on a work vehicle equipped with a vibration sensor, comprising: a screen; and a control device for controlling the display of an image on the screen, the control device for generating an image including an input interface for a user to perform input operations, and for displaying the image on the screen, wherein the control device changes the position of the display of the input interface on the screen based on the vibration of the work vehicle detected by the vibration sensor.

[0008] A work vehicle according to one aspect of this disclosure comprises a vibration sensor and the display system.

[0009] The comprehensive or specific embodiments of this disclosure may be implemented by apparatus, systems, methods, integrated circuits, computer programs, or computer-readable non-temporary storage media, or any combination thereof. Computer-readable storage media may include volatile storage media or non-volatile storage media. Apparatus may consist of multiple devices. If apparatus consists of two or more devices, these two or more devices may be located in a single device or in two or more separate devices. [Effects of the Invention]

[0010] According to embodiments of this disclosure, it is possible to improve the operability of the input interface displayed on the screen when the work vehicle vibrates or tilts. [Brief explanation of the drawing]

[0011] [Figure 1] This figure shows an example of an operating terminal and a group of operating switches installed inside the cabin of a work vehicle. [Figure 2] It is a block diagram illustrating the hardware configuration of the operation terminal. [Figure 3] It is a diagram for explaining the coordinate system and displacement components of the vibration of the work vehicle according to an embodiment of the present disclosure. [Figure 4A] It is a diagram showing an example of the display of an image on the screen of the operation terminal in the case of small shaking. [Figure 4B] It is a diagram showing an example of the display of an image on the screen of the operation terminal in the case of large shaking. [Figure 5A] It is a diagram showing another example of the display of an image on the screen of the operation terminal in the case of small shaking. [Figure 5B] It is a diagram showing another example of the display of an image on the screen of the operation terminal in the case of large shaking. [Figure 6] It is a schematic diagram showing an example of the configuration of the HUD unit. [Figure 7] It is a diagram showing the state of a work vehicle traveling on a bumpy field. [Figure 8A] It is a diagram showing an example of the display of an image on the screen in the case of small shaking. [Figure 8B] It is a diagram showing an example of the display of an image on the screen in the case of large shaking. [Figure 9A] It is a diagram showing another example of the display of an image on the screen in the case of small shaking. [Figure 9B] It is a diagram showing another example of the display of an image on the screen in the case of large shaking. [Figure 10] It is a perspective view showing an example of the appearance of the work vehicle. [Figure 11] It is a side view schematically showing an example of the work vehicle with the work implement attached. [Figure 12] It is a block diagram showing an example of the configuration of the work vehicle and the work implement. [Figure 13] It is a conceptual diagram showing an example of a work vehicle performing positioning by RTK-GNSS.

Mode for Carrying Out the Invention

[0012] (Definition of Terms) In the present disclosure, "agricultural machinery" means machinery used for agricultural purposes. Examples of agricultural machinery include tractors, harvesters, rice transplanters, ride-on mowers, vegetable transplanters, lawn mowers, seeders, fertilizer spreaders, and agricultural mobile robots. Not only can a work vehicle such as a tractor function as "agricultural machinery" alone, but in some cases, the entire combination of a work vehicle and an implement (attachment) mounted on or towed by the work vehicle can function as one "agricultural machinery". Agricultural machinery performs agricultural operations such as tilling, seeding, pest control, fertilizing, planting crops, or harvesting on the ground in a field. These agricultural operations may be referred to as "ground operations" or simply "operations". The act of a vehicle-type agricultural machinery traveling while performing agricultural operations may be referred to as "working travel".

[0013] "Autonomous driving" means controlling the movement of agricultural machinery through the operation of a control device, without manual operation by a driver. Agricultural machinery that performs autonomous driving is sometimes called "autonomous farm machinery" or "robot farm machinery." During autonomous driving, not only the movement of the agricultural machinery but also the actions of agricultural work (e.g., the operation of implements) may be controlled automatically. If the agricultural machinery is a vehicle-type machine, the movement of the agricultural machinery by autonomous driving is called "autonomous driving." The control device may control at least one of the following necessary actions for the movement of the agricultural machinery: steering, adjustment of movement speed, starting and stopping of movement. When controlling a work vehicle equipped with implements, the control device may also control actions such as raising and lowering the implements, and starting and stopping the operation of the implements. Movement by autonomous driving may include not only movement of agricultural machinery along a predetermined path toward a destination but also movement following a target. Agricultural machinery that performs autonomous driving may move partially based on user instructions. In addition to the autonomous driving mode, agricultural machinery that performs autonomous driving may also operate in a manual driving mode in which it is moved by manual operation by a driver. The steering of agricultural machinery by a control device, without manual intervention, is called "automatic steering." Part or all of the control device may be located outside the agricultural machinery. Communication, such as control signals, commands, or data, may take place between the external control device and the agricultural machinery. An autonomously operating agricultural machine may move autonomously while sensing its surroundings, without human intervention in controlling its movement. During autonomous movement, it may detect and avoid obstacles.

[0014] An "environmental map" is data that represents the location or area of ​​objects in the environment in which agricultural machinery operates, using a predetermined coordinate system. Environmental maps are sometimes simply referred to as "maps" or "map data." The coordinate system that defines an environmental map may be a world coordinate system, such as a geographic coordinate system fixed to the Earth. Environmental maps may include information other than location about objects in the environment (e.g., attribute information or other information). Environmental maps include various forms of maps, such as point cloud maps or grid maps. Data of local maps or submaps generated or processed in the process of constructing an environmental map is also called a "map" or "map data."

[0015] "Agricultural roads" refer to roads primarily used for agricultural purposes. Agricultural roads are not limited to asphalt-paved roads, but also include unpaved roads covered with soil or gravel. Agricultural roads include roads exclusively passable by agricultural machinery (such as tractors and other work vehicles) (including private roads) and roads that can also be used by general vehicles (passenger cars, trucks, buses, etc.). Work vehicles may travel automatically on general roads in addition to agricultural roads. General roads are roads that are maintained for the traffic of general vehicles.

[0016] (Embodiment) Embodiments of the present disclosure are described below. However, descriptions that are unnecessarily detailed may be omitted. For example, detailed descriptions of already well-known matters and redundant descriptions of substantially identical configurations may be omitted. This is to avoid the following description becoming unnecessarily verbose and to facilitate understanding for those skilled in the art. The inventors provide the accompanying drawings and the following description so that those skilled in the art can fully understand the present disclosure, and not to limit the subject matter described in the claims. In the following description, components having the same or similar function are denoted by the same reference numerals.

[0017] The following embodiments are illustrative, and the technology of this disclosure is not limited to these embodiments. For example, the numerical values, shapes, materials, steps, order of steps, and display screen layouts shown in the following embodiments are merely examples, and various modifications are possible as long as they do not create a technical inconsistency. Furthermore, it is possible to combine one embodiment with another as long as it does not create a technical inconsistency.

[0018] The display system according to the embodiment of this disclosure is mounted on a work vehicle equipped with a vibration sensor. The work vehicle is capable of being operated by a person and can be operated manually or automatically. The display system comprises a screen and a control device. Examples of vibration sensors include a gyroscope, an accelerometer, a combination of a gyroscope and an accelerometer, or an inertial measuring unit (IMU). For example, the screen of the display system may function as the touchscreen of an operating terminal, or the front windshield (or front panel) of the work vehicle that displays a virtual image formed by a head-up display (hereinafter referred to as "HUD"), or a combiner of the HUD.

[0019] The control device is configured to control the display of an image on a screen, generate an image that includes an input interface for the user to perform input operations, and display that image on the screen. The control device changes the position of the input interface display on the screen based on the vibration of the work vehicle detected by the vibration sensor. For example, if the magnitude of the vibration of the work vehicle detected by the vibration sensor is less than a threshold, the control device keeps the input interface display stationary at a first predetermined position on the screen, and if the magnitude of the vibration of the work vehicle is greater than or equal to the threshold, it moves the input interface display from the first predetermined position. The magnitude of vibration refers to the velocity, acceleration, or displacement of the vibration. In embodiments of this disclosure, unless otherwise specified, the magnitude of vibration refers to the magnitude of the vibration displacement (amount of displacement).

[0020] In the control device according to the embodiment of this disclosure, if the magnitude of vibration of the work vehicle exceeds a threshold, the display of the input interface moves from a first predetermined position in the same direction as the direction of displacement due to vibration of the work vehicle. By shifting the display of the input interface in the video in the same direction as the direction of displacement due to vibration in this way, the operability of the input interface displayed on the screen can be improved when the work vehicle vibrates or tilts.

[0021] [1. Display System] Figure 1 shows an example of an operating terminal 200 and a group of operating switches 191 installed inside the cabin of a work vehicle. Figure 2 is a block diagram illustrating the hardware configuration of the operating terminal 200.

[0022] Inside the cabin is a group of control switches 191, which includes multiple switches that can be operated by the user. The group of control switches 191 may include, for example, a switch for selecting the gear of the main or sub-transmission, a switch for switching between automatic and manual driving modes, a switch for switching between forward and reverse, a switch for switching between four-wheel drive and two-wheel drive, a switch for releasing the coupling between the left and right brakes, and a switch for raising and lowering the implement.

[0023] The operating terminal 200 is a terminal for the user to perform operations related to the driving of the work vehicle and the operation of the implement, and is also called a virtual terminal (VT). The operating terminal 200 may be equipped with a touchscreen display and / or one or more buttons. The display may be a display such as a liquid crystal or organic light-emitting diode (OLED). By operating the touchscreen of the operating terminal 200, the user can perform various operations such as switching the automatic driving mode on / off, switching the remote control mode on / off, recording or editing the environmental map, setting a target route, changing control quantities for the work vehicle such as vehicle speed or engine speed, and switching the implement on / off. At least some of these operations can also be performed by operating the group of operating switches 191. The operating terminal 200 may be configured to be detachable from the work vehicle. A user located away from the work vehicle may control the operation of the work vehicle by operating the detached operating terminal 200.

[0024] A laptop computer with a touchscreen, installed with application software necessary to change the position of the input interface display on the screen based on vibrations of the work vehicle detected by a vibration sensor, may be placed inside the cabin in place of the operating terminal 200. Alternatively, a mobile device such as a smartphone or tablet computer with the same application software installed may be placed inside the cabin.

[0025] The operating terminal 200 shown in Figure 2 comprises an input device 210, a display device 220, a control device 230, a ROM 240, a RAM 250, a storage device 260, and a communication device 270. These components are connected to each other via a bus so that they can communicate with one another.

[0026] The input device 210 is a device for converting user instructions into data and inputting it into a computer. The input device 210 may be, for example, a keyboard or a mouse. The display device 220 may be, for example, a liquid crystal display or an organic EL display. The display device 220 has a touchscreen and, in addition to displaying images, also performs the functions of the input device 210.

[0027] The control device 230 includes a processor. The processor may be a semiconductor integrated circuit including, for example, a central processing unit (CPU). The processor may be implemented by a microprocessor or microcontroller. Alternatively, the processor may be implemented by an FPGA (Field Programmable Gate Array) equipped with a CPU, a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an ASSP (Application Specific Standard Product), or a combination of two or more circuits selected from these circuits. The processor sequentially executes a computer program stored in the ROM 240, which describes a set of instructions for performing at least one process, to achieve the desired process.

[0028] ROM240 can be, for example, writable memory (e.g., PROM), rewritable memory (e.g., flash memory), or read-only memory. ROM240 stores programs that control the operation of the processor. ROM240 does not have to be a single storage medium; it may be a collection of multiple storage media. Some of these storage media may be removable memory.

[0029] RAM250 provides a workspace for temporarily unpacking the control program stored in ROM240 during boot-up. RAM250 does not need to be a single storage medium; it may be a collection of multiple storage media.

[0030] The storage device 260 may be, for example, a magnetic storage device or a semiconductor storage device. An example of a magnetic storage device is a hard disk drive (HDD). An example of a semiconductor storage device is a solid-state drive (SSD).

[0031] The communication device 270 is a communication module for communicating via a network with, for example, a cloud server for managing agricultural work, a work vehicle, or a terminal device that can be used by a user (such as an agricultural manager or farm worker). The communication device 270 can perform wired communication compliant with communication standards such as IEEE 1394 (registered trademark) or Ethernet (registered trademark). The communication device 270 may also perform wireless communication compliant with Bluetooth (registered trademark) or Wi-Fi standards, or cellular mobile communication such as 3G, 4G, or 5G.

[0032] Figure 3 is a diagram illustrating the coordinate system and displacement components of vibrations in the work vehicle 100. Figure 3 shows a three-axis XYZ coordinate system with mutually orthogonal axes. Vibrations that may occur in the work vehicle 100 are represented by displacement components along the X, Y, and Z axes and rotational components around each axis. The X axis extends in the longitudinal direction of the work vehicle 100. The Y axis extends in the vertical direction of the work vehicle 100. The Z axis extends in the lateral direction of the work vehicle 100. Translations along the X, Y, and Z axes without rotation are called longitudinal displacement (X), vertical displacement (Y), and lateral displacement (Z), respectively. Rotational displacement includes rolling, pitching, and yawing. Rotation around the X axis is called rolling, rotation around the Y axis is called yawing, and rotation around the Z axis is called pitching.

[0033] In this specification, the vibration or tilting of a work vehicle may be described as "the work vehicle shaking." A work vehicle shakes due to, for example, unevenness of the off-road surface, including fields, acceleration or deceleration of the vehicle, or turning. Shaking can be described by three-axis translation, three-axis rotational displacement, or any combination thereof. In embodiments of this disclosure, in particular, shaking described by lateral displacement (Z), rolling, or a combination thereof is referred to as "lateral shaking," and shaking described by vertical displacement (Y), pitching, or a combination thereof is referred to as "longitudinal shaking."

[0034] The work vehicle according to the embodiment of this disclosure is equipped with an IMU as a vibration sensor, as described later. The IMU includes an accelerometer and a gyroscope. The accelerometer measures the translation of the three axes mentioned above. The gyroscope measures the rotational displacement of the three axes mentioned above. For example, the velocity is obtained by integrating the acceleration in the Y-axis direction measured by the accelerometer once, and the vertical displacement (Y) is obtained by integrating the velocity once again. In addition, the rotation angle is obtained by integrating the angular velocity of rotation around the Z-axis measured by the gyroscope once. In this way, the tilt and vibration of the work vehicle can be measured by the IMU.

[0035] The following describes examples of the operation of the control device in the event of lateral or vertical shaking. However, when a work vehicle is actually traveling off-road, shaking can occur in any direction. In the embodiments of this disclosure, the degree of shaking is distinguished based on a threshold. Shaking where the magnitude of the vibration displacement (or the magnitude of the vibration) is less than the threshold is called a relatively "small shaking," and shaking where the magnitude of the vibration displacement is greater than or equal to the threshold is called a relatively "large shaking." In the case of large shaking, the vibration frequency of the work vehicle is, for example, several Hz to several tens of Hz. Therefore, the driver sitting in the driver's seat may be shaken relatively greatly by the shaking of the work vehicle.

[0036] Figure 4A shows an example of the image display on the screen S of the operating terminal 200 in the case of a small tremor. Figure 4B shows an example of the image display on the screen S of the operating terminal 200 in the case of a large tremor. Both Figure 4A and Figure 4B show a two-axis uv coordinate system fixed on the screen S, with the upper left corner of the screen S as the origin. For reference, the Y and Z axes of the work vehicle's XYZ coordinate system are also shown. In the embodiments of this disclosure, for the sake of explanation, the direction in which the u axis extends is parallel to the direction in which the Y axis extends, and the direction in which the v axis extends is parallel to the direction in which the Z axis extends.

[0037] In the illustrated example, screen S is a touchscreen. The image on screen S includes an input interface 280 for the user to perform input operations. The input interface 280 includes an input unit for inputting at least one piece of information, such as information about the work vehicle, information about the work vehicle's movement, and information about the work performed by the work vehicle. Information about the work vehicle includes, for example, vehicle speed (km / h), tilling depth (cm), and engine speed (rpm). Information about the work vehicle's movement includes, for example, setting a target route, correcting the target route, switching the automatic driving mode on / off, and emergency stop. Examples of information about the work performed by the work vehicle may include a work plan or the type of work performed by the implement.

[0038] The image on screen S may further include a display 290 of at least one of the following pieces of information: information about the work vehicle, information about the work vehicle's movement, and information about the work performed by the work vehicle. Hereafter, these displays of information will simply be referred to as "display 290". The information about the work vehicle included in display 290 may include, in addition to the examples described above, fuel level (%), water temperature (°C), and transmission oil temperature (°C). An example of the information about the work vehicle's movement included in display 290 is the position information of the work vehicle traveling along a target route set in the field. The image on screen S in the illustrated example includes a display 290 of a map image showing the current position of the work vehicle. An example of the information about the work performed by the work vehicle included in display 290 is the work history. The image on screen S may further include camera footage taken by a camera mounted on the work vehicle.

[0039] The control device 230 controls the display of images on the screen S. The control device 230 generates an image including the input interface 280 and displays the image on the screen S. The control device 230 may further generate an image including the display 290.

[0040] The input interface 280 illustrated in Figure 4A includes buttons that serve as inputs for zooming in or out of the display, and setting change buttons for changing the vehicle speed and engine RPM, respectively. The user can perform the desired operation by touching the touchscreen and operating the + and - buttons or the zoom in and out buttons. However, the image on screen S illustrated in Figure 4A is just an example and may include various video displays as described above.

[0041] The control device 230 changes the position of the display on the input interface 280 on the screen S based on the vibration of the work vehicle detected by the vibration sensor. Specifically, if the magnitude of the vibration of the work vehicle detected by the vibration sensor (e.g., the amount of vibration displacement) is less than a threshold, that is, in the case of a small tremor, the control device 230 keeps the display on the input interface 280 stationary at a first predetermined position on the screen S. The threshold can be set, for example, in the range of 5 cm to 10 cm. However, this range of the threshold is just an example and is not limited thereto. For example, the display on the input interface 280 may be moved when there is vibration of the work vehicle. Furthermore, if the magnitude of the vibration of the work vehicle is less than a threshold, the control device 230 keeps the display 290 stationary at a second predetermined position on the screen S. In this way, even if the work vehicle is displaced in a certain direction, if the magnitude of the vibration is less than a threshold, the control device 230 keeps the display on the input interface 280 and the display 290 stationary at the first and second predetermined positions, respectively.

[0042] In the example shown in Figure 4A, in the case of small tremors, the input interface 280 is located approximately in the center of the screen S. The first predetermined position on the screen S in this example is approximately in the center of the screen S. Most of the display 290 is located in the left-hand region of the screen S. The second predetermined position on the screen S in this example is located to the left of the center of the screen. However, the first and second predetermined positions are not limited to the example shown. The first and second predetermined positions are identified by their position coordinates in the uv coordinate system.

[0043] Space is provided above, below, to the left and right of the input interface 280 on screen S to allow for movement of the input interface 280. In the illustrated example, the input interface 280 is displayed at the forefront, and a portion of it overlaps with a portion of the display 290. Displaying the input interface 280 at the forefront can improve its usability.

[0044] The control device 230 moves the display of the input interface 280 from a first predetermined position when the magnitude of the vibration of the work vehicle exceeds a threshold, that is, when there is a large tremor. More specifically, when there is a large tremor, the control device 230 moves the display of the input interface 280 from the first predetermined position in the same direction as the direction of displacement of the vibration of the work vehicle. The amount of movement of the input interface 280 may be determined, for example, by the amount of displacement and the size of the margins on the screen S above, below, left, and right of the input interface 280.

[0045] Figure 4B shows an example of how the image on screen S is displayed when lateral swaying occurs. For example, if the work vehicle is displaced in the -Z direction and the magnitude of the vibration is greater than or equal to a threshold, the control device 230 moves the display of the input interface 280 from the first predetermined position in the same direction as the direction of the work vehicle's vibration displacement (-Z direction), i.e., in the -v direction. As a result, the input interface 280 moves in the -v direction from the first predetermined position illustrated in Figure 4A due to the lateral swaying of the work vehicle. As another example, if the work vehicle is displaced in the +Z direction and the magnitude of the vibration is greater than or equal to a threshold, the control device 230 moves the display of the input interface 280 from the first predetermined position in the +v direction. Alternatively, if the work vehicle is displaced in the upper right direction (direction P shown in the figure) and the magnitude of the vibration is greater than or equal to a threshold, the control device 230 moves the display of the input interface 280 from the first predetermined position in the direction P shown in the figure.

[0046] When the magnitude of vibration exceeds a threshold, i.e., when there is a large sway, the driver sitting in the driver's seat will be shaken relatively large at a low frequency. For example, the driver may perform a turning maneuver on a headland while changing speed or engine RPM. However, the work vehicle is prone to shaking during headland turns, and therefore the operability of the input interface displayed on the screen was not very good.

[0047] According to embodiments of this disclosure, in the event of a large tremor, the control device 230 moves the display of the input interface 280 from a first predetermined position in the same direction as the direction of the vibration displacement of the work vehicle. As a result, the input interface 280 moves in the direction of the vibration displacement of the work vehicle. This makes it easier for the driver, who is being shaken by the vibration or tilt of the work vehicle, to operate the input interface 280 displayed on the screen S, for example, with their finger.

[0048] Figure 5A shows another example of the image displayed on the screen S of the control terminal 200 during a small tremor. Figure 5B shows another example of the image displayed on the screen of the control terminal 200 during a large tremor. The uv coordinate system is shown in both Figure 5A and Figure 5B, and for reference, the Y and Z axes of the work vehicle's XYZ coordinate system are also shown.

[0049] The image on screen S in the illustrated example includes an input interface 280 and a display 290. The input interface 280 includes a numeric keypad for the user to input numbers. The display 290 includes information about the work vehicle.

[0050] In the example shown in Figure 5A, in the case of small tremors, the input interface 280 is located approximately in the center of the screen. The first predetermined position on the screen S in this example is approximately in the center of the screen S. The display 290 is located in the left-hand region of the screen S. The second predetermined position on the screen S in this example is located to the left of the center of the screen. However, the first and second predetermined positions are not limited to the examples shown. Similar to the example shown in Figure 4A, space is provided above, below, to the left and to the right of the input interface 280 on the screen S for moving the input interface 280.

[0051] Figure 5B shows an example of how the image on screen S is displayed when lateral shaking occurs. For example, if the work vehicle is displaced in the +Z direction and the magnitude of the vibration is greater than or equal to a threshold, the control device 230 moves the display of the input interface 280 from the first predetermined position in the same direction as the direction of the vibration displacement of the work vehicle (+Z direction), i.e., in the +v direction. As a result, the input interface 280 moves in the +v direction from the first predetermined position illustrated in Figure 5A due to the lateral shaking of the work vehicle.

[0052] If the magnitude of vibration of the work vehicle exceeds a threshold, i.e., in the case of a large sway, the control device 230 may move the display 290 from the second predetermined position in the same direction as the displacement of the work vehicle's vibration. As illustrated in Figure 5B, if the work vehicle is displaced in the +Z direction and the magnitude of vibration exceeds a threshold, the control device 230 may move the display 290 from the second predetermined position in the +v direction. In this example, the display 290 also moves in the +v direction from the first predetermined position illustrated in Figure 5A due to the lateral sway of the work vehicle. Alternatively, if the work vehicle is displaced in the upper right direction (direction P shown) and the magnitude of vibration exceeds a threshold, the control device 230 may move the display 290 from the second predetermined position in the direction P shown. The amount of movement of the display 290 can be determined, for example, according to the amount of displacement and the size of the margins on the screen S in the top, bottom, left, and right directions of the display 290. The amount of movement of the input interface 280 and the amount of movement of the display 290 may be the same or different from each other.

[0053] A display system according to the embodiments of this disclosure may include a light source controlled by a control device and an optical system that receives light emitted from the light source and forms a virtual image in front of a screen. In other words, the display system may include a HUD. HUDs, which project information within a person's field of view, are used to assist driving by displaying information on the front windshield of a vehicle.

[0054] Figure 6 is a schematic diagram showing an example of a HUD unit configuration. One type of HUD is one that uses a virtual image optical system, which will be described below. However, the configuration of the HUD unit is not limited to the example shown in Figure 6.

[0055] The HUD unit 400 comprises a light source 410, a transmissive screen 420, a field lens 430, and a combiner 440. The optical system of the HUD unit 400 has the transmissive screen 420, the field lens 430, and the combiner 440, and may further include a MEME mirror, a movable lens, and the like. The HUD unit 400 is mounted, for example, on the ceiling surface of the roof inside the cabin of a work vehicle.

[0056] A light beam emitted from the light source 410 is focused by the transmissive screen 420 to form a real image. The transmissive screen 420 functions as a secondary light source, directing the focused light beam toward the combiner 440 so that its illumination area is approximately rectangular. The combiner 440 forms a virtual image based on the illuminated light beam. This allows the driver to view the image along with the scenery through the combiner 440.

[0057] The light source 410 is a device that displays images. The light source 410 is configured to emit display light toward a transmissive screen 420. For example, known methods for displaying images include DLP (Digital Light Processing) and laser projectors. The light source 410 may have a laser projector and a MEME mirror that scans the light beam emitted from the laser projector. An example of a laser projector is an RGB laser projector.

[0058] The transmissive screen 420 has a microlens array on the light-receiving surface side. The transmissive screen 420 functions to broaden the incident beam. The field lens 430 is positioned between the transmissive screen 420 and the combiner 440, and is located near the transmissive screen 420. The field lens 430 is formed, for example, from a convex lens and changes the direction of propagation of the light beam emitted from the transmissive screen 420. By using the field lens 430, the efficiency of light utilization can be further increased. However, the field lens 430 is not essential.

[0059] A half-mirror is commonly used for the combiner 440, but a holographic element may also be used. The combiner 440 reflects the divergent light beam from the transmissive screen 420 to form a virtual image of light. The combiner 440 has the function of displaying the image formed on the transmissive screen 420 at a distance, and also has the function of superimposing the image onto the scenery. Hereinafter, the combiner 440 may be referred to as the screen. This allows the driver to view the image along with the scenery through the combiner 440. In other words, the driver can view the image projected on the screen along with the scenery. Depending on the curvature of the combiner 440, the size of the virtual image or the position where the virtual image is formed can be changed.

[0060] The control device 450 has the same structure as the control device 230 of the aforementioned operating terminal 200 and is configured to perform the same functions. Therefore, a detailed explanation is omitted.

[0061] Figure 8A shows an example of the image displayed on screen S during a small tremor. Figure 8B shows an example of the image displayed on screen S during a large tremor. The uv coordinate system is shown in both Figure 8A and Figure 8B, and for reference, the Y and Z axes of the work vehicle's XYZ coordinate system are also shown.

[0062] In the illustrated example, screen S is the combiner 440 of the HUD unit 400. However, as will be described later, the front windshield of the work vehicle may function as the screen instead of the combiner. The image on screen S includes an input interface 480 for the user to perform input operations. In the illustrated example, the input interface 480 includes an operation button for instructing the work vehicle to start autonomous driving, and an operation button for emergency stopping the work vehicle while it is driving autonomously. The image on screen S in the illustrated example further includes a display 491 (hereinafter simply referred to as "display 491") related to information about the work vehicle and a display 492 (hereinafter simply referred to as "display 492") related to information about the work vehicle's movement. In this example, display 492 is a display of a line indicating the target route. The input interface 480, display 491, and display 492 are superimposed on the external scenery of the front windshield 500. Of these displays, the input interface 480 may be displayed in the foreground.

[0063] In embodiments of this disclosure, an input interface 480 (e.g., an operation button) displayed on the screen S can be operated remotely. For example, the display system may include an infrared camera for detecting the driver's line of sight. The infrared camera is installed, for example, above the front windshield 500 so as to be located in front of the driver. The control device 450 processes data output from the infrared camera to detect the driver's line of sight. When the control device 450 determines that the driver's line of sight is directed towards an operation button displayed on the screen S, it can cause the work vehicle to perform a predetermined operation assigned to that operation button. An example of an algorithm for remote operation based on the driver's line of sight is described in Japanese Patent Application Publication No. 2022-72453. All disclosures of Japanese Patent Application Publication No. 2022-72453 are incorporated herein by reference.

[0064] In the example shown in Figure 8A, the input interface 480 is located below the screen S during small tremors. The first predetermined position on the screen S in this example is below the screen S. Display 491 is located in the upper left region of the screen S. The second predetermined position on the screen S in this example is located in the upper left region of the screen S. However, the first and second predetermined positions are not limited to the illustrated example. Display 492 is displayed along a target path set on the ground surface of the field.

[0065] Figure 7 shows a work vehicle 100 traveling on an undulating field. The right and left sides of Figure 7 show the work vehicle 100 approaching an uphill and downhill slope, respectively. The work vehicle 100 shown on the right and left sides of Figure 7 pitches by an angle θ in the upward and downward directions, respectively. Figures 8A and 8B show examples of superimposed images of the work vehicle 100 shown on the right side of Figure 7 before and after it approaches an uphill slope, respectively.

[0066] Figure 8B shows an example of how the image on screen S is displayed when vertical shaking occurs. As shown in Figure 7, when the work vehicle is displaced in the +Y direction (upward direction) due to pitching and the magnitude of the vibration is greater than or equal to a threshold, the control device 450 moves the display of the input interface 480 from the first predetermined position in the same direction as the direction of the vibration displacement of the work vehicle (+Y direction), i.e., in the -u direction. As a result, the input interface 480 moves from the bottom to the top of screen S due to the vertical shaking of the work vehicle. As another example, when the work vehicle is displaced in the upper right direction (direction P shown in the figure) and the magnitude of the vibration is greater than or equal to a threshold, the control device 450 moves the display of the input interface 480 from the first predetermined position in the direction P shown in the figure.

[0067] If the magnitude of the vibration of the work vehicle exceeds a threshold, the control device 450 may move each of the displays 491 and 492 from their respective second predetermined positions in the opposite direction to the direction of the vibration displacement of the work vehicle. In other words, the control device 450 may move each of the displays 491 and 492 from their respective second predetermined positions in a direction that counteracts the vibration displacement. This maintains the display positions of each of the displays 491 and 492 relative to the landscape. The amount of movement of the displays 491 and 492 is determined based on the amount of vibration displacement.

[0068] As illustrated in Figure 7, if the work vehicle is displaced in the +Y direction (upward direction) due to pitching and the magnitude of the vibration is greater than or equal to a threshold, the control device 450 may move each of the displays 491 and 492 from their second predetermined positions in the opposite direction to the direction of the work vehicle's vibration displacement (+Y direction), i.e., in the +u direction. As a result, each of the displays 491 and 492 moves in the opposite direction to the direction in which the input interface 480 moves due to the vertical swaying of the work vehicle. As another example, if the work vehicle is displaced in the lower left direction (Q direction in the figure) and the magnitude of the vibration is greater than or equal to a threshold, the control device 450 may move each of the displays 491 and 492 from their second predetermined positions in the upper right direction (P direction in the figure).

[0069] According to the HUD unit 400 of the embodiment of this disclosure, in the event of a large tremor, the control device 450 moves the display of the input interface 480 from the first predetermined position in the same direction as the direction of displacement of the work vehicle's vibration. As a result, the input interface 480 moves in the direction of displacement of the work vehicle's vibration. For this reason, the driver, who is being shaken by the vibration or tilt of the work vehicle, can easily operate the input interface 480 displayed on the screen S, for example, with their eyes. Furthermore, in the event of a large tremor, the control device 450 may move each of the displays 491 and 492 from the second predetermined position in the opposite direction to the direction of displacement of the work vehicle's vibration. This is advantageous in that it can reduce the superposition misalignment of the display 491 relative to the scenery, and in particular in that it can reduce the superposition misalignment of the display 492 relative to the field ground.

[0070] Figure 9A shows another example of the image displayed on screen S during a small tremor. Figure 9B shows another example of the image displayed on screen S during a large tremor. Both Figure 9A and Figure 9B show the uv coordinate system, and for reference, the Y and Z axes of the work vehicle's XYZ coordinate system are also shown. Figures 9A and 9B show examples of superimposed images before and after the work vehicle 100 on the left, shown in Figure 7, approaches a downhill slope.

[0071] In the illustrated example, the front window 500 functions as a screen S. In this example, unlike the example shown in Figure 8B, the control device 450 dynamically changes the position of the image display area 600 in the uv coordinate system of the screen S in the event of large shaking. In other words, the control device 450 dynamically changes the relative position of the display area 600 to the front window 500 in response to the shaking of the work vehicle. The movement of the display area 600 on the screen S can be achieved by changing the angle of, for example, a MEMS mirror and / or a movable lens included in the optical system of the HUD unit 400.

[0072] Figure 9B shows an example of the image display on screen S when vertical vibration occurs. As shown in Figure 7, when the work vehicle is displaced in the -Y direction (downward direction) due to pitching and the magnitude of the vibration is greater than or equal to a threshold, the control device 450 moves the display of the input interface 480 from the first predetermined position in the same direction as the direction of the vibration displacement of the work vehicle (-Y direction), i.e., in the +u direction. As a result, when focusing on the uv coordinate system, the input interface 480 moves downward from the first predetermined position shown in Figure 9A due to the vertical vibration of the work vehicle. Similarly, when focusing on the display area 600, the input interface 480 also moves downward from the first predetermined position shown in Figure 9A. Furthermore, the control device 450 may move displays 491 and 492, respectively, from the second predetermined position in the opposite direction to the direction of the vibration displacement of the work vehicle (-Y direction), i.e., in the -u direction. As a result, when focusing on the uv coordinate system, displays 491 and 492 each move upward from the second predetermined position shown in Figure 9A. On the other hand, when focusing on the display area 600, the display positions of displays 491 and 492 do not change before and after the displacement. As illustrated in Figure 9B, when the input interface 480 and display 491 overlap, the input interface 480 may be displayed in the foreground.

[0073] [2. Examples of agricultural machinery configurations] The work vehicle according to the embodiment of this disclosure includes a vibration sensor and the display system described above.

[0074] Referring to Figures 10 to 13, embodiments of the technology disclosed herein, applied to work vehicles with autonomous driving capabilities, such as tractors, which are an example of agricultural machinery, will be mainly described. However, autonomous driving capabilities are not essential. The technology disclosed herein can be applied not only to agricultural machinery such as tractors but also to construction machinery. Below, as an example, an embodiment in which a driving control system for realizing autonomous driving capabilities is mounted on the work vehicle will be described. At least some of the functions of the driving control system may be implemented in other devices (e.g., a server) that communicate with the work vehicle.

[0075] Figure 10 is a perspective view showing an example of the external appearance of the work vehicle 100 in an embodiment of the present disclosure. Figure 11 is a schematic side view showing an example of the work vehicle 100 with the implement 300 attached. The work vehicle 100 in an embodiment of the present disclosure is an agricultural tractor (work vehicle) with the implement 300 attached. The work vehicle 100 is not limited to a tractor, nor is it necessary for the implement 300 to be attached.

[0076] As shown in Figure 11, the work vehicle 100 comprises a vehicle body 101, a prime mover (engine) 102, and a transmission 103. The vehicle body 101 is provided with a running gear including wheels 104 with tires, and a cabin 105. The running gear includes four wheels 104, axles that rotate the four wheels, and brakes that brake each axle. The wheels 104 include a pair of front wheels 104F and a pair of rear wheels 104R. Inside the cabin 105 are a driver's seat 107, a steering gear 106, an operating terminal 200, and a group of switches for operation. When the work vehicle 100 is performing work in a field, one or both of the front wheels 104F and the rear wheels 104R may be replaced with multiple wheels (crawlers) fitted with tracks instead of wheels with tires.

[0077] The work vehicle 100 may include at least one sensing device for sensing the environment around the work vehicle 100, and a processing device for processing sensing data output from at least one sensing device. In the example shown in Figure 11, the work vehicle 100 includes multiple sensing devices. The sensing devices include multiple cameras 120, a LiDAR sensor 140, and multiple obstacle sensors 130.

[0078] Cameras 120 may be installed, for example, on the front, rear, left, and right sides of the work vehicle 100. Cameras 120 capture images of the environment around the work vehicle 100 and generate image data. The images acquired by cameras 120 can be output to a processing device mounted on the work vehicle 100 and transmitted to a terminal device for remote monitoring. The images can be displayed, for example, on the screen of an operating terminal 200. Cameras 120 may also be used to generate images for recognizing surrounding features or obstacles, white lines, signs, or markings when the work vehicle 100 is traveling on a road outside the field (agricultural road or public road).

[0079] In the example shown in Figure 11, the LiDAR sensor 140 is located at the lower front of the vehicle body 101. The LiDAR sensor 140 may be located in other positions. While the work vehicle 100 is mainly traveling outside the field, the LiDAR sensor 140 repeatedly outputs sensor data indicating the distance and direction to each measurement point of objects in the surrounding environment, or the two-dimensional or three-dimensional coordinate values ​​of each measurement point. The sensor data output from the LiDAR sensor 140 is processed by the control device of the work vehicle 100. The control device can estimate the self-position of the work vehicle 100 by matching the sensor data with an environmental map. The control device can further detect objects such as obstacles present around the work vehicle 100 based on the sensor data. The control device can also generate or edit an environmental map using algorithms such as SLAM (Simultaneous Localization and Mapping). The work vehicle 100 may be equipped with multiple LiDAR sensors located in different positions and oriented in different directions.

[0080] The multiple obstacle sensors 130 shown in Figure 11 are located at the front and rear of the cabin 105. Obstacle sensors 130 may also be located in other areas. For example, one or more obstacle sensors 130 may be provided at any location on the sides, front, and rear of the vehicle body 101. Obstacle sensors 130 may include, for example, laser scanners or ultrasonic sonar. Obstacle sensors 130 are used to detect surrounding obstacles during autonomous driving and to stop or bypass the work vehicle 100. A LiDAR sensor 140 may be used as one of the obstacle sensors 130.

[0081] The work vehicle 100 further includes a GNSS unit 110. The GNSS unit 110 includes a GNSS receiver. The GNSS receiver may include an antenna that receives signals from GNSS satellites and a processor that calculates the position of the work vehicle 100 based on the signals received by the antenna. The GNSS unit 110 receives satellite signals transmitted from multiple GNSS satellites and performs positioning based on the satellite signals. GNSS is a general term for satellite positioning systems such as GPS (Global Positioning System), QZSS (Quasi-Zenith Satellite System, e.g., Michibiki), GLONASS, Galileo, and BeiDou. In embodiments of this disclosure, the GNSS unit 110 is located on top of the cabin 105, but it may be located in other locations.

[0082] The GNSS unit 110 may include an IMU. The signal from the IMU can be used to supplement the position data. The IMU can measure the tilt and minute movements of the work vehicle 100. By using the data acquired by the IMU to supplement the position data based on satellite signals, the positioning performance can be improved.

[0083] The control device of the work vehicle 100 may use sensing data acquired by sensing devices such as a camera 120 or LiDAR sensor 140 for positioning, in addition to the positioning results from the GNSS unit 110. If there are features that function as characteristic points in the environment in which the work vehicle 100 travels, such as farm roads, forest roads, public roads, or orchards, the position and orientation of the work vehicle 100 can be estimated with high accuracy based on the data acquired by the camera 120 or LiDAR sensor 140 and an environmental map stored in a storage device in advance. By correcting or supplementing the position data based on satellite signals using the data acquired by the camera 120 or LiDAR sensor 140, the position of the work vehicle 100 can be determined with even higher accuracy.

[0084] The prime mover 102 may be, for example, a diesel engine. An electric motor may be used instead of a diesel engine. The transmission 103 can change the propulsion force and travel speed of the work vehicle 100 by shifting gears. The transmission 103 can also switch the work vehicle 100 between forward and reverse.

[0085] The steering system 106 includes a steering wheel, a steering shaft connected to the steering wheel, and a power steering system that assists steering by the steering wheel. The front wheels 104F are steering wheels, and the direction of travel of the work vehicle 100 can be changed by changing their steering angle (also referred to as the "steering angle"). The steering angle of the front wheels 104F can be changed by operating the steering wheel. The power steering system includes a hydraulic system or electric motor that supplies auxiliary force to change the steering angle of the front wheels 104F. When automatic steering is performed, the steering angle is automatically adjusted by the force of the hydraulic system or electric motor under control from a control device located inside the work vehicle 100.

[0086] A coupling device 108 is provided at the rear of the vehicle body 101. The coupling device 108 includes, for example, a three-point support device (also referred to as a "three-point link" or "three-point hitch"), a PTO (Power Take Off) shaft, a universal joint, and a communication cable. The coupling device 108 allows the work implement 300 to be attached to and detached from the work vehicle 100. The coupling device 108 can change the position or orientation of the work implement 300 by raising and lowering the three-point link, for example, by a hydraulic system. Power can also be supplied from the work vehicle 100 to the work implement 300 via the universal joint. The work vehicle 100 can pull the work implement 300 and cause the work implement 300 to perform a predetermined operation. The coupling device may also be provided at the front of the vehicle body 101. In that case, an implement can be connected to the rear of the work vehicle 100.

[0087] The implement 300 shown in Figure 11 is a rotary tiller, but the implement 300 is not limited to a rotary tiller. For example, any implement such as a seeder, spreader, transplanter, mower, rake, baler, harvester, spreader, or harrow can be connected to the work vehicle 100 and used.

[0088] Figure 12 is a block diagram showing an example configuration of a work vehicle 100 and a work machine 300. The work vehicle 100 and the work machine 300 can communicate with each other via a communication cable included in the coupling device 108. The work vehicle 100 can communicate via a network with, for example, terminal devices and a server that manages agricultural work.

[0089] In the example shown in Figure 12, the work vehicle 100 includes a GNSS unit 110, a camera 120, an obstacle sensor 130, a LiDAR sensor 140, and an operating terminal 200, as well as a group of sensors 150 for detecting the operating status of the work vehicle 100, a control system 160, a communication device 190, a group of operating switches 191, a buzzer 192, and a drive device 193. These components are connected to each other via a bus for communication. The GNSS unit 110 includes a GNSS receiver 111, an RTK receiver 112, an IMU 115, and a processing circuit 116. The group of sensors 150 includes a steering wheel sensor 152, a steering angle sensor 154, and an axle sensor 156. The control system 160 includes a storage device 170 and a control device 180. The control device 180 includes a plurality of electronic control units (ECUs) 181 to 185. The work machine 300 includes a drive unit 340, a control device 380, and a communication device 390. Figure 12 shows components that are relatively highly relevant to the operation of the work vehicle 100's automatic driving function, and other components are not shown.

[0090] The GNSS receiver 111 in the GNSS unit 110 receives satellite signals transmitted from multiple GNSS satellites and generates GNSS data based on the satellite signals. The GNSS data is generated in a predetermined format, such as NMEA-0183 format. The GNSS data may include, for example, the identification number, elevation angle, azimuth angle, and received signal strength of each satellite from which the satellite signal was received.

[0091] The GNSS unit 110 shown in Figure 3 uses RTK (Real Time Kinematic)-GNSS to position the work vehicle 100. Figure 13 is a conceptual diagram showing an example of a work vehicle 100 performing positioning using RTK-GNSS. In RTK-GNSS positioning, in addition to satellite signals transmitted from multiple GNSS satellites 50, a correction signal transmitted from a base station 60 is used. The base station 60 may be installed near the field where the work vehicle 100 is performing its work (for example, within 10 km of the work vehicle 100). Based on the satellite signals received from multiple GNSS satellites 50, the base station 60 generates a correction signal, for example, in RTCM format and transmits it to the GNSS unit 110. The RTK receiver 112 includes an antenna and a modem and receives the correction signal transmitted from the base station 60. The processing circuit 116 of the GNSS unit 110 corrects the positioning result from the GNSS receiver 111 based on the correction signal. By using RTK-GNSS, it is possible to perform positioning with an accuracy of, for example, a few centimeters. Position information, including latitude, longitude, and altitude, is acquired by high-precision positioning using RTK-GNSS. The GNSS unit 110 calculates the position of the work vehicle 100 at a frequency of, for example, 1 to 10 times per second.

[0092] Furthermore, the positioning method is not limited to RTK-GNSS; any positioning method that can obtain the necessary accuracy of positional information (such as interferometric positioning or relative positioning) can be used. For example, positioning using VRS (Virtual Reference Station) or DGPS (Differential Global Positioning System) may be performed. If the necessary accuracy of positional information can be obtained without using the correction signal transmitted from the base station 60, the positional information may be generated without using the correction signal. In that case, the GNSS unit 110 does not need to be equipped with an RTK receiver 112.

[0093] Even when using RTK-GNSS, in locations where correction signals from the base station 60 cannot be obtained (for example, on a road far from the field), the position of the work vehicle 100 is estimated by other means, without relying on signals from the RTK receiver 112. For example, the position of the work vehicle 100 can be estimated by matching data output from the LiDAR sensor 140 and / or camera 120 with a high-precision environmental map.

[0094] The GNSS unit 110 in the embodiments of this disclosure further comprises an IMU 115. The IMU 115 may include a 3-axis accelerometer and a 3-axis gyroscope. The IMU 115 may also include an orientation sensor, such as a 3-axis geomagnetic sensor. The IMU 115 functions as a motion sensor and can output signals indicating various quantities such as acceleration, velocity, displacement, and attitude of the work vehicle 100. The processing circuit 116 can estimate the position and orientation of the work vehicle 100 with higher accuracy based on the signals output from the IMU 115 in addition to the satellite signals and correction signals. The signals output from the IMU 115 can be used to correct or complement the position calculated based on the satellite signals and correction signals. The IMU 115 outputs signals at a higher frequency than the GNSS receiver 111. Using these high-frequency signals, the processing circuit 116 can measure the position and orientation of the work vehicle 100 at a higher frequency (e.g., 10 Hz or higher). Instead of the IMU 115, a 3-axis accelerometer and a 3-axis gyroscope may be provided separately. The IMU115 may be provided as a separate device from the GNSS unit 110.

[0095] In the embodiments of this disclosure, the IMU 115 functions as the vibration sensor described above. However, the work vehicle 100 may be equipped with a different IMU as a vibration sensor than the IMU 115. The control device 230 or 450 described above is communicated with the IMU 115, for example, via a bus. This allows the control device 230 or 450 to acquire data output from the IMU 115 and measure the magnitude of vibration of the work vehicle 100.

[0096] Camera 120 is an imaging device that captures the environment around the work vehicle 100. Camera 120 includes an image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). Camera 120 may also include an optical system including one or more lenses and a signal processing circuit. While the work vehicle 100 is in motion, Camera 120 captures the environment around the work vehicle 100 and generates image (e.g., video) data. Camera 120 can capture video at a frame rate of, for example, 3 frames per second (fps) or higher. The images generated by Camera 120 can be used, for example, when a remote observer uses a terminal device to check the environment around the work vehicle 100. The images generated by Camera 120 may also be used for positioning or obstacle detection. As shown in Figure 11, multiple cameras 120 may be installed at different locations on the work vehicle 100, or a single camera may be installed. A visible light camera that generates visible light images and an infrared camera that generates infrared images may be provided separately. Both the visible light camera and the infrared camera may be provided as cameras that generate surveillance images. The infrared camera can also be used for detecting obstacles at night.

[0097] The obstacle sensor 130 detects objects present around the work vehicle 100. The obstacle sensor 130 may include, for example, a laser scanner or an ultrasonic sonar. The obstacle sensor 130 outputs a signal indicating the presence of an obstacle when an object is closer than a predetermined distance from the obstacle sensor 130. Multiple obstacle sensors 130 may be installed at different locations on the work vehicle 100. For example, multiple laser scanners and multiple ultrasonic sonars may be placed at different locations on the work vehicle 100. By providing many such obstacle sensors 130, blind spots in monitoring obstacles around the work vehicle 100 can be reduced.

[0098] The steering wheel sensor 152 measures the rotation angle of the steering wheel of the work vehicle 100. The steering angle sensor 154 measures the steering angle of the front wheels 104F, which are the steering wheels. The values ​​measured by the steering wheel sensor 152 and the steering angle sensor 154 are used for steering control by the control device 180.

[0099] The axle sensor 156 measures the rotational speed of the axle connected to the wheel 104, i.e., the number of rotations per unit time. The axle sensor 156 may be a sensor that utilizes, for example, a magnetoresistive element (MR), a Hall element, or an electromagnetic pickup. The axle sensor 156 outputs a numerical value indicating, for example, the number of rotations of the axle per minute (in rpm). The axle sensor 156 is used to measure the speed of the work vehicle 100.

[0100] The drive unit 193 includes various devices necessary for the movement of the work vehicle 100 and the driving of the work equipment 300, such as the prime mover 102, the transmission 103, the steering device 106, and the coupling device 108. The prime mover 102 may be an internal combustion engine, such as a diesel engine. The drive unit 193 may also be equipped with an electric motor for traction, either in place of or in conjunction with the internal combustion engine.

[0101] The buzzer 192 is an audio output device that emits a warning sound to notify of an abnormality. For example, the buzzer 192 emits a warning sound when an obstacle is detected during autonomous driving. The buzzer 192 is controlled by the control device 180.

[0102] The storage device 170 includes one or more storage media such as flash memory or magnetic disks. The storage device 170 stores various data generated by the GNSS unit 110, camera 120, obstacle sensor 130, LiDAR sensor 140, sensor group 150, and control device 180. The data stored in the storage device 170 may include map data of the environment in which the work vehicle 100 travels (environmental map) and target route data for autonomous driving. The environmental map includes information on multiple fields where the work vehicle 100 performs agricultural work and the surrounding roads. The environmental map and target route may be generated by a processor in a server that manages agricultural work. The control device 180 may also have a function to generate or edit the environmental map and target route. The control device 180 can edit the environmental map and target route acquired from the server according to the driving environment of the work vehicle 100. The storage device 170 also stores work plan data received by the communication device 190 from the server.

[0103] The storage device 170 also stores computer programs that cause each ECU in the control unit 180 to perform various operations described later. Such computer programs can be provided to the work vehicle 100 via a storage medium (e.g., semiconductor memory or optical disc) or a telecommunications line (e.g., the Internet). Such computer programs may be sold as commercial software.

[0104] The control device 180 includes a plurality of ECUs. These plurality of ECUs include, for example, an ECU 181 for speed control, an ECU 182 for steering control, an ECU 183 for implement control, an ECU 184 for autonomous driving control, and an ECU 185 for route generation.

[0105] The ECU 181 controls the speed of the work vehicle 100 by controlling the prime mover 102, the transmission 103, and the brakes, which are included in the drive unit 193.

[0106] The ECU 182 controls the steering of the work vehicle 100 by controlling the hydraulic system or electric motor included in the steering device 106 based on the measurements of the steering wheel sensor 152.

[0107] The ECU 183 controls the movement of the three-point linkage and PTO shaft, etc., included in the coupling device 108, in order to make the work implement 300 perform the desired operation. The ECU 183 also generates signals to control the operation of the work implement 300 and transmits these signals from the communication device 190 to the work implement 300.

[0108] The ECU 184 performs calculations and controls to achieve autonomous driving based on data output from the GNSS unit 110, camera 120, obstacle sensor 130, LiDAR sensor 140, and sensor group 150. For example, the ECU 184 determines the position of the work vehicle 100 based on data output from at least one of the GNSS unit 110, camera 120, and LiDAR sensor 140. Within the field, the ECU 184 may determine the position of the work vehicle 100 based only on data output from the GNSS unit 110. The ECU 184 may estimate or correct the position of the work vehicle 100 based on data acquired by the camera 120 or LiDAR sensor 140. By utilizing data acquired by the camera 120 or LiDAR sensor 140, the accuracy of positioning can be further improved. Outside the field, the ECU 184 estimates the position of the work vehicle 100 using data output from the LiDAR sensor 140 or camera 120. For example, ECU 184 may estimate the position of the work vehicle 100 by matching data output from the LiDAR sensor 140 or camera 120 with an environmental map. During autonomous driving, ECU 184 performs calculations necessary for the work vehicle 100 to travel along the target path based on the estimated position of the work vehicle 100. ECU 184 sends a command to ECU 181 to change speed and a command to ECU 182 to change steering angle. ECU 181 changes the speed of the work vehicle 100 by controlling the engine 102, transmission 103, or brakes in response to the command to change speed. ECU 182 changes the steering angle by controlling the steering device 106 in response to the command to change steering angle.

[0109] While the work vehicle 100 is in motion, the ECU 185 recognizes obstacles present around the work vehicle 100 based on data output from the camera 120, obstacle sensor 130, and LiDAR sensor 140. The ECU 185 can also determine the destination of the work vehicle 100 based on the work plan stored in the storage device 170, and determine the target path from the starting point to the destination point of the work vehicle 100's movement.

[0110] Through the operation of these ECUs, the control unit 180 enables autonomous driving. During autonomous driving, the control unit 180 controls the drive unit 193 based on the measured or estimated position of the work vehicle 100 and the target path. This allows the control unit 180 to drive the work vehicle 100 along the target path. Multiple ECUs included in the control unit 180 may cooperate to perform these processes.

[0111] Multiple ECUs included in the control unit 180 can communicate with each other according to a vehicle bus standard such as CAN (Controller Area Network). Instead of CAN, a faster communication method such as Automotive Ethernet (registered trademark) may be used. In Figure 12, each of the ECUs 181 to 185 is shown as a separate block, but each of their functions may be implemented by multiple ECUs. An on-board computer integrating at least some of the functions of ECUs 181 to 185 may be provided. The control unit 180 may also include ECUs other than ECUs 181 to 185, and any number of ECUs can be provided depending on their function. Each ECU includes a processing circuit containing one or more processors.

[0112] The communication device 190 is a device that includes circuits for communicating with the work machine 300, terminal equipment, and a server that manages agricultural work. The communication device 190 includes circuits for transmitting and receiving signals compliant with ISOBUS standards, such as ISOBUS-TIM, to and from the communication device 390 of the work machine 300. This allows the work machine 300 to perform desired operations or to obtain information from the work machine 300. The communication device 190 may further include antennas and communication circuits for transmitting and receiving signals over a network to and from the respective communication devices of the terminal equipment and the server. The network may include, for example, a cellular mobile communication network such as 3G, 4G, or 5G and the Internet. The communication device 190 may also have a function to communicate with a mobile terminal used by a supervisor near the work vehicle 100. Communication with such a mobile terminal may be conducted in accordance with any wireless communication standard, such as Wi-Fi®, cellular mobile communication such as 3G, 4G, or 5G, or Bluetooth®.

[0113] The drive unit 340 in the work machine 300 shown in Figure 12 performs the operations necessary for the work machine 300 to perform a predetermined operation. The drive unit 340 includes devices such as a hydraulic system, an electric motor, or a pump, depending on the application of the work machine 300. The control device 380 controls the operation of the drive unit 340. The control device 380 causes the drive unit 340 to perform various operations in response to signals transmitted from the work vehicle 100 via the communication device 390. It can also transmit signals corresponding to the status of the work machine 300 from the communication device 390 to the work vehicle 100.

[0114] An example of the operation of the work vehicle 100 in automatic driving mode will be described. In this embodiment of the disclosure, the work vehicle 100 can drive automatically both inside and outside the field. Inside the field, the work vehicle 100 drives along a pre-set target route and drives the work machine 300 to perform predetermined agricultural work. If the work vehicle 100 detects an obstacle by the obstacle sensor 130 while driving inside the field, it stops driving and performs actions such as emitting a warning sound from the buzzer 192 and transmitting a warning signal to a terminal device. Inside the field, the positioning of the work vehicle 100 is mainly based on data output from the GNSS unit 110. On the other hand, outside the field, the work vehicle 100 drives automatically along a target route set on an agricultural road or public road outside the field. While driving outside the field, the work vehicle 100 uses data acquired by the camera 120 or LiDAR sensor 140 to drive. Outside the field, if the work vehicle 100 detects an obstacle, it will either avoid the obstacle or stop in place. Outside the field, the position of the work vehicle 100 is estimated based on positioning data output from the GNSS unit 110, as well as data output from the LiDAR sensor 140 or camera 120.

[0115] The configurations and operations of the embodiments described above are illustrative only, and this disclosure is not limited to the embodiments described above. For example, the various embodiments described above may be combined as appropriate to form other embodiments.

[0116] The display systems in the above embodiments can also be retrofitted to work vehicles (agricultural or construction machinery) that do not possess those functions. Such systems can be manufactured and sold independently of the work vehicles. Computer programs used in such systems can also be manufactured and sold independently of the work vehicles. Computer programs can be provided, for example, stored in a computer-readable non-temporary storage medium. Computer programs can also be provided by download via telecommunications lines (e.g., the Internet).

[0117] As described above, this disclosure includes the display system and work vehicles described in the following items.

[0118] [Item 1] A display system mounted on a work vehicle equipped with a vibration sensor, The screen and, A control device for controlling the display of an image on the screen, comprising: a control device that generates an image including an input interface for a user to perform an input operation, and a control device that displays the image on the screen; Equipped with, The control device is a display system that changes the position of the display of the input interface on the screen based on the vibration of the work vehicle detected by the vibration sensor.

[0119] [Item 2] The display system according to item 1, wherein the control device, when the magnitude of vibration of the work vehicle detected by the vibration sensor is less than a threshold, keeps the display of the input interface stationary at a first predetermined position on the screen, and when the magnitude of vibration of the work vehicle is greater than or equal to the threshold, moves the display of the input interface from the first predetermined position.

[0120] [Item 3] The display system according to item 2, wherein the control device moves the display of the input interface from the first predetermined position in the same direction as the direction of displacement of the vibration of the work vehicle when the magnitude of vibration of the work vehicle is greater than or equal to the threshold.

[0121] [Item 4] The display system described in item 2 or 3, wherein the screen is a touchscreen.

[0122] [Item 5] The control device is The image is generated and the image is displayed on the screen, further including the display of at least one piece of information: information about the work vehicle, information about the movement of the work vehicle, and information about the work performed by the work vehicle. If the magnitude of vibration of the work vehicle is less than the threshold, the display of at least one piece of information is stopped at a second predetermined position on the screen. The display system according to item 4, wherein if the magnitude of vibration of the work vehicle is greater than or equal to the threshold, the display of at least one piece of information is moved from the second predetermined position in the same direction as the direction of displacement of the vibration of the work vehicle.

[0123] [Item 6] A light source controlled by the control device, An optical system that receives light emitted from the light source and forms a virtual image in front of the screen, A display system as described in item 2 or 3, comprising:

[0124] [Item 7] The control device is The image is generated and the image is displayed on the screen, further including the display of at least one piece of information: information about the work vehicle, information about the movement of the work vehicle, and information about the work performed by the work vehicle. If the magnitude of vibration of the work vehicle is less than the threshold, the display of at least one piece of information is stopped at a second predetermined position on the screen. The display system according to item 6, wherein if the magnitude of vibration of the work vehicle is greater than or equal to the threshold, the display of at least one piece of information is moved from the second predetermined position in the opposite direction to the direction of displacement of the vibration of the work vehicle.

[0125] [Item 8] The display system according to any one of items 1 to 7, wherein the input interface includes an input unit for inputting at least one piece of information: information relating to the work vehicle, information relating to the movement of the work vehicle, and information relating to the work performed by the work vehicle.

[0126] [Item 9] Vibration sensor and, A display system as described in any of items 1 through 8, A work vehicle equipped with the following features. [Industrial applicability]

[0127] The technology of this disclosure can be applied to a display system for agricultural machinery or construction machinery. [Explanation of symbols]

[0128] 50: GNSS satellite, 60: base station, 100: work vehicle, 101: vehicle body, 102: prime mover, 103: transmission, 104: wheels, 104F: front wheels, 104R: rear wheels, 105: cabin, 106: steering system, 107: driver's seat, 108: coupling device, 110: GNSS unit, 111: GNSS receiver, 112: RTK receiver, 116: processing circuit, 120: camera, 130: obstacle sensor, 140: LiDAR sensor, 150: sensor group, 152: steering wheel sensor, 154: steering angle sensor, 156: axle sensor, 160: control system, 170: memory device, 180: control device, 181~185: ECU, 1 90: Communication device, 191: Operation switch group, 192: Buzzer, 193: Drive device, 200: Operation terminal, 210: Input device, 220: Display device, 230, 450: Control device, 240: ROM, 250: RAM, 260: Storage device, 270: Communication device, 280: Input interface, 290: Display, 300: Work machine, 340: Drive device, 380: Control device, 390: Communication device, 400: HUD unit, 410: Light source, 420: Transparent screen, 430: Field lens, 440: Combiner, 480: Input interface, 491, 492: Display, 500: Front window, 600: Display area, S: Screen

Claims

1. A display system mounted on a work vehicle equipped with a vibration sensor, The screen and, A control device for controlling the display of an image on the screen, comprising: a control device that generates an image including an input interface for a user to perform an input operation, and a control device that displays the image on the screen; Equipped with, The control device is a display system that, when the magnitude of vibration of the work vehicle detected by the vibration sensor is less than a threshold, keeps the display of the input interface stationary at a first predetermined position on the screen, and when the magnitude of vibration of the work vehicle is greater than or equal to the threshold, moves the display of the input interface from the first predetermined position in the same direction as the direction of displacement of the vibration of the work vehicle.

2. The display system according to claim 1, wherein the screen is a touchscreen.

3. A display system mounted on a work vehicle equipped with a vibration sensor, Touchscreen and A control device for controlling the display of an image on the touchscreen, comprising: a control device that generates an image including an input interface for a user to perform an input operation, and a control device that displays the image on the touchscreen; Equipped with, The control device is The system generates an image that further includes the display of at least one piece of information, such as information relating to the work vehicle, information relating to the movement of the work vehicle, and information relating to the work performed by the work vehicle, and displays the image on the touchscreen. If the magnitude of the vibration of the work vehicle detected by the vibration sensor is below a threshold, the display of at least one piece of information is stopped at a second predetermined position on the touchscreen. A display system that, when the magnitude of vibration of the work vehicle exceeds the threshold, moves the display of at least one piece of information from the second predetermined position in the same direction as the direction of displacement of the vibration of the work vehicle.

4. A light source controlled by the control device, An optical system that receives light emitted from the light source and forms a virtual image in front of the screen, The display system according to claim 1, comprising:

5. The control device is An image is generated that further includes the display of at least one piece of information: information about the work vehicle, information about the movement of the work vehicle, and information about the work performed by the work vehicle; and the image is displayed on the screen. If the magnitude of vibration of the work vehicle is less than the threshold, the display of at least one piece of information is stopped at a second predetermined position on the screen. The display system according to claim 4, wherein if the magnitude of vibration of the work vehicle is greater than or equal to the threshold, the display of at least one piece of information is moved from the second predetermined position in the opposite direction to the direction of displacement of the vibration of the work vehicle.

6. The display system according to claim 1, wherein the input interface includes an input unit for inputting at least one piece of information: information relating to the work vehicle, information relating to the movement of the work vehicle, and information relating to the work performed by the work vehicle.

7. Vibration sensor and, A display system according to any one of claims 1 to 6, A work vehicle equipped with the following features.

Images