Remote devices and remote control systems

Abstract

To assist (support) remote control.SOLUTION: A remote device 30 comprises: a control device 35 for remotely controlling a work vehicle 1; a communication device 33 for receiving traveling information indicating a speed or acceleration of the work vehicle 1; a display device 34; and a control device 31 for causing the display device 34 to display a highlighting display that changes depending on the speed of the work vehicle 1 which is indicated by the traveling information when the work vehicle 1 is remotely controlled by the control device 35.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a remote control device for remotely controlling a work vehicle and a remote control system for remotely controlling a work vehicle.

Background Art

[0002] For example, Patent Document 1 discloses a driving support system including an anger determination unit that determines the anger of a driver, an acceleration control unit that suppresses acceleration with respect to the operation amount of an accelerator pedal based on the determination result of the anger determination unit, and a perceived speed variable unit that increases the perceived speed based on the determination result of the anger determination unit. According to this driving support system, when detecting the anger of the driver, by displaying an increase in the driver's perceived speed, it is possible to make the driver aware that the state of mind has been lost due to anger and quickly return to a normal state of mind.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Since the speed range of industrial machines is biased towards low speeds, unlike ordinary vehicles, it is difficult to sense changes in vehicle speed. Remote driving is already difficult to sense vehicle speed, but when driving on grasslands or fields, there is no center line and the change in the surrounding scenery is scarce, making it even more difficult to sense vehicle speed.

[0005] In view of the above problems, an object of the present invention is to provide a remote control device and a remote control system that can make it easier for a remote operator to sense the speed of a work vehicle and support remote control.

Means for Solving the Problems

[0006] The technical means of the present invention for solving the above technical problems are characterized by the following points. A remote control device according to one aspect of the present invention comprises: a control device for remotely controlling a work vehicle; a communication device for receiving driving information indicating the speed or acceleration of the work vehicle; a display device; and a control device that, when the work vehicle is remotely controlled by the control device, causes the display device to display a highlighting that changes according to the driving information. The communication device is used when the work vehicle is remotely operated. In this case, the device sequentially receives captured images of the direction of travel of the work vehicle, the display device sequentially displays the captured images on the remote operation screen, the control device displays the highlighting on the remote operation screen, the control device displays the highlighting on the remote operation screen when the first condition is met, and does not display the highlighting on the remote operation screen when the first condition is not met, the control device determines that the first condition is met when the amount of change in the multiple captured images is less than a threshold, and determines that the first condition is not met when the amount of change in the multiple captured images is greater than or equal to a threshold. .

[0010] A remote control device according to one aspect of the present invention comprises: a control device for remotely controlling a work vehicle; a communication device for receiving driving information indicating the speed or acceleration of the work vehicle; a display device; and a control device for causing the display device to display a highlighting display that changes according to the driving information when the work vehicle is remotely controlled by the control device, wherein the communication device sequentially receives captured images of the direction of travel of the work vehicle when the work vehicle is remotely driven, the display device sequentially displays the captured images on the remote driving screen, the control device displays the highlighting display on the remote driving screen when a first condition is met, and does not display the highlighting display on the remote driving screen when the first condition is not met. The control device determines that the first condition is met if the captured image does not include road markings, and determines that the first condition is not met if the captured image includes road markings. do .

[0011] A remote control device according to one aspect of the present invention comprises: a control device for remotely controlling a work vehicle; a communication device for receiving driving information indicating the speed or acceleration of the work vehicle; a display device; and a control device that, when the work vehicle is remotely controlled by the control device, causes the display device to display a highlighting display that changes according to the driving information, wherein the communication device sequentially receives captured images of the direction of travel of the work vehicle when the work vehicle is remotely controlled; the display device sequentially displays the captured images on the remote control screen; the control device causes the highlighting display on the remote control screen when a first condition is met; and when the first condition is not met, the highlighting display on the remote control screen Do not display it. The control device uses the location information of the work vehicle and the map information to determine if the current location indicated by the location information of the work vehicle is within a predetermined area of ​​the map indicated by the map information, and determines that the first condition is not met if the vehicle is not within the predetermined area. do .

[0012] before The control device displays the highlighting on the peripheral edge of the remote operation screen or the captured image, and the control device changes the area of ​​the peripheral edge according to the speed or acceleration of the work vehicle.

[0013] A remote control device according to one aspect of the present invention comprises: a control device for remotely controlling a work vehicle; a communication device for receiving driving information indicating the speed or acceleration of the work vehicle; a display device; and a control device for displaying a highlighting on the display device that changes according to the driving information when the work vehicle is remotely controlled by the control device, wherein the communication device sequentially receives captured images of the direction of travel of the work vehicle when the work vehicle is remotely driven, the display device sequentially displays the captured images on a remote driving screen, the control device displays the highlighting on the remote driving screen, and the control device superimposes the highlighting on the captured images on the remote driving screen. The control device changes the movement speed of the display unit according to the speed or acceleration of the work vehicle. do . A remote control device according to one aspect of the present invention comprises: a control device for remotely controlling a work vehicle; a communication device for receiving driving information indicating the speed or acceleration of the work vehicle; a display device; and a control device for displaying a highlighting on the display device that changes according to the driving information when the work vehicle is remotely controlled by the control device, wherein the communication device sequentially receives captured images of the direction of travel of the work vehicle when the work vehicle is remotely driven, the display device sequentially displays the captured images on a remote driving screen, the control device displays the highlighting on the remote driving screen, and the control device superimposes the highlighting on the captured images on the remote driving screen. The control device changes the display configuration according to the speed or acceleration of the work vehicle. do .

[0014] The display body may be a display body that extends in the direction of travel of the work vehicle. The display body may be a plurality of virtual display objects arranged in the traveling direction of the work vehicle.

[0016] before The control device changes the color of a specific location other than the captured image of the remote operation screen according to the speed or acceleration of the work vehicle as the emphasized display.

[0017] before The control device changes the color of the frame of the remote operation screen according to the speed or acceleration of the work vehicle as the emphasized display.

[0018] A remote control device according to one aspect of the present invention comprises: a control device for remotely controlling a work vehicle; a communication device for receiving driving information indicating the speed or acceleration of the work vehicle; a display device; and a control device that, when the work vehicle is remotely controlled by the control device, causes the display device to display a highlight that changes according to the driving information, wherein the communication device sequentially receives captured images of the direction of travel of the work vehicle when the work vehicle is remotely driven, the display device sequentially displays the captured images on the remote driving screen, and the control device displays the highlight on the remote driving screen. The remote operation screen includes a front captured image and a rear captured image, and the control device displays the emphasized display on the front captured image during forward travel and on the rear captured image during reverse travel. ru 。

[0019] A remote control device according to one aspect of the present invention comprises a control device for remotely controlling a work vehicle, a communication device for receiving driving information indicating the speed or acceleration of the work vehicle, a display device, and a control device that causes the display device to display a highlighting that changes according to the driving information when the work vehicle is remotely controlled by the control device, wherein the communication device is used when the work vehicle is remotely operated. In this case, the system receives sequentially captured images of the direction of travel of the work vehicle, the display device displays the captured images sequentially on the remote operation screen, and the control device displays the highlighting on the remote operation screen. When the work vehicle is traveling in reverse, the control device displays the captured image of the work vehicle during reverse travel on the remote operation screen, and changes the mode of the guide line displayed on the remote operation screen according to the speed or acceleration of the work vehicle as the emphasized display. ru 。

[0020] A remote control device according to one aspect of the present invention comprises: a control device for remotely controlling a work vehicle; a communication device for receiving driving information indicating the speed or acceleration of the work vehicle; a display device; and a control device that, when the work vehicle is remotely controlled by the control device, causes the display device to display a highlight that changes according to the driving information, wherein the communication device sequentially receives captured images of the direction of travel of the work vehicle when the work vehicle is remotely driven, the display device sequentially displays the captured images on the remote driving screen, and the control device displays the highlight on the remote driving screen. When the work vehicle is accelerating, the control device changes the range displayed as the captured image on the remote operation screen upward according to the change in acceleration, and when the work vehicle is decelerating, changes the range displayed as the captured image on the remote operation screen downward according to the change in acceleration. ru 。

[0021] When the work vehicle is turning left, the control device changes the range displayed as the captured image on the remote operation screen to the right according to the steering angle of the left turn, and when the work vehicle is turning right, changes the range displayed as the captured image on the remote operation screen to the left according to the steering angle of the right turn.

[0022] A remote control system according to an aspect of the present invention includes a work vehicle and the remote device described above. The work vehicle includes a detection device that detects the speed of the work vehicle, an imaging device that images the traveling direction of the work vehicle, travel information indicating the speed or acceleration detected by the detection device, and the an in-vehicle communication device that transmits correspondence data in which the imaging image captured by the imaging device is associated with the travel information. The communication device of the remote device receives the correspondence data transmitted from the in-vehicle communication device.

Advantages of the Invention

[0023] According to the present invention, it is possible to make it easier for a remote operator to feel the speed of the work vehicle and support remote control.

Brief Description of the Drawings

[0024] [Figure 1] It is a configuration diagram of a remote control system 100 according to an embodiment of the present invention. [Figure 2] It is a side view of a tractor which is an example of the work vehicle 1. [Figure 3] It is a diagram showing an example of the correspondence data. [Figure 4] It is a diagram showing an example of a planned travel route L1. [Figure 5A] It is a diagram showing an example of a remote operation screen G2 without the highlighting K. [Figure 5B] It is a diagram showing an example of a remote operation screen G2 with the highlighting K. [Figure 5C] It is a diagram showing the highlighting K of the second display form on the remote operation screen G2. [Figure 6] It is a diagram showing the highlighting K of the third display form on the remote operation screen G2. [Figure 7A] It is a diagram showing the highlighting K of the fourth display form on the remote operation screen G2. [Figure 7B] It is a diagram showing the highlighting K of the fourth display form on the remote operation screen G2. [Figure 8A] It is a diagram showing the highlighting K of the fifth display form on the remote operation screen G2. [Figure 8B] This figure shows the seventh display mode, highlighted K, on ​​the remote driving screen G2. [Figure 8C] This figure shows the eighth display mode, highlighted K, on ​​the remote driving screen G2. [Figure 9A] This figure shows an example of the selection screen G1 of the display device 34. [Figure 9B] This figure shows an example of the selection screen G1 of the display device 34. [Figure 9C] This figure shows an example of the selection screen G1 of the display device 34. [Figure 9D] This figure shows an example of the selection screen G1 of the display device 34 in a first modified form. [Figure 10A] This flowchart shows the operation of work vehicle 1 during remote operation. [Figure 10B] This is a flowchart showing the operation of the remote control device 30 when remotely controlling the work vehicle 1. [Figure 11] This is a flowchart showing the screen display update process. [Figure 12A] This figure shows an example of the remote control screen G2 of the first modified example. [Figure 12B] This figure shows the highlighted K on the remote control screen G2 of the first modified example. [Figure 12C] This figure shows the highlighted K on the remote control screen G2 of the first modified example. [Figure 12D] This figure shows the highlighted K on the remote control screen G2 of the first modified example. [Figure 12E] This figure shows the highlighted K on the remote control screen G2 of the first modified example. [Figure 13] This flowchart shows the screen display update process for the second modified example. [Modes for carrying out the invention]

[0025] Figure 1 is a diagram showing the configuration of a remote control system 100 according to one embodiment of the present invention. The remote control system 100 comprises a work vehicle 1 and a remote device 30. The remote control system 100 and the remote device 30 enable remote control (or remote operation) and remote monitoring of the work vehicle 1. The work vehicle 1 is an agricultural machine (also called a "remotely controlled agricultural machine") that can be remotely operated (for example, remote driving and remote work) by the remote device 30. The work vehicle 1 is, for example, a tractor. A tractor is an example of an agricultural machine that performs farm work in a field. The work vehicle 1 may be an agricultural machine other than a tractor, a construction machine, or a work machine.

[0026] Figure 2 is a side view of a tractor, which is an example of a work vehicle 1. The work vehicle 1 has a body 3. The body 3 is equipped with a running gear 7. The running gear 7 has front wheels 7F and rear wheels 7R on both the left and right sides of the body 3, respectively, and supports the body 3 so that it can move. The running gear 7 may be a crawler type device.

[0027] The vehicle body 3 is equipped with a prime mover 4, a transmission 5, a braking system 13 (Figure 1), and a steering system 14 (Figure 1). The prime mover 4 consists of an engine (diesel engine, gasoline engine) or an electric motor, etc. The transmission 5 varies the propulsion force of the running gear 7 and switches the running gear 7 between forward and reverse by, for example, shifting gears. The braking system 13 brakes the vehicle body 3. The steering system 14 steers the vehicle body 3.

[0028] A cabin 9, which is an example of a protective mechanism, is provided on the upper part of the vehicle body 3. Inside the cabin 9 are a driver's seat 10 and a control device 11. The work vehicle 1 is a tractor that can be driven (operated) unmanned and perform work using the work device 2, but it is also possible for an operator seated in the driver's seat 10 to operate the control device 11 to drive the work vehicle 1 and perform work using the work device 2. The cabin 9 protects the driver's seat 10 by surrounding it from the front, rear, above, and left and right sides. Note that the protective mechanism is not limited to the cabin 9 and may also be a locating device, etc.

[0029] In Figure 2, arrow A1 points in the direction of the front of work vehicle 1. Arrow A2 points in the direction of the rear of work vehicle 1. Arrow Z1 points in the direction of the top of work vehicle 1. Arrow Z2 points in the direction of the bottom of work vehicle 1. The directions perpendicular to arrows A1, A2, Z1, and Z2 represent the width direction (left-right direction) of work vehicle 1. In Figure 2, the near side is the left side of work vehicle 1, and the far side is the right side of work vehicle 1.

[0030] A coupling device 8 is provided at the rear of the vehicle body 3. The coupling device 8 is composed of a three-point linkage mechanism, etc. A work device 2 (implement, etc.) can be attached to and detached from the coupling device 8. By connecting the work device 2 to the coupling device 8 and driving the running gear 7, the work vehicle 1 (vehicle body 3) can be driven, thereby towing the work device 2. In addition, the coupling device 8 can raise and lower the work device 2 and change the posture of the work device 2.

[0031] The working equipment 2 includes tilling equipment, fertilizer spreading equipment, pesticide spraying equipment, harvesting equipment, mowing equipment, spreading equipment, hay collecting equipment, and shaping equipment. Each of these devices can be attached to and detached from the working vehicle 1 by a coupling device 8. The working vehicle 1 performs agricultural work on the field using the working equipment 2.

[0032] A bonnet 12 is provided at the front of the cabin 9. The bonnet 12 is attached to the vehicle body 3. A storage compartment is formed between the bonnet 12 and the vehicle body 3. This storage compartment houses not only the prime mover 4, but also a cooling fan, radiator, and battery, among other things.

[0033] As shown in Figure 1, the work vehicle 1 is equipped with an on-board control device 21, an on-board communication device 23, a position detection device 24, a sensing device 25, a state detection device 26, a steering device 11, an actuator group 27, a prime mover 4, a running device 7, a transmission 5, a braking device 13, a steering device 14, and a coupling device 8. The work vehicle 1 is also equipped with an on-board network such as CAN, LIN, or FlexRay. The on-board control device 21 is electrically connected to the on-board communication device 23, the position detection device 24, the sensing device 25, the state detection device 26, the steering device 11, the actuator group 27, and the work equipment 2 connected to the work vehicle 1 via this on-board network.

[0034] The on-board control unit 21 consists of an ECU (Electronic Control Unit) including a processor 21a and a memory 21b. The on-board control unit 21 is a controller that controls the operation of various parts of the work vehicle 1. The memory 21b consists of volatile or non-volatile memory, etc. The memory 21b of the on-board control device 21 stores various information and data for the on-board control device 21 to control the operation of each part of the work vehicle 1 in a read-write manner.

[0035] The in-vehicle communication device 23 consists of an antenna, an IC (integrated circuit), and electrical circuits for wireless communication via a mobile phone network, the internet, or a wireless LAN. The in-vehicle control device 21 communicates wirelessly with the remote device 30 using the in-vehicle communication device 23.

[0036] In this embodiment, an example is shown in which the work vehicle 1 and the remote device 30 communicate via a mobile phone network, but in addition to this, for example, the work vehicle 1 and the remote device 30 may communicate via an external device such as a server or relay to the mobile phone network. Alternatively, the work vehicle 1 and the remote device 30 may communicate directly using short-range wireless signals such as BLU (Bluetooth® Low Energy) signals or UHF (Ultra High Frequency) signals. In this case, it is sufficient that the in-vehicle communication device 23 and the remote device 30 are each provided with an interface for short-range wireless communication.

[0037] The position detection device 24 is installed, for example, on top of the cabin 9 (Figure 2). However, the installation location of the position detection device 24 is not limited to the top of the cabin 9; it may be installed at another location on the vehicle body 3 or at a designated location on the work device 2. The position detection device 24 uses a satellite positioning system to detect its own position (positioning information including latitude and longitude). That is, the position detection device 24 receives signals transmitted from positioning satellites (position of the positioning satellite, transmission time, correction information, etc.) and detects its own position based on these signals. The position detection device 24 may also detect a position corrected based on correction signals from a base station (reference station) capable of receiving signals from positioning satellites as its own position.

[0038] Furthermore, the position detection device 24 may have inertial measuring devices such as a gyro sensor or an acceleration sensor. In this case, the position detection device 24 may correct the position (latitude, longitude) detected based on the signal received from the positioning satellite using the inertial measuring device, and detect the corrected position as its own position. The position detection device 24 uses its detected position as the position of the work vehicle 1. The position detection device 24 may also calculate the position of the work vehicle 1 based on its detected position and pre-stored external shape information of the work vehicle 1. The position detection device 24 may also calculate the position of the work device 2 based on its detected position, pre-stored external shape information of the work device 2, and the mounting position of the work device 2 on the vehicle body 3.

[0039] The sensing device 25 senses (monitors) the area around the work vehicle 1. Specifically, the sensing device 25 includes a laser sensor 25a, an ultrasonic sensor 25b, a camera 25c, and an object detection unit 25d. For example, there are multiple laser sensors 25a and multiple ultrasonic sensors 25b. The multiple laser sensors 25a and multiple ultrasonic sensors 25b are installed at predetermined locations such as the front, rear, and left and right sides of the work vehicle 1, respectively, to detect the surrounding conditions in front of, behind, and to the left and right sides of the work vehicle 1, as well as objects in those surroundings. For example, the laser sensors 25a and ultrasonic sensors 25b are installed at predetermined locations on the vehicle body 3 that are within a predetermined target detection distance from the work vehicle 1 and can detect objects that are lower than the vehicle body 3.

[0040] The laser sensor 25a and the ultrasonic sensor 25b are examples of object sensors. At least one of the laser sensor 25a and the ultrasonic sensor 25b may be provided in multiples on the sensing device 25 as object sensors. Furthermore, other object sensors may be provided in multiples on the sensing device 25.

[0041] The laser sensor 25a is composed of optical sensors such as LiDAR (Light Detection and Ranging). The laser sensor 25a is composed of light sources such as laser diodes. The laser sensor 25a emits pulsed measurement light (laser light) millions of times per second, and by reflecting this measurement light with a rotating mirror, it scans horizontally or vertically and projects the light into a predetermined detection range (sensing range). The laser sensor 25a then receives the reflected light from the object being measured using a light-receiving element.

[0042] The object detection unit 25d consists of an electrical circuit or IC that detects the presence or absence of an object, its position, and its type based on the light received signal output from the light receiving element of the laser sensor 25a. The object detection unit 25d detects the distance to the object based on the time from when the laser sensor 25a irradiates with measurement light until the reflected light is received (TOF (Time of Flight) method). Objects detected by the object detection unit 25d include the work site where the work vehicle 1 travels and works, fields, crops in the fields, the ground, roads, other objects, and people.

[0043] The ultrasonic sensor 25b is composed of an airborne ultrasonic sensor such as a sonar. The ultrasonic sensor 25b transmits a measurement wave (ultrasound) to a predetermined detection range using a transmitter, and receives the reflected wave when the measurement wave is reflected by an object using a receiver. The object detection unit 25d detects the presence or absence of an object, the position of the object, and the type of object based on the signal output from the receiver of the ultrasonic sensor 25b. The object detection unit 25d also detects the distance to the object (TOF method) based on the time from when the ultrasonic sensor 25b transmits the measurement wave until the reflected wave is received.

[0044] Camera 25c consists of a CCD camera equipped with a CCD (Charge Coupled Devices) image sensor, or a CMOS camera equipped with a CMOS (Complementary Metal Oxide Semiconductor) image sensor. As shown in Figure 2, Camera 25c is installed in predetermined locations such as the front, rear, left and right sides of the work vehicle 1, and inside the cabin 9, and captures images of the surrounding area such as the front, rear, and left and right sides of the work vehicle 1, and outputs the data of the captured images. Camera 25c is an example of an imaging device.

[0045] For example, the work vehicle 1 is equipped with multiple cameras 25c. Of the multiple cameras 25c installed on the work vehicle 1, the internal camera 25c1, installed inside the cabin 9 as shown in Figure 2, captures images of the area in front of the work vehicle 1 from the driver's seat 10. More specifically, the internal camera 25c1 captures images of the area in front of the work vehicle 1 (direction of travel) with a field of view that is approximately the same as that of a worker seated in the driver's seat 10. In other words, the internal camera 25c1 provides images of the direction of travel of the work vehicle 1.

[0046] Furthermore, among the multiple cameras 25c, the rear camera 25c2, installed at the rear of the cabin 9 as shown in Figure 2, captures images of the area behind the work vehicle 1. More specifically, when the shift lever is operated to the reverse position, for example, the rear camera 25c2 captures images of the area behind the work vehicle 1 (in the reverse direction) from the rear of the cabin 9. In other words, the rear camera 25c2 provides captured images of the area behind the work vehicle 1 (hereinafter, these may be referred to as rear captured images as appropriate). Note that the rear camera 25c2 may also capture images of the area behind the work vehicle 1 at all times, regardless of the position of the shift lever (i.e., the forward position, the neutral position, or the reverse position).

[0047] The object detection unit 25d can also detect the presence or absence of an object, the position of the object, and the type of object based on the data of the captured image output from the camera 25c.

[0048] The sensing device 25 senses (monitors) the surrounding conditions of the work vehicle 1 and work equipment 2 using a laser sensor 25a, an ultrasonic sensor 25b, a camera 25c, and an object detection unit 25d, and outputs sensing information showing the results to the on-board control device 21. This includes at least the detection information from the object detection unit 25d and the image data from the camera 25c, but the sensing information may also include detection information from the laser sensor 25a and the ultrasonic sensor 25b.

[0049] The state detection device 26 detects the operating status of the work vehicle 1 and the work device 2. Specifically, the state detection device 26 includes various sensors installed on various parts of the work vehicle 1 and the work device 2, and a calculator. The calculator detects (calculates) the operating status of the work vehicle 1 and the work device 2 based on the output signals from the various sensors. The state of the work vehicle 1 detected by the state detection device 26 includes the drive and stop status of each part of the work vehicle 1, the direction of travel, travel speed, acceleration, and posture of the work vehicle 1. The state of the work device 2 detected by the state detection device 26 includes the drive and stop status and posture of each part of the work device 2.

[0050] The state detection device 26 may acquire the position of the vehicle body 3 (position of the work vehicle 1) detected by the position detection device 24 at a predetermined period, and detect (calculate) the position of the work device 2 from the position of the vehicle body 3, or detect changes (transitions) in the position of the vehicle body 3. The state detection device 26 may also detect the travel speed of the vehicle body 3 from the change in the position of the vehicle body 3. As another example, a rotation speed sensor may be provided to detect the rotation speed of the front wheels 7F and rear wheels 7R of the travel device 7, or the rotation speed of the travel motors that roll the front wheels 7F and rear wheels 7R, and the state detection device 26 may detect the travel speed of the vehicle body 3 based on the output signal of the rotation speed sensor. The state detection device 26 may also have a speedometer and acquire the travel speed of the vehicle body 3 measured by the speedometer. The state detection device 26 may also detect acceleration from the change in speed per unit time. The state detection device 26 may also have an accelerometer and acquire the acceleration of the vehicle body 3 measured by the accelerometer.

[0051] The state detection device 26 generates detection information indicating the operating status of the detected work vehicle 1 and work device 2, and outputs this detection information to the on-board control device 21. For example, the detection information of the state detection device 26 includes operating information for the work vehicle 1 and work device 2. This operating information includes, for example, information on the speed and acceleration of the work vehicle 1, the gear shift position of the transmission 5, the braking position of the braking device 13, and the operating position of the work device 2.

[0052] The position detection device 24 and the state detection device 26 output detection information indicating the results of detection at predetermined intervals or predetermined timings to the on-board control device 21 as needed. The sensing device 25 also outputs sensing information indicating the results of sensing at predetermined intervals or predetermined timings to the on-board control device 21 as needed. The on-board control device 21 stores the detection information and sensing information input from the position detection device 24, the state detection device 26, and the sensing device 25 in its internal memory 21b. In the case of remote operation, the on-board control device 21 sequentially transmits the detection information and sensing information stored in its internal memory 21b to the remote device 30 via the on-board communication device 23 at predetermined intervals or predetermined timings.

[0053] The detection and sensing information transmitted from the work vehicle 1 includes correspondence data (see Figure 3) that associates the position information of the work vehicle 1 with the driving information including the speed or acceleration of the work vehicle 1 and the captured image of the direction of travel of the work vehicle 1. Figure 3 is a diagram showing an example of correspondence data. Specifically, correspondence data that associates the detection information of the position detection device 24 (i.e., the position information of the work vehicle 1), the driving information of the work vehicle 1 detected by the state detection device 26, and the sensing information of the sensing device 25 (for example, the captured image by the internal camera 25c1) is transmitted sequentially to the remote device 30. As shown in Figure 3, correspondence data that associates the position PA1 of the work vehicle 1 with the speed SD1 of the work vehicle 1 and the captured image GPA1 taken by the camera 25c is transmitted to the remote device 30. In addition, correspondence data that associates the position PA2 of the work vehicle 1 with the speed SD2 of the work vehicle 1 and the captured image GPA2 taken by the camera 25c is transmitted to the remote device 30. Regarding the captured images, as shown in Figure 3, when moving forward, the image is captured from the front (when moving backward, the image is captured from the rear), along with the driving information of the work vehicle 1 and the work vehicle Correspondence data, which associates the location information of both devices 1 with the captured image, is transmitted sequentially to the remote device 30. In the embodiment described above, the correspondence data included the location information, driving information, and captured image of the work vehicle 1, but the location information, driving information, and captured image of the work vehicle 1 may be acquired separately and associated with each other based on time, etc.

[0054] The actuator group 27 includes electric or hydraulic motors, cylinders, and control valves for operating various parts of the work vehicle 1, such as the prime mover 4, running gear 7, transmission 5, braking system 13, and coupling device 8. The steering device 11 includes a steering wheel 11a (Figure 2), accelerator pedal, brake pedal, and gear shift lever 11d (Figure 1). The on-board control device 21 operates predetermined actuators included in the actuator group 27 according to the operating state of the steering device 11 to drive the prime mover 4, running gear 7, transmission 5, braking system 13, and steering device 14, thereby controlling the movement and steering of the work vehicle 1.

[0055] Furthermore, the on-board control device 21 communicates with the control unit 2a provided in the work device 2, causing the control unit 2a to control the operation of the work device 2. In other words, the on-board control device 21 performs work on the field by controlling the operation of the work device 2 via the control unit 2a. For example, the control unit 2a is composed of a CPU, memory, etc. Note that some work devices 2 do not have a control unit 2a. In this case, the on-board control device 21 controls the posture of the work device 2 using the coupling device 8, and the work device 2 performs work on the field.

[0056] The on-board control device 21 controls the movement of the work vehicle 1, the work performed by the work device 2, and other operations of the work vehicle 1 based on sensing information from the sensing device 25, detection information from the state detection device 26, and detection information from the position detection device 24. Furthermore, when the on-board control device 21 receives a remote control signal transmitted from the remote device 30 via the on-board communication device 23, it controls the movement of the work vehicle 1, the work performed by the work device 2, and other operations of the work vehicle 1 based on the remote control signal, in addition to the above information.

[0057] Furthermore, when the on-board control device 21 controls the movement of the work vehicle 1 or the work performed by the work device 2, it determines from the detection information of the object detection unit 25d whether or not an object is approaching the work vehicle 1 or the work device 2 within a predetermined distance and posing a risk of contact. If the on-board control device 21 determines that an object is approaching the work vehicle 1 or the work device 2 within a predetermined distance and posing a risk of contact, it controls the travel device 7 or the work device 2, etc., to automatically stop the movement or work of the work vehicle 1 and avoid contact with the object.

[0058] Next, the remote control device 30 will be described. As shown in Figure 1, the remote control device 30 is located at a distance from the work vehicle 1. With the remote control device 30, the work vehicle 1 can be remotely controlled by a remote operator, and the status of the work vehicle 1, the surrounding conditions of the work vehicle 1, etc. can be monitored. The remote control device 30 includes a control device 31, a storage device 32, a communication device 33, a display device 34, a control device 35, and a notification device 36.

[0059] The control device 31 is a processor that controls the operation of each part of the remote device 30. This processor functions as a control device 31 that controls the operation of each part of the remote device 30 by, for example, executing a remote control program stored in the storage device 32. The internal memory 32a provided in the control device 31 is either volatile or non-volatile memory. Various information and data for the control device 31 to control the operation of each part of the remote device 30 are stored in the internal memory 32a in a read-write manner.

[0060] The storage device 32 pre-stores control programs such as a remote control program for remotely operating the work vehicle 1, a remote monitoring program for remotely monitoring the work vehicle 1, and various data. The storage device 32 is, for example, an SSD (Solid State Drive), H Examples include DD (Hard Disk Drive).

[0061] The communication device 33 consists of an antenna, IC, and electrical circuits for wireless communication via a mobile phone network, the internet, or a wireless LAN. The communication device 33 communicates wirelessly with the work vehicle 1 under the control of the control device 31. The communication device 33 receives various data transmitted from the on-board communication device 23 (such as detection information from the position detection device 24 and the state detection device 26, and sensing information from the sensing device 25). For example, the communication device 33 receives correspondence data that associates the position information of the work vehicle 1 with the driving information of the work vehicle 1 and an image captured of the direction of travel of the work vehicle 1.

[0062] The display device 34 is, for example, a liquid crystal display or an organic EL display. The display device 34 displays information for remotely operating the work vehicle 1 through display control by the control device 31. Figure 5A is a diagram showing an example of a remote operation screen G2 without highlighting K. For example, the display device 34 displays the remote operation screen G2 as shown in Figure 5A.

[0063] The remote operation screen G2 is an operation screen that displays various information for remotely operating the work vehicle 1. For example, the remote operation screen G2 has a window 43a that displays a forward image 42a captured by an internal camera 25c1 in front of the work vehicle 1, a window 43b that displays a rear image 42b captured by a rear camera 25c2 (Figure 2) installed at the rear of the vehicle body 3 in rear of the work vehicle 1, and windows 41a and 41b that display various information indicating the status of the work vehicle 1. In other words, both the forward image 42a and the rear image 42b are displayed. The control device 31 displays a highlight K on the forward image 42a when moving forward, and on the rear image 42b when moving backward. Note that the remote operation screen G2 may also be configured to display only the forward image 42a when the work vehicle 1 is moving forward, and only the rear image 42b (see Figure 8C) when the work vehicle 1 is moving backward.

[0064] Furthermore, the display device 34 is equipped with, for example, a touch panel on the front of the display screen, and touch operations on the display screen can be detected by the touch panel.

[0065] The control device 31 of the remote device 30 displays the status of the work vehicle 1, as detected by the position detection device 24 and the on-board control device 21, in windows 41a and 41b of the remote driving screen G2. In Figure 5A, window 41a displays that the direction of travel of the running gear 7 has been switched to the forward direction ("Shuttle: F"), the sub-transmission of the transmission 5 has been switched to a high gear ("Sub-transmission: High"), the main transmission (continuously variable transmission) is at 50% ("Main transmission: 50%"), the work vehicle 1 is driving in two-wheel drive mode ("Driving mode: 2WD"), and the accelerator pedal is at 40%. Window 41b displays that the work vehicle 1 is being driven remotely ("Remotely controlled"), the driving speed of the work vehicle 1 (vehicle body 3) is 2.9 km / h, and the rotational speed of the prime mover 4 is 1600 rpm.

[0066] The information displayed in windows 41a and 41b is not limited to the status of the work vehicle 1 described above. Also, the number of windows 41a and 41b is not limited to two; there may be one or three or more. Furthermore, the control device 31 may display not only the status of the work vehicle 1, but also whether or not the work device 2 is connected to the work vehicle 1, the type of work device 2, etc., in the windows of the remote operation screen G2 based on detection information such as the position detection device 24 or sensing information from the sensing device 25.

[0067] The control device 35 is a device for remotely controlling the work vehicle 1. The control device 35 comprises a steering wheel 35a, an accelerator pedal 35b, a brake pedal 35c, and a gear shift lever 35d, which are arranged around the remote operator's seat. The remote operator sits in the remote operator's seat and controls the vehicle. By operating the device 35, the movement of the work vehicle 1 or the work performed by the work device 2 is remotely controlled. The remote operator also monitors the work vehicle 1 and the surrounding conditions using the display device 34. Furthermore, the remote operator can input predetermined information or instructions to the remote device 30 by operating the control device 35. The control device 35 may be a touchpad or a hardware-type switch, etc.

[0068] The notification device 36 is equipped with a speaker 36a that outputs a notification sound to the remote operator.

[0069] When a remote operator operates the control device 35 to input an operation command for the work vehicle 1, the control device 31 generates a remote control signal corresponding to the operation command and transmits the remote control signal to the work vehicle 1 via the communication device 33. In other words, remote control signals corresponding to the operation of the steering wheel 35a, accelerator pedal 35b, brake pedal 35c, and gear shift lever 35d are transmitted to the work vehicle 1. When the on-board control device 21 of the work vehicle 1 receives the remote control signal from the remote device 30 via the on-board communication device 23, it operates various parts of the work vehicle 1 based on the remote control signal, the detection information from the position detection device 24, the sensing information from the sensing device 25, and the detection information from the state detection device 26 to control the driving and steering of the work vehicle 1, and to control the work operations performed by the work device 2.

[0070] Furthermore, the in-vehicle control device 21 transmits the detection information from the position detection device 24 and the state detection device 26, as well as the sensing information from the sensing device 25, to the remote device 30 via the in-vehicle communication device 23. When the control device 31 of the remote device 30 receives the detection information from the position detection device 24 and the state detection device 26, as well as the sensing information from the sensing device 25, via the communication device 33, it stores this information in the internal memory 32a or displays it on the display device 34.

[0071] As shown in Figure 1, the remote device 30 may consist of a display terminal 70 and a control device 35. That is, the display terminal 70 may be a terminal device equipped with a control device 31, a storage device 32, a communication device 33, a display device 34, and a notification device 36. The display terminal 70 may be, for example, a portable terminal device such as a tablet or smartphone, or a stationary computer installed at a base station (not shown). The display terminal 70 may also be a user interface device.

[0072] Here, the planned route L1 of the work vehicle 1 will be described in detail. Figure 4 shows an example of the planned route L1. The remote device 30 can set the planned route L1. For example, map information including field H1 is pre-stored in the storage device 32. If map information including field H1 is not stored in the storage device 32, the remote device 30 can also access a map server (not shown) to obtain map information including field H1 and store it in the storage device 32. The control device 31 reads the map information including field H1 from the storage device 32 and displays field H1 as shown in Figure 4 on the display screen of the display device 34. By touching the work area WA1 on the display screen of the display device 34 (for example, using pen input), the planned route L1 can be pre-set for the work area WA1 of field H1 as shown in Figure 4. The planned route L1 consists of a plurality of straight paths L1a and a plurality of semi-circular turning paths L1b that connect the ends of two adjacent straight paths L1a. The set planned route L1 is registered in the storage device 32.

[0073] Furthermore, the work vehicle 1 can also have a pre-set planned route L1. For example, when the driver boards the work vehicle 1 and actually drives it in field H1, the on-board control device 21 of the work vehicle 1 can set the planned route L1 to the work area WA1 of field H1. The remote device 30 may receive this planned route L1 from the work vehicle 1 and store it in the storage device 32.

[0074] Incidentally, the speed range of work vehicles 1, such as tractors, is biased towards low speeds, unlike that of general vehicles such as automobiles. For this reason, even when a driver is actually on board and driving work vehicle 1 (actual driving), it is difficult to perceive changes in vehicle speed. Furthermore, in remote driving, it is already difficult to perceive vehicle speed, but when driving in fields such as rice paddies, fields, and pastures, there are no center lines and the surrounding scenery does not change much, making it even more difficult to perceive vehicle speed.

[0075] Therefore, the remote control system 100 and remote device 30 of this embodiment make it easier for the remote operator to perceive the speed of the work vehicle 1 by displaying a vehicle speed highlight K on the display device 34, as shown in Figure 5B, for example. Figure 5B is a diagram showing an example of a remote operation screen G2 with the highlight K.

[0076] When the work vehicle 1 is remotely controlled by the control device 35, the control device 31 displays a highlight K on the display device 34 that changes according to the driving information. For example, the control device 31 displays a vehicle speed highlight K on the display device 34 that changes according to the speed or acceleration of the work vehicle 1 indicated by the driving information. Here, the speed of the work vehicle 1 refers to either the speed per unit time, such as kilometers per hour, meters per minute, or meters per second, or the acceleration, which is the change in speed.

[0077] Specifically, the communication device 33 sequentially receives captured images of the direction of travel of the work vehicle 1 when the work vehicle 1 is being remotely operated. The display device 34 sequentially displays the captured images on the remote operation screen G2. The control device 31 displays a highlight K on the remote operation screen G2 when the first condition, which will be described later, is met. More specifically, when the first condition is met, the control device 31 displays one of the highlight K selected from the first to eighth display modes.

[0078] Figure 5B shows the highlighting K of the first display mode. For example, the control device 31 superimposes the highlighting K of the first display mode onto the captured image of the remote operation screen G2. The control device 31 superimposes a display body K1 extending in the direction of travel of the work vehicle 1 as the highlighting K of the first display mode, and changes the movement speed of the display body K1 according to the speed or acceleration of the work vehicle 1. The display body K1 is, for example, a dashed line demarcation line (dashed center line, dashed lane boundary line, etc.), and consists of multiple line segments Ka arranged in a line along the direction of travel of the work vehicle 1.

[0079] The control device 31 performs enhancement processing so that the perceived speed of the remote operator viewing the remote operation screen G2 feels faster than the actual speed. In the enhancement processing, if the actual speed (e.g., km / h) or acceleration of the work vehicle 1 per unit time increases, the control device 31 performs enhancement display K so that the perceived speed of the remote operator viewing the display device 34 feels faster than the actual speed. In addition, in the enhancement processing, even if the actual speed or acceleration of the work vehicle 1 per unit time decreases, the control device 31 performs enhancement display K so that the perceived speed of the remote operator feels faster than the actual speed of the work vehicle 1.

[0080] When the vehicle speed or acceleration of work vehicle 1 increases, the display's movement speed also increases, and when the vehicle speed or acceleration decreases, the display's movement speed decreases. However, in both cases, it is preferable to set it so that the operator perceives the speed as faster than the actual vehicle speed.

[0081] Generally, the vehicle speed of work vehicle 1 (e.g., a tractor) is biased towards the low-speed range, making it difficult for the operator to perceive the speed when remotely controlled. However, with the above-mentioned highlighting K, the perceived speed can be exaggerated to feel faster than the actual speed.

[0082] Furthermore, in the enhancement process, the control device 31 will react if the actual acceleration of the work vehicle 1 increases. In addition, the control device 31 performs an emphasis K so that the remote operator perceives the speed as faster than the actual acceleration of the work vehicle 1. Furthermore, in the emphasis process, even if the acceleration of the work vehicle 1 decreases, the control device 31 performs an emphasis K so that the remote operator perceives the speed as faster than the actual acceleration of the work vehicle 1.

[0083] For example, in the enhancement process, if the travel speed of the work vehicle 1 is [1 km / h], the movement display speed of the display unit K1 on the remote driving screen G2 is set to the first movement display speed. The first movement display speed may be the same as or greater than the actual speed [1 km / h] (a speed obtained by multiplying the actual speed or acceleration increase by a coefficient of 1 or more). Then, in the enhancement process, if the travel speed of the work vehicle 1 is [2 km / h], the movement display speed of the display unit K1 is set to the second movement display speed, which is faster than the first movement display speed. The second movement display speed may be the same as or greater than the actual speed [2 km / h], as long as it is faster than the first movement display speed.

[0084] When the first condition is met, the control device 31 displays the highlight K on the remote operation screen G2 as shown in Figure 5B, etc., and when the first condition is not met, it does not display the highlight K on the remote operation screen G2 as shown in Figure 5A.

[0085] Specifically, the control device 31 determines that the first condition is met if the amount of change in multiple captured images is less than a threshold, and determines that the first condition is not met if the amount of change in multiple captured images is greater than or equal to the threshold. For example, the control device 31 can determine whether the amount of change in multiple captured images is greater than or equal to the threshold by using known difference image processing. For example, the control device 31 generates a difference image by differentiating two captured images. Then, the control device 31 determines that the amount of change in multiple captured images is less than a threshold and determines that the first condition is met if the total number of difference pixel values ​​greater than or equal to a predetermined value in a predetermined range of this difference image is less than a specified number. The predetermined range may be the entire difference image or a part of the difference image (for example, a part that exclusively corresponds to the road surface). On the other hand, the control device 31 determines that the amount of change in multiple captured images is greater than or equal to the threshold and determines that the first condition is not met if the total number of difference pixel values ​​greater than or equal to a predetermined value is greater than or equal to a specified number.

[0086] Furthermore, the control device 31 determines that the first condition is met if the captured image does not contain road markings, and determines that the first condition is not met if the captured image contains road markings. Examples of road markings include road markings on the road surface (indicator markings such as center lines and lane boundaries, and regulatory markings such as marks indicating driving restrictions). The control device 31 determines that the first condition is met if it determines, through known image analysis processing (e.g., pattern matching processing), that there are no road markings in the captured image. In other words, it can determine that the area in which the work vehicle 1 is traveling via remote control is an area with little variation in the surrounding scenery (e.g., pasture, field, etc.). On the other hand, the control device 31 determines that the first condition is not met if it determines, through known image analysis processing (e.g., pattern matching processing), that the captured image contains road markings. In other words, it can determine that the area in which the work vehicle 1 is traveling via remote control is an area with a lot of variation in the surrounding scenery (e.g., public road).

[0087] Figure 5C shows the highlighting K of the second display mode on the remote driving screen G2. As shown in Figure 5C, the control device 31 can superimpose a display element K2 extending in the direction of travel of the work vehicle 1 onto the captured image of the remote driving screen G2 as the highlighting K of the second display mode, and can also change the color of the display element K2 according to the speed of the work vehicle 1. The display element K2 is, for example, a solid line lane marking (solid center line, solid lane boundary line, etc.), with one line Kb arranged along the direction of travel of the work vehicle 1.

[0088] Specifically, the control device 31, according to the speed (travel speed) of the work vehicle 1, as shown in Figure 5C. The color of the highlight K (display unit K2) in the second display mode on the remote driving screen G2 is changed. For example, if the vehicle speed of the work vehicle 1 is low, the display unit K2 is displayed in blue, and if it is high speed, the display unit K2 is displayed in red. The control device 31 may also change the color of the highlight K (display unit K2) as the travel speed increases, for example, in the order of green, yellow-green, yellow, yellowish-orange, orange, reddish-orange, and red in the Ostwald color system. For example, when the travel speed is [0 km / h], the color of the highlight K is green, and for each unit speed increase in travel speed (for example, [0.5 km / h]), it changes in the order of yellow-green, yellow, yellowish-orange, orange, reddish-orange, and red. This is just one example, and it may also change in the order of violet, indigo, blue, green, yellow, orange, and red, or in the order of green, yellow, orange, and red.

[0089] Furthermore, the control device 31 may change the color of the highlighting K (display element K1) of the first display mode on the remote operation screen G2 shown in Figure 5B, according to the speed (travel speed) of the work vehicle 1.

[0090] Figure 6 shows the highlighting display K of the third display mode on the remote operation screen G2. As shown in Figure 6, the control device 31 can display multiple virtual display objects Kc in the direction of travel of the work vehicle 1 on the captured image of the remote operation screen G2 as the highlighting display K of the third display mode, and can also change the movement display speed of the multiple virtual display objects Kc according to the speed of the work vehicle 1. The virtual display objects Kc are, for example, road cones, poles, etc. The highlighting display K3 shown in Figure 6 is composed of multiple virtual display objects Kc.

[0091] For example, when the travel speed of the work vehicle 1 is [1 km / h], the control device 31 sets the movement display speed of the virtual display object Kc on the remote operation screen G2 as the first movement display speed. The first movement display speed may be the same as the actual speed [1 km / h] or it may be different. Then, when the travel speed of the work vehicle 1 is [2 km / h], the control device 31 sets the movement display speed of the virtual display object Kc as a second movement display speed that is faster than the first movement display speed. The second movement display speed may be the same as the actual speed [2 km / h] or it may be different, as long as it is faster than the first movement display speed.

[0092] Figure 7A shows the highlighting K of the fourth display mode on the remote operation screen G2. As shown in Figure 7A, the control device 31 can display the highlighting K of the fourth display mode on the peripheral edge PP of the remote operation screen G2. Alternatively, the highlighting K of the fourth display mode may be displayed on the peripheral edge PP of the captured image. The peripheral edge PP corresponds to the acceleration effect area K4. The peripheral edge PP can be said to be the area where the acceleration effect area K4 is displayed. For example, the control device 31 changes the area of ​​the peripheral edge PP according to the speed or acceleration of the work vehicle 1. Changing the area of ​​the peripheral edge PP here includes changing the size of the area, changing the shape, color, and other design elements. For example, when the speed or acceleration of the work vehicle 1 increases, the control device 31 increases the area of ​​the peripheral edge PP (i.e., the acceleration effect area K4). Since the acceleration effect area K4 has a rectangular frame shape, it has areas on the left side, top side, right side, and bottom side. In the acceleration effect region K4 shown in Figure 7A, for example, the widths d1 of the left and right sides are equal to the heights d2 of the top and bottom sides, but they may be different lengths.

[0093] In the acceleration effect area K4, flow lines indicating speed are represented in a manner that extends roughly radially from the outer contour of the captured image. In other words, a flow display is shown in the acceleration effect area K4. The control device 31 displays the captured image covering the entire remote operation screen G2, but is not limited to this, as it is reduced in size to fit within the area of ​​the remote operation screen G2 excluding the peripheral area PP. For example, the control device 31 may superimpose the rectangular frame-shaped acceleration effect area K4 on the captured image without changing the size of the captured image covering the entire remote operation screen G2. In this case, the acceleration effect area K4 may be displayed transparent or semi-transparent in areas other than the black lines indicating flow.

[0094] The control device 31 increases the size of the acceleration display area K4 as the speed or acceleration of the work vehicle 1 increases. In the acceleration display area K4 shown in Figure 7B, for example, the widths d3 of the left and right sides are larger than the widths d1, and the heights d4 of the top and bottom sides are larger than the heights d2. Therefore, the acceleration display area K4 shown in Figure 7B is larger than the acceleration display area K4 shown in Figure 7A. In other words, the control device 31 increases the widths and heights of the acceleration display area K4 as the speed or acceleration of the work vehicle 1 increases. Note that in the acceleration display area K4 shown in Figure 7B, the widths d3 of the left and right sides and the heights d4 of the top and bottom sides are assumed to be equal, but they may be of different lengths.

[0095] If the first condition is met, the control device 31 may, for example, display the acceleration effect area K4 shown in Figure 7A on the remote control screen G2 if the travel speed of the work vehicle 1 is [1 km / h], and display the acceleration effect area K4 shown in Figure 7B on the remote control screen G2 if the travel speed of the work vehicle 1 is [2 km / h]. Alternatively, for example, if the acceleration of the work vehicle 1 is the first acceleration, the control device 31 may display the acceleration effect area K4 shown in Figure 7A on the remote control screen G2, and if the acceleration of the work vehicle 1 is the second acceleration which is greater than the first acceleration, the control device 31 may display the acceleration effect area K4 shown in Figure 7B on the remote control screen G2.

[0096] Furthermore, the control device 31 can display the acceleration effect area K5 shown in Figure 8A instead of the acceleration effect area K4 shown in Figures 7A and 7B. Figure 8A is a diagram showing the highlighting K of the fifth display mode on the remote operation screen G2. The acceleration effect area K4 shown in Figures 7A and 7B is a display mode that has flow lines to indicate speed. In contrast, the acceleration effect area K5 shown in Figure 8A is a display mode that has blurring to indicate speed. In other words, the acceleration effect area K5 shown in Figure 8A is displayed as the highlighting K of the fifth display mode. For example, in the acceleration effect area K5, the density of the blur indicating speed increases as it moves approximately radially away from the outer contour of the captured image. In other words, the acceleration effect area K5 is displayed with blurring. Also, similar to the case of the acceleration effect area K4 shown in Figures 7A and 7B, the control device 31 can increase the display of the acceleration effect area K5 as the speed or acceleration of the work vehicle 1 increases.

[0097] The control device 31 can change the color of specific areas (e.g., windows 41a, 41b) other than the captured image on the remote control screen G2 as a highlighting K, according to the speed or acceleration of the work vehicle 1. The control device 31 displays a sixth display mode of highlighting K, for example, as shown in Figure 5A, which changes the color of specific areas on the remote control screen G2. For example, if the speed of the work vehicle 1 is low, the specific areas on the remote control screen G2 are displayed in blue, and if the speed is high, the specific areas on the remote control screen G2 are displayed in red. The control device 31 may also change the color of specific areas on the remote control screen G2 in the order of green, yellow-green, yellow, yellowish-orange, orange, reddish-orange, and red in the Ostwald color system as the travel speed increases. The color of the entire remote control screen G2 may also be changed.

[0098] Figure 8B shows the seventh display mode highlighting K on the remote operation screen G2. The control device 31 can change the color of the frame F on the remote operation screen G2 according to the speed or acceleration of the work vehicle 1 as highlighting K. As shown in Figure 8B, the control device 31 changes the color of the frame F on the remote operation screen G2. For example, if the speed of the work vehicle 1 is low, the frame F on the remote operation screen G2 is displayed in blue, and if it is high, the frame F on the remote operation screen G2 is displayed in red. Alternatively, the control device 31 may change the color of the frame F on the remote operation screen G2 in the order of green, yellow-green, yellow, yellowish-orange, orange, reddish-orange, and red in the Ostwald color system as the travel speed increases.

[0099] Figure 8C shows the eighth display mode highlighting K in window 43b of the remote operation screen G2. As shown in Figure 8C, when the work vehicle 1 is moving in reverse, the control device 31 displays the captured image of the work vehicle 1 moving in reverse on the remote operation screen G2 and guides the remote operation screen G2. The guide lines K6 are displayed. The guide lines K6 are, for example, the expected path of the work vehicle 1 when reversing, the parking guide line for the work vehicle 1, and the expected path of the work machine attached to the work vehicle 1. In Figure 8C, a pair of guide lines K6 with an angle θ1 are displayed. The control device 31 can change the appearance (shape or color) of the guide lines K6 displayed on the remote operation screen G2 as a highlight K according to the speed or acceleration of the work vehicle 1. For example, according to the speed or acceleration of the work vehicle 1, the angle of the pair of guide lines K6 changes from angle θ1 to angle θ2. Angle θ2 is smaller than angle θ1. In other words, as the speed or acceleration of the work vehicle 1 increases, a guide line K61 with a smaller angle is displayed. Alternatively, a guide line K61 with a larger angle may be displayed. The guide lines K61 may also be displayed by their color.

[0100] For example, when the working vehicle 1 is moving in reverse at a speed of [1 km / h], the control device 31 displays the guide line K6 shown in Figure 8C on the remote operation screen G2, and when the working vehicle 1 is moving in reverse at a speed of [2 km / h], it changes the guide line K6 shown in Figure 8C to the guide line K61.

[0101] The control device 31 can select the type of highlighting K from the first to eighth display modes according to the selection operation by the remote operator. Figures 9A and 9C show examples of the selection screen G1 of the display device 34. The control device 31 can change to the selected highlighting K from the first to eighth display modes by a selection operation on the selection screen G1. Specifically, when the remote operator gives an instruction to select a highlighting K, the control device 31 displays the selection screen G1 on the display device 34 as shown in Figure 9A. The selection screen G1 shown in Figure 9A shows that the current highlighting type is set to the center line (dashed line) shown in Figure 5B.

[0102] When the remote operator touches the change button B1 on the selection screen G1 shown in Figure 9A, the control device 31 displays selectable items on the display device 34, as shown in Figure 9B, indicating the type of highlighting K for the first to eighth display modes. The display device 34 displays eight selectable items: the highlighting K for the first display mode (center line (dashed line)) shown in Figure 5B, the highlighting K for the second display mode (center line (color)) shown in Figure 5C, the highlighting K for the third display mode (road cone) shown in Figure 6, the highlighting K for the fourth display mode (acceleration effect area K4 (blur)) shown in Figure 7A, the highlighting K for the fifth display mode (acceleration effect area K5 (blur)) shown in Figure 8A, the highlighting K for the sixth display mode (overall color of the remote driving screen G2), the highlighting K for the seventh display mode (color of the frame F of the remote driving screen G2) shown in Figure 8B, and the highlighting K for the eighth display mode (back monitor guide line K6) shown in Figure 8C. The control device 31 moves the selectable items upward each time the remote operator operates the upward button B3, and moves the selectable items downward each time the remote operator operates the downward button B4. In Figure 9B, the downward button B4 has been operated twice, and the display has been changed to the third display mode shown in Figure 6, which is highlighted as a road cone K.

[0103] As shown in Figure 9C, when the remote operator operates the OK button B2, the control device 31 determines the selected item to be highlighted K. Figure 9C shows that the fourth display mode highlighted K (acceleration effect area K4 (flow)) shown in Figure 7A has been determined. The control device 31 may also change the type of highlighted K by the remote operator operating one or more selection buttons (not shown) located around the driver's seat 10.

[0104] Here, using Figures 10A and 10B, we will explain the process of overlaying a vehicle speed highlight K on the remote operation screen G2 of the display device 34 when the work vehicle 1 is remotely controlled. Figure 10A is a flowchart showing the operation of the work vehicle 1 during remote operation. Figure 10B is a flowchart showing the operation of the remote device 30 when the work vehicle 1 is remotely controlled.

[0105] When a remote operator requests the start of remote control, the control device 31 causes the communication device 33 to send a request signal to the work vehicle 1 requesting information detected by the work vehicle 1, as shown in Figure 10B (S21).

[0106] As shown in Figure 10A, when the onboard control device 21 of the work vehicle 1 receives a request signal from the remote device 30 via the onboard communication device 23 (S11), it causes the onboard communication device 23 to transmit the detection information from the position detection device 24 and the state detection device 26, and the sensing information from the sensing device 25 to the remote device 30 (S12). As mentioned above, the detection information from the state detection device 26 includes operation information for the work vehicle 1 and the work device 2 (at least one of the following: speed (or acceleration) of the work vehicle 1, gear shift position of the transmission 5, braking position of the braking device 13, and operating position of the work device 2).

[0107] Returning to Figure 10B, when the control device 31 of the remote device 30 receives detection information from the position detection device 24 and the state detection device 26, and sensing information from the sensing device 25, via the communication device 33 (S22), it stores this information in the storage device 32. The control device 31 also reads the detection information from the position detection device 24 and the state detection device 26, the sensing information from the sensing device 25, the device information indicating the specifications of the work vehicle 1 and the work equipment 2, and the map information of the area around the work vehicle 1, stored in the storage device 32, into the internal memory 32a (S23).

[0108] The communication device 33 receives device information indicating the specifications of the work vehicle 1 and the work equipment 2, and stores it in the storage device 32. The storage device 32 also has map information of the area where the work vehicle 1 is located pre-stored in it. In step S23, the control device 31 extracts the location of the work vehicle 1 from the detection information of the location detection device 24, considers the area within a predetermined distance from the location of the work vehicle 1 as the surrounding area of ​​the work vehicle 1, and reads the map information of that area from the storage device 32 into the internal memory 32a. As another example, in step S23, the control device 31 may receive the map information of the area within a predetermined distance from the location of the work vehicle 1 from an external server via the internet or the like using the communication device 33, and read the received map information.

[0109] Then, the control device 31 displays the remote operation screen G2 on the display device 34 based on the detection information from the position detection device 24 and the state detection device 26, the sensing information from the sensing device 25, the device information, and the map information (S24).

[0110] The control device 31 determines whether or not a type of highlighting K has been selected (S25). If, for example, the remote operator has given instructions to select a type of highlighting K on the selection screen G1 shown in Figures 9A to 9C (S25: YES), the control device 31 determines the selected type of highlighting K (S26). On the other hand, if the remote operator has not given instructions to select a type of highlighting K (S25: NO), the control device 31 determines the default value or the type of highlighting K that was set last time (S26).

[0111] Here, in S26, the control device 31 has decided on the first display mode shown in Figure 5B, highlighted K (center line (dashed line)), as shown in Figure 9A.

[0112] The control device 31 determines whether or not there is a steering operation by the steering device 35 (S27). If there is a steering operation by the steering device 35 (S27: YES), the control device 31 causes the communication device 33 to transmit a remote control signal corresponding to the steering operation by the steering device 35 to the work vehicle 1 (S28). For example, a remote control signal including various operation signals corresponding to the operation of the steering wheel 35a, accelerator pedal 35b, brake pedal 35c, and gear shift lever 35d by the remote operator is transmitted from the remote device 30 to the work vehicle 1.

[0113] Returning to Figure 10A, after S12, the on-board control device 21 determines whether or not there is a remote control signal from the remote device 30 (S13). If the on-board control device 21 receives a remote control signal from the remote device 30 via the on-board communication device 23 (S13: YES), it controls the movement of the work vehicle 1, the work performed by the work device 2, and other operations of the work vehicle 1 based on the sensing information from the sensing device 25, the detection information from the state detection device 26, the detection information from the position detection device 24, and the remote control signal (S14).

[0114] The work vehicle 1 operates according to remote control signals from the remote control device 30. In other words, the onboard control device 21 operates the steering wheel 11a (Figure 2), accelerator pedal 35b, brake pedal 35c, and gear shift lever 35d of the control device 11 according to various operation signals corresponding to the operation of the steering wheel 35a, accelerator pedal 35b, brake pedal 35c, and gear shift lever 35d by the remote operator.

[0115] On the other hand, the control device 31 of the remote device 30 proceeds to the screen display update process (S29) after S28, or if there is no control operation by the control device 35 (S27: NO).

[0116] The control device 31 performs a screen display update process (S29). That is, the control device 31 performs a screen display update process each time it receives corresponding data from the work vehicle 1 during remote operation. Figure 11 is a flowchart of the screen display update process. The control device 31 performs an image analysis process (S41).

[0117] Specifically, the communication device 33 of the remote device 30 sequentially receives detection information from the position detection device 24 and state detection device 26 of the work vehicle 1, as well as sensing information from the sensing device 25. The communication device 33 sequentially receives the corresponding data included in the detection information (i.e., corresponding data that associates the image captured by the internal camera 25c1, the driving information of the work vehicle 1 detected by the state detection device 26, and the position information of the work vehicle 1 detected by the position detection device 24).

[0118] The control device 31 performs image analysis processing on the sequentially received captured images (images captured by the internal camera 25c1) (S41). The control device 31 determines whether or not road markings are included in the captured images by known image analysis processing (e.g., pattern matching processing). The control device 31 determines whether or not the first condition is met (S42). If the captured images do not contain road markings, the control device 31 determines that the first condition is met (S42: YES) and that highlighting is present (S43). On the other hand, if the captured images contain road markings, the control device 31 determines that the first condition is not met (S42: NO) and that there is no highlighting (S44).

[0119] The control device 31 may also determine in S41 whether the amount of change in multiple captured images is greater than or equal to a threshold by performing known differential image processing. If the amount of change in multiple captured images is less than the threshold, the control device 31 determines that the first condition is met (S42: YES) and determines that highlighting is enabled (S43). On the other hand, if the amount of change in multiple captured images is greater than or equal to the threshold, the control device 31 determines that the first condition is not met (S42: NO) and determines that highlighting is disabled (S44).

[0120] The control device 31 updates the screen display (S45). Specifically, the control device 31 updates the captured image displayed on the remote operation screen G2 with the captured image of the corresponding data received by the communication device 33, and if the first condition is met (S42: YES), it superimposes the highlighting K onto the captured image. Here, since the control device 31 has decided on the highlighting K of the first display mode in S26, it superimposes the highlighting K of the first display mode (center line (dashed line)) as shown in Figure 5B.

[0121] Meanwhile, the control device 31 updates the captured image displayed on the remote operation screen G2 with the captured image of the corresponding data received by the communication device 33. If the first condition is not met, (S42:NO) the highlighting K is not superimposed on the captured image. As shown in Figure 5A, the remote operation screen G2 without the highlighting K is displayed.

[0122] Returning to Figure 10A, the on-board control device 21 sets an emergency stop area as the region from the work vehicle 1 to a predetermined distance in the direction of travel of the work vehicle 1. Therefore, when the work vehicle 1 is being driven by remote control via the remote device 30, if the on-board control device 21 detects that an obstacle has entered the emergency stop area based on sensing information from the sensing device 25, it automatically stops the work vehicle 1 to an emergency stop to avoid contact between the work vehicle 1 and the obstacle (S15:YES), and returns to S12.

[0123] On the other hand, if the on-board control unit 21 does not detect that an obstacle has entered the emergency stop area (S15: NO), that is, if there is no obstacle in the emergency stop area, it determines whether or not to terminate the remote operation (S16). For example, if the on-board control unit 21 receives a remote operation termination signal from the remote device 30 (S16: YES), it terminates the remote operation. On the other hand, if the on-board control unit 21 has not received a remote operation termination signal from the remote device 30 (S16: NO), it returns to S12.

[0124] Returning to Figure 10B, the control device 31 determines whether or not to terminate the remote operation (S30). For example, if the remote operator has not instructed the remote operation to terminate (S30: NO), the control device 31 returns to S27. On the other hand, if the remote operation has been instructed to terminate (S30: YES), the control device 31 terminates the remote operation.

[0125] In the above embodiment, when remotely operating the work vehicle 1 within a field, the highlight K is superimposed on the remote operation screen G2 when it is determined that the first condition has been met, but the embodiment is not limited to this. For example, when remotely operating the work vehicle 1 for movement between fields, movement between a field and a barn, or movement on a farm road or public road, the highlight K may be superimposed on the remote operation screen G2 when it is determined that the first condition has been met.

[0126] The remote control device 30 of this embodiment described above comprises a control device 35 for remotely controlling the work vehicle 1, a communication device 33 for receiving driving information indicating the speed of the work vehicle 1, a display device 34, and a control device 31 that, when the work vehicle 1 is remotely controlled by the control device 35, causes the display device 34 to display a vehicle speed highlight K that changes according to the speed of the work vehicle 1 indicated by the driving information. With this configuration, when the work vehicle 1 is remotely driven, a highlight K that changes according to the speed of the work vehicle 1 (i.e., a vehicle speed highlight K) is displayed. This highlight K makes it easier for the remote operator to perceive the speed of the work vehicle 1. In other words, it makes the remote operator aware of the speed of the work vehicle 1.

[0127] Furthermore, when the work vehicle 1 is remotely operated, the communication device 33 sequentially receives captured images of the direction of travel of the work vehicle 1, the display device 34 sequentially displays the captured images on the remote operation screen G2, and the control device 31 displays highlighting K on the remote operation screen G2. With this configuration, since highlighting K is displayed on the remote operation screen G2, which sequentially displays captured images of the direction of travel of the work vehicle 1, it is possible to add a sense of the speed of the work vehicle 1 to the captured images on the remote operation screen G2, making it easier to perceive the speed of the work vehicle 1 from the remote operation screen G2.

[0128] Furthermore, the control device 31 displays the highlight K on the remote operation screen G2 when the first condition is met, and does not display the highlight K on the remote operation screen G2 when the first condition is not met. With this configuration, it is possible to appropriately switch whether or not to display the highlight K on the remote operation screen G2. This allows for the appropriate switching of whether or not the remote operator is provided with a sense of the speed of the work vehicle 1.

[0129] Furthermore, the control device 31 determines that the first condition is met if the amount of change in multiple captured images is less than a threshold, and determines that the first condition is not met if the amount of change in multiple captured images is greater than or equal to the threshold. With this configuration, if the amount of change in multiple captured images is less than a threshold, it can be determined that the area in which the work vehicle 1 is traveling by remote operation is an area with little change in the surrounding scenery. For example, pastures and fields are generally areas (land) of the same color and do not have center lines, so they are areas (land) with little change in color. In such areas with little change in the surrounding scenery (e.g., pastures and fields), there is also the problem that it is difficult to perceive the speed of the vehicle. In contrast, in areas with little change in the surrounding scenery, highlighting K is applied, so the remote operator can more easily perceive the speed of the work vehicle 1 in areas with little change in the surrounding scenery. In other words, when remotely operating the work vehicle 1 in a place where it is difficult to perceive the speed of the work vehicle 1, the remote operator can be made aware of the speed of the work vehicle 1. On the other hand, if the amount of change in multiple captured images exceeds a threshold, it can be determined that the area in which the work vehicle 1 is traveling is an area with varied surrounding scenery. In this way, in areas with varied surrounding scenery, the vehicle speed is easier to perceive compared to areas with little variation in surrounding scenery, so the highlight K is not displayed.

[0130] Furthermore, the control device 31 determines that the first condition is met if the captured image does not include road markings, and determines that the first condition is not met if the captured image includes road markings. With this configuration, if there are no road markings in the captured image (for example, road markings on the road surface (indicating markings such as center lines and lane boundaries, and regulatory markings such as marks indicating driving restrictions)), it can be determined that the area in which the work vehicle 1 is traveling via remote control is an area with little variation in the surrounding scenery (for example, a pasture or a field). In such areas with little variation in the surrounding scenery, highlighting K is displayed, making it easier for the remote operator to perceive the speed of the work vehicle 1 in areas with little variation in the surrounding scenery. On the other hand, if the captured image includes road markings, it can be determined that the area in which the work vehicle 1 is traveling via remote control is an area with a lot of variation in the surrounding scenery (for example, a public road). In this way, in areas with a lot of variation in the surrounding scenery, it is easier to perceive the vehicle speed compared to areas with little variation in the surrounding scenery, so highlighting K is not displayed.

[0131] Furthermore, the control device 31 superimposes a display element K1 extending in the direction of travel of the work vehicle 1 onto the captured image of the remote driving screen G2 as a highlighting display K, and changes the movement speed of the display element K1 according to the speed of the work vehicle 1. With this configuration, the control device 31 superimposes a display element K1 (for example, a center line, lane boundary line, etc.) extending in the direction of travel of the work vehicle 1 onto the captured image of the remote driving screen G2 as a highlighting display K, and changes the movement speed of the display element K1 according to the speed of the work vehicle 1. In other words, by increasing the movement speed of the display element K1, the vehicle speed can be emphasized. In this way, since the display element K1 whose movement speed is changed according to the speed of the work vehicle 1 is superimposed on the captured image of the remote driving screen G2, a sense of the speed of the work vehicle 1 can be added to the captured image of the remote driving screen G2, making it easier to perceive the speed of the work vehicle 1 from the remote driving screen G2.

[0132] Furthermore, the control device 31 superimposes display elements K1 and K2, which extend in the direction of travel of the work vehicle 1, onto the captured image of the remote driving screen G2 as a highlighting display K, and changes the color of the display elements K1 and K2 according to the speed of the work vehicle 1. With this configuration, the control device 31 superimposes display elements K1 and K2 (for example, a center line, lane boundary line, etc.) that extend in the direction of travel of the work vehicle 1 onto the captured image of the remote driving screen G2 as a highlighting display K, and changes the color of the display elements K1 and K2 according to the speed of the work vehicle 1. In other words, by changing the color of the display elements K1 and K2, the vehicle speed can be highlighted. In this way, the display elements K1 and K2, whose color changes according to the speed of the work vehicle 1, are superimposed on the captured image of the remote driving screen G2. Since it is displayed, a sense of the speed of the work vehicle 1 can be added to the captured image on the remote driving screen G2, making it easier to perceive the speed of the work vehicle 1 from the remote driving screen G2.

[0133] Furthermore, the control device 31 displays multiple virtual display objects Kc as highlighting K on the captured image of the remote driving screen G2, arranged in the direction of travel of the work vehicle 1, and changes the movement display speed of the multiple virtual display objects Kc according to the speed of the work vehicle 1. With this configuration, the control device 31 displays multiple virtual display objects Kc (for example, road cones, etc.) as highlighting K on the captured image of the remote driving screen G2, arranged in the direction of travel of the work vehicle 1, and changes the movement display speed of the multiple virtual display objects Kc according to the speed of the work vehicle 1. In other words, by increasing the movement display speed of the multiple virtual display objects Kc, the vehicle speed can be emphasized. In this way, since multiple virtual display objects Kc whose movement display speed is changed according to the speed of the work vehicle 1 are displayed as highlighting K on the captured image of the remote driving screen G2, a sense of the speed of the work vehicle 1 can be added to the captured image of the remote driving screen G2, making it easier to feel the speed of the work vehicle 1 from the remote driving screen G2.

[0134] Furthermore, the control device 31 displays acceleration effect areas K4 and K5 on the peripheral edge PP of the remote control screen G2 as a highlighting indicator K, according to the speed or acceleration of the work vehicle 1. With this configuration, the control device 31 displays acceleration effect areas K4 and K5 on the peripheral edge PP of the remote control screen G2 as a highlighting indicator K, according to the speed or acceleration of the work vehicle 1. In other words, the vehicle speed can be highlighted by the acceleration effect areas K4 and K5 displayed on the peripheral edge PP of the remote control screen G2.

[0135] Furthermore, the control device 31 increases the size of the acceleration effect areas K4 and K5 as the speed or acceleration of the work vehicle 1 increases. With this configuration, the acceleration effect areas K4 and K5 displayed on the peripheral PP of the remote operation screen G2 increase in size as the speed or acceleration of the work vehicle 1 increases, so the size of the captured image on the remote operation screen G2 decreases. Therefore, it is possible to provide a display effect in which the field of view narrows as the speed or acceleration of the work vehicle 1 increases. This makes it possible to further enhance the sense of speed of the work vehicle 1 in the captured image on the remote operation screen G2, and makes it easier to feel the speed of the work vehicle 1 from the remote operation screen G2.

[0136] Furthermore, the control device 31 changes the overall color of the remote control screen G2 according to the speed or acceleration of the work vehicle 1 as a highlighting indicator K. With this configuration, the control device 31 changes the overall color of the remote control screen G2 according to the speed or acceleration of the work vehicle 1 as a highlighting indicator K. In other words, by changing the overall color of the remote control screen G2 according to the speed or acceleration of the work vehicle 1, the vehicle speed or acceleration can be highlighted and displayed.

[0137] Furthermore, the control device 31 changes the color of the frame F of the remote control screen G2 according to the speed or acceleration of the work vehicle 1 as a highlighting indicator K. With this configuration, the control device 31 changes the color of the frame F of the remote control screen G2 according to the speed or acceleration of the work vehicle 1 as a highlighting indicator K. In other words, by changing the color of the frame F of the remote control screen G2 according to the speed or acceleration of the work vehicle 1, the vehicle speed or acceleration can be highlighted and displayed.

[0138] Furthermore, when the work vehicle 1 is moving in reverse, the control device 31 displays the captured image of the work vehicle 1 in reverse on the remote driving screen G2, and as a highlighting K, changes the form of the guide lines displayed on the remote driving screen G2 according to the speed or acceleration of the work vehicle 1. With this configuration, the control device 31 changes the form (color, shape, etc.) of the guide lines (predicted route, parking guide lines, etc.) displayed on the remote driving screen G2 as a highlighting K according to the speed or acceleration of the work vehicle 1. In other words, by changing the form of the guide lines on the remote driving screen G2 according to the speed or acceleration of the work vehicle 1, the vehicle speed or acceleration can be highlighted and displayed. ru.

[0139] Furthermore, the remote control system 100 comprises a work vehicle 1 and a remote device 30. The work vehicle 1 includes a detection device (state detection device 26) for detecting the speed of the work vehicle 1, an imaging device (camera 25c) for imaging the direction of travel of the work vehicle 1, and an on-board communication device 23 for transmitting correspondence data that associates the speed or acceleration information detected by the state detection device 26 with the image captured by the camera 25c. The communication device 33 of the remote device 30 receives the correspondence data transmitted from the on-board communication device 23. With this configuration, when the work vehicle 1 is remotely controlled by the control device 35 of the remote device 30, a highlighting K (i.e., a vehicle speed highlighting K) that changes according to the speed of the work vehicle 1 is displayed on the display device 34 of the remote device 30. This highlighting K makes it easier for the remote operator to perceive the speed of the work vehicle 1. In other words, it makes the remote operator aware of the speed of the work vehicle 1.

[0140] <First variation> In the first modified remote device 30 and remote control system 100, as shown in Figures 12A to 12E, the control device 31 can display a highlighting K on the remote operation screen G2 that changes the range displayed as an captured image when the work vehicle 1 is accelerated, decelerated, or has a sudden steering operation. Figure 12A is a diagram showing an example of the remote operation screen G2 of the first modified example. Figures 12B to 12E are diagrams showing the eighth display mode of the highlighting K on the remote operation screen G2 of the first modified example, respectively.

[0141] As shown in Figure 12A, the range of the image captured by camera 25c that is displayed as the captured image (i.e., the range displayed in window 43a) is predetermined. For example, the range to be displayed as the captured image is a rectangular area with the center line of the direction in which camera 25c is pointed (shooting direction) as its center point.

[0142] As shown in Figure 12B, when the work vehicle 1 is accelerating, the control device 31 changes the range displayed as the captured image on the remote control screen G2 upwards by a distance D corresponding to the change in acceleration. For example, when the acceleration of the work vehicle 1 is above a specified value, the control device 31 moves the range displayed as the captured image on the remote control screen G2 upwards by a distance D corresponding to the change in acceleration. In other words, the captured image is displayed as if the camera direction has been shifted upwards. On the other hand, when the acceleration of the work vehicle 1 is below a specified value, the control device 31 does not move the range displayed as the captured image on the remote control screen G2. Thus, if there is rapid acceleration (acceleration above a specified value), the highlighting K shown in Figure 12B is executed, and if there is gradual acceleration (acceleration below a specified value), the highlighting K shown in Figure 12B is not executed, and the remote control screen G2 shown in Figure 12A is displayed.

[0143] As shown in Figure 12C, the control device 31 changes the range displayed as the captured image on the remote control screen G2 by a distance D corresponding to the change in acceleration when the work vehicle 1 is decelerating. For example, if the deceleration of the work vehicle 1 is greater than or equal to a specified value, the control device 31 moves the range displayed as the captured image on the remote control screen G2 downward by a distance D corresponding to the change in acceleration. In other words, the captured image is displayed as if the camera direction has been shifted downward. On the other hand, if the deceleration of the work vehicle 1 is less than a specified value, the control device 31 does not move the range displayed as the captured image on the remote control screen G2. Thus, if there is a rapid deceleration (deceleration greater than or equal to a specified value), the highlighting K shown in Figure 12C is executed, and if there is a gradual deceleration (deceleration less than a specified value), the highlighting K shown in Figure 12C is not executed, and the remote control screen G2 shown in Figure 12A is displayed.

[0144] As shown in Figure 12D, when the work vehicle 1 turns left, the control device 31 changes the range displayed as the captured image on the remote control screen G2 to the right by a distance D corresponding to the steering angle of the left turn. For example, if the steering angle of the work vehicle 1 when turning left is greater than or equal to a specified value, the range displayed as the captured image on the remote driving screen G2 is moved to the right by a distance D corresponding to the steering angle. If the steering angle of the work vehicle 1 when turning left is less than the specified value, the range displayed as the captured image on the remote driving screen G2 is not moved. In this way, if the left steering operation is sudden, the highlighting K shown in Figure 12D is executed, and if the left steering operation is gentle, the highlighting K shown in Figure 12D is not executed, and the remote driving screen G2 shown in Figure 12A is displayed.

[0145] As shown in Figure 12E, when the work vehicle 1 turns to the right, the control device 31 shifts the range displayed as the captured image on the remote control screen G2 to the left by a distance D corresponding to the steering angle of the right turn. For example, if the steering angle of the work vehicle 1 when turning to the right is greater than or equal to a specified value, the range displayed as the captured image on the remote control screen G2 is shifted to the left by a distance D corresponding to the steering angle, and if the steering angle of the work vehicle 1 when turning to the right is less than a specified value, the range displayed as the captured image on the remote control screen G2 is not shifted. In this way, if the right steering operation is sudden, the highlighting K shown in Figure 12E is executed, and if the right steering operation is gradual, the highlighting K shown in Figure 12E is not executed, and the remote control screen G2 shown in Figure 12A is displayed.

[0146] Furthermore, instead of changing the range of the captured image captured by the camera 25c described above, the range captured by the camera 25c (i.e., the orientation of the camera 25c) may be changed. That is, if a sudden change in the speed of the work vehicle 1 (for example, a change in acceleration exceeding a specified value) is detected by the state detection device 26 on the work vehicle 1, the on-board control device 21 of the work vehicle 1 automatically controls and changes the mounting angle of the camera 25c to the work vehicle 1. For example, if the state detection device 26 detects a sudden acceleration, the on-board control device 21 may control the mounting angle of the camera 25c so that it points upward by an angle corresponding to the change in acceleration, so that the camera 25c takes pictures of the upper side. Furthermore, the same control may be applied to sudden deceleration and sudden turns to the left or right. That is, the on-board control device 21 may control the mounting angle of the camera 25c so that it points downward, to the left, or to the right by an angle corresponding to the change in acceleration, so that the camera 25c takes pictures of the downward, to the left, or to the right side.

[0147] The control device 31 can determine whether or not to perform the eighth display mode highlighting K shown in Figures 12B to 12E, according to the selection operation by the remote operator on the selection screen G1 shown in Figure 9D. Figure 9D is a diagram showing an example of the selection screen G1 of the first modified display device 34.

[0148] Specifically, when the remote operator gives a predetermined additional instruction, such as an instruction to set an additional effect (tactile display effect), the control device 31 displays the selection screen G1 on the display device 34, as shown in Figure 9D. On the selection screen G1 shown in Figure 9D, the three effects that make up the tactile display effect can be individually set to on or off. The three effects are an upward effect of the captured image during rapid acceleration, a downward effect of the captured image during rapid deceleration, and a left-right swing effect of the captured image during sudden steering. In Figure 9D, all three effects are set to on.

[0149] If the first condition shown in Figure 11 is met (S42: YES), the control device 31 displays the highlighting display K (tactile display effect) shown in Figure 12B if the work vehicle 1 accelerates rapidly (acceleration above a specified value), displays the highlighting display K (tactile display effect) shown in Figure 12C if the work vehicle 1 decelerates rapidly (deceleration above a specified value), and displays the highlighting display K (tactile display effect) shown in Figures 12D and 12E if there is a sudden steering maneuver (steering angle above a specified value). The control device 31 may also execute the highlighting display K (tactile display effect) shown in Figures 12B to 12E in addition to Figures 5B, 5C, 6, 7A, 7B, 8A, 8B, and 8C.

[0150] According to the first modified remote device 30, when the work vehicle 1 is accelerating, the control device 31 moves the display position of the captured image on the remote operation screen G2 upward by a distance D corresponding to the change in acceleration, and when the work vehicle 1 is decelerating, it moves the display position of the captured image on the remote operation screen G2 downward. The image is moved by a distance D corresponding to the change in speed. With this configuration, when the work vehicle 1 is accelerating, the display position of the captured image on the remote control screen G2 is moved upward by a distance D corresponding to the change in acceleration, thereby providing the remote operator with a sense of acceleration. Also, when the work vehicle 1 is decelerating, the display position of the captured image on the remote control screen G2 is moved downward by a distance D corresponding to the change in acceleration, thereby providing the remote operator with a sense of deceleration.

[0151] Furthermore, when the work vehicle 1 turns left, the control device 31 moves the display position of the captured image on the remote control screen G2 to the left by a distance D corresponding to the steering angle of the left turn, and when the work vehicle 1 turns right, it moves the display position of the captured image on the remote control screen G2 to the right by a distance D corresponding to the steering angle of the right turn. With this configuration, when the work vehicle 1 turns left, the display position of the captured image on the remote control screen G2 is moved to the left by a distance D corresponding to the steering angle of the left turn, thus providing the remote operator with the impression of a sharp left turn. Also, when the work vehicle 1 turns right, the display position of the captured image on the remote control screen G2 is moved to the right by a distance D corresponding to the steering angle of the right turn, thus providing the remote operator with the impression of a sharp right turn.

[0152] <Second variation> In the remote device 30 and remote control system 100 of the above embodiment, the control device 31 determines whether the first condition is met or not based on the captured image, but is not limited to this. In the remote device 30 and remote control system 100 of the second modified example, the determination of whether the first condition is met or not can be made based on map information.

[0153] For example, the control device 31 of the second modified example uses the location information of the work vehicle 1 and map information to determine that the first condition is met if the current location indicated by the location information of the work vehicle 1 is within a predetermined area on the map indicated by the map information (for example, field H1), and determines that the first condition is not met if it is not within the predetermined area (for example, field H1).

[0154] Figure 13 is a flowchart showing the screen display update process of the second modified example. The control device 31 performs map determination processing (S51). Specifically, the storage device 32 stores map information including, for example, field H1. In the map determination processing (S51), the control device 31 uses the location information of the work vehicle 1 and the map information stored in the storage device 32 to determine whether the current location indicated by the location information of the work vehicle 1 is located in a predetermined area of ​​the map indicated by the map information (for example, field H1). If the current location of the work vehicle 1 is located in field H1, the control device 31 determines that the first condition is met (S42: YES), and if the current location of the work vehicle 1 is not located in field H1, it determines that the first condition is not met (S42: NO). S43 to S45 are the same as in Figure 11, so their explanation is omitted here.

[0155] According to the second modified configuration, the control device 31 uses the location information of the work vehicle 1 and the map information to determine if the first condition is met if the current location indicated by the location information of the work vehicle 1 is within a predetermined area on the map indicated by the map information, and if it is not within the predetermined area, it determines that the first condition is not met. With this configuration, if the location of the remotely operated work vehicle 1 is within a predetermined area on the map (for example, a field, pasture, farm road, etc.), it is determined that the first condition is met, so there is no need to perform image analysis on the captured image, and the meeting or non-measuring of the first condition can be easily determined.

[0156] In the above embodiment and the first and second modifications, the highlighting K is displayed on the remote operation screen G2, but the highlighting K may also be displayed on a peripheral device of the control device 35 of the remote device 30 shown in Figure 1 (for example, the steering wheel 35a).

[0157] Furthermore, in addition to the highlighting K in the above embodiment and the first and second modified examples, the remote control of the remote cockpit The wind is directed towards the operator, and the wind strength may be changed according to the speed of the work vehicle 1. For example, the wind strength increases as the speed of the work vehicle 1 increases.

[0158] In addition to the highlighting K in the above embodiment and the first and second modifications, the engine sound of the work vehicle 1 may be output to the remote operator in the remote control seat, and the volume and type of the engine sound may be changed according to the travel speed of the work vehicle 1. For example, the engine sound may get louder or change as the travel speed of the work vehicle 1 increases. For example, the remote device 30 may store the engine sound in the storage device 32 in advance and output it from the speaker 36a so that the engine sound gets louder or changes as the travel speed of the work vehicle 1 increases. Alternatively, the remote device 30 may detect the engine sound with a sound collection device installed on the actual work vehicle 1 and transmit it to the remote device 30 as sound information, and the engine sound reproduced by the sound information received by the remote device 30 with an audio playback device may be output from the speaker 36a.

[0159] Although the present invention has been described above, the embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the present invention is indicated by the claims rather than by the foregoing description, and all modifications within the meaning and scope equivalent to the claims are intended to be included. [Explanation of Symbols]

[0160] 1. Work vehicles 2. Working equipment 21. In-vehicle communication device 24 Position detection device 25c Camera (imaging device) 25c1 Internal camera (imaging device) 25c2 Rear camera (imaging device) 30 Remote devices 31 Control device 32 Storage device 33 Communication equipment 34 Display device 35 Control devices 70 Display terminals 100 Remote Control Systems K Highlight

Claims

1. A control device for remotely operating work vehicles, A communication device that receives driving information indicating the speed or acceleration of the aforementioned work vehicle, Display device and The control device, when the work vehicle is remotely controlled using the aforementioned control device, includes a control device that causes the display device to display a highlighting that changes according to the driving information, The communication device receives sequentially captured images of the direction of travel of the work vehicle when the work vehicle is being remotely operated. The display device sequentially displays the captured images on the remote operation screen. The control device displays the highlighting on the remote operation screen, The control device displays the highlighting on the remote operation screen when the first condition is met, and does not display the highlighting on the remote operation screen when the first condition is not met. The control device is a remote device that determines that the first condition is met when the amount of change in a plurality of captured images is less than a threshold, and determines that the first condition is not met when the amount of change in a plurality of captured images is equal to or greater than the threshold.

2. A control device for remotely operating work vehicles, A communication device that receives driving information indicating the speed or acceleration of the aforementioned work vehicle, Display device and The control device, when the work vehicle is remotely controlled using the aforementioned control device, includes a control device that causes the display device to display a highlighting that changes according to the driving information, The communication device receives sequentially captured images of the direction of travel of the work vehicle when the work vehicle is being remotely operated. The display device sequentially displays the captured images on the remote operation screen. The control device displays the highlighting on the remote operation screen, The control device displays the highlighting on the remote operation screen when the first condition is met, and does not display the highlighting on the remote operation screen when the first condition is not met. The control device is a remote device that determines that the first condition is met if the captured image does not include road markings, and determines that the first condition is not met if the captured image includes road markings.

3. A control device for remotely operating work vehicles, A communication device that receives driving information indicating the speed or acceleration of the aforementioned work vehicle, Display device and The control device, when the work vehicle is remotely controlled using the aforementioned control device, includes a control device that causes the display device to display a highlighting that changes according to the driving information, The communication device receives sequentially captured images of the direction of travel of the work vehicle when the work vehicle is being remotely operated. The display device sequentially displays the captured images on the remote operation screen. The control device displays the highlighting on the remote operation screen, The control device displays the highlighting on the remote operation screen when the first condition is met, and does not display the highlighting on the remote operation screen when the first condition is not met. The control device is a remote device that uses the location information of the work vehicle and map information to determine that the first condition is met if the current location indicated by the location information of the work vehicle is located within a predetermined area of ​​the map indicated by the map information, and determines that the first condition is not met if the vehicle is not located within the predetermined area.

4. The control device causes the highlighting to be displayed on the remote operation screen or the peripheral edge of the captured image. The remote control device according to any one of claims 1 to 3, wherein the control device changes the area of ​​the peripheral portion according to the speed or acceleration of the work vehicle.

5. A control device for remotely operating work vehicles, A communication device that receives driving information indicating the speed or acceleration of the aforementioned work vehicle, Display device and The control device, when the work vehicle is remotely controlled using the aforementioned control device, includes a control device that causes the display device to display a highlighting that changes according to the driving information, The communication device receives sequentially captured images of the direction of travel of the work vehicle when the work vehicle is being remotely operated. The display device sequentially displays the captured images on the remote operation screen. The control device displays the highlighting on the remote operation screen, The control device superimposes the highlighting on the captured image on the remote operation screen. The control device is a remote device that changes the movement speed of the display body according to the speed or acceleration of the work vehicle.

6. A control device for remotely operating work vehicles, A communication device that receives driving information indicating the speed or acceleration of the aforementioned work vehicle, Display device and The control device, when the work vehicle is remotely controlled using the aforementioned control device, includes a control device that causes the display device to display a highlighting that changes according to the driving information, The communication device receives sequentially captured images of the direction of travel of the work vehicle when the work vehicle is being remotely operated. The display device sequentially displays the captured images on the remote operation screen. The control device displays the highlighting on the remote operation screen, The control device superimposes the highlighting on the captured image on the remote operation screen. The control device is a remote device that changes the appearance of the display body according to the speed or acceleration of the work vehicle.

7. The remote control device according to claim 5 or 6, wherein the display body is a display body that extends in the direction of travel of the work vehicle.

8. The remote device according to claim 5 or 6, wherein the display body is a plurality of virtual display objects arranged in the direction of travel of the work vehicle.

9. The remote control device according to any one of claims 1 to 3, wherein the control device changes the color of a specific location other than the captured image on the remote control screen according to the speed or acceleration of the work vehicle as the highlighting.

10. The remote control device according to any one of claims 1 to 3, wherein the control device changes the color of the frame of the remote control screen according to the speed or acceleration of the work vehicle as the highlighting indicator.

11. A control device for remotely operating work vehicles, A communication device that receives driving information indicating the speed or acceleration of the aforementioned work vehicle, Display device and The control device, when the work vehicle is remotely controlled using the aforementioned control device, includes a control device that causes the display device to display a highlighting that changes according to the driving information, The communication device receives sequentially captured images of the direction of travel of the work vehicle when the work vehicle is being remotely operated. The display device sequentially displays the captured images on the remote operation screen. The control device displays the highlighting on the remote operation screen, The aforementioned remote driving screen includes a forward-facing image and a rear-facing image. The control device is a remote device that displays the highlighting on the forward-facing image when moving forward, and on the rear-facing image when moving backward.

12. A control device for remotely operating work vehicles, A communication device that receives driving information indicating the speed or acceleration of the aforementioned work vehicle, Display device and The control device, when the work vehicle is remotely controlled using the aforementioned control device, includes a control device that causes the display device to display a highlighting that changes according to the driving information, The communication device receives sequentially captured images of the direction of travel of the work vehicle when the work vehicle is being remotely operated. The display device sequentially displays the captured images on the remote operation screen. The control device displays the highlighting on the remote operation screen, The control device is a remote device that, when the work vehicle is moving in reverse, displays an image of the work vehicle in reverse on the remote operation screen, and changes the appearance of the guide lines displayed on the remote operation screen as a highlighting feature according to the speed or acceleration of the work vehicle.

13. A control device for remotely operating work vehicles, A communication device that receives driving information indicating the speed or acceleration of the aforementioned work vehicle, Display device and The control device, when the work vehicle is remotely controlled using the aforementioned control device, includes a control device that causes the display device to display a highlighting that changes according to the driving information, The communication device receives sequentially captured images of the direction of travel of the work vehicle when the work vehicle is being remotely operated. The display device sequentially displays the captured images on the remote operation screen. The control device displays the highlighting on the remote operation screen, The control device is a remote device that, when the work vehicle is accelerating, changes the range displayed as the captured image on the remote operation screen upwards in accordance with the change in acceleration, and when the work vehicle is decelerating, changes the range displayed as the captured image on the remote operation screen downwards in accordance with the change in acceleration.

14. The remote control device according to claim 13, wherein when the work vehicle turns left, the control device changes the range displayed as the captured image on the remote control screen to the right in accordance with the steering angle of the left turn, and when the work vehicle turns right, the control device changes the range displayed as the captured image on the remote control screen to the left in accordance with the steering angle of the right turn.

15. A work vehicle and a remote control device according to any one of claims 1 to 3, 5, 6, 11 to 13, The work vehicle comprises a detection device for detecting the speed of the work vehicle, an imaging device for capturing images of the direction of travel of the work vehicle, and an on-board communication device for transmitting correspondence data that associates driving information indicating the speed or acceleration detected by the detection device with the captured images captured by the imaging device. The communication device of the remote device is a remote control system that receives the corresponding data transmitted from the in-vehicle communication device.

Images