Agricultural machinery, sensing system, sensing method, remote control system, and control method

Abstract

This agricultural machine comprises a vehicle body, a sensing device that senses the environment around the vehicle body and outputs sensing data, and a communication device that transmits the sensing data output from the sensing device to a remote device. The sensing device changes the amount of sensing data output to the communication device in accordance with the speed of communication from the communication device to the remote device.

Description

[Technical Field]

[0001] The present disclosure relates to an agricultural machine, a sensing system, a sensing method, a remote control system, and a control method. [Background technology]

[0002] Research and development is underway to automate agricultural machinery used in fields. For example, work vehicles such as tractors, combine harvesters, and rice transplanters that use positioning systems such as the Global Navigation Satellite System (GNSS) to navigate autonomously within fields have been put to practical use. Research and development is also underway on work vehicles that can navigate autonomously not only within fields but also outside of them. Technology for remotely operating agricultural machinery is also being developed.

[0003] Patent Documents 1 and 2 disclose examples of systems that automatically drive an unmanned work vehicle between two fields separated by a road. Patent Document 3 discloses an example of a device that remotely controls an autonomously driving work vehicle. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-073602 [Patent Document 2] Patent Publication No. 2021-029218 [Patent Document 3] International Publication No. 2016 / 017367 Summary of the Invention [Problem to be solved by the invention]

[0005] The present disclosure provides at least one of a technique for appropriately transferring data between a communication device and a remote device in accordance with a communication speed, and a technique for assisting in remote operation of an agricultural machine. [Means for solving the problem]

[0006] An agricultural machine according to one aspect of the present disclosure comprises a vehicle body, a sensing device that senses the environment around the vehicle body and outputs sensing data, and a communication device that transmits the sensing data output from the sensing device to a remote device, and the sensing device changes the amount of sensing data output to the communication device depending on the communication speed from the communication device to the remote device.

[0007] A sensing system according to one aspect of the present disclosure includes a sensing device that senses the environment surrounding an agricultural machine and outputs sensing data, and a communication device that transmits the sensing data output from the sensing device to a remote device, and the sensing device changes the amount of sensing data output to the communication device depending on the communication speed from the communication device to the remote device.

[0008] A sensing method according to one aspect of the present disclosure includes the steps of acquiring sensing data from a sensing device that senses the environment surrounding an agricultural machine and outputs the sensing data, transmitting the sensing data from a communication device to a remote device, and causing the sensing device to change the amount of data of the sensing data depending on the communication speed from the communication device to the remote device.

[0009] A remote control system for an agricultural machine according to one aspect of the present disclosure includes a sensing device that senses the environment around the agricultural machine and outputs sensing data, a communication device that transmits transmission data based on the sensing data output from the sensing device to a remote device that transmits control commands to the agricultural machine, and a control device that causes at least one of the agricultural machine and the sensing device to perform compensatory operations to reduce the impact on the remote control caused by deterioration of a display image showing the situation around the agricultural machine based on the transmission data received by the remote device when the communication speed from the communication device to the remote device decreases.

[0010] A general or specific aspect of the present disclosure may be realized by an apparatus, a system, a method, an integrated circuit, a computer program, or a computer-readable non-transitory storage medium, or any combination thereof. The computer-readable storage medium may include a volatile storage medium or a non-volatile storage medium. An apparatus may be composed of multiple devices. When an apparatus is composed of two or more devices, the two or more devices may be located in a single device or may be located separately in two or more separate devices. [Effects of the Invention]

[0011] According to the embodiment of the present disclosure, data transfer can be appropriately performed according to the communication speed between the communication device and the remote device, or the impact on remote control when the communication speed between the communication device and the remote device in the agricultural machine decreases can be reduced. [Brief explanation of the drawings]

[0012] [Figure 1A] FIG. 1 is a diagram illustrating an example of the configuration of an agricultural management system. [Figure 1B] FIG. 10 is a diagram illustrating another example of the configuration of an agricultural management system. [Figure 2] 1 is a side view schematically showing an example of a work vehicle and an implement coupled to the work vehicle. FIG. [Figure 3] FIG. 2 is a block diagram showing an example of the configuration of a work vehicle and an implement. [Figure 4] FIG. 1 is a conceptual diagram showing an example of a work vehicle that performs positioning using RTK-GNSS. [Figure 5] 3A and 3B are diagrams illustrating an example of an operation terminal and an operation switch group provided inside a cabin. [Figure 6] FIG. 2 is a block diagram illustrating an example of the hardware configuration of a management device and a remote device. [Figure 7] FIG. 1 is a diagram schematically illustrating an example of a work vehicle that automatically travels along a target route in a farm field. [Figure 8]10 is a flowchart illustrating an example of the operation of steering control during automatic driving. [Figure 9A] 1 is a diagram showing an example of a work vehicle traveling along a target route P. FIG. [Figure 9B] FIG. 10 is a diagram showing an example of a work vehicle at a position shifted to the right from the target route P. [Figure 9C] FIG. 10 is a diagram showing an example of a work vehicle at a position shifted to the left from a target route P. [Figure 9D] 10 is a diagram showing an example of a work vehicle facing in a direction inclined with respect to a target route P. FIG. [Figure 10] FIG. 1 is a diagram schematically illustrating an example of a situation in which a plurality of work vehicles are automatically traveling on roads inside and outside a farm field. [Figure 11A] FIG. 10 is a diagram showing an example of a display screen in an automatic driving mode. [Figure 11B] FIG. 10 is a diagram showing an example of a display screen in a remote operation mode. [Figure 11C] FIG. 10 is a diagram showing another example of the display screen in the remote control mode. [Figure 12] 10 is a graph illustrating an example of a change in the communication speed of a communication device and a change in the amount of sensing data output from a sensing device in response to the change in the communication speed. [Figure 13] 10A and 10B are diagrams for explaining the state transition of the operation of the sensing device according to the communication speed of the communication device. [Figure 14] FIG. 10 is a diagram for explaining reducing an image size while maintaining resolution. [Figure 15A] 10 is a diagram schematically illustrating a sensing area set in a first angle range θ1 of sensing in the horizontal direction. FIG. [Figure 15B] 10 is a diagram schematically illustrating a sensing area set in a second angle range θ2 of sensing in the horizontal direction. FIG. [Figure 16A] 10 is a diagram schematically illustrating a sensing region set in a third angular range θ3 of sensing in the vertical direction. FIG. [Figure 16B]FIG. 10 is a diagram schematically illustrating a sensing region set in a fourth angular range θ4 of sensing in the vertical direction. [Figure 17A] 1 is a diagram schematically illustrating a sensing region in which the maximum detection distance from a LiDAR sensor is a first distance L1. [Figure 17B] 10 is a diagram schematically illustrating a sensing region in which the maximum detection distance from the LiDAR sensor is a second distance L2. FIG. [Figure 18] 10 is a flowchart illustrating an example of a control operation according to a communication speed. [Figure 19A] FIG. 4 is a diagram showing an example of the relationship between a speed command value and an actual speed of a work vehicle. [Figure 19B] FIG. 10 is a diagram showing another example of the relationship between the speed command value and the actual speed of the work vehicle. [Figure 20] 10 is a flowchart showing another example of a control operation according to a communication speed. [Figure 21A] FIG. 4 is a diagram showing an example of the relationship between a steering angle command value and an actual steering angle of a work vehicle. [Figure 21B] FIG. 10 is a diagram showing another example of the relationship between the steering angle command value and the actual steering angle of the work vehicle. [Figure 22] 10 is a diagram illustrating an example of a ratio (target value / command value) of a target value to a value (command value) of a maneuver command in control during a compensation operation. FIG. [Figure 23] 10 is a flowchart showing yet another example of a control operation according to a communication speed. [Figure 24] 10A and 10B are diagrams illustrating an example of a change over time in the communication speed of a communication device and a change over time in the amount of sensing data. [Figure 25A] FIG. 10 is a diagram schematically illustrating sensing by a first camera that captures the side in the traveling direction of the work vehicle and a second camera that captures the side opposite the traveling direction. [Figure 25B] FIG. 10 is a diagram schematically illustrating sensing by a first LiDAR sensor that senses the side in the traveling direction of the work vehicle and a second LiDAR sensor that senses the side opposite the traveling direction. DETAILED DESCRIPTION OF THE INVENTION

[0013] (Definition of terms) In this disclosure, "agricultural machinery" refers to machinery used for agricultural purposes. Examples of agricultural machinery include tractors, harvesters, rice transplanters, riding cultivators, vegetable transplanters, mowers, seed sowing machines, fertilizer applicators, agricultural drones (i.e., unmanned aerial vehicles: UAVs), and agricultural mobile robots. Not only can a work vehicle such as a tractor function alone as "agricultural machinery," but the entire work vehicle and an implement attached to or towed by the work vehicle can also function as one "agricultural machinery." Agricultural machinery performs agricultural tasks on the ground in a field, such as plowing, sowing seeds, pest control, fertilizing, planting crops, or harvesting. These agricultural tasks are sometimes referred to as "ground work" or simply "work." Traveling while performing agricultural tasks by a vehicle-type agricultural machine is sometimes referred to as "work driving."

[0014] "Autonomous driving" refers to controlling the movement of an agricultural machine through the action of a control device, without manual operation by a driver. Agricultural machines that perform autonomous driving are sometimes called "autonomous agricultural machines" or "robotic agricultural machines." During autonomous driving, not only the movement of the agricultural machine but also the agricultural work operations (e.g., the operation of the implement) may be automatically controlled. When the agricultural machine is a vehicle-type machine, the movement of the agricultural machine through autonomous driving is referred to as "autonomous driving." The control device may control at least one of the following operations required for the movement of the agricultural machine: steering, adjusting the movement speed, and starting and stopping the movement. When controlling a work vehicle equipped with implements, the control device may control operations such as raising and lowering the implement, starting and stopping the operation of the implement, etc. Autonomous driving movement may include not only movement of the agricultural machine toward a destination along a predetermined route, but also movement of the agricultural machine following a tracking target. An autonomously driving agricultural machine may move partially based on user instructions. Furthermore, an autonomously driving agricultural machine may operate in a manual driving mode, in addition to an autonomous driving mode, in which the agricultural machine moves through manual operation by the driver. Steering an agricultural machine by the action of a control device, without manual operation, is called "automatic steering." Part or all of the control device may be external to the agricultural machine. Control signals, commands, data, and the like may be communicated between the agricultural machine and a control device external to the agricultural machine. An agricultural machine that performs automatic driving may move autonomously while sensing the surrounding environment, without a human being being involved in controlling the movement of the agricultural machine. An agricultural machine capable of autonomous movement can travel unmanned within a field or outside a field (e.g., on a road). During autonomous movement, the machine may detect obstacles and take action to avoid them.

[0015] "Teleoperation" or "telecontrol" means operating an agricultural machine using a remote control device. Remote operation can be performed by an operator (e.g., a system administrator or a user of the agricultural machine) who is located away from the agricultural machine. "Teleoperated driving" means that the agricultural machine drives in response to signals transmitted from a remote control device. The remote control device can include a device with a signal transmission function, such as a personal computer (PC), laptop computer, tablet computer, smartphone, or remote controller. By operating the remote control device, the operator can give commands to the agricultural machine, such as starting, stopping, accelerating, decelerating, or changing the direction of travel. The mode in which the control device controls the driving of the agricultural machine in response to these commands is called the "remote control mode."

[0016] A "remote device" is a device equipped with a communication function and located away from the agricultural machine. The remote device may be, for example, a remote control device used by an operator to remotely control the agricultural machine. The remote device may include a display device or may be connected to a display device. The display device may display an image (or video) that visualizes the situation around the agricultural machine based on sensor data (also referred to as "sensing data") output from a sensing device such as a camera or LiDAR sensor equipped on the agricultural machine. The operator can grasp the situation around the agricultural machine while looking at the displayed image and operate the remote control device as necessary to remotely control the agricultural machine.

[0017] "Communication speed" refers to the amount of data transmitted per unit time in data communication. Communication speed is also referred to as "communication volume per unit time." The communication speed from a communication device to a remote device refers to the amount of data transmitted from the communication device to the remote device per unit time. Communication speed can be expressed in units such as bps (bits per second), Mbps (megabits per second), Gbps (gigabits per second), etc.

[0018] A "work plan" is data that schedules one or more agricultural tasks to be performed by an agricultural machine. The work plan may include, for example, information indicating the order of agricultural tasks to be performed by the agricultural machine and the field on which each task will be performed. The work plan may also include information on the scheduled date and time for each task. A work plan that includes information on the scheduled date and time for each task is particularly referred to as a "work schedule" or simply a "schedule." The work schedule may include information on the scheduled start and / or end times for each task performed on each work day. The work plan or work schedule may include information for each task, such as the content of the task, the implements to be used, and / or the type and amount of agricultural materials to be used. Here, "agricultural materials" refers to materials used in agricultural tasks performed by an agricultural machine. Agricultural materials may also be simply referred to as "materials." Agricultural materials may include materials consumed in agricultural tasks, such as pesticides, fertilizers, seeds, or seedlings. The work plan may be created by a processing device that communicates with the agricultural machine to manage the agricultural task, or a processing device installed on the agricultural machine. The processing device can create a work plan based on information input by a user (such as a farm manager or farm worker) by operating a terminal device, for example. In this specification, a processing device that communicates with agricultural machines and manages agricultural work is referred to as a "management device." The management device may manage the agricultural work of multiple agricultural machines. In this case, the management device may create a work plan that includes information about each agricultural work to be performed by each of the multiple agricultural machines. The work plan can be downloaded by each agricultural machine and stored in a storage device. Each agricultural machine can automatically head to a field and perform the scheduled agricultural work in accordance with the work plan.

[0019] An "environmental map" is data that represents the positions or areas of objects in the environment in which the agricultural machine moves using a specified coordinate system. An environmental map may be simply referred to as a "map" or "map data." The coordinate system that defines the environmental map may be, for example, a world coordinate system such as a geographic coordinate system fixed relative to the Earth. An environmental map may also include information other than the positions of objects in the environment (e.g., attribute information and other information). Environmental maps include maps in various formats, such as point cloud maps or grid maps. Data for local or partial maps that are generated or processed in the process of constructing an environmental map are also referred to as a "map" or "map data."

[0020] "Farm road" means a road that is primarily used for agricultural purposes. Farm roads are not limited to roads paved with asphalt, but also include unpaved roads covered with dirt or gravel. Farm roads include roads (including private roads) that are exclusively passable by vehicle-type agricultural machinery (for example, work vehicles such as tractors) and roads that are also passable by general vehicles (passenger cars, trucks, buses, etc.). Work vehicles may automatically travel on general roads in addition to farm roads. General roads are roads that have been developed for the traffic of general vehicles.

[0021] (Embodiment) Hereinafter, embodiments of the present disclosure will be described. However, more detailed descriptions than necessary may be omitted. For example, detailed descriptions of well-known matters and redundant descriptions of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the inventors provide the accompanying drawings and the following description to enable those skilled in the art to fully understand the present disclosure, and do not intend for them to limit the subject matter described in the claims. In the following description, components having the same or similar functions are designated by the same reference numerals.

[0022] The following embodiments are examples, and the technology of the present disclosure is not limited to the following embodiments. For example, the numerical values, shapes, materials, steps, step order, display screen layout, etc. shown in the following embodiments are merely examples, and various modifications are possible as long as no technical contradiction occurs. Furthermore, one aspect can be combined with another aspect as long as no technical contradiction occurs.

[0023] The following mainly describes an embodiment in which the technology of the present disclosure is applied to a work vehicle such as a tractor, which is an example of agricultural machinery. The technology of the present disclosure is not limited to tractors, but can also be applied to other types of agricultural machinery capable of remotely controlled driving (e.g., rice transplanters, combine harvesters, harvesters, riding cultivators, vegetable transplanters, mowers, seed sowing machines, fertilizer applicators, agricultural drones, agricultural mobile robots, etc.). As an example, the following describes an embodiment in which a driving control system for realizing an automatic driving function and a remote control function is mounted on a work vehicle. At least some of the functions of the driving control system may be implemented in another device that communicates with the work vehicle (e.g., a remote device that remotely controls the work vehicle, a server, etc.).

[0024] FIG. 1A is a diagram illustrating an overview of an agricultural management system according to an exemplary embodiment of the present disclosure. The agricultural management system illustrated in FIG. 1A includes a work vehicle 100, a remote device 400, and a management device 600. The remote device 400 is a computer used by a user to remotely monitor the work vehicle 100. The management device 600 is a computer managed by a business operator who operates the agricultural management system. The work vehicle 100, the remote device 400, and the management device 600 can communicate with each other via a network 80. While FIG. 1A illustrates a single work vehicle 100, the agricultural management system may include multiple work vehicles or other agricultural machinery. The agricultural management system in this embodiment includes a remote control system for the work vehicle 100. The remote control system includes a sensing device, a communication device, and a control device in the work vehicle 100, as well as the remote device 400. The portion of the entire remote control system in this embodiment that includes the sensing device and communication device in the work vehicle 100 may be referred to as the "sensing system." In other words, the sensing system is part of the remote control system.

[0025] The work vehicle 100 in this embodiment is a tractor. The work vehicle 100 can be fitted with an implement at either the rear or the front, or both. The work vehicle 100 can travel within a field while performing agricultural work according to the type of implement. The work vehicle 100 may also travel within or outside a field without an implement attached.

[0026] The work vehicle 100 in this embodiment is equipped with an automatic driving function. In other words, the work vehicle 100 can travel by the operation of a control device, without manual operation. The control device in this embodiment is provided inside the work vehicle 100, and can control both the speed and steering of the work vehicle 100. The work vehicle 100 can travel automatically not only within a field, but also outside the field (for example, on a road). The mode in which the control device causes the work vehicle 100 to travel automatically is called the "automatic travel mode."

[0027] The work vehicle 100 is also equipped with a remotely controlled traveling function. The control device can change the traveling speed and traveling direction by controlling the traveling device of the work vehicle 100 in response to remote operation by the user using the remote device 400. The work vehicle 100 can perform remotely controlled traveling not only within a farm field but also outside a farm field. The mode in which the control device remotely controls the work vehicle 100 to travel is called the "remotely controlled mode."

[0028] The work vehicle 100 is equipped with devices used for positioning or self-location estimation, such as a GNSS receiver and a LiDAR sensor. In the autonomous driving mode, the control device of the work vehicle 100 causes the work vehicle 100 to travel automatically based on the position of the work vehicle 100 and information on the target route generated by the management device 600. In addition to controlling the travel of the work vehicle 100, the control device also controls the operation of the implement. This allows the work vehicle 100 to perform agricultural work using the implement while traveling autonomously within a field. Furthermore, the work vehicle 100 can automatically travel along roads outside the field (e.g., farm roads or public roads) along the target route. When traveling autonomously along roads outside the field, the work vehicle 100 travels while generating a local route along the target route that can avoid obstacles based on data output from sensing devices such as a camera or LiDAR sensor. Within the field, the work vehicle 100 may travel while generating a local route as described above, or may travel along a target route without generating a local route and stop if an obstacle is detected.

[0029] The management device 600 is a computer that manages agricultural work performed by the work vehicle 100. The management device 600 may be, for example, a server computer that centrally manages information about farm fields on the cloud and supports agriculture by utilizing data on the cloud. The management device 600 may, for example, create a work plan for the work vehicle 100 and generate a target route for the work vehicle 100 in accordance with the work plan. Alternatively, the management device 600 may generate a target route for the work vehicle 100 in response to an operation performed by a user using the remote device 400.

[0030] Remote device 400 is a computer used by a user located away from work vehicle 100. Remote device 400 shown in Fig. 1A is a laptop computer, but is not limited to this. Remote device 400 may be a stationary computer such as a desktop personal computer (PC), or a mobile terminal such as a smartphone or tablet computer.

[0031] Remote device 400 can be used to remotely monitor and remotely operate work vehicle 100. For example, remote device 400 can display on a display video captured by one or more cameras equipped on work vehicle 100. A user can view the video to check the situation around work vehicle 100 and send instructions to work vehicle 100 to stop, start, accelerate, decelerate, change direction, etc.

[0032] FIG. 1B is a diagram showing another example of an agricultural management system. The agricultural management system shown in FIG. 1B includes multiple work vehicles 100. Although FIG. 1B illustrates three work vehicles 100, the number of work vehicles 100 is arbitrary. Agricultural machinery other than the work vehicles 100 (e.g., agricultural drones) may also be included in the system. The remote device 400 in the example of FIG. 1B is not a home terminal device, but a computer installed in a remote monitoring center for the agricultural machinery. The remote device 400 may be connected to a remote control device 500 and one or more displays 430 used by an operator at the remote monitoring center. Although FIG. 1B illustrates five displays 430, the number of displays 430 is arbitrary. The remote control device 500 may include various devices for remotely controlling the work vehicle 100 (e.g., a steering wheel, an accelerator pedal, left and right brake pedals, a clutch pedal, various switches or levers, etc.). The remotely controlled device 500 shown in FIG. 1B is a device that imitates an operating device used to manually drive the work vehicle 100, but the remotely controlled device 500 is not limited to such a device. For example, remote control may be performed by a controller such as a joystick. Each display 430 can display, for example, an environmental map of the area including the field where the work vehicle 100 will perform agricultural work, and images (e.g., video images) captured by one or more cameras mounted on the work vehicle 100. The operator can grasp the situation around the work vehicle 100 while viewing the images displayed on the display 430. The operator can switch between automatic driving mode and remote control mode and remotely control each agricultural machine depending on the situation around each work vehicle 100.

[0033] The configuration and operation of the system in this embodiment will be described in more detail below.

[0034] [1. Configuration] FIG. 2 is a side view schematically illustrating an example of a work vehicle 100 and an implement 300 coupled to the work vehicle 100. The work vehicle 100 in this embodiment can operate in both a manual driving mode and an automatic driving mode. In the automatic driving mode, the work vehicle 100 can travel unmanned. The work vehicle 100 is capable of automatic driving both inside and outside of a field. In the automatic driving mode, the control device can operate in an automatic driving mode in which the work vehicle 100 travels along a predetermined target route, and a remote control mode in which the work vehicle 100 travels in response to a user's operation using the remote device 400. The user may be, for example, a user of the work vehicle 100 or an operator at a remote monitoring center. Switching between the automatic driving mode and the remote control mode can be performed when the user performs a predetermined operation using the remote device 400. For example, in the automatic driving mode, if the user performs an operation using the remote device 400 to instruct the start of remote operation, the mode transitions to the remote control mode. In addition, in the remote control mode, when the user performs an operation using the remote device 400 to instruct the start of automatic driving, the mode transitions to automatic driving mode.

[0035] As shown in FIG. 2, work vehicle 100 includes a vehicle body 101, a prime mover (engine) 102, and a transmission 103. Vehicle body 101 is provided with a traveling device including wheels 104 with tires, and a cabin 105. The traveling device includes four wheels 104, axles that rotate the four wheels, and braking devices (brakes) that brake each axle. Wheels 104 include a pair of front wheels 104F and a pair of rear wheels 104R. Inside cabin 105 are provided a driver's seat 107, a steering device 106, an operation terminal 200, and a group of switches for operation. When work vehicle 100 travels in a field, one or both of front wheels 104F and rear wheels 104R may be replaced with multiple wheels (crawlers) equipped with tracks rather than with tires.

[0036] The work vehicle 100 can switch between a four-wheel drive (4W) mode in which both the front wheels 104F and the rear wheels 104R are drive wheels, and a two-wheel drive (2W) mode in which either the front wheels 104F or the rear wheels 104R are drive wheels. The work vehicle 100 can also switch between a state in which the left and right brakes are coupled and a state in which the coupling is released. By releasing the coupling between the left and right brakes, the left and right wheels 104 can be braked independently. This allows for turning with a small turning radius.

[0037] The work vehicle 100 is equipped with multiple sensing devices that sense the surroundings of the work vehicle 100. In the example of Fig. 2, the sensing devices include multiple cameras 120, a LiDAR sensor 140, and multiple obstacle sensors 130. The sensing devices may include only some of the cameras 120, the LiDAR sensors 140, and the obstacle sensors 130. The sensing devices sense the environment around the vehicle body and output sensing data.

[0038] The cameras 120 may be installed, for example, on the front, rear, left and right sides of the work vehicle 100. The cameras 120 capture images of the environment around the work vehicle 100 and generate image data. In this specification, the image data generated by the cameras 120 may be simply referred to as "images." The act of capturing images and generating image data may also be expressed as "acquiring images." The images acquired by the cameras 120 may be transmitted to a remote device 400 for remote monitoring. These images may be used to monitor the work vehicle 100 during unmanned operation. The cameras 120 may also be used to generate images for recognizing surrounding features or obstacles, white lines, signs, or markings when the work vehicle 100 travels on roads outside of fields (farm roads or public roads).

[0039] In the example of FIG. 2, the LiDAR sensor 140 is disposed on the lower front side of the vehicle body 101. The LiDAR sensor 140 may be disposed in another location. While the work vehicle 100 is traveling mainly outside the field, the LiDAR sensor 140 repeatedly outputs sensor data indicating the distance and direction to each measurement point of an object in the surrounding environment, or the two-dimensional or three-dimensional coordinate values ​​of each measurement point. The sensor data output from the LiDAR sensor 140 is processed by a control device of the work vehicle 100. The control device can estimate the self-position of the work vehicle 100 by matching the sensor data with an environmental map. The control device can further detect objects such as obstacles present around the work vehicle 100 based on the sensor data, and generate a local route that the work vehicle 100 should actually travel along, along a target route (also called a global route). The control device can also generate or edit an environmental map using an algorithm such as SLAM (Simultaneous Localization and Mapping). Work vehicle 100 may be equipped with multiple LiDAR sensors positioned at different locations and with different orientations.

[0040] The multiple obstacle sensors 130 shown in FIG. 2 are provided at the front and rear of the cabin 105. The obstacle sensors 130 may also be located in other locations. For example, one or more obstacle sensors 130 may be provided at any position on the side, front, and rear of the vehicle body 101. The obstacle sensors 130 may include, for example, a laser scanner or ultrasonic sonar. The obstacle sensors 130 are used to detect surrounding obstacles during autonomous driving and to stop or detour the work vehicle 100. A LiDAR sensor 140 may be used as one of the obstacle sensors 130.

[0041] The work vehicle 100 further includes a GNSS unit 110. The GNSS unit 110 includes a GNSS receiver. The GNSS receiver may include an antenna that receives signals from GNSS satellites and a processor that calculates the position of the work vehicle 100 based on the signals received by the antenna. The GNSS unit 110 receives satellite signals transmitted from multiple GNSS satellites and performs positioning based on the satellite signals. In this embodiment, the GNSS unit 110 is provided on top of the cabin 105, but it may be provided in another location.

[0042] The GNSS unit 110 may include an inertial measurement unit (IMU). Signals from the IMU can be used to complement position data. The IMU can measure the tilt and minute movements of the work vehicle 100. By complementing position data based on satellite signals with data acquired by the IMU, positioning performance can be improved.

[0043] The control device of the work vehicle 100 may use, in addition to the positioning results from the GNSS unit 110, sensing data acquired by sensing devices such as the camera 120 or LiDAR sensor 140 for positioning. If there are features that function as characteristic points in the environment in which the work vehicle 100 is traveling, such as farm roads, forest roads, public roads, or orchards, the position and orientation of the work vehicle 100 can be estimated with high accuracy based on the data acquired by the camera 120 or LiDAR sensor 140 and an environmental map that has been stored in advance in a storage device. By using the data acquired by the camera 120 or LiDAR sensor 140 to correct or complement position data based on satellite signals, the position of the work vehicle 100 can be determined with higher accuracy.

[0044] The prime mover 102 may be, for example, a diesel engine. An electric motor may be used instead of a diesel engine. The transmission 103 can change the propulsive force and travel speed of the work vehicle 100 by changing gears. The transmission 103 can also switch the work vehicle 100 between forward and reverse travel.

[0045] The steering device 106 includes a steering wheel, a steering shaft connected to the steering wheel, and a power steering device that assists steering by the steering wheel. The front wheels 104F are steerable wheels, and the traveling direction of the work vehicle 100 can be changed by changing the turning angle (also referred to as the "steering angle"). The steering angle of the front wheels 104F can be changed by operating the steering wheel. The power steering device includes a hydraulic device or an electric motor that supplies an assisting force to change the steering angle of the front wheels 104F. When automatic steering is performed, the steering angle is automatically adjusted by the force of the hydraulic device or electric motor under control of a control device arranged inside the work vehicle 100.

[0046] A coupling device 108 is provided at the rear of the vehicle body 101. The coupling device 108 includes, for example, a three-point support device (also referred to as a "three-point link" or "three-point hitch"), a PTO (Power Take Off) shaft, a universal joint, and a communication cable. The coupling device 108 allows the implement 300 to be attached to and detached from the work vehicle 100. The coupling device 108 can raise and lower the three-point link using, for example, a hydraulic device, thereby changing the position or attitude of the implement 300. Power can also be sent from the work vehicle 100 to the implement 300 via the universal joint. The work vehicle 100 can pull the implement 300 and cause the implement 300 to perform a predetermined task. The coupling device may be provided at the front of the vehicle body 101. In this case, the implement can be connected to the front of the work vehicle 100.

[0047] 2 is a rotary tiller, the implement 300 is not limited to a rotary tiller. For example, any implement such as a seeder (seed sowing machine), a spreader (fertilizer applicator), a transplanter, a mower (grass cutter), a rake, a baler (grass collector), a harvester (harvesting machine), a sprayer, or a harrow can be connected to the work vehicle 100 and used.

[0048] 2 is capable of being driven by a driver, but may also be capable of being driven only unmanned. In that case, components required only for driven operation, such as the cabin 105, steering device 106, and driver's seat 107, may not be provided in the work vehicle 100. The unmanned work vehicle 100 can travel autonomously or by remote control by a user.

[0049] 3 is a block diagram showing an example configuration of the work vehicle 100 and the implement 300. The work vehicle 100 and the implement 300 can communicate with each other via a communication cable included in the coupling device 108. The work vehicle 100 can communicate with the remote device 400 and the management device 600 via the network 80.

[0050] In the example of FIG. 3 , the work vehicle 100 includes a GNSS unit 110, a sensing device 250 (camera 120, obstacle sensor 130, LiDAR sensor 140), and an operation terminal 200, as well as a group of sensors 150 that detect the operating state of the work vehicle 100, a cruise control system 160, a communication device 190, a group of operation switches 210, a buzzer 220, and a drive unit 240. These components are communicatively connected to each other via a bus. The GNSS unit 110 includes a GNSS receiver 111, an RTK receiver 112, an inertial measurement unit (IMU) 115, and a processing circuit 116. The group of sensors 150 includes a steering wheel sensor 152, a turning angle sensor 154, and an axle sensor 156. The cruise control system 160 includes a memory device 170 and a control device 180. The control device 180 includes multiple electronic control units (ECUs) 181 to 186. Implement 300 includes a drive unit 340, a control unit 380, and a communication unit 390. Note that Fig. 3 shows components that are relatively closely related to the operations of automatic driving and remote control by work vehicle 100, and does not show other components.

[0051] The GNSS receiver 111 in the GNSS unit 110 receives satellite signals transmitted from multiple GNSS satellites and generates GNSS data based on the satellite signals. The GNSS data is generated in a predetermined format, such as the NMEA-0183 format. The GNSS data may include, for example, values ​​indicating the identification number, elevation angle, azimuth angle, and reception strength of each satellite from which a satellite signal is received.

[0052] The GNSS unit 110 shown in FIG. 3 performs positioning of the work vehicle 100 using RTK (Real Time Kinematic)-GNSS. FIG. 4 is a conceptual diagram showing an example of a work vehicle 100 performing positioning using RTK-GNSS. Positioning using RTK-GNSS uses satellite signals transmitted from multiple GNSS satellites 50 as well as correction signals transmitted from a reference station 60. The reference station 60 may be installed near the field where the work vehicle 100 will be traveling (for example, within 10 km of the work vehicle 100). The reference station 60 generates correction signals, for example in RTCM format, based on the satellite signals received from the multiple GNSS satellites 50 and transmits them to the GNSS unit 110. The RTK receiver 112 includes an antenna and a modem and receives the correction signals transmitted from the reference station 60. The processing circuit 116 of the GNSS unit 110 corrects the positioning results obtained by the GNSS receiver 111 based on the correction signals. By using RTK-GNSS, it is possible to perform positioning with an accuracy of, for example, a few centimeters. Position information including latitude, longitude, and altitude information is obtained through highly accurate positioning using RTK-GNSS. The GNSS unit 110 calculates the position of the work vehicle 100, for example, at a frequency of approximately 1 to 10 times per second.

[0053] The positioning method is not limited to RTK-GNSS, and any positioning method (such as interferometric positioning or differential positioning) that can obtain position information with the required accuracy can be used. For example, positioning may be performed using a Virtual Reference Station (VRS) or a Differential Global Positioning System (DGPS). If position information with the required accuracy can be obtained without using a correction signal transmitted from the reference station 60, the position information may be generated without using a correction signal. In this case, the GNSS unit 110 does not need to be equipped with the RTK receiver 112.

[0054] Even when RTK-GNSS is used, in places where correction signals from the reference station 60 cannot be obtained (for example, on a road far from a field), the position of the work vehicle 100 is estimated by other methods without relying on signals from the RTK receiver 112. For example, the position of the work vehicle 100 can be estimated by matching data output from the LiDAR sensor 140 and / or camera 120 with a highly accurate environmental map.

[0055] The GNSS unit 110 in this embodiment further includes an IMU 115. The IMU 115 may include a three-axis acceleration sensor and a three-axis gyroscope. The IMU 115 may also include a direction sensor such as a three-axis geomagnetic sensor. The IMU 115 functions as a motion sensor and can output signals indicating various quantities such as the acceleration, velocity, displacement, and attitude of the work vehicle 100. The processing circuit 116 can estimate the position and orientation of the work vehicle 100 with higher accuracy based on the signals output from the IMU 115 in addition to the satellite signals and correction signals. The signals output from the IMU 115 can be used to correct or complement the position calculated based on the satellite signals and correction signals. The IMU 115 outputs signals at a higher frequency than the GNSS receiver 111. Using these high-frequency signals, the processing circuit 116 can measure the position and orientation of the work vehicle 100 at a higher frequency (e.g., 10 Hz or higher). A three-axis acceleration sensor and a three-axis gyroscope may be provided separately instead of the IMU 115. The IMU 115 may be provided as a device separate from the GNSS unit 110.

[0056] The camera 120 is an imaging device that captures images of the environment surrounding the work vehicle 100. The camera 120 includes an image sensor, such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The camera 120 may also include an optical system including one or more lenses and a signal processing circuit. The camera 120 captures images of the environment surrounding the work vehicle 100 while the work vehicle 100 is traveling and generates image (e.g., video) data. The camera 120 can capture video at a frame rate of, for example, 3 frames per second (fps) or higher. The images generated by the camera 120 can be used, for example, when a remote observer checks the environment surrounding the work vehicle 100 using the remote device 400. The images generated by the camera 120 may be used for positioning or obstacle detection. As shown in FIG. 2, multiple cameras 120 may be installed at different positions on the work vehicle 100, or a single camera may be installed. A visible light camera that generates a visible light image and an infrared camera that generates an infrared image may be provided separately. Both a visible light camera and an infrared camera may be provided as cameras that generate images for surveillance. The infrared camera can also be used to detect obstacles at night.

[0057] The obstacle sensor 130 detects objects present around the work vehicle 100. The obstacle sensor 130 may include, for example, a laser scanner or an ultrasonic sonar. The obstacle sensor 130 outputs a signal indicating the presence of an obstacle when an object is present closer than a predetermined distance from the obstacle sensor 130. Multiple obstacle sensors 130 may be provided at different positions on the work vehicle 100. For example, multiple laser scanners and multiple ultrasonic sonars may be arranged at different positions on the work vehicle 100. By providing such a large number of obstacle sensors 130, blind spots in monitoring obstacles around the work vehicle 100 can be reduced.

[0058] The steering wheel sensor 152 measures the rotation angle of the steering wheel of the work vehicle 100. The turning angle sensor 154 measures the turning angle of the front wheels 104F, which are the steered wheels. The measurement values ​​from the steering wheel sensor 152 and the turning angle sensor 154 are used for steering control by the control device 180.

[0059] The axle sensor 156 measures the rotational speed of the axle connected to the wheel 104, i.e., the number of rotations per unit time. The axle sensor 156 may be a sensor that uses, for example, a magnetoresistive element (MR), a Hall element, or an electromagnetic pickup. The axle sensor 156 outputs a numerical value that indicates, for example, the number of rotations per minute (unit: rpm) of the axle. The axle sensor 156 is used to measure the speed of the work vehicle 100.

[0060] The drive device 240 includes various devices necessary for the travel of the work vehicle 100 and the driving of the implement 300, such as the prime mover 102, transmission 103, steering device 106, and coupling device 108 described above. The prime mover 102 may be equipped with an internal combustion engine such as a diesel engine. The drive device 240 may be equipped with an electric motor for traction instead of or in addition to the internal combustion engine.

[0061] The buzzer 220 is an audio output device that emits a warning sound to notify of an abnormality. For example, the buzzer 220 emits the warning sound when an obstacle is detected during automatic driving. The buzzer 220 is controlled by the control device 180.

[0062] The storage device 170 includes one or more storage media, such as a flash memory or a magnetic disk. The storage device 170 stores various data generated by the GNSS unit 110, the camera 120, the obstacle sensor 130, the LiDAR sensor 140, the sensor group 150, and the control device 180. The data stored in the storage device 170 may include map data (environmental map) of the environment in which the work vehicle 100 travels, and data on a global route (target route) for autonomous driving. The environmental map includes information on multiple fields in which the work vehicle 100 will perform agricultural work and the roads in their surroundings. The environmental map and target route may be generated by a processing device (i.e., a processor) in the management device 600. Note that the control device 180 in this embodiment may have a function for generating or editing the environmental map and target route. The control device 180 can edit the environmental map and target route obtained from the management device 600 according to the travel environment of the work vehicle 100.

[0063] The storage device 170 also stores work plan data received by the communication device 190 from the management device 600. The work plan includes information regarding multiple agricultural tasks to be performed by the work vehicle 100 over multiple work days. The work plan may be, for example, work schedule data including information on the scheduled times for each agricultural task to be performed by the work vehicle 100 on each work day. The storage device 170 also stores computer programs that cause each ECU in the control device 180 to perform various operations, which will be described later. Such computer programs may be provided to the work vehicle 100 via a storage medium (e.g., a semiconductor memory or an optical disk) or an electric communication line (e.g., the Internet). Such computer programs may also be sold as commercial software.

[0064] The control device 180 includes a plurality of ECUs, such as an ECU 181 for speed control, an ECU 182 for steering control, an ECU 183 for implement control, an ECU 184 for automatic driving control, an ECU 185 for route generation, and an ECU 186 for map generation.

[0065] The ECU 181 controls the speed of the work vehicle 100 by controlling the prime mover 102 , the transmission 103 , and the brakes included in the drive unit 240 .

[0066] The ECU 182 controls the steering of the work vehicle 100 by controlling the hydraulic device or electric motor included in the steering device 106 based on the measurement value of the steering wheel sensor 152 .

[0067] The ECU 183 controls the operation of the three-point link and PTO shaft included in the coupling device 108, etc., in order to cause the implement 300 to perform a desired operation. The ECU 183 also generates a signal that controls the operation of the implement 300, and transmits the signal from the communication device 190 to the implement 300.

[0068] The ECU 184 performs calculations and controls to achieve autonomous driving based on data output from the GNSS unit 110, the camera 120, the obstacle sensor 130, the LiDAR sensor 140, and the sensor group 150. For example, the ECU 184 determines the position of the work vehicle 100 based on data output from at least one of the GNSS unit 110, the camera 120, and the LiDAR sensor 140. Within a farm field, the ECU 184 may determine the position of the work vehicle 100 based solely on data output from the GNSS unit 110. The ECU 184 may also estimate or correct the position of the work vehicle 100 based on data acquired by the camera 120 or the LiDAR sensor 140. By using the data acquired by the camera 120 or the LiDAR sensor 140, the accuracy of positioning can be further improved. Outside of a farm field, the ECU 184 estimates the position of the work vehicle 100 using data output from the LiDAR sensor 140 or the camera 120. For example, ECU 184 may estimate the position of work vehicle 100 by matching data output from LiDAR sensor 140 or camera 120 with an environmental map. During autonomous driving, ECU 184 performs calculations necessary for work vehicle 100 to travel along a target path or a local path based on the estimated position of work vehicle 100. ECU 184 sends a speed change command to ECU 181 and a steering angle change command to ECU 182. In response to the speed change command, ECU 181 changes the speed of work vehicle 100 by controlling prime mover 102, transmission 103, or brakes. In response to the steering angle change command, ECU 182 changes the steering angle by controlling steering device 106.

[0069] ECU 184 also controls the remotely controlled traveling of work vehicle 100. In the remote control mode, ECU 184 controls ECUs 181, 182, and 183 in response to signals received by communication device 190 from remote device 400. This makes it possible to perform operations such as speed control of work vehicle 100, steering control, raising and lowering of implement 300, and turning implement 300 on and off in response to remote control from the user.

[0070] ECU 185 sequentially generates local paths that can avoid obstacles while work vehicle 100 is traveling along the target path. While work vehicle 100 is traveling, ECU 185 recognizes obstacles that exist around work vehicle 100 based on data output from camera 120, obstacle sensor 130, and LiDAR sensor 140. ECU 185 generates local paths that avoid the recognized obstacles.

[0071] ECU 185 may have a function for performing global route planning instead of management device 600. In this case, ECU 185 may determine the destination of work vehicle 100 based on the work plan stored in storage device 170, and determine a target route from the start point of work vehicle 100's movement to the destination point. ECU 185 may create, for example, a route that will allow the work vehicle to reach the destination in the shortest time as the target route based on an environmental map that includes road information stored in storage device 170. Alternatively, ECU 185 may generate, as the target route, a route that prioritizes specific types of roads (for example, roads that follow specific features such as farm roads or waterways, or roads that have good reception of satellite signals from GNSS satellites) based on attribute information of each road included in the environmental map.

[0072] The ECU 186 generates or edits a map of the environment in which the work vehicle 100 travels. In this embodiment, an environmental map generated by an external device such as the management device 600 is transmitted to the work vehicle 100 and recorded in the storage device 170, but the ECU 186 can also generate or edit the environmental map instead. The following describes the operation when the ECU 186 generates an environmental map. The environmental map can be generated based on sensor data output from the LiDAR sensor 140. When generating the environmental map, the ECU 186 sequentially generates three-dimensional point cloud data based on the sensor data output from the LiDAR sensor 140 while the work vehicle 100 is traveling. The ECU 186 can generate the environmental map by connecting the sequentially generated point cloud data using an algorithm such as SLAM. The environmental map generated in this manner is a highly accurate three-dimensional map and can be used by the ECU 184 for self-localization estimation. A two-dimensional map used for global path planning can be generated based on this three-dimensional map. In this specification, both the 3D map used for self-localization and the 2D map used for global route planning are referred to as “environment maps.” ECU 186 can also edit the map by adding various attribute information to the map, such as features (e.g., waterways, rivers, grass, trees, etc.) recognized based on data output from camera 120 or LiDAR sensor 140, road type (e.g., whether it is a farm road), road surface condition, or road passability.

[0073] Through the operation of these ECUs, the control device 180 realizes automatic driving and remotely controlled driving. During automatic driving, the control device 180 controls the drive device 240 based on the measured or estimated position of the work vehicle 100 and the generated route. In this way, the control device 180 can cause the work vehicle 100 to travel along a target route. During remotely controlled driving, the control device 180 controls the driving of the work vehicle 100 based on signals (also referred to as "steering commands") transmitted from the remote device 400. In other words, the control device 180 controls the drive device 240 in response to operations by the user using the remote device 400. In this way, the control device 180 can cause the work vehicle 100 to travel according to instructions from the user.

[0074] The multiple ECUs included in the control device 180 can communicate with each other in accordance with a vehicle bus standard such as CAN (Controller Area Network). Instead of CAN, a faster communication method such as Automotive Ethernet (registered trademark) may be used. In FIG. 3, each of the ECUs 181 to 186 is shown as an individual block, but the functions of each of these may be realized by multiple ECUs. An on-board computer that integrates at least some of the functions of the ECUs 181 to 186 may be provided. The control device 180 may include ECUs other than the ECUs 181 to 186, and any number of ECUs may be provided depending on the functions. Each ECU includes a processing circuit including one or more processors.

[0075] The communication device 190 includes circuits for communicating with the implement 300, the remote device 400, and the management device 600. The communication device 190 transmits sensing data output from the sensing device 250 or transmission data, such as image data based on the sensing data, to the remote device 400. The communication device 190 includes circuits for transmitting and receiving signals compliant with ISOBUS standards, such as ISOBUS-TIM, between the communication device 390 of the implement 300. This allows the implement 300 to perform desired operations and acquire information from the implement 300. The communication device 190 may further include an antenna and communication circuits for transmitting and receiving signals between the communication devices of the remote device 400 and the management device 600 via the network 80. The network 80 may include, for example, a cellular mobile communication network such as 3G, 4G, or 5G, and the Internet. The communication device 190 may also have a function for communicating with a mobile terminal used by a supervisor near the work vehicle 100. Communication between such mobile terminals may be performed in accordance with any wireless communication standard, such as Wi-Fi (registered trademark), cellular mobile communication such as 3G, 4G or 5G, or Bluetooth (registered trademark).

[0076] The operation terminal 200 is a terminal through which a user performs operations related to the travel of the work vehicle 100 and the operation of the implement 300, and is also referred to as a virtual terminal (VT). The operation terminal 200 may include a display device such as a touch screen and / or one or more buttons. The display device may be, for example, a liquid crystal display or an organic light-emitting diode (OLED) display. By operating the operation terminal 200, a user can perform various operations, such as switching the autonomous driving mode on / off, switching the remote operation mode on / off, recording or editing an environmental map, setting a target route, and switching the implement 300 on / off. At least some of these operations can also be achieved by operating the operation switch group 210. The operation terminal 200 may be configured to be detachable from the work vehicle 100. A user located away from the work vehicle 100 may operate the detached operation terminal 200 to control the operation of the work vehicle 100. Instead of operating terminal 200, the user may control the operation of work vehicle 100 by operating a computer, such as remote device 400, on which necessary application software has been installed.

[0077] FIG. 5 is a diagram showing an example of operation terminal 200 and operation switch group 210 provided inside cabin 105. Operation switch group 210 including a plurality of switches that can be operated by the user is arranged inside cabin 105. Operation switch group 210 may include, for example, a switch for selecting a gear position of the main transmission or auxiliary transmission, a switch for switching between automatic driving mode and manual driving mode, a switch for switching between forward and reverse, a switch for switching between four-wheel drive and two-wheel drive, a switch for disconnecting the left and right brakes, and a switch for raising and lowering implement 300. Note that if work vehicle 100 only performs unmanned operation and does not have a function for manned operation, work vehicle 100 does not need to be equipped with operation switch group 210.

[0078] At least some of the operations that can be performed by the operation terminal 200 or the operation switch group 210 can also be performed by remote operation using the remote device 400. When a user performs a predetermined operation on the screen displayed on the display of the remote device 400, any of the above operations can be performed.

[0079] The drive device 340 in the implement 300 shown in FIG. 3 performs the operations required for the implement 300 to perform a predetermined task. The drive device 340 includes devices appropriate for the application of the implement 300, such as a hydraulic device, an electric motor, or a pump. The control device 380 controls the operation of the drive device 340. The control device 380 causes the drive device 340 to perform various operations in response to signals transmitted from the work vehicle 100 via the communication device 390. In addition, a signal appropriate for the state of the implement 300 can also be transmitted from the communication device 390 to the work vehicle 100.

[0080] Next, the configurations of the management device 600 and the remote device 400 will be described with reference to Fig. 6. Fig. 6 is a block diagram illustrating a schematic hardware configuration of the management device 600 and the remote device 400.

[0081] The management device 600 includes a storage device 650, a processor 660, a read-only memory (ROM) 670, a random access memory (RAM) 680, and a communication device 690. These components are communicatively connected to each other via a bus. The management device 600 manages the schedule of agricultural work performed in the field by the work vehicle 100 and can function as a cloud server that supports agriculture by utilizing the data it manages. A user can input information necessary for creating a work plan using the remote device 400 and upload that information to the management device 600 via the network 80. The management device 600 can create a schedule for agricultural work, i.e., a work plan, based on that information. The management device 600 can also generate or edit an environmental map and perform global path planning for the work vehicle 100. The environmental map may be distributed from a computer external to the management device 600.

[0082] The communication device 690 is a communication module for communicating with the work vehicle 100 and the remote device 400 via the network 80. The communication device 690 can perform wired communication in accordance with communication standards such as IEEE1394 (registered trademark) or Ethernet (registered trademark). The communication device 690 may also perform wireless communication in accordance with the Bluetooth (registered trademark) standard or the Wi-Fi standard, or cellular mobile communication such as 3G, 4G, or 5G.

[0083] The processor 660 may be, for example, a semiconductor integrated circuit including a central processing unit (CPU). The processor 660 may be implemented by a microprocessor or a microcontroller. Alternatively, the processor 660 may be implemented by a field programmable gate array (FPGA) equipped with a CPU, a graphics processing unit (GPU), an application specific integrated circuit (ASIC), an application specific standard product (ASSP), or a combination of two or more circuits selected from these circuits. The processor 660 sequentially executes a computer program stored in the ROM 670, which describes a group of instructions for executing at least one process, to achieve the desired process.

[0084] The ROM 670 is, for example, a writable memory (e.g., a PROM), a rewritable memory (e.g., a flash memory), or a read-only memory. The ROM 670 stores a program that controls the operation of the processor 660. The ROM 670 does not need to be a single storage medium, but may be a collection of multiple storage media. Part of the collection of multiple storage media may be removable memory.

[0085] The RAM 680 provides a working area for temporarily loading the control program stored in the ROM 670 at boot time. The RAM 680 does not have to be a single storage medium, but may be a collection of multiple storage media.

[0086] The storage device 650 mainly functions as database storage. The storage device 650 may be, for example, a magnetic storage device or a semiconductor storage device. An example of a magnetic storage device is a hard disk drive (HDD). An example of a semiconductor storage device is a solid state drive (SSD). The storage device 650 may be a device independent of the management device 600. For example, the storage device 650 may be a storage device connected to the management device 600 via the network 80, such as a cloud storage device.

[0087] The remote device 400 shown in FIG. 6 includes an input device 420, a display device (display) 430, a storage device 450, a processor 460, a ROM 470, a RAM 480, and a communication device 490. These components are communicatively connected to one another via a bus. The input device 420 converts user instructions into data and inputs the data to a computer. The input device 420 may be, for example, a keyboard, a mouse, or a touch panel. The display device 430 may be, for example, a liquid crystal display or an organic electroluminescence (EL) display. The processor 460, ROM 470, RAM 480, storage device 450, and communication device 490 are described in the hardware configuration example of the management device 600, and therefore will not be described here.

[0088] In the example of Figure 6, the remote device 400 is a computer with a built-in display and input devices, as shown in Figure 1A. The remote device 400 may also be a computer connected to a remote control 500 and a display 430 at a remote monitoring center, as illustrated in Figure 1B.

[0089] [2. Operation] Next, the operations of the work vehicle 100, the remote device 400, and the management device 600 will be described.

[0090] [2-1.Automatic driving operation] First, an example of the operation of autonomous driving by the work vehicle 100 will be described. The work vehicle 100 in this embodiment can travel autonomously both inside and outside the field. In the field, the work vehicle 100 drives the implement 300 to perform predetermined agricultural work while traveling along a predetermined target route. If the obstacle sensor 130 detects an obstacle while traveling in the field, the work vehicle 100 stops traveling, emits a warning sound from the buzzer 220, and transmits a warning signal to the remote device 400. In the field, the positioning of the work vehicle 100 is performed mainly based on data output from the GNSS unit 110. On the other hand, outside the field, the work vehicle 100 travels autonomously along a target route set on a farm road or public road outside the field. While traveling outside the field, the work vehicle 100 travels while performing local route planning based on data acquired by the camera 120 or the LiDAR sensor 140. When the work vehicle 100 detects an obstacle outside the field, it either avoids the obstacle or stops on the spot. Outside the field, the position of the work vehicle 100 is estimated based on the data output from the LiDAR sensor 140 or the camera 120 in addition to the positioning data output from the GNSS unit 110.

[0091] Below, we will first explain the operation of the work vehicle 100 when it travels automatically within a farm field. The operation of the work vehicle 100 when it travels automatically outside a farm field will be described later.

[0092] FIG. 7 is a diagram schematically illustrating an example of a work vehicle 100 that automatically travels through a field along a target route. In this example, the field includes a work area 72 where the work vehicle 100 performs work using an implement 300, and a headland 74 located near the outer periphery of the field. The user can set in advance which areas of the field on the map correspond to the work area 72 or the headland 74. The target route in this example includes multiple parallel main routes P1 and multiple turning routes P2 that connect the multiple main routes P1. The main routes P1 are located within the work area 72, and the turning routes P2 are located within the headland 74. Although each main route P1 shown in FIG. 7 is a straight route, each main route P1 may also include curved portions. The main path P1 can be automatically generated, for example, by a user viewing a map of the field displayed on the operation terminal 200 or the remote device 400 and specifying two points near the edge of the field (points A and B in FIG. 7). In this case, multiple main paths P1 are set parallel to a line segment connecting points A and B specified by the user, and a target path within the field is generated by connecting these main paths P1 with a turning path P2. The dashed line in FIG. 7 represents the working width of the implement 300. The working width is set in advance and recorded in the storage device 170. The working width can be set and recorded by the user operating the operation terminal 200 or the remote device 400. Alternatively, the working width can be automatically recognized and recorded when the implement 300 is connected to the work vehicle 100. The spacing between the multiple main paths P1 can be set to match the working width. The target path can be created based on user operation before automatic driving begins. The target route can be created so as to cover the entire work area 72 in a farm field, for example. The work vehicle 100 automatically travels back and forth from the start point of the work to the end point of the work along the target route as shown in Fig. 7. Note that the target route shown in Fig. 7 is merely an example, and the target route can be determined in any way.

[0093] Next, an example of control by the control device 180 during automatic operation in a farm field will be described.

[0094] FIG. 8 is a flowchart showing an example of the operation of steering control during automatic driving executed by the control device 180. The control device 180 performs automatic steering by executing the operations of steps S121 to S125 shown in FIG. 8 while the work vehicle 100 is traveling. The speed is maintained at, for example, a preset speed. While the work vehicle 100 is traveling, the control device 180 acquires data indicating the position of the work vehicle 100 generated by the GNSS unit 110 (step S121). Next, the control device 180 calculates the deviation between the position of the work vehicle 100 and the target route (step S122). The deviation represents the distance between the position of the work vehicle 100 at that time and the target route. The control device 180 determines whether the calculated position deviation exceeds a preset threshold (step S123). If the deviation exceeds the threshold, the control device 180 changes the steering angle by changing the control parameters of the steering device included in the drive device 240 so as to reduce the deviation. If the deviation does not exceed the threshold value in step S123, the operation of step S124 is skipped. In the following step S125, the control device 180 determines whether or not a command to end the operation has been received. A command to end the operation may be issued, for example, when a user remotely instructs the work vehicle 100 to stop autonomous driving, or when the work vehicle 100 reaches its destination. If a command to end the operation has not been issued, the process returns to step S121, and the same operation is performed based on the newly measured position of the work vehicle 100. The control device 180 repeats the operations of steps S121 to S125 until a command to end the operation is issued. The above operations are executed by the ECUs 182 and 184 in the control device 180.

[0095] 8, the control device 180 controls the drive device 240 based only on the deviation between the position of the work vehicle 100 identified by the GNSS unit 110 and the target route, but the control may also take into consideration the deviation in heading. For example, when the heading deviation, which is the angular difference between the orientation of the work vehicle 100 identified by the GNSS unit 110 and the direction of the target route, exceeds a preset threshold, the control device 180 may change the control parameters (e.g., steering angle) of the steering device of the drive device 240 in accordance with the deviation.

[0096] An example of steering control by the control device 180 will be described in more detail below with reference to FIGS. 9A to 9D.

[0097] FIG. 9A is a diagram showing an example of a work vehicle 100 traveling along a target route P. FIG. 9B is a diagram showing an example of a work vehicle 100 shifted to the right from the target route P. FIG. 9C is a diagram showing an example of a work vehicle 100 shifted to the left from the target route P. FIG. 9D is a diagram showing an example of a work vehicle 100 facing in an inclined direction with respect to the target route P. In these figures, the pose indicating the position and orientation of the work vehicle 100 measured by the GNSS unit 110 is expressed as r(x, y, θ). (x, y) are coordinates representing the position of the reference point of the work vehicle 100 in the XY coordinate system, which is a two-dimensional coordinate system fixed to the Earth. In the examples shown in FIGS. 9A to 9D, the reference point of the work vehicle 100 is located at the position where the GNSS antenna is installed on the cabin, but the position of the reference point is arbitrary. θ is an angle representing the measured orientation of the work vehicle 100. In the examples shown, the target route P is parallel to the Y axis, but generally, the target route P is not necessarily parallel to the Y axis.

[0098] As shown in FIG. 9A, if the position and orientation of the work vehicle 100 do not deviate from the target route P, the control device 180 maintains the steering angle and speed of the work vehicle 100 unchanged.

[0099] As shown in Fig. 9B, when the position of work vehicle 100 has shifted to the right from target route P, control device 180 changes the steering angle so that the traveling direction of work vehicle 100 leans to the left and approaches target route P. At this time, the speed may also be changed in addition to the steering angle. The magnitude of the steering angle may be adjusted, for example, according to the magnitude of position deviation Δx.

[0100] As shown in Fig. 9C, when the position of the work vehicle 100 has shifted to the left from the target route P, the control device 180 changes the steering angle so that the traveling direction of the work vehicle 100 leans to the right and approaches the target route P. In this case, too, the speed may be changed in addition to the steering angle. The amount of change in the steering angle may be adjusted, for example, according to the magnitude of the position deviation Δx.

[0101] As shown in FIG. 9D , when the position of the work vehicle 100 is not significantly deviated from the target route P but the heading is different from the direction of the target route P, the control device 180 changes the steering angle to reduce the azimuth deviation Δθ. In this case, the speed may also be changed in addition to the steering angle. The magnitude of the steering angle may be adjusted, for example, according to the magnitudes of the position deviation Δx and the azimuth deviation Δθ. For example, the smaller the absolute value of the position deviation Δx, the greater the amount of change in the steering angle according to the azimuth deviation Δθ. When the absolute value of the position deviation Δx is large, the steering angle will be changed significantly to return to the target route P, which inevitably increases the absolute value of the azimuth deviation Δθ. Conversely, when the absolute value of the position deviation Δx is small, it is necessary to bring the azimuth deviation Δθ closer to zero. For this reason, it is appropriate to relatively increase the weight of the azimuth deviation Δθ (i.e., the control gain) used to determine the steering angle.

[0102] Control techniques such as PID control or MPC control (model predictive control) can be applied to the steering control and speed control of work vehicle 100. By applying these control techniques, it is possible to smooth the control that brings work vehicle 100 closer to target path P.

[0103] If an obstacle is detected by one or more obstacle sensors 130 while the work vehicle 100 is traveling, the control device 180 will stop the work vehicle 100. At this time, the buzzer 220 may be caused to emit a warning sound or a warning signal may be sent to the remote device 400. If it is possible to avoid the obstacle, the control device 180 may control the drive device 240 to avoid the obstacle.

[0104] The work vehicle 100 in this embodiment is capable of autonomous driving not only in farm fields but also outside of farm fields. Outside of farm fields, the control device 180 can detect objects (e.g., other vehicles or pedestrians) that are located relatively far from the work vehicle 100 based on data output from the camera 120 or the LiDAR sensor 140. The control device 180 generates a local route to avoid the detected object, and performs speed control and steering control along the local route, thereby realizing autonomous driving on roads outside of farm fields.

[0105] In this manner, the work vehicle 100 in this embodiment can autonomously travel unmanned within and outside a field. FIG. 10 is a diagram schematically illustrating an example of a situation in which multiple work vehicles 100 are autonomously traveling within a field 70 and on a road 76 outside the field 70. An environmental map and a target route for an area including multiple fields and their surrounding roads are stored in the storage device 170. The environmental map and target route can be generated by the management device 600 or the ECU 185. When the work vehicle 100 travels on a road, the work vehicle 100 travels along the target route with the implement 300 raised while sensing the surroundings using sensing devices such as the camera 120 and the LiDAR sensor 140. While traveling, the control device 180 sequentially generates local routes and causes the work vehicle 100 to travel along the local routes. This enables autonomous traveling while avoiding obstacles. The target route may be changed during travel depending on the situation. In this way, the control device 180 in this embodiment can generate a target route for autonomous driving within the field and on roads around the field. In the autonomous driving mode, the control device 180 causes the work vehicle 100 to autonomously drive within an autonomous driving area defined by the field and roads for which the target route has been generated.

[0106] [2-2. Remote Control] Next, the operation relating to the remote control of the work vehicle 100 will be described.

[0107] When the work vehicle 100 is traveling autonomously, the user can remotely monitor and remotely control the work vehicle 100 using the remote device 400. When the work vehicle 100 is traveling autonomously, the control device 180 transmits images (e.g., video images) captured by one or more cameras 120 mounted on the work vehicle 100 to the remote device 400 via the communication device 190. The remote device 400 displays the images on the display 430. While viewing the displayed images, the user can check the situation around the work vehicle 100 and start remote-controlled traveling as necessary.

[0108] FIG. 11A is a diagram showing an example of an image displayed on the display 430 of the remote device 400. The image shown in FIG. 11A shows a field 70, a road 76, sky 79, and the front of the work vehicle 100. This image was captured by a camera 120 capturing an image in front of the work vehicle 100. Images captured by a camera 120 capturing an image in front of the work vehicle 100, as well as images captured by a camera 120 capturing an image in the rear, right, or left direction, may also be displayed on the display 430. The display 430 displays moving images at a frame rate of, for example, 3 fps or more (typically, 30 fps or 60 fps, etc.). Multiple images captured by multiple cameras 120 may be displayed on multiple displays. For example, as shown in FIG. 1B, multiple images may be displayed on multiple displays 430 at a remote monitoring center. In this case, the user (i.e., the operator) who is the monitor can check the situation around the work vehicle 100 in detail by viewing the multiple images displayed on the multiple displays 430. In addition to the image captured by the camera 120, a map of the area including the work vehicle 100 may be displayed on the display.

[0109] In the example shown in FIG. 11A, the displayed image includes a button (remote control start button) 81 for instructing the start of remote control, and a button (emergency stop button) 82 for making an emergency stop of work vehicle 100. The user can switch from automatic driving mode to remote control mode by touching or clicking remote control start button 81. The user can also make an emergency stop of work vehicle 100 by touching or clicking emergency stop button 82.

[0110] FIG. 11B is a diagram showing an example of the display screen after remote control start button 81 has been pressed. When remote control start button 81 is pressed, remote control becomes possible. For example, in the example of FIG. 1B, the operator can remotely control work vehicle 100 using remote controller 500. In response to an operation by the user, control device 180 causes work vehicle 100 to perform the instructed operation. The user can switch from remote control mode to automatic driving mode by touching or clicking automatic driving start button 84 on the screen shown in FIG. 11B.

[0111] In this example, when the remote control start button 81 shown in Fig. 11A is pressed, a barrier 77 is displayed, as shown in Fig. 11B, indicating an area where remotely controlled driving of the work vehicle 100 is not permitted at that time. The barrier 77 shown in Fig. 11B is displayed on the boundary between the road 76 and the field 70, excluding the entrance / exit 71 of the field 70. The areas in which remotely controlled driving is permitted and the position at which the barrier 77 is displayed are determined in advance. Remote control of the work vehicle 100 to enter the barrier 77 is disabled.

[0112] 11C is a diagram showing an example of a display screen in remote operation mode when the work vehicle 100 is located within the field 70. In this example, a barrier 77 is displayed on the outer periphery of the field 70 excluding the entrance / exit 71. This disables remote operation of the work vehicle 100 to exit the field 70 from any location other than the entrance / exit 71.

[0113] There may be cases where no area is set where remote control of the work vehicle 100 is prohibited. In such cases, the barrier 77 is not displayed on the monitoring screen, and remote control is possible regardless of the location of the work vehicle 100 as long as it is within the range where a remote control command can be received.

[0114] In the above example, an image acquired by the camera 120 mounted on the work vehicle 100 (hereinafter also referred to as a "camera image") is displayed on the display 430 of the remote device 400. In addition to camera images, images based on point cloud data acquired by the LiDAR sensor 140 or other sensing data may also be displayed on the display 430. Images based on point cloud data show the distribution of features present around the work vehicle 100, and can therefore be used for monitoring in the same way as camera images. Based on camera images or images based on point cloud data, the operator can grasp the situation around the work vehicle 100. In the following description, image data generated by the camera 120 and images based on point cloud data generated by the LiDAR sensor 140 may be referred to as "visualized images." Furthermore, moving images or videos based on such visualized images may be referred to as "time-series images."

[0115] [3. Adaptive change of output data volume from sensing device according to communication speed] In the remote control system described above, as the number of work vehicles connected to the network increases, many work vehicles simultaneously use radio waves on the same or adjacent frequencies for wireless communication, which can easily cause problems such as radio wave interference or crosstalk. As a result, communication between the communication device and the remote device can become unstable or the communication speed can decrease. With the spread of smart agriculture, the number of work vehicles and remote devices connected to the network is expected to increase, making problems such as radio wave interference or crosstalk more likely to occur. It is expected that the communication speed between the communication device and the remote device will frequently decrease. This decrease in communication speed can increase the processing load on the communication device.

[0116] Using new communication technologies such as the fifth-generation mobile communication system (5G), remote control systems that enable low-latency and large-capacity data communication are being developed. However, as sensing devices such as LiDAR or cameras process large amounts of data due to, for example, higher image resolution, the amount of data output from the sensing devices also increases. As a result, this places an increased burden on communication devices that process the sensing data output from the sensing devices. Therefore, it is desirable to suppress the increase in the processing load on communication devices.

[0117] Taking remote control as an example, an image showing the environmental conditions around the work vehicle 100 based on sensing data output from a sensing device is displayed on one or more displays 430 installed in the remote monitoring center shown in Fig. 1B. If the communication speed from the communication device 190 to the remote device 400 decreases, this image may be continuously or temporarily degraded (or distorted), which may interfere with the remote control of the user using the remote controller 500.

[0118] The sensing device 250 in this embodiment is configured to change the amount of sensing data to be output to the communication device 190 in accordance with the communication speed from the communication device 190 to the remote device 400. Specifically, when the communication speed is below a threshold, the sensing device 250 outputs sensing data with an amount of data smaller than the amount of sensing data output when the communication speed is equal to or greater than the threshold. While the communication speed is equal to or greater than the threshold, the sensing device 250 outputs, for example, the maximum amount of data that can be output, and when the communication speed falls below the threshold, outputs, for example, one-half or one-third of the maximum amount of data.

[0119] According to the agricultural machine or sensing system of this embodiment, the amount of sensing data output from the sensing device 250 can be adaptively changed in accordance with the communication speed of the communication device, thereby reducing the processing load on the communication device. Furthermore, data communication between the communication device and a remote device can be realized in accordance with the communication congestion state.

[0120] Fig. 12 is a graph illustrating an example of changes in the communication speed of the communication device 190 and changes in the amount of sensing data output from the sensing device in response to the changes in the communication speed. The upper graph in Fig. 12 shows changes in the communication speed over time, and the lower graph shows changes in the amount of sensing data over time. The horizontal axis represents time, and the vertical axis represents the communication speed (upper graph) and the amount of data (lower graph). Fig. 13 is a diagram for explaining state transitions in the operation of the sensing device 250 in response to the communication speed of the communication device 190.

[0121] In this embodiment, a period in which the communication speed of the communication device 190 is equal to or higher than a threshold is called a "steady section," and a period in which the communication speed of the communication device 190 is lower than the threshold is called a "deteriorating section," to distinguish between the two. The threshold may be determined appropriately depending on, for example, the environment in which the sensing system is used. The sensing device 250 is configured to output sensing data according to a first operation setting during the steady section, and to output sensing data according to a second operation setting during the deteriorating section.

[0122] As illustrated in FIG. 13, it is assumed that the sensing device 250 outputs sensing data according to the first operation setting during a steady period in which the communication speed of the communication device 190 is equal to or higher than a threshold. The sensing device 250 repeatedly outputs sensing data according to the first operation setting while the communication speed of the communication device 190 is equal to or higher than the threshold (condition A). If the communication speed of the communication device 190 falls below the threshold at a certain timing, the sensing device 250 outputs sensing data according to the second operation setting (condition B). If the communication speed of the communication device 190 is below the threshold, the sensing device 250 repeatedly outputs sensing data according to the second operation setting (condition C). If the communication speed of the communication device 190 recovers to equal to or higher than the threshold at a certain timing, the sensing device 250 outputs sensing data according to the first operation setting. The first operation setting and the second operation setting will be described in detail later.

[0123] First, the operation of the sensing device 250 in the first example will be described. The sensing device 250 in this example includes a camera 120 equipped with an RGB sensor. The camera 120 is capable of capturing color images. When the communication speed of the communication device 190 is below a threshold, the camera 120 reduces at least one of the resolution, amount of color information, and image size during image capture compared to when the communication speed is equal to or greater than the threshold. This information is set, for example, in a register within a processor of the sensing device 250. For example, the processor of the sensing device 250 may communicate with the processor of the communication device 190 and receive a signal (e.g., 1 bit) indicating whether the communication speed is equal to or greater than the threshold from the processor of the communication device 190. This allows the sensing device 250 to determine whether the communication speed has fallen below the threshold. The control device 180 may also receive a signal indicating whether the communication speed is equal to or greater than the threshold from the processor of the communication device 190. For example, in response to the assertion of this signal, the control device 180 may change the setting of a register within the processor of the sensing device 250. The setting of the register before the change corresponds to the first operational setting described above, and the setting of the register after the change corresponds to the second operational setting described above. In other words, in response to the assertion of the signal, the control device 180 changes the setting of the register from the first operational setting to the second operational setting.

[0124] Reducing the resolution includes, for example, thinning the maximum vertical resolution of the image to half or one-quarter the resolution, or thinning the maximum horizontal and vertical resolutions of the image to half or one-quarter the resolution. Reducing the amount of color information means, for example, when the output format is RGB, outputting an RG image, GB image, or RB image of the RGB pixel values, or outputting any one of the RGB pixel values. Alternatively, when the output format is YCbCr, thinning the color difference information by chroma sampling to change the maximum 4:4:4 format to 4:4:2, 4:2:2, 4:2:0, or 4:1:1. In this example, the first operational setting is the 4:4:4 format, and the second operational setting is the 4:4:2, 4:2:2, 4:2:0, or 4:1:1 format.

[0125] FIG. 14 is a diagram illustrating reducing an image size while maintaining resolution. The rectangular area surrounded by a solid line in FIG. 14 represents a full-size image 700, and the rectangular area surrounded by a dashed line represents a reduced-size image 701 reduced to 1 / 4 of the full size (reduced to 1 / 2 in both the horizontal and vertical directions). Note that the position of the reduced-size image 701 within the full-size image 700 shown in FIG. 14 is merely an example and is arbitrary. The camera 120 outputs full-size image data when the communication speed is equal to or higher than a threshold. On the other hand, when the communication speed of the communication device 190 is less than the threshold, the camera 120 outputs data of a reduced-size image having the same resolution as the full-size image, for example, 1 / 4, by cropping a predetermined area from the full-size image 700. In this example, the first operation setting is full size, and the second operation setting is a reduced size of 1 / 4. However, the reduction ratio is not limited to 1 / 4 and is arbitrary.

[0126] Camera 120 may be a stereo camera. In this example, camera 120 can reduce the parallax when the communication speed of communication device 190 is below a threshold. By adjusting the amount of parallax to be relatively smaller than the amount of parallax when the communication speed is equal to or higher than the threshold, the stereoscopic effect expressed by the parallax becomes softer, but it is possible to reduce the amount of image data.

[0127] In the above example, one threshold is set, and the amount of sensing data output from the sensing device 250 is changed depending on the magnitude relationship between the communication speed and the threshold, but the present disclosure is not limited to this. Two or more thresholds may be set, and the amount of sensing data output from the sensing device 250 may be changed in stages depending on the magnitude relationship between the communication speed and each threshold.

[0128] Next, the operation of the sensing device 250 in the second example will be described. The sensing device 250 in this example includes the LiDAR sensor 140. When the communication speed of the communication device 190 is below a threshold, the LiDAR sensor 140 reduces at least one of the amount of point cloud data output per unit time and the angle range of sensing, compared to when the communication speed is equal to or higher than the threshold.

[0129] The sensing device 250 can efficiently reduce the point cloud data acquired by sensing, for example, by applying a filtering process. For example, the sensing device 250 may extract and reduce data points in areas with low point density, thin out data points evenly in both the horizontal and vertical directions, or reduce data points whose reflection intensity is lower than a threshold. This reduces the amount of point cloud data output per unit time.

[0130] The LiDAR sensor 140 may change the angular range or resolution of sensing in the horizontal and / or vertical directions between the steady state and the declining state. The resolution refers to the number of laser beams emitted from the LiDAR sensor 140 that are included in the angular range of sensing.

[0131] Fig. 15A is a diagram schematically showing a sensing area 710 set in a first angular range θ1 for sensing in the horizontal direction. Fig. 15B is a diagram schematically showing a sensing area 720 set in a second angular range θ2 for sensing in the horizontal direction. Fig. 16A is a diagram schematically showing a sensing area 730 set in a third angular range θ3 for sensing in the vertical direction. Fig. 16B is a diagram schematically showing a sensing area 740 set in a fourth angular range θ4 for sensing in the vertical direction.

[0132] During the steady period, the LiDAR sensor 140 scans the surrounding environment in a first angular range θ1 in the horizontal direction, as illustrated in FIG. 15A. During the decline period, the LiDAR sensor 140 scans the surrounding environment in a second angular range θ2 in the horizontal direction, as illustrated in FIG. 15B. The second angular range θ2 is smaller than the first angular range θ1. By reducing the angular range of sensing in the horizontal direction and lowering the horizontal resolution, the number of acquired data points is reduced. Similarly to the horizontal direction, the LiDAR sensor 140 may scan the surrounding environment in a third angular range θ3 in the vertical direction during the steady period, as illustrated in FIG. 16A. During the decline period, the LiDAR sensor 140 may scan the surrounding environment in a fourth angular range θ4 in the vertical direction, as illustrated in FIG. 16B. The fourth angular range θ4 is smaller than the third angular range θ3. In this way, by reducing the angular range of sensing in the horizontal and / or vertical directions during the decline period, the amount of point cloud data acquired by the LiDAR sensor 140 per unit time can be reduced. Therefore, compared to the steady section, it is possible to reduce the amount of point cloud data output by the LiDAR sensor 140.

[0133] 17A is a diagram schematically illustrating a sensing area 710 whose maximum detection distance from the LiDAR sensor 140 is a first distance L1. FIG. 17B is a diagram schematically illustrating a sensing area 720 whose maximum detection distance from the LiDAR sensor 140 is a second distance L2.

[0134] Instead of or in addition to changing the sensing angle range, the LiDAR sensor 140 may change the maximum detection distance from the LiDAR sensor 140 between the steady section and the decreasing section. The maximum detection distance can be adjusted by adjusting the laser output. In the steady section, the LiDAR sensor 140 scans a sensing area 710 having a maximum detection distance of a first distance L1 as illustrated in FIG. 17A . In the decreasing section, the LiDAR sensor 140 scans a sensing area 720 having a maximum detection distance of a second distance L2 as illustrated in FIG. 17B . The second distance L2 is shorter than the first distance L1. By shortening the maximum detection distance in the decreasing section, the amount of point cloud data acquired by the LiDAR sensor 140 per unit time can be reduced. This makes it possible to reduce the amount of point cloud data output by the LiDAR sensor 140 compared to the steady section. Furthermore, by narrowing the sensing angle range and shortening the maximum detection distance in the decreasing section, the amount of point cloud data can be efficiently reduced.

[0135] Next, the operation of the sensing device 250 in a third example will be described. The sensing device 250 in this example includes an obstacle sensor 130. When the obstacle sensor 130 does not detect an obstacle, it may output sensing data with an amount of data smaller than the amount of sensing data output when an obstacle is detected. If the obstacle sensor 130 includes a laser scanner, the laser scanner may reduce the amount of acquired point cloud data while an obstacle is detected, for example, by applying the filtering process described above. Obstacle detection may be performed based on sensing data output from the obstacle sensor 130. For example, the control device 180 may have a function to detect an obstacle, or the remote device 400 may have a function to detect an obstacle.

[0136] The communication device 190 may be configured to reduce the amount of sensing data corresponding to areas with little change in the video before transmitting the sensing data to the remote device 400, based on time-series images (or videos) that visualize the environment around the work vehicle 100, which are generated based on sensing data output from the sensing device 250.

[0137] Referring again to FIG. 11A , in an image of the environment around the work vehicle 100, for example, the sky 79 included in the background is one of the areas that shows little change between successive frames of a frame group that makes up a time-series image. For example, the control device 180 may have a function for detecting areas with little change in the image. Before transmitting the sensing data to the remote device 400, the communication device 190 in this embodiment performs preprocessing to reduce the amount of sensing data corresponding to the area of ​​the sky 79 that shows little change in the image. This preprocessing enables the communication device 190 to efficiently compress the sensing data and reduce the data amount of the video stream.

[0138] The communication device 190 may include an encoder, and the remote device 400 may include a decoder. The encoder of the communication device 190 may encode the sensing data output from the sensing device 250 to generate streaming data. For example, the streaming data may be transmitted to the remote device 400 or to the management device 600 and recorded in the storage device 650. The streaming data recorded in the storage device 650 may be used for purposes other than remote control, such as observing the growth status of crops or pests, which do not require real-time performance. The decoder of the remote device 400 receives and decodes the streaming data to generate a video of the environment surrounding the work vehicle 100. This video is video with reduced degradation or distortion. A user at the remote monitoring center can monitor the autonomous driving of the work vehicle 100 or remotely control the work vehicle 100 using the remote control 500 while viewing the video displayed on the display 430.

[0139] According to the agricultural machine or sensing system of this embodiment, the amount of sensing data output from the sensing device is adaptively changed according to the communication speed of the communication device, thereby making it possible to reduce the amount of communication data when the communication speed drops. Furthermore, by applying data compression processing on the communication device 190 side to the sensing data with reduced data volume output from the sensing device, the effect of reducing the amount of communication data can be further enhanced. However, data compression processing on the communication device 190 side is not essential.

[0140] An agricultural machine 100 according to an embodiment of the present disclosure includes a vehicle body 101, a sensing device 250 that senses the environment around the vehicle body 101 and outputs sensing data, and a communication device 190 that transmits the sensing data output from the sensing device 250 to a remote device 400. The sensing device 250 changes the amount of sensing data to be output to the communication device 190 depending on the communication speed from the communication device 190 to the remote device 400. According to this agricultural machine, it is possible to reduce the amount of data transmitted when the communication speed between the communication device 190 and the remote device 400 decreases.

[0141] When the communication speed is below the threshold, the sensing device 250 can output a smaller amount of sensing data than the amount of sensing data output when the communication speed is equal to or greater than the threshold. With this configuration, the amount of sensing data can be adaptively changed depending on the magnitude relationship between the communication speed and the threshold.

[0142] The sensing device 250 may include a camera 120. When the communication speed is below a threshold, the camera 120 may reduce at least one of the resolution, amount of color information, parallax, and image size during image capture compared to when the communication speed is equal to or higher than the threshold. With this configuration, the amount of data output from the camera can be adaptively changed according to the communication speed.

[0143] The sensing device 250 may include a LiDAR sensor 140. When the communication speed is below a threshold, the LiDAR sensor 140 may reduce at least one of the amount of point cloud data output per unit time and the angle range of sensing compared to when the communication speed is equal to or greater than the threshold. With this configuration, the amount of data output from the LiDAR sensor can be adaptively changed according to the communication speed.

[0144] The sensing device 250 may include an obstacle sensor 130 that detects obstacles present in the environment surrounding the vehicle body 101. When no obstacle is detected, the obstacle sensor 130 may output sensing data with a smaller amount of data than the amount of sensing data output when an obstacle is detected. With this configuration, the amount of data output from the obstacle sensor can be adaptively changed according to the communication speed.

[0145] Based on the time-series images that visualize the environment around the vehicle body 101, which are generated based on the sensing data, the communication device 190 can reduce the amount of sensing data corresponding to areas with little change in the time-series images before transmitting the sensing data to the remote device 400. According to this configuration, by performing preprocessing to reduce the amount of sensing data corresponding to areas with little change in the video, it becomes possible to efficiently compress the sensing data on the communication device side and reduce the data amount of the video stream.

[0146] The control device 180 can control the traveling of the vehicle body 101 based on signals transmitted from the remote device 400. This configuration enables remote control from a location distant from the agricultural machine.

[0147] The sensing system according to the embodiment of the present disclosure includes a sensing device 250 that senses the environment around the agricultural machine and outputs sensing data, and a communication device 190 that transmits the sensing data output from the sensing device 250 to a remote device 400. The sensing device 250 changes the amount of sensing data to be output to the communication device 190 depending on the communication speed from the communication device 190 to the remote device 400.

[0148] A sensing method according to an embodiment of the present disclosure includes the steps of acquiring sensing data from a sensing device 250 that senses the environment around the agricultural machine 100 and outputs the sensing data, transmitting the sensing data from a communication device 190 to a remote device 400, and causing the sensing device 250 to change the amount of sensing data depending on the communication speed from the communication device 190 to the remote device 400.

[0149] [4. Compensation operation when communication speed drops] In the remote control system described above, the communication device 190 of the work vehicle 100 transmits transmission data based on sensing data output from a sensing device 250, such as the camera 120 or LiDAR sensor 140, to a remote device 400, which transmits control commands to the work vehicle 100. The transmission data may be the sensing data itself, or may be visualized data, such as an image, generated by the control device 180 based on the sensing data. Data communication from the communication device 190 to the remote device 400 may be performed via a mobile communication network, such as 3G, LTE, 4G, or 5G. The remote device 400 displays, on the display 430, time-series images (i.e., video) based on the transmission data sequentially transmitted from the communication device 190. As a result, a video showing the status of the environment around the work vehicle 100 is displayed on the display 430. The operator can perform remote monitoring and remote control while viewing the video displayed on the display 430.

[0150] To achieve the above-described remote monitoring and remote control, it is important that data communication from the communication device 190 to the remote device 400 be stable. However, depending on the driving environment of the work vehicle 100, data communication from the communication device 190 to the remote device 400 may be unstable, resulting in a decrease in communication speed (i.e., the amount of data communication per unit time). For example, if there are many other mobile objects or communication devices communicating using the same wireless communication base station around the work vehicle 100, problems such as radio wave interference or crosstalk are likely to occur. As a result, communication between the communication device 190 and the remote device 400 may become unstable or the communication speed may decrease. As smart agriculture becomes more widespread, the number of agricultural machinery and remote devices connected to the network will continue to increase, making problems such as radio wave interference or crosstalk more likely to occur, and communication speed is expected to frequently decrease. Communication speed also decreases when the work vehicle 100 is driving in an area where it is difficult to receive radio waves for communication. If the communication speed from the communication device 190 to the remote device 400 decreases, the image displayed on the display 430 of the remote device 400 may be continuously or temporarily distorted, making remote control difficult.

[0151] Therefore, in this embodiment, when the communication speed from communication device 190 to remote device 400 drops, control device 180 causes at least one of drive device 240 and sensing device 250 of work vehicle 100 to perform a compensating operation to reduce the impact on remote operation caused by degradation of the image displayed on display 430. The compensating operation may include, for example, at least one of the following operations (a) to (d): (a) The movement speed of the work vehicle 100 is reduced to a speed lower than that specified by the steering command transmitted from the remote device 400. (b) The steering angle when the work vehicle 100 changes direction is reduced below the steering angle specified by the steering command. (c) Before the communication device 190 transmits the transmission data to the remote device, the amount of sensing data corresponding to an area with little temporal change in the visualization data (e.g., time-series image data) generated based on the sensing data is reduced. (d) The amount of sensing data output by a sensor that senses the surrounding environment in a direction different from the direction of travel of the work vehicle 100 is reduced to be smaller than the amount of sensing data output by a sensor that senses the surrounding environment in the direction of travel.

[0152] The compensation operation is not limited to the above operations (a) to (d), and may be other operations that support remote control. The compensation operation may be, for example, an operation to suppress changes or deterioration of the displayed image, or an operation to reduce the amount of sensing data corresponding to an area of ​​relatively low importance in the displayed image.

[0153] Control of driving device 240 or sensing device 250 to perform such a compensating operation may be executed by ECU 184 in control device 180. When the communication speed from communication device 190 to remote device 400 decreases, the compensating operation described above can reduce some data that has a relatively small effect on remote control, or reduce changes in the display image generated based on the transmitted data. This reduces the effect on remote control when the communication speed decreases.

[0154] In the autonomous driving mode and the remote control mode, the control device 180 may perform the above control at all times, or may perform the control only when the work vehicle 100 is traveling in a specific area. For example, the control device 180 may cause the drive device 240 or the sensing device 250 of the work vehicle 100 to perform a compensatory operation only when the control device 180 detects that the work vehicle 100 is traveling on a farm road or a field and that the communication speed has decreased. Whether the work vehicle 100 is traveling on a farm road or a field can be determined, for example, by comparing the position of the work vehicle 100 identified by positioning using the GNSS unit 110 or by self-localization estimation using the camera 120 and / or LiDAR sensor 140 with an environmental map that includes position information for the farm road and the field. Alternatively, it is also possible to determine whether the work vehicle 100 is traveling on a farm road or a field by image recognition processing using the camera 120.

[0155] In performing the above control, the control device 180 monitors the communication speed (i.e., bit rate) of data transmitted from the communication device 190 to the remote device 400. A decrease in the communication speed can be detected based on a comparison between the communication speed and a threshold value. For example, the control device 180 may cause the work vehicle 100 or the sensing device 250 to perform a compensatory operation when the communication speed drops below a preset threshold value.

[0156] The compensating operation is not limited to one type, and multiple compensating operations may be performed. Compensating operations may be performed in stages depending on the degree of decrease in communication speed. For example, the control device 180 may control the driving device 240 and / or the sensing device 250 so that a first compensating operation is performed when the communication speed falls below a first threshold, and a second compensating operation different from the first compensating operation is performed when the communication speed falls below a second threshold that is lower than the first threshold. In other words, the control device 180 may control at least one of the driving device 240 and the sensing device 250 so that different compensating operations are performed in stages depending on the degree of decrease in communication speed based on a comparison between the communication speed and multiple thresholds. Such control enables more flexible operation depending on the communication speed. As a result, the impact on remote control of degradation of the display image due to a decrease in communication speed can be more effectively reduced.

[0157] A more specific example of the above control by the control device 180 will be described below.

[0158] Fig. 18 is a flowchart showing an example of a control operation according to a communication speed by the control device 180. In the example of Fig. 18, the above-mentioned compensation operation (a) is performed. The control device 180 executes the operations from steps S201 to S207. The operation of each step will be described below.

[0159] In step S201, the control device 180 acquires sensing data output from the sensing device 250. The sensing data may be, for example, image data output from the camera 120 and / or point cloud data output from the LiDAR sensor 140. The sensing device 250 repeatedly generates and outputs sensing data at a preset cycle. The generated sensing data may be sequentially stored in the storage device 170.

[0160] In step S202, the control device 180 instructs the communication device 190 to transmit transmission data based on the sensing data to the remote device 400. The control device 180 may cause the communication device 190 to transmit the sensing data itself as transmission data. Alternatively, the control device 180 may cause the communication device 190 to transmit visualization data (for example, time-series image data) newly generated based on the sensing data as transmission data.

[0161] In step S203, the control device 180 determines whether the communication device 190 has received a steering command from the remote device 400. If a steering command has been received, the process proceeds to step S204. If a steering command has not been received, the process returns to step S201. The steering command may include, for example, a signal indicating a command value for the travel speed and / or steering angle of the work vehicle 100. The steering angle is the angle at which the steered wheels (e.g., the front wheels) point relative to the neutral direction (i.e., the straight-ahead direction of the work vehicle 100). The steering command may include a signal that specifies not only the travel state of the work vehicle 100 but also the operating state of the implement 300.

[0162] In step S204, the control device 180 determines whether the communication speed from the communication device 190 to the remote device 400 is less than a preset threshold. During communication, the control device 180 monitors the communication speed (i.e., bit rate) of data transmitted from the communication device 190 to the remote device 400 and compares the communication speed with the threshold. For example, if the bit rate of the most recently transmitted data is less than the threshold, the control device 180 can determine that the communication speed has decreased. The threshold may be, for example, 100 kbps, 500 kbps, 1 Mbps, or 5 Mbps. The threshold may be set to an appropriate value depending on various conditions, such as the resolution, frame rate, amount of color information, and type of encoder of the transmitted visualization data. In one example, the threshold may be set to a value within a range from 50 kbps to 10 Mbps. In another example, the threshold may be set to a value within a range from 100 kbps to 5 Mbps. If the communication speed is less than the threshold, the process proceeds to step S205. If the communication speed is equal to or greater than the threshold, the process proceeds to step S206.

[0163] In step S205, the control device 180 causes the drive device 240 of the work vehicle 100 to perform the above-described compensation operation (a). That is, the control device 180 controls the drive device 240 so that the work vehicle 100 travels at a speed lower than the speed specified by the steering command received from the remote device 400. For example, the control device 180 may perform speed control using a value obtained by multiplying the speed command value indicated by the steering command by a positive coefficient smaller than 1 as the target speed. As a result, the work vehicle 100 travels at a speed lower than the speed command value.

[0164] FIG. 19A is a diagram showing an example of the relationship between the speed command value and the actual speed of the work vehicle 100. In the example of FIG. 19A, when the communication speed is equal to or greater than the threshold (communication speed: high), the travel speed of the work vehicle 100 is controlled so that the actual travel speed is equal to the speed command value V0. On the other hand, when the communication speed is less than the threshold (communication speed: low), the actual travel speed of the work vehicle 100 is controlled to a value V0 that is smaller than the speed command value V0. l The running speed is controlled so that

[0165] Instead of comparing the communication speed with one threshold, the control device 180 may compare the communication speed with multiple thresholds and gradually reduce the travel speed of the work vehicle 100 depending on the degree of reduction in the communication speed. For example, as shown in the example of FIG. 19B, when the communication speed is less than the first threshold and equal to or greater than the second threshold (communication speed: medium), the actual speed of the work vehicle 100 is reduced to V, which is lower than the speed command value V0. m When the communication speed is less than the second threshold (communication speed: low), the actual speed of the work vehicle 100 is reduced to speed V m Lower V l may be reduced to

[0166] Instead of comparing the communication speed with a threshold value, the control device 180 may determine a speed limit according to the communication speed based on data such as a pre-created table or function. In this case, the control device 180 controls the speed limit of the work vehicle 100 so that it is lower the lower the communication speed. This allows for more precise speed control according to the communication conditions.

[0167] The compensation operation in step S205 causes the actual speed of the work vehicle 100 to be lower than the speed specified by the steering command. This reduces the movement of the image displayed on the display 430 of the remote device 400, making remote monitoring and remote control easier. Furthermore, when the communication device 190 compresses and transmits video data using, for example, intra-frame prediction and inter-frame prediction, the amount of data transmission can be reduced by reducing the change in the video. This suppresses degradation of the displayed image due to communication delays, making it easier for the operator to grasp the situation around the work vehicle 100 based on the displayed image.

[0168] In step S206, the control device 180 instructs the drive device 240 to travel the work vehicle 100 in accordance with the steering command. That is, if the communication speed is equal to or greater than the threshold, the control device 180 causes the work vehicle 100 to travel at the speed specified by the steering command.

[0169] In step S207, the control device 180 determines whether an instruction to end the operation has been issued. The instruction to end the operation can be sent from the remote device 400 to the communication device 190, for example, when the operator performs an operation to stop remote control or automatic traveling of the work vehicle 100. If an instruction to end the operation has been issued, the control device 180 stops the work vehicle 100 and ends the operation. If an instruction to end the operation has not been issued, the process returns to step S201. Thereafter, the operations of steps S201 to S207 are repeated until an instruction to end the operation is issued.

[0170] Through the above operations, the operator can remotely monitor and remotely control the work vehicle 100 while viewing the image displayed on the display 430 of the remote device 400. In the example of Fig. 18, when the control device 180 detects a decrease in communication speed, it causes the work vehicle 100 to travel at a speed lower than the speed specified by the maneuvering command. This reduces the impact on remote control of degradation of the displayed image caused by a decrease in communication speed.

[0171] Fig. 20 is a flowchart showing another example of the control operation according to the communication speed by the control device 180. In the example of Fig. 20, the above-mentioned compensation operation (b) is performed. The operation shown in Fig. 20 is the same as the operation shown in Fig. 18 except that step S205 is replaced with step S215. Below, differences from the example of Fig. 18 will be described.

[0172] In step S215, the control device 180 causes the drive device 240 of the work vehicle 100 to perform the above-described compensation operation (b). That is, the control device 180 controls the drive device 240 so that the work vehicle 100 travels at a steering angle that is smaller than the steering angle specified by the steering command received from the remote device 400. For example, the control device 180 may perform steering control using a target steering angle that is a value obtained by multiplying the steering angle command value indicated by the steering command by a positive coefficient that is smaller than 1. Alternatively, when a steering command instructing a right or left turn is issued, the control device 180 can perform the above-described control by reducing the gain of the control that changes the steering angle. By reducing the gain, fluctuations in the steering angle are reduced, and changes in the displayed image can be suppressed.

[0173] 21A is a diagram showing an example of the relationship between the steering angle command value and the actual steering angle of the work vehicle 100. In the example of FIG. 21A, when the communication speed is equal to or higher than the threshold (communication speed: high), the steering angle is controlled so that the actual steering angle of the work vehicle 100 is equal to the steering angle command value θ0. On the other hand, when the communication speed is lower than the threshold (communication speed: low), the actual steering angle of the work vehicle 100 is controlled to a value θ smaller than the steering angle command value θ0. l The running speed is controlled so that

[0174] In this example, the control device 180 may also compare the communication speed with multiple thresholds instead of comparing the communication speed with one threshold, and gradually reduce the steering angle of the work vehicle 100 depending on the degree of reduction in the communication speed. For example, as shown in the example in FIG. 21B, when the communication speed is less than the first threshold and equal to or greater than the second threshold (communication speed: medium), the actual steering angle of the work vehicle 100 is reduced to a value θ smaller than the steering angle command value θ0. m When the communication speed is less than the second threshold (communication speed: low), the actual steering angle of the work vehicle 100 is reduced to an angle θ m A value θ smaller than l may be reduced to

[0175] In this example, too, the control device 180 may determine the upper limit of the steering angle according to the communication speed based on data such as a pre-created table or function, instead of comparing the communication speed with a threshold value. In this case, the control device 180 controls the upper limit of the steering angle of the work vehicle 100 so that the lower the communication speed, the lower the steering angle. This allows for more precise steering control according to the communication conditions.

[0176] As described above, the control device 180 may decrease the gain of the steering control as the communication speed decreases. For example, the control device 180 may monotonically decrease the gain of the steering control as the communication speed decreases. In this case, data indicating a table or function that defines the relationship between the measured value of the communication speed and the gain of the steering control may be stored in advance in the storage device 170. The control device 180 may determine the gain of the steering control based on the data and the measured value of the communication speed.

[0177] The compensation operation in step S215 causes the actual steering angle of the work vehicle 100 to be smaller than the steering angle specified by the steering command. As a result, when the work vehicle 100 changes direction, changes in the image displayed on the display 430 of the remote device 400 are reduced, making remote monitoring easier. Furthermore, when the communication device 190 compresses and transmits video data using, for example, intra-frame prediction and inter-frame prediction, the amount of data transmission can be reduced by reducing changes in the video. This suppresses degradation of the displayed image due to communication delays, making it easier for the operator to grasp the situation around the work vehicle 100 based on the displayed video.

[0178] In step S215 shown in Fig. 20, the control device 180 may also perform the operation of step S205 shown in Fig. 18. That is, the control device 180 may reduce both the traveling speed and steering angle of the work vehicle 100 to values ​​less than the command values ​​specified by the steering command. Such a compensating operation can further suppress changes or deterioration in the displayed image when the communication speed drops, making remote control even easier.

[0179] FIG. 22 is a diagram illustrating the ratio (target value / command value) between the value of the control command (command value) during the above-described compensation operation and the target value set by the control device 180 in control. The command value is the above-described speed command value V0 or steering angle command value θ0 or the like. The target value is the above-described speed V l 、V m (<V0), or the angle θ l 、θ m (<θ0) or the like. FIG. 22 shows three examples of how to set the target value according to the communication speed. The control device 180 can execute feedback control (such as PID control or the like) to bring the control amount (for example, traveling speed or steering angle) closer to the target value. When the control device 180 detects a decrease in the communication speed, the control device 180 can realize the above-described compensation operation by setting the target value in the control to be lower than the command value.

[0180] As shown in FIG. 22, when the communication speed is equal to or higher than the threshold value (that is, when the communication speed is high), the control device 180 sets the target value in control to the same value as the command value. In this case, the target value / command value is 1.0. When the communication speed is less than the threshold value (that is, when the communication speed is low), the control device 180 makes the target value in control smaller than the command value. For example, when the communication speed is less than the threshold value, as shown by line L101, the control device 180 may fix the target value to a predetermined value lower than the command value. In this case, the target value / command value is X1. Alternatively, when the communication speed is less than the threshold value, as shown by line L102, the control device 180 may gradually change the target value to a value lower than the command value according to the communication speed. In this case, the target value / command value is any one of X2 to X5 according to the value of the communication speed. Alternatively, when the communication speed is less than the threshold value, as shown by line L103, the control device 180 may continuously change the target value to a value lower than the command value according to the communication speed. In this case, the target value / command value continuously decreases in response to the decrease in the communication speed. ​​22, when the communication speed is equal to or higher than the threshold (i.e., when the communication speed is high), the control device 180 sets the target value to the same value as the command value, and when the communication speed is less than the threshold (i.e., when the communication speed is low), the control device 180 sets the target value to a value smaller than the command value. This makes it possible to reduce the movement of the work vehicle 100 compared to when the target value is the same as the command value, making remote control easier.

[0182] Fig. 23 is a flowchart showing yet another example of the control operation according to the communication speed by the control device 180. In the example of Fig. 23, the above-mentioned compensation operation (c) is performed. The operation shown in Fig. 23 differs from the operation shown in Fig. 18 in that step S222 is added between step S201 and step S202, the process proceeds to step S206 if the determination in step S203 is Yes, the process proceeds to step S204 if the determination in step S203 is No, step S205 is replaced with step S225, and the process proceeds to step S207 if the determination in step S204 is No. The differences from the example of Fig. 18 will be described below.

[0183] In the example of FIG. 23 , after step S201, the control device 180 generates visualization data showing the situation around the work vehicle 100 based on the sensing data (step S222). The visualization data may be, for example, image data based on data output from the camera 120 or the LiDAR sensor 140. The visualization data is not limited to general color image data, and may be, for example, image data that visualizes the position distribution or distance distribution of a point cloud for which ranging has been performed. This visualization data may be transmitted to the remote device 400 by the communication device 190 in the subsequent step S202. Note that the operation of step S222 may be performed between steps S201 and S202 in the examples shown in FIGS. 18 and 20 . In this case, the visualization data generated in step S222 may also be transmitted to the remote device 400 as transmission data in FIG. 18 or step S202 shown in FIG. 18 .

[0184] In the example of FIG. 23, if it is determined in step S203 that a steering command has been received, the control device 180 proceeds to step S206 and controls the drive device 240 to travel in accordance with the steering command.

[0185] If it is determined in step S203 that a steering command has not been received, the control device 180 proceeds to step S204, where it compares the communication speed with a threshold value. If the communication speed is less than the threshold value, it proceeds to step S225. If the communication speed is equal to or greater than the threshold value, it proceeds to step S207.

[0186] In step S225, the control device 180 instructs the sensing device 250, such as the camera 120 or the LiDAR sensor 140, to reduce the amount of sensing data to be output below a specified value. For example, the control device 180 instructs the sensing device 250 to reduce the amount of sensing data corresponding to an area with little temporal change (i.e., little movement) in the visualization data generated in step S222 below a specified value. More specifically, the control device 180 may lower the resolution of sensing data corresponding to an area with little change in the image represented by the visualization data. In the next and subsequent sensing operations, the sensing device 250 outputs sensing data in which the amount of data corresponding to the area with little change has been reduced. As a result, in step S202 from the next time onwards, visualization data based on the sensing data with the reduced amount of data is transmitted.

[0187] 23 is configured to be able to change the amount of sensing data to be output in response to a request from the control device 180. When the communication speed is below a threshold, the sensing device 250 outputs a smaller amount of sensing data than the amount of sensing data output when the communication speed is equal to or higher than the threshold. The sensing device 250 can be controlled to output, for example, the maximum amount of data that can be output while the communication speed is equal to or higher than the threshold, and to output, for example, one-half or one-third of the maximum amount of data when the communication speed falls below the threshold.

[0188] In an image showing the environment around the work vehicle 100, for example, the sky 79 included in the background shown in FIG. 11A is one of the areas that change little between consecutive frames of a group of frames that make up a time-series image. The control device 180 may have a function for detecting areas in the image that change little, such as the sky 79. Before the communication device 190 transmits the sensing data to the remote device 400, the control device 180 performs preprocessing to reduce the amount of sensing data corresponding to areas in the image that change little, such as the sky 79. This preprocessing enables the communication device 190 to efficiently compress the sensing data and reduce the data amount of the video stream.

[0189] The communication device 190 may include an encoder, and the remote device 400 may include a decoder. The encoder of the communication device 190 may encode the sensing data output from the sensing device 250 to generate streaming data. For example, the streaming data may be transmitted to the remote device 400 or to the management device 600 and recorded in the storage device 650. The streaming data recorded in the storage device 650 may be used for purposes other than remote control, such as observing the growth status of crops or pests, which do not require real-time performance. The decoder of the remote device 400 receives and decodes the streaming data to generate a video of the environment around the work vehicle 100. This video is video with reduced degradation or distortion. An operator or user at the remote monitoring center can monitor the autonomous driving of the work vehicle 100 or remotely control the work vehicle 100 using the remote control device 500 while viewing the video displayed on the display 430.

[0190] 23, the amount of sensing data output from the sensing device 250 can be adaptively changed depending on the communication speed of the communication device 190. For example, when the communication speed decreases, the amount of sensing data corresponding to areas with little change in the displayed image can be reduced while maintaining the amount of sensing data corresponding to areas with a lot of change. This makes it possible to reduce the amount of data transmission while maintaining information on areas important for remote monitoring and remote control. As a result, it is possible to reduce the impact on remote control of degradation of the displayed image caused by a decrease in communication speed.

[0191] Fig. 24 is a diagram showing an example of the change over time in the communication speed of the communication device 190 and the change over time in the amount of sensing data output from the sensing device 250. The upper graph in Fig. 24 shows an example of the change over time in the communication speed. The lower graph in Fig. 24 shows an example of the change over time in the amount of sensing data output. The horizontal axis represents time, and the vertical axis represents the communication speed in the upper graph and the data amount in the lower graph.

[0192] The communication speed may fluctuate due to changes in the driving environment of the work vehicle 100. In the example shown in FIG. 24, a period in which the communication speed is sufficiently high and stable is called a "steady section," and a period in which the communication speed falls below a threshold is called a "decreasing section." The threshold may be determined appropriately depending on the environment in which the system is used. The sensing device 250 is controlled to output sensing data according to a first operational setting during the steady section and to output sensing data according to a second operational setting during the decreasing section. During the steady section, the amount of sensing data output from the sensing device 250 is maintained at a predetermined upper limit (Max value). If the communication speed drops from that state and falls below the threshold, the control device 180 controls the sensing device 250 to perform a compensating operation, for example, to suppress the output of some of the sensing data that is less important. This reduces the amount of sensing data output, and also reduces the amount of transmission data generated based on the sensing data. This suppresses degradation of the monitoring video generated by the remote device 400, thereby reducing the impact on remote control.

[0193] 23, the control device 180 may compare the communication speed with multiple thresholds instead of comparing the communication speed with one threshold, and vary the amount of sensing data reduction depending on the degree of decrease in communication speed. Alternatively, instead of comparing the communication speed with a threshold, the control device 180 may determine the amount of data reduction depending on the communication speed based on data such as a pre-created table or function. In this case, the control device 180 controls the amount of sensing data output from the sensing device 250 so that the amount of data decreases as the communication speed decreases. This enables more precise control depending on the communication conditions.

[0194] The operation shown in Fig. 23 may be combined with the operations shown in either or both of Fig. 18 and Fig. 20. That is, when the control device 180 detects a decrease in the communication speed, it may instruct the sensing device 250 to reduce the amount of sensing data, and may instruct the drive device 240 to reduce at least one of the traveling speed and the steering angle to be smaller than the values ​​specified by the steering command. By combining such multiple types of compensation operations, the effect of assisting remote steering can be further enhanced.

[0195] Next, a specific example of operation will be described in which the sensing device 250 includes a first sensor that senses the surrounding environment in the direction of travel of the work vehicle 100, and a second sensor that senses the surrounding environment in a direction different from the direction of travel.

[0196] FIG. 25A is a diagram schematically illustrating sensing performed by sensing device 250 using first camera 120F, which captures the image in the direction of travel of work vehicle 100, and second camera 120R, which captures the image opposite the direction of travel of work vehicle 100. In this example, first camera 120F corresponds to the first sensor, and second camera 120R corresponds to the second sensor. In this configuration, control device 180 can cause second camera 120R to perform the compensation operation (d) described above. That is, control device 180 can cause second camera 120R to perform a compensation operation to reduce the amount of image data output by second camera 120R to make the amount of image data smaller than the amount of image data output by first camera 120F. Control device 180 can reduce the amount of data by, for example, lowering the resolution of the image output from second camera 120R, reducing the color information of the image, or outputting an image obtained by cropping a portion of the image, such as the center, from the image. 25A, when the communication speed drops, control device 180 may also reduce the amount of image data output from first camera 120F. In this case, however, the reduction rate of the amount of image data output from first camera 120F may be kept lower than the reduction rate of the amount of image data output from second camera 120R.

[0197] FIG. 25B is a diagram showing a modified example of FIG. 25A. In this example, the sensing device 250 includes a first LiDAR sensor 140F that senses the side in the traveling direction of the work vehicle 100, and a second LiDAR sensor 140R that senses the side opposite the traveling direction of the work vehicle 100. In the example of FIG. 25B, the first LiDAR sensor 140F corresponds to the first sensor, and the second LiDAR sensor 140R corresponds to the second sensor. Even in this configuration, the control device 180 can cause the second LiDAR sensor 140R to perform the compensation operation (d) described above. That is, the control device 180 can cause the second LiDAR sensor 140R to perform, as a compensation operation, an operation of reducing the amount of point cloud data output by the second LiDAR sensor 140R to make the amount of data smaller than the amount of point cloud data output by the first LiDAR sensor 140F. The control device 180 can reduce the amount of data by, for example, reducing the number of point clouds in the point cloud data output from the second LiDAR sensor 140R or narrowing the scanning range. In the example of Fig. 25B, when the communication speed decreases, the control device 180 may also reduce the amount of point cloud data output from the first LiDAR sensor 140F. In this case, however, the reduction rate of the data amount of the point cloud data output from the first LiDAR sensor 140F may be kept smaller than the reduction rate of the data amount of the point cloud data output from the second LiDAR sensor 140R.

[0198] The operation of the control device 180 in the example shown in Figures 25A and 25B is basically the same as the operation shown in Figure 23. When compensation operation (d) is performed, the control device 180 instructs the sensing device 250 to reduce the amount of sensing data output by the second sensor in step S225. If the sensing device 250 includes three or more sensors, the amount of sensing data may be reduced for sensors other than the sensor that senses the front direction. This further reduces the amount of transmitted data, thereby further reducing degradation of the displayed image.

[0199] As described above, the remote control system in this embodiment comprises a sensing device 250 that senses the environment around the work vehicle 100 (i.e., agricultural machinery) and outputs sensing data, a communication device 190 that transmits transmission data based on the sensing data output from the sensing device 250 to a remote device 400 that transmits control commands to the work vehicle 100, and a control device 180 that causes at least one of the work vehicle 100 and the sensing device 250 to perform compensatory operations to reduce the impact on remote control caused by deterioration of the display image showing the situation around the work vehicle 100 based on the transmission data received by the remote device 400 when the communication speed from the communication device 190 to the remote device 400 decreases.

[0200] The above configuration can solve the problem of remote control becoming difficult due to degradation of the display image caused by a decrease in communication speed.

[0201] The control device 180 may cause the work vehicle 100 to perform a compensating operation to reduce the movement speed of the work vehicle 100 below the speed specified by the steering command. This makes it possible to suppress changes in the display image that accompany the movement of the work vehicle 100, making remote steering easier.

[0202] The control device 180 may cause the work vehicle 100 to execute a compensating operation to reduce the steering angle when the work vehicle 100 changes direction below the steering angle specified by the steering command. This makes it possible to suppress changes in the displayed image when the work vehicle 100 changes direction, making remote control easier.

[0203] The control device 180 may generate visualization data that visualizes the environment around the work vehicle 100 based on the sensing data, and may cause the sensing device 250 to execute a compensating operation to reduce the amount of sensing data corresponding to areas of the visualization data that change little over time before the communication device 190 transmits the transmission data to the remote device 400. This makes it possible to reduce data in areas that are relatively less important and change little, thereby suppressing degradation of the displayed image due to a decrease in communication speed.

[0204] The sensing device 250 may include a first sensor that senses the surrounding environment in the traveling direction of the work vehicle 100, and a second sensor that senses the surrounding environment in a direction different from the traveling direction of the work vehicle 100. The control device 180 may cause the second sensor to perform a compensating operation to reduce the amount of sensing data output by the second sensor so that the amount of sensing data output by the second sensor is smaller than the amount of sensing data output by the first sensor. This makes it possible to reduce the amount of sensing data that indicates conditions in a direction different from the traveling direction, which is relatively less important, and thereby suppress degradation of the display image caused by a decrease in communication speed.

[0205] When the control device 180 detects that the work vehicle 100 is traveling on a farm road or in a field and that the communication speed has decreased, it may cause the sensing device 250 and / or the work vehicle 100 to perform a compensating operation. This makes it possible to assist remote control by performing a compensating operation when the work vehicle 100 is traveling on a farm road or in a field.

[0206] When the communication speed drops below a threshold, the control device 180 may cause at least one of the work vehicle 100 and the sensing device 250 to perform a compensating operation. Alternatively, the control device 180 may cause at least one of the work vehicle 100 and the sensing device 250 to perform a compensating operation based on data such as a table that defines the correspondence between the communication speed and the compensating operation.

[0207] The control device 180 may cause at least one of the work vehicle 100 and the sensing device 250 to perform a first compensating operation when the communication speed drops below a first threshold, and may cause at least one of the work vehicle 100 and the sensing device 250 to perform a second compensating operation different from the first compensating operation when the communication speed drops below a second threshold that is smaller than the first threshold. This makes it possible to perform appropriate compensating operations in stages depending on the degree of the drop in communication speed.

[0208] The remote control system may further include a remote device 400. Alternatively, the remote device 400 may be an element external to the remote control system. The remote control system may further include a display device 430 for displaying a display image. Alternatively, the display device 430 may be an element external to the remote control system.

[0209] The work vehicle 100 in this embodiment includes a remote control system according to any of the above aspects, and a traveling device controlled by the control device 180.

[0210] A control method according to an embodiment of the present disclosure is used in a remote control system for remotely controlling an agricultural machine. The control method includes the steps of acquiring sensing data generated by a sensing device that senses the environment around the agricultural machine, transmitting transmission data based on the sensing data to a remote device that transmits control commands to the agricultural machine, and, when the communication speed from the communication device to the remote device decreases, causing at least one of the agricultural machine and the sensing device to perform a compensating operation to reduce the impact on the remote control caused by deterioration of a display image showing the situation around the agricultural machine based on the transmission data.

[0211] The configurations and operations of the above-described embodiments are merely examples, and the present disclosure is not limited to the above-described embodiments. For example, other embodiments may be configured by appropriately combining the above-described various embodiments.

[0212] In the above embodiment, the agricultural machine performs automatic driving, but the agricultural machine does not have to have an automatic driving function. The technology of the present disclosure can be widely applied to agricultural machines that can be remotely controlled.

[0213] The system for controlling automatic traveling and / or remote-controlled traveling in the above embodiments can also be retrofitted to an agricultural machine that does not have these functions. Such a system can be manufactured and sold independently of the agricultural machine. The computer program used in such a system can also be manufactured and sold independently of the agricultural machine. The computer program can be provided, for example, by being stored on a computer-readable non-transitory storage medium. The computer program can also be provided by downloading via a telecommunications line (for example, the Internet).

[0214] As described above, the present disclosure includes the agricultural machinery, sensing system, sensing method, remote control system, and control method described in the following items.

[0215] [Item 1] The vehicle body, a sensing device that senses the environment around the vehicle body and outputs sensing data; a communication device that transmits the sensing data output from the sensing device to a remote device; An agricultural machine equipped with The agricultural machine, wherein the sensing device changes the amount of the sensing data to be output to the communication device in accordance with a communication speed from the communication device to the remote device.

[0216] [Item 2] Item 2. The agricultural machine according to item 1, wherein, when the communication speed is less than a threshold, the sensing device outputs a smaller amount of sensing data than the amount of sensing data output when the communication speed is equal to or greater than the threshold.

[0217] [Item 3] the sensing device includes a camera; 3. The agricultural machine according to item 1 or 2, wherein when the communication speed is below a threshold, the camera reduces at least one of the resolution, the amount of color information, the parallax, and the image size when capturing an image, compared to when the communication speed is equal to or higher than the threshold.

[0218] [Item 4] the sensing device includes a LiDAR sensor; 4. The agricultural machine according to any one of items 1 to 3, wherein when the communication speed is less than a threshold, the LiDAR sensor reduces at least one of the amount of point cloud data output per unit time and the angle range of sensing, compared to when the communication speed is equal to or greater than the threshold.

[0219] [Item 5] the sensing device includes an obstacle sensor that detects an obstacle present in the environment around the vehicle body; 5. The agricultural machine according to any one of items 1 to 4, wherein, when the obstacle sensor does not detect the obstacle, it outputs sensing data with an amount of data smaller than the amount of sensing data that is output when the obstacle is detected.

[0220] [Item 6] 6. The agricultural machine according to any one of items 1 to 5, wherein the communication device reduces a data amount of the sensing data corresponding to a region with little change in the time-series image, which visualizes an environment around the vehicle body and is generated based on the sensing data, before transmitting the sensing data to the remote device.

[0221] [Item 7] 7. The agricultural machine according to any one of items 1 to 6, further comprising a control device that controls the traveling of the vehicle body based on a signal transmitted from the remote device.

[0222] [Item 8] a sensing device that senses the environment around the agricultural machine and outputs sensing data; a communication device that transmits the sensing data output from the sensing device to a remote device; Equipped with the sensing device changes the amount of the sensing data to be output to the communication device in accordance with a communication speed from the communication device to the remote device. Sensing system.

[0223] [Item 9] a step of acquiring sensing data from a sensing device that senses an environment around the agricultural machine and outputs the sensing data; transmitting the sensing data from a communication device to a remote device; causing the sensing device to change the amount of the sensing data in accordance with a communication speed from the communication device to the remote device; A sensing method comprising:

[0224] [Item 10] A remote control system for agricultural machinery, comprising: a sensing device that senses the environment around the agricultural machine and outputs sensing data; a communication device that transmits transmission data based on the sensing data output from the sensing device to a remote device that transmits an operation command to the agricultural machine; a control device that causes at least one of the agricultural machine and the sensing device to execute a compensating operation to reduce an impact on the remote operation caused by deterioration of a display image that shows the situation around the agricultural machine based on the transmission data received by the remote device when the communication speed from the communication device to the remote device decreases; A remote control system comprising:

[0225] [Item 11] Item 11. The remote control system according to item 10, wherein the control device causes the agricultural machine to perform the compensating operation of reducing the moving speed of the agricultural machine below the speed specified by the steering command.

[0226] [Item 12] Item 12. The remote control system according to item 10 or 11, wherein the control device causes the agricultural machine to perform, as the compensating operation, an operation of reducing the steering angle when the agricultural machine changes direction to be less than the steering angle specified by the steering command.

[0227] [Item 13] The control device generating visualization data for visualizing an environment around the agricultural machine based on the sensing data; 13. The remote control system according to any one of items 10 to 12, wherein, before the communication device transmits the transmission data to the remote device, the communication device causes the sensing device to perform the compensation operation of reducing the amount of the sensing data corresponding to an area of ​​the visualization data that changes little over time.

[0228] [Item 14] The sensing device is a first sensor that senses the surrounding environment in the direction of travel of the agricultural machine; a second sensor that senses the surrounding environment in a direction different from the traveling direction of the agricultural machine, 14. The remote control system according to any one of items 10 to 13, wherein the control device causes the second sensor to perform the compensating operation of reducing the amount of sensing data output by the second sensor so that the amount of sensing data output by the second sensor is smaller than the amount of sensing data output by the first sensor.

[0229] [Item 15] 15. The remote control system according to any one of items 10 to 14, wherein the control device causes at least one of the sensing device and the agricultural machine to perform the compensating operation when the control device detects that the agricultural machine is traveling on a farm road or a field and that the communication speed has decreased.

[0230] [Item 16] 16. The remote control system according to any one of items 10 to 15, wherein the control device causes at least one of the agricultural machine and the sensing device to perform the compensating operation when the communication speed drops below a threshold.

[0231] [Item 17] The control device when the communication speed falls below a first threshold, causing at least one of the agricultural machine and the sensing device to perform a first compensating operation; 17. The remote control system according to any one of items 10 to 16, wherein, when the communication speed drops below a second threshold value that is lower than the first threshold value, at least one of the agricultural machine and the sensing device is caused to perform a second compensating operation that is different from the first compensating operation.

[0232] [Item 18] 18. The remote control system of any one of items 10 to 17, further comprising the remote device.

[0233] [Item 19] 19. The remote control system of any one of items 10 to 18, further comprising a display device that displays the display image.

[0234] [Item 20] A remote control system according to any one of items 10 to 19; a traveling device controlled by the control device; Agricultural machinery equipped with:

[0235] [Item 21] A control method used in a remote control system for remotely controlling an agricultural machine, comprising: acquiring sensing data generated by a sensing device that senses an environment around the agricultural machine; transmitting transmission data based on the sensing data to a remote device that transmits a steering command to the agricultural machine; a step of causing at least one of the agricultural machine and the sensing device to perform a compensating operation to reduce an effect on the remote operation caused by deterioration of a display image showing the situation around the agricultural machine based on the transmitted data when the communication speed from the communication device to the remote device decreases; A control method comprising: [Industrial Applicability]

[0236] The technology disclosed herein can be applied to control systems for autonomous agricultural machinery or agricultural machinery that can be operated by remote control, such as tractors, harvesters, rice transplanters, riding tillers, vegetable transplanters, mowers, seed sowing machines, fertilizer applicators, agricultural drones, or agricultural robots. [Explanation of symbols]

[0237] 50 GNSS satellite, 60 Reference station, 70 Field, 71 Entrance / exit, 72 Work area, 74 Headland, 76 Road, 79 Sky, 80 Network, 90 Storage area, 96 Waiting area, 100 Work vehicle, 101 Vehicle body, 102 Prime mover (engine), 103 Transmission, 104 Wheels, 1 05···Cabin, 106···Steering device, 107···Driver's seat, 108···Coupling device, 110···Positioning device, 111···GNSS receiver, 112···RTK receiver, 115···Inertial measurement unit (IMU), 116···Processing circuit, 120···Camera, 130···Obstacle sensor, 140···LiDAR sensor, 150···Sensor group, 152···Steering wheel steering sensor, 154, turning angle sensor, 156, rotation sensor, 160, control system, 170, storage device, 180, control device, 181-186, ECU, 190, communication device, 200, operation terminal, 210, operation switch group, 220, buzzer, 240, drive device, 300, implement, 340, drive device, 380, control device, 3 90 communication device, 400 remote device, 420 input device, 430 display device, 450 storage device, 460 processor, 470 ROM, 480 RAM, 490 communication device, 500 remote control device, 600 management device, 660 processor, 670 storage device, 670 ROM, 680 RAM, 690 communication device

Claims

1. A remote control system for agricultural machinery, comprising: a sensing device that senses the environment around the agricultural machine and outputs sensing data; a communication device that transmits transmission data based on the sensing data output from the sensing device to a remote device that transmits an operation command to the agricultural machine; a control device that causes at least one of the agricultural machine and the sensing device to execute a compensating operation to reduce an impact on the remote operation caused by deterioration of a display image that shows the situation around the agricultural machine based on the transmission data received by the remote device when the communication speed from the communication device to the remote device decreases; Equipped with The control device causes the agricultural machine to perform the compensating operation of reducing the steering angle when the agricultural machine changes direction below the steering angle specified by the steering command.

2. The remote control system according to claim 1 , wherein the control device causes the agricultural machine to perform the compensating operation by reducing the moving speed of the agricultural machine below a speed specified by the steering command.

3. The control device generating visualization data for visualizing an environment around the agricultural machine based on the sensing data; 3. The remote control system according to claim 1, wherein before the communication device transmits the transmission data to the remote device, the sensing device is caused to perform the compensating operation of reducing the amount of the sensing data corresponding to an area in the visualization data that changes little over time.

4. The sensing device is a first sensor that senses the surrounding environment in the direction of travel of the agricultural machine; a second sensor that senses the surrounding environment in a direction different from the traveling direction of the agricultural machine, 3. The remote control system according to claim 1, wherein the control device causes the second sensor to perform the compensating operation of reducing the amount of sensing data output by the second sensor so that the amount of sensing data output by the second sensor is smaller than the amount of sensing data output by the first sensor.

5. 3. The remote control system according to claim 1, wherein the control device causes at least one of the sensing device and the agricultural machine to perform the compensating operation when the control device detects that the agricultural machine is traveling on a farm road or a field and that the communication speed has decreased.

6. The remote control system according to claim 1 or 2, wherein the control device causes at least one of the agricultural machine and the sensing device to perform the compensating operation when the communication speed drops below a threshold value.

7. The control device when the communication speed drops below a first threshold, causing at least one of the agricultural machine and the sensing device to perform a first compensating operation; 3. The remote control system according to claim 1, wherein, when the communication speed drops below a second threshold value that is lower than the first threshold value, at least one of the agricultural machine and the sensing device is caused to perform a second compensating operation that is different from the first compensating operation.

8. The remote control system of claim 1 or 2, further comprising the remote device.

9. The remote control system according to claim 1 or 2, further comprising a display device that displays the display image.

10. The remote control system according to claim 1 or 2; a traveling device controlled by the control device; Agricultural machinery equipped with:

11. A control method used in a remote control system for remotely controlling an agricultural machine, comprising: acquiring sensing data generated by a sensing device that senses an environment around the agricultural machine; transmitting transmission data based on the sensing data to a remote device that transmits a steering command to the agricultural machine; a step of causing at least one of the agricultural machine and the sensing device to perform a compensating operation to reduce an impact on the remote operation caused by deterioration of a display image showing the situation around the agricultural machine based on the transmitted data when a communication speed from a communication device to the remote device decreases; Including, a control method in which the step of executing the compensating operation includes causing the agricultural machine to execute, as the compensating operation, an operation of reducing the steering angle when the agricultural machine changes direction to be lower than the steering angle specified by the steering command.

Images