Map generation system and map generation method

Abstract

This map generation system generates map data for an agricultural machine that travels autonomously on a road around a field. The map generation system comprises: a storage device for storing a plurality of geographical feature block images associated with each of a plurality of types of geographical features that can exist around the road; and a processing device. The processing device performs the following: acquires position distribution data relating to one or more types of geographical features present around the road, said position distribution data being generated on the basis of at least one of image data from an imaging device and sensor data from a LiDAR sensor output while a moving body equipped with at least one of the LiDAR sensor and the imaging device moves along the road; reads one or more types of geographical block images associated with the one or more types of geographical features from the storage device; and arranges the geographical feature block images in accordance with the position distribution data and generates map data of an area including the road.

Description

[Technical Field]

[0001] The present disclosure relates to a map generation system and a map generation method. [Background technology]

[0002] Research and development is underway to automate agricultural machinery used in fields. For example, work vehicles such as tractors, combine harvesters, and rice transplanters that can navigate autonomously within fields using positioning systems such as the Global Navigation Satellite System (GNSS) have been put to practical use. Research and development is also underway on work vehicles that can navigate autonomously not only within fields but also outside of them.

[0003] Patent Documents 1 and 2 disclose examples of a system in which an unmanned work vehicle automatically travels between two farm fields separated by a road.

[0004] Meanwhile, development is also underway for autonomously moving vehicles that use distance sensors such as LiDAR (Light Detection and Ranging). For example, Patent Document 3 discloses an example of a work vehicle that uses LiDAR to automatically navigate between rows of crops. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2021-073602 [Patent Document 2] Patent Publication No. 2021-029218 [Patent Document 3] Japanese Patent Application Publication No. 2019-154379 Summary of the Invention [Problem to be solved by the invention]

[0006] In order for agricultural machinery to automatically travel not only within a field but also on roads outside the field (for example, farm roads or public roads), a map of the outside of the field is required. Such a map can be used to generate routes for the automatic travel of the agricultural machinery and to display the position of the agricultural machinery while it is traveling automatically.

[0007] The present disclosure provides a system and method for efficiently generating map data for an agricultural machine that navigates autonomously within an environment that includes a field and surrounding roads. [Means for solving the problem]

[0008] A map generation system according to one aspect of the present disclosure generates map data for an agricultural machine that automatically travels along roads around a field. The map generation system includes a storage device that stores multiple feature block images associated with multiple types of features that may exist around the road, and a processing device. The processing device acquires position distribution data for one or more types of features that exist around the road. The feature position distribution data is generated based on at least one of sensor data from a LiDAR sensor and image data from an imaging device, output by a mobile object equipped with at least one of a LiDAR sensor and an imaging device while the mobile object is moving along the road. The processing device reads one or more types of feature block images associated with the one or more types of features from the storage device, and arranges the feature block images according to the position distribution data to generate map data for an area including the road.

[0009] A general or specific aspect of the present disclosure may be realized by an apparatus, a system, a method, an integrated circuit, a computer program, or a computer-readable non-transitory storage medium, or any combination thereof. The computer-readable storage medium may include a volatile storage medium or a non-volatile storage medium. An apparatus may be composed of multiple devices. When an apparatus is composed of two or more devices, the two or more devices may be located in a single device or may be located separately in two or more separate devices. [Effects of the Invention]

[0010] According to an embodiment of the present disclosure, it is possible to efficiently generate map data for an agricultural machine that automatically navigates within an environment including a field and roads in the surrounding area. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram illustrating an example of the configuration of a map generation system. [Figure 2A] FIG. 10 is a diagram showing an example of a feature block image. [Figure 2B] FIG. 10 is a diagram showing an example of a screen for setting the size of each feature block image. [Figure 3A] 1 is a flowchart showing the basic operation of the processing device. [Figure 3B] FIG. 10 is a diagram illustrating an example of the relationship between feature position distribution data and map data. [Figure 4] FIG. 10 is a diagram illustrating an example of a portion of a map generated by a processing device. [Figure 5] FIG. 10 is a diagram illustrating another example of the configuration of the map generation system. [Figure 6] FIG. 1 is a diagram schematically illustrating a state in which a mobile object performs sensing while traveling on a farm road around a farm field. [Figure 7] 10 is a flowchart illustrating an example of a process for generating feature position distribution data. [Figure 8] FIG. 1 is a diagram for explaining an overview of an agricultural management system according to an exemplary embodiment of the present disclosure. [Figure 9] 1 is a side view schematically showing an example of a work vehicle and an implement coupled to the work vehicle. [Figure 10] FIG. 2 is a block diagram showing an example of the configuration of a work vehicle and an implement. [Figure 11] FIG. 1 is a conceptual diagram showing an example of a work vehicle that performs positioning using RTK-GNSS. [Figure 12] 3A and 3B are diagrams illustrating an example of an operation terminal and an operation switch group provided inside a cabin. [Figure 13] FIG. 2 is a block diagram illustrating an example of the hardware configuration of a management device and a terminal device. [Figure 14] FIG. 1 is a diagram schematically illustrating an example of a work vehicle that automatically travels along a target route in a farm field. [Figure 15] 10 is a flowchart illustrating an example of the operation of steering control during automatic driving. [Figure 16A] 1 is a diagram showing an example of a work vehicle traveling along a target route P. FIG. [Figure 16B] FIG. 10 is a diagram showing an example of a work vehicle at a position shifted to the right from the target route P. [Figure 16C] FIG. 10 is a diagram showing an example of a work vehicle at a position shifted to the left from a target route P. [Figure 16D] 10 is a diagram showing an example of a work vehicle facing in a direction inclined with respect to a target route P. FIG. [Figure 17] FIG. 1 is a diagram schematically illustrating an example of a situation in which a plurality of work vehicles are automatically traveling on roads inside and outside a farm field. [Figure 18] FIG. 10 is a diagram showing an example of a farm work schedule created by the management device. [Figure 19] FIG. 2 is a diagram showing an example of a map to be referred to when planning a route. [Figure 20] FIG. 1 is a diagram illustrating an example of a global path. [Figure 21] FIG. 10 is a diagram illustrating an example of a global path and a local path generated in an environment where obstacles are present. DETAILED DESCRIPTION OF THE INVENTION

[0012] (Definition of terms) In this disclosure, "agricultural machinery" refers to machinery used for agricultural purposes. Examples of agricultural machinery include tractors, harvesters, rice transplanters, riding cultivators, vegetable transplanters, mowers, seed sowing machines, fertilizer applicators, and agricultural mobile robots. Not only can a work vehicle such as a tractor function alone as an "agricultural machinery," but the entire work vehicle and an implement attached to or towed by the work vehicle can also function as a single "agricultural machinery." Agricultural machinery performs agricultural work on the ground in a field, such as plowing, sowing, pest control, fertilizing, planting crops, or harvesting. These agricultural works are sometimes referred to as "ground work" or simply "work." Traveling while performing agricultural work by a vehicle-type agricultural machine is sometimes referred to as "work travel."

[0013] "Autonomous driving" refers to controlling the movement of an agricultural machine through the action of a control device, without manual operation by a driver. Agricultural machines that perform autonomous driving are sometimes called "autonomous agricultural machines" or "robotic agricultural machines." During autonomous driving, not only the movement of the agricultural machine but also the agricultural work operations (e.g., the operation of the implements) may be automatically controlled. When the agricultural machine is a vehicle-type machine, the movement of the agricultural machine through autonomous driving is referred to as "autonomous driving." The control device may control at least one of the steering, speed adjustment, and start and stop of movement required for the movement of the agricultural machine. When controlling a work vehicle equipped with implements, the control device may control operations such as raising and lowering the implements and starting and stopping their operation. Autonomous driving movement includes not only movement of the agricultural machine toward a destination along a predetermined route, but also movement of the agricultural machine following a tracking target. An autonomously driving agricultural machine may move partially based on user instructions. Furthermore, an autonomously driving agricultural machine may operate in a manual driving mode, in which it moves through manual operation by the driver, in addition to an autonomous driving mode. Steering an agricultural machine by the action of a control device, without manual operation, is called "automatic steering." Part or all of the control device may be external to the agricultural machine. Control signals, commands, data, and the like may be communicated between the agricultural machine and a control device external to the agricultural machine. An agricultural machine that performs automatic driving may move autonomously while sensing the surrounding environment, without a human being being involved in controlling the movement of the agricultural machine. An agricultural machine capable of autonomous movement can travel unmanned within a field or outside a field (e.g., on a road). During autonomous movement, the machine may detect obstacles and take action to avoid them.

[0014] A "work plan" is data that schedules one or more agricultural tasks to be performed by an agricultural machine. The work plan may include, for example, information indicating the order of agricultural tasks to be performed by the agricultural machine and the field on which each task will be performed. The work plan may also include information on the scheduled date and time for each task. A work plan that includes information on the scheduled date and time for each task is particularly referred to as a "work schedule" or simply a "schedule." The work schedule may include information on the scheduled start and / or end times for each task performed on each work day. The work plan or work schedule may include information for each task, such as the content of the task, the implements to be used, and / or the type and amount of agricultural materials to be used. Here, "agricultural materials" refers to materials used in agricultural tasks performed by an agricultural machine. Agricultural materials may also be simply referred to as "materials." Agricultural materials may include materials consumed in agricultural tasks, such as pesticides, fertilizers, seeds, or seedlings. The work plan may be created by a processing device that communicates with the agricultural machine to manage the agricultural task, or a processing device installed on the agricultural machine. The processing device can create a work plan based on information input by a user (such as a farm manager or farm worker) by operating a terminal device, for example. In this specification, a processing device that communicates with agricultural machines and manages agricultural work is referred to as a "management device." The management device may manage the agricultural work of multiple agricultural machines. In this case, the management device may create a work plan that includes information about each agricultural work to be performed by each of the multiple agricultural machines. The work plan can be downloaded by each agricultural machine and stored in a storage device. Each agricultural machine can automatically head to a field and perform the scheduled agricultural work in accordance with the work plan.

[0015] An "environmental map" is data that represents the positions or areas of objects in the environment in which the agricultural machine moves using a specified coordinate system. An environmental map may be simply referred to as a "map" or "map data." The coordinate system that defines the environmental map may be, for example, a world coordinate system such as a geographic coordinate system fixed relative to the Earth. An environmental map may also include information other than the positions of objects in the environment (e.g., attribute information and other information). Environmental maps include maps in various formats, such as point cloud maps or grid maps. Data for local or partial maps that are generated or processed in the process of constructing an environmental map are also referred to as a "map" or "map data."

[0016] A "feature" means something that exists on the ground. Examples of features include grass, trees, roads, fields, waterways, ditches, rivers, bridges, forests, mountains, rocks, buildings, and railroad tracks. Things that do not exist in the real world, such as boundary lines, place names, building names, field names, and road names, are not included in the "feature" in this disclosure.

[0017] A "feature block image" refers to an image of a predetermined size that represents a feature, and the data of that image. A feature block image may be an image that simply represents a feature. A feature block image may be, for example, a low-resolution bitmap image (i.e., a raster image) such as a pixel art, or a relatively small vector image. A feature block image may also be a color image or a monochrome image. A feature block image is not limited to two-dimensional data, but may also be three-dimensional data that includes height information. By arranging one or more types of feature block images, a simple environmental map can be generated that represents the environment around the road on which agricultural machinery travels. An environmental map generated by arranging feature block images is not limited to two-dimensional data, but may also be three-dimensional data that includes height information.

[0018] A "global path" refers to data on a path connecting a starting point to a destination point when an agricultural machine moves automatically, generated by a processing device that performs path planning. Generating a global path is called global path planning or global path design. In the following description, the global path is also called a "target path" or simply a "path." A global path can be defined, for example, by the coordinate values ​​of multiple points through which the agricultural machine must pass. A point through which the agricultural machine must pass is called a "waypoint," and a line segment connecting adjacent waypoints is called a "link."

[0019] A "local path" refers to a local path that can avoid obstacles and is generated sequentially when an agricultural machine automatically moves along a global path. Generating a local path is called local path planning or local path design. A local path is generated sequentially based on data acquired by one or more sensing devices equipped on the agricultural machine while the agricultural machine is moving. A local path may be defined by multiple waypoints along a portion of the global path. However, if an obstacle exists near the global path, waypoints may be set to bypass the obstacle. The length of the link between waypoints on a local path is shorter than the length of the link between waypoints on the global path. The device that generates the local path may be the same as or different from the device that generates the global path. For example, a management device that manages farm work by an agricultural machine may generate the global path, and a control device installed on the agricultural machine may generate the local path. In this case, the combination of the management device and the control device functions as a "processing device" that performs path planning. The agricultural machine's controller may function as a processor for both global and local path planning.

[0020] "Farm road" means a road that is primarily used for agricultural purposes. Farm roads are not limited to roads paved with asphalt, but also include unpaved roads covered with dirt or gravel. Farm roads include roads (including private roads) that are exclusively passable by vehicle-type agricultural machinery (for example, work vehicles such as tractors) and roads that are also passable by general vehicles (passenger cars, trucks, buses, etc.). Work vehicles may automatically travel on general roads in addition to farm roads. General roads are roads that have been developed for the traffic of general vehicles.

[0021] (Embodiment) Hereinafter, embodiments of the present disclosure will be described. However, more detailed descriptions than necessary may be omitted. For example, detailed descriptions of well-known matters and redundant descriptions of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the inventors provide the accompanying drawings and the following description to enable those skilled in the art to fully understand the present disclosure, and do not intend for them to limit the subject matter described in the claims. In the following description, components having the same or similar functions are designated by the same reference numerals.

[0022] The following embodiments are examples, and the technology of the present disclosure is not limited to the following embodiments. For example, the numerical values, shapes, materials, steps, step order, display screen layout, etc. shown in the following embodiments are merely examples, and various modifications are possible as long as no technical contradiction occurs. Furthermore, one aspect can be combined with another aspect as long as no technical contradiction occurs.

[0023] <Embodiment of Map Generation System> An example configuration of a map generation system according to an exemplary embodiment of the present disclosure will be described.

[0024] 1 is a block diagram showing an example of the configuration of a map generation system 1000. The map generation system 1000 is a system that generates map data for agricultural machines (e.g., work vehicles such as tractors) that automatically travel on roads around the fields, in addition to the fields themselves. The map generation system 1000 includes a storage device 10 and a processing device 20.

[0025] The storage device 10 is a device that includes one or more storage media, such as a semiconductor storage medium, a magnetic storage medium, or an optical storage medium. The storage device 10 stores one or more types of feature block images associated with one or more types of features that may exist around a road (e.g., a farm road or a public road) on which an agricultural machine travels. The feature block images may be images that simply represent features that may exist around the road (e.g., grass, trees, ditches, waterways, bridges, fields, buildings, mountains, forests, etc.). The storage device 10 may also store feature block images associated with roads (hereinafter also referred to as "road block images"). The feature block images may be bitmap images with relatively low resolution, such as pixel art. The feature block images are not limited to bitmap images, and may also be images in other formats, such as vector images.

[0026] FIG. 2A is a diagram showing an example of a feature block image. In this example, the storage device 10 stores multiple dot pictures representing multiple types of features as multiple feature block images. FIG. 2A illustrates eight types of feature block images: a field, grass, a road, a tree, a waterway, a ditch, a house, and a building. Each feature block image shown in FIG. 2A is a dot picture of a relatively small size (for example, approximately 32 dots x 32 dots to 256 dots x 256 dots). Such feature block images may be created in advance and stored in the storage device 10. The processing device 20 can generate a map of the environment in which the agricultural machine will automatically travel by arranging these feature block images. The number of feature block images is not limited to the eight types shown in the figure, and may be seven or fewer types, or nine or more types. Only one type of feature block image may be used. For example, only one type of feature block image representing a road or one type of feature block image representing a feature other than a road may be stored in the storage device 10. Even in this case, the processing device 20 can generate a simple map that makes it possible to distinguish between roads on which agricultural machines travel and other features.

[0027] The size (e.g., width and height) of each feature block image may be set to a fixed value in advance, or may be set manually by a system administrator or a user of the agricultural machine. For example, the administrator or user may be able to set the size of each feature block image using a terminal device that can communicate with the processing device 20.

[0028] Fig. 2B shows an example of a screen for setting the size of each feature block image. The setting screen shown in Fig. 2B is an example of a graphical user interface (GUI) for setting the width W (i.e., horizontal length) and height H (i.e., vertical length) of each of a plurality of feature block images. As in this example, the processing device 20 can display the GUI for setting the width and height of each feature block image on the display of the terminal device.

[0029] Using a setting screen such as that shown in FIG. 2B, it is possible to set the width W and height H of each of a plurality of feature block images (including road block images) as corresponding to a length in the real world (e.g., meters). By operating a terminal device, an administrator or user can set the width W and height H of each of a plurality of feature block images, such as fields, grass, roads, trees, waterways, ditches, houses, and buildings. Once the width W and height H of each of a plurality of feature block images have been set, each feature block image is associated with the set width W and height H and stored in the storage device 10. The processing device 20 generates map data by arranging one or more types of feature block images having the width W and height H set on the GUI.

[0030] In the example shown in FIG. 2B, the aspect ratio of a feature block image can be automatically changed according to the settings of the width W and height H. The aspect ratio is expressed, for example, as a value of "width W:height H." If the width W and height H are the same value, the aspect ratio of the feature block image is set to "1:1." On the other hand, if the width W is 1.0 m and the height H is 2.0 m, the aspect ratio of the feature block image is set to "1:2." The aspect ratio of the feature block image may be set by a terminal device that displays a setting screen, or by the processing device 20.

[0031] In the example of Fig. 2B, one feature block image is prepared for each type of feature, and the width W and height H of each feature block image are individually set. However, the present invention is not limited to this example, and for example, multiple feature block images may be prepared for the same type of feature, and the width W and height H may be set for each of these feature block images.

[0032] As in this embodiment, by providing a function for setting the size of each feature block image depending on the type of feature, it is possible to generate a map that more accurately reflects the actual arrangement of features.

[0033] Instead of providing a function to set the size of each feature block image, the width W and height H of each feature block image may be fixed to lengths according to the type of the corresponding feature. Even in this case, it is possible to generate a suitable map by the map generation process described later.

[0034] The processing device 20 may be a computer including one or more processors and one or more memories. The processing device 20 performs an operation of generating map data by having the processor execute a computer program stored in the memory. The processing device 20 may also be a collection of multiple computers. For example, the processing device 20 may be configured by multiple computers connected to each other so as to be able to communicate with each other via a network such as the Internet.

[0035] Fig. 3A is a flowchart showing the basic operation of the processing device 20. The processing device 20 can generate map data of the environment in which the agricultural machine travels by executing the operations of steps S11 to S13 shown in Fig. 3A. The operation of each step will be described below.

[0036] (Step S11) The processing device 20 acquires position distribution data of one or more types of features present around the road on which the agricultural machine travels. The position distribution data of the features can be generated based on at least one of sensor data from a LiDAR sensor and image data from an imaging device output while a mobile object equipped with at least one of a LiDAR sensor and an imaging device travels along the road. The mobile object may be a work vehicle similar to the above-mentioned agricultural machine, or may be a mobile object different from the agricultural machine (e.g., a drone, i.e., an unmanned aerial vehicle (UAV)). The position distribution data of the features can be data indicating which features are present at which positions in the environment in which the agricultural machine travels. The position distribution data of the features can be expressed, for example, by a combination of a position (e.g., latitude and longitude) in a global coordinate system (e.g., a geographic coordinate system) fixed to the Earth and an identifier indicating the type of feature at that position.

[0037] The feature location distribution data may be generated by the processing device 20 itself or by another device. For example, a computer installed in the mobile object or another device, such as a server computer on the cloud, may generate the feature location distribution data and transmit it to the processing device 20. The processing device 20 or another device may recognize one or more types of features existing around the road based on at least one of sensor data and image data, and generate location distribution data for the recognized one or more types of features. The recognized features may be, for example, at least one of grass, trees, ditches, waterways, fields, and buildings. The processing device 20 or another device may generate the location distribution data for the recognized features based on GNSS data output from a GNSS receiver installed in the mobile object and sensor data. A more specific example of the processing of step S11 will be described later.

[0038] (Step S12) The processing device 20 reads out one or more types of feature block images associated with one or more types of features in the feature position distribution data acquired in step S11 from the storage device 10. For example, if the feature position distribution data includes position information of eight types of features exemplified in Fig. 2A, the processing device 20 reads out feature block images corresponding to those features.

[0039] (Step S13) The processing device 20 arranges the feature block images read out in step S12 according to the feature position distribution data acquired in step S11, and generates map data for an area including a road on which the agricultural machine travels. More specifically, the processing device 20 arranges one or more types of feature block images associated with one or more recognized types of features within an area in the map data where the features exist, according to the feature position distribution data, to generate map data. For example, the processing device 20 can generate map data by executing a process including the following steps S1 to S5 for each of one or more types of features. (S1) Based on the location distribution data, the horizontal and vertical lengths of the continuous area occupied by the feature are determined. Here, the horizontal direction corresponds to the width direction of the feature block image, and the vertical direction corresponds to the height direction of the feature block image. (S2) The horizontal length is divided by the width of the feature block image corresponding to the feature to determine the horizontal number of feature block images. (S3) The vertical number of feature block images is determined by dividing the vertical length by the height of the feature block image corresponding to that feature. (S4) The feature block images are arranged horizontally by the number of horizontal rows. (S5) The feature block images are arranged vertically in the same number as the vertical number.

[0040] The above steps (S1) to (S5) are preferably applied when the planar shape of the feature can be approximated by a rectangle. If the planar shape of the feature cannot be approximated by a rectangle, the area occupied by the feature can be divided into multiple rectangular or strip-shaped areas of different sizes, and the above steps (S1) to (S5) can be applied to each area to generate map data.

[0041] An example of the process of arranging feature block images will be described in detail below with reference to FIG. 3B.

[0042] Figure 3B shows an example of the relationship between feature location distribution data and map data generated by arranging one or more types of feature block images. The top diagram (a) in Figure 3B shows an example of the location of features indicated by the location distribution data. The bottom diagram (b) in Figure 3B shows an example of the generated map. The following explanation will be made using the x-y coordinate system shown in Figure 3B. The x-axis and y-axis are orthogonal to each other. The x-axis direction is referred to as the "horizontal direction," and the y-axis direction is referred to as the "vertical direction." The horizontal direction corresponds to the width direction of the feature block image, and the vertical direction corresponds to the height direction of the feature block image. In this example, the feature location distribution data includes the location information of grass, roads, and houses. For ease of understanding, the continuous areas occupied by the grass, roads, and houses are assumed to have a rectangular planar shape with two sides parallel to the x-axis and two sides parallel to the y-axis.

[0043] In the example of FIG. 3B, the processing device 20 sets an arbitrary point in the position distribution data as a reference point, acquires position information for each feature, such as grass, roads, and houses, shown in the position distribution data along the horizontal and vertical directions from the reference point, and identifies the continuous area occupied by each feature. For example, focusing on the grass on the left side shown in FIG. 3B, an area with a horizontal length L1 and a vertical length L2 from the reference point Pa is identified as the area occupied by the left side grass. The processing device 20 divides the horizontal length L1 of the left side grass by the width W of the feature block image representing the grass to determine the number of feature block images to be arranged horizontally (hereinafter referred to as the "horizontal number"). The processing device 20 also divides the vertical length L2 of the left side grass by the height H of the feature block image representing the grass to determine the number of feature block images to be arranged vertically (hereinafter referred to as the "vertical number"). The processing device 20 arranges the feature block images horizontally as many times as the number of feature blocks, and arranges the feature block images vertically as many times as the number of feature blocks, starting from the reference point Pa. This allows the generation of the grass area A1 on the left side of the map, as shown in the lower diagram of Figure 3B.

[0044] Note that there are cases where the result of dividing the horizontal length L1 by the width W of the feature block image, or the result of dividing the vertical length L2 by the height H, does not result in an integer. In such cases, the processing device 20 may generate map data corresponding to the feature position distribution data by enlarging or reducing some of the arranged feature block images in at least one of the horizontal and vertical directions.

[0045] Similarly, for the right-hand grass, the processing device 20 determines the horizontal length L3 and vertical length L4 of the right-hand grass area from the position distribution data. The processing device 20 calculates the horizontal number by dividing L3 by W, and calculates the vertical number by dividing L4 by H. The processing device 20 arranges feature block images in the horizontal direction by the number of horizontal feature block images, and arranges feature block images in the vertical direction by the number of vertical feature block images, starting from the reference point Pb. This allows the right-hand grass area A2 on the map to be generated.

[0046] The processing device 20 performs the same processing as above for roads and houses. When the processing device 20 detects a boundary between different types of features as a result of searching for features horizontally and vertically in the feature position distribution data, it sets the point on the boundary as a new reference point. For example, if the processing device 20 detects a boundary between grass and a house at point Pc in FIG. 3B, it sets point Pc as a new reference point. The processing device 20 calculates the horizontal and vertical numbers of feature block images of the house from the horizontal and vertical lengths of the area occupied by the house, and arranges the feature block images by that number.

[0047] Generally, the area occupied by each feature, such as grass, roads, and houses, has a planar shape other than a rectangle. The processing device 20 regards the area occupied by each feature as a collection of multiple rectangular or strip-shaped areas of different sizes, and applies the above processing to each area, thereby generating map data.

[0048] The above process is merely an example, and the processing device 20 may generate map data using other methods. For example, the processing device 20 may detect the width of roads around a field based on at least one of sensor data and image data, and generate map data that reflects the width of the roads by arranging road block images in the width direction in an amount equal to the width of the roads. In this case, features other than roads may be represented by a single type of feature block image. After arranging the feature block images, the processing device 20 may generate final map data by smoothing boundaries between feature blocks of different types or interpolating areas where the feature type is unknown with images of surrounding features.

[0049] FIG. 4 is a diagram showing an example of a portion of a map generated in step S13. The map in this example is a bitmap map created by arranging nine types of feature block images (dot pictures in this example) shown in the lower part of FIG. 4. The processing device 20 can generate a map such as that shown in FIG. 4 by processing to place a corresponding feature block image at the position of each feature indicated by the feature position distribution data. In this way, the processing device 20 can generate map data by arranging multiple types of feature block images recorded in the storage device 10.

[0050] The processing device 20 may arrange the feature block images for each feature by enlarging or reducing them to an appropriate size. For example, feature block images representing structures such as houses and buildings may be arranged relatively large, and feature block images representing grass, farm roads, waterways, and ditches may be arranged relatively small. Such processing may smooth the boundaries between the arranged feature block images, making it possible to generate a map that is more realistic.

[0051] In the example of Fig. 4, road block images associated with roads are also used, and a map is generated that reflects the actual position and width of each road. In this way, the processing device 20 may generate map data by arranging one or more types of feature block images associated with one or more types of feature and road block images associated with roads according to the feature position distribution data. Even if road block images are not used, the processing device 20 may arrange block images of grass or the like on both sides of the road, and then generate map data that represents the area between those block images as a road.

[0052] The map shown in Figure 4 provides a simplified representation of the environment in which agricultural machinery will travel, while accurately representing the location and width of roads around the field. Creating such a map makes it possible to properly plan routes for automated driving of agricultural machinery.

[0053] Unlike conventional autonomous vehicles, autonomous agricultural machinery often travels at low speeds and often travels on farm roads surrounding fields rather than on public roads where pedestrians and vehicles frequently pass by. Such autonomous agricultural machinery is required to navigate farm roads appropriately and smoothly enter and exit fields automatically. To achieve this, maps accurately depicting the location and width of farm roads surrounding fields are required. However, such maps are not generally available. While maps for general autonomous vehicles (not intended for agricultural use) are widely available, such maps are often inapplicable to autonomous agricultural machinery. While maps for general autonomous vehicles contain detailed information about roads, such as public roads and expressways, they often lack information about fields and farm roads. For example, they often do not include information about field entrances and exits, and the shape of fields and the width of farm roads adjacent to fields are often inaccurate. Furthermore, maps for general autonomous vehicles often contain a lot of information that is unnecessary for agricultural machinery (e.g., information about roads, such as expressways, that agricultural machinery does not travel on). For this reason, general maps are not necessarily suitable for creating and displaying routes for autonomous agricultural machinery.

[0054] The map generation system 1000 of this embodiment can generate, through simple processing, a map that accurately represents the positions and widths of roads around a field and simply represents features other than roads. This makes it possible to efficiently generate maps that are suitable for creating and displaying routes for agricultural machinery.

[0055] Next, a more specific example of the configuration and operation of the map generation system 1000 according to this embodiment will be described.

[0056] 5 is a block diagram showing another example configuration of a map generation system 1000. In this example, the map generation system 1000 further includes a mobile object 30 in addition to the storage device 10 and the processing device 20. The processing device 20 in this example is configured to perform each of the processes of feature recognition, generation of feature position distribution data, arrangement of feature block images, and map data generation based on data acquired from the mobile object 30. Each of these processes can be implemented as a software module or a hardware module (e.g., a circuit).

[0057] The mobile object 30 is equipped with a LiDAR sensor 33, a camera 34, and a GNSS unit 35. While moving, the mobile object 30 uses the LiDAR sensor 33, the camera 34, and the GNSS unit 35 to collect data necessary for generating map data to be generated by the processing device 20. The data collected by the LiDAR sensor 33, the camera 34, and the GNSS unit 35 is provided to the processing device 20 via a wired or wireless network or a recording medium. The mobile object 30 may be the same as or different from the agricultural machine that uses the map data generated by the processing device 20. The mobile object 30 is not limited to agricultural machines such as work vehicles, but may also be an air vehicle such as a drone.

[0058] The LiDAR sensor 33 is a sensor that uses light to measure distances to surrounding objects. The LiDAR sensor 33 generates and outputs sensor data that indicates the distance distribution or position distribution of objects present around the mobile unit 30. The LiDAR sensor 33 may be, for example, a scanning sensor that acquires information about the distance distribution or position distribution of objects in space by scanning a laser beam. Alternatively, the LiDAR sensor 33 may be a flash-type sensor that acquires information about the distance distribution or position distribution of objects in space by using light that diffuses over a wide area.

[0059] The LiDAR sensor 33 may be configured to, for example, emit a pulsed laser beam (i.e., a laser pulse), measure the time it takes for the laser pulse to be reflected by a surrounding object and return to the LiDAR sensor 33, and calculate the distance to a reflection point on the object surface based on that time. Alternatively, the LiDAR sensor 33 may be a sensor that measures distances using FMCW (Frequency Modulated Continuous Wave) technology. A LiDAR sensor that uses FMCW technology emits laser light whose frequency is linearly modulated, and can determine the distance to and speed of the reflection point based on the frequency of a beat signal obtained by detecting interference light between the emitted light and the reflected light.

[0060] The LiDAR sensor 33 may be configured to output sensor data including information on the distance and direction to each reflection point. Alternatively, the LiDAR sensor 33 may calculate the coordinate values ​​of each reflection point in a coordinate system fixed to the mobile body 30 (hereinafter referred to as the "mobile body coordinate system") from the information on the distance and direction to each reflection point, and output sensor data including the coordinate values. The sensor data may include brightness information of each reflection point. The brightness information of each reflection point may be used to identify the type of feature located at that reflection point.

[0061] The LiDAR sensor 33 may be a two-dimensional LiDAR sensor or a three-dimensional LiDAR sensor. A two-dimensional LiDAR sensor may scan the environment with a laser beam rotating in a single plane. In contrast, a three-dimensional LiDAR sensor may scan the environment with multiple laser beams rotating along different conical planes. The three-dimensional LiDAR sensor may be used to collect data necessary to generate a three-dimensional map.

[0062] The coordinate values ​​(two-dimensional or three-dimensional coordinate values) of each reflection point calculated based on the distance and direction information generated by the LiDAR sensor 33 are expressed in a mobile body coordinate system. The processing device 20 can obtain the coordinate values ​​of each reflection point in the world coordinate system by converting the coordinate values ​​of each reflection point from the mobile body coordinate system to a world coordinate system fixed to the Earth. This makes it possible to generate two-dimensional or three-dimensional point cloud data. To convert from the mobile body coordinate system to the world coordinate system, information on the pose (i.e., position and orientation) of the mobile body 30 is required. The pose of the mobile body 30 can be determined based on the GNSS data output from the GNSS unit 35.

[0063] The camera 34 is an imaging device that captures images of the environment surrounding the mobile object 30 and generates image data. The camera 34 may include, for example, an image sensor that captures visible light images or infrared images, and a lens optical system that forms an image on the image sensor. The camera 34 generates moving image data at a predetermined frame rate. The image data output from the camera 34 can be used to recognize objects present in space.

[0064] The GNSS unit 35 includes a GNSS receiver and generates GNSS data including position information of the mobile object 30. The GNSS receiver includes an antenna for receiving signals from GNSS satellites and a receiver for generating GNSS data based on the signals received by the antenna. Mobile 30and a processor that calculates the position of the vehicle. The GNSS unit 35 receives satellite signals transmitted from multiple GNSS satellites and performs positioning based on the satellite signals. GNSS is a general term for satellite positioning systems such as GPS (Global Positioning System), QZSS (Quasi-Zenith Satellite System, e.g., Michibiki), GLONASS, Galileo, and BeiDou.

[0065] The GNSS unit 35 may include an inertial measurement unit (IMU). The IMU may include, for example, a 3-axis acceleration sensor, a 3-axis gyroscope, and a 3-axis geomagnetic sensor or other orientation sensors. The IMU measures the tilt, minute movements, and orientation of the mobile object 30. The GNSS unit 35 can estimate the position and orientation of the mobile object 30 with high accuracy by complementing position data based on satellite signals with data acquired by the IMU. Note that the IMU may be provided independently of the GNSS unit 35.

[0066] FIG. 6 is a diagram schematically illustrating a mobile object 30 performing sensing using a LiDAR sensor 33 and a camera 34 while traveling on a farm road around a field. In this example, the mobile object 30 is a work vehicle similar to an agricultural machine (e.g., a tractor) that uses map data. The mobile object 30 may also be another type of mobile object, such as a drone. The dashed sector in FIG. 6 schematically illustrates an example of the sensing range of the LiDAR sensor 33 or the camera 34. The mobile object 30 may be driven or remotely controlled by a driver (operator). Alternatively, the mobile object 30 may move automatically while simultaneously estimating its own position and generating a map based on sensor data from the LiDAR sensor 33 using an algorithm such as SLAM (Simultaneous Localization and Mapping).

[0067] While the mobile object 30 is moving, the LiDAR sensor 33 measures the distance to each feature present around the mobile object 30, and the camera 34 captures images of the environment surrounding the mobile object 30. The GNSS unit 35 repeatedly measures the position and orientation of the mobile object 30. While the mobile object 30 is moving, the sensor data output from the LiDAR sensor 33, the image data output from the camera 34, and the GNSS data output from the GNSS unit 35 are stored in a storage device provided in the mobile object 30, associated with, for example, date and time information. The stored sensor data, image data, and GNSS data are sent to the processing device 20 at a certain timing and used to generate map data.

[0068] In this example, the processing device 20 generates map data based on the sensor data, image data, and GNSS data. More specifically, the processing device 20 generates map data based on the sensor data, image data, and GNSS data. A Map data is generated by executing the operations shown in Figure 3. A In step S11 shown in FIG. 1, the processing device 20 generates position distribution data of features based on the sensor data, image data, and GNSS data.

[0069] 7 is a flowchart showing an example of a process for generating location distribution data of features by the processing device 20. In this example, the processing device 20 performs the process shown in FIG. A In step S11 shown in FIG. 1, the processes of steps S111, S112, and S113 are executed. The processes of each step will be described below.

[0070] (Step S111) The processing device 20 acquires sensor data, image data, and GNSS data collected by the mobile object 30 while it is moving along roads around the farm field.

[0071] (Step S112) The processing device 20 recognizes one or more types of features existing around the road based on the acquired image data. For example, the processing device 20 recognizes features existing around the road, including at least one of grass, trees, ditches, waterways, fields, and buildings, based on the image data. The processing device 20 can recognize one or more types of features existing around the road by performing processing such as pattern matching or segmentation based on the image data. Image recognition using machine learning may be used to recognize the features. For example, the processing device 20 may recognize features in the image by semantic segmentation using a convolutional neural network (CNN). The image recognition algorithm is not limited to a specific one, and any algorithm may be used. The processing device 20 may detect boundaries between the road and other features or the width of the road through image recognition. The processing device 20 may use information regarding the boundaries between the road and other features and / or the width of the road in the subsequent arrangement processing of the feature block images in step S13.

[0072] (Step S113) The processing device 20 generates position distribution data of the features recognized in step S112 based on the GNSS data and sensor data. The processing device 20 calculates the position coordinates (e.g., latitude and longitude) of each reflection point in a geographic coordinate system, for example, based on the distance and direction to each reflection point indicated by the sensor data and the position and orientation of the mobile body 30 indicated by the GNSS data. At this time, the processing device 20 may calculate three-dimensional position coordinates including altitude information of the reflection points. This allows the processing device 20 to generate two-dimensional or three-dimensional point cloud data indicating the distribution of objects present in the environment surrounding the mobile body 30. The processing device 20 matches the image data with the point cloud data to identify the position range of points corresponding to each feature recognized in step S112. The processing device 20 can generate, for example, data indicating an identifier indicating the type of each recognized feature and the range of position coordinates of the corresponding point as the position distribution data of the features. In this way, the processing device 20 can generate position distribution data indicating the distribution of the positions (e.g., latitude and longitude) of one or more types of features based on the GNSS data and sensor data.

[0073] By the above process, the processing device 20 can generate position distribution data of the features recognized based on the image data. After step S113, the processing device 20 performs the process shown in FIG. A By executing the processes of steps S12 and S13 shown in the figure, map data is generated.

[0074] In step S13, the processing device 20 generates map data by arranging one or more types of feature block images associated with one or more types of features recognized in step S112 within the area on the map data where the features exist, according to the position distribution data. The processing device 20, for example, arranges the feature block images so that the width of the road recognized in step S112 is accurately reflected. This makes it possible to generate a map suitable for route planning for agricultural machinery. The processing device 20 may also generate a map without using road block images indicating roads. For example, the processing device 20 may arrange one or more types of feature block images other than roads along the boundary between roads and features other than roads outside the boundary, and may generate a map by placing an image representing the road (e.g., an image with a specific color or pattern) inside the boundary.

[0075] In the above example, the mobile object 30 includes both the LiDAR sensor 33 and the camera 34 (i.e., an imaging device). However, the mobile object 30 may include only one of them. Even in this case, it is possible to detect and recognize features and generate location distribution data of the features based on data output from the LiDAR sensor 33 or the camera 34. For example, it is possible to generate location distribution data indicating the distribution of roads and non-road features based on information indicating the distance or position of each reflection point and brightness information of each reflection point included in the sensor data output from the LiDAR sensor 33. Because light reflectance varies depending on the feature, it is possible to distinguish features based on brightness information of the reflection points. Alternatively, it is possible to achieve the same results as when using the LiDAR sensor 33 by using an imaging device capable of acquiring distance information, such as a ToF (Time of Flight) camera or a stereo camera. Furthermore, the mobile object 30 may perform positioning or self-localization of the mobile object 30 using other methods instead of the GNSS unit 35. For example, the mobile object 30 may move automatically while simultaneously performing self-localization and map generation based on sensor data using an algorithm such as SLAM. In this case, the processing device 20 may generate final map data by executing the above-described feature recognition processing and feature block image arrangement processing based on a map (for example, a point cloud map) generated by the mobile body 30.

[0076] After generating the map data in the above-described manner, the processing device 20 may generate a route for the agricultural machine to automatically travel along roads based on the map data. The generation of the route may be performed by a device different from the processing device 20 (for example, a computer installed in the agricultural machine or a server computer on the cloud). The agricultural machine automatically travels along the generated route (hereinafter, sometimes referred to as a "target route" or a "global route"). The processing device 20 may display the generated map and route on the display of a terminal device used by the user.

[0077] <Embodiment of agricultural management system> Next, an embodiment of a system for managing agricultural machinery that operates automatically based on map data created by the above method will be described. The following mainly describes an embodiment in which the technology of the present disclosure is applied to a work vehicle such as a tractor, which is an example of agricultural machinery. The technology of the present disclosure is not limited to work vehicles such as tractors, but can also be applied to other types of agricultural machinery.

[0078] FIG. 8 is a diagram for explaining an overview of an agricultural management system according to an exemplary embodiment of the present disclosure. The agricultural management system shown in FIG. 8 includes a work vehicle 100, a terminal device 400, and a management device 600. The terminal device 400 is a computer used by a user to remotely monitor the work vehicle 100. The management device 600 is a computer managed by a business operator that operates the agricultural management system. The work vehicle 100, the terminal device 400, and the management device 600 can communicate with each other via a network 80. Although FIG. 8 illustrates one work vehicle 100, the agricultural management system may include multiple work vehicles or other agricultural machinery.

[0079] The work vehicle 100 in this embodiment is a tractor. The work vehicle 100 can be fitted with an implement at either the rear or the front, or both. The work vehicle 100 can travel within a field while performing agricultural work according to the type of implement. The work vehicle 100 may also travel within or outside a field without an implement attached.

[0080] The work vehicle 100 has an automatic driving function. That is, the work vehicle 100 can travel by the operation of a control device, without manual operation. The control device in this embodiment is provided inside the work vehicle 100, and can control both the speed and steering of the work vehicle 100. The work vehicle 100 can travel automatically not only within a field, but also outside the field (for example, on a road).

[0081] The work vehicle 100 is equipped with devices used for positioning or self-location estimation, such as a GNSS receiver and a LiDAR sensor. A control device of the work vehicle 100 causes the work vehicle 100 to travel automatically based on the position of the work vehicle 100 and information about a target route. In addition to controlling the travel of the work vehicle 100, the control device also controls the operation of the implement. This allows the work vehicle 100 to perform agricultural work using the implement while traveling automatically within a field. Furthermore, the work vehicle 100 can automatically travel along roads outside the field (e.g., farm roads or public roads) along a target route. When traveling automatically along roads outside the field, the work vehicle 100 travels while generating a local route along the target route that can avoid obstacles, based on data output from sensing devices such as a camera or LiDAR sensor. Within the field, the work vehicle 100 may travel while generating a local route as described above, or may travel along a target route without generating a local route and stop if an obstacle is detected.

[0082] The management device 600 is a computer that manages agricultural work performed by the work vehicle 100. The management device 600 may be, for example, a server computer that centrally manages information about farm fields on the cloud and supports agriculture by utilizing data on the cloud. The management device 600, for example, creates a map of the environment in which the work vehicle 100 will travel and a work plan for the work vehicle 100, and plans a global path for the work vehicle 100 according to the map and work plan. The management device 600 generates a map of the environment in which the work vehicle 100 will travel using a method similar to the method described with reference to FIGS. 1 to 7. That is, the management device 600 recognizes one or more types of features present in the environment based on data collected by the work vehicle 100 or other moving objects using at least one of a LiDAR sensor and an imaging device, and generates an environmental map by arranging feature block images corresponding to the recognized features. The management device 600 generates a global path (target path) for the work vehicle 100 based on the generated environmental map. In this embodiment, the management device 600 functions as a map generation system that generates map data for automatic operation of agricultural machinery.

[0083] The management device 600 generates target routes within the field and outside the field using different methods. The management device 600 generates target routes within the field based on information about the field. For example, the management device 600 can generate target routes within the field based on various information, such as pre-registered field outlines, field area, the location of field entrances and exits, the width of the work vehicle 100, the width of the implement, the type of work being performed, the type of crop being cultivated, the crop growing area, the crop growth conditions, or the spacing between crop rows or furrows. The management device 600 generates target routes within the field based on information input by the user using the terminal device 400 or another device, for example. The management device 600 generates target routes within the field so as to cover the entire work area where work is to be performed, for example. On the other hand, the management device 600 generates target routes outside the field in accordance with a work plan or user instructions. For example, the management device 600 can generate a target route outside the field based on various information such as the order of farm work indicated in the work plan, the location of the field where each farm work will be performed, the location of the entrance and exit to the field, the scheduled start and end times of each farm work, road surface conditions, weather conditions, or traffic conditions. The management device 600 may generate a target route based on information indicating a route or waypoints specified by a user operating the terminal device 400, rather than based on a work plan. The management device 600 transmits data on the generated work plan, target route, and environmental map to the work vehicle 100. The work vehicle 100 automatically moves and performs farm work based on this data.

[0084] It should be noted that the global route design and the generation (or editing) of the environmental map may be performed by other devices, not just the management device 600. For example, the control device of the work vehicle 100 may perform the global route design or the generation or editing of the environmental map.

[0085] The terminal device 400 is a computer used by a user located remotely from the work vehicle 100. The terminal device 400 shown in FIG. 8 is a laptop computer, but is not limited to this. The terminal device 400 may be a stationary computer such as a desktop PC (personal computer), or a mobile terminal such as a smartphone or tablet computer. The terminal device 400 may be used to remotely monitor or remotely operate the work vehicle 100. For example, the terminal device 400 can display on a display image captured by one or more cameras (imaging devices) equipped on the work vehicle 100. The user can view the image to check the situation around the work vehicle 100 and send instructions to the work vehicle 100 to stop or start. The terminal device 400 can also display on a display a setting screen that allows the user to input information necessary to create a work plan for the work vehicle 100 (e.g., a schedule for each agricultural work). When the user inputs the necessary information on the setting screen and performs a send operation, the terminal device 400 transmits the input information to the management device 600. The management device 600 creates a work plan based on that information. The terminal device 400 can also be used to register one or more fields where the work vehicle 100 will perform agricultural work. The terminal device 400 may further have a function to display a setting screen on the display for the user to input information necessary to set a target route. The terminal device 400 may display a map generated by the management device 600 (for example, a map such as that shown in FIG. 4) on the setting screen. The user may perform an operation to set a target route on the displayed map.

[0086] The configuration and operation of the system in this embodiment will be described in more detail below.

[0087] [1. Configuration] FIG. 9 is a side view that schematically shows an example of a work vehicle 100 and an implement 300 coupled to the work vehicle 100. The work vehicle 100 in this embodiment can operate in both a manual driving mode and an automatic driving mode. In the automatic driving mode, the work vehicle 100 can travel unmanned. The work vehicle 100 can be driven automatically both inside and outside a field.

[0088] As shown in Figure 9, work vehicle 100 includes a vehicle body 101, a prime mover (engine) 102, and a speed change device (transmission) 103. Vehicle body 101 is provided with wheels 104 with tires, and a cabin 105. Wheels 104 include a pair of front wheels 104F and a pair of rear wheels 104R. Inside cabin 105 are provided a driver's seat 107, a steering device 106, an operation terminal 200, and a group of switches for operation. When work vehicle 100 travels for work in a field, crawlers instead of tires may be attached to one or both of front wheels 104F and rear wheels 104R.

[0089] The work vehicle 100 is equipped with a plurality of sensing devices that sense the surroundings of the work vehicle 100. In the example of Figure 9, the sensing devices include a plurality of cameras 120, a LiDAR sensor 140, and a plurality of obstacle sensors 130.

[0090] Cameras 120 may be installed, for example, on the front, rear, left and right sides of work vehicle 100. Cameras 120 capture images of the environment around work vehicle 100 and generate image data. Images acquired by cameras 120 may be transmitted to terminal device 400 for remote monitoring. These images may be used to monitor work vehicle 100 during unmanned operation. Cameras 120 may also be used to generate images for recognizing surrounding features or obstacles, white lines, signs, or markings when work vehicle 100 travels on roads outside of fields (farm roads or public roads).

[0091] In the example of FIG. 9 , the LiDAR sensor 140 is disposed on the lower front side of the vehicle body 101. The LiDAR sensor 140 may be disposed in another location. While the work vehicle 100 is traveling mainly outside the field, the LiDAR sensor 140 repeatedly outputs sensor data indicating the distance and direction to each measurement point of an object in the surrounding environment, or the two-dimensional or three-dimensional coordinate values ​​of each measurement point. The sensor data output from the LiDAR sensor 140 is processed by a control device of the work vehicle 100. The control device can estimate the self-localization of the work vehicle 100 by matching the sensor data with an environmental map. The control device can further detect objects such as obstacles present around the work vehicle 100 based on the sensor data and generate a local path that the work vehicle 100 should actually travel along the global path. The control device can also generate or edit an environmental map using an algorithm such as SLAM. The work vehicle 100 may be equipped with multiple LiDAR sensors disposed at different positions and with different orientations.

[0092] The multiple obstacle sensors 130 shown in FIG. 9 are provided at the front and rear of the cabin 105. The obstacle sensors 130 may also be located in other locations. For example, one or more obstacle sensors 130 may be provided at any position on the side, front, or rear of the vehicle body 101. The obstacle sensors 130 may include, for example, a laser scanner or ultrasonic sonar. The obstacle sensors 130 are used to detect surrounding obstacles during autonomous driving and to stop or detour the work vehicle 100. A LiDAR sensor 140 may be used as one of the obstacle sensors 130.

[0093] The work vehicle 100 further includes a GNSS unit 110. The GNSS unit 110 includes a GNSS receiver. The GNSS receiver may include an antenna that receives signals from GNSS satellites and a processor that calculates the position of the work vehicle 100 based on the signals received by the antenna. The GNSS unit 110 receives satellite signals transmitted from multiple GNSS satellites and performs positioning based on the satellite signals. In this embodiment, the GNSS unit 110 is provided on top of the cabin 105, but it may be provided in another location.

[0094] The GNSS unit 110 may include an inertial measurement unit (IMU). Signals from the IMU can be used to complement position data. The IMU can measure the tilt and minute movements of the work vehicle 100. By complementing position data based on satellite signals with data acquired by the IMU, positioning performance can be improved.

[0095] The control device of the work vehicle 100 may use, in addition to the positioning results from the GNSS unit 110, sensing data acquired by sensing devices such as the camera 120 or LiDAR sensor 140 for positioning. If there are features that function as characteristic points in the environment in which the work vehicle 100 is traveling, such as farm roads, forest roads, public roads, or orchards, the position and orientation of the work vehicle 100 can be estimated with high accuracy based on the data acquired by the camera 120 or LiDAR sensor 140 and an environmental map that has been stored in advance in a storage device. By using the data acquired by the camera 120 or LiDAR sensor 140 to correct or complement position data based on satellite signals, the position of the work vehicle 100 can be determined with higher accuracy.

[0096] The prime mover 102 may be, for example, a diesel engine. An electric motor may be used instead of a diesel engine. The transmission 103 can change the propulsive force and travel speed of the work vehicle 100 by changing gears. The transmission 103 can also switch the work vehicle 100 between forward and reverse travel.

[0097] The steering device 106 includes a steering wheel, a steering shaft connected to the steering wheel, and a power steering device that assists steering by the steering wheel. The front wheels 104F are steerable wheels, and the traveling direction of the work vehicle 100 can be changed by changing the turning angle (also referred to as the "steering angle"). The steering angle of the front wheels 104F can be changed by operating the steering wheel. The power steering device includes a hydraulic device or an electric motor that supplies an assisting force to change the steering angle of the front wheels 104F. When automatic steering is performed, the steering angle is automatically adjusted by the force of the hydraulic device or electric motor under control of a control device arranged inside the work vehicle 100.

[0098] A coupling device 108 is provided at the rear of the vehicle body 101. The coupling device 108 includes, for example, a three-point support device (also referred to as a "three-point link" or "three-point hitch"), a PTO (Power Take Off) shaft, a universal joint, and a communication cable. The coupling device 108 allows the implement 300 to be attached to and detached from the work vehicle 100. The coupling device 108 can raise and lower the three-point link using, for example, a hydraulic device, thereby changing the position or attitude of the implement 300. Power can also be sent from the work vehicle 100 to the implement 300 via the universal joint. The work vehicle 100 can pull the implement 300 and cause the implement 300 to perform a predetermined task. The coupling device may be provided at the front of the vehicle body 101. In this case, the implement can be connected to the front of the work vehicle 100.

[0099] 9 is a rotary tiller, but the implement 300 is not limited to a rotary tiller. For example, any implement such as a seeder (seed sowing machine), a spreader (fertilizer applicator), a transplanter, a mower (grass cutter), a rake, a baler (grass collector), a harvester (harvesting machine), a sprayer, or a harrow can be connected to the work vehicle 100 and used.

[0100] The work vehicle 100 shown in Fig. 9 is capable of being driven by a driver, but may also be capable of being driven only unmanned. In that case, components that are only required for driven operation, such as the cabin 105, steering device 106, and driver's seat 107, may not be provided in the work vehicle 100. The unmanned work vehicle 100 can travel autonomously or by remote control by a user.

[0101] 10 is a block diagram showing an example configuration of the work vehicle 100 and the implement 300. The work vehicle 100 and the implement 300 can communicate with each other via a communication cable included in the coupling device 108. The work vehicle 100 can communicate with the terminal device 400 and the management device 600 via the network 80.

[0102] In the example of FIG. 10 , the work vehicle 100 includes a GNSS unit 110, a camera 120, an obstacle sensor 130, a LiDAR sensor 140, and an operation terminal 200, as well as a group of sensors 150 that detect the operating state of the work vehicle 100, a control system 160, a communication device 190, a group of operation switches 210, a buzzer 220, and a drive unit 240. These components are communicatively connected to each other via a bus. The GNSS unit 110 includes a GNSS receiver 111, an RTK receiver 112, an inertial measurement unit (IMU) 115, and a processing circuit 116. The group of sensors 150 includes a steering wheel sensor 152, a turning angle sensor 154, and an axle sensor 156. The control system 160 includes a memory device 170 and a control device 180. The control device 180 includes multiple electronic control units (ECUs) 181 to 186. Implement 300 includes a drive unit 340, a control unit 380, and a communication unit 390. Note that Fig. 10 shows components that are relatively highly related to the operation of the autonomous driving by work vehicle 100, and does not show other components.

[0103] The GNSS receiver 111 in the GNSS unit 110 receives satellite signals transmitted from multiple GNSS satellites and generates GNSS data based on the satellite signals. The GNSS data is generated in a predetermined format, such as the NMEA-0183 format. The GNSS data may include, for example, values ​​indicating the identification number, elevation angle, azimuth angle, and reception strength of each satellite from which a satellite signal is received.

[0104] The GNSS unit 110 shown in FIG. 10 performs positioning of the work vehicle 100 using RTK (Real Time Kinematic)-GNSS. FIG. 11 is a conceptual diagram showing an example of a work vehicle 100 performing positioning using RTK-GNSS. Positioning using RTK-GNSS uses satellite signals transmitted from multiple GNSS satellites 50 as well as correction signals transmitted from a reference station 60. The reference station 60 may be installed near the field where the work vehicle 100 will be traveling (for example, within 10 km of the work vehicle 100). The reference station 60 generates correction signals, for example in RTCM format, based on the satellite signals received from the multiple GNSS satellites 50 and transmits them to the GNSS unit 110. The RTK receiver 112 includes an antenna and a modem and receives the correction signals transmitted from the reference station 60. The processing circuit 116 of the GNSS unit 110 corrects the positioning results obtained by the GNSS receiver 111 based on the correction signals. By using RTK-GNSS, it is possible to perform positioning with an accuracy of, for example, a few centimeters. Position information including latitude, longitude, and altitude information is obtained through highly accurate positioning using RTK-GNSS. The GNSS unit 110 calculates the position of the work vehicle 100, for example, at a frequency of approximately 1 to 10 times per second.

[0105] The positioning method is not limited to RTK-GNSS, and any positioning method (such as interferometric positioning or differential positioning) that can obtain position information with the required accuracy can be used. For example, positioning may be performed using a Virtual Reference Station (VRS) or a Differential Global Positioning System (DGPS). If position information with the required accuracy can be obtained without using a correction signal transmitted from the reference station 60, the position information may be generated without using a correction signal. In this case, the GNSS unit 110 does not need to be equipped with the RTK receiver 112.

[0106] Even when RTK-GNSS is used, in places where correction signals from the reference station 60 cannot be obtained (for example, on a road far from a field), the position of the work vehicle 100 is estimated by other methods without relying on signals from the RTK receiver 112. For example, the position of the work vehicle 100 can be estimated by matching data output from the LiDAR sensor 140 and / or camera 120 with a highly accurate environmental map.

[0107] The GNSS unit 110 in this embodiment further includes an IMU 115. The IMU 115 may include a three-axis acceleration sensor and a three-axis gyroscope. The IMU 115 may also include a direction sensor such as a three-axis geomagnetic sensor. The IMU 115 functions as a motion sensor and can output signals indicating various quantities such as the acceleration, velocity, displacement, and attitude of the work vehicle 100. The processing circuit 116 can estimate the position and orientation of the work vehicle 100 with higher accuracy based on the signals output from the IMU 115 in addition to the satellite signals and correction signals. The signals output from the IMU 115 can be used to correct or complement the position calculated based on the satellite signals and correction signals. The IMU 115 outputs signals at a higher frequency than the GNSS receiver 111. Using these high-frequency signals, the processing circuit 116 can measure the position and orientation of the work vehicle 100 at a higher frequency (e.g., 10 Hz or higher). A three-axis acceleration sensor and a three-axis gyroscope may be provided separately instead of the IMU 115. The IMU 115 may be provided as a device separate from the GNSS unit 110.

[0108] The camera 120 is an imaging device that captures images of the environment surrounding the work vehicle 100. The camera 120 includes an image sensor, such as a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The camera 120 may also include an optical system including one or more lenses and a signal processing circuit. The camera 120 captures images of the environment surrounding the work vehicle 100 while the work vehicle 100 is traveling and generates image (e.g., video) data. The camera 120 can capture video at a frame rate of, for example, 3 frames per second (fps) or higher. The images generated by the camera 120 can be used, for example, when a remote observer uses the terminal device 400 to check the environment surrounding the work vehicle 100. The images generated by the camera 120 may be used for positioning or obstacle detection. As shown in FIG. 9, multiple cameras 120 may be installed at different positions on the work vehicle 100, or a single camera may be installed. A visible light camera that generates a visible light image and an infrared camera that generates an infrared image may be provided separately. Both a visible light camera and an infrared camera may be provided as cameras that generate images for surveillance. The infrared camera can also be used to detect obstacles at night.

[0109] The obstacle sensor 130 detects objects present around the work vehicle 100. The obstacle sensor 130 may include, for example, a laser scanner or an ultrasonic sonar. The obstacle sensor 130 outputs a signal indicating the presence of an obstacle when an object is present closer than a predetermined distance from the obstacle sensor 130. Multiple obstacle sensors 130 may be provided at different positions on the work vehicle 100. For example, multiple laser scanners and multiple ultrasonic sonars may be arranged at different positions on the work vehicle 100. By providing such a large number of obstacle sensors 130, blind spots in monitoring obstacles around the work vehicle 100 can be reduced.

[0110] The steering wheel sensor 152 measures the rotation angle of the steering wheel of the work vehicle 100. The turning angle sensor 154 measures the turning angle of the front wheels 104F, which are the steered wheels. The measurement values ​​from the steering wheel sensor 152 and the turning angle sensor 154 are used for steering control by the control device 180.

[0111] The axle sensor 156 measures the rotational speed of the axle connected to the wheel 104, i.e., the number of rotations per unit time. The axle sensor 156 may be a sensor that uses, for example, a magnetoresistive element (MR), a Hall element, or an electromagnetic pickup. The axle sensor 156 outputs a numerical value that indicates, for example, the number of rotations per minute (unit: rpm) of the axle. The axle sensor 156 is used to measure the speed of the work vehicle 100.

[0112] The drive device 240 includes various devices necessary for the travel of the work vehicle 100 and the driving of the implement 300, such as the prime mover 102, transmission 103, steering device 106, and coupling device 108 described above. The prime mover 102 may be equipped with an internal combustion engine such as a diesel engine. The drive device 240 may be equipped with an electric motor for traction instead of or in addition to the internal combustion engine.

[0113] The buzzer 220 is an audio output device that emits a warning sound to notify of an abnormality. For example, the buzzer 220 emits the warning sound when an obstacle is detected during automatic driving. The buzzer 220 is controlled by the control device 180.

[0114] The storage device 170 includes one or more storage media, such as a flash memory or a magnetic disk. The storage device 170 stores various data generated by the GNSS unit 110, the camera 120, the obstacle sensor 130, the LiDAR sensor 140, the sensor group 150, and the control device 180. The data stored in the storage device 170 may include map data (environmental map) of the environment in which the work vehicle 100 travels and data on a global route (target route) for autonomous driving. The environmental map includes information on multiple fields in which the work vehicle 100 performs agricultural work and the roads in their surrounding areas. The environmental map may be created by processing multiple types of feature block images, as shown in FIG. 4, for example. In addition to the simple environmental map shown in FIG. 4, a detailed environmental map used for self-localization using the LiDAR sensor 140 or the camera 120 may also be stored in the storage device 170. The environmental map and the target route may be generated by a processor in the management device 600. The control device 180 in this embodiment has the function of generating or editing an environmental map and a target route. The control device 180 may generate the environmental map and target route itself, or may edit the environmental map and target route obtained from the management device 600 according to the driving environment of the work vehicle 100. The storage device 170 also stores work plan data received by the communication device 190 from the management device 600. The work plan includes information about multiple agricultural tasks to be performed by the work vehicle 100 over multiple work days. The work plan may be, for example, work schedule data including information about the scheduled time of each agricultural task to be performed by the work vehicle 100 on each work day. The storage device 170 also stores computer programs that cause each ECU in the control device 180 to perform various operations, which will be described later. Such computer programs may be provided to the work vehicle 100 via a storage medium (e.g., a semiconductor memory or an optical disk) or an electric communication line (e.g., the Internet). Such computer programs may be sold as commercial software.

[0115] The control device 180 includes a plurality of ECUs, such as an ECU 181 for speed control, an ECU 182 for steering control, an ECU 183 for implement control, an ECU 184 for automatic driving control, an ECU 185 for route generation, and an ECU 186 for map generation.

[0116] The ECU 181 controls the speed of the work vehicle 100 by controlling the prime mover 102 , the transmission 103 , and the brakes included in the drive unit 240 .

[0117] The ECU 182 controls the steering of the work vehicle 100 by controlling the hydraulic device or electric motor included in the steering device 106 based on the measurement value of the steering wheel sensor 152 .

[0118] The ECU 183 controls the operation of the three-point link and the PTO shaft included in the coupling device 108 to cause the implement 300 to perform a desired operation. A signal for controlling the operation of the implement 300 is generated and transmitted from the communication device 190 to the implement 300 .

[0119] The ECU 184 performs calculations and controls to achieve autonomous driving based on data output from the GNSS unit 110, the camera 120, the obstacle sensor 130, the LiDAR sensor 140, and the sensor group 150. For example, the ECU 184 determines the position of the work vehicle 100 based on data output from at least one of the GNSS unit 110, the camera 120, and the LiDAR sensor 140. Within a farm field, the ECU 184 may determine the position of the work vehicle 100 based solely on data output from the GNSS unit 110. The ECU 184 may also estimate or correct the position of the work vehicle 100 based on data acquired by the camera 120 or the LiDAR sensor 140. By using the data acquired by the camera 120 or the LiDAR sensor 140, the accuracy of positioning can be further improved. Outside of a farm field, the ECU 184 estimates the position of the work vehicle 100 using data output from the LiDAR sensor 140 or the camera 120. For example, ECU 184 may estimate the position of work vehicle 100 by matching data output from LiDAR sensor 140 or camera 120 with an environmental map. During autonomous driving, ECU 184 performs calculations necessary for work vehicle 100 to travel along a target path or a local path based on the estimated position of work vehicle 100. ECU 184 sends a speed change command to ECU 181 and a steering angle change command to ECU 182. In response to the speed change command, ECU 181 changes the speed of work vehicle 100 by controlling prime mover 102, transmission 103, or brakes. In response to the steering angle change command, ECU 182 changes the steering angle by controlling steering device 106.

[0120] While work vehicle 100 is traveling along the target route, ECU 185 sequentially generates local routes that can avoid obstacles. While work vehicle 100 is traveling, ECU 185 recognizes obstacles that exist around work vehicle 100 based on data output from camera 120, obstacle sensor 130, and LiDAR sensor 140. ECU 185 generates local routes that avoid the recognized obstacles. ECU 185 may also have a function for performing global route design instead of management device 600. In this case, ECU 185 determines the destination of work vehicle 100 based on the work plan stored in storage device 170 and determines a target route from the start point of work vehicle 100's movement to the destination point. ECU 185 can create a target route that, for example, allows work vehicle 100 to reach the destination in the shortest time based on an environmental map that includes information about roads around the field stored in storage device 170.

[0121] The ECU 186 generates or edits a map of the environment in which the work vehicle 100 travels. In this embodiment, an environmental map generated by an external device such as the management device 600 is transmitted to the work vehicle 100 and recorded in the storage device 170, but the ECU 186 can also generate or edit the environmental map instead. The following describes the operation when the ECU 186 generates an environmental map. The environmental map can be generated based on sensor data output from the LiDAR sensor 140. When generating the environmental map, the ECU 186 sequentially generates three-dimensional point cloud data based on the sensor data output from the LiDAR sensor 140 while the work vehicle 100 is traveling. The ECU 186 can generate the environmental map by connecting the sequentially generated point cloud data using an algorithm such as SLAM. The environmental map generated in this manner is a highly accurate three-dimensional map and can be used for self-localization estimation by the ECU 184. A two-dimensional map (for example, a bitmap map as shown in FIG. 4) used for global path planning may be generated based on this three-dimensional map. In this specification, the 3D map used for localization and the 2D map used for global route planning are both referred to as “environment maps.” ECU 186 may further edit the map by adding various attribute information related to structures, road surface conditions, road passability, etc., recognized based on data output from camera 120 or LiDAR sensor 140.

[0122] Through the operation of these ECUs, control device 180 realizes autonomous driving. During autonomous driving, control device 180 controls drive device 240 based on the measured or estimated position of work vehicle 100 and the target route. In this way, control device 180 causes work vehicle 100 to travel along the target route.

[0123] The multiple ECUs included in the control device 180 can communicate with each other in accordance with a vehicle bus standard such as CAN (Controller Area Network). Instead of CAN, a faster communication method such as Automotive Ethernet (registered trademark) may be used. In FIG. 10, each of the ECUs 181 to 186 is shown as an individual block, but the functions of each of these may be realized by multiple ECUs. An on-board computer that integrates at least some of the functions of the ECUs 181 to 186 may be provided. The control device 180 may include ECUs other than the ECUs 181 to 186, and any number of ECUs may be provided depending on the functions. Each ECU includes a processing circuit including one or more processors.

[0124] The communication device 190 includes circuits for communicating with the implement 300, the terminal device 400, and the management device 600. The communication device 190 includes circuits for transmitting and receiving signals compliant with ISOBUS standards, such as ISOBUS-TIM, between the communication device 390 of the implement 300. This allows the implement 300 to perform desired operations and acquire information from the implement 300. The communication device 190 may further include an antenna and communication circuits for transmitting and receiving signals via the network 80 between the communication devices of the terminal device 400 and the management device 600. The network 80 may include, for example, a cellular mobile communication network such as 3G, 4G, or 5G, and the Internet. The communication device 190 may also have a function for communicating with a mobile device used by a supervisor near the work vehicle 100. Communication between such mobile terminals may be performed in accordance with any wireless communication standard, such as Wi-Fi (registered trademark), cellular mobile communication such as 3G, 4G or 5G, or Bluetooth (registered trademark).

[0125] The operation terminal 200 is a terminal through which a user performs operations related to the travel of the work vehicle 100 and the operation of the implement 300, and is also referred to as a virtual terminal (VT). The operation terminal 200 may include a display device such as a touch screen and / or one or more buttons. The display device may be, for example, a liquid crystal display or an organic light-emitting diode (OLED) display. By operating the operation terminal 200, a user can perform various operations, such as switching the autonomous driving mode on / off, recording or editing an environmental map, setting a target route, and switching the implement 300 on / off. At least some of these operations can also be achieved by operating the operation switch group 210. The operation terminal 200 may be configured to be detachable from the work vehicle 100. A user located remotely from the work vehicle 100 may operate the detached operation terminal 200 to control the operation of the work vehicle 100. Instead of the operation terminal 200, the user may control the operation of the work vehicle 100 by operating a computer, such as a terminal device 400, on which necessary application software is installed.

[0126] 12 is a diagram showing an example of operation terminal 200 and operation switch group 210 provided inside cabin 105. Operation switch group 210 including a plurality of switches that can be operated by the user is arranged inside cabin 105. Operation switch group 210 may include, for example, a switch for selecting the gear stage of the main transmission or auxiliary transmission, a switch for switching between automatic driving mode and manual driving mode, a switch for switching between forward and reverse, and a switch for raising and lowering implement 300. Note that if work vehicle 100 only performs unmanned operation and does not have a function for manned operation, work vehicle 100 does not need to be equipped with operation switch group 210.

[0127] The drive device 340 in the implement 300 shown in FIG. 10 performs the operations required for the implement 300 to perform a predetermined task. The drive device 340 includes devices appropriate for the application of the implement 300, such as a hydraulic device, an electric motor, or a pump. The control device 380 controls the operation of the drive device 340. The control device 380 causes the drive device 340 to perform various operations in response to signals transmitted from the work vehicle 100 via the communication device 390. In addition, a signal appropriate for the state of the implement 300 can also be transmitted from the communication device 390 to the work vehicle 100.

[0128] Next, the configurations of the management device 600 and the terminal device 400 will be described with reference to Fig. 13. Fig. 13 is a block diagram illustrating a schematic hardware configuration of the management device 600 and the terminal device 400.

[0129] The management device 600 includes a storage device 650, a processor 660, a ROM (Read Only Memory) 670, a RAM (Random Access Memory) 680, and a communication device 690. These components are communicatively connected to each other via a bus. The management device 600 manages the schedule of agricultural work performed in the field by the work vehicle 100 and can function as a cloud server that supports agriculture by utilizing the data it manages. A user can input information necessary for creating a work plan using the terminal device 400 and upload that information to the management device 600 via the network 80. The management device 600 can create a schedule for agricultural work, i.e., a work plan, based on that information. The management device 600 can also generate or edit an environmental map and perform global path planning for the work vehicle 100.

[0130] The communication device 690 is a communication module for communicating with the work vehicle 100 and the terminal device 400 via the network 80. The communication device 690 can perform wired communication in accordance with communication standards such as IEEE1394 (registered trademark) or Ethernet (registered trademark). The communication device 690 may also perform wireless communication in accordance with the Bluetooth (registered trademark) standard or the Wi-Fi standard, or cellular mobile communication such as 3G, 4G, or 5G.

[0131] The processor 660 may be, for example, a semiconductor integrated circuit including a central processing unit (CPU). The processor 660 may be realized by a microprocessor or a microcontroller. Alternatively, the processor 660 may be realized by a field programmable gate array (FPGA) equipped with a CPU, a graphics processing unit (GPU), an application specific integrated circuit (ASIC), or an application specific standard product (ASSP). The processor 660 sequentially executes a computer program stored in the ROM 670, which describes a set of instructions for executing at least one process, to realize a desired process.

[0132] The ROM 670 is, for example, a writable memory (e.g., a PROM), a rewritable memory (e.g., a flash memory), or a read-only memory. The ROM 670 stores a program that controls the operation of the processor 660. The ROM 670 does not need to be a single storage medium, but may be a collection of multiple storage media. Part of the collection of multiple storage media may be removable memory.

[0133] The RAM 680 provides a working area for temporarily loading the control program stored in the ROM 670 at boot time. The RAM 680 does not have to be a single storage medium, but may be a collection of multiple storage media.

[0134] The storage device 650 mainly functions as database storage. The storage device 650 may be, for example, a magnetic storage device or a semiconductor storage device. An example of a magnetic storage device is a hard disk drive (HDD). An example of a semiconductor storage device is a solid state drive (SSD). The storage device 650 may be a device independent of the management device 600. For example, the storage device 650 may be a storage device connected to the management device 600 via the network 80, such as a cloud storage device.

[0135] The terminal device 400 includes an input device 420, a display device 430, a storage device 450, a processor 460, a ROM 470, a RAM 480, and a communication device 490. These components are communicatively connected to one another via a bus. The input device 420 is a device for converting user instructions into data and inputting the data to a computer. The input device 420 may be, for example, a keyboard, a mouse, or a touch panel. The display device 430 may be, for example, a liquid crystal display or an organic EL display. The processor 460, the ROM 470, the RAM 480, the storage device 450, and the communication device 490 are described in the hardware configuration example of the management device 600, and therefore their description will be omitted.

[0136] [2. Operation] Next, the operations of the work vehicle 100, the terminal device 400, and the management device 600 will be described.

[0137] [2-1.Automatic driving operation] First, an example of the operation of autonomous driving by the work vehicle 100 will be described. The work vehicle 100 in this embodiment can travel autonomously both inside and outside the field. In the field, the work vehicle 100 drives the implement 300 to perform predetermined agricultural work while traveling along a predetermined target route. If the obstacle sensor 130 detects an obstacle while traveling in the field, the work vehicle 100 stops traveling, emits a warning sound from the buzzer 220, and transmits a warning signal to the terminal device 400. In the field, the position of the work vehicle 100 is determined mainly based on data output from the GNSS unit 110. On the other hand, outside the field, the work vehicle 100 travels autonomously along a target route set on a farm road or public road outside the field. The processor 660 of the management device 600 generates a map as shown in FIG. 4 by arranging one or more types of feature block images, and sets a target route on the road on that map. While traveling outside the field, the work vehicle 100 travels while performing local route planning based on data acquired by the camera 120 or LiDAR sensor 140. When the work vehicle 100 detects an obstacle outside the field, it either avoids the obstacle or stops on the spot. Outside the field, the position of the work vehicle 100 is estimated based on the positioning data output from the GNSS unit 110 as well as the data output from the LiDAR sensor 140 or camera 120.

[0138] Below, we will first explain the operation of the work vehicle 100 when it travels automatically within a farm field. The operation of the work vehicle 100 when it travels automatically outside a farm field, and the processing of global route design and local route design outside a farm field will be described later.

[0139] FIG. 14 is a schematic diagram illustrating an example of a work vehicle 100 automatically traveling through a field along a target route. In this example, the field includes a work area 72 where the work vehicle 100 performs work using an implement 300 and a headland 74 located near the outer edge of the field. The user can set in advance which areas of the field on the map correspond to the work area 72 or the headland 74. The target route in this example includes multiple parallel main routes P1 and multiple turning routes P2 connecting the multiple main routes P1. The main routes P1 are located within the work area 72, and the turning routes P2 are located within the headland 74. Although each main route P1 shown in FIG. 14 is a straight route, each main route P1 may also include curved portions. The dashed line in FIG. 14 represents the working width of the implement 300. The working width is set in advance and recorded in the storage device 170. The working width may be set and recorded by the user operating the operation terminal 200 or the terminal device 400. Alternatively, the working width may be automatically recognized and recorded when the implement 300 is connected to the work vehicle 100. The spacing between the multiple main paths P1 may be set to match the working width. A target route may be created based on user operation before automatic driving begins. The target route may be created to cover, for example, the entire work area 72 in a field. The work vehicle 100 automatically travels back and forth from the start point of work to the end point of work along a target route such as that shown in FIG. 14. Note that the target route shown in FIG. 14 is merely an example, and the target route may be defined in any manner.

[0140] Next, an example of control by the control device 180 during automatic operation in a farm field will be described.

[0141] FIG. 15 is a flowchart showing an example of the operation of steering control during automatic driving executed by the control device 180. The control device 180 performs automatic steering by executing the operations of steps S121 to S125 shown in FIG. 15 while the work vehicle 100 is traveling. The speed is maintained at, for example, a preset speed. While the work vehicle 100 is traveling, the control device 180 acquires data indicating the position of the work vehicle 100 generated by the GNSS unit 110 (step S121). Next, the control device 180 calculates the deviation between the position of the work vehicle 100 and the target route (step S122). The deviation represents the distance between the position of the work vehicle 100 at that time and the target route. The control device 180 determines whether the calculated position deviation exceeds a preset threshold (step S123). If the deviation exceeds the threshold, the control device 180 changes the steering angle by changing the control parameters of the steering device included in the drive device 240 so as to reduce the deviation. If the deviation does not exceed the threshold value in step S123, the operation of step S124 is skipped. In the following step S125, the control device 180 determines whether or not a command to end the operation has been received. A command to end the operation may be issued, for example, when a user remotely instructs the work vehicle 100 to stop autonomous driving, or when the work vehicle 100 reaches its destination. If a command to end the operation has not been issued, the process returns to step S121, and the same operation is performed based on the newly measured position of the work vehicle 100. The control device 180 repeats the operations of steps S121 to S125 until a command to end the operation is issued. The above operations are executed by the ECUs 182 and 184 in the control device 180.

[0142] 15, the control device 180 controls the drive device 240 based only on the deviation between the position of the work vehicle 100 identified by the GNSS unit 110 and the target route, but the control may also take into consideration the deviation in heading. For example, when the heading deviation, which is the angular difference between the orientation of the work vehicle 100 identified by the GNSS unit 110 and the direction of the target route, exceeds a preset threshold, the control device 180 may change the control parameters (e.g., steering angle) of the steering device of the drive device 240 in accordance with the deviation.

[0143] An example of steering control by the control device 180 will be described in more detail below with reference to FIGS. 16A to 16D.

[0144] FIG. 16A is a diagram showing an example of a work vehicle 100 traveling along a target route P. FIG. 16B is a diagram showing an example of a work vehicle 100 shifted to the right from the target route P. FIG. 16C is a diagram showing an example of a work vehicle 100 shifted to the left from the target route P. FIG. 16D is a diagram showing an example of a work vehicle 100 facing in an inclined direction with respect to the target route P. In these figures, the pose indicating the position and orientation of the work vehicle 100 measured by the GNSS unit 110 is expressed as r(x, y, θ). (x, y) are coordinates representing the position of the reference point of the work vehicle 100 in the XY coordinate system, which is a two-dimensional coordinate system fixed to the Earth. In the examples shown in FIGS. 16A to 16D, the reference point of the work vehicle 100 is located at the position where the GNSS antenna on the cabin is installed, but the position of the reference point is arbitrary. θ is an angle representing the measured orientation of the work vehicle 100. In the illustrated example, the target path P is parallel to the Y axis, but in general, the target path P is not necessarily parallel to the Y axis.

[0145] As shown in FIG. 16A, if the position and orientation of the work vehicle 100 do not deviate from the target route P, the control device 180 maintains the steering angle and speed of the work vehicle 100 unchanged.

[0146] As shown in Fig. 16B, when the position of work vehicle 100 has shifted to the right from target route P, control device 180 changes the steering angle so that the traveling direction of work vehicle 100 leans leftward and approaches route P. At this time, the speed may also be changed in addition to the steering angle. The magnitude of the steering angle can be adjusted, for example, according to the magnitude of position deviation Δx.

[0147] As shown in Fig. 16C, when the position of the work vehicle 100 has shifted to the left from the target route P, the control device 180 changes the steering angle so that the traveling direction of the work vehicle 100 tilts to the right and approaches the route P. In this case, too, the speed may be changed in addition to the steering angle. The amount of change in the steering angle may be adjusted, for example, according to the magnitude of the position deviation Δx.

[0148] As shown in FIG. 16D , when the position of the work vehicle 100 is not significantly deviated from the target route P but the heading is different from the direction of the target route P, the control device 180 changes the steering angle to reduce the azimuth deviation Δθ. In this case, the speed may also be changed in addition to the steering angle. The magnitude of the steering angle may be adjusted, for example, according to the magnitudes of the position deviation Δx and the azimuth deviation Δθ. For example, the smaller the absolute value of the position deviation Δx, the greater the amount of change in the steering angle according to the azimuth deviation Δθ. When the absolute value of the position deviation Δx is large, the steering angle will be changed significantly to return to the route P, which inevitably increases the absolute value of the azimuth deviation Δθ. Conversely, when the absolute value of the position deviation Δx is small, it is necessary to bring the azimuth deviation Δθ closer to zero. For this reason, it is appropriate to relatively increase the weight of the azimuth deviation Δθ (i.e., the control gain) used to determine the steering angle.

[0149] Control techniques such as PID control or MPC control (model predictive control) can be applied to the steering control and speed control of work vehicle 100. By applying these control techniques, it is possible to smooth the control that brings work vehicle 100 closer to target path P.

[0150] If an obstacle is detected by one or more obstacle sensors 130 while the work vehicle 100 is traveling, the control device 180 will stop the work vehicle 100. At this time, the buzzer 220 may be caused to emit a warning sound or a warning signal may be sent to the terminal device 400. If it is possible to avoid the obstacle, the control device 180 may control the drive device 240 to avoid the obstacle.

[0151] The work vehicle 100 in this embodiment is capable of autonomous driving not only in farm fields but also outside of farm fields. Outside of farm fields, the control device 180 can detect objects (e.g., other vehicles or pedestrians) that are located relatively far from the work vehicle 100 based on data output from the camera 120 or the LiDAR sensor 140. The control device 180 generates a local route to avoid the detected object, and performs speed control and steering control along the local route, thereby realizing autonomous driving on roads outside of farm fields.

[0152] In this manner, the work vehicle 100 in this embodiment can autonomously travel unmanned within and outside a field. FIG. 17 is a diagram schematically illustrating an example of a situation in which multiple work vehicles 100 are autonomously traveling within a field 70 and on a road 76 outside the field 70. An environmental map and a target route for an area including multiple fields and their surrounding roads are stored in the storage device 170. The environmental map and target route can be generated by the management device 600 or the ECU 185. When the work vehicle 100 travels on a road, the work vehicle 100 travels along the target route with the implement 300 raised while sensing its surroundings using sensing devices such as the camera 120 and the LiDAR sensor 140. While traveling, the control device 180 sequentially generates local routes and causes the work vehicle 100 to travel along the local routes. This enables autonomous traveling while avoiding obstacles. The target route may be changed during travel depending on the situation.

[0153] [2-2. Creating a work plan] In this embodiment, the work vehicle 100 automatically moves between fields and performs agricultural work in each field according to a work plan created by the management device 600. The work plan includes information about one or more agricultural works to be performed by the work vehicle 100. For example, the work plan includes information about one or more agricultural works to be performed by the work vehicle 100 and the fields on which each agricultural work will be performed. The work plan may include information about multiple agricultural works to be performed by the work vehicle 100 over multiple work days and the fields on which each agricultural work will be performed. More specifically, the work plan may be a database containing information about a work schedule that indicates which agricultural machine will perform which agricultural work in which field at what time for each work day. The work plan may be created by the processor 660 of the management device 600 based on information input by the user using the terminal device 400.

[0154] FIG. 18 is a diagram showing an example of an agricultural work schedule created by the management device 600. The schedule in this example includes information indicating, for each registered agricultural machine, the date and time the agricultural work will be performed, the field, the work content, and the implement to be used. In addition to the information shown in FIG. 18, the schedule may also include other information depending on the work content, such as the type of pesticide or the amount of pesticide to be sprayed. In accordance with this schedule, the processor 660 of the management device 600 issues agricultural work instructions to the work vehicle 100. The schedule may be downloaded by the control device 180 of the work vehicle 100 and also stored in the storage device 170. In this case, the control device 180 may autonomously start operating according to the schedule stored in the storage device 170.

[0155] In this embodiment, the work plan is created by management device 600, but the work plan may also be created by another device. For example, processor 460 of terminal device 400 or control device 180 in work vehicle 100 may have a function for generating or updating a work plan.

[0156] [2-3. Route planning] Next, the operation of the route planning in this embodiment will be described in more detail.

[0157] In this embodiment, the management device 600 and the control system 160 of the work vehicle 100 work together to function as a route planning system for the work vehicle 100. The storage device 650 stores a map that includes multiple fields and roads around those fields. The processor 660 of the management device 600 generates a map and functions as a processing device that generates a route for the work vehicle 100 on the map. The management device 600 generates a route for the work vehicle 100 based on the map and the work plan. Note that part or all of the map generation processing and route generation processing performed by the management device 600 may be executed by the ECU 185 of the control device 180 of the work vehicle 100. Also, part or all of the map generation processing and route generation processing performed by the management device 600 may be executed by the operation terminal 200 or terminal device 400 of the work vehicle 100.

[0158] FIG. 19 is a diagram showing an example of a map referenced during route planning. This map is a two-dimensional digital map and can be generated or updated by the management device 600 or the ECU 185. The map can be generated by the method described with reference to FIGS. 1 to 7. In this embodiment, before autonomous driving, the work vehicle 100 may manually drive along the planned route while collecting data for map generation using the LiDAR sensor 140 and the camera 120. The map shown in FIG. 19 includes information on the positions (e.g., latitude and longitude) of multiple fields 70 where the work vehicle 100 will perform agricultural work and each point on the roads 76 surrounding them. The map in this example also includes location information for a storage location 90 for the work vehicle 100 and a waiting location 96 where the work vehicle 100 will temporarily wait. The storage location 90 and the waiting location 96 can be registered by a user's operation using the terminal device 400. A map such as the one shown in FIG. 19 can be created for the entire area in which the work vehicle 100 can travel. Although the map shown in FIG. 19 is a two-dimensional map, a three-dimensional map may also be used for route planning.

[0159] The storage location 90 may be, for example, a garage, barn, or parking lot adjacent to the user's home or business. The waiting location 96 may be, for example, a location jointly managed or used by multiple users. The waiting location 96 may be a facility such as a parking lot or garage managed and operated by a city, town, or village, an agricultural cooperative, or a company. If the waiting location 96 is a facility that is locked at night, theft of the work vehicle 100 parked in the waiting location 96 can be prevented. While FIG. 19 illustrates one waiting location 96, multiple waiting locations 96 may be provided. Furthermore, if the work vehicle 100 moves within a relatively small area, there is no need to provide a waiting location 96 separate from the storage location 90.

[0160] Before agricultural work begins on each work day, the management device 600 reads from the storage device 650 a map of the area including the field where agricultural work is scheduled to be performed on that work day, and generates a route for the work vehicle 100 based on that map. More specifically, the management device 600 generates a first route (also referred to as a "work travel route") in the field 70 on the map along which the work vehicle 100 will travel while performing agricultural work in that field, and also generates a second route on the map along which the work vehicle 100 will travel toward the field 70. After generating the first and second routes, the management device 600 connects the two to generate a global route for the work vehicle 100.

[0161] FIG. 20 is a diagram showing an example of a generated global route. In FIG. 20, of the global routes generated by the management device 600, a route generated on a road 76 is indicated by an arrow. The first route (work travel route) generated within the field 70 is omitted from the illustration. The work travel route is, for example, a route such as that shown in FIG. 14, and is connected to the second route generated on the road 76. The arrows in FIG. 20 show an example of a route for the work vehicle 100 on a certain work day.

[0162] In the example shown in FIG. 20 , the management device 600 generates a route for one work day that starts from the storage location 90 and passes through eight fields 70 in order to reach the waiting location 96. This is because the waiting location 96 is the closest waiting location to the group of fields where work is scheduled for the next work day. If the group of fields where work is scheduled for the next work day is close to the storage location 90, the management device 600 may return the work vehicle 100 to the storage location 90 after the work for the day is completed. In this way, the management device 600 may generate a route from the last field where work is performed on each work day to the waiting location 96 that is the shortest on average distance from the group of fields where work will be performed on the next work day. The work vehicle 100 travels along the generated route.

[0163] In the example shown in FIG. 20 , on a certain work day, the work vehicle 100 departs from the storage location 90 and sequentially visits the group of fields where work is scheduled for that day, performing the work indicated in the schedule in each field. In each field, the work vehicle 100 performs the work while automatically traveling along the work travel route, for example, using the method described with reference to FIGS. 14 to 16D . When work in one field is completed, the work vehicle 100 enters the next field and performs the work in the same manner. In this way, when work in the last field of the day is completed, the work vehicle 100 moves to the waiting location 96. The work vehicle 100 waits at the waiting location 96 until the next work day. On the next work day, the work vehicle 100 departs from the waiting location 96 and sequentially visits the group of fields where work is scheduled for that day, performing the work indicated in the schedule in each field. By performing such operations, the planned farm work can be completed while the work vehicle 100 is moved efficiently along the optimal route according to the schedule.

[0164] When the work vehicle 100 is traveling outside a field, an obstacle such as a pedestrian or another vehicle may be present on or near the global route. To prevent the work vehicle 100 from colliding with the obstacle, the ECU 185 in the control device 180 sequentially generates a local route that can avoid the obstacle while the work vehicle 100 is traveling. The ECU 185 generates the local route based on sensing data acquired by sensing devices (such as the obstacle sensor 130, the LiDAR sensor 140, and the camera 120) equipped on the work vehicle 100 while the work vehicle 100 is traveling. The local route is defined by multiple waypoints along a portion of the global route. The ECU 185 determines whether an obstacle is present on or near the path of the work vehicle 100 based on the sensing data. If such an obstacle is present, the ECU 185 sets multiple waypoints to avoid the obstacle and generates a local route. If no obstacles are present, ECU 185 generates a local route that is approximately parallel to the global route. Information indicating the generated local route is sent to ECU 184 for automatic driving control. ECU 184 controls ECU 181 and ECU 182 so that work vehicle 100 travels along the local route. This allows work vehicle 100 to travel while avoiding obstacles. Note that if there are traffic lights on the road on which work vehicle 100 is traveling, work vehicle 100 may, for example, recognize the traffic lights based on images captured by camera 120, stop when the light is red, and start when the light is green.

[0165] FIG. 21 is a diagram illustrating an example of a global route and a local route generated in an environment where obstacles are present. In FIG. 21, a global route 31 is illustrated by a dotted arrow, and a local route 32, which is generated sequentially during travel, is illustrated by a solid arrow. The global route 31 is defined by multiple waypoints 31p. The local route 32 is defined by multiple waypoints 32p set at intervals shorter than the waypoints 31p. Each waypoint has, for example, position and orientation information. The management device 600 generates the global route 31 by setting multiple waypoints 31p at multiple locations, including intersections on a road 76. The intervals between the waypoints 31p may be relatively long, for example, several meters to several tens of meters. The ECU 185 generates the local route 32 by setting multiple waypoints 32p based on sensing data output from the sensing device while the work vehicle 100 is traveling. The intervals between waypoints 32p on the local route 32 are shorter than the intervals between waypoints 31p on the global route 31. The intervals between waypoints 32p can be, for example, several tens of centimeters (cm) to several meters (m). The local route 32 is generated within a relatively small range (for example, a range of about several meters) starting from the position of the work vehicle 100. FIG. 21 shows an example of a series of local routes 32 generated while the work vehicle 100 travels along a road 76 between fields 70 and turns left at an intersection. While the work vehicle 100 is moving, the ECU 185 repeats the operation of generating a local route from the position of the work vehicle 100 estimated by the ECU 184 to a point, for example, several meters ahead. The work vehicle 100 travels along the local routes that are successively generated.

[0166] In the example shown in FIG. 21 , an obstacle 40 (e.g., a person) is present ahead of the work vehicle 100. In FIG. 21 , an example of a range sensed by a sensing device such as the camera 120, the obstacle sensor 130, or the LiDAR sensor 140 mounted on the work vehicle 100 is illustrated as a sector. In such a situation, the ECU 185 generates a local route 32 so as to avoid the obstacle 40 detected based on the sensing data. The ECU 185 determines whether there is a possibility that the work vehicle 100 will collide with the obstacle 40, for example, based on the sensing data and the width of the work vehicle 100 (including the width of the implement if one is attached). If there is a possibility that the work vehicle 100 will collide with the obstacle 40, the ECU 185 sets multiple waypoints 32p so as to avoid the obstacle 40, and generates the local route 32. Note that the ECU 185 may recognize not only the presence or absence of an obstacle 40 but also the state of the road surface (for example, mud, depressions, etc.) based on sensing data, and if a location where travel is difficult is detected, it may generate a local route 32 to avoid such a location. The work vehicle 100 travels along the local route 32. If the obstacle 40 cannot be avoided no matter how the local route 32 is set, the control device 180 may stop the work vehicle 100. At this time, the control device 180 may send a warning signal to the terminal device 400 to alert the monitor. After stopping, if it is recognized that the obstacle 40 has moved and there is no longer any risk of collision, the control device 180 may resume travel of the work vehicle 100.

[0167] [2-4. Map Update] As described above, global path planning is performed based on a map of the environment in which the work vehicle 100 travels. The map includes position information for multiple farm fields and the roads surrounding the fields in the area in which the work vehicle 100 travels. The map may also include information indicating the distribution of features (e.g., grass, trees, waterways, buildings, etc.) around the road. The information indicating the distribution of features may be generated based on data obtained by sensing with a sensing device such as a LiDAR sensor or a camera while the work vehicle 100 or other mobile object is moving. In an environment where vegetation grows along the road or where tall buildings exist near the road, reception of radio waves from GNSS satellites may be hindered, and travel of the work vehicle 100 may be hindered. For example, in an environment where trees are densely distributed around farm roads, the leaves growing at the top of the trees form a canopy that acts as an obstacle or multiple reflector for radio waves from the satellites. Accurate positioning using GNSS is difficult on such farm roads. To avoid generating a route on such roads, ECU 186 may recognize structures based on sensing data acquired by a sensing device while work vehicle 100 is traveling autonomously, and reflect the recognized structures on the map. Recognized structures may be, for example, trees, buildings, waterways, signs, etc. ECU 186 may also recognize at least one of the condition of the road on which work vehicle 100 is traveling and the condition of vegetation around work vehicle 100 based on the sensing data, and reflect the recognized conditions on the map.

[0168] In an environment where vegetation grows densely around roads, as in the above example, the condition of the vegetation varies depending on the season, and the ease of passage may change. For example, the proportion of roads covered with vegetation is higher in summer than in winter, and the number of roads that are difficult to pass tends to increase. Therefore, the ECU 186 may generate multiple maps corresponding to the seasons and record them in the storage device 170. For example, the ECU 186 may generate four types of maps: spring, summer, autumn, and winter. These maps share common geographic information such as fields, roads, waiting areas, and buildings, but may differ in attribute information regarding the state of vegetation or the passability of roads. The multiple generated maps are transmitted to the management device 600 and recorded in the storage device 650. The management device 600 may extract, from the multiple recorded maps, a map corresponding to the season in which the work vehicle 100 will perform agricultural work, and generate a route for the work vehicle 100 based on the extracted map. This makes it possible to realize appropriate global route design according to the season.

[0169] (Other embodiments) The configurations and operations of the above-described embodiments are merely examples, and the present disclosure is not limited to the above-described embodiments. Other embodiments will be described below.

[0170] In the above embodiment, the processor 660 of the management device 600 creates a work plan, generates an environmental map, and designs a global route for the work vehicle 100, and the control device 180 inside the work vehicle 100 designs a local route and controls the travel of the work vehicle 100. Alternatively, some of the operations of the management device 600 described above may be executed by the control device 180, the operation terminal 200, or the terminal device 400. For example, the generation of the environmental map and the generation of the global route may be executed by the control device 180, the operation terminal 200, or the terminal device 400.

[0171] The management device 600 may manage the operations of multiple agricultural machines including the work vehicle 100. In this case, the management device 600 may perform global route planning and driving instructions for each agricultural machine based on the schedule of farm work to be performed by each agricultural machine.

[0172] The systems for generating maps, planning routes, or performing automatic driving control in the above embodiments can also be retrofitted to agricultural machines that do not have these functions. Such systems can be manufactured and sold independently of the agricultural machines. The computer programs used in such systems can also be manufactured and sold independently of the agricultural machines. The computer programs can be provided, for example, by being stored on a computer-readable non-transitory storage medium. The computer programs can also be provided by downloading via a telecommunications line (for example, the Internet).

[0173] As described above, a map generation system according to an embodiment of the present disclosure is a system for generating map data for an agricultural machine that automatically travels on roads around a field. The map generation system includes a storage device that stores multiple feature block images associated with multiple types of features that may be present around the road, and a processing device. The processing device acquires position distribution data of one or more types of features present around the road, the position distribution data being generated based on at least one of sensor data from the LiDAR sensor and image data from the image capture device output by a mobile object equipped with at least one of a LiDAR sensor and an image capture device while the mobile object is traveling along the road. The processing device reads one or more types of feature block images associated with the one or more types of features from the storage device, and arranges the feature block images according to the position distribution data to generate map data of an area including the road.

[0174] According to the above configuration, it is possible to efficiently generate map data suitable for route planning for agricultural machinery that travels automatically on roads around a field.

[0175] The processing device may display on a display a graphical user interface (GUI) for setting the width and height of each of the plurality of feature block images, and may generate the map data by arranging the one or more types of feature block images having the width and height set on the GUI.

[0176] According to the above configuration, the system administrator or the agricultural machine user can set the width and height of each feature block image on the GUI. This allows the feature block image to be set to an appropriate size depending on the type of feature, making it easier to generate a suitable map.

[0177] The processing device may generate the map data for each of the one or more types of features by a process including the following steps (S11) to (S14). (S11) Based on the position distribution data, the horizontal length corresponding to the width direction and the vertical length corresponding to the height direction of the area occupied by the feature are determined. (S12) The horizontal length is divided by the width of the feature block image corresponding to the feature to determine the horizontal number of the feature block images. (S13) The vertical length is divided by the height of the feature block image to determine the vertical number of the feature block images. (S14) The feature block images are arranged in the horizontal direction by the number of horizontal pieces, and the feature block images are arranged in the vertical direction by the number of vertical pieces.

[0178] The above process allows the generation of a map by appropriately arranging the corresponding feature block images according to the horizontal and vertical lengths of the continuous area occupied by each feature. Note that the processing device may apply the above process after dividing the continuous area occupied by each feature into multiple rectangular or strip-shaped areas. This process allows the appropriate arrangement of feature block images regardless of the shape of the area occupied by the feature.

[0179] The processing device may recognize the one or more types of features present around the road based on at least one of the sensor data and the image data, and generate the position distribution data of the one or more types of features based on the GNSS data output from a GNSS receiver equipped on the mobile body and the sensor data.

[0180] According to the above configuration, the processing device can generate position distribution data by itself, and generate map data based on the position distribution data and one or more types of feature block images.

[0181] The processing device may recognize one or more types of features including at least one of grass, trees, ditches, waterways, fields, and buildings present around the road based on at least one of the sensor data and the image data, and may generate the map data by arranging the one or more types of feature block images associated with the recognized one or more types of features within an area on the map data where the features exist, according to the position distribution data.

[0182] According to the above configuration, map data can be generated based on one or more types of feature block images corresponding to at least one of grass, trees, ditches, waterways, fields, and buildings, thereby enabling the generation of suitable map data that reflects the types and locations of features around roads on which agricultural machinery travels.

[0183] The processing device may recognize the one or more types of features present around the road by pattern matching or segmentation based on the image data.

[0184] According to the above process, features can be recognized more accurately from image data.

[0185] The processing device may detect a width of the road based on at least one of the sensor data and the image data, and generate the map data that reflects the width of the road.

[0186] According to the above process, map data that more accurately reflects road widths can be generated, making it possible to more appropriately execute processes such as route planning based on map data.

[0187] The processing device may detect a boundary between the road and the one or more types of features based on at least one of the sensor data and the image data, arrange the one or more types of feature block images associated with the one or more types of features along the boundary outside the boundary, and place an image representing the road inside the boundary, thereby generating the map data.

[0188] According to the above process, even if feature block images corresponding to roads are not prepared, suitable map data can be generated.

[0189] The processing device may generate the position distribution data indicating a distribution of latitudes and longitudes of the one or more types of features based on the GNSS data and the sensor data.

[0190] This makes it possible to generate map data that more accurately reflects the distribution of latitudes and longitudes of one or more types of features based on the position distribution data.

[0191] The storage device may store a plurality of pixel arts representing the plurality of types of features as the plurality of feature block images, and the processing device may generate the map data by arranging the pixel arts.

[0192] This makes it possible to efficiently generate small-sized map data expressed in pixel art.

[0193] The storage device may store a road block image associated with the road as one of the plurality of feature block images. The processing device may generate the map data by arranging the one or more types of feature block images associated with the one or more types of features other than roads and the road block image associated with the road in accordance with the position distribution data.

[0194] According to the above process, suitable map data can be generated by arranging road block images and other feature block images.

[0195] The processing device may generate a route for the agricultural machine to automatically travel along the road based on the map data. The processing device may transmit data indicating the route to the agricultural machine.

[0196] The map generation system may further include the LiDAR sensor and the imaging device.

[0197] The map generation system may further include the mobile body.

[0198] The mobile object may be the agricultural machine. The mobile object may be different from the agricultural machine.

[0199] A map generation method according to another embodiment of the present disclosure is a method for generating map data for an agricultural machine that automatically travels on a road around a field. The map generation method includes acquiring position distribution data of one or more types of features present around the road, the position distribution data being generated based on at least one of sensor data from the LiDAR sensor and image data from the image capture device output by a mobile object equipped with at least one of a LiDAR sensor and an image capture device while the mobile object is moving along the road, reading out one or more types of feature block images associated with the one or more types of features from a storage device, and arranging the feature block images according to the position distribution data to generate map data of an area including the road.

[0200] A map generation program according to yet another embodiment of the present disclosure is a computer program for generating map data for an agricultural machine that automatically travels on roads around a field. The program is stored on a computer-readable non-transitory storage medium. The program causes the computer to acquire position distribution data of one or more types of features present around the road, the position distribution data being generated based on at least one of sensor data from the LiDAR sensor and image data from the image capture device output by a mobile object equipped with at least one of a LiDAR sensor and an image capture device while the mobile object is traveling along the road; read one or more types of feature block images associated with the one or more types of features from a storage device; and generate map data of an area including the road by arranging the feature block images according to the position distribution data. [Industrial Applicability]

[0201] The techniques of the present disclosure can be applied to path planning systems for agricultural machines such as tractors, harvesters, rice transplanters, riding tillers, vegetable transplanters, mowers, seed sowing machines, fertilizer applicators, or agricultural robots. [Explanation of symbols]

[0202] 10... storage device, 20... processing device, 30... mobile object, 31... global route, 32... local route, 33... LiDAR sensor, 34... camera, 35... GNSS unit, 40... obstacle, 50... GNSS satellite, 60... reference station, 70... field, 72... work area, 74... headland, 76... road, 80... network, 90... storage area, 96... waiting area, 100... work vehicle, 101... vehicle body, 102... engine, 103... transmission, 104... wheels, 105... cabin, 106... steering device, 107... driver's seat, 108... coupling device, 110... GNSS unit111···GNSS receiver, 112···RTK receiver, 115···Inertial measurement unit (IMU), 116···Processing circuit, 120···Camera, 130···Obstacle sensor, 140···LiDAR sensor, 150···Sensor group, 152···Steering wheel sensor, 154···Turning angle sensor, 156···Axle sensor, 160···Control system, 170···Storage device, 180···Control device, 181 to 186···ECU 190....Communication device, 200...Operation terminal, 210...Operation switch group, 220...Buzzer, 240...Drive device, 300...Implement, 340...Drive device, 380...Control device, 390...Communication device, 400...Terminal device, 420...Input device, 430...Display device, 450...Storage device, 460...Processor, 470...ROM, 480...RAM, 490...Communication device, 600...Management Device , 650...Storage device, 660 processor 、6 70···ROM, 680···RAM, 690···Communication device, 1000···Map generation system

Claims

1. A map generation system that generates map data for an agricultural machine that automatically travels on roads around a field, a storage device that stores a plurality of feature block images that correspond to a plurality of types of features that may exist around the road; a processing device; Equipped with The processing device includes: displaying a graphical user interface (GUI) on a display for setting the width and height of each of the plurality of feature block images; acquiring position distribution data of one or more types of features present around the road, the position distribution data being generated based on at least one of sensor data from the LiDAR sensor and image data from the imaging device output while a mobile body equipped with at least one of a LiDAR sensor and an imaging device is moving along the road; reading out from the storage device one or more types of feature block images associated with the one or more types of feature; generating map data of an area including the road by arranging the one or more types of feature block images having widths and heights set on the GUI in accordance with the position distribution data; Map generation system.

2. The processing device includes: For each of the one or more types of features: determining a horizontal length corresponding to the width direction and a vertical length corresponding to the height direction of an area occupied by the feature based on the position distribution data; Dividing the horizontal length by the width of a feature block image corresponding to the feature Therefore, the horizontal number of the feature block image is determined, determining the vertical number of the feature block images by dividing the vertical length by the height of the feature block images; arranging the feature block images in the horizontal direction by the number of horizontal images; The feature block images are arranged in the vertical direction by the number of vertical images. generating the map data by a process including: The map generation system of claim 1 .

3. The processing device includes: Recognizing the one or more types of features present around the road based on at least one of the sensor data and the image data; generating the position distribution data of the one or more types of features based on the GNSS data output from a GNSS receiver included in the mobile object and the sensor data; The map generation system of claim 1 .

4. The processing device includes: recognize one or more types of features including at least one of grass, trees, ditches, waterways, fields, and buildings present around the road based on at least one of the sensor data and the image data; generating the map data by arranging the one or more types of feature block images associated with the one or more types of recognized features within an area on the map data where the features exist, in accordance with the position distribution data; The map generation system according to claim 3 .

5. The map generating system according to claim 3 , wherein the processing device recognizes the one or more types of features present around the road by pattern matching or segmentation based on the image data.

6. The processing device includes: Detecting the width of the road based on at least one of the sensor data and the image data; generating the map data that reflects the width of the road; The map generation system according to claim 3 .

7. The processing device includes: detecting a boundary between the road and the one or more types of feature based on at least one of the sensor data and the image data; generating the map data by arranging the one or more types of feature block images associated with the one or more types of feature along the boundary outside the boundary, and arranging the image representing the road inside the boundary; The map generation system according to claim 3 .

8. The map generation system according to claim 3 , wherein the processing device generates the position distribution data indicating a distribution of latitudes and longitudes of the one or more types of features based on the GNSS data and the sensor data.

9. the storage device stores a plurality of pixel arts representing the plurality of types of features as the plurality of feature block images; the processing device arranges the pixel art to generate the map data; The map generation system of claim 1 .

10. the storage device stores a road block image associated with the road as one of the plurality of feature block images; the processing device generates the map data by arranging the one or more types of feature block images associated with the one or more types of feature other than roads and the road block images associated with the roads in accordance with the position distribution data. The map generation system of claim 1 .

11. The map generating system according to claim 1 , wherein the processing device generates a route for the agricultural machine to automatically travel along the road based on the map data.

12. A map generation system that generates map data for an agricultural machine that automatically travels on roads around a field, a storage device that stores a plurality of feature block images that correspond to a plurality of types of features that may exist around the road; a processing device; Equipped with The processing device includes: Recognizing one or more types of features present around the road based on at least one of sensor data from the LiDAR sensor and image data from the imaging device output while a mobile body equipped with at least one of a LiDAR sensor and an imaging device is moving along the road; generating position distribution data of the one or more types of features based on the GNSS data output from a GNSS receiver equipped in the mobile object and the sensor data; reading out from the storage device one or more types of feature block images associated with the one or more types of feature; arranging the feature block images in accordance with the position distribution data to generate map data for an area including the road; Map generation system.

13. the LiDAR sensor; and the imaging device; The map generation system of claim 1 , further comprising:

14. The map generating system according to claim 1 , further comprising the moving body.

15. The map generation system according to claim 14 , wherein the mobile object is the agricultural machine.

16. 1. A map generation method executed by a computer for generating map data for an agricultural machine that automatically travels on roads around a field, comprising: displaying on a display a graphical user interface (GUI) for setting the width and height of each of a plurality of feature block images respectively associated with a plurality of types of feature that may exist around the road; Acquiring position distribution data of one or more types of features present around the road, the position distribution data being generated based on at least one of sensor data from the LiDAR sensor and image data from the image capture device output while a mobile body equipped with at least one of a LiDAR sensor and an image capture device is moving along the road; reading out from a storage device one or more types of feature block images associated with the one or more types of feature; generating map data of an area including the road by arranging the one or more types of feature block images having widths and heights set on the GUI in accordance with the position distribution data; A map generation method including:

17. 1. A map generation method executed by a computer for generating map data for an agricultural machine that automatically travels on roads around a field, comprising: Recognizing one or more types of features present around the road based on at least one of sensor data from the LiDAR sensor and image data from the imaging device output while a mobile body equipped with at least one of a LiDAR sensor and an imaging device is moving along the road; and generating position distribution data of the one or more types of features based on the GNSS data output from a GNSS receiver included in the mobile object and the sensor data; reading out from a storage device one or more types of feature block images associated with the one or more types of feature; generating map data of an area including the road by arranging the feature block images according to the position distribution data; A map generation method including:

Images