Work vehicle

Abstract

To provide a working vehicle capable of improving safety by accurately calculating a distance to a person from image information captured by a camera.SOLUTION: A work vehicle 1 configured to be capable of autonomously driving in a field includes a foreground imaging camera 13a configured to generate foreground image data obtained by imaging a front side of the work vehicle 1, an image analysis unit configured to perform image analysis on the foreground image data, and a person position estimation model configured to receive input of attribute information including information relating to a position and a size of a person captured in the foreground image data by the image analysis unit and output an estimation value of a direction and a distance of the person with respect to a machine body included in the foreground image data, the estimation value being calculated based on a region of the person image occupying in the foreground image data. The above problem is solved by a work vehicle 1 configured to perform an avoidance operation for avoiding contact with the captured person based on an estimation value of a direction and a distance of the captured person with respect to a machine body, the estimation value being output by a person position estimation model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a work vehicle that performs agricultural work while automatically driving in a field. [Background technology]

[0002] For example, as shown in Patent Document 1 below, there is known a work vehicle that can switch between manual driving, where an operator operates a steering wheel (so-called handle), and automatic driving, where the steering wheel is automatically steered by an on-board computer. This conventional work vehicle calculates its own position using position information obtained from a satellite positioning system, and automatically controls the steering angle of the steering wheel to eliminate any deviation between the calculated own position and a target travel route, thereby enabling the vehicle to perform agricultural work (hereinafter sometimes simply referred to as work) while automatically traveling through a field along a target travel route.

[0003] Furthermore, as shown in Patent Document 2 below, for example, conventionally, work vehicles are known that are equipped with detection means such as ultrasonic sonar, laser scanners, and cameras on the vehicle body in order to detect people and obstacles around the work vehicle while it is traveling. Patent Document 3 also discloses technology that recognizes and avoids people and obstacles based on image information captured by a camera during autonomous driving. In recent years, against the backdrop of developments in artificial intelligence, various efforts have been made in the field of agricultural machinery and implements to analyze and utilize image information captured by cameras. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-069291 [Patent Document 2] Japanese Patent Application Publication No. 2018-116611 [Patent Document 3] Japanese Patent Publication No. 2022-123742 Summary of the Invention [Problem to be solved by the invention]

[0005] However, if conventional work vehicles are configured to simply recognize and avoid people or obstacles that need to be avoided based on image information captured by a camera during autonomous driving, it is difficult to accurately calculate the distance to the person or obstacle from the image information, which increases the risk of contact with these objects and reduces safety. This problem is particularly pronounced when the object to be avoided is a person, as not only the work vehicle but also the person is moving.

[0006] Therefore, the present invention aims to solve such problems and provide a work vehicle that can accurately calculate the distance to a person from image information captured by a camera, prevent contact, and improve safety. [Means for solving the problem]

[0007] In order to achieve the above object, the first invention is: A work vehicle comprising a traveling body that travels in a field, a work implement attached to the traveling body, a positioning device that measures the position of the work implement itself, and a steering device that steers a steering wheel, and is configured to automatically steer the steering wheel by controlling the steering device based on position information of the work implement itself acquired by the positioning device, thereby enabling automatic driving in a field, a front view photographing camera that photographs an area ahead of the work vehicle and generates front view image data; an image analysis unit that performs image analysis on the foreground image data; a human position estimation model that is calculated by the image analysis unit based on the area of ​​a human image that occupies an image of the foreground image data, receives input of attribute information including information about the location and size of a person photographed by the foreground photographing camera, and outputs estimated values ​​of the direction and distance of the photographed person relative to the aircraft; A work vehicle is provided that is configured to perform avoidance operations to avoid contact with the photographed person based on the estimated values ​​of the direction and distance of the photographed person relative to the vehicle body output by the human position estimation model.

[0008] According to the first invention, by performing evasive action to avoid contact with a person based on the estimated values ​​of the direction and distance of the photographed person relative to the aircraft output by the person position estimation model, the distance to the person can be accurately calculated from the image information captured by the camera, preventing contact and improving safety.

[0009] The second invention is the first invention, a steering wheel photographing camera that photographs an area in front of the steering wheel and generates steering wheel image data; a target marker that is disposed within a photographing range of the steering wheel photographing camera; and a steering angle estimation model that receives input of the steering wheel image data and outputs an estimated value of the steering angle of the steering wheel, The vehicle is characterized in that it is configured to perform automatic driving based on the estimated steering angle value output by the steering angle estimation model.

[0010] According to the second invention, in addition to the effects of the first invention, By performing automatic driving based on the estimated steering angle value output by the steering angle estimation model, the possibility of the work vehicle coming into contact with people can be further reduced, thereby improving safety.

[0011] The third invention is the first or second invention, The image analysis unit is configured to determine whether the person photographed is an adult or a child from the area of ​​the human image, and when performing obstacle avoidance operations during autonomous driving, if the image analysis unit determines that the person photographed is a child, it is configured to increase the steering angle during detouring compared to when it determines that the person is an adult.

[0012] According to the third invention, in addition to the effects of the first or second invention, When the work vehicle determines that a person photographed by the front view camera is a child, it will avoid the person by steering at a larger angle during detouring when performing obstacle avoidance operations during autonomous driving than when it determines that the person is an adult. As a result, when a child approaches the vehicle, it will detour at a larger angle than when it determines that the person is an adult, thereby reducing the risk of contact and further improving safety. [Effects of the Invention]

[0013] According to the present invention, it is possible to provide a work vehicle that can accurately calculate the distance to a person or an obstacle from image information captured by a camera, thereby improving safety. Work vehicles can be provided. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a left side view of a work vehicle according to an embodiment of the present invention. [Figure 2] FIG. 2 is a plan view of the same. [Figure 3] FIG. 3 is an external view of the cockpit interior as seen from the cockpit. [Figure 4] FIG. 4 is a partially enlarged view of the steering wheel periphery of FIG. [Figure 5A] FIG. 5A is a block diagram showing the configuration of a control system including a control device for a work vehicle. [Figure 5B] FIG. 5B is a block diagram of the same. [Figure 6] FIG. 6 is an image diagram of an example of foreground image data captured by a foreground camera. [Figure 7] Fig. 7(a) is an image diagram of an example of obstacle extraction image data extracted by an obstacle extraction unit from the foreground image data of Fig. 6. Fig. 7(b) is an image diagram for explaining attribute information calculated by an attribute information calculation unit from the obstacle extraction image data of Fig. 7(a). [Figure 8] FIG. 8 is an image diagram showing a case where a marker extraction image is generated from steering wheel image data by the marker extraction image generating unit. [Figure 9] FIG. 9 is a block diagram for explaining the function of the human position estimation unit. [Figure 10] FIG. 10 is a block diagram for explaining the function of the steering angle estimating unit. [Figure 11] FIG. 11 is a schematic diagram of a model showing inputs and outputs of a human position estimation model. [Figure 12] FIG. 12 is a model schematic diagram showing inputs and outputs of the steering angle estimation model. [Figure 13] Figure 13 is a schematic explanatory diagram showing an example of how an AI camera is used. [Figure 14] FIG. 14 is a schematic explanatory diagram of the same. [Figure 15] FIG. 15 is a schematic explanatory diagram of the same. [Figure 16] FIG. 16 is a schematic explanatory diagram of the same. [Figure 17] FIG. 17 is a schematic explanatory diagram of the same. [Figure 18] FIG. 18 is a schematic explanatory diagram of the same. DETAILED DESCRIPTION OF THE INVENTION

[0015] A preferred embodiment of the present invention will be described below with reference to the accompanying drawings. In the following description, unless otherwise specified, the forward direction of the work vehicle 1 (the direction from the operator's seat 8 to the steering wheel 9, which will be described later) is referred to as the front, the opposite direction is referred to as the rear, and the right side when facing forward is referred to as the right, and the left side is referred to as the left. The main body of the work vehicle 1 may also be referred to as the fuselage.

[0016] <1. Basic configuration of work vehicle> First, the basic configuration of a work vehicle 1 according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a left side view of the work vehicle 1 according to an embodiment, and Fig. 2 is a plan view. Note that the following description will be given using a tractor as an example of the work vehicle 1. Therefore, in the following description, the work vehicle 1 will mainly be referred to as a tractor 1.

[0017] The tractor 1, which is a work vehicle, is an agricultural tractor that travels autonomously to perform work in fields, etc. The tractor 1 is operated by an operator (also called an operator) and travels within the field to perform predetermined tasks, and is also configured to perform predetermined tasks while automatically driving within the field by controlling each part using a control system centered on a control device C (see Figure 3), which will be described later, that is disposed at an appropriate position on the vehicle body.

[0018] As shown in Fig. 1, the tractor 1 comprises a traveling body 2, which is a vehicle body that can travel, and a work implement W attached to the traveling body 2. The traveling body 2 comprises a body frame 3, front wheels 4, rear wheels 5, a bonnet 6, an engine E, a control section 7, and a transmission case 10. The body frame 3 and the transmission case 10 form the vehicle body skeleton and function as the main frame of the traveling body 2.

[0019] The front wheels 4 are a pair of left and right wheels, and are primarily used for steering (i.e., steering wheels). The rear wheels 5 are a pair of left and right wheels, and are primarily used for driving (i.e., driving wheels). The tractor 1 may be configured to be switchable between two-wheel drive (2WD) in which the rear wheels 5 are driven, and four-wheel drive (4WD) in which both the front wheels 4 and the rear wheels 5 are driven. In this case, both the front wheels 4 and the rear wheels 5 are driven wheels. The traveling body 2 may be equipped with crawler devices instead of wheels (front wheels 4 and rear wheels 5). In this case, the traveling crawlers function as driving wheels.

[0020] The hood 6 is provided at the front of the traveling vehicle body 2 so as to be able to be opened and closed freely. The hood 6 can be rotated (opened and closed) in the vertical direction with the rear part as the rotation center. When closed, the hood 6 covers the engine E mounted on the vehicle body frame 3. The engine E is the driving source of the tractor 1 and is a heat engine such as a diesel engine or a gasoline engine.

[0021] The control unit 7 functions to control the work vehicle 1 by receiving operations from the worker, and a control room 7r is provided in a cabin box 7a that covers the top of the traveling vehicle body 2, forming a cabin. A control seat 8 where the worker sits is disposed within the control room 7r, and various operating members that receive operations from the worker, such as a steering wheel 9, are disposed in front of the control seat 8, and an air conditioning unit 7e is disposed behind the control seat 8. The steering wheel 9 is a member that steers the front wheels 4, which are steered wheels, and is manually steered by the worker during manual operation, and is automatically steered by a steering device 31 that is configured including a steering actuator and the like (not shown) during automatic operation. The configuration of the control room 7r will be described later.

[0022] A steering angle sensor 25 is provided at the rotation base of the steering wheel 9, which is capable of detecting a steering angle corresponding to the direction of rotation and amount of operation of the steering wheel 9 (see FIG. 5A). This steering angle sensor 25 is configured, for example, with a rotary encoder or the like, and is capable of detecting the angle of rotation (steering angle) of the input shaft (not shown) of the steering wheel 9 about the rotation axis. Since the traveling direction of the work vehicle A is determined according to this steering angle, a control device C, which will be described later, acquires information related to the detection value of the steering angle sensor 25 (hereinafter referred to as steering angle data Da2) and controls the steering angle of the steering wheel 9 using the steering device 31, thereby making it possible to control the traveling direction of the work vehicle A. The control device C appropriately stores the acquired steering angle data Da2 together with the time of acquisition.

[0023] The transmission case 10 houses a transmission (a speed change device 32). The power (rotational power) output from the engine E is appropriately reduced (shifted) by the transmission and transmitted to the front wheels 4 and rear wheels 5 via the front axle 4j and rear axle 5j, as well as to the PTO shaft 16. A PTO clutch, a PTO speed change device, and an auto-brake device (braking device) 33 (not shown) are also housed within the transmission case 10, making it possible to control the transmission (on / off, shifting) of power to the PTO shaft. This enables the work vehicle 1 to control the drive of the work implement W.

[0024] A work implement W that performs work in the field is connected to the rear of the traveling body 2, and a lifting device 12 that raises and lowers the work implement W is provided. Furthermore, a PTO shaft 16 that transmits power to drive the work implement W is disposed so as to protrude rearward from the transmission case 10. The PTO shaft 16 transmits rotational power that has been appropriately reduced by the transmission to the work implement W connected to the rear of the traveling body 2.

[0025] The lifting device 12 includes a hydraulic lifting cylinder 121, a lift arm 122, a lift rod 123, a lower link 124, and a top link 125. The lifting device 12 can move the work implement W to a non-working position by raising it. The non-working position is a position of the work implement W that is raised when, for example, the traveling body 2 moves backward or turns, and the work implement W is supported by the vehicle body in the air away from the field. The lifting device 12 can also move the work implement W to a ground work position by lowering it. The ground work position is a position of the work implement W that has been lowered to perform work, and the work implement W is supported by the vehicle body while in contact with the field.

[0026] When hydraulic oil is supplied to the lift cylinder 121, the lift arm 122 rotates around the axis AX serving as the rotation fulcrum to raise the work implement W, and when hydraulic oil is discharged from the lift cylinder 121, the lift arm 122 rotates around the axis AX to lower the work implement W. A lift arm sensor 26 that detects the rotation angle of the lift arm 122 is provided at the base of the lift arm 122 (near the axis AX). The height of the work implement W is calculated by the control device C based on the detection result of the lift arm sensor 26.

[0027] The lift arm 122 is connected to the lower link 124 via the lift rod 123. In this way, the lifting device 12 connects the work machine W to the traveling body 2 via the lower link 124 and the top link 125 so that the work machine W can be raised and lowered. The lower link 124 is attached to the rear of the transmission case 10.

[0028] Although not described in detail, the tractor 1 is also equipped with an engine rotation sensor 23 that detects the rotation speed of the engine E and a vehicle speed sensor 24 that detects the vehicle speed, in addition to the above-mentioned lift arm sensor 26 (see FIG. 5A). Also, although not shown, a turning angle sensor that detects the turning angle of the front wheels 4, which are steered wheels, and a lever sensor that detects the operating positions of various operating levers such as the sub-transmission lever 14 are disposed at appropriate positions. The control device C is configured to acquire the detection information of these sensors via wired or wireless communication.

[0029] The positioning device 30 is a device that measures the position of the vehicle (its own vehicle) and is configured, for example, to include a GNSS (Global Navigation Satellite System) antenna. It can receive radio waves from navigation satellites S orbiting the sky to measure position and time. It can also calculate the vehicle speed from the history of positioning results and the Doppler effect of radio waves. During autonomous driving, a control device C (described later) acquires positioning information (i.e., vehicle position information) from the positioning device 30 to calculate the vehicle's position and controls the steering device 31 to eliminate deviation from a predetermined target driving route, thereby enabling autonomous driving. The positioning device 30 also includes an IMU (Inertial Measurement Unit), which can simultaneously measure the tilt angle of the traveling vehicle body 2 (i.e., the tilt of the field).

[0030] The work implement W is a machine that performs work in a farm field. In the example shown in Fig. 1, the work implement W is a rotary tiller that performs tilling work in a farm field, but the type of work implement W is not limited to this. Note that the rotary tiller tills the farm field (soil) by rotating the tiller tines 61 using power transmitted from the PTO shaft 16.

[0031] The tractor 1 also includes a control device C (see FIG. 3). The control device C controls the engine E and also controls the traveling speed of the traveling body 2. The control device C also controls the lifting and lowering of the work implement W.

[0032] Furthermore, the tractor 1 is configured to be able to wirelessly communicate with a mobile information terminal (mobile terminal such as a smartphone or tablet terminal) 100, and the worker can operate the mobile information terminal 100 to check various settings and information of the tractor 1. The mobile information terminal 100 includes a storage unit made up of, for example, a hard disk, a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and a display unit and operation unit made up of a touch-type liquid crystal display 110. Note that various keys, buttons, etc. may also be provided separately as the operation unit.

[0033] <2. Obstacle detection sensor> The tractor 1 is equipped with an obstacle detection sensor 20 as an obstacle detection means for detecting obstacles (people, animals, objects, etc. that impede the tractor's travel in a field). The obstacle detection sensor 20 detects obstacles around the tractor and is configured as a 3D sensor called a Lidar (Light Detection and Ranging) that measures scattered light from laser irradiation. The Lidar method detects objects by irradiating them with near-infrared light, visible light, or ultraviolet light and detecting the reflected light with an optical sensor to obtain the direction and distance to the object, thereby acquiring 3D point cloud data around the tractor. The obstacle detection sensor 20 in this embodiment is configured to detect obstacles based on a cross section perpendicular to the traveling direction. It has a detection range of at least several tens of meters and can measure the size and shape of the detected object (i.e., obstacle), its direction relative to the tractor, and its distance from the tractor. Note that the obstacle detection sensor 20 can also be replaced by other mid-range sensors, such as an infrared sensor, ultrasonic sonar, or millimeter-wave radar, or a combination of these. Information about the obstacle detected by the obstacle detection sensor 20 (hereinafter referred to as obstacle detection data Da1) is appropriately transmitted to and stored in a control device C (described later). The obstacle detection data Da1 includes at least information about the direction and distance of the obstacle relative to the aircraft, and information about the time the obstacle was detected.

[0034] The obstacle detection sensor 20 is attached to the front of the traveling vehicle body 2 so as to be able to detect obstacles ahead of the vehicle body. More specifically, it is attached to a sensor attachment stay 13 provided in front of the hood 6, has a detection area extending forward of the vehicle body over a predetermined range, and detects obstacles present in front of (the traveling direction of) the traveling vehicle body 2. Note that the obstacle detection sensor 20 may also be attached to the rear of the traveling vehicle body 2 so as to be able to detect obstacles present behind the vehicle body in addition to those in front of it.

[0035] <3. Cockpit Interior Configuration> Next, the configuration inside the cockpit 7r will be described. FIG. 3 is an external view of the inside of the cockpit 7r as viewed from the cockpit 8, and FIG. 4 is an enlarged view of the area around the steering wheel in FIG.

[0036] As described above, the steering wheel 9 is provided in front of the driver's seat 8. In addition, the clutch pedal 18 is provided on the lower left side of the handlebar post 350 to which the steering wheel 9 is attached, and the accelerator pedal 19 and the brake pedal 15 are provided on the lower right side of the handlebar post 350. The brake pedal 15 is a pair of left and right brake pedals 15L, 15R to enable operation of the left and right brakes, respectively.

[0037] A forward / reverse lever 201 is provided on the upper left side of the handle post 350. An accelerator lever 351 for adjusting the rotation speed of the engine E, a blinker lever 352, and the like are provided on the upper right side of the handle post 350. An engine key switch 353 and a PTO shift lever 354 for operating the drive (on / off) of the engine E are provided on the driver's seat 8 side of the handle post 350.

[0038] 3, a dashboard cover 355 is provided in front of the steering wheel 9. The dashboard cover 355 is provided with a meter panel 11 so as to be visible to the driver in the cockpit 8. The meter panel 11 is provided with a display unit (touch panel) 356, an engine revolution meter (tachometer) 357, and the like.

[0039] Furthermore, although this configuration is well known and will not be described in detail, a main shift lever, a sub-shift lever, an accelerator lever, a position lever, a lift position setting means (lift height dial), a public road driving button, an operation panel storage section, and the like are provided on the left and right sides of the cockpit 8. Of these, the position lever is operated when raising or lowering the lift arm 122. In addition, various operation switches are provided, such as a PTO automatic / manual changeover switch, a PTO on / off switch, an engine rotation indicator, a rotation speed increase adjustment switch, and a rotation speed decrease adjustment switch. The operation panel storage section also stores an operation panel on which operation switches other than those mentioned above are provided. Furthermore, audio output devices (so-called speaker devices) capable of audio output are provided at appropriate locations within the cockpit 7r, enabling various audio notifications to be made.

[0040] <4. Imaging Device> Next, the photographing device 13 of the tractor 1 will be described. As the first photographing device 13, a foreground photographing camera 13a, which is a ceiling-suspended photographing device, is provided in the cockpit 7r on the ceiling substantially above the cockpit 8. This foreground photographing camera 13a is a photographing device (camera) that photographs the foreground (front scenery) of the work vehicle 1 at predetermined time intervals and generates image data of the area in front of the work vehicle 1 (hereinafter referred to as foreground image data G). More specifically, in this embodiment, the foreground photographing camera 13a is configured to photograph a predetermined range of the area in front from within the cockpit 7r, and is configured to include at least the windshield 14f in front of the cabin box 7a as its photographing range. It is more preferable that the photographing range of the foreground photographing camera 13a be configured to include the left and right side windows 14s, 14s of the cabin box 7a in addition to the front windshield 14f. The foreground image data G also includes information regarding the time the image was photographed. In addition, in this embodiment, the foreground photographing camera 13a is shown to be arranged on the ceiling approximately above the cockpit 8, but the arrangement position is not limited to this and may be arranged, for example, on the dashboard cover 355.

[0041] A cross-shaped reference marker m1 is provided on the windshield 14f at approximately the center in the vertical and horizontal directions so as to be included in the shooting range of the foreground photographing camera 13a. This reference marker m1 is made of, for example, black adhesive tape that can be attached to the windshield 14f, and has a pre-designed size in terms of length and width (or top and bottom and left and right) (for example, 20 cm long x 20 cm wide). The position of this reference marker m1 is not limited to approximately the center in the vertical and horizontal directions of the windshield 14f, as long as it is included in the shooting range of the foreground photographing camera 13a.

[0042] The height position of this foreground photographing camera 13a is set close to the eye level of the operator (hereinafter referred to as the driver) seated in the cockpit 8, but is arranged at a position slightly higher than the seat height of the driver (for example, at a height of about 110 cm above the seat of the cockpit 8) so as not to obstruct the field of view. Note that the configuration of the foreground photographing camera 13a is not limited to this, and for example, it may be configured with a plurality of cameras that photograph the windshield 14f and the left and right side windows 14s, 14s, respectively.

[0043] In addition, as the second photographing device 13, a handle photographing camera 13b is attached to the upper end of the grip portion (rim portion) of the steering wheel 9. This handle photographing camera 13b The steering wheel photographing camera 13b is a photographing device (camera) fixed to the upper end of the steering wheel 9 with a fixing clip, photographs the area in front of the steering wheel 9 at predetermined intervals, and generates image data of the area in front of the steering wheel 9 (hereinafter referred to as steering wheel image data G2). The steering wheel photographing camera 13b is configured to rotate together with the steering wheel 9 when the steering wheel 9 is turned. The steering wheel photographing camera 13b is preferably a small and lightweight camera, such as a CCD camera, with a viewing angle of approximately 150 degrees facing forward, thereby enabling it to photograph substantially the entire area of ​​the dashboard cover 355. A pair of targeting markers m2, m2 is provided on the upper part of the dashboard cover 355. The pair of targeting markers m2, m2 are, for example, circular reflective stickers attached to the dashboard cover 355, and are provided at positions that are included in at least the photographing range of the steering wheel photographing camera 13b. The steering wheel image data G2 also includes information regarding the time the image was captured.

[0044] <5. Control device configuration> 5A and 5B are block diagrams showing the configuration of a control system including a control device C of a work vehicle A. The control device C is an information processing device configured by combining multiple ECUs (Electronic Control Units). Each of these multiple ECUs is configured with a CPU that performs arithmetic processing and memory that can read and write information necessary for the arithmetic processing, and the control device C realizes the configuration shown as functional blocks in FIG. 5 by the CPU operating in accordance with various control programs stored in the memory.

[0045] 6, the control device C has on its input side connected to the positioning device 30, the photographing device 13 (foreground photographing camera 13a, handlebar photographing camera 13b), obstacle detection sensor 20, steering angle sensor 25, engine rotation sensor 23, vehicle speed sensor 24, lift arm sensor 26, engine key switch 353, and touch panel 356 via an input / output signal processing unit (including a communication unit) not shown, thereby obtaining positioning information (machine position information) from the positioning device 30, image information from the photographing device 13 (foreground photographing camera 13a, handlebar photographing camera 13b), and detection and sensing information from various sensors and switches. Furthermore, the control device C obtains operation information from an operator's operation from the touch panel 356, which is connected so as to be able to send and receive information bidirectionally, and also transmits display information indicating an image to be displayed on the touch panel 356.

[0046] The control device C is equipped with ECUs for controlling each mechanism of the vehicle, and more specifically, is equipped with an operation system ECU 50 that controls the operation of each mechanism of the work vehicle 1, and a control system ECU 54 that determines the operation method (operation rules) of each mechanism. As shown in Figure 5, the operation system ECU 50 is equipped with an engine ECU 51 that controls the operation of the engine E, a travel system ECU 52 that controls the operation of mechanisms related to travel, and a work implement ECU 53 that controls the operation of the work implement W.

[0047] As shown in FIG. 5, the engine ECU 51 is connected to the engine E, the travel system ECU 52 is connected to the steering device 31, the transmission 32, and the auto-brake device 33, and the work system ECU is connected to the lifting device 12 and the PTO clutch 13.

[0048] As described above, the steering device 31 includes a steering actuator that rotates the steering wheel 9, and is a device that automatically steers the steering wheel 9 during automatic driving. The speed change device 32 is a transmission housed in the transmission case 10, and is a device that changes the speed of the rotational power output from the engine E. The auto-brake device 33 is a device that can brake the rear wheels 5 (restrict rotation) without operating the brake pedal 15.

[0049] The control device C also includes a communication unit 59, which is a communication mechanism that connects to an external device physically separated from the control device C via a network NW and exchanges information through communication. The communication unit 59 is, for example, a wireless communication module that can be connected to a wireless router outside the device. In this embodiment, the communication unit 59 is connected to the mobile information terminal 100 via the network NW, and is capable of sending and receiving information through two-way communication.

[0050] The mobile information terminal 100 is a small information processing terminal such as a mobile phone (smartphone), a tablet, a notebook computer, or a wearable device such as glasses or a wristwatch, and is equipped with a touch panel display 101 (see FIG. 1).

[0051] The control system ECU 54 includes a manual driving control unit 54a that controls driving in the manual driving mode, and an automatic driving control unit 54b that controls driving in the automatic driving mode. Here, controlling driving means, in more detail, acquiring necessary detection information from various sensors to execute each mode, sending necessary control commands to the operation system ECU 50 when each mode is selected, and causing the traveling vehicle body 2 to travel.

[0052] Work information is stored in a work information DB (database) 54c connected to the control system ECU 54. The work information includes, for example, field information, which is information about the field, a planned travel route for travel, and the work width of the work implement W. The field information also includes field information necessary for the work, such as information about the shape, position, size, range, latitude, longitude, and altitude of each field to be worked on.

[0053] Each operation mode is well known and will not be described in detail. However, in the automatic operation mode, a planned travel route corresponding to the work to be performed by the work implement W is determined in advance for each field, digitized, and stored in a work information DB (54c). Based on position information acquired from the positioning device 30, various components such as the engine E, steering device 31, transmission 32, braking device 53, and lifting device 12 are controlled so that the work implement W travels along the planned travel route designed in advance. Note that the planned travel route is usually designed to alternate between straight travel routes and turning routes in order to travel efficiently throughout the entire field, and the spacing between the straight travel routes is determined by the working width of the work implement W so that the work areas do not overlap. Furthermore, the specific planned travel route is designed depending on the shape and size of the field, the width, length, and number of ridges formed in the field, etc. The planned travel route may be designed by the control device C upon receiving information about the field and input of operations by the operator, or the control device C may be configured to acquire information about the planned travel route designed by an external computer via the network NW. In addition, in the manual driving mode, the operator can steer the machine by operating the steering wheel 9 to travel in the field.

[0054] <6. Image analysis of control devices> The control device C includes an image analysis unit 55 that analyzes image information acquired from the photographing device 13. The image analysis unit 55 includes an obstacle extraction unit 55a, a person determination unit 55b, an attribute information calculation unit 55c, and a marker extraction image creation unit 55d. The image information acquired from the photographing device 13 and the information analyzed by the image analysis unit 55 are stored in an image information DB (database) 55e as appropriate. The analysis process of the image information by the image analysis unit 55 is automatically performed each time foreground image data G is acquired by the foreground photographing camera 13a.

[0055] Here, the processing of each part of the image analysis unit 55 will be explained with reference to Fig. 6 to Fig. 7. Fig. 6 is an image diagram of an example of foreground image data captured by the foreground photographing camera 13a. The image of the foreground image data G shown in Fig. 6 is an example, and the foreground image data G includes an image g1 of a person and an image g2 of a standard marker m1.

[0056] Fig. 7(a) is an image diagram of an example of obstacle extraction image data extracted by an obstacle extraction unit from the foreground image data of Fig. 6. Fig. 7(b) is an image diagram for explaining attribute information calculated by an attribute information calculation unit from the obstacle extraction image data of Fig. 7(a). The obstacle extraction unit 55a generates obstacle-extracted image data G1 by extracting an image g2 (extracted image g12) of the standard marker m1 and an image g1 of an obstacle (person, small animal, object, etc.) from the foreground image data G using a known image analysis process (e.g., pattern matching process). As shown in FIG. 7(a), the obstacle-extracted image data G1 is image data in which a rectangular region g11 surrounding the image g1 of the obstacle (person, small animal, object, etc.) and an image g2 (extracted image g12) of the standard marker m1 are extracted from the foreground image data G, and the image data G1 has the same size as the foreground image data G. In other words, the obstacle-extracted image data G1 is image data in which all areas other than the rectangular region g11 containing the image of the obstacle and the image g2 (extracted image g12) of the standard marker m1 are removed (e.g., whitened) from the foreground image data G. The rectangular region g11 is designed to be as small as possible so as to surround the entire outer edge of the image g1 of the obstacle (person, small animal, object, etc.). Furthermore, when there are multiple images g1 of obstacles in the foreground image data G, the obstacle extraction unit 55a extracts the rectangular area g11 of the image g1 of the obstacle having the largest rectangular area g11, in order to perform processing preferentially on the obstacle closest to the work vehicle 1.

[0057] The person determination unit 55b refers to the obstacle extraction image data G1 and determines whether the obstacle image g1 included in the obstacle extraction image data G1 is a person or not through pattern matching processing, thereby enabling the control device C to determine whether a person has been photographed by the foreground photographing camera 13a.

[0058] When the person determination unit 55b determines that the obstacle extraction image data G1 includes an image of a person, the attribute information calculation unit 55c calculates attribute information based on the image g1 of the person included in the obstacle extraction image data G1. Here, the attribute information Da3 is information about the attributes of the person photographed by the foreground photographing camera 13a, and is calculated based on the area of ​​the person image that occupies the image of the obstacle extraction image data G1 (it can also be said that the attribute information Da3 is calculated based on the area of ​​the person image that occupies the image of the foreground image data G, which is data before processing (before noise removal)). The attribute information Da3 includes, for example, information such as the size, head-to-body ratio, and location (coordinates) of the photographed person, which can be calculated based on the area of ​​the person image that occupies the image of the obstacle extraction image data G1. Specifically, the obstacle extraction unit 55a generates obstacle extraction image data G1 each time from the foreground image data G generated by the foreground photographing camera 13a, and the attribute information calculation unit 55c, which has acquired the generated obstacle extraction image data G1, executes image analysis processing and calculates, as attribute information Da3, the point P where the photographed person exists, the width Hx and height Hy of the area of ​​the person image (in this embodiment, the area of ​​the person image is regarded as a rectangular area g11 and processing is performed), the height r1 of the head, and the height r2 of the body, based on the area of ​​the person image occupying the image of the obstacle extraction image data G1, and further calculates the size (width Hx / hx, height Hy / hy) of the person image relative to the image g2 (image g12 after extraction) of the standard marker m1 in the image, and stores the calculated attribute information in the image information DB (55e). Here, the attribute information calculation unit 55c is configured to calculate the point P(Px,Py) where the photographed person exists by specifying the point P(Px,Py) using two-dimensional coordinates with the center point of the image g2 of the standard marker m1 (image g12 after extraction) as the reference point Po(0,0), as shown in the illustrated example, and further calculates the two-dimensional coordinates of the midpoints of the coordinates of the lower left and right corners of the area of ​​the person image (rectangular area g11) as the point P(Px,Py) where the photographed person exists. For example, when the coordinates of the lower left and lower right of the rectangular area g11 are (4,2) and (10,2), the point P(7,2) where the person exists can be calculated.In addition, the attribute information calculation unit 55c may be configured to determine whether the person photographed by the foreground shooting camera 13a is an adult or a child based on the ratio of the vertical width r1 of the head to the vertical width r2 of the body of the person image, and to include the determination result in the attribute information Da3.

[0059] Furthermore, as described above, the information regarding the size H of the photographed person included in the attribute information Da3 is calculated based on the ratio of the width Hx and height Hy of the rectangular area g11 to the width hx and height hy of the image g12 in which the standard marker m1 was captured. For example, when the width Hx = 200 pixels (px) and the height Hy = 500 pixels (px), and the width hx = 100 pixels (px) and the height hy = 100 pixels (px), the size H of the person in the image is calculated as width Hx / hx = 2 and height Hy / hy = 5. As a result, even if the photographing position of the foreground photographing camera 13a moves forward or backward, the attribute information calculation unit 55c calculates the size H of the person based on the image g12 in which the standard marker m1 was captured. This allows the attribute information calculation unit 55c to obtain more accurate values ​​indicating the location, width, and height of the person, thereby preventing errors caused by moving or replacing the foreground photographing camera 13a.

[0060] FIG. 8 is an illustration of a case where a marker extraction image is generated from steering wheel image data by the marker extraction image creation unit 55d. The marker extraction image creation unit 55d generates marker extraction image data G3 by extracting an image g21 of the target marker m2 from steering wheel image data G2 using a known image analysis process (e.g., pattern matching process). In FIG. 8, the left side of the table shows an image of the steering wheel image data G2 before extraction by the marker extraction image creation unit 55d, and the right side of the table shows an image of the steering wheel image data G2 after extraction (i.e., the image of the steering wheel image data G2). The upper row of the table shows images when the steering wheel 9 is in the neutral position, and the lower row of the table shows images when the steering wheel 9 is turned to the right (rotated clockwise) from the neutral position.

[0061] As shown in Fig. 8, the marker extraction image data G3 is image data in which an image g21 (extracted image g31) of the target marker m2 is extracted from the steering wheel image data G2. In the illustrated example, the image g31 of the target marker m2 extracted from the steering wheel image data G2 is shown in the marker extraction image data G3. The marker extraction image data G3 is image data of the same size as the steering wheel image data G2. In other words, it is image data in which all parts of the steering wheel image data G2 except for the image g21 of the target marker m2 have been removed (e.g., whitened out).

[0062] <7. Configuration of the human position estimation unit> Returning to Fig. 5, the control device C includes a person position estimation unit 56 that estimates the position of a person photographed by the foreground photographing camera 13a. This person position estimation unit 56 includes a person position learning data creation unit 56a, a person position learning unit 56b, and a person position estimation model 56c. These will be explained in order below.

[0063] FIG. 9 is a block diagram for explaining the function of the human position estimation unit 56. The human position estimation unit 56 has a function of creating learning data and a function of learning a neural network model (human position estimation model 56c) that estimates the position of a person based on the created learning data. The execution of this function is started and ended by a predetermined operation by the worker (operation of the touch panel 356 or the mobile information terminal 100).

[0064] The human position estimation unit 56 includes a human position learning data creation unit 56a that creates learning data for estimating the position of a person. The human position learning data creation unit 56a acquires obstacle detection data Da1 and attribute information Da3, and creates human position learning data Da4, which is learning data for estimating the position of a person, as shown in Fig. 9. Here, the human position learning data Da4 is data that pairs attribute information Da3 (including at least information about the position of a point P (Px, Py) where a person included in the obstacle extraction image data G1 is present and the calculated values ​​of the width Hx / hx and height Hy / hy) calculated based on foreground image data G acquired by the foreground photographing camera 13a at the same time, with obstacle detection data Da1 acquired by the obstacle detection sensor 20 as correct answer data (correct answer label) (note that here, information about the direction and distance of the obstacle relative to the aircraft is used). It is also possible to create learning data by pairing foreground image data G and obstacle detection data Da1, but by using attribute information Da3 calculated by the image analysis unit 55, noise is removed from the foreground image data G, thereby improving the accuracy of the learning model.

[0065] Here, the foreground image data G includes attribute information (the position of the point P (Px, Py) where the person is present, the position of the point P (Px, Py), and the size H of the photographed person, i.e., the width Hx / hx and the height Hy / hy) in the image of the person photographed by the foreground photographing camera 13a, and this attribute information (the position of the point P (Px, Py) where the person is present, the position of the point P (Px, Py) of the person, and the size H of the photographed person, i.e., the width Hx / hx and the height Hy / hy) will change depending on the direction and distance of the person relative to the aircraft. As described above, by machine learning the attribute information (the position of point P (Px, Py) where the person is located and the size H of the photographed person, i.e., width Hx / hx, height Hy / hy) and the obstacle detection data Da1 (i.e., the direction and distance of the obstacle) detected by the obstacle detection sensor 20, it becomes possible to estimate the direction and distance of the person relative to the aircraft based on the image of the person photographed by the foreground photographing camera 13a (i.e., the area of ​​the person image), the position (point) and size of the person in the image, without relying on the obstacle detection sensor 20.

[0066] The person position learning unit 56b learns a person position estimation model 56c that estimates the position of a person photographed by the photographing device 13 (foreground photographing camera 13a) based on the person position learning data Da4 created by the person position learning data creation unit 56a. The learned person position estimation model 56c is stored in a memory area provided in the control device C using a large amount of person position learning data Da4. As a result, the control device C is configured to be able to estimate the position of a person based on image information photographed by the photographing device 13 (foreground photographing camera 13a) using the learned person position estimation model 56c.

[0067] <8. Configuration of the steering angle estimator> Returning to Fig. 5, the control device C includes a steering angle estimation unit 57 that estimates the steering angle based on image information captured by the photographing device 13 (steering wheel photographing camera 13b). The steering angle estimation unit 57 includes a steering angle learning data creation unit 57a, a steering angle learning unit 57b, and a steering angle estimation model 57c. These will be described in order below.

[0068] FIG. 10 is a block diagram for explaining the function of the steering angle estimation unit 57. The steering angle estimation unit 57 has a function of creating learning data and a function of learning a neural network model (steering angle estimation model 57c) that estimates the steering angle based on the created learning data. The start and end of execution of these functions are instructed by a predetermined operation by the operator (operation of the touch panel 356 or the mobile information terminal 100).

[0069] The steering angle estimation unit 57 includes a steering angle learning data creation unit 57a that creates learning data for estimating the steering angle. As shown in Fig. 10, the steering angle learning data creation unit 57a acquires steering angle data Da2 and marker-extracted image data G3 and creates steering angle learning data Da5, which is learning data for estimating the steering angle. Here, the steering angle learning data Da5 is data that pairs marker-extracted image data G3 created by the image analysis unit 55 from steering wheel image data G2 acquired by the steering wheel photographing camera 13b at the same time, and steering angle detection data Da2 acquired by the steering angle sensor 25 as correct answer data (correct answer label). Note that data that pairs the steering wheel image data G2 with the steering angle detection data Da2 acquired by the steering angle sensor 25 as correct answer data can also be used as the steering angle learning data Da5. However, by using the marker-extracted image data G3, noise can be removed and accuracy can be improved.

[0070] Here, the marker extraction image data G3 includes information regarding the positions of the pair of target markers m2, m2 in the image captured by the steering wheel photographing camera 13b. The positions of the pair of target markers m2, m2 in the image change in response to the operation of the steering wheel 9, and the steering angle detected by the steering angle sensor 25 also changes. Therefore, as described above, by performing machine learning on the positions of the target markers m2, m2 in the image and the steering angle of the steering angle sensor 25, it becomes possible to estimate the steering angle based on the positions of the pair of target markers m2, m2 in the image captured by the steering wheel photographing camera 13b, without relying on the steering angle sensor 25. Note that while it is acceptable to photograph only one target marker m2, providing a pair of target markers m2 on the left and right sides with respect to the neutral position of the steering wheel photographing camera 13b (a position where the steering wheel 9 is not being operated) allows for accurate estimation of the steering angle.

[0071] The steering angle learning unit 57b learns a steering angle estimation model 57c that estimates the steering angle of the steering wheel 9 photographed by the photographing device 13 (steering wheel photographing camera 13b) based on the steering angle learning data Da5 created by the steering angle learning data creating unit 57a. The learned steering angle estimation model 57c is stored in a storage area of ​​the control device C using a large amount of steering angle learning data Da5. As a result, the control device C is configured to be able to estimate the steering angle of the steering wheel 9 photographed by the photographing device 13 (steering wheel photographing camera 13b) using the learned steering angle estimation model 57c.

[0072] FIG. 11 is a schematic model diagram showing the input and output of the human position estimation model 56c. In the model schematic diagram shown in Fig. 11, when attribute information generated (by image analysis unit 55) based on foreground image data G acquired by foreground photographing camera 13a, i.e., the calculated coordinates of point P (Px, Py) where a person included in the image is located, and the calculated size H of the photographed person (i.e., width Hx / hx, height Hy / hy) are input, trained human position estimation model 56c is designed to output estimated values ​​of distance D and direction α from the aircraft. As a result, a work vehicle 1 equipped with trained human position estimation model 56c can acquire estimated values ​​of distance D and direction α of the photographed person from the aircraft based on image information (foreground image data G) captured by foreground photographing camera 13a.

[0073] FIG. 12 is a model schematic diagram showing inputs and outputs of the steering angle estimation model 57c. In the model schematic diagram shown in FIG. 12, when marker-extracted image data G3 generated (by the image analysis unit 55) based on steering wheel image data G2 acquired by the steering wheel photographing camera 13b is input, the trained steering angle estimation model 57c is designed to output an estimated value of the steering angle. In other words, a work vehicle 1 equipped with the trained steering angle estimation model 57c is able to acquire an estimated value of the steering angle An of the photographed steering wheel 9 based on image information (steering wheel image data G2) captured by the steering wheel photographing camera 13b. The work vehicle 1 in this embodiment is configured to perform autonomous driving based on the estimated value of the steering angle An output by this steering angle estimation model 57c. As a result, the work vehicle 1 can be driven autonomously by estimating the steering angle based on the image captured by the steering wheel photographing camera 13b, thereby enabling accurate autonomous driving without relying on the steering angle sensor 25. The work vehicle 1 may also be configured to be switchable between automatic driving based on steering angle estimation based on steering wheel image data G2 acquired by the steering wheel photographing camera 13b and automatic driving based on the steering angle measured by the steering angle sensor 25. Furthermore, during automatic driving based on the steering angle measured by the steering angle sensor 25, if a malfunction of the steering wheel angle sensor 25 is detected (for example, when an abnormal value of the sensor is detected), the work vehicle 1 may be configured to automatically switch to automatic driving based on steering wheel image data G2 acquired by the steering wheel photographing camera 13b. This makes it possible to effectively deal with a malfunction of the steering wheel angle sensor 25, thereby further effectively reducing the possibility of the work vehicle 1 coming into contact with a person and improving safety. Note that when data paired with the steering wheel image data G2 and steering angle detection data Da2 acquired by the steering angle sensor 25 as correct answer data is used as steering angle learning data Da5, the steering wheel image data G2 can be input to the steering angle estimation model 57c to obtain an estimated value of the steering angle An.

[0074] <9. Obstacle avoidance processing> Returning to Figure 5, the processing of the obstacle avoidance unit 58 will be described. The obstacle avoidance unit 58 is a program for avoiding obstacles while the work vehicle 1 is autonomously driving, and processing starts when autonomous driving starts and ends when autonomous driving ends. Note that when the processing of the obstacle avoidance unit 58 is being executed, the control device C is assumed to be equipped with a trained human position estimation model 56c and a trained steering angle estimation model 57c.

[0075] During automatic driving, the obstacle avoidance unit 58 acquires foreground image data G acquired by the foreground photographing camera 13a from the photographing device 13 at predetermined time intervals, and the image analysis unit 55 determines whether an obstacle is included in the acquired foreground image data G, and if an obstacle is included, the person determination unit 55b determines whether the obstacle is a person or not.

[0076] Here, when the obstacle avoidance unit 58 determines that an obstacle is included in the foreground image data G but is not a person, if the size of the obstacle in the foreground image data G (more specifically, the area of ​​the rectangular region surrounding the image of the obstacle) is equal to or greater than a predetermined value, the obstacle avoidance unit 58 determines that the vehicle has approached an obstacle, and, under the control of the operation system ECU 50, executes a known obstacle avoidance operation during autonomous driving (an operation of the vehicle to avoid contact with the obstacle; for example, if the distance between the obstacle and the vehicle is equal to or greater than a certain distance, a detour route that bypasses the obstacle is generated, and the vehicle temporarily deviates from the planned driving route and travels along the detour route. If the distance between the obstacle and the vehicle is less than the certain distance, it determines that the obstacle is too close and therefore a detour route cannot be generated, and the vehicle temporarily suspends its travel). This known obstacle avoidance operation will not be described in detail, but please refer to, for example, Japanese Patent Application Laid-Open Nos. 2019-101931 and 2023-127875. After the avoidance operation is completed, autonomous driving is resumed. Furthermore, if the size of the obstacle in the foreground image data G is below a predetermined value, it is determined that there is sufficient distance between the aircraft and the obstacle and the possibility of contact is low, and automatic driving continues.

[0077] Furthermore, if the person determination unit 55b determines that a person is included in the foreground image data G, the image analysis unit 55 generates attribute information Da3 from the foreground image data G, and inputs the generated attribute information Da3 into the trained human position estimation model 56c to obtain estimated values ​​of the distance D and direction α of the person relative to the aircraft (see FIG. 11). If the obstacle avoidance unit 58 determines that the aircraft is approaching a person based on the acquired estimated values ​​of distance D and direction α, it executes a known obstacle avoidance operation during autonomous driving (stopping the aircraft or bypassing the obstacle) under the control of the operation system ECU 50. After the avoidance operation is completed, autonomous driving is resumed. If the acquired estimated values ​​of distance D and direction α determine that there is a sufficient distance between the aircraft and the obstacle and the possibility of contact is low, autonomous driving is continued. In determining whether or not the aircraft and a person have come close to each other, the closer the person's direction α is to the front of the aircraft (for example, within plus or minus 30 degrees of the aircraft's direction of travel), the longer the distance D at which it is determined that the person has come close (for example, 20 meters), and the further the person's direction α is from the front of the aircraft (for example, not within plus or minus 30 degrees of the aircraft's direction of travel), the shorter the distance D at which it is determined that the person has come close (for example, 15 meters).This makes it possible to accurately calculate the distance to the person from image information captured by the camera (imaging device 13), prevent contact, and improve safety.

[0078] The embodiments of the present invention have been described above. The present invention is not limited to the above-described embodiments. It goes without saying that modifications may be made as appropriate within the scope of the technical concept. Other embodiments will be described below.

[0079] <10. Changes in processing based on the judgment of adults and children> FIG. 11 illustrates an example in which, when attribute information Da3 generated (by the image analysis unit 55) based on foreground image data G acquired by the foreground photographing camera 13a is input, i.e., calculated values ​​for the coordinates of the point P (Px, Py) where the photographed person is located and calculated values ​​for the size H (the person's width Hx / hx, height Hy / hy), the trained human position estimation model 56c outputs estimated values ​​for the distance D and direction α from the aircraft. Here, when the control device C (obstacle avoidance unit 58) inputs the attribute information generated (by the image analysis unit 55) based on the foreground image data G acquired by the foreground photographing camera 13a to the trained human position estimation model 56c, the control device C (obstacle avoidance unit 58) may be configured to determine whether the person included in the foreground image data G is an adult or a child by performing a known image analysis process (e.g., a pattern matching process) or analyzing the head-to-body ratio of the person image. Note that if the attribute information Da3 is configured to include a determination result of whether the person is an adult or a child, the determination result may be used.

[0080] Furthermore, when a person included in an image of foreground image data G is determined to be a child, the size H (the person's width Hx, height Hy) of the generated attribute information (by image analysis unit 55) may be corrected to be larger (enlarged) (for example, the person's width Hx × 1.2 times, height Hy × 1.5 times), and then input to trained person position estimation model 56c. Generally, when a child is photographed, the size of the child in the image will be smaller than that of an adult, so by correcting the size to be enlarged, the distance D to the aircraft can be estimated to be longer than usual, thereby improving safety.

[0081] Furthermore, if a person included in the image of the foreground image data G is determined to be a child, the obstacle avoidance unit 58 may correct the distance used to determine whether the aircraft is approaching a person to be longer than that used for an adult. For example, the distance D used to determine proximity may be corrected to 1.5 times the original distance. This can further improve safety. Furthermore, the emergency stopping distance of the aircraft when it is determined to be very close to a person may be corrected to be longer than that used for an adult (for example, 5 meters for an adult and 10 meters for a child).

[0082] Furthermore, if a person included in an image of the foreground image data G is determined to be a child, when performing obstacle avoidance (detouring) during autonomous driving, the steering angle (in other words, the steering amount) may be increased (for example, 1.2 times) compared to the normal case (for an adult) to avoid the obstacle. As a result, when a child approaches the aircraft, the detouring is performed to a greater extent than in the case of an adult, thereby reducing the risk of contact and improving safety.

[0083] <11. About omission of learning function> The work vehicle 1 can also be configured such that the learned steering angle estimation model 57c and the learned human position estimation model 56c are stored in advance in the control device C. In this case, the learning functions (human position learning data creation unit 56a, human position learning unit 56b, steering angle learning data creation unit 57a, steering angle learning unit 57b) can be omitted. Furthermore, it is also possible to omit the obstacle detection sensor 20 and steering angle sensor 25. This makes it possible to provide a work vehicle that is capable of highly safe autonomous driving while reducing the cost of the work vehicle 1.

[0084] <12. Use cases of AI cameras> Next, examples of using AI cameras will be described with reference to Figures 13 to 18. Note that an AI camera refers to an imaging device that is equipped with artificial intelligence (AI) or that is configured to be available via a network.

[0085] Figure 13 shows an example of a work vehicle (grass mower) that is equipped with an AI camera to mow the grass on a golf course, and that uses the AI ​​camera to collect images of the area ahead when mowing the grass. The work vehicle analyzes the boundaries of the rough, fairway, and green from the camera images. Steering assistance is provided so that the vehicle travels within the same area and does not enter other areas. As a result, steering assistance is provided to the vehicle so that it travels and works only within a specific area, which reduces the burden on the worker and helps prevent work mistakes even for beginners.

[0086] Figure 14 shows an example of a configuration in which an AI camera and monitor are installed on a work vehicle (snow blower) that removes snow. This work vehicle uses an AI camera to collect image data of the area where snow removal work is to be performed before snow accumulates in winter. When performing snow removal work in winter, the location of obstacles and ditches under the snow along the driving route is identified from images taken before the snow accumulates. The current camera image is displayed on the monitor, and an image is displayed that combines the locations of obstacles and ditches based on image analysis. When an obstacle or a ditch is approached, an alarm sounds. Even when there is a large amount of snow, the location of obstacles and ditches under the snow can be identified, preventing the risk of an accident.

[0087] Figure 15 shows an example of a configuration in which a work vehicle (rice transplanter) equipped with GNSS is equipped with an AI camera that photographs the seedling tray and spare seedling tray. The AI ​​camera analyzes the amount of seedlings consumed in one round trip for each row. The remaining number of possible round trips is calculated from the amount of seedlings consumed in one round trip, the current amount of seedlings remaining, and the amount of seedlings remaining on the spare seedling tray (number of mats). If the remaining number of possible round trips falls to "one round trip" or less, a notice is issued to the operator to replenish seedlings as there is a possibility that seedlings will run out during work. By calculating the remaining number of round trips from the total amount of seedlings remaining on the work vehicle and the amount of seedlings consumed in one round trip, it is possible to prevent seedling shortages during work.

[0088] Furthermore, when the above-mentioned work vehicle (rice transplanter) performs work such as tilling, transplanting, and harvesting, it does not use GNSS information but instead uses an AI camera equipped on the machine to determine which steps have been completed and which remain to be completed. It is also possible to configure a work planning system that calculates the next step based on the results of this determination. In situations where GNSS cannot be used (such as mountain valleys, near windbreaks, or inside greenhouses), the AI ​​camera can be used to enable autonomous driving. Furthermore, by presetting the working width of the work vehicle, it is possible to distinguish between areas that have actually been tilled and areas that are simply covered with soil.

[0089] Figure 16 shows an example of a configuration in which an AI camera is installed on a work vehicle (vegetable harvester). Reference markers are set up in the field beforehand, and before snow accumulates in winter, the AI ​​camera on the work vehicle collects information on the positions of the ridges and vegetables in the field relative to the reference markers. When harvesting vegetables buried under the snow in winter, the positions of the vegetables buried under the snow are identified from images taken before the snow accumulates. Based on the positions of the ridges and vegetables, steering assistance is provided to align the vehicle with the ridges. Even when there is a large amount of snow, there is no need to remove snow before harvesting. Furthermore, steering assistance is provided to align the work machine with the ridges, reducing the burden on the worker. Setting a reference can improve accuracy.

[0090] Figure 17 shows an example of a configuration in which an AI camera is mounted on a small unmanned aerial vehicle (a so-called drone). The AI ​​camera analyzes the direction and speed of the wind that the crops are exposed to. From the analyzed wind direction and speed, the impact of the wind on the drone is calculated. From the analysis results and the height to the crops, the distance that the pesticides and fertilizers will be carried by the wind is calculated. The drone's flight path and attitude are corrected so that the spraying position of the pesticides and fertilizers matches the target crops. In this way, by correcting the distance that the pesticides and fertilizers will be carried by the wind and the drone's flight path and attitude, it becomes possible to spray the pesticides and fertilizers at the target location.

[0091] Figure 18 shows an example of a configuration in which an AI camera and thermometer are installed inside a greenhouse, ventilation windows are installed in the greenhouse that are controlled to open and close by a computer, and the AI ​​camera monitors the impact of the wind on the crops when the ventilation windows are open. If the crops are moving significantly due to the wind, the computer closes the ventilation windows. If the temperature inside the greenhouse falls below a threshold, the ventilation windows close automatically. By automatically opening and closing the ventilation windows according to the condition of the crops, the burden on workers can be reduced and crops can be grown. [Explanation of symbols]

[0092] 1 Tractor (work vehicle) 2 Running vehicle 3 Body frame 4 front wheels 5 rear wheels 6. Bonnet 7 Control Unit 7a Cabin Box (Cabin) 7r cockpit 8. Cockpit 9. Steering wheel 10 Mission Case 11. Meter panel 12 Lifting device 13 Imaging equipment 13a Foreground camera (first imaging device) 13b Handlebar camera (secondary camera) 14th floor windshield 14s side glass 15(15L,15R) Brake pedal 16 PTO shaft 18 Clutch pedal 19 Accelerator pedal 20 Obstacle detection sensor 26 Lift arm sensor 30 Positioning device 100 Mobile Information Terminals 101 Display 201 Forward / reverse lever 350 handlebar post 351 Accelerator lever 352 Winker lever 353 Engine key switch 354 PTO shift lever 355 dashboard cover 356 Display unit (touch panel) 357 Engine revolution meter (tachometer) E-Engine S navigation satellite W Work Machine m1 reference marker m2 aiming marker

Claims

1. A work vehicle comprising a traveling body that travels in a field, a work implement attached to the traveling body, a positioning device that measures the position of the work implement itself, and a steering device that steers a steering wheel, and is configured to automatically steer the steering wheel by controlling the steering device based on position information of the work implement itself acquired by the positioning device, thereby enabling automatic driving in a field, a front view photographing camera that photographs an area ahead of the work vehicle and generates front view image data; an image analysis unit that performs image analysis on the foreground image data; a human position estimation model that is calculated by the image analysis unit based on the area of ​​a human image that occupies an image of the foreground image data, receives input of attribute information including information about the location and size of a person photographed by the foreground photographing camera, and outputs estimated values ​​of the direction and distance of the photographed person relative to the aircraft; A work vehicle configured to perform evasive maneuvers to avoid contact with the photographed person based on the estimated direction and distance of the photographed person relative to the vehicle, output by the human position estimation model.

2. a steering wheel photographing camera that photographs an area in front of the steering wheel and generates steering wheel image data; a target marker that is disposed within a photographing range of the steering wheel photographing camera; and a steering angle estimation model that receives input of the steering wheel image data and outputs an estimated value of the steering angle of the steering wheel, 2. The work vehicle according to claim 1, wherein the work vehicle is configured to perform automatic driving based on the estimated steering angle output by the steering angle estimation model.

3. The work vehicle described in claim 1 or claim 2, characterized in that the image analysis unit is configured to determine whether the photographed person is an adult or a child from the area of ​​the human image, and when performing obstacle avoidance operations during autonomous driving, if the image analysis unit determines that the photographed person is a child, the steering angle during detouring is larger than if the image analysis unit determines that the person is an adult.

Images