Work Management System

Abstract

We provide a work management system that allows for easy and quick review of images before and after work. [Solution] A work management system comprising a work management device for processing work management information and terminal devices for various information, wherein the work vehicle is configured to acquire location information and automatically travel along a target travel route, and is equipped with a camera that takes pictures of the area around the vehicle and generates field image information, the work management device comprises a field image acquisition unit that acquires image information from the work vehicle, a pre- and post-work determination unit that determines whether the acquired image information is of a field before or after work, and a field image merging unit that separates and combines the field image determined by the pre- and post-work determination unit into a field image before work and a field image after work, and is configured to display the field image before and after work on the terminal device.

Description

Technical Field

[0001] The present invention relates to a work management system for managing work vehicles.

Background Art

[0002] Conventionally, as disclosed in Patent Document 1 below, there is known a technique in which a camera is mounted on an agricultural vehicle and the vehicle is driven through a farm field, thereby obtaining both an overhead image capturing the entire farm field similar to aerial photography and a local enlarged image that is difficult to obtain by aerial photography.

Prior Art Literature

Patent Literature

[0003]

Patent Document 1

Summary of Invention

Problem to be Solved by the Invention

[0004] However, according to the above conventional technique, when it is desired to check images of a farm field before and after work performed by an agricultural vehicle, it is necessary to drive the agricultural vehicle separately before and after the work, which is very time-consuming. In addition, it is difficult to check the conditions before and after work in real time with images while performing work, which is inconvenient.

[0005] Accordingly, an object of the present invention is to solve such problems and provide a work management system that can easily and quickly check images before and after work.

Means for Solving the Problem

[0006] In order to achieve the above object, a first aspect of the present invention provides: A work management system comprising: a work vehicle that travels in a farm field; a work management device that processes information related to work management; and a terminal device that handles various types of information, The aforementioned work vehicle is configured to acquire positional information using a positioning device and to automatically travel within the field along a target route, and is equipped with a camera that takes pictures of the area around the vehicle to generate image information of the field. The work management device comprises a field image acquisition unit that acquires image information of the field from the work vehicle, a pre- and post-work determination unit that determines whether the acquired field image information is of the field before or after work, and a field image merging unit that separates and combines the field image determined by the field image determination unit to be before work and the field image determined to be after work. The field image merging unit is configured to generate pre-work field image data, which is an image of the entire field before work, and post-work field image data, which is an image of the entire field after work, so that the images of the entire field before and after work can be displayed on the terminal device.

[0007] According to the first invention described above, images before and after the work can be easily and quickly confirmed.

[0008] The second invention, in addition to the configuration of the first invention, The work management device includes a field image conversion unit that converts the acquired field image information into multiple rectangular images of the field viewed vertically from above, The field image conversion unit generates rectangular images showing images of the field in at least four directions: front, rear, left, and right of the work vehicle. The field image determination unit is characterized in that, when the work vehicle is traveling along a straight path after turning right, it determines the rectangular image on the right side of the machine as the image after work and the rectangular image on the left side of the machine as the image before work, and when the work vehicle is traveling along a straight path after turning left, it determines the rectangular image on the left side of the machine as the image after work and the rectangular image on the right side of the machine as the image before work.

[0009] According to the second invention described above, in addition to the effects of the first invention described above, Based on information about the target travel route, it is possible to accurately determine whether a rectangular image has been processed before or after the operation.

[0010] The third invention, in addition to the configuration of the first invention, The work management device includes a field image conversion unit that converts the acquired field image information into multiple rectangular images of the field viewed vertically from above, and a work status recording unit that creates a work status record map that records work information of work vehicles. The field image conversion unit generates rectangular images showing images of the field in at least four directions: front, rear, left, and right of the work vehicle. The aforementioned work status recording unit is configured to record the pre-work area, which is the area in the field where no work has been performed, and the post-work area, which is the area where work has been performed, on the work status recording map. The field image determination unit is characterized by determining whether the rectangular image is an image before or after work, based on the work status record map and information on the shooting center point indicating the position of the rectangular image on the field.

[0011] According to the third invention described above, in addition to the effects of the first invention described above, it is possible to accurately and effectively determine whether a rectangular image has been processed before or after processing based on the information in the work information recording map.

[0012] The fourth invention, in addition to the configuration of the second invention, The field image joining unit is characterized in that, when joining the rectangular images, it lowers the priority of overwriting the rectangular image at the rear of the machine compared to the rectangular images at other locations when the rectangular images overlap, so that in the display of the overall field image, the rectangular images acquired from locations other than the rear of the machine are displayed preferentially.

[0013] The fourth invention provides, in addition to the effects of the second invention, The rectangular image at the rear of the machine tends to become blurred due to dust and other debris from the implements, so this allows for the display of a clearer image of the entire field. [Effects of the Invention]

[0014] According to the present invention, a work management system can be provided that allows for easy and rapid confirmation of images before and after work. [BRIEF DESCRIPTION OF THE DRAWINGS]

[0015] [Figure 1] FIG. 1 is a schematic configuration diagram of a work management system according to a preferred embodiment of the present invention. [Figure 2] FIG. 2 is a left side view of a work vehicle 100 according to the embodiment. [Figure 3] FIG. 3 is a plan view of the same as above. [Figure 4] FIG. 4 is a functional block diagram of the control unit of the same as above. [Figure 5A] FIG. 5A is an explanatory diagram showing an example of designing a target travel route by a target travel route design unit. [Figure 5B] FIG. 5B is an explanatory diagram of the same as above. [Figure 6A] FIG. 6A is an explanatory diagram showing how work of the work vehicle progresses based on the designed target travel route. [Figure 6B] FIG. 6B is an explanatory diagram of the same as above. [Figure 6C] FIG. 6C is an explanatory diagram of the same as above. [Figure 7] FIG. 7 is an image diagram of a work status record map created by a work information recording unit. [Figure 8] FIG. 8 is a plan view of the work vehicle for explaining acquisition of field image information by a camera. [Figure 9] FIG. 9 is a plan view of the work vehicle for explaining conversion of field image information acquired by the camera. [Figure 10] FIG. 10 is a flowchart showing a procedure of entire field image creation processing. [Figure 11] FIG. 11 is an explanatory diagram for explaining a rectangular image combining method by a field image combining unit in FIG. 1. [Figure 12] FIG. 12 is an image diagram of a bird's-eye view image of a field displayed on a terminal device. DESCRIPTION OF EMBODIMENTS

[0016] Hereinafter, preferred embodiments of the present invention will be described with reference to the attached drawings. In the following description, unless otherwise specified, the forward direction of the work vehicle 1 will be referred to as "forward" (see Figure 1), the opposite direction as "rear," the right side when facing forward will be referred to as "right," and the left side as "left." The work vehicle 1 body may also be referred to as the machine body.

[0017] <1. Outline of the Work Management System 1> Preferred embodiments of the present invention will be described below with reference to the attached drawings. Figure 1 is a schematic diagram of a work management system 1 according to a preferred embodiment of the present invention. As shown in Figure 1, the work management system 1 comprises a work vehicle 100 that travels through the field H, a work management device 200 that processes information related to work management, and terminal devices 300 for various information, all connected by a network NW to enable the mutual transmission and reception of various information. The network NW may consist of, for example, the Internet, a mobile communication system (e.g., 3G, 4G, 5G, LTE (Long Term Evolution), etc.), Wi-Fi (Wireless Fidelity), Bluetooth (registered trademark), other communication lines, or a combination thereof. Each configuration will be described in detail below.

[0018] <2. Composition of work vehicle 100> Figure 2 is a left side view of the work vehicle 100 according to the embodiment, Figure 3 is a top view of the same, and Figure 4 is a functional block diagram of the control unit of the same. In this embodiment, a tractor is used as an example of the work vehicle 100, but it is not limited to this, and a rice transplanter or the like may also be used. In the following description, unless otherwise specified, the forward direction of the work vehicle 1 is referred to as "forward" (see Figure 1), the opposite direction is referred to as "rear," the right side when facing forward is referred to as "right," and the left side is referred to as "left." The work vehicle 1 body may also be simply referred to as the machine body.

[0019] In this embodiment, the work vehicle 100 is an agricultural tractor that performs work in fields while self-propelled. In addition to performing predetermined tasks while a driver (also called an operator) is on board and driving around the field, the work vehicle 100 can also perform predetermined tasks while automatically driving around the field, controlled by a control system centered on a control device C (see Figure 4), which will be described later, located at an appropriate position on the machine.

[0020] As shown in Figure 2, the work vehicle 100 comprises a vehicle body 2 and a work machine 30. The vehicle body 2 comprises a vehicle frame 3, front wheels 4, rear wheels 5, a bonnet 6, an engine E, a control unit 7, and a transmission case 10. The vehicle frame 3 and the transmission case 10 constitute the main frame of the vehicle body 2.

[0021] The control unit 7 receives input from the operator and functions to control the work vehicle 100, and includes a cabin box 7a that covers the upper part of the vehicle body 2. Inside the cabin box 7a, various operating members that receive input from the operator are arranged, such as a driver's seat 8 where the operator sits and a steering wheel 9. The steering wheel 9 is a component that steers the front wheels 4, which are the steering wheels. In manual operation, the operator steers manually, and in automatic operation, the steering is automatically controlled by a steering device 21 which includes a steering actuator (not shown) that rotates the steering wheel 9.

[0022] The transmission case 10 houses the transmission 22. The power output from engine E is appropriately reduced (shifted) by the transmission and transmitted to the front wheels 4 and rear wheels 5 via the front axle 4j and rear axle 5j, and also transmitted (supplied) to the PTO shaft 16. In addition, a PTO clutch, PTO transmission, and braking device (not shown) are housed in the transmission case 10, and the transmission of power to the PTO shaft (on / off, shifting) can be controlled. As a result, the work vehicle 100 is able to control the drive of the work implement 30.

[0023] A work implement 30 for use in the field is connected to the rear of the vehicle body 2, and a PTO shaft 16 that transmits power to drive the work implement 30 protrudes rearward from the transmission case 10. The PTO shaft 16 transmits rotational power, which has been appropriately reduced by the transmission, to the work implement 30 mounted at least at the rear of the vehicle body 2.

[0024] Furthermore, a lifting device 23 for raising and lowering the work implement 30 is provided at the rear of the vehicle body 2. The lifting device 23 moves the work implement 30 to a non-working position by raising it. The non-working position is, for example, the position in which the work implement 30 is raised when the vehicle body 2 is moving backward or when the vehicle body 2 is turning. The lifting device 23 also moves the work implement 30 to a ground-level working position by lowering it. The lifting device 23 comprises a hydraulic lifting cylinder 24, a lift arm 25, a lift rod 26, a lower link 27, and a top link 28.

[0025] As shown in Figure 3, the work vehicle 100 is equipped with cameras 40 for photographing the field surrounding the vehicle. These cameras 40 are positioned at the front, rear, left, and right of the roof portion (cabin roof) of the cabin 7a, enabling them to photograph the four sides of the work vehicle 1. Specifically, the cameras 40 consist of a front camera 41 for photographing the front of the work vehicle 1, a rear camera 42 for photographing the rear, a left camera 43 for photographing the left side, and a right camera 44 for photographing the right side. This allows the cameras 40 to photograph the area around the work vehicle 100 in a 360° range. The image information of the field H captured and generated by the cameras 40 is transmitted to the control device C, which will be described later. The cameras 40 may also be attached to the roof portion of the cabin 7a via stays.

[0026] The control unit C of the work vehicle 100 is capable of controlling each part by electronic control and includes a processing unit having a CPU (Central Processing Unit), as well as a storage unit consisting of, for example, a hard disk, ROM (Read Only Memory), RAM (Random Access Memory), etc., which stores various programs and necessary data such as target travel routes L (see Figure 6) set for each field H.

[0027] As shown in Figure 4, the control unit C has the following connected to its input side: positioning device 31, azimuth sensor 32, engine speed sensor 33, vehicle speed sensor 34, steering angle sensor 35, PTO speed sensor 36, fuel sensor 37, lift arm sensor 29, camera 40, etc. The control unit C also has the following connected to its output side: engine E, steering device 21, transmission 22, lifting device 23, etc.

[0028] The positioning device 31 performs the function of acquiring the aircraft's position information. For example, it is a GNSS (Global Navigation Satellite System) antenna and can perform positioning and timing by receiving radio waves from the navigation satellite Q orbiting overhead. The azimuth sensor 32 detects the azimuth of the work vehicle 100. The engine speed sensor 33 detects the rotational speed of the engine E. The vehicle speed sensor 34 detects the travel speed (vehicle speed) of the work vehicle 100. The steering angle sensor 35 detects the steering angle of the steering wheels (front wheels). The PTO rotation speed sensor 36 detects the rotational speed of the PTO shaft. The fuel sensor 37 detects the remaining fuel level of the work vehicle 100.

[0029] The control unit C comprises an engine ECU (Electronic Control Unit) 101, a travel system ECU 102, and a work equipment lifting system ECU 103. The engine ECU 101 controls the rotational speed of the engine E. The travel system ECU 102 controls the travel speed of the work vehicle 10, controls the direction of travel during automatic travel by controlling the steering device 21, and controls the rotation of the drive wheels (rear wheels) by controlling the transmission 22. The work equipment lifting system ECU 103 controls the raising and lowering of the work equipment 30 by controlling the lifting device 23. Furthermore, the control unit C can be connected to a network by a communication mechanism (not shown).

[0030] <3. Configuration of terminal device 300> Returning to Figure 1, the terminal device 300 is an information processing device capable of inputting, outputting, and displaying various types of information, and comprises a processor 301, memory 302, storage 303, communication IF 304, input / output IF 305, input unit 307, and display unit 308. These components of the terminal device 300 are connected to each other by a communication bus.

[0031] The processor 301 controls the overall operation of the terminal device 300. The processor 301 may include a CPU, MPU, and GPU. The processor 301 reads a program from the storage 303 and loads it into the memory 302. The processor 301 executes the loaded program.

[0032] Memory 302 is the main memory. Memory 302 is composed of storage devices such as ROM and RAM. Memory 302 provides the processor 301 with a workspace by temporarily storing programs and various data read by the processor 301 from storage 303. Memory 302 also temporarily stores various data generated by the processor 301 while it is operating according to the program.

[0033] Storage 303 is an auxiliary storage device. Storage 303 is comprised of, for example, a storage device such as flash memory or an HDD. Various data related to work management are stored in Storage 303.

[0034] The communication IF304 controls the transmission and reception of various data via the network NW between the terminal device 300 and the work management device 200, etc.

[0035] The input / output IF 305 is an interface for the terminal device 300 to receive data input and an interface for the terminal device 300 to output data. The input / output IF 305 may perform data input and output via, for example, USB (Universal Serial Bus). The input / output IF 305 may include an input unit 307 or a display unit 308, etc.

[0036] The input unit 307 accepts input from the operator (user). The input unit 307 may be a pointing device such as a touchpad.

[0037] The input unit 307 has the function of detecting the position input to the input surface by user operation (e.g., touch operation, tap operation, slide operation, swipe operation, and flick operation, etc.) and transmitting information indicating the detected position as an input signal. The input unit 307 may be, for example, a keyboard, various physical buttons, various sensors (e.g., acceleration sensor, angular velocity sensor, magnetic sensor, GPS sensor, motion sensor, gaze sensor, biopotential sensor, fingerprint sensor, breath sensor, pressure sensor, or image sensor, etc.), an operation stick, a camera, or a microphone, etc.

[0038] The display unit 308 displays various images. The display unit 308 may be, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 18 may be, for example, a projector. The terminal device 300 includes, for example, a touchscreen 306, which is an electronic component combining an input unit 307 and a display unit 308.

[0039] <4. Outline configuration of the work management device 200> The work management device 200 is an information processing device (computer) used to execute processes related to work management. For example, it has mechanical elements such as a CPU and a GPU, and various functions are realized by executing programs stored internally using RAM or the like as a work area. The work management device 200 may be a general-purpose computer such as a workstation or a personal computer. The work management device 200 includes a target travel path design unit 201 for designing the target travel path L, a work information recording unit 202 for creating a work status record map Ms (see Figure 7) that records work information, a creation mode setting unit 203 for setting the creation mode for the entire field image, a field image creation unit 204 for creating the entire field image, and a storage unit 205 for storing various information necessary for work management.

[0040] Furthermore, the field image creation unit 204 includes a field image acquisition unit 211 that acquires image information of the field taken by the work vehicle 100, a field image conversion unit 212 that converts the field image information acquired by the field image acquisition unit 211, a pre / pre-work determination unit 213 that determines whether the field image information is before or after work, and a field image merging unit 214 that combines the field image information. Details of these will be described later.

[0041] The memory unit 205 stores a field map Mh, a work information record map Ms, pre-work field overall image data J1, and pre-work field overall image data J2. The field map Mh is map information about the field and includes information such as the size, shape, and location of the field being worked on, as well as location data of ridges and entrances that define the boundaries of the field. Details of the work information record map Ms, pre-work field overall image data J1, and pre-work field overall image data J2 will be described later.

[0042] <5. Design of the target travel route> Figures 5A and 5B are explanatory diagrams showing an example of a target travel route L designed by the target travel route design unit 201. As shown in Figure 5A, the operator first manually drives the work vehicle 1 to set a reference driving line L0, which serves as the basis for the direction of travel during automatic driving (teaching drive). At this time, the work management device 200 acquires position information from the work vehicle 100 and records position information indicating the driving trajectory of the work vehicle 100 on field H based on the field map Mh. At the start point Ls and end point Le of the reference driving line L0, the operator performs predetermined operations to instruct the input unit 307 to start and end the setting, thereby transmitting the setting instruction information to the work management device 200 and completing the setting of the reference driving line L0. Once the setting of the reference driving line L0 is complete, as shown in Figure 5B, a target driving path L is designed that alternately repeats a straight path Lj and a turning path Lt in order to efficiently travel within field H. At this time, multiple straight paths Lj are designed to be parallel to the reference driving line L0, and their length is designed to be approximately the same as the reference driving line L0. Information regarding the designed target travel route L is linked to the field map Mh, stored in the memory unit 205, and transmitted to the work vehicle 100.

[0043] Furthermore, the spacing between straight paths Lj is determined by the working width of the implement 30, which is set in advance, so that the working areas do not overlap. In addition, the number of straight paths Lj is determined based on information such as the shape and size of the field H contained in the field map Mh stored in the memory unit 205, and as a result, the working area Hr, which is the area to be worked on, is defined. In the illustrated example, six straight paths Lj, indicated by symbols L1 to L6, are designed. In addition, the turning path Lt is a path for the work vehicle 100 to move around, and multiple turning paths Lt are designed to connect the starting point Ls and ending point Le of the straight paths Lj.

[0044] Figures 6A, 6B, and 6C are explanatory diagrams showing how the work of the work vehicle 100 progresses based on the designed target travel path L. Specifically, Figure 6A shows the vehicle traveling along the first straight path L1 (also called the first outbound path), Figure 6B shows the vehicle traveling along the second straight path L2 (also called the first return path), and Figure 6C shows the vehicle traveling along the third straight path L3 (also called the second outbound path). The figures also show the cultivated land area Hk, which represents the area of ​​the field H after work has been done, and the uncultivated land area Hm, which represents the area of ​​the field H where work has not yet been done, both included in the work target area Hr. Information regarding the target travel path L designed by the target travel path design unit 201 is transmitted to the work vehicle 100 via the network NW. Automatic driving of the work vehicle 100 is initiated when an operator performs a predetermined operation to instruct the control unit 7 or terminal device 300 of the work vehicle 100 to start automatic driving. This operation transmits instruction information to the work vehicle 100 from the work management device 200. Upon receiving this instruction information to start automatic driving, the work vehicle 100 is configured to automatically drive along the target driving path L using automatic steering while acquiring the vehicle's position information using the positioning device 31. During this time, the work management device 200 is configured to acquire the position information of the work vehicle 100 as appropriate while the work vehicle 100 is driving automatically.

[0045] <6. Recording of work status> Figure 7 is an illustrative diagram of the work status record map Ms created by the work information recording unit 202. The work status record map Ms is created based on the location information of the work vehicle 100 and the field map Mh. While the work vehicle 100 is automatically driving, as shown in Figure 7, it is map information that shows the pre-work area Rm (areas where no work has been performed), the post-work area Ra (areas where work has been performed), and the working area Rt (areas where work is in progress) for the area Rh of field H. The work management device 200 creates this map in real time, and the created work status record map Ms is updated at predetermined time intervals. This work status record map Ms is used by the determination unit, which will be described later, to determine whether or not work has been performed. The size of the working area Rt is calculated based on the total length, width, and working width W of the work vehicle 100, which are set in advance. Furthermore, the position of the working area Rt on the work status record map Ms is set based on the current position Pm of the work vehicle 100 (the position Pm is calculated from the location information of the current position of the work vehicle 100). In other words, the position Pm of the work vehicle 100 on the work status record map Ms is calculated using the field map Mh and the position information of the work vehicle 100. The post-work area Ra indicates the area that the work vehicle 100 passed through while performing work after the start of work. The pre-work area Rm indicates the area that the work vehicle 100 did not pass through while performing work after the start of work. Note that the example shown in Figure 7 shows an image of the work status record map Ms created in the work situation of Figure 6B. More specifically, the area Rh of field H in the work status record map Ms of Figure 7 indicates the area h2 within the field in Figure 6B, the post-work area Ra in Figure 7 indicates the area of ​​cultivated land Hk in Figure 6B, the pre-work area Rm in Figure 7 indicates the area of ​​uncultivated land Hm in Figure 6B, and the working area Rt in Figure 7 indicates the area where the work vehicle 100 is performing work (the area occupied by the work vehicle 100) in Figure 6B. By referring to the work status record map Ms created as described above, it is possible to determine whether or not work has been performed at a desired location in field H. More specifically, by calculating the location on the work status record map Ms based on the location information of that desired location, if the location on the work status record map Ms belongs to the post-work area Ra, it can be determined that work has been performed (post-work), and if the location on the work status record map Ms belongs to the pre-work area Rm, it can be determined that work has not been performed (pre-work).

[0046] <7. Acquisition and conversion of field image information> Figure 8 is a plan view of a work vehicle used to explain the acquisition of field image information by a camera, and Figure 9 is a plan view of a work vehicle used to explain the conversion of field image information acquired by a camera. As shown in Figure 8, the camera 100 of the work vehicle 100 is configured to capture the area D around the vehicle. Specifically, the front camera 41 captures the area D1 in front of the vehicle, the right camera 44 captures the area D2 to the right of the vehicle, the rear camera 42 captures the area D4 in front of the vehicle, and the left camera 43 captures the area D3 to the left of the vehicle. This makes it possible to capture images of the area around the work vehicle 100 over 360°. The image information of the field captured (acquired) by the camera 100 at predetermined time intervals is transmitted to the work management device 200 each time and acquired by the image information acquisition unit 211.

[0047] Furthermore, the image information of field H acquired by the image information acquisition unit 211 is passed to the field image conversion unit 212, which converts it into multiple rectangular images Df as shown in Figure 9. That is, in Figure 8, the image information of field H is a photograph of the field taken from an oblique angle, but this is converted into multiple rectangular images Df of the field viewed vertically from above using a known image conversion method. More specifically, in this embodiment, based on the image information of field H in the area D surrounding the aircraft that has been photographed, a rectangular image df1 of the left front of the aircraft, a rectangular image df2 of the front of the aircraft, a rectangular image df3 of the right front of the aircraft, a rectangular image df4 of the left side of the aircraft, a rectangular image df5 of the right side of the aircraft, a rectangular image df6 of the left rear of the aircraft, a rectangular image df7 of the rear of the aircraft, and a rectangular image df8 of the right rear of the aircraft are created with the current position Pm of the aircraft as the reference. Simultaneously, positional information for the shooting center points Pf (pf1 to pf8), which indicate the positions of multiple rectangular images Df, is calculated based on the positional information of the aircraft's current position Pm. Furthermore, it is preferable that the image range of each rectangular image Df in the field H be designed to be approximately square, with the front, back, left, and right sides having a working width W.

[0048] <8. Regarding the creation of an overall field image> Next, the creation of a field image by the creation mode setting unit 203 and the field image creation unit 204 will be explained. A field image is image information that shows the entire field before and after work, and consists of pre-work field image data J1, which is the image before work, and post-work field image data J2, which is the image after work (see Figure 11).

[0049] The creation mode setting unit 203 allows the user to select and set one of several creation modes, each with a different method for creating an overall field image. Specifically, this embodiment includes a target travel route reference mode that creates an overall field image using information from the target travel route L, and a work status record reference mode that creates an overall field image using the work status record map Ms. The operator can select the creation mode in advance before starting work by operating the terminal device 300. The creation mode selected by the operator is stored in the storage unit 205.

[0050] Figure 10 is a flowchart showing the procedure for creating an image of the entire field. The process of creating an image of the entire field is initiated by the field information creation unit 210, for example, when the automatic driving of the work vehicle 100 begins (when instruction information to start automatic driving is transmitted from the work management device 200 to the work vehicle 100).

[0051] When the process of creating an overall field image is started, the work management device 200 (field image acquisition unit 211) acquires image information of field H along with the current position information from the work vehicle 100 (step #1). Next, the field image conversion unit 212 converts the field image acquired in step #1 into multiple rectangular images Df (see Figure 9) that view the field from above in a vertical direction (step #2). Subsequently, according to the creation mode set in advance by the creation mode setting unit 203, the rectangular images Df of field H to be combined (used) are selected (step #3), and for each of the selected rectangular images Df, the pre- and post-work determination unit 213 determines whether it is a pre-work image (image of uncultivated land Hm) or a post-work image (image of cultivated land Hk) (step #4). The selection of rectangular images Df according to the creation mode and the determination of pre- and post-work will be described later.

[0052] In step #4, the rectangular image Df determined to be a pre-work image is combined with the pre-work overall field image data J1 by the field image combining unit 214 (steps #5 and #6), and the rectangular image Df determined to be a post-work image is combined with the post-work overall field image data J2 by ​​the field image combining unit 214 (steps #5 and #7). Details on how to combine the rectangular images Df will be described later. Subsequently, when the field image creation is completed (for example, when the automatic driving of the work vehicle 100 is completed), the work management device 200 terminates the overall field image creation process, and if the field image creation is not completed (for example, when the automatic driving of the work vehicle 100 continues), it returns to step #1. Through the above procedure, pre-work overall field image data J1 and post-work overall field image data J2, which are overall field images before and after the work, are created.

[0053] <9. Creation Mode and Method for Combining Rectangular Images Df> If the target travel path reference mode is selected as the creation mode, in step #3 of Figure 10, the rectangular images Df2 in front of the machine, df4 to the left of the machine, df5 to the right of the machine, and df7 to the rear of the machine are selected as the rectangular images Df of the field H to be combined (used) (see Figure 9). Subsequently, in step #4, a determination is made as to whether each of the selected rectangular images Df is before or after work. Specifically, when the work vehicle 100 is traveling along the first straight path L1 (see Figure 5B), the rectangular images df2 in front of the machine, df4 to the left of the machine, and df5 to the right of the machine are determined to be images before work, and the rectangular image df7 to the rear of the machine is determined to be an image after work.

[0054] Furthermore, during travel along the subsequent straight path Lj (straight path L2~L6), when traveling along the straight path Lj after a right turn, the rectangular image df5 on the right side of the aircraft is determined to be the image after the operation, and the rectangular image df4 on the left side of the aircraft is determined to be the image before the operation. Similarly, when traveling along the straight path Lj after a left turn, the rectangular image df4 on the left side of the aircraft is determined to be the image after the operation, and the rectangular image df5 on the right side of the aircraft is determined to be the image before the operation. Regardless of whether it is a right or left turn, the rectangular image df2 in front of the aircraft is determined to be the image before the operation, and the rectangular image df7 behind the aircraft is determined to be the image after the operation. This determination method allows for accurate determination of whether the rectangular image Df is before or after the operation based on the information of the target travel path L. Furthermore, by acquiring images of the work completed from the left and right sides of the work vehicle 100 (rectangular image df4 on the left side of the machine, or rectangular image df5 on the right side of the machine), it is possible to effectively prevent the images from becoming unclear due to dust and other debris generated immediately after the work vehicle passes by.

[0055] Furthermore, if the work status record reference mode is selected as the creation mode, in step #3 of Figure 10, the following rectangular images Df of the field H to be combined (used) are selected based on the current position Pm of the aircraft: rectangular image df1 to the left front of the aircraft, rectangular image df2 to the front of the aircraft, rectangular image df3 to the right front of the aircraft, rectangular image df4 to the left of the aircraft, rectangular image df5 to the right of the aircraft, rectangular image df6 to the left rear of the aircraft, rectangular image df7 to the rear of the aircraft, and rectangular image df8 to the right rear of the aircraft (see Figure 9). Subsequently, in step #4, a determination is made as to whether each selected rectangular image Df is before or after work. Specifically, by referring to the work information record map Ms, if the position of the shooting center point Pf (pf1~pf8) of each rectangular image Df belongs to the pre-work area Rm, it is determined to be a pre-work image, and if it belongs to the post-work area Ra, it is determined to be a post-work image. In the example shown in Figure 7, the center points (pf2, pf4) of the rectangular image df2 in front of the aircraft and the rectangular image df4 to the left of the aircraft belong to the pre-work region Rm, therefore, the rectangular image df2 in front of the aircraft and the rectangular image df4 to the left of the aircraft are determined to be pre-work images. Similarly, the center points Pf (pf5, pf7) of the rectangular image df2 to the right of the aircraft and the rectangular image df4 to the left of the aircraft belong to the post-work region Ra, therefore, the rectangular image df2 in front of the aircraft and the rectangular image df to the left of the aircraft are determined to be pre-work images. The center points Pf of the rectangular image df1 to the left front, the rectangular image df3 to the right front of the aircraft, the rectangular image df6 to the left rear of the aircraft, and the rectangular image df8 to the right rear of the aircraft are not shown in Figure 7, but are determined in a similar manner. With this determination method, the pre- and post-work determination of rectangular images Df can be performed accurately and well based on the information in the work information recording map Ms. Furthermore, by acquiring images after the work is completed from the front of the work vehicle 100 (a rectangular image df1 to the left front, or a rectangular image df3 to the right front of the machine), it is possible to effectively prevent the image from becoming unclear due to dust and other debris generated immediately after the work vehicle passes.

[0056] Next, we will explain the method for combining the rectangular images Df, referring to Figure 11. Figure 11 is an explanatory diagram illustrating the method of combining rectangular images Df using the field image joining unit 214. As shown in Figure 11, in step #4, the rectangular image Df determined to be before work is combined with the pre-work field overall image data J1, and the rectangular image Df determined to be after work is combined with the pre-work field overall image data J1. Based on the field map Mh, the pre-work field overall image data J1 defines the outline Jh1 of the image representing the entire field H. Within this outline Jh1, the position of the shooting center point Pf in field H is calculated from the position information of the shooting center point Pf, and the rectangular image Df is placed and combined at the corresponding position within the outline Jh1. Similarly, the rectangular image Df determined to be post-work is combined with the post-work field image data J2. Based on the field map Mh, the post-work field image data J2 defines an image outline Jh2 representing the entire field H. Within this outline Jh2, the position of the shooting center point Pf in field H is calculated from the position information of the shooting center point Pf, and the rectangular image Df is placed and combined at the corresponding position within the outline Jh2. When combining the rectangular image Df, any overlapping parts of the rectangular image Df may be overwritten, or only the parts where the images do not overlap may be combined. As a result, the pre-work field image data J1 and post-work field image data J2 are created in real time, and the worker can check the post-work field image data J1 and post-work field image data J2 displayed on the terminal device 300 in real time. This allows the worker to easily and quickly check images of field H before and after work. Furthermore, areas (locations) in the outlines Jh1 and Jh2 of the image showing the entire field H where the rectangular image Df has not been acquired will be displayed in white, for example. Also, since the rectangular image df7 at the rear of the machine is prone to becoming unclear due to dust and other debris from the implement 30, the field image joining unit 214 may be configured such that, when joining rectangular images Df, the priority of overwriting the rectangular image df7 at the rear of the machine is lowered compared to the rectangular images df7 at locations other than the rear of the machine, so that in the image of the entire field, rectangular images Df acquired from locations other than the rectangular images df7 are displayed preferentially. This makes it possible to display a clearer image of the entire field.

[0057] <10. Around View> The work management device 200 is capable of displaying a bird's-eye view (around view) G of field H on the terminal device 300 in real time during work, based on image information acquired from the camera 40. Figure 12 is an illustrative diagram of the bird's-eye view G of field H displayed on the terminal device 300. As shown in the figure, the bird's-eye view G displays information such as the field g1, work vehicle g2, target travel route g3, status of work vehicle 100 g4 (in detail, vehicle speed, engine speed, rotation speed of the PTO shaft 16, and fuel level), and work progress g5 (in detail, the percentage of the work target area Hr occupied by the already cultivated area Hk is calculated and visually displayed). The work vehicle 100 is displayed as a computer graphics (CG), and the raising and lowering of the implement 30 is visually changed based on detection information from the lift arm sensor 29. The CG of the implement 30 may be changed according to the implement information. The CG of the front wheels 4 may be changed based on the steering sensor value or steering instruction value. The position information of the work vehicle 100 is acquired and the target travel path Lwo is superimposed and displayed on the camera image. The target travel path L is superimposed so that it appears to be attached to the field scene, and the parts hidden by the work vehicle g2 are displayed so that the front-to-back relationship between the work vehicle g2 and the target travel path g3 can be seen.

[0058] <11. Others> The embodiments of the present invention have been described above. The present invention is not limited to the embodiments described above. It goes without saying that modifications can be made as appropriate within the scope of the technical idea.

[0059] The rectangular image Df selected in step #3 of Figure 10 may be configured so that the operator can arbitrarily set it in advance from the following: rectangular image df1 for the left front of the aircraft, rectangular image df2 for the front of the aircraft, rectangular image df3 for the right front of the aircraft, rectangular image df4 for the left side of the aircraft, rectangular image df5 for the right side of the aircraft, rectangular image df6 for the left rear of the aircraft, rectangular image df7 for the rear of the aircraft, and rectangular image df8 for the right rear of the aircraft.

[0060] The work management device 200 may be configured to acquire settings for the work machine 30 and, if the work position is offset from the travel position of the work vehicle 100, to acquire an image according to the offset position.

[0061] The system may be configured to acquire a post-work image using the camera 40 on the rear side if it can be detected from the lifting position of the work equipment 30 or the rotation speed of the PTO shaft 16 that work was being performed before the work vehicle 100 rotates.

[0062] In the above embodiment, the work management device 200 was shown as being configured to manage one field H and one work vehicle 100, but it may also be configured to manage multiple fields H and work vehicles 100. [Explanation of symbols]

[0063] 1. Work Management System 2. Running vehicle 3. Vehicle frame 4 Front wheels 5 Rear wheels 6. Hood 7. Control Unit 7a Cabin 8. Cockpit 9 Steering wheel 10 Mission Case 16 PTO shafts 21 Steering gear 22 Transmission 23 Lifting device 24 Lifting Cylinder 25 Lift Arm 26 Lift Rod 27 Lower Link 28 Top Links 29. Lift arm sensor 30 Work Machines 31 Positioning device 32. Azimuth sensor 33. Engine speed sensor 34. Vehicle speed sensor 35 Steering angle sensor 36 PTO rotation speed sensor 37 Fuel Sensor 40 Cameras 41 Front Camera 42 Rear camera 43 Left camera 44 Right camera 100 work vehicles 101 Engine ECU 102 Driving system ECU 103 Work equipment lifting system ECU 200 Work management device 300 terminal devices 301 Processor 302 memory 303 Storage 304 Communication IF 305 Input / Output Interface 306 Touchscreen 307 Input section 308 Display section Df Rectangular image G Bird's-eye view g1 field g2 work vehicle g3 Target driving route g4 Condition of the work vehicle G5 Work progress Df(df1~df8) Rectangular image Field H L Target driving route NW Network Pm Aircraft's current position Pf (pf1~pf8) Center point of shooting Q Navigation satellite Rh field area Ra after work area Rm pre-work area W Working width

Claims

1. A work management system comprising a work vehicle that travels in a field, a work management device that processes information related to work management, and terminal devices for various types of information, The aforementioned work vehicle is configured to acquire positional information using a positioning device and to automatically travel within the field along a target route, and is equipped with a camera that takes pictures of the area around the vehicle to generate image information of the field. The work management device comprises a field image acquisition unit that acquires image information of the field from the work vehicle, a pre- and post-work determination unit that determines whether the acquired field image information is of the field before or after work, and a field image merging unit that separates and merges the field images determined by the pre- and post-work determination unit into images of the field before work and images of the field determined to be after work. The work management system is characterized in that the field image merging unit generates pre-work field image data, which is an image of the entire field before work, and post-work field image data, which is an image of the entire field after work, and is configured to display the images of the entire field before and after work on the terminal device.

2. The work management device includes a field image conversion unit that converts the acquired field image information into multiple rectangular images of the field viewed vertically from above, The field image conversion unit generates rectangular images showing images of the field in at least four directions: front, rear, left, and right of the work vehicle. The work management system according to claim 1, characterized in that the pre- and post-work determination unit determines, when the work vehicle is traveling along a straight path after a right turn, the rectangular image on the right side of the machine is a post-work image and the rectangular image on the left side of the machine is a pre-work image, and when the work vehicle is traveling along a straight path after a left turn, the rectangular image on the left side of the machine is a post-work image and the rectangular image on the right side of the machine is a pre-work image.

3. The work management device includes a field image conversion unit that converts acquired field image information into multiple rectangular images of the field viewed vertically from above, and a work status recording unit that creates a work status record map that records work information of work vehicles. The field image conversion unit generates rectangular images showing images of the field in at least four directions: front, rear, left, and right of the work vehicle. The aforementioned work status recording unit is configured to record the pre-work area, which is the area in the field where no work has been performed, and the post-work area, which is the area where work has been performed, on the work status recording map. The work management system according to claim 1, wherein the pre- and post-work determination unit determines whether the rectangular image is a pre- or post-work image based on the work status record map and information on the shooting center point indicating the position of the rectangular image on the field.

4. The work management system according to claim 2, characterized in that when the field image joining unit joins the rectangular images, it lowers the priority of overwriting the rectangular image at the rear of the machine to the priority of rectangular images other than those at the rear of the machine, so that in the display of the overall field image, rectangular images acquired from areas other than the rear of the machine are displayed preferentially.

Images