Control method for work vehicles

Abstract

This invention provides a control method for work vehicles that allows operators to easily perform operations and set parameters related to driving and work without requiring extra effort from the operator, thereby improving work efficiency. [Solution] The work vehicle 1, which travels and performs work in the field, includes a display control unit 62 that displays various information on the terminal display unit 53 of the mobile terminal 5, a vehicle imaging unit 34 that captures images of workers, including the worker of the work vehicle 1, a driving posture determination unit 60 that determines the driving posture of the worker captured in the worker image based on a learning model that has been trained on training images of the worker, and an operation candidate determination unit 61 that determines an operation candidate for the work vehicle 1 corresponding to the driving posture determined by the driving posture determination unit 60, based on a learning model that has been trained on the worker's driving posture information and operation candidate information for the work vehicle 1. The display control unit 62 displays a screen on the terminal display unit 53 according to the operation candidate determined by the operation candidate determination unit 61.

Description

Technical Field

[0001] The present invention relates to a control method for controlling a work vehicle that travels and works on a farm field.

Background Art

[0002] Conventionally, a work vehicle such as a tractor is equipped with a working machine that works on a farm field, and performs work with the working machine while manually or automatically traveling. Further, the work vehicle includes a display unit such as a touch panel that displays a setting screen on which various parameters related to traveling and work can be set.

[0003] For example, in Patent Document 1, a remote operation device for a construction machine includes a display device that displays a work video at a remote location. When it is determined by an operation determination unit that the operator's operation is a fine operation and it is determined by a posture determination unit that the operator's posture has changed from a reference posture to a gaze posture, this remote operation device includes a video generation unit that generates an enlarged video obtained by an eye tracking device in a portion related to the gaze position in the work video of the construction machine and displays the enlarged video on the display device. Thereby, workability can be improved by displaying the video required by the operator clearly at an appropriate timing.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, with conventional work vehicles, changing the settings of various parameters requires switching from the home screen to the parameter settings screen on the display unit. The parameter settings screens have a hierarchical structure with the home screen at the top, and the operator must manually switch to the desired settings screen and then manually set the parameters on that screen. As a result, in addition to driving and operating the vehicle, the operator has to spend time setting parameters, which may reduce work efficiency.

[0006] The present invention aims to provide a control method for a work vehicle that allows operators to easily perform operations and set parameters related to driving and work without requiring extra effort from the operator, thereby improving work efficiency. [Means for solving the problem]

[0007] To solve the above problems, the present invention provides a control method for controlling a work vehicle that travels and works in a field, comprising: a display step of displaying various information on a display unit; an imaging step of capturing an image of a worker, including the worker of the work vehicle; a driving posture determination step of determining the driving posture of the worker captured in the worker image based on a learning model that has been trained on training images of the worker; and an operation candidate determination step of determining an operation candidate for the work vehicle corresponding to the driving posture determined in the driving posture determination step, based on a learning model that has been trained on the worker's driving posture information and operation candidate information for the work vehicle, wherein the display step displays a screen on the display unit according to the operation candidate determined in the operation candidate determination step. [Effects of the Invention]

[0008] The present invention provides a control method for a work vehicle that allows operators to easily perform operations and set parameters related to driving and work without requiring extra effort from the operator, thereby improving work efficiency. [Brief explanation of the drawing]

[0009] [Figure 1] This is a side view showing an example of a work vehicle according to an embodiment of the present invention. [Figure 2] This is a block diagram of a work vehicle according to an embodiment of the present invention. [Figure 3] This table shows examples of the operator's driving posture, candidate operation of the work vehicle, display control, and voice input control determined by a work vehicle according to an embodiment of the present invention. [Figure 4] This is a plan view showing an example of a home screen in a work vehicle according to an embodiment of the present invention. [Figure 5] This is a plan view showing an example of a horizontal control setting screen in a work vehicle according to an embodiment of the present invention. [Figure 6] This is a plan view showing another example of the home screen in a work vehicle according to an embodiment of the present invention. [Figure 7] This table shows examples of voice inputs received in voice input permission mode in a work vehicle according to an embodiment of the present invention. [Modes for carrying out the invention]

[0010] The following describes a work vehicle 1 according to one embodiment of the present invention with reference to the drawings. The work vehicle 1 of the present invention performs work by driving on a field and working with an implement 3, either by automatic driving or manual operation. The work vehicle 1 of the present invention is an agricultural vehicle such as a tractor, combine harvester, rice transplanter, or lawnmower, and in the following embodiment, the work vehicle 1 being a tractor will be described.

[0011] As shown in Figure 1, the tractor-type work vehicle 1 comprises a body 2 and implements 3, and is configured to perform tasks such as tilling using the implements 3 while the vehicle 2 moves. Various implements 3, such as rotary tillers, harrows, loaders, plows, and box scrapers, are attached to the vehicle 2 as needed. Furthermore, as shown in Figure 2, the work vehicle 1 is equipped with a control device 4 that controls the operation of each part.

[0012] In work vehicle 1, either manual driving mode or automatic driving mode is set. When manual driving mode is set, work vehicle 1 is driven manually within the field in response to the operator's operation of various control devices (steering wheel, accelerator pedal, shift lever, etc.).

[0013] When the automatic driving mode is set, the control device 4 controls the vehicle speed and steering of the work vehicle 1 so that it drives automatically along the set driving path, and the control device 4 also controls the work machine 3 so that it performs automatic work associated with the automatic driving.

[0014] For example, when operating a work vehicle 1 automatically to perform work in a field, first, the work vehicle 1 is driven around the outer perimeter of the field by manual operation or other means to acquire information about the field's external shape. Then, based on the acquired information, the work vehicle 1 is operated automatically in the central work area of ​​the field to perform work, such as back-and-forth driving along multiple straight paths in the central work area, or circular driving where the vehicle repeatedly circles a straight path while shifting from the center of the field outwards. Furthermore, the work vehicle 1 is operated automatically in the headland area surrounding the central work area, following the outer perimeter of the field.

[0015] A pair of front wheels 10 are provided on the lower front of the vehicle body 2, and a pair of rear wheels 11 are provided on the lower rear of the vehicle body 2. A cabin 12 for the operator (worker) is provided on the upper part of the vehicle body 2, and the interior of the cabin 12 is equipped with a driver's seat and various operating devices (steering wheel, accelerator pedal, shift lever, etc.).

[0016] The vehicle body 2 is also equipped with an engine 13, a transmission 14, a work equipment lifting mechanism 15, a positioning unit 16 (see Figure 2), and a control device 4.

[0017] An engine 13 is housed in the front of the vehicle body 2, and a transmission 14 is provided between the pair of rear wheels 11. The power from the engine 13 is shifted by the transmission 14 and transmitted to each of the front wheels 10 and each of the rear wheels 11.

[0018] At the rear of the transmission 14, a pair of left and right lower links 18, a top link 19, and a PTO shaft 20 are connected. Each of the lower links 18, the top link 19, and the PTO shaft 20 extends rearward. The working machine 3 is connected to the rear ends of each of the lower links 18, the top link 19, and the PTO shaft 20 and is driven by the PTO shaft 20.

[0019] The working machine lifting mechanism 15 includes a pair of left and right lift arms 21 and a lift cylinder 22 composed of a hydraulic cylinder. The tip of one lift arm 21 is connected to one lower link 18 via a link member 23, and the tip of the other lift arm 21 is connected to the other lower link 18 via a rolling cylinder 24. The working machine lifting mechanism 15 can change the height of the working machine 3 supported by the vehicle body 2 by driving the lift cylinder 22.

[0020] The working machine 3 is composed of, for example, a rotary tiller or a harrow for secondary tillage and includes a tillage cover 25 extending in the left - right direction. A tillage rotating shaft 26 having a rotating shaft extending in the left - right direction is rotatably attached to the tillage cover 25. The tillage rotating shaft 26 rotates by the power transmitted from the PTO shaft 20. A plurality of tillage claws 27 are provided at intervals in the left - right direction on the tillage rotating shaft 26. The working machine 3 is configured such that the tillage rotating shaft 26 and the tillage claws 27 rotate with respect to the soil in the field for tillage.

[0021] The positioning unit 16 is configured to acquire the position information (positioning point) of the work vehicle 1 using a satellite positioning system such as GNSS or a relative positioning system such as RTK. It receives a positioning signal from a positioning satellite via a positioning antenna and acquires the position information of the positioning unit 16, that is, the position information of the work vehicle 1 based on the positioning signal.

[0022] In the cabin 12, the area around the driver's seat is equipped with a vehicle display unit 30 such as an LCD display, a vehicle audio output unit 31 such as a speaker, a vehicle operation unit 32 such as a keyboard and various buttons, and a vehicle sound collection unit 33 such as a microphone. The vehicle display unit 30 is controlled by the control device 4 to display various information to the operator. The vehicle audio output unit 31 is controlled by the control device 4 to output various information to the operator as audio. The vehicle operation unit 32 is controlled by the control device 4 to receive information input from the operator via manual operation. The vehicle sound collection unit 33 is controlled by the control device 4 to receive information input via voice input from the operator. The vehicle display unit 30 and the vehicle operation unit 32 may be configured as an integrated touch panel.

[0023] Furthermore, the vehicle body 2 is equipped with a vehicle imaging unit 34 that captures images of a worker seated in the driver's seat inside the cabin 12. For example, the vehicle imaging unit 34 may be located at the front or rear of the upper part of the cabin 12. Multiple vehicle imaging units 34 may be located at multiple locations inside the cabin 12. The vehicle imaging unit 34 is controlled by the control device 4 to capture images of the worker seated in the driver's seat and input them to the control device 4.

[0024] Next, the control device 4 will be described. The control device 4 is composed of a computer such as a CPU and, as shown in Figure 2, is connected to a storage unit 41 such as ROM, RAM, hard disk drive, and flash memory, and a communication unit 42 that communicates with external devices such as a management server 6 that manages the mobile terminal 5 and the work vehicle 1, and an artificial intelligence server 7. Although Figure 2 illustrates an example in which the management server 6 and the artificial intelligence server 7 are provided separately, the artificial intelligence server 7 may be included in the management server 6.

[0025] The storage unit 41 stores programs and data for controlling various components and functions of the work vehicle 1, and the control device 4 controls the various components and functions by executing calculations based on the programs and data stored in the storage unit 41. For example, the control device 4 controls the positioning unit 16 to acquire the position of the work vehicle 1. The control device 4 operates as an imaging control unit 45 and an operation control unit 46 by executing the programs stored in the storage unit 41. The imaging control unit 45 and the operation control unit 46 implement the imaging process and the operation control process of the control method for the work vehicle 1 according to the present invention.

[0026] The communication unit 42 can communicate wirelessly with external devices such as the mobile terminal 5 held by the worker, the management server 6, and the artificial intelligence server 7 via a wireless communication antenna. The control device 4 controls the communication unit 42 to communicate wirelessly with the mobile terminal 5, the management server 6, and the artificial intelligence server 7, and sends and receives various information between the mobile terminal 5, the management server 6, and the artificial intelligence server 7.

[0027] The imaging control unit 45 controls the vehicle imaging unit 34 at predetermined imaging timings to capture images of workers, including those seated in the driver's seat inside the cabin 12. The imaging control unit 45 receives these worker images from the vehicle imaging unit 34 and transmits them to the mobile terminal 5 via the communication unit 42. For example, the imaging control unit 45 may use the start of the engine 13 as the imaging timing to have the vehicle imaging unit 34 capture images of workers. In addition, the imaging control unit 45 may use the elapsed time of a predetermined time interval during the operation or work of the work vehicle 1 as the imaging timing to have the vehicle imaging unit 34 capture images of workers so that changes in the worker's driving posture can always be detected. If multiple vehicle imaging units 34 are provided, the imaging control unit 45 may have each of the multiple vehicle imaging units 34 capture images of workers.

[0028] The driving control unit 46 controls the vehicle speed and steering of the work vehicle 1 by controlling the engine 13, the transmission 14 and the steering device (not shown), and controls the operation of the work implement 3 by controlling the work implement lifting mechanism 15 and the PTO shaft 20, thereby controlling the driving operation of the work vehicle 1.

[0029] For example, when the automatic driving mode is set, the driving control unit 46 controls the automatic driving of the work vehicle 1 based on the driving path set for the field. When automatic driving starts, the driving control unit 46 obtains the position of the work vehicle 1 from the positioning unit 16 and controls each part so that the work vehicle 1 performs automatic driving work along the driving path based on the position of the work vehicle 1, field information and the driving path.

[0030] Furthermore, the driving control unit 46 controls the movement of the work vehicle 1 to start straight-line assist driving in response to a signal indicating that the vehicle will travel in a straight line without requiring the operator to operate the steering wheel, on a straight-line driving path among the driving paths set for the field. For example, the driving control unit 46 receives a signal indicating the start of straight-line assist driving from the mobile terminal 5 via the communication unit 42.

[0031] The operation control unit 46 controls the movement of the work vehicle 1 to start automatic turning in response to an automatic turning start signal, which is a signal that enables the vehicle to turn along a set travel path in the field without requiring the operator to operate the steering wheel. For example, the operation control unit 46 receives an automatic turning start signal from the mobile terminal 5 via the communication unit 42.

[0032] The mobile terminal 5 is one of the components of the work vehicle 1 and is a terminal capable of remotely controlling the work vehicle 1. For example, it can be a tablet terminal equipped with a touch panel or a notebook-type personal computer. In this invention, the work vehicle 1 and the mobile terminal 5 constitute a control system for the work vehicle 1.

[0033] As shown in Figure 2, the mobile terminal 5 is equipped with a control device 50 which consists of a computer such as a CPU. The control device 50 is connected to a storage unit 51 which includes ROM, RAM, a hard disk drive, and flash memory, and to a communication unit 52 which communicates with external devices such as the work vehicle 1, the management server 6, and the artificial intelligence server 7.

[0034] The mobile terminal 5 is equipped with a terminal display unit 53 such as an LCD display, a terminal audio output unit 54 such as a speaker, a terminal operation unit 55 such as a keyboard and various buttons, and a terminal sound collection unit 56 such as a microphone. The terminal display unit 53 is controlled by the control device 50 to display various information to the worker. The terminal audio output unit 54 is controlled by the control device 50 to output various information to the worker as audio. The terminal operation unit 55 is controlled by the control device 50 to accept information input by the worker through manual operation. The terminal sound collection unit 56 is controlled by the control device 50 to accept information input by voice input from the worker. The terminal display unit 53 and the terminal operation unit 55 may be configured as an integrated touch panel.

[0035] The mobile terminal 5 has a terminal imaging unit 57 for capturing images of a worker seated in the driver's seat inside the cabin 12. The terminal imaging unit 57 is controlled by the control device 50 to capture an image of the worker seated in the driver's seat and input it to the control device 50.

[0036] The memory unit 51 stores programs and data for controlling various components and functions of the mobile terminal 5, and the control device 50 controls the various components and functions of the mobile terminal 5 by performing calculations based on the programs and data stored in the memory unit 51.

[0037] The storage unit 51 stores field information of the field that the work vehicle 1 is working on. This field information includes, for example, the shape, size, and location information (coordinates, etc.) of the field edges that constitute the outer perimeter of the field, as well as the shape, size, and location information (coordinates, etc.) of the work area of ​​the field. The field information also includes information such as the shape, size, and location information (coordinates, etc.) of unworked areas where work has not yet been performed, and completed work areas where work has already been completed.

[0038] The communication unit 52 can communicate wirelessly with external devices such as the work vehicle 1, the management server 6, and the artificial intelligence server 7 via a wireless communication antenna. The control device 50 controls the communication unit 52 to communicate wirelessly with the work vehicle 1, the management server 6, and the artificial intelligence server 7, and transmits and receives various information between the work vehicle 1, the management server 6, and the artificial intelligence server 7.

[0039] Furthermore, the control device 50 operates as a driving posture determination unit 60, an operation candidate determination unit 61, a display control unit 62, a voice input control unit 63, and a route creation unit 64 by executing a program stored in the memory unit 51. The driving posture determination unit 60, the operation candidate determination unit 61, the display control unit 62, the voice input control unit 63, and the route creation unit 64 realize the driving posture determination step, the operation candidate determination step, the display step, the voice input step, and the route creation step of the control method for the work vehicle 1 according to the present invention.

[0040] The driving posture determination unit 60 determines the driving posture of a worker based on the worker image captured in the worker image when the mobile terminal 5 receives a worker image captured by the vehicle imaging unit 34 of the work vehicle 1 from the work vehicle 1. The driving posture determination unit 60 may determine the driving posture by inputting a worker image captured by another imaging device, such as the terminal imaging unit 57, not just the vehicle imaging unit 34 of the work vehicle 1. For example, the driving posture determination unit 60 may determine the worker's static posture, gaze, dynamic posture (movement), etc., as a specific driving posture. Figure 3 shows an example of a worker's driving posture determined by the driving posture determination unit 60.

[0041] Specifically, regarding static posture, the driving posture determination unit 60 determines that a forward-facing posture, where the worker seated in the driver's seat is facing forward, is the normal posture. The driving posture determination unit 60 also determines other static postures, such as a rearward-facing posture, where the worker seated in the driver's seat is facing backward, or a sideways posture, where the worker is facing to the right or left. Furthermore, for the forward-facing, rearward-facing, and sideways postures, the driving posture determination unit 60 determines whether the worker is facing nearby or far away.

[0042] Regarding the line of sight, the driving posture determination unit 60 determines whether the operator seated in the driver's seat is looking at the mirrors to check the work behind them via the side mirrors or rearview mirror.

[0043] Furthermore, the driving posture determination unit 60 determines the diagonal forward line of sight when the driver is looking diagonally forward to check the front wheels 10 or the front end of the vehicle body 2 (for example, the front end of the harvesting part such as a divider if the work vehicle 1 is a combine harvester) diagonally forward to check the distant diagonal forward area when the driver is driving.

[0044] Furthermore, the driving posture determination unit 60 determines the lateral near line of sight, in which the driver is intently looking to the sides to check the positions of the front wheels 10 and rear wheels 11 and the ends of the work equipment 3 below and to the left and right while driving, and the lateral far line of sight, in which the driver is intently looking to the sides to check the vicinity of adjacent driving paths to the left and far away while driving.

[0045] Regarding dynamic posture, the driving posture determination unit 60 determines whether the vehicle is moving backward from a forward-facing posture to a rearward-facing posture, moving forward from a rearward-facing posture to a forward-facing posture, or moving sideways from a forward-facing posture to a sideways-facing posture.

[0046] The driving posture determination unit 60 may use artificial intelligence (AI) to determine the worker's driving posture based on the worker's image, and may use a generating AI to create the driving posture information resulting from the determination. For example, the driving posture determination unit 60 generates a machine learning model by machine learning a training image of a worker seated in the driver's seat of the work vehicle 1, and stores it in the storage unit 51 or the artificial intelligence server 7. Then, the driving posture determination unit 60 uses the machine learning model to analyze the captured worker image using AI and determines the worker's driving posture included in the worker image.

[0047] The operation candidate determination unit 61 estimates and determines candidate operations (operation candidates) that the worker will perform for driving the work vehicle 1 or for performing work, based on the worker's driving posture determined by the driving posture determination unit 60. Figure 3 shows examples of operation candidates for the work vehicle 1 determined by the operation candidate determination unit 61.

[0048] For example, the operation candidate determination unit 61 determines that the operation candidate is either the operation of the work implement 3 (work implement operation) or the operation when checking the rear area worked on by the work implement 3 (rearward check operation) when the driving posture is a static rearward posture. The operation candidate determination unit 61 determines that the operation candidate is the operation when the work vehicle 1 is moving forward (forward driving operation) when the driving posture is a static forward posture and the driver is looking through the mirrors. The operation candidate determination unit 61 determines that the operation candidate is the operation of the work implement 3 via the mirrors (work implement operation) when the driving posture is a static forward posture and the driver is looking through the mirrors.

[0049] The operation candidate determination unit 61 determines that an operation to drive the work vehicle 1 in a straight line along the edge or outer perimeter of a field (straight-line driving operation) is an operation candidate when the driving posture is a static forward-facing posture and the line of sight is diagonally forward or nearby, or when the driving posture is a static sideways posture and the line of sight is to the side or nearby. The operation candidate determination unit 61 determines that an operation to drive the work vehicle 1 in a turning position (turning driving operation) is an operation candidate when the driving posture is a static forward-facing posture and the line of sight is diagonally forward or far away, or when the driving posture is a static sideways posture and the line of sight is to the side or far away.

[0050] Furthermore, the operation candidate determination unit 61 determines, as an operation candidate, that the driving posture is in a dynamic backward-facing position, that the operation is for operating a work implement, or that the operation is for checking behind, or that the operation is for driving the work vehicle 1 in reverse (operation during reverse driving). The operation candidate determination unit 61 determines, as an operation candidate, that the operation candidate is for driving forward, when the driving posture is in a dynamic forward-facing position. The operation candidate determination unit 61 determines, as an operation candidate, that the operation candidate is for driving while turning, when the driving posture is in a dynamic sideways-facing position.

[0051] The operation candidate determination unit 61 may use artificial intelligence (AI) to determine operation candidates for the work vehicle 1 based on the operator's driving posture, and may use a generating AI to create information on the operation candidates and instructions based on the determination results. For example, the operation candidate determination unit 61 generates a machine learning model by machine learning the driving posture information related to various driving postures of the operator and the operation candidate information related to various operation candidates for the work vehicle 1, and stores it in the memory unit 51 or the artificial intelligence server 7. Then, the operation candidate determination unit 61 uses the machine learning model of the driving posture information and operation candidate information to analyze the operator's driving posture determined by the driving posture determination unit 60, and determines the operation candidates for the work vehicle 1 based on the operator's driving posture.

[0052] The operation candidate determination unit 61 may determine the operation candidate for the work vehicle 1 by considering not only the operator's driving posture, but also the driving status of the work vehicle 1, the working conditions of the field, the operator's operating tendencies, etc. For example, the operation candidate determination unit 61 may use a machine learning model that has been trained on the driving status of the work vehicle 1, the working conditions of the field, the operator's operating tendencies, etc., in addition to the driving posture information, to determine the operation candidate for the work vehicle 1 based on the operator's driving posture using AI.

[0053] Here, if there are multiple operation candidates for the work vehicle 1 corresponding to the operator's driving posture, the operation candidate determination unit 61 may determine one operation candidate based on the operator's behavior information, as well as the driving status of the work vehicle 1, the working conditions of the field, the operator's operating tendencies, etc. Alternatively, if there are multiple operation candidates for the work vehicle 1 corresponding to the operator's driving posture, the operation candidate determination unit 61 may pre-set one operation candidate that is determined in accordance with the driving posture determination by the driving posture determination unit 60, according to any operation.

[0054] The display control unit 62 displays screens on the terminal display unit 53 that provide guidance on the driving and operation of the work vehicle 1, as well as screens for setting various parameters. For example, in a hierarchical structure with the home screen 70 at the top and various guidance screens and various setting screens below it, as shown in Figure 4, the display control unit 62 displays the home screen 70 when the engine 13 is started.

[0055] The display control unit 62 displays each screen according to the hierarchical structure; in other words, it displays the guidance screen or setting screen of the next level in response to screen transition operations on the currently displayed screen (home screen 70, guidance screen, setting screen). For example, the display control unit 62 displays the horizontal control setting screen 75 for the work machine 3, as shown in Figure 5, in response to the operation of the UFO setting button 71 on the home screen 70. Regardless of the type of current screen (home screen 70, guidance screen, setting screen), it is preferable that the display control unit 62 display shortcut buttons 72 that accept operations such as driving the work vehicle 1, operating the work, and setting parameters, at the same position on each screen.

[0056] Furthermore, when the operation candidate determination unit 61 determines an operation candidate based on the operator's driving posture, the display control unit 62 displays the screen according to the operation candidate. At this time, the display control unit 62 may, depending on the operation candidate, transition the screen currently displayed on the terminal display unit 53 (current screen) to the operation screen corresponding to the operation candidate, change a part of the current screen to correspond to the operation candidate and display it, or change a button on the current screen that transitions to another screen (one shortcut button 72) to a button that transitions to the operation screen corresponding to the operation candidate (another shortcut button 72') and display it. Figure 3 illustrates the display control of the screen by the display control unit 62.

[0057] For example, if the display control unit 62 determines that work equipment operation is a candidate operation, it displays a work equipment setting screen on the terminal display unit 53, which allows the parameters of work equipment 3 to be set, instead of the screen currently displayed on the terminal display unit 53 (current screen). Alternatively, if the display control unit 62 determines that work equipment operation is a candidate operation, it changes the shortcut button 72 displayed on the current screen to a work equipment setting button 72' that allows the parameters of work equipment 3 to be set, as shown in Figure 6, and displays it on the terminal display unit 53. Alternatively, if the display control unit 62 determines that work equipment operation is a candidate operation, it displays a simplified work equipment setting screen on the terminal display unit 53, which allows the parameters of work equipment 3 to be easily set when the rear view is visible, instead of the current screen.

[0058] Furthermore, if the display control unit 62 determines that a rearward confirmation operation is an operation candidate, it changes the shortcut button 72 displayed on the current screen to a work equipment setting button 72' that allows setting the parameters of the work equipment 3 and displays it on the terminal display unit 53.

[0059] Furthermore, if the display control unit 62 determines that a forward driving operation is a candidate operation, it will display a guidance screen for straight-line assist driving on the terminal display unit 53 instead of the current screen. Alternatively, if the display control unit 62 determines that a forward driving operation is a candidate operation, it will display a screen for setting the engine speed and vehicle speed on the terminal display unit 53 instead of the current screen.

[0060] Furthermore, if the display control unit 62 determines that a straight-line driving operation is a candidate operation, it will display the guidance screen or setting screen for the headland straight-line driving mode on the terminal display unit 53 instead of the current screen. Alternatively, if the display control unit 62 determines that a straight-line driving operation is a candidate operation, it will display the guidance screen for straight-line assist driving on the terminal display unit 53 instead of the current screen. Alternatively, if the display control unit 62 determines that a straight-line driving operation is a candidate operation, it will display the guidance screen or setting screen for the field registration mode on the terminal display unit 53 instead of the current screen.

[0061] Furthermore, if the display control unit 62 determines that a turning operation is a candidate operation, it will display the driving settings screen on the terminal display unit 53 instead of the current screen. Alternatively, if the display control unit 62 determines that a turning operation is a candidate operation, it will display the engine speed and vehicle speed settings screen on the terminal display unit 53 instead of the current screen. Alternatively, if the display control unit 62 determines that a turning operation is a candidate operation, it will change the shortcut button 72 displayed on the current screen to a turning start button to start automatic turning and display it on the terminal display unit 53. Alternatively, if the display control unit 62 determines that a turning operation is a candidate operation, it will display the automatic turning settings screen on the terminal display unit 53 instead of the current screen.

[0062] The display control unit 62 may use artificial intelligence (AI) to determine the display pattern of the terminal display unit 53 corresponding to the operation candidate of the work vehicle 1, and may use a generating AI to create the display pattern information and instructions resulting from the determination. For example, the display control unit 62 generates a machine learning model by machine learning the operation candidate information related to various operation candidates of the work vehicle 1 and the display pattern information related to various display patterns of the terminal display unit 53, and stores it in the storage unit 51 or the artificial intelligence server 7. Then, the display control unit 62 uses the machine learning model of the operation candidate information and display pattern information to analyze the operation candidate of the work vehicle 1 determined by the operation candidate determination unit 61, and determines a display pattern suitable for that operation candidate.

[0063] Furthermore, the display control unit 62 may determine the display pattern of the terminal display unit 53 by considering not only the candidate operations for the work vehicle 1, but also the operator's driving posture, the operating status of the work vehicle 1, the working conditions in the field, the operator's operating tendencies, etc. For example, the display control unit 62 may use a machine learning model that has been trained on the operator's driving posture, the operating status of the work vehicle 1, the working conditions in the field, the operator's operating tendencies, etc., in addition to the candidate operations for the work vehicle 1, to determine the display pattern of the terminal display unit 53 corresponding to the candidate operations for the work vehicle 1 using AI.

[0064] Here, if there are multiple display patterns for the terminal display unit 53 corresponding to the operation candidates of the work vehicle 1, the display control unit 62 may determine one display pattern based on the operator's driving posture, the operating status of the work vehicle 1, the working conditions of the field, the operator's operating tendencies, etc., in addition to the operation candidates of the work vehicle 1. Alternatively, if there are multiple display patterns for the terminal display unit 53 corresponding to the operation candidates of the work vehicle 1, the display control unit 62 may pre-set one display pattern determined in response to the operation candidate determination by the operation candidate determination unit 61, according to any operation.

[0065] The voice input control unit 63 controls voice input from the worker via the terminal sound collection unit 56 regarding the operation and work of the work vehicle 1. The voice input control unit 63 may also receive voice input via other sound collection devices, such as the vehicle sound collection unit 33 of the work vehicle 1, rather than just the terminal sound collection unit 56. In particular, when the operation candidate determination unit 61 determines an operation candidate based on the worker's driving posture, the voice input control unit 63 accepts voice input according to that operation candidate. Figure 3 illustrates the voice input control by the voice input control unit 63.

[0066] At this time, the voice input control unit 63 performs voice analysis of the voice input received via the terminal sound collection unit 56, converts the voice input into text, and generates text data.

[0067] Furthermore, the voice input control unit 63 may perform voice analysis of the voice input using artificial intelligence (AI), and may generate text data based on the voice input using a generation AI. For example, the voice input control unit 63 generates a machine learning model by machine learning voice data of commands such as "start straight-line assist driving," "start automatic turning driving," and other similar terms, and stores it in the memory unit 51 or the artificial intelligence server 7. Then, the voice input control unit 63 analyzes the input voice input using the machine learning model with AI and generates text data by converting the voice input into text.

[0068] Furthermore, the voice input control unit 63 analyzes the text data, which is a transcription of the voice input, to determine the content of the instructions included in the text data, such as the start of straight-line assist driving, the start of automatic turning driving, and other instructions.

[0069] Furthermore, the voice input control unit 63 may use artificial intelligence (AI) to analyze the text data, and the generation AI may determine the content of instructions such as starting straight-line assist driving, starting automatic turning driving, and other instructions based on the text data. For example, the voice input control unit 63 generates a machine learning model by machine learning the text data of instructions such as starting straight-line assist driving, starting automatic turning driving, and other similar terms, and stores it in the memory unit 51 or the artificial intelligence server 7. Then, the voice input control unit 63 uses the machine learning model to analyze the text data, which is the text generated from the voice input, and determines the content of instructions such as starting straight-line assist driving, starting automatic turning driving, and other instructions contained in the text data.

[0070] For example, if the voice input control unit 63 determines that the operation is to check behind the vehicle as a candidate operation, it controls the system to allow voice input for starting straight-line assist driving while reversing, or to allow other voice inputs. In response to the voice input for starting straight-line assist driving, the voice input control unit 63 transmits a signal to start straight-line assist driving to the work vehicle 1 via the communication unit 52.

[0071] Furthermore, if the voice input control unit 63 determines that a forward driving operation is a candidate operation, it controls the system to allow voice input for starting straight-line assist driving while driving forward. In response to the voice input for starting straight-line assist driving, the voice input control unit 63 transmits a signal to start straight-line assist driving to the work vehicle 1 via the communication unit 52.

[0072] Furthermore, if the voice input control unit 63 determines that a turning operation is a candidate operation, it controls the system to allow voice input for starting automatic turning while the vehicle is moving forward. In response to the voice input for starting automatic turning, the voice input control unit 63 transmits an automatic turning start signal to the work vehicle 1 via the communication unit 52.

[0073] The route creation unit 64 creates a travel route for the implement 3 to automatically travel while working in the field to be worked on, and transmits it to the work vehicle 1 via the communication unit 52. For example, the route creation unit 64 creates multiple straight-line routes for the field, according to the selected travel pattern (reciprocating mowing or circular mowing), and creates a travel route by combining multiple straight-line routes and multiple turning routes that connect each straight-line route.

[0074] As described above, according to this embodiment, the work vehicle 1 that travels and works in the field includes a display control unit 62 that displays various information on the terminal display unit 53 of the mobile terminal 5, a vehicle imaging unit 34 that captures images of workers, including the worker of the work vehicle 1, a driving posture determination unit 60 that determines the driving posture of the worker captured in the worker image based on a learning model that has been trained on training images of the worker, and an operation candidate determination unit 61 that determines an operation candidate for the work vehicle 1 corresponding to the driving posture determined by the driving posture determination unit 60, based on a learning model that has been trained on the worker's driving posture information and operation candidate information for the work vehicle 1. The display control unit 62 displays a screen on the terminal display unit 53 according to the operation candidate determined by the operation candidate determination unit 61.

[0075] In other words, the control method for controlling a work vehicle 1 that travels and works in a field in the present invention includes a display step of displaying various information on the terminal display unit 53 of a mobile terminal 5, an imaging step of capturing an image of a worker including the worker of the work vehicle 1, a driving posture determination step of determining the driving posture of the worker captured in the worker image based on a learning model that has been trained on training images of the worker, and an operation candidate determination step of determining an operation candidate for the work vehicle 1 corresponding to the driving posture determined in the driving posture determination step, based on a learning model that has been trained on the worker's driving posture information and operation candidate information for the work vehicle 1, and the display step displays a screen on the terminal display unit 53 according to the operation candidate determined in the operation candidate determination step.

[0076] As a result, according to the present invention, a screen can be displayed on the terminal display unit 53 that enables operations suitable for the operation and work of the work vehicle 1 (for example, operations and parameter settings related to operation and work) based on a learning model that has been machine-learned to capture the operator's driving posture. Therefore, operations and parameter settings related to operation and work can be easily performed without requiring the operator to do anything, thereby improving work efficiency.

[0077] Specifically, according to the work vehicle 1 of this embodiment, the display control unit 62 transitions from the current screen of the terminal display unit 53 to an operation screen corresponding to an operation candidate and displays it. Alternatively, the display control unit 62 modifies a part of the current screen of the terminal display unit 53 to correspond to an operation candidate and displays it. In this case, the display control unit 62 may change the buttons on the current screen of the terminal display unit 53 that transition to other screens to buttons that transition to an operation screen corresponding to an operation candidate and display them.

[0078] This allows for the automatic display of an operation screen suitable for the driving and operation of the work vehicle 1, based on the driver's posture, thereby improving work efficiency.

[0079] In the above-described embodiment, an example was explained in which a home screen 70, guidance screen, setting screen, etc., related to the driving and operation of the work vehicle 1 are displayed on the terminal display unit 53 of the mobile terminal 5. However, the present invention is not limited to this example. In other examples, the control device 4 of the work vehicle 1 may operate as a display control unit 62 and display a home screen 70, guidance screen, setting screen, etc., related to the driving and operation of the work vehicle 1 on the vehicle display unit 30 of the work vehicle 1. In this case, the control device 4 operating as a display control unit 62 displays a screen on the vehicle display unit 30 according to operation candidates based on the operator's driving posture.

[0080] In the above-described embodiment, the voice input control unit 63 was shown to accept voice inputs for starting straight-line assist driving while reversing, starting straight-line assist driving while moving forward, and starting automatic turning driving while moving forward, respectively, when it determined that the operation candidates were rearward confirmation operation, forward driving operation, and turning operation. However, the present invention is not limited to this example.

[0081] In a modified version of this embodiment, the voice input control unit 63 may be configured to switch to a voice input permission mode that accepts various voice inputs related to the driving and operation of the work vehicle 1 and the setting of various parameters when the driving posture determination unit 60 determines a specific driving posture.

[0082] For example, if the driving posture determination unit 60 determines that the driving posture (static posture, gaze, dynamic posture, etc.) is such as looking at the position of the vehicle sound collection unit 33 inside the cabin 12 or the voice input sign, or any other driving posture predetermined with the intention of granting voice input, then the voice input control unit 63 will switch to voice input permission mode. The voice input control unit 63 may cancel the voice input permission mode after a predetermined time has elapsed since the voice input posture was canceled.

[0083] Furthermore, the voice input control unit 63 may switch to voice input permission mode when the work vehicle 1 is changed from automatic driving mode to manual driving mode, and may also release the voice input permission mode after a predetermined time has elapsed since the change to manual driving mode.

[0084] The voice input control unit 63 may, when the voice input indicates the setting of a parameter, control the variables that can be changed according to whether the work vehicle 1 is in automatic driving mode or manual driving mode. Furthermore, the voice input control unit 63 may, when the voice input indicates the setting of a parameter, control the variables that can be changed according to the PTO rotation speed, vehicle speed, forward / reverse movement, and height of the work implement 3 of the work vehicle 1. The voice input control unit 63 may accept voice input for setting parameters that can be set on the current screen displayed on the terminal display unit 53, or it may accept voice input for setting parameters that cannot be set on the current screen without switching the current screen displayed on the terminal display unit 53.

[0085] When the voice input control unit 63 receives voice input indicating driving or work operations or the setting of various parameters, or when it executes driving or work operations or the setting of various parameters in accordance with the voice input, it is preferable to notify the operator of this fact by voice, message, lamp, etc.

[0086] If the voice input control unit 63 determines that the operation of driving or working, or the setting of various parameters indicated by the voice input, is impossible due to the operator's driving posture, the operating status of the work vehicle 1, or the working conditions in the field, it will notify the operator of this impossibility through voice, messages, lights, etc.

[0087] Figure 7 illustrates the voice input received when voice input is enabled.

[0088] The voice input control unit 63 may switch to voice input permission mode when it receives voice input of a predetermined startup command, in addition to, or instead of, determining the posture during voice input. The startup command may be set to, for example, "Hey, ○○○○", "OK, ○○○○", or "○○○○" when the proper noun of the system is "○○○○".

[0089] Alternatively, the voice input control unit 63 may be configured to allow switching the voice input permission mode on or off via the touch panel on the terminal display unit 53 or the physical buttons on the terminal operation unit 55.

[0090] At this time, the voice input control unit 63 performs voice analysis of the voice input received in voice input permission mode, converts the voice input into text, and generates text data.

[0091] Furthermore, the voice input control unit 63 may use artificial intelligence (AI) to analyze the voice input received when voice input is enabled, and may use a generating AI to generate text data based on the voice input. For example, the voice input control unit 63 generates a machine learning model by machine learning the voice input and similar terminology data, and stores it in the memory unit 51 or the artificial intelligence server 7. The voice input control unit 63 then uses the machine learning model to analyze the received voice input using AI and generates text data by converting the voice input into text.

[0092] Furthermore, the voice input control unit 63 determines the content of the voice input contained in the text data by analyzing the text data, which is a text representation of the voice input received when voice input is enabled.

[0093] Furthermore, the voice input control unit 63 may perform text data analysis using artificial intelligence (AI), and may determine the content of the voice input based on the text data using a generating AI. For example, the voice input control unit 63 generates a machine learning model by machine learning the text data of voice input and similar terms, and stores it in the memory unit 51 or the artificial intelligence server 7. Then, the voice input control unit 63 uses the machine learning model to analyze the text data, which is the text of the voice input received in voice input permission mode, and determines the content of the voice input contained in the text data.

[0094] In this case, the voice input control unit 63 may use a machine learning model that has been trained on the operator's driving posture, the operating status of the work vehicle 1, the working conditions in the field, the operator's operational tendencies, etc., in addition to the machine learning model for voice input, to determine the content of the voice input using AI.

[0095] For example, when the voice input control unit 63 receives a voice input indicating an intention to tilt to the right, such as "tilt to the right," if the work vehicle 1 is a tractor, it sends a control signal to the work vehicle 1 to tilt the implement 3 to the right relative to the vehicle direction. If the work vehicle 1 is a combine harvester, it sends a control signal to the work vehicle 1 to tilt the vehicle body 2 to the right relative to the vehicle direction. In addition, if the work vehicle 1 is a tractor, in addition to a voice input indicating an intention to tilt to the right, it may also send a control signal to tilt the implement 3 to the right based on conditions such as the driving posture being facing backward, looking in the mirror, or moving backward.

[0096] When the voice input control unit 63 receives voice input indicating that the vehicle is already tilted to the right, such as "it's tilted to the right," if the work vehicle 1 is a tractor, it sends a control signal to the work vehicle 1 to tilt the implement 3 to the left relative to the vehicle direction, and if the work vehicle 1 is a combine harvester, it sends a control signal to the work vehicle 1 to tilt the vehicle body 2 to the left relative to the vehicle direction.

[0097] Furthermore, when the voice input control unit 63 receives voice input such as "tilt to the right" and the driver's posture is facing backward, the operator's line of sight is reversed. Therefore, if the work vehicle 1 is a tractor, the control unit 63 sends a control signal to the work vehicle 1 to tilt the implement 3 to the left relative to the vehicle direction. If the work vehicle 1 is a combine harvester, the control unit 63 sends a control signal to the work vehicle 1 to tilt the vehicle body 2 to the left relative to the vehicle direction.

[0098] When the voice input control unit 63 receives a voice input indicating the intention to start straight-line assist driving, such as "automatic start," "automatic driving start," or "straight assist start," it turns on the automatic driving mode and sends a signal to start straight-line assist driving to the work vehicle 1. The voice input control unit 63 may also turn on the automatic driving mode on the condition that the driver's gaze is directed towards the straight-line assist driving guidance screen or start button. Furthermore, the voice input control unit 63 may turn on the automatic driving mode on the condition that the driver's posture is facing forward when straight-line assist driving is performed while moving forward, or on the condition that the driver's posture is facing backward when straight-line assist driving is performed while moving in reverse.

[0099] When the voice input control unit 63 receives voice input indicating an increase in the control sensitivity of the automatic driving system, such as "increase sensitivity" or "turning," or voice input indicating a decrease in the control sensitivity of the automatic driving system, while the work vehicle 1 is driving automatically, it increases the set value of the control sensitivity of the automatic driving system by a predetermined amount. Furthermore, considering the relationship between the steering angle of the front wheels 10 and the control sensitivity of the automatic driving system, if the steering angle is relatively large and increasing the control sensitivity would result in dangerous driving such as sudden steering, the voice input control unit 63 will notify the operator that the control sensitivity cannot be changed and will inform the operator of this fact by voice, message, lamp, etc.

[0100] When the voice input control unit 63 receives a voice input such as "increase sensitivity" while the work vehicle 1 is being driven manually, indicating an intention to increase the sensitivity of the horizontal control of the work implement 3, it increases the set value of the sensitivity of the horizontal control of the work implement 3 by a predetermined amount. The voice input control unit 63 may also change the sensitivity of the horizontal control of the work implement 3 on the condition that the driver's posture is facing backward or looking in the mirror and the work implement 3 is visible.

[0101] When the voice input control unit 63 receives voice input indicating an increase in the power of the engine 13, such as "Raise the engine," "The engine is weak," or "The engine is losing power," or voice input indicating a decrease in the power of the engine 13, it increases the set value of the rotational speed of the engine 13 by a predetermined amount.

[0102] When the voice input control unit 63 receives voice input indicating the target and degree of setting of a parameter, such as "set XX to 50%", it changes the setting value of the target function (for example, raising and lowering the work machine 3) according to the degree of setting.

[0103] When the voice input control unit 63 receives voice input indicating the setting target and the value to be changed regarding parameter settings, such as "set XX to 9", it changes the setting value of the function to be set (for example, driving sensitivity, etc.) to the changed value.

[0104] When the voice input control unit 63 receives voice input indicating a change in the method of acquiring location information, such as "Prioritize RTK accuracy," it sets the acquisition of location information in automatic driving mode to prioritize RTK accuracy. However, when the work vehicle 1 is driving automatically, the method of acquiring location information cannot be changed, so the voice input control unit 63 notifies the operator that the change is not possible through voice, message, lights, etc.

[0105] As described above, according to a modified version of this embodiment, the work vehicle 1 includes a voice input control unit 63 that receives voice input to change the setting value of a parameter that cannot be set on the current screen of the terminal display unit 53.

[0106] Specifically, according to the modified work vehicle 1, the voice input control unit 63 changes the parameter settings based on the voice input. At this time, the work vehicle 1 controls its operation according to the parameter settings changed based on the voice input.

[0107] As a result, according to the present invention, the operation of the work vehicle 1, the operation of the work, and the setting of various parameters can be performed without changing the screen displayed on the terminal display unit 53. Therefore, operations related to driving and work, and the setting of parameters can be easily performed without requiring extra effort from the operator, thereby improving work efficiency.

[0108] In the embodiments described above, examples were given in which various machine learning models are stored in the memory unit 51 of the mobile terminal 5 or in the artificial intelligence server 7, but the present invention is not limited to these examples. In other examples, various machine learning models may be stored in the memory unit 41 of the work vehicle 1.

[0109] Furthermore, in the above-described embodiment, an example was explained in which the driving posture determination unit 60, operation candidate determination unit 61, display control unit 62, voice input control unit 63, and route creation unit 64 operate on the computer of the control device 50 of the mobile terminal 5, but the present invention is not limited to this example. In other examples, the driving posture determination unit 60, operation candidate determination unit 61, display control unit 62, voice input control unit 63, and route creation unit 64 may be configured to operate on the computer of the control device 4 of the work vehicle 1 or the computer of the artificial intelligence server 7. In this case, the driving posture determination unit 60, operation candidate determination unit 61, display control unit 62, voice input control unit 63, and route creation unit 64 may operate as artificial intelligence (AI) or generative AI on the computer of the control device 4 of the work vehicle 1 or the computer of the artificial intelligence server 7.

[0110] In other words, according to this embodiment, the control system of the work vehicle 1 only needs to include an imaging control unit 45, a driving control unit 46, a driving posture determination unit 60, an operation candidate determination unit 61, a display control unit 62, a voice input control unit 63, and a route creation unit 64. Here, the driving control unit 46, the driving posture determination unit 60, the operation candidate determination unit 61, the display control unit 62, the voice input control unit 63, and the route creation unit 64 may be executed by the control device 50 of the mobile terminal 5, or they may be executed by the computer of the control device 4 of the work vehicle 1 or the computer of the artificial intelligence server 7.

[0111] In the embodiments described above, an example was given in which the work vehicle 1 is composed of a tractor, but the present invention is not limited to this example. For example, the work vehicle of the present invention may be composed of other agricultural machinery such as a combine harvester, rice transplanter, or lawnmower, or it may be composed of other work vehicles other than agricultural machinery.

[0112] Furthermore, the present invention may be modified as appropriate, provided that it does not contradict the gist or idea of ​​the invention as can be inferred from the claims and the specification as a whole, and a control method for a work vehicle involving such modifications is also included in the technical concept of the present invention.

[0113] [Notes on the invention] The following is an overview of the invention extracted from the above-described embodiments. Note that each configuration and processing function described below can be selected and combined as desired.

[0114] <Note 1> A control method for controlling work vehicles that travel and perform tasks in a field, A display process in which various information is displayed on the display unit, The imaging step involves capturing images of workers, including the workers on the aforementioned work vehicle. A driving posture determination step, which determines the driving posture of the worker captured in the worker image based on a learning model that has been trained on training images of the worker, The system includes an operation candidate determination step that determines an operation candidate for the work vehicle corresponding to the driving posture determined in the driving posture determination step, based on a machine learning model that has acquired information on the operator's driving posture and information on candidate operations for the work vehicle. The control method is characterized in that the display step displays a screen on the display unit according to the operation candidate determined in the operation candidate determination step.

[0115] <Note 2> The control method according to Appendix 1, characterized in that the display step transitions from the current screen of the display unit to an operation screen corresponding to the operation candidate and displays the result.

[0116] <Note 3> The control method according to Appendix 1 or 2, characterized in that the display step involves changing and displaying a part of the current screen of the display unit to correspond to the operation candidate.

[0117] <Note 4> The control method according to Appendix 3, characterized in that the display step is to change the button for transitioning to another screen on the current screen of the display unit to a button for transitioning to an operation screen corresponding to the operation candidate and display it.

[0118] <Note 5> The control method according to any one of the appendices 1 to 4, further comprising a voice input step for receiving voice input to change the setting value of a parameter that cannot be set on the current screen of the display unit.

[0119] <Note 6> The control method according to Appendix 5, characterized in that the voice input step changes the set value of the parameter based on the voice input.

[0120] <Note 7> The control method according to Appendix 6, characterized in that it controls the operation of the work vehicle according to the setting value of the parameter that has been changed based on the voice input. [Explanation of symbols]

[0121] 1. Work vehicles 2 car bodies 3. Work equipment 4. Control device 5 Mobile devices 6. Management Server 7. Artificial Intelligence Server 13 Engine 14. Transmission 15. Lifting mechanism for work equipment 16 Positioning Units 30 Vehicle display section 31 Vehicle audio output unit 32 Vehicle Control Unit 33 Vehicle sound collection unit 34 Vehicle body imaging unit 41 Storage section 45 Imaging Control Unit 46 Operation Control Unit 50 Control device 51 Storage section 53 Terminal display unit (display unit) 54 Terminal audio output section 55 Terminal operation section 56 Terminal sound collection unit 57 Terminal imaging unit 60 Driving posture determination unit 61 Operation candidate determination section 62 Display Control Unit 63. Voice Input Control Unit 64 Route Creation Unit 72 Shortcut buttons (buttons) 72' Implement setting button (button)

Claims

1. A control method for controlling work vehicles that travel and perform tasks in a field, A display process in which various information is displayed on the display unit, The imaging step involves capturing images of workers, including the workers on the aforementioned work vehicle. A driving posture determination step, which determines the driving posture of the worker captured in the worker image based on a learning model that has been trained on training images of the worker, The system includes an operation candidate determination step that determines an operation candidate for the work vehicle corresponding to the driving posture determined in the driving posture determination step, based on a machine learning model that has acquired information on the operator's driving posture and information on candidate operations for the work vehicle. The control method is characterized in that the display step displays a screen on the display unit according to the operation candidate determined in the operation candidate determination step.

2. The control method according to claim 1, characterized in that the display step transitions from the current screen of the display unit to an operation screen corresponding to the operation candidate and displays the result.

3. The control method according to claim 1, characterized in that the display step involves changing and displaying a part of the current screen of the display unit to correspond to the operation candidate.

4. The control method according to claim 3, characterized in that the display step is to change the button for transitioning to another screen on the current screen of the display unit to a button for transitioning to an operation screen corresponding to the operation candidate and display it.

5. The control method according to claim 1, further comprising a voice input step for receiving voice input to change the setting value of a parameter that cannot be set on the current screen of the display unit.

6. The control method according to claim 5, characterized in that the voice input step changes the set value of the parameter based on the voice input.

7. The control method according to claim 6, characterized in that the operation of the work vehicle is controlled according to the setting value of the parameter which has been changed based on the voice input.

Images