Control method

Abstract

The present invention provides a control method for a work vehicle that allows for the appropriate handling of the work vehicle and the work equipment, according to the type and / or model of the attached work equipment, without requiring extra effort from the operator. [Solution] The control system 1 for the work vehicle 2 and / or work machine 11 includes a learning unit 50 in the management server 4 that stores a machine learning model created by machine learning the learning images of the work machine 11 and the learning parameters of the work vehicle 2 and / or work machine 11; an imaging control unit 23 in the work vehicle 2 that captures images of the work machine 11 using a vehicle imaging unit 18 and a terminal imaging unit 37; a determination unit 51 in the management server 4 that determines the work machine 11 captured in the work machine image based on the machine learning model; and a proposal unit 52 and a parameter setting unit 25 in the management server 4 and the work vehicle 2, respectively, that determine and propose parameters to be set for the work vehicle 2 and / or work machine 11 according to the work machine 11 determined by the determination unit 51, based on the machine learning model.

Description

Technical Field

[0006] , ,

[0001] The present invention relates to a method for controlling a work vehicle and / or a work implement.

Background Art

[0002] A work vehicle such as an agricultural work machine mounts a detachable work implement and performs work on a field by the work implement while traveling on the field.

[0003] For example, in a work implement management system including a work vehicle to which a work implement is mounted and a management server for managing the work implement in Patent Document 1, the work vehicle has a photographing unit that photographs the work implement mounted on the work vehicle and acquires a photographed image, and a photographed image transmission unit that transmits the photographed image to the management server, and the management server has a work implement discrimination unit that analyzes the photographed image and discriminates at least the type of the work implement mounted on the work vehicle.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In a work vehicle, appropriate parameters of the work vehicle and / or the work implement (for example, working width, vehicle speed, PTO rotation speed, PTO rotation direction, etc.) may differ depending on the type and / or model of the work implement. Therefore, the operator needs to manually derive appropriate parameters of the work vehicle and / or the work implement while driving the work vehicle equipped with the work implement. Alternatively, each time the operator mounts a work implement on the work vehicle, the operator needs to manually reset appropriate parameters of the work vehicle and / or the work implement. Therefore, it takes time and effort for the operator to set appropriate parameters of the work vehicle and / or the work implement.

[0006] Furthermore, in the case of work vehicles, the means of detecting and responding to abnormalities in the work vehicle or work equipment may differ depending on the type and / or model of the work equipment. Therefore, each time a work equipment is attached to the work vehicle, the operator must visually and audibly judge the abnormality, which differs for each type and / or model of work equipment, manually reset the detection means for that abnormality, or look up the means of responding to the abnormality, which differs for each type and / or model of work equipment, in the instruction manual or other documents. Consequently, detecting and responding to abnormalities in the work vehicle or work equipment requires considerable effort on the operator's part.

[0007] Thus, with work vehicles equipped with detachable implements, operators need to perform different handling procedures for each type and / or model of implement, which adds extra work for the operator.

[0008] The present invention aims to provide a control method for a work vehicle that allows for the appropriate handling of the work vehicle and work equipment according to the type and / or model of the attached work equipment, without requiring extra effort from the operator. [Means for solving the problem]

[0009] To solve the above problems, the present invention provides a control method for a work vehicle and / or work machine, comprising: a learning step of accumulating a learning model obtained by machine learning using a learning image of the work machine and learning parameters of the work vehicle and / or the work machine; an imaging step of capturing an image of the work machine; a determination step of determining the work machine captured in the image based on the learning model; and a proposal step of determining and proposing parameters to be set on the work vehicle and / or the work machine according to the work machine determined in the determination step, based on the learning model.

[0010] Furthermore, in order to solve the above problems, the control method of the present invention is a control method for a work vehicle and / or work machine, characterized by comprising: a learning step of accumulating a learning model obtained by machine learning from learning images of the work machine in normal and abnormal conditions; an imaging step of capturing an image of the work machine; and a detection step of detecting an abnormality of the work machine captured in the image based on the learning model. [Effects of the Invention]

[0011] The present invention provides a control method for a work vehicle that allows for appropriate handling of the work vehicle and work equipment according to the type and / or model of the attached work equipment, without requiring extra effort from the operator. [Brief explanation of the drawing]

[0012] [Figure 1] This is a block diagram showing a control system according to an embodiment of the present invention. [Figure 2] This is a side view showing an example of a work vehicle according to an embodiment of the present invention. [Figure 3] This is a front view showing an example of a parameter suggestion screen in a control system according to an embodiment of the present invention. [Figure 4] This is a front view showing an example of an individual parameter setting screen in a control system according to an embodiment of the present invention. [Figure 5] Block diagram showing a control system according to a modified example of the present invention. [Figure 6] This is a front view showing an example of an abnormality notification screen in a control system according to a modified example of the present invention. [Modes for carrying out the invention]

[0013] An embodiment of the present invention, a control system 1, will be described with reference to the drawings. As shown in Figure 1, the control system 1 includes a work vehicle 2, which is an agricultural machine such as a tractor, rice transplanter, or combine harvester handled by an operator, a portable terminal 3 owned by the operator, and a management server 4 that can communicate with the work vehicle 2 and the portable terminal 3 via a network such as the Internet.

[0014] In this embodiment, an example is described in which the work vehicle 2 is a tractor. Furthermore, in this embodiment, an example is described in which the control system 1 has one work vehicle 2 and one portable terminal 3, but the present invention is not limited to this example, and the control system 1 may have two or more work vehicles 2, or two or more portable terminals 3.

[0015] As shown in Figure 2, the work vehicle 2 operates in a work area such as a field, driving the vehicle body 10 while performing work using implements 11 attached to the vehicle body 10. For example, it can perform agricultural work such as tilling, puddling, and sowing on the same field. The implements 11 are detachably attached to the vehicle body 10. The work vehicle 2 can attach one of the various implements 11 to the implement 11, and the control system 1 sets different parameters for the work vehicle 2 and the implements 11 for each type and / or model of the implement 11.

[0016] The work vehicle 2 is equipped with a power source 12 including an engine (not shown) and a transmission (not shown) that generates power, and a PTO shaft 13 that transmits power from the power source 12. The work implement 11 is connected to the PTO shaft 13 and driven by the power transmitted from the power source 12 of the work vehicle 2. The work vehicle 2 is capable of adjusting the PTO rotation speed and PTO rotation direction of the PTO shaft 13.

[0017] The work vehicle 2 includes a vehicle control device 20 composed of a computer such as a CPU. The vehicle control device 20 is connected to a vehicle storage unit 21 such as a ROM, a RAM, a hard disk drive, and a flash memory, and a vehicle communication unit 22 that communicates with external devices such as a mobile terminal 3 and a management server 4. For example, while performing farming operations in a farm field, the work vehicle 2 measures field information regarding the farm field (such as the shape, size, and position information of the field edges constituting the outer periphery of the farm field), and transmits the measured field information to the mobile terminal 3.

[0018] The vehicle storage unit 21 stores programs and data for controlling various components and functions of the work vehicle 2. The vehicle control device 20 executes arithmetic processing based on the programs and data stored in the vehicle storage unit 21 to control various components and functions of the work vehicle 2. For example, the vehicle control device 20 controls a positioning unit (not shown) to obtain the position of the work vehicle 2 itself. An operation example of the vehicle control device 20 will be described later.

[0019] The vehicle storage unit 21 stores vehicle information regarding the work vehicle 2. The vehicle information includes, for example, information such as the model number, model name, identification number, size (dimensions), traveling method (wheel type having a pair of left and right front wheels and rear wheels, half crawler type in which a pair of left and right rear wheels are replaced with crawler devices, etc.), traveling width, vehicle speed and engine speed during non-operation, and vehicle speed and engine speed during operation of the work vehicle 2. Further, the vehicle information may include information such as the model number, model name, identification number, size (dimensions), and working width of the work implement 11 attached to the work vehicle 2.

[0020] The vehicle storage unit 21 stores parameters for controlling the work vehicle 2 and the work implement 11. The work vehicle 2 performs different operations for each type and / or model of the work implement 11. The parameters stored in the vehicle storage unit 21 include parameters commonly set for the work vehicle 2 and the work implement 11 regardless of the type and model of the work implement 11, and parameters set for the work vehicle 2 and the work implement 11 corresponding to the work implement 11 mounted on the work vehicle 2. The parameters set for the work vehicle 2 and the work implement 11 according to the work implement 11 include, for example, the vehicle speed of the work vehicle 2, the engine speed, the on / off of the lift function during turning travel, the on / off of the lift function during reverse travel, the on / off of the automatic deceleration function during overload, etc. Also, it includes the depth of the work implement 11, the PTO rotation speed, the PTO rotation direction, the uppermost / lowermost position of the work implement 11, etc. Further, it includes the working width, the wrap width, the control sensitivity indicating how much steering should be cut to return to the target path when deviating from the target path, the turning assist setting that enables automatic turning without the operator operating the steering wheel, etc.

[0021] In addition, the vehicle storage unit 21 stores the field information of the field that is the work target of the work vehicle 2. The field information includes, for example, the shape, size, and position information (coordinates, etc.) of the field edge that constitutes the field perimeter, and the shape, size, and position information (coordinates, etc.) of the working area of the field. Also, the field information includes information such as the shape, size, and position information (coordinates, etc.) of the unworked land where work has not been done yet and the worked land where work has already been completed.

[0022] Note that the vehicle control device 20 of the work vehicle 2 may store the vehicle information and field information received from the mobile terminal 3 or the management server 4 in the vehicle storage unit 21, or may store the vehicle information and field information input according to an arbitrary operation of the operator in the vehicle storage unit 21.

[0023] The vehicle communication unit 22 can communicate wirelessly with external devices such as the mobile terminal 3 and the management server 4 via a wireless communication antenna. The vehicle control device 20 controls the vehicle communication unit 22 to communicate wirelessly with the mobile terminal 3 and the management server 4, and sends and receives various information between the vehicle and the mobile terminal 3 and the management server 4. The work vehicle 2 is connected to the management server 4 via a network and is also connected to the mobile terminal 3 via a network or short-range wireless communication.

[0024] Furthermore, the work vehicle 2 is equipped with a vehicle display unit 14 such as an LCD display, a vehicle audio output unit 15 such as a speaker, a vehicle operation unit 16 such as a keyboard and various buttons, and a vehicle sound collection unit 17 such as a microphone, around the driver's seat. The vehicle display unit 14 is controlled by the vehicle control device 20 to display various information to the worker. The vehicle audio output unit 15 is controlled by the vehicle control device 20 to output various information to the worker as audio. The vehicle operation unit 16 is controlled by the vehicle control device 20 to receive information input from the worker via manual operation. The vehicle sound collection unit 17 is controlled by the vehicle control device 20 to receive information input via voice input from the worker. Note that the vehicle display unit 14 and the vehicle operation unit 16 may be configured as an integrated touch panel.

[0025] The work vehicle 2 has a vehicle imaging unit 18 for capturing images of the status of the work vehicle 2 and the work machine 11, as well as the working conditions in the field. The vehicle imaging unit 18 is controlled by the vehicle control device 20 to capture desired images and input them to the vehicle control device 20.

[0026] Furthermore, the work vehicle 2 may be configured as an automated driving vehicle that automatically drives along a pre-set driving route. In this case, the work vehicle 2 receives field data and driving route from the mobile terminal 3 and is controlled to automatically drive according to this field data and driving route. The work vehicle 2 also has driving data related to driving conditions and work data related to working conditions, and controls automatic driving based on this data.

[0027] An example of the operation of the vehicle control device 20 will be described. The vehicle control device 20 operates as an imaging control unit 23, a driving control unit 24, and a parameter setting unit 25 by executing a program stored in the vehicle memory unit 21. The imaging control unit 23, the driving control unit 24, and the parameter setting unit 25 realize the imaging process, the driving control process, and the parameter setting process of the control method for the work vehicle 2 according to the present invention.

[0028] The imaging control unit 23 controls the vehicle imaging unit 18 at a predetermined imaging timing to capture an image of the work equipment including the work equipment 11, inputs the work equipment image from the vehicle imaging unit 18, and transmits it to the management server 4 via the vehicle communication unit 22. For example, the imaging control unit 23 may use the start of the engine of the power source 12 as the imaging timing to have the vehicle imaging unit 18 capture an image of the work equipment. Alternatively, the imaging control unit 23 may have the vehicle imaging unit 18 capture an image of the work equipment at an imaging timing corresponding to an operation of the operator. If multiple vehicle imaging units 18 are provided, the imaging control unit 23 may have each of the multiple vehicle imaging units 18 capture an image of the work equipment.

[0029] The driving control unit 24 controls the vehicle speed and steering of the work vehicle 2 by using parameters for controlling the work vehicle 2 and work implement 11 stored in the vehicle memory unit 21 to control the engine, transmission, and steering device (not shown) of the power source 12, and also controls the operation of the work implement 11 by controlling the PTO shaft 13, thereby controlling the driving operation of the work vehicle 2.

[0030] For example, when the operation control unit 24 performs automatic driving of the work vehicle 2, it controls the automatic driving of the work vehicle 2 based on the driving path set for the field. When automatic driving starts, the operation control unit 24 obtains the position of the work vehicle 2 from the positioning unit and controls each part so that the work vehicle 2 performs automatic driving work along the driving path based on the position of the work vehicle 2, field information and the driving path.

[0031] The parameter setting unit 25 receives parameter suggestions for the work vehicle 2 and work equipment 11, which differ for each type and / or model of the work equipment 11, from the management server 4. For example, as shown in Figure 3, the parameter setting unit 25 displays a parameter suggestion screen 70 on the vehicle display unit 14, which displays the parameters for the work vehicle 2 and work equipment 11 suggested by the management server 4, thereby suggesting the parameters for the work vehicle 2 and work equipment 11 to the worker. Alternatively, the parameter setting unit 25 suggests the parameters for the work vehicle 2 and work equipment 11 to the worker by outputting the parameters for the work vehicle 2 and work equipment 11 suggested by the management server 4 via the vehicle voice output unit 15.

[0032] The parameter setting unit 25 may also propose parameters for the work vehicle 2 and work machine 11 to the worker by displaying a selection screen on the vehicle display unit 14 that allows the worker to choose whether or not to set the parameters for the work vehicle 2 and work machine 11 proposed by the management server 4. For example, the parameter setting unit 25 may use the parameter proposal screen 70 as a selection screen to accept the choice of whether or not to set the parameters. On the selection screen, the parameter setting unit 25 displays notifications such as "A change in work machine has been detected. Do you want to change to the recommended settings?" or "A change in work machine has been detected. Do you want to change to the recommended settings?". The parameter proposal screen 70, which is a selection screen, displays three buttons for selecting whether or not to change all the parameters proposed by the management server 4: an "OK All" button to change all of them, an "NG All" button to not change any of them, and an "Individual Settings" button to change them individually. If the "Individual Settings" button is selected, the parameter setting unit 25 displays an individual settings screen 71 that accepts individual parameter changes, as shown in Figure 4. If the operator selects to set parameters on the selection screen, the parameter setting unit 25 may set the parameters for the work vehicle 2 and work machine 11 proposed by the management server 4 and store them in the vehicle storage unit 21.

[0033] Alternatively, the parameter setting unit 25 may automatically set the parameters of the work vehicle 2 and work equipment 11 proposed by the management server 4 and store them in the vehicle storage unit 21 without displaying such a selection screen on the vehicle display unit 14.

[0034] In this embodiment, an example was described in which the parameter setting unit 25 of the vehicle control device 20 proposes different parameters for the work vehicle 2 and the work equipment 11 to the worker for each type and / or model of the work equipment 11. However, the present invention is not limited to this example, and a mobile terminal 3 may make similar proposals.

[0035] The mobile terminal 3 is one of the components of the work vehicle 2 and is a terminal that can remotely control the work vehicle 2. For example, it can be a smartphone, a tablet, a notebook computer, or the like. A similar operating device to the mobile terminal 3 may also be provided in the driver's seat of the work vehicle 2.

[0036] The mobile terminal 3 is equipped with a terminal control unit 30 which consists of a computer such as a CPU. The terminal control unit 30 is connected to a terminal storage unit 31 which includes ROM, RAM, a hard disk drive, and flash memory, and to a terminal communication unit 32 which communicates with external devices such as a work vehicle 2 and a management server 4.

[0037] The mobile terminal 3 also includes a terminal display unit 33 such as an LCD display, a terminal audio output unit 34 such as a speaker, a terminal operation unit 35 such as a keyboard and various buttons, and a terminal sound collection unit 36 ​​such as a microphone. The terminal display unit 33 is controlled by the terminal control device 30 to display various information to the worker. The terminal audio output unit 34 is controlled by the terminal control device 30 to output various information to the worker as audio. The terminal operation unit 35 is controlled by the terminal control device 30 to accept information input by manual operation by the worker. The terminal sound collection unit 36 ​​is controlled by the terminal control device 30 to accept information input by voice input from the worker. The terminal display unit 33 and the terminal operation unit 35 may be configured as an integrated touch panel.

[0038] The mobile terminal 3 has a terminal imaging unit 37 for capturing images of the status of the work vehicle 2 and work equipment 11, as well as the work conditions in the field. The terminal imaging unit 37 is controlled by the terminal control device 30 to capture the desired image and input it to the terminal control device 30. For example, the mobile terminal 3 transmits the work equipment image captured by the terminal imaging unit 37 to the management server 4 via the terminal communication unit 32.

[0039] The terminal storage unit 31 stores programs and data for controlling various components and functions of the mobile terminal 3, and the terminal control device 30 controls the various components and functions of the mobile terminal 3 by performing calculations based on the programs and data stored in the terminal storage unit 31.

[0040] The terminal communication unit 32 can communicate wirelessly with external devices such as the work vehicle 2 and the management server 4 via a wireless communication antenna. The terminal control device 30 controls the terminal communication unit 32 to communicate wirelessly with the work vehicle 2 and the management server 4, and sends and receives various information between the two. The mobile terminal 3 is connected to the management server 4 via a network and is also connected to the work vehicle 2 via a network or short-range wireless communication.

[0041] The management server 4 is an external server for managing the work vehicles 2 and work machines 11. It manages vehicle information for the work vehicles 2 and field information for each field, and is wirelessly connected to the work vehicles 2 and mobile terminals 3 via a network such as the Internet.

[0042] The management server 4 is equipped with a server control unit 40, which consists of a computer such as a CPU. The server control unit 40 is connected to a server storage unit 41, which includes ROM, RAM, hard disk drives, and flash memory, and a server communication unit 42, which communicates with external devices such as work vehicles 2 and mobile terminals 3. The server control unit 40 is also connected to a database 43 for storing machine learning models used by artificial intelligence (AI).

[0043] The server storage unit 41 stores programs and data for controlling various components and functions of the management server 4, and the server control device 40 controls the various components and functions of the management server 4 by executing calculations based on the programs and data stored in the server storage unit 41.

[0044] For example, the server storage unit 41 stores field information for each field, vehicle information for each work vehicle 2, and electronic data of instruction manuals for each work vehicle 2 and / or each work machine 11. The server control device 40 of the management server 4 receives field information and vehicle information from the connected work vehicles 2 and mobile terminals 3 and stores them in the server storage unit 41.

[0045] The server communication unit 42 can communicate wirelessly with external devices such as the work vehicle 2 and the mobile terminal 3 via a wireless communication antenna. The server control device 40 controls the server communication unit 42 to communicate wirelessly with the work vehicle 2 and the mobile terminal 3, and sends and receives various information between the two. The management server 4 is connected to the work vehicle 2 and the mobile terminal 3 via a network so that it can communicate with them.

[0046] Furthermore, the server control device 40 operates as a learning unit 50, a determination unit 51, and a proposal unit 52 by executing a program stored in the server storage unit 41, and the management server 4 operates as an artificial intelligence server. The learning unit 50, the determination unit 51, and the proposal unit 52 realize the learning process, the determination process, and the proposal process of the control method for the work vehicle 2 according to the present invention.

[0047] The learning unit 50 stores in the database 43 a machine learning model (learning model) that has been machine-learned by taking various learning images of each type and model of work equipment 11 and various learning parameters set on the work vehicle 2 and / or work equipment 11 when each type and model of work equipment 11 is attached. The learning unit 50 may input images taken by the vehicle imaging unit 18 or the terminal imaging unit 37 as learning images of the work equipment 11, or it may input images that have been made public on the internet by a website or the like. The learning unit 50 may input parameters that have been set on the work vehicle 2 and / or work equipment 11 in the past as learning parameters, or it may generate parameters that are appropriately set on the work vehicle 2 and / or work equipment 11 according to the type and model of work equipment 11 using a generation AI and input them as learning parameters.

[0048] The work vehicle 2 performs different tasks depending on the type and / or model of the work implement 11. The parameters set for the work vehicle 2 and the work implement 11 include parameters that are common regardless of the type and model of the work implement 11, and parameters that differ depending on the type and model of the work implement 11. The learning parameters that the learning unit 50 uses machine learning are parameters that differ depending on the type and model of the work implement 11, and include, for example, the vehicle speed of the work vehicle 2, engine speed, on / off of the lifting function during turning, on / off of the lifting function during reverse driving, on / off of the automatic deceleration function in case of overload, etc., as well as the depth of the work implement 11, PTO rotation speed, PTO rotation direction, highest / lowest position of the work implement 11, etc., as well as the working width, overlap width, control sensitivity, turning assist setting, etc. Specifically, if the implement 11 is a sprayer or cultivator, the vehicle speed should be set to a relatively fast speed (for example, 5 km / h), and if the implement 11 is a rotary tiller or the like, the vehicle speed should be set to a relatively slow speed (for example, 1.5 km / h).

[0049] For example, the learning unit 50 analyzes the training images of the work equipment 11 and determines the type and model of the work equipment 11 captured in the training images. At this time, the learning unit 50 may add information to the training images that is determined to identify the type and model of the work equipment 11 from among the features that indicate the work equipment 11 captured in the training images. Furthermore, the learning unit 50 generates a machine learning model by linking the training images of each type and model of work equipment 11 with the learning parameters that are appropriately set for the work vehicle 2 and / or work equipment 11 when the work equipment 11 is attached, and stores it in the database 43.

[0050] The learning unit 50 may use artificial intelligence (AI) to analyze the training images of the work implement 11 and to determine the characteristics of the work implement 11 in order to identify its type and model. For example, the learning unit 50 may use machine learning to analyze the characteristics of the work implement 11 captured in the training images, thereby generating a machine learning model of the characteristics of the work implement 11 in the training images and storing it in the database 43.

[0051] The determination unit 51 determines the type and model of the work equipment 11 mounted on the work vehicle 2 based on images of the work equipment captured by the vehicle imaging unit 18 and the terminal imaging unit 37. For example, the determination unit 51 determines the type and model of the work equipment 11 captured in the work equipment image by artificial intelligence (AI) based on a machine learning model of the work equipment 11's training images, thereby determining the work equipment 11 mounted on the work vehicle 2. At this time, the learning unit 50 may use a machine learning model of the features of the work equipment 11 to perform analysis of the work equipment 11 captured in the training image and determine the features for identifying the type and model by artificial intelligence (AI). The learning unit 50 may also use machine learning to learn the work equipment images used by the determination unit 51 and the determination results based on those work equipment images to generate a machine learning model of the training images of the work equipment 11.

[0052] The proposal unit 52 determines appropriate parameters to be set for the work vehicle 2 and / or the work machine 11 according to the type and model of the work machine 11 attached to the work vehicle 2, and proposes the determined parameters to the work vehicle 2 and / or the mobile terminal 3 via the server communication unit 42. For example, the proposal unit 52 uses artificial intelligence (AI) to determine the parameters to be appropriately set for the work vehicle 2 and / or the work machine 11 according to the type and model of the work machine 11 determined by the determination unit 51, based on a machine learning model that has been trained on a learning image of the work machine 11 and the learning parameters to be set for the work vehicle 2 and / or the work machine 11, with respect to the work machine 11 captured in the work machine image. At this time, the proposal unit 52 may use a generation AI to generate the parameters to be appropriately set for the work vehicle 2 and / or the work machine 11 according to the type and model of the work machine 11. Furthermore, the proposed unit 52 may generate parameters using a generative AI, taking into account other factors such as field conditions, in addition to a machine learning model that has been trained on the learning images of the implement 11 and the learning parameters set for the work vehicle 2 and / or implement 11.

[0053] As described above, according to this embodiment, the control system 1 for the work vehicle 2 and / or work machine 11 has a learning unit 50 in the management server 4 that stores a machine learning model (learning model) that has been machine-learned using learning images of the work machine 11 and learning parameters of the work vehicle 2 and / or work machine 11; an imaging control unit 23 in the work vehicle 2 that captures work machine images of the work machine 11 using a vehicle imaging unit 18 and a terminal imaging unit 37; a determination unit 51 in the management server 4 that determines the work machine 11 captured in the work machine image based on the machine learning model; and a proposal unit 52 and a parameter setting unit 25 in the management server 4 and the work vehicle 2, respectively, that determine and propose parameters to be set for the work vehicle 2 and / or work machine 11 according to the work machine 11 determined by the determination unit 51 based on the machine learning model.

[0054] In other words, in the present invention, the control method for the work vehicle 2 and / or work machine 11 includes a learning step of accumulating a machine learning model (learning model) that has been machine-learned using learning images of the work machine 11 and learning parameters of the work vehicle 2 and / or work machine 11; an imaging step of capturing images of the work machine 11; a determination step of determining the work machine 11 captured in the image based on the machine learning model; and a proposal step of determining and proposing parameters to be set for the work vehicle 2 and / or work machine 11 according to the work machine 11 determined in the determination step, based on the machine learning model.

[0055] As a result, according to the present invention, even if the parameters set for the work vehicle 2 and the work equipment 11 differ depending on the type and model of the work equipment 11, the type and model of the work equipment 11 can be automatically determined by using artificial intelligence (AI) based on a machine learning model that has learned the learning images of the work equipment 11 and the learning parameters set for the work vehicle 2 and / or work equipment 11. The appropriate parameters to be set for the work vehicle 2 and work equipment 11 can then be automatically determined according to the work equipment 11. Therefore, appropriate parameters can be set for the work vehicle 2 and work equipment 11 according to the type and / or model of the work equipment 11 installed, without requiring any effort from the operator, and the work vehicle 2 and work equipment 11 can be handled appropriately.

[0056] Furthermore, according to this embodiment, the proposal unit 52 and the parameter setting unit 25 propose parameters by displaying them on the parameter proposal screen 70.

[0057] This allows the appropriate parameters for the work vehicle 2 and work machine 11 to be concretely displayed on the screen and visually output.

[0058] Alternatively, according to this embodiment, the suggestion unit 52 and the parameter setting unit 25 suggest parameters by outputting them as audio.

[0059] This allows the appropriate parameters for the work vehicle 2 and work machine 11 to be output via voice, without requiring the operator to visually confirm them or take other steps.

[0060] Furthermore, according to this embodiment, the suggestion unit 52 and the parameter setting unit 25 display a selection screen, such as the parameter suggestion screen 70, which allows the user to choose whether or not to set the suggested parameters.

[0061] This allows the operator to choose whether or not to set appropriate parameters for the work vehicle 2 and the work machine 11, according to their own preference.

[0062] Alternatively, according to this embodiment, the suggestion unit 52 and the parameter setting unit 25 automatically set the suggested parameters.

[0063] This allows for the immediate setting of appropriate parameters for the work vehicle 2 and work machine 11 without requiring any extra effort from the operator.

[0064] Next, a modified control system 1 of the above-described embodiment will be explained. In the modified embodiment as well, the work vehicle 2 is equipped with one of the various work implements 11, while in the modified embodiment in particular, the control system 1 provides different detection and response means for each type and / or model of work implement 11 in response to abnormalities occurring in the work implement 11. In the modified embodiment, the same components as in the above-described embodiment will not be explained.

[0065] The work vehicle 2 performs different tasks depending on the type and / or model of the work equipment 11, and the abnormalities that occur in the work equipment 11 (e.g., abnormal location or abnormal condition) differ depending on the type and / or model of the work equipment 11. Therefore, in the modified example, the vehicle memory unit 21 of the work vehicle 2 stores detection means and response means for abnormalities in the work equipment 11 mounted on the work vehicle 2.

[0066] Furthermore, the vehicle control device 20 of the work vehicle 2 operates as an imaging control unit 23 and a driving control unit 24, as shown in Figure 5, by executing a program stored in the vehicle memory unit 21, and also operates as an abnormality notification unit 26 instead of a parameter setting unit 25, similar to the embodiment described above. Alternatively, the vehicle control device 20 may operate as an imaging control unit 23, a driving control unit 24, and a parameter setting unit 25, and in addition, as an abnormality notification unit 26. The abnormality notification unit 26 implements the notification process of the control method for the work vehicle 2 according to the present invention.

[0067] The abnormality notification unit 26 receives input from the management server 4 regarding notifications of abnormalities in the work equipment 11 and the provision of means for responding to such abnormalities, which differ for each type and / or model of the work equipment 11. For example, as shown in Figure 6, the abnormality notification unit 26 notifies the worker of the abnormality in the work equipment 11 by displaying the notification of the abnormality in the work equipment 11 provided by the management server 4 on the vehicle display unit 14 via the abnormality notification screen 72. Alternatively, the abnormality notification unit 26 notifies the worker of the abnormality in the work equipment 11 by outputting the notification of the abnormality in the work equipment 11 provided by the management server 4 as audio via the vehicle audio output unit 15.

[0068] Furthermore, the abnormality notification unit 26 notifies the operator of the means to respond to the abnormality of the work equipment 11, provided by the management server 4, by displaying the means on the vehicle display unit 14 via the abnormality notification screen 72. Alternatively, the abnormality notification unit 26 notifies the operator of the means to respond to the abnormality of the work equipment 11, provided by the management server 4, by outputting the means to respond to the abnormality of the work equipment 11 via the vehicle voice output unit 15. For example, the abnormality notification unit 26 displays a notification on the screen such as, "An abnormality has been detected in the work equipment 11. Please stop work and check it. Please set the height of the work equipment to ○○," when an abnormality occurs in the work equipment 11 or as a means to respond to the abnormality of the work equipment 11.

[0069] In the modified example, an example was described in which, in a work vehicle 2, the abnormality notification unit 26 of the vehicle control device 20 notifies the worker of the occurrence of an abnormality in the work equipment 11 and the means of responding to the abnormality in the work equipment 11, which differ for each type and / or model of the work equipment 11. However, the present invention is not limited to this example, and a mobile terminal 3 may also provide similar notifications.

[0070] Furthermore, in the modified version, the server control device 40 of the management server 4 operates as a learning unit 60, a detection unit 61, and a notification unit 62 by executing a program stored in the server storage unit 41, and the management server 4 operates as an artificial intelligence server. The learning unit 60, detection unit 61, and notification unit 62 realize the learning process, detection process, and notification process of the control method for the work vehicle 2 according to the present invention.

[0071] The learning unit 60 stores in the database 43 machine learning models (learning models) that have been trained on various learning images of each type and model of work equipment 11 under normal and abnormal conditions. The learning unit 60 may input images previously taken by the vehicle imaging unit 18 or the terminal imaging unit 37 as learning images of the work equipment 11, or it may input images that have been made public on the internet by websites, etc.

[0072] For example, the learning unit 60 analyzes training images of the work implement 11 to determine the type and model of the work implement 11 captured in the training images, and to determine whether the work implement 11 captured in the training images is normal or abnormal. At this time, the learning unit 60 may add information to the training images that is determined to identify the type and model of the work implement 11 from among the features that indicate the work implement 11 captured in the training images. The learning unit 60 may also add a flag to the training images indicating whether the work implement 11 is normal or abnormal. Furthermore, for each type and model of work implement 11, the learning unit 60 generates a machine learning model of the training images by machine learning the features that differ between the training images when the work implement 11 is normal and when it is abnormal, and stores it in the database 43.

[0073] The learning unit 60 may use artificial intelligence (AI) to analyze training images of the work implement 11, determine the characteristics of the work implement 11 to identify its type and model, and determine whether the work implement 11 is normal or abnormal. For example, the learning unit 60 may use machine learning to analyze the characteristics of the work implement 11 captured in the training images, thereby generating a machine learning model of the characteristics of the work implement 11 in the training images and storing it in the database 43.

[0074] Furthermore, the learning unit 60 stores in the database 43 learning images of each type and model of work machine 11 in both normal and abnormal states, as well as machine learning models (learning models) that have been trained on means for detecting and / or responding to abnormalities occurring in each type and model of work machine 11.

[0075] The learning unit 60 may learn conditions for detecting abnormalities in the work machine 11 based on the differences in the characteristics of the learning images under normal conditions and abnormal conditions, as a means for detecting abnormalities in the work machine 11. Furthermore, the learning unit 60 may learn conditions for detecting abnormalities in the work machine 11 by analyzing the instruction manual for the work machine 11 stored in the server storage unit 41 and information about the work machine 11 published on the internet via websites, etc., as a means for detecting abnormalities in the work machine 11.

[0076] Furthermore, the learning unit 60 may learn countermeasures for abnormalities occurring in the work machine 11 by analyzing the instruction manual for the work machine 11 stored in the server storage unit 41 and information about the work machine 11 published on the internet via websites, etc.

[0077] Furthermore, the learning unit 60 generates a machine learning model by linking learning images of each type and model of work machine 11 in normal and abnormal conditions with means for detecting and / or responding to abnormalities occurring in each type and model of work machine 11, and stores it in the database 43.

[0078] The learning unit 50 may generate detection and response methods for abnormalities occurring in the work machine 11 using a generating AI. For example, the learning unit 50 generates machine learning models for detection and response methods for abnormalities occurring in the work machine 11 by machine learning the differences in the characteristics of learning images under normal conditions and abnormal conditions, as well as the instruction manual for the work machine 11 and information about the work machine 11 published on the internet, and stores them in the database 43.

[0079] The detection unit 61 determines the type and model of the work equipment 11 mounted on the work vehicle 2, and detects any abnormalities in the work equipment 11, based on images of the work equipment 11 captured by the vehicle imaging unit 18 and the terminal imaging unit 37. For example, the detection unit 61 determines the type and model of the work equipment 11 captured in the work equipment image by using artificial intelligence (AI) based on a machine learning model of learning images of the work equipment 11 in normal and abnormal states, thereby determining the work equipment 11 mounted on the work vehicle 2. Furthermore, the detection unit 61 uses artificial intelligence (AI) to detect any abnormalities in the work equipment 11 captured in the work equipment image, based on a machine learning model of learning images of the work equipment 11 in normal and abnormal states, and a machine learning model of the abnormality detection means for the work equipment 11.

[0080] The notification unit 62, when the detection unit 61 detects an abnormality in the work equipment 11 whose type and model have been determined by the detection unit 61, transmits the occurrence of the abnormality in the work equipment 11 to the work vehicle 2 and the mobile terminal 3 via the server communication unit 42 to notify them, and also determines the means to respond to the abnormality and transmits the determined means to the work vehicle 2 and the mobile terminal 3 via the server communication unit 42 to propose the means to respond to the abnormality in the work equipment 11. For example, based on a machine learning model of the means to respond to the abnormality of the work equipment 11, the notification unit 62 uses artificial intelligence (AI) to determine the appropriate means to respond to the abnormality of the work equipment 11, according to the type and model of the work equipment 11 and the abnormality, as captured in the work equipment image.

[0081] As described above, according to the modified example, the control system 1 for the work vehicle 2 and / or work machine 11 has a learning unit 60 in the management server 4 that stores a machine learning model (learning model) that has been trained on learning images of the work machine 11 in normal and abnormal conditions, an imaging control unit 23 in the work vehicle 2 that captures images of the work machine 11 using a vehicle imaging unit 18 and a terminal imaging unit 37, and a detection unit 61 in the management server 4 that detects abnormalities of the work machine 11 captured in the work machine image based on the machine learning model.

[0082] In other words, in the modified example, the control method for the work vehicle 2 and / or work machine 11 includes a learning step of accumulating a machine learning model (learning model) that has been trained on learning images of the work machine 11 in normal and abnormal conditions; an imaging step of capturing images of the work machine 11 using the vehicle imaging unit 18 and the terminal imaging unit 37; and a detection step of detecting abnormalities of the work machine 11 captured in the work machine image based on the machine learning model.

[0083] As a result, according to the modified version, even if the abnormalities that occur in the work equipment 11, as well as the means for detecting and responding to them, differ depending on the type and model of the work equipment 11, artificial intelligence (AI) based on a machine learning model that has learned images of the work equipment 11 in both normal and abnormal states can be used to automatically determine the type and model of the work equipment 11 and automatically determine any abnormalities that occur in the work equipment 11. Therefore, without requiring any extra effort from the operator, it is possible to understand any abnormalities that occur in the work equipment 11 according to the type and / or model of the work equipment 11 that is installed, and to handle the work vehicle 2 and the work equipment 11 appropriately.

[0084] Furthermore, according to the modified version, the notification unit 62 of the management server 4 and the abnormality notification unit 26 of the work vehicle 2 notify the abnormality of the work machine 11 by displaying it on the abnormality notification screen 72.

[0085] This allows abnormalities occurring in the work machine 11 to be visualized and output through a screen display.

[0086] Alternatively, according to a modified version, the notification unit 62 and the abnormality notification unit 26 notify of any abnormality that occurs in the work machine 11 by outputting an audible signal.

[0087] This allows abnormalities in the work machine 11 to be output via voice, without requiring the operator to visually confirm the abnormality.

[0088] In a modified version, the learning unit 60 stores learning images of the working machine 11 in both normal and abnormal states, as well as a machine learning model (learning model) that has been trained on means for detecting and / or responding to abnormalities in the working machine 11. The notification unit 62 and the abnormality notification unit 26 then determine and propose means for responding to abnormalities in the working machine 11 captured in the working machine image, based on the machine learning model.

[0089] This allows the system to automatically determine and propose countermeasures for abnormalities occurring in the work equipment 11, which vary depending on the type and model. Therefore, without requiring any extra effort from the operator, they can understand the countermeasures for abnormalities occurring in the installed work equipment 11, and can handle the work vehicle 2 and work equipment 11 appropriately.

[0090] Furthermore, in a modified version, the notification unit 62 and the abnormality notification unit 26 display and propose countermeasures for abnormalities in the work machine 11 on the abnormality notification screen 72.

[0091] This allows the means of responding to abnormalities in the work machine 11 to be concretely represented and visually output through a screen display.

[0092] Alternatively, according to a modified version, the notification unit 62 and the abnormality notification unit 26 propose a response to an abnormality in the work machine 11 by outputting an audio signal.

[0093] This allows for the output of a response to any abnormalities occurring in the work machine 11 via voice, without requiring the operator to visually confirm the information.

[0094] In the embodiments and modifications described above, the control system 1 operates the parameter setting unit 25 and the abnormality notification unit 26 using the computer of the vehicle control device 20 of the work vehicle 2, and the work vehicle 2 proposes parameters and countermeasures. However, the present invention is not limited to this example.

[0095] In another example, the control system 1 may be configured to operate the parameter setting unit 25 and the abnormality notification unit 26 using the computer of the terminal control device 30 of the mobile terminal 3, so that the mobile terminal 3 can suggest parameters and countermeasures. In this case, the setting of parameters for the work vehicle 2 is performed by the parameter setting unit 25 provided on the work vehicle 2, rather than the parameter setting unit 25 provided on the mobile terminal 3.

[0096] Furthermore, in the embodiments and modifications described above, the learning unit 50, determination unit 51, proposal unit 52, learning unit 60, detection unit 61, and notification unit 62 in the control system 1 are operated by the computer of the server control device 40 of the management server 4, and the management server 4 is equipped with a database 43, thereby operating the management server 4 as an artificial intelligence server. However, the present invention is not limited to this example.

[0097] In another example, the control system 1 may operate the learning unit 50, the determination unit 51, the proposal unit 52, and the learning unit 60, the detection unit 61, and the notification unit 62 on a computer of an artificial intelligence server different from the management server 4, and may also have a database 43 on such another artificial intelligence server. In this case, the learning unit 50, the determination unit 51, the proposal unit 52, and the learning unit 60, the detection unit 61, and the notification unit 62 may operate as artificial intelligence (AI) or generative AI on the computer of such another artificial intelligence server. Furthermore, the work vehicle 2 and the mobile terminal 3 will not only be able to communicate with the management server 4, but will also be able to communicate with such other artificial intelligence servers.

[0098] Alternatively, in another example, the control system 1 may operate the learning unit 50, judgment unit 51, suggestion unit 52, or the learning unit 60, detection unit 61, and notification unit 62 on the computer of the vehicle control device 20 of the work vehicle 2 or the terminal control device 30 of the mobile terminal 3, thereby operating the work vehicle 2 or mobile terminal 3 as an artificial intelligence server. Furthermore, the database 43 may be provided on the work vehicle 2 or mobile terminal 3, or on another artificial intelligence server or other computer different from the management server 4. In this case, the learning unit 50, judgment unit 51, suggestion unit 52, or the learning unit 60, detection unit 61, and notification unit 62 may operate as artificial intelligence (AI) or generative AI on the computer of the vehicle control device 20 of the work vehicle 2 or the terminal control device 30 of the mobile terminal 3. In addition, the work vehicle 2 or mobile terminal 3 may be connected not only to the management server 4 but also to a database 43 outside the management server 4.

[0099] In other words, according to this embodiment and its modifications, the control system 1 only needs to include a learning unit 50, a determination unit 51, a proposal unit 52, or a learning unit 60, a detection unit 61, and a notification unit 62. Here, the learning unit 50, the determination unit 51, the proposal unit 52, the learning unit 60, the detection unit 61, and the notification unit 62 may be executed by any of the computers of the vehicle control device 20 of the work vehicle 2, the terminal control device 30 of the mobile terminal 3, the server control device 40 of the management server 4, or other artificial intelligence servers.

[0100] In the embodiments described above, an example was given in which the work vehicle 2 is composed of a tractor, but the present invention is not limited to this example. For example, the work vehicle of the present invention may be composed of other agricultural machinery such as a combine harvester, rice transplanter, or lawnmower, or it may be composed of other work vehicles other than agricultural machinery.

[0101] Furthermore, the present invention may be modified as appropriate, provided that it does not contradict the gist or idea of ​​the invention as can be inferred from the claims and the specification as a whole, and control methods involving such modifications are also included in the technical concept of the present invention.

[0102] [Notes on the invention] The following is an overview of the invention extracted from the above-described embodiments. Note that each configuration and processing function described below can be selected and combined as desired.

[0103] <Note 1> A control method for work vehicles and / or work machines, A learning process for accumulating a learning model obtained by machine learning using the learning images of the work machine and the learning parameters of the work vehicle and / or the work machine, The imaging step involves capturing an image of the work machine, A determination step is to determine the work machine captured in the work machine image based on the learning model, A control method characterized by comprising a suggestion step of determining and suggesting parameters to be set on the work vehicle and / or the work machine according to the work machine determined in the determination step, based on the learning model.

[0104] <Note 2> The control method according to Appendix 1, characterized in that the proposed step involves displaying the parameters on a screen and making a proposal.

[0105] <Note 3> The control method according to Appendix 1, characterized in that the proposed step involves proposing the parameters by outputting them as audio.

[0106] <Note 4> The control method according to any one of the appendices 1 to 3, characterized in that the proposed step displays a screen on which the user can select whether or not to set the proposed parameters.

[0107] <Note 5> The control method according to any one of the appendices 1 to 3, characterized in that the proposed step automatically sets the proposed parameters.

[0108] <Note 6> A control method for work vehicles and / or work machines, A learning process for accumulating a learning model that has been trained using machine learning on learning images of the work machine in both normal and abnormal conditions, The imaging step involves capturing an image of the work machine, A control method characterized by comprising a detection step of detecting an abnormality of the work machine captured in the work machine image based on the learning model.

[0109] <Note 7> The control method according to Appendix 6, characterized in that it includes a notification step of displaying an abnormality of the work machine on a screen to notify the user.

[0110] <Note 8> The control method according to Appendix 6, characterized in that it includes a notification step of notifying of an abnormality in the work machine by outputting an audible signal.

[0111] <Note 9> The learning process involves accumulating the learning model, which has been trained on the detection and / or response means for detecting abnormalities in the work machine, in addition to the learning images of the work machine in both normal and abnormal conditions. The control method according to Appendix 7 or 8, characterized in that the notification step determines and proposes a means to respond to an abnormality of the work machine captured in the work machine image based on the learning model.

[0112] <Note 10> The control method according to Appendix 9, characterized in that the notification step proposes a means for responding to an abnormality in the work machine by displaying it on a screen.

[0113] <Note 11> The control method according to Appendix 9, characterized in that the notification step proposes a means for responding to an abnormality in the work machine by outputting it as an audible signal. [Explanation of symbols]

[0114] 1. Control System 2 Work vehicles 3 Mobile devices 4. Management Server 11. Work equipment 14. Vehicle display section 15. Vehicle audio output unit 18. Vehicle imaging unit 20 Vehicle control system 21 Vehicle memory unit 23 Imaging control unit 24 Operation Control Unit 25 Parameter setting section 26 Anomaly Reporting Department 40 Server Control Units 41 Server Storage Unit 43 Databases 50 Learning Department 51 Judgment section 52 Proposal Department 60 Learning Department 61 Detection unit 62 Hochi Department 70 Parameter suggestion screen 71 Individual Settings Screen 72 Anomaly Notification Screen

Claims

1. A control method for work vehicles and / or work machines, A learning process for accumulating a learning model obtained by machine learning using the learning images of the work machine and the learning parameters of the work vehicle and / or the work machine, The imaging step involves capturing an image of the work machine, A determination step is to determine the work machine captured in the work machine image based on the learning model, A control method characterized by comprising: a proposal step of determining and proposing parameters to be set on the work vehicle and / or the work machine according to the work machine determined in the determination step, based on the learning model.

2. The control method according to claim 1, characterized in that the proposed step involves displaying the parameters on a screen.

3. The control method according to claim 1, characterized in that the proposed step involves proposing the parameters by outputting them as audio.

4. The control method according to claim 1, characterized in that the proposed step displays a screen on which the user can select whether or not to set the proposed parameters.

5. The control method according to claim 1, characterized in that the proposed step automatically sets the proposed parameters.

6. A control method for work vehicles and / or work machines, A learning process for accumulating a learning model that has been trained using machine learning on learning images of the work machine in both normal and abnormal conditions, The imaging step involves capturing an image of the work machine, A control method characterized by comprising a detection step of detecting an abnormality of the work machine captured in the work machine image based on the learning model.

7. The control method according to claim 6, further comprising a notification step of displaying an abnormality of the work machine on a screen to notify the user.

8. The control method according to claim 6, characterized in that it includes a notification step of notifying of an abnormality in the work machine by outputting an audible signal.

9. The learning process involves accumulating the learning model, which has been trained on the detection and / or response means for detecting abnormalities in the work machine, in addition to the learning images of the work machine in both normal and abnormal conditions. The control method according to claim 7 or 8, characterized in that the notification step determines and proposes a means to respond to an abnormality of the work machine captured in the work machine image based on the learning model.

10. The control method according to claim 9, characterized in that the notification step proposes a means for responding to an abnormality in the work machine by displaying it on the screen.

11. The control method according to claim 9, characterized in that the notification step proposes a means for responding to an abnormality in the work machine by outputting an audio signal.

Images