Work vehicle

Abstract

To solve the problems that in a conventional technique, images photographed from a fixed position in the same condition are used when performing difference analysis by comparing images, but it is difficult to compare images photographed from an unstable position like a work vehicle.SOLUTION: In detection of a difference between reference images and newly photographed images, photographing is performed with the same work vehicle in the same places in a field toward the same directions, and further, photographing is performed so as to make a part of the work vehicle get in images of an imaging device, and correction processing of overlapping reference parts of the work vehicle getting in the images, and matching a scale for a size and inclination of the work vehicle with a tone of color of the work vehicle is performed so as to detect a difference between the reference images and the newly photographed images.SELECTED DRAWING: Figure 12

Description

[Technical Field]

[0001] The present invention uses an imaging device to analyze the difference between a reference image and a new image, and This relates to a work vehicle that operates in a specific work mode or takes safety measures. [Background technology]

[0002] In the prior art, there is a system that performs image matching of pest footprints and manages information related to the images. (Patent Document 1) [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7322775 Summary of the Invention [Problem to be solved by the invention]

[0004] In the prior art, the accuracy of matching is high because the main focus is on comparing and matching the captured data. Furthermore, in order to match the image data, it is necessary to regulate the imaging conditions for acquiring the image, but this is not described in detail.

[0005] The present invention aims to provide a work vehicle that can analyze the correlation between various types of data, avoid abnormalities, and detect defects by restricting imaging conditions for comparison and utilizing a matching function using artificial intelligence to process data that cannot be simply matched. [Means for solving the problem]

[0006] The present invention is solved by the following technical means.

[0007] A work vehicle capable of detecting a difference between a reference image previously captured by an imaging device and an image newly captured by the imaging device, The reference image and the newly captured image are both captured at the same location in the field and in the same direction, A part of the work vehicle is included in both the reference image and the newly captured image, and a reference part of the work vehicle included in the image is set; superimposing the reference portions of the reference image and the newly captured image; By performing a correction process to match the scale and inclination of the size of the work vehicle that has entered the image and the color of the work vehicle that has entered the image, it is possible to detect the difference between the reference image and the newly captured image.

[0008] A configuration is provided in which a change between a reference image and a newly captured image can be detected, If a change of a magnitude greater than a predetermined range is detected, the area where the change was detected is set as a specific area, the work vehicle is stopped, and continuous image capture is taken.If movement is confirmed in the specific area, it is determined to be a moving object, and a pre-set moving object avoidance mode is entered.

[0009] A configuration is provided in which a change between a reference image and a newly captured image can be detected, It is desirable to take an image of the growing part of a crop in a field, and if a change in color is detected in the imaged growing part, to determine that it is pest damage and enter a preset pest response mode. .

[0010] A configuration is provided in which a change between a reference image and a newly captured image can be detected, It is desirable to take an image of the water surface of a paddy field, and if a change in color or shape is detected in the image of the water surface, to determine that it is a weed and enter a preset weed control mode. .

[0011] The system is configured to be able to detect changes between a reference image and a newly captured image, The input data is a difference determination between the reference image and the newly captured image, the driving data of the work vehicle at the time of the image capture, and the map data of the foreign object detection position. When an abnormality is detected or a setting change is made manually by controlling the work vehicle based on the output results of the neural network, It is desirable to have a function to add the data of the abnormality detection and the data of the manual setting change, and to learn the correlation between the input and output of each data. .

[0012]

[0013]

[0014]

[0015] [Effects of the Invention]

[0016] According to the present invention, the captured images are taken under conditions that make it easy to match, and there are standards for confirming the match in subsequent image data matching, so that changed areas can be extracted with high accuracy.

[0017]

[0018]

[0019]

[0020] [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a diagram showing a reference image of the present invention during tillage work; [Figure 2]New images of tillage work according to the present invention, with different imaging scales [Figure 3] A diagram showing a moving object being detected in a new image taken during tillage work according to the present invention. [Figure 4] A new image of the present invention during tillage work, showing different field conditions. [Figure 5] A diagram showing a case where an obstacle is confirmed in a new image during tillage work according to the present invention. [Figure 6] 1 is a diagram showing a reference image during pest control according to the present invention. [Figure 7] This is a new image of the present invention, showing damage to crops caused by pests. [Figure 8] Image processing analysis of the present invention when food is eaten by pests [Figure 9] 1 is a diagram showing a reference image for weeding work after rice planting according to the present invention; [Figure 10] This is a new image of weeding work after rice planting in this invention, showing weeds and other weeds being identified. [Figure 11] The figure shows the reference image and the new image during the tilling work of the present invention, and shows that the movement amount of the moving object is calculated using a neural network. [Figure 12] This figure shows a reference image and a new image during tillage work in accordance with the present invention, and illustrates how the scale and tilt are corrected based on the work vehicle to perform image matching using a neural network. [Figure 13] Configuration diagram of a dedicated machine for handling moving objects according to the present invention. [Figure 14] Equipment configuration diagram when using the dedicated machine for moving objects of the present invention [Figure 15] Configuration diagram of the dedicated pest control machine of the present invention [Figure 16] Configuration diagram of the pest control machine of the present invention [Figure 17] Configuration diagram of the weed-control dedicated machine of the present invention [Figure 18] Operation diagram of the weed control machine of the present invention when it is manually operated DETAILED DESCRIPTION OF THE INVENTION

[0022] The present invention will be described below with reference to the embodiments shown in the drawings.

[0023] The work vehicle shown in FIGS. 1 to 18 shows an example of this embodiment.

[0024] The work vehicle of the present invention will now be described.

[0025] This invention includes tractors, rice transplanters, combine harvesters, and cultivators. Each work vehicle has its own unique movement characteristics, but here we will explain using tractors as a representative model. Each work vehicle is capable of automatic operation and is equipped with a satellite positioning device and an inertial positioning device to enable unmanned operation, and travel steering and work equipment operation operate based on registered data. The reference travel route is based on the field, and is registered and managed using map data.

[0026] The background of the present invention will be explained.

[0027] As automation advances in agriculture, it would be ideal for work vehicles to perform tasks automatically. However, farm work does not simply involve repeating the same tasks. A variety of crops are cultivated in vast fields, and there are seasonal weather variations, making reproducibility as high as in industrial production sites. While fields are privately managed land, they are not enclosed within buildings, allowing for free movement of moving objects. Pests, as crops provide convenient food, sometimes enter fields. Similarly, humans may become interested in automated work vehicles and approach them.

[0028] Moving objects during such work can be a risk factor, and this can be used as a safety measure for autonomous driving. Furthermore, autonomous driving allows for operation during times when humans cannot cope, long-term work, and repeated operation. If this function and the image capture device can be used to analyze the data, continuous extermination of pests and vermin is also possible. Furthermore, in weed control of rice paddies, removing weeds and weeding work can be done simultaneously, which saves labor.

[0029] The first aspect of the present invention will be described.

[0030] 1 is a diagram of a reference image during tillage work. The work vehicle is a tractor 10.

[0031] The entire screen of the imaging device is captured as shown in Figure 1. The first important point is that when registering the image of the reference image G1, the latitude and longitude of the image must be registered and the direction of the image must be clearly defined. Regarding the direction of the imaging device, a digital camera is equipped with an infrared sensor or other device to measure the distance to the subject. Alternatively, the direction of the imaging device can be the direction of the focal point of the imaging device's lens, but this direction of the imaging device to the subject is superimposed on the latitude and longitude of the imaging location. The imaging device also has an angle in the vertical direction. The direction is registered by registering three directions, including this angle, namely the latitude and longitude of the imaging direction from the registered position of the imaging device to the subject, and the imaging angle of the imaging device from the ground. This determines the standard for the imaging device, which is used to compare the difference between an image captured by the same work vehicle at the same location in the field facing the same direction and a newly captured image taken by the same work vehicle at the same location in the field facing the same direction.

[0032] By performing this type of work, newly captured images can be compared. In general image comparisons, the image capture device and the subject are placed on a fixed jig, and images are captured by matching target points. This method of placing the subject in a predetermined position as a condition for comparison is an easy method for image comparison, but since moving objects entering the field do not move to a predetermined position according to instructions, the image capturer must capture an image to be compared with the reference image.

[0033] However, even if these conditions are known, comparison of image G1 is not easy based on this alone. The reason is that the imaging device is mounted on a mobile work vehicle such as the tractor 10, and the tractor 10 travels through a field, not like a car traveling on an asphalt road. The unevenness and softness of the field cause the work vehicle to tilt forward, backward, left, and right. Because this tilt occurs three-dimensionally, even if images are taken with the same work vehicle at the same location in the field, facing the same direction, correcting for the three-dimensional tilt may result in the image scale being slightly different. This difference in size could result in the same object being judged to have a different shape. Therefore, the scale of this image needs to be corrected. Since tilt also occurs, tilt correction is also required.

[0034] Furthermore, the present invention also performs comparisons based on color. Even if the shape is the same, differences in color indicate changes. For example, green and brown plants show different growth patterns. The RGB color, which represents the color, is corrected to match the color of the work vehicle. If the captured image is taken at a different time or season, the angle of the sunlight, etc., can cause the color of the same work vehicle to appear different. Therefore, since the work vehicles captured in the reference image and new image are the same, this color difference can be corrected so that the color of the work vehicle in the new image is the same as the color of the work vehicle in the reference image. By applying this color correction to the entire image, it is possible to compare the colors of other parts of the field, thereby enabling a comparison of shape and color.

[0035] In addition, for shadows that occur depending on the angle of the sunlight, a filtering effect is applied using artificial intelligence to erase the image, but the angle of the sunlight is calculated based on the date and time the image was taken, and the position of the shadow is moved and the color is corrected to match the angle of the sunlight at the date and time of the reference image.

[0036] Note that reference numbers 20 to 57 in Figure 1 represent rice stubbles left in the field after harvesting with a combine harvester. By matching the color of the work vehicle, it becomes possible to confirm changes in the color of the rice stubbles. The reference image was taken immediately after rice harvesting, and the new image was taken immediately after land leveling work. If the images were taken at different times, it is possible to distinguish between the images by color, even if the shapes of the rice stubs are in a similar range.

[0037] Figure 2 is a new image taken during tillage work. As mentioned above, when the three-dimensional tilt is corrected, the scale and tilt of the image are slightly different. The difference between image G1 in Figure 1 and image G2 in Figure 2 shows the difficulty of comparing these captured images.

[0038] As such, comparing size using images is not easy. For example, consider the deployment of LIDAR (Light Detection and Ranging). LIDAR is a type of remote sensing technology that uses light to measure scattered light from pulsed laser radiation, making it possible to analyze the shape and properties of an object. While it is possible to measure the distance to an object, and while this is effective for analyzing individual data, when comparing captured image data, the images are calculated using dots or polygon shapes, making it difficult to determine minutely rounded shapes using numerical comparisons. Rather, it is more accurate to simply compare the image data and analyze the shape of a certain area using a convolutional neural network.

[0039] Therefore, it is necessary to align the scale and angle of the image, but this reference becomes a problem. It is difficult to find the reference within the field when the work vehicle is constantly moving within the field.

[0040] Therefore, in this invention, an image of a portion of a work vehicle is captured by an imaging device. Since the work vehicles are the same, for example, if an imaging device is installed inside the cabin of the work vehicle, it is possible to compare three-dimensional distances from the width and length of the work vehicle's hood, the width and position of the front wheels, or the size of the cabin window frame (not shown). However, since it is difficult to match completely with no difference in all directions, a tolerance is set in each direction, and differences between them are processed to be considered as a match, allowing for immediate response.

[0041] By superimposing a portion of the work vehicle that has entered the image and performing a process to align the scale and inclination of the work vehicle's size, it is possible to detect changes between the reference image and the newly captured image, making it possible to compare the reference image with the new image. Differences between the images are judged to be a different situation in unmanned operation of a robotic work vehicle, etc., and it is possible to respond in accordance with the comparison standard that states are different from usual, based on human experience.

[0042] The relationship in which the process of matching the scale and inclination of the size of this work vehicle is performed is explained in Figure 12. Image G1 in Figure 1 is the reference image, and image G2 in Figure 2 is a newly captured image. As explained above, it can be seen that image G2 has a different scale and inclination than image G1. If the images were simply compared in this state, the size of the field, the size of the rice stubbles after harvesting, and the pitch between the stubbles would be different, and different parts would be detected as being different in each area.

[0043] Therefore, as mentioned above, a reference for correcting the scale and tilt is calculated by capturing part of the work vehicle in the image. Image 91 is a diagram in which the tractor 10 (solid line) in image G1 is superimposed on the tractor 10A (two-dot chain line) in image 2. Because the tractor 10 and tractor 10A are the same, detecting this difference allows the difference in the scale and tilt of the image to be detected.

[0044] First, the reference point of the tractor 10 must be registered. In this embodiment, the meter panel position 92 of the tractor 10 is used as the reference point. By aligning this image 91, the two are compared, centering on the outline of the image shape. The image is expressed as a collection of dots of a certain size, and a convolutional neural network is used to calculate the amount of movement required to compare the deviation of the outline and to perform calculations to compare the colors of each dot.

[0045] As a result, in the image 91, it is detected that the tractor 10A is tilted 3 degrees to the left, and the scale is 80% (560 mm / 700 mm) in the images of the tractor 10 and tractor 10A.

[0046] Referring to the comparison flowchart 80, if there is a difference in the work vehicles in S80, then the tilt comparison is performed in S82. Then, the scale is compared in S84, and the image comparison in S86 begins.

[0047] Through this image processing, image G2 is corrected to tilt 3 degrees to the right, resulting in image G2A. Image G2B is then enlarged by 125% (700 mm / 560 mm), and by comparing image G1 with image G2B, any changes between the reference image and the newly captured image can be detected.

[0048] This type of image matching makes it possible to detect changes in the field. As an example, the differences will be explained using image G1 in Figure 1 and image G2 in Figure 2. In image G2 in Figure 2, it can be seen that the rice stubbles at 22A, 36A, 44A, and 45A have disappeared. In this way, differences in the field can be detected. However, since the rice stubbles do not pose a problem for work, if the shape of the rice stubbles is set to be excluded from detection in advance, it is possible to prevent them from being detected using the filtering function.

[0049] By using artificial intelligence for such image filtering functions and specific image detection, it is possible to detect similar shapes quickly and accurately. Data 87 in Figure 12 is generated by breaking down the detected image from image G2B into areas and inputting information about various objects based on color and image contours. Data 88 weights the relevance of data 87, data 89 predicts objects that can be inferred, and data 90 outputs the identified object. In other words, by capturing an image and using other measurement data (input data), it is possible to identify the object captured in the image (output data). While it is easy for humans to infer the shape of an object from memory and identify the relevant object from a captured image, this is difficult for machines, so artificial intelligence is used to make the identification.

[0050] The output of the artificial intelligence is used not only to identify objects but also to determine differences in images, and in the image comparison process, it is possible to identify objects in areas where changes have occurred and analyze the causes.

[0051] The second aspect of the present invention will now be described.

[0052] When detecting changes between the reference image described above and a newly captured image, if there is a change of a magnitude greater than a predetermined range, the area of ​​the change is set as a specific area, the work vehicle is stopped, and continuous images are taken.If movement is confirmed in the specific area, it is determined to be a moving object, and a pre-set moving object avoidance mode is entered.

[0053] First, even if the moving object is palm-sized, it is likely a small bird or animal, and compared to the size of a work vehicle, it may run away when the work vehicle approaches, so it can be excluded as a moving object. However, if it is larger than the size of both hands, it becomes the size of a dog, cat, wild boar, weasel, etc. If it is the size of a human child, it is even more serious, and if there is a change in size beyond a specified range, the area where the change occurred is designated as a specific area, and continuous images are taken and analyzed.

[0054] Image G3 in Figure 3 is a newly captured image in which an object 60 large enough to be determined as an obstacle has been detected. The reference image for this image is image G1 in Figure 1. Here, the scale and tilt of the image are the same, and correction will not be mentioned, but the same work as in the first invention is performed.

[0055] Fig. 11 explains how to analyze whether an object is a moving object. This figure shows how the amount of movement of a moving object is calculated using a neural network in a reference image and a new image during tillage work.

[0056] The image marked G1 is the reference image during tillage work. Image G3 is the image captured this time. In image G3, it can be seen that object 60 is different from image G1. In this case, the work vehicle makes an emergency stop. The emergency stop is made to improve the accuracy of the image capture and, above all, to prevent damage caused by an approaching moving object.

[0057] Image G3A is an image captured after the emergency stop, and it can be seen that object 60 has been detected as moving as object 61, and that its shape is also different. This allows object 60 to be recognized as a moving object. Furthermore, in image G3B, it is determined that object 62 is approaching the work vehicle even closer.

[0058] In this way, when the movement of an object larger than a predetermined size is confirmed, the system enters moving object avoidance mode. In moving object avoidance mode, not only does the work vehicle stop moving, but the work machine itself also stops working. In the case of a robotic work machine, the operator's seat is locked. Furthermore, sound may be used to intimidate or verbal warnings may be issued. A lighting device may also be used to flash to indicate danger. Data communication is also performed, and captured images are sent to the user to check for misrecognition. The decision to resume operation is made at the user's command.

[0059] In this way, if the moving object is a living thing, the moving object avoidance mode may be able to avoid a dangerous situation. In the example, the case of a human is described, and it can be seen that the human turns its back on the work vehicle and moves away, as shown in image G3C.

[0060] A neural network can be used to determine whether object 60 in image G3, object 61 in image G3A, object 62 in image G3B, and object 63 in image G3C are the same object. Data 70 compares the data of captured images G1 to G3C and the data held by the work vehicle with registered data. Data input and output are the same as those described in FIG. 12.

[0061] Data 70 in FIG. 11 is created by breaking down images G1 to G3C into areas and inputting information about various objects based on color and image contours. Data 71 weights the relevance of data 70, data 72 assumes possible objects, and data 73 outputs the identified object. In other words, by capturing an image and using other measurement data (input data), it is possible to identify the object captured in the image (output data). While it is easy for a human to infer the shape of an object from memory and determine the relevant object from a captured image, this is difficult for a machine, so artificial intelligence is used to identify the object.

[0062] The output of the artificial intelligence is used not only to identify objects but also to determine differences in images, and in the image comparison process, it is possible to identify objects in areas where changes have occurred and analyze the causes.

[0063] In this case, the object was determined to be a moving object, but in the analysis method, once similarity matching of a human-shaped object has been performed, position detection is performed using face position 60A as a point. When the images are combined, it is confirmed that the face position moves from face position 60A ⇒ face position 61A ⇒ face position 62A ⇒ face position 63A. If a change in the shape and movement of the image outline is confirmed by the variation in the distance and direction of this movement per unit time and by combining images 60, 61, 62, and 63, it is determined that the object is a living thing and is highly likely to be a human.

[0064] Furthermore, even if the image is not determined to be the contour of a human, a detection standard for a moving object is established, and if the part shows a movement line such as face position 60A ⇒ face position 61A ⇒ face position 62A ⇒ face position 63A, the movement line of this reference position is confirmed and a judgment is made as to whether it is a moving object.

[0065] The movement of this line of movement is also used to determine whether it is a moving object or a living thing other than a human. Anything with irregular movement amount per unit time or direction of change is determined to be a living thing. In particular, if there is a change in the vertical direction of the reference line of movement, it can be determined that it is not a moving object such as a work vehicle. Also, by checking the change in movement when a work vehicle emits a sound to warn, it is possible to determine whether it is a living thing, but a human or other living thing. For example, if a work vehicle emits a voice saying, "Danger. Please move away," it is assumed that if it is a human, there will be a change in behavior, and this can be used as a basis for judgment.

[0066] Such image analysis and judgment is also performed from the input of data 70 to the output of data 73. In other words, it is possible to judge the phenomenon of a person approaching a work vehicle in a farm field from the captured image data.

[0067] Although the flow for entering the moving object avoidance mode is simply described as S74 to S78, the moving object avoidance mode S78 is entered depending on whether a different part is moving in S76. In a case like image G4 in Figure 4, the moving object avoidance mode is not entered, and S77 for detecting a part different from the normal operation mode is selected.

[0068] An example of this is shown in image G4. When compared with the reference image G1, rice stubs 20C to 44C were matched, but no movement of objects was confirmed. This also applies to cases where rice stubs 50C to 54C on the left side were not detected.

[0069] Image G5 is another example. In this image, object 80 is detected as a different part. This time, the number of objects has increased. In this case, the type of object object 80 is analyzed using a neural network. Since information on foreign objects found in fields has been accumulated through extensive experience, comparison images are generally available. Image data of stones, cardboard boxes, plastic boxes, umbrellas, nylon bags, wrapping paper, magazines, newspapers, tools, etc. are registered and can be compared based on similar shapes and colors. Data on living creatures such as humans, dogs, cats, weasels, and mice is also registered.

[0070] However, there are classifications for foreign objects, and they are classified according to the method of subsequent driving response, such as when the foreign object interferes with work and driving is not possible unless the foreign object is removed, when conditional driving is possible, when normal driving is possible, and when entering the moving object avoidance mode S78 mentioned above.

[0071] The criteria for classifying such foreign objects must be improved through a learning function. To do this, when detecting changes between a reference image and a newly captured image, the system uses image synthesis processing, work vehicle data, and map data as input data, and uses a neural network to learn correlations so that data on the changed parts of the image can be output as a method for controlling future autonomous driving.

[0072] The fifth aspect of the present invention will now be described.

[0073] This is one example of an embodiment of the present invention. Input data 70 of the neural network includes the reference image G1, image data at the time of foreign object detection (images G3 to G3C), driving data of the work vehicle at the time of detection (each setting of the tractor 10 and detection data when images G3 to G3C were taken), and map data of the foreign object detection position (position information of the moving object 60 when images G3 to G3C were taken), and the output data 73 is used to determine whether the moving object avoidance mode S78 responded correctly, and is evaluated based on whether any abnormalities occurred after the content of the response made by automatic driving, or whether there were any elements that were manually changed by a human.

[0074] This process of detecting anomalies or making manual changes is deemed to be an error or poor accuracy in the AI's judgment criteria, and the changes must be learned. These anomaly detections and manual changes are added as change elements to the input data 70. By repeating this process, the correlation increases further, making it possible to calculate the best output conditions for the field environment.

[0075] An example of a learning function part is the above-mentioned foreign object classification standard, which is improved by the learning function, and classifies the situation according to the subsequent driving response method, such as when it is determined that the foreign object is interfering with work and operation is not possible unless the foreign object is removed, when it is determined that conditional operation is possible, when it is determined that normal operation is possible, and when the above-mentioned moving object avoidance mode S78 is entered, and the standard value is changed.

[0076] For example, the criteria for determining that a foreign object must be removed before driving can be improved through learning. If an object is judged to be a stone in the captured image but is actually a black lump of soil, there is no problem with driving even if it is not removed, so the system will learn to distinguish between images of stones and soil lumps. In this case, the neural network is used to change the weighting of color discrimination, and by changing the accuracy of distinguishing between stones and soil lumps, the system will use this information to learn the criteria for classifying foreign objects.

[0077] The third aspect of the present invention will now be described.

[0078] When detecting changes between a reference image and a newly captured image, the system is set to capture an image of the growing area of ​​the crop, and if a change in color is confirmed within this area, the system will enter a pre-set pest response mode.

[0079] Image G6 in Figure 6 is a reference image, and image G7 in Figure 7 is a newly captured image. When compared, changes can be seen in plant 102B, plant 103C, plant 111A, plant 112B, and plant 113C in image G7. These changes are mainly due to the detection of differences in color rather than changes in shape.

[0080] If plant 111 becomes plant 111A, a close-up analysis of area 120 and area 121 where the change was detected reveals that pests have been discovered and that the color is different. When such a detection is made, disinfection is carried out mainly on the affected area. Note that disinfection is carried out in pest response mode, but since pest damage has occurred, the location of the affected plant is registered, and its location is registered on a map so that it can be excluded from harvesting.

[0081] The fourth aspect of the present invention will now be described.

[0082] When detecting changes between the reference image and a newly captured image, the system is set to capture the water surface of a rice paddy, and if a change in color or shape is confirmed within this range, the system will enter a pre-set weed control mode.

[0083] Image 9 in Figure 9 is the reference image, and image G10 in Figure 10 is the new image. Both images were taken after rice planting was completed and the paddy fields were being maintained. This is the time when weeds begin to appear in paddy fields. Weeds include Monochoria vaginalis, Kuroguai, and Japanese barnyard grass. If left unchecked, weeds 161-174 will spread across the entire area of ​​the waterway between rice plants 141A-154A, as shown in Image 10. This image shows a significant amount of weed growth, but if weeds are detected through image comparison, we will immediately begin work to capture the weeds and also muddy the paddy water, blocking sunlight from reaching the field surface in the areas between the rice plants, thereby preventing weeds from growing. This response is the weed response mode.

[0084] The focus of image judgment in the third and fourth inventions is color judgment. The reference image and the new image are superimposed, and the focus is on detecting differences in color tone in specific areas. Color tone is judged based on the degree of the three primary colors. Reference colors for each position are registered in the neural network, and if the area difference between different colors from these colors exceeds a predetermined value, an abnormality is detected. However, in the early stages of both image G7 and image G10, the changes are small, making judgment difficult. Therefore, it is advisable to use a neural network to comprehensively compare and judge changes in shape and color.

[0085] The second, third, and fourth aspects of the present invention are ways of responding when an abnormality is detected in an image, and more accurate responses are possible by launching a dedicated work vehicle in addition to the tractor 10, which is the representative model of the present invention.

[0086] The moving object avoidance mode of the second invention is described with reference to Figure 13. The moving object-compatible dedicated device 200 is a dedicated device used in the moving object avoidance mode. It enables automatic travel using a satellite positioning device 204. It is equipped with a water gun 207, which fires water like a bullet at moving objects to check their condition. It is assumed that animals will retreat, making this the primary response in the moving object avoidance mode. The imaging device 266 is a LIDAR and CCD camera that detects the shape and distance from the captured image to estimate the moving object. The crawler 209 allows it to travel over some uneven terrain. The head 210 rotates, causing the light 205 to rotate and emit light, which can intimidate moving objects and serve as a warning signal to nearby residents. The figures are a front view 201, a top view 203, and a side view 202.

[0087] In addition, the system can be used in a different manner as shown in Figure 14 (B), in which an imaging device 211 is installed in the corner of a field, and when a moving object 220 is confirmed to have entered the field 212, a dedicated machine 200 for handling moving objects is dispatched from a warehouse 213.

[0088] 14(A) shows a system that uses a drone 230 instead of the dedicated moving object handling machine 200. Similarly, when an intrusion of a moving object 220 into a farm field 212 is confirmed, the drone 230 is dispatched from a warehouse / charging device 231. The drone 230 is a system that emits sound to scare away the moving object 220.

[0089] The pest control mode of the third invention will be explained with reference to Figure 15. 251 is a top view of the pest control dedicated machine 250, and 252 is a front view showing the machine in spraying mode. 253 is a view showing the machine in charging mode, showing the connection with the charger 254.

[0090] The pest control dedicated machine 250 is capable of automatic travel using a satellite positioning device 262. The travel section can move using wheels 260 and crawlers 259, and the main body is equipped with a disinfectant tank 257, with a charging device and battery section 258 located below. A spraying device 255 is provided above, which emits a mist of disinfectant that is sprayed on the target area of ​​the plant 261 where the pests have occurred. Linked to images captured by the tractor 10, the position of the plant where the pests have occurred can be confirmed by latitude and longitude, and the height above ground can also be confirmed from the image, the target position can be confirmed, and measures can be taken to increase the amount sprayed at that location.

[0091] It is also possible to provide an ultraviolet lamp 256 that can be used to irradiate the area at the same time as spraying the disinfectant. The ultraviolet lamp is used to determine if eggs have been laid in the presence of pests, and to prevent hatching and remove the eggs.

[0092] The dedicated pest control machine 270 in Figure 16 is designed for tall plants such as crops 275. In addition to disinfecting, it also sprays water, and is equipped with piping 271 and nozzles 272 for use at high altitudes. The tank 273 is installed at a slightly higher altitude, and is slightly pressurized with the pump pressure used for spraying. It is equipped with a satellite positioning device 280 and is capable of automatic operation. The traveling section is the same as that of the dedicated pest control machine 250.

[0093] 16(A) shows the state during spraying, and FIG. 16(B) shows the state during charging. Charging is performed by contactlessly connecting charging unit 276 to power supply unit 278. At this time, disinfectant 277 is supplied.

[0094] The weeding mode of the fourth invention will be described using the dedicated weeding machine 300 in Figure 17. The views are a front view 303, a top view 301, and a side view 302. The dedicated weeding machine 300 is capable of automatic travel using a satellite positioning device 307. While it is possible to weed by attaching a dedicated weeding machine to the tractor 130, it is more effective to detect weeds in a rice paddy using an image, transmit the positional relationship data to the work vehicle 300, and have the dedicated weeding machine carry out the weeding.

[0095] This weed-killing machine 300 is equipped with a small chain 304 between front wheels 305 and rear wheels 306, and as it moves, it drags the small chain 304 across the muddy layer at the top of the rice paddy, stirring up the mud. In addition to stirring up mud, the chain can also entangle and tear off the grass in the paddy field, thereby removing weeds that have already grown.

[0096] Multiple small chains 304 are arranged in a row, and there are no chains in the areas where the rice grows, so the rice is protected and only weeds are entangled. The front wheels 305 have a larger diameter, so the chains do not get caught when turning.

[0097] Working in this way in conjunction with data from the weed control dedicated machine 300 is effective and can be done as part of the weed control mode.

[0098] 18A shows the state in which the tractor 130 has entered the weeding mode for the entire field, regardless of whether it has detected weeds. The tractor 130 takes a travel route across the entire field 308 and automatically travels to weed.

[0099] 18(B) shows a manual operation performed by the external controller 305. The position of weeds is detected by the tractor 130, but when weeding is performed at that position, weeds such as floating weeds may drift into an area where there are no weeds. To deal with this, an image capturing device is provided in the image capturing device 310, and the captured image is sent to the external controller 305, allowing weeding to be performed while checking the image. [Explanation of symbols]

[0100] 10 Tractor 20 Rice stubble (reference image) 20A Rice stubble (new image) 60 Objects (Human) 70 Data (input, neural network) 73 Data (output, neural network) 91 images (reference and new composite) 111 Plants (reference image) 111A Plants (new image) 120 Pests 161 Weeds 200 Dedicated machine for moving objects 250 Pest Control Machine 300 Weed Control Machine

Claims

1. A work vehicle capable of detecting a difference between a reference image previously captured by an imaging device and an image newly captured by the imaging device, The reference image and the newly captured image are both captured at the same location in the field and in the same direction, A part of the work vehicle is included in both the reference image and the newly captured image, and a reference part of the work vehicle included in the image is set; superimposing the reference portions of the reference image and the newly captured image; This work vehicle is capable of detecting differences between a reference image and a newly captured image by performing a correction process to match the scale and inclination of the size of the work vehicle that has entered the image and the color of the work vehicle that has entered the image.

2. A configuration capable of detecting changes between a reference image and a newly captured image, 2. The work vehicle of claim 1, wherein, when a change of a magnitude exceeding a predetermined range is detected, the part where the change is detected is set as a specific part, the work vehicle is stopped, and continuous image capture is performed, and when movement is confirmed in the specific part, it is determined to be a moving object, and a preset moving object avoidance mode is entered.

3. A configuration capable of detecting changes between a reference image and a newly captured image, 2. The work vehicle of claim 1, wherein an image of a growing part of a crop in a field is captured, and if a change in color is detected in the captured growing part, the image is determined to be damage caused by pests, and the work vehicle enters a preset pest response mode.

4. A configuration capable of detecting changes between a reference image and a newly captured image, 2. The work vehicle of claim 1, wherein the work vehicle captures an image of the water surface of a paddy field, and when a change in color or shape is detected in the image of the water surface, the work vehicle determines that the water surface is a weed and enters a preset weed control mode.

5. A configuration capable of detecting changes between a reference image and a newly captured image, The input data is a difference determination between the reference image and the newly captured image, the driving data of the work vehicle at the time of the image capture, and the map data of the foreign object detection position. When an abnormality is detected or a setting change is made manually by controlling the work vehicle based on the output results of the neural network, 2. The work vehicle according to claim 1, further comprising a function for adding the data on abnormality detection and data on manual setting changes, and for learning the correlation between the input and output of each of the data.

Images