Work vehicle

Abstract

To solve the problems that in a conventional technique, images photographed from a fixed position in the same condition are used when performing difference analysis by comparing images, but it is difficult to compare images photographed from an unstable position like a work vehicle.SOLUTION: In detection of a difference between reference images and newly photographed images, photographing is performed with the same work vehicle in the same places in a field toward the same directions, and further, photographing is performed so as to make a part of the work vehicle get in images of an imaging device, and correction processing of overlapping reference parts of the work vehicle getting in the images, and matching a scale for a size and inclination of the work vehicle with a tone of color of the work vehicle is performed so as to detect a difference between the reference images and the newly photographed images.SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] The present invention relates to a work vehicle that uses an imaging device to analyze the difference between a reference image and a new image, and operates in a pre-registered work mode or takes safety measures. [Background technology]

[0002] In the prior art, there is a system that performs image matching of pest footprints and manages information related to the images. (Patent Document 1) [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7322775 Summary of the Invention [Problem to be solved by the invention]

[0004] In the conventional technology, the accuracy of matching is high because the main focus is on comparing and matching the captured data. In addition, in order to match image data, it is necessary to regulate the imaging conditions for acquiring the image, but this is not described in detail.

[0005] The objective of the present invention is to provide a work vehicle that can analyze the correlation between various types of data by restricting imaging conditions for comparison and utilizing an artificial intelligence matching function to collate data that cannot be simply compared, thereby avoiding abnormalities and detecting malfunctions. [Means for solving the problem]

[0006] The first aspect of the present invention is achieved by the following technical means.

[0007] In detecting the difference between the reference image and a newly captured image, images are taken with the same work vehicle at the same location in the field in the same direction, and further, images are taken so that part of the work vehicle is included in the image of the imaging device, and a correction process is performed to superimpose the reference part of the work vehicle included in the image and to match the scale and inclination of the size of the work vehicle and the color of the work vehicle, thereby making it possible to detect the difference between the reference image and the newly captured image.

[0008] The second aspect of the present invention is achieved by the following technical means.

[0009] When detecting a change between a reference image and a newly captured image, if there is a change larger than a predetermined range, the part where the change occurred is set as a specific part, the work vehicle is stopped, and images are taken continuously. If movement is confirmed in the specific part, it is determined to be a moving object, and a preset moving object avoidance mode is entered.

[0010] The third aspect of the present invention is achieved by the following technical means.

[0011] When detecting changes between a reference image and a newly captured image, the camera is set to capture an image of the growing area of ​​the crop, and if a change in color is confirmed within this area, the camera enters a pre-set pest response mode.

[0012] The fourth aspect of the present invention is achieved by the following technical means.

[0013] When detecting changes between the reference image and a newly captured image, the system is set to capture the water surface of a rice paddy, and if a change in color or shape is confirmed within this range, the system enters a pre-set weed control mode.

[0014] The fifth aspect of the present invention is achieved by the following technical means.

[0015] In detecting changes between a reference image and a newly captured image, the input data is a determination of the differences between the reference image and the new image, the driving data of the work vehicle at the time of capture, and map data of the location of the foreign object detection. If an abnormality is detected or manual settings are changed when controlling the work vehicle based on the output results from the neural network, the system has the function of learning the correlation between the input and output of each piece of data from the changed data. Effect of the Invention

[0016] According to the first aspect of the present invention, the captured images are taken under conditions that facilitate matching, and there is a standard for confirming the match in subsequent image data matching, so that changed areas can be extracted with high accuracy.

[0017] The second invention enables the vehicle to automatically respond when there is a moving object.

[0018] The third invention makes it possible to automatically respond to pest infestations and the like by distinguishing color tones.

[0019] The fourth invention makes it possible to automatically control weeds in rice paddies.

[0020] According to the fifth invention, accuracy is improved by utilizing artificial intelligence and having it learn. [Brief description of the drawings]

[0021] [Figure 1] FIG. 1 is a diagram showing a reference image during tilling work according to the present invention. [Diagram 2] A diagram showing the new image of the present invention during tillage work when the imaging scale is different. [Diagram 3] A diagram showing a moving object being confirmed in a new image taken during cultivation work according to the present invention. [Figure 4] A new image of the present invention during tillage work, showing different field conditions. [Diagram 5] A diagram showing a case where an obstacle is identified in a new image during cultivation work according to the present invention. [Figure 6] FIG. 1 is a diagram showing a reference image during pest control according to the present invention. [Figure 7]A new image of the present invention showing damage to crops caused by pests. [Figure 8] Image processing analysis of the present invention when the insect is eaten [Figure 9] FIG. 1 is a diagram showing a reference image for weeding work after rice planting according to the present invention. [Figure 10] A new image of weeding work after rice planting according to the present invention, showing when weeds are confirmed [Figure 11] A diagram showing how the amount of movement of a moving object is calculated using a neural network in a reference image and a new image during cultivation work according to the present invention. [Figure 12] A diagram showing a reference image and a new image during tilling work according to the present invention, in which the scale and inclination are corrected based on the work vehicle in order to match the images, and the matching is performed using a neural network. [Figure 13] A diagram showing the configuration of a dedicated machine for handling moving objects according to the present invention. [Figure 14] Equipment configuration diagram when using the moving object handling dedicated machine of the present invention [Figure 15] Configuration diagram of the pest control machine of the present invention [Figure 16] A diagram showing the configuration of the pest control machine of the present invention during pest control work. [Figure 17] Configuration diagram of the weed-killing machine of the present invention [Figure 18] Operation diagram of the weed-killing machine of the present invention when controlled during manual operation DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] The present invention will now be described with reference to the embodiments shown in the drawings.

[0023] The work vehicle shown in Figs. 1 to 18 shows an example of this embodiment.

[0024] The work vehicle of the present invention will now be described.

[0025] In this invention, there are tractors, rice transplanters, combine harvesters, and cultivators. Each work machine has its own characteristics in the way it moves, but here, a tractor will be used as a representative model. Each work vehicle is capable of automatic driving, and is equipped with a satellite positioning device and an inertial positioning device to enable unmanned driving, and driving steering and work machine operation are performed based on registered data. The reference driving route is based on the farm field, and is registered and managed using map data.

[0026] The background of the present invention will now be described.

[0027] As automation in agriculture progresses, it would be ideal if work vehicles could perform tasks automatically. However, even farm work does not simply mean repeating the same tasks. In vast fields, various crops are cultivated, and there are seasonal weather differences, so the reproducibility is not as high as in industrial production sites. In particular, although the fields are privately managed land, they are not inside buildings that are isolated from the outside, so moving objects can enter and exit freely. Creatures that are considered pests may enter fields because crops are convenient food. Similarly, humans may be interested in automated work vehicles and approach them.

[0028] There are ways to deal with moving objects during such work, which can become a risk factor, and these are items to be used as safe driving measures for autonomous driving. Furthermore, autonomous driving allows for driving during times when humans cannot keep up, long-term work, and repeated driving. If this function and the image capture device can be used to analyze, it would also be possible to continuously eradicate pests and vermin. Furthermore, in weed management for rice paddies, removing weeds and weeding work can be done at the same time, which would save labor.

[0029] The first aspect of the present invention will be described.

[0030] 1 is a diagram of a reference image during tillage work. The work vehicle is a tractor 10.

[0031] The entire screen of the imaging device is captured so that it is in the state shown in FIG. 1. The first important point is that when the image of the reference image G1 is registered, the latitude and longitude of the image must be registered and the direction of the image must be clearly defined. The direction of the imaging device is determined by a digital camera equipped with an infrared sensor or other device to measure the distance to the subject. The direction of the imaging device may be the direction in which the lens of the imaging device is focused, but the direction of the imaging device to the subject is superimposed on the latitude and longitude of the imaging position. The imaging device also has an angle in the height direction with the ground. The direction is registered by registering three directions, including this angle, the latitude and longitude of the imaging direction from the registration position of the imaging device to the subject, and the imaging angle of the imaging device from the ground. This determines the standard in the imaging device for comparing the difference between an image captured by the same work vehicle at the same place in the field in the same direction as the reference image and a newly captured image captured by the same work vehicle at the same place in the field in the same direction.

[0032] By carrying out such work, the newly captured image can be compared. In general image comparison, the image capture device and the subject are placed on a fixed jig, and the target points are matched and captured. This method of placing the subject in a predetermined position as a condition for comparison is easily handled in image comparison, but since a moving object entering the field does not move to a predetermined position by command, the image capture side must capture an image to be matched with the reference image.

[0033] However, even if the conditions are known, this alone does not make it easy to compare the images G1. The reason is that the imaging device is mounted on a moving work vehicle such as the tractor 10, and the tractor 10 travels in a field, not like an automobile travels on an asphalt road. Due to the unevenness and softness of the field, the work vehicle will tilt forward, backward, left and right. Since this tilt occurs three-dimensionally, even if the images are taken in the same place in the field and in the same direction by the same work vehicle as described above, when the three-dimensional tilt is corrected, the scale of the images may differ slightly. If there is a difference in size, the same object may be judged to have a different shape. Therefore, the scale of this image needs to be corrected. Since tilt also occurs, tilt correction is also required.

[0034] Furthermore, the present invention also performs matching by color. Even if the shape is the same, differences in color mean changes. For plants, green and brown indicate different growth. The RGB color, which is the color of the color, is corrected to match the color of the work vehicle. If the captured image is taken at a different time or season, the angle of the sunlight, etc., may cause the color of the same work vehicle to appear different. Therefore, since the work vehicles captured in the reference image and the new image are the same, if the color difference is corrected so that the color of the work vehicle in the new image is the same as the color of the work vehicle in the reference image, this correction of color can be used for the entire image to compare the color of other parts of the field, which makes it possible to compare shapes and colors.

[0035] In regards to shadows that occur depending on the angle of sunlight, a filtering effect is applied using artificial intelligence to eliminate shadows from the image, but the angle of the sunlight is also calculated based on the date and time the image was taken, and the position of the shadow is moved and the color is corrected to match the angle of the sunlight at the date and time of the reference image.

[0036] In addition, reference numbers 20 to 57 in Figure 1 are rice stubbles left in the field after harvesting with a combine harvester. By matching the color of the work vehicle, it is possible to confirm the change in color of the rice stubbles. The reference image is taken immediately after rice harvesting, and the new image is taken immediately after land leveling work. If the images were taken at different times, it is possible to distinguish the difference between the images by color even if the shapes of the rice stubbles are in a similar shape range.

[0037] Figure 2 is a new image taken during tillage work. As mentioned above, when the three-dimensional tilt is corrected, the scale and tilt of the image are slightly different. The difference between image G1 in Figure 1 and image G2 in Figure 2 shows the difficulty of comparing the captured images.

[0038] Thus, it is not easy to compare size with images. For example, suppose LIDAR (Light Detection and Ranging) is deployed. LIDAR is a type of remote sensing technology that uses light, and it is possible to analyze the shape and properties of an object by measuring the scattered light from a pulsed laser irradiation. It also makes it possible to measure the distance to an object, and although this is effective for individual data analysis, when comparing captured image data, the images are calculated as dots or polygon shapes, and it is difficult to judge shapes that are slightly rounded by numerical comparison. Rather, it is more accurate to simply compare the image data and analyze the shape of a certain area using a convolution neural network.

[0039] Therefore, work must be done to align the scale and angle of the images, but finding this reference point poses a problem. For a work vehicle that is constantly moving within a field, it is difficult to find the reference point within the field.

[0040] Therefore, in the present invention, an image of a part of the work vehicle is captured by the imaging device. Since the work vehicles are the same, for example, if an imaging device is installed inside the cabin of the work vehicle, it is possible to compare the three-dimensional distance from the width and length of the bonnet of the work vehicle, the width and position of the front wheels, or the size of the window frame of the cabin (not shown). However, since it is difficult to match completely without difference in all directions, a tolerance is set in each direction, and the difference between them is processed as a match, allowing for immediate response.

[0041] By overlaying a portion of the work vehicle that has entered the image and performing a process to align the scale and inclination of the work vehicle's size, it is possible to detect changes between the reference image and the newly captured image, making it possible to compare the reference image with the new image. Differences between the images are judged to be a different situation in unmanned operation of a robotic work vehicle, etc., making it possible to respond in accordance with the comparison standard that the condition is different from usual based on human experience.

[0042] The relationship in which the process of matching the scale and inclination of the size of this work vehicle is performed is explained in Figure 12. Image G1 in Figure 1 is the reference image, and image G2 in Figure 2 is a newly captured image. As explained above, it can be seen that image G2 has a different scale and inclination compared to image G1. If the images were simply synthesized and compared as they are, the size of the field, the size of the rice stubbles after harvesting, and the pitch of the stubbles would be different, and it would be detected that there are different parts in each part.

[0043] Therefore, as described above, a reference for correcting the scale and inclination is calculated by capturing part of the work vehicle in the image. Image 91 is a diagram in which the tractor 10 (solid line) in image G1 and the tractor 10A (two-dot chain line) in image 2 are superimposed. Since the tractor 10 and the tractor 10A are the same, by detecting this difference, the difference in the scale and inclination of the images is detected.

[0044] First, the reference part of the tractor 10 must be registered. In this embodiment, the meter panel position 92 of the tractor 10 is used as the reference part. By matching this image 91, the two are compared with each other, centering on the outline of the image shape. The image is expressed as a collection of dots of a certain size, and a convolutional neural network is used to calculate the amount of movement required to compare the deviation of the outline and to perform calculations to compare the colors of each dot.

[0045] As a result, in the image 91, it is detected that the images of the tractor 10 and the tractor 10A show that the tractor 10A is tilted 3 degrees to the left, and the scale is 80% (560 mm / 700 mm).

[0046] Explaining this with reference to the comparison flow chart 80, if there is a difference in the work vehicles in S80, then the inclination comparison is performed in S82. After that, the scale is compared in S84, and the image comparison in S86 is started.

[0047] Through this image processing, image G2 is corrected so that it is tilted 3 degrees to the right, and is corrected to image G2A. Image G2B is then enlarged by 125% (700 mm / 560 mm), and by comparing image G1 with image G2B, any changes between the reference image and the newly captured image can be detected.

[0048] By matching images in this way, it becomes possible to detect changes in the field. As an example, the differences will be explained using image G1 in Figure 1 and image G2 in Figure 2. In image G2 in Figure 2, it can be seen that the rice stubbles at 22A, 36A, 44A, and 45A have disappeared. In this way, the difference in the field can be detected. However, since the rice stubbles do not pose a problem for work, if the shape of the rice stubbles is set to be excluded from detection in advance, it is possible to prevent them from being detected using the filtering function.

[0049] If artificial intelligence is used for such image filtering functions and specific image detection, it is possible to detect similar shapes quickly and with high accuracy. Data 87 in FIG. 12 is obtained by decomposing the detected image from image G2B into areas, and inputting information on various objects based on color and image contours. Data 88 weights the relevance of data 87, data 89 assumes an object that can be inferred, and data 90 outputs the specified object. In other words, by capturing an image or using other measurement data (input data), it is possible to identify the object captured in the image (output data). It is easy for a human being to infer the shape of an object from memory from a captured image and determine the relevant object, but this is difficult for a machine, so artificial intelligence is used to identify the object.

[0050] The output of the artificial intelligence is used not only to identify objects, but also to determine differences in images. When comparing images, it is possible to identify objects in areas where changes have occurred and analyze the causes.

[0051] The second aspect of the present invention will now be described.

[0052] When detecting changes between the reference image described above and a newly captured image, if there is a change greater than a specified range, the area where the change occurred is set as a specific area, the work vehicle is stopped, and images are taken continuously.If movement is confirmed in the specific area, it is determined to be a moving object, and a pre-set moving object avoidance mode is entered.

[0053] First, even if it is a moving object, if it is palm-sized, it is a small bird or a small animal, and compared to the size of the work vehicle, it may run away when the work vehicle approaches, so it can be excluded as a moving object. However, if it is larger than the size of both hands, it becomes the size of a dog, cat, wild boar, weasel, etc. Furthermore, if it is the size of a human child, it is serious, and if there is a change in size beyond a specified range, the part where the change occurred is designated as a specific part, and images are continuously taken and analyzed.

[0054] Image G3 in Fig. 3 is a newly captured image in which an object 60 is detected that is large enough to be determined as an obstacle. The reference image for this image is image G1 in Fig. 1. Here, the scale and inclination of the image are the same, and correction will not be mentioned, but the same operations as in the first invention are performed.

[0055] Fig. 11 explains how to analyze whether an object is a moving object. This figure shows that the amount of movement of a moving object is calculated using a neural network in a reference image and a new image during tillage work.

[0056] The image marked G1 is the reference image during tillage work. Image G3 is the image captured this time. It can be seen that object 60 is different in image G3 compared to image G1. In this case, the work vehicle makes an emergency stop. The emergency stop is performed to increase the accuracy of the image capture and, above all, to prevent a moving object from approaching and causing damage.

[0057] Image G3A is an image captured after an emergency stop, and it can be seen that object 60 is detected as moving as object 61, and that its shape is also different. This allows object 60 to be recognized as a moving object. Furthermore, in image G3B, it is determined that object 62 is approaching even closer to the work vehicle.

[0058] In this way, when the movement of an object larger than a predetermined size is confirmed, the moving object avoidance mode is entered. In the moving object avoidance mode, not only is the work vehicle stopped from traveling, but the work machine itself also stops working. In the case of a robotic work machine, the operator's seat is locked. Furthermore, sound may be used to intimidate or a warning may be given verbally. A lighting device may also be used to flash to indicate danger. Data communication is also performed, and the captured image is sent to the user to check for any misrecognition. The decision to resume operation is made at the user's instruction.

[0059] In this way, if the moving object is a living thing, the moving object avoidance mode may be able to avoid a dangerous situation. In the embodiment, the case of a human is described, and it is confirmed that the human turns its back on the work vehicle and moves away, as in image G3C.

[0060] It is possible to determine that object 60 in image G3, object 61 in image G3A, object 62 in image G3B, and object 63 in image G3C are the same object using a neural network. Data 70 compares the data of captured images G1 to G3C and the data held by the work vehicle with registered data. Data input and output are the same as those explained in FIG. 12.

[0061] Data 70 in FIG. 11 is obtained by decomposing the images G1 to G3C into areas, and inputting information on various objects based on color and image contours. Data 71 weights the relevance of data 70, data 72 assumes an object that can be inferred, and data 73 outputs the specified object. In other words, by capturing an image and using other measurement data (input data), it is possible to identify the object captured in the image (output data). It is easy for a human being to infer the shape of an object from memory from a captured image and determine the relevant object, but this is difficult for a machine, so artificial intelligence is used to make the identification.

[0062] The output of the artificial intelligence is used not only to identify objects, but also to determine differences in images. When comparing images, it is possible to identify objects in areas where changes have occurred and analyze the causes.

[0063] In this case, the object was determined to be a moving object, but in the analysis method, once a similarity match of a humanoid object has been made, position detection is performed using face position 60A as a point. When the images are combined, it is confirmed that the face position moves from face position 60A ⇒ face position 61A ⇒ face position 62A ⇒ face position 63A. If a change in shape and movement of the image outline is confirmed by the variation in the distance per unit time and change in the direction of movement, and by combining images 60, 61, 62, and 63, it is determined that the object is a living thing and is highly likely to be human.

[0064] Even if the image is not determined to be the contour of a human, a detection criterion for a moving object is set, and if the part shows a movement line such as face position 60A ⇒ face position 61A ⇒ face position 62A ⇒ face position 63A, the movement line of this reference position is confirmed and a moving object is determined.

[0065] The movement of this line of movement is also used to determine whether it is a moving object or a living thing other than a human. Anything with an irregular amount of movement per unit time or direction of change is determined to be a living thing. In particular, if there is a change in the vertical direction of the reference line of movement, it can be determined that it is not a moving object such as a work vehicle. In addition, by checking the change in movement when a sound is emitted from a work vehicle to warn, it is possible to determine whether an object is a human or other living thing. For example, if a work vehicle emits a voice saying, "It is dangerous. Please move away," it is assumed that if it is a human, there will be a change in behavior, and this can be used as the basis for judgment.

[0066] Such image analysis and judgment is also performed from the input of data 70 to the output of data 73. In other words, the phenomenon of a person approaching a work vehicle in a farm field can be judged from the captured image data.

[0067] Although the flow for entering the moving object avoidance mode is simply described in S74 to S78, the moving object avoidance mode S78 is entered depending on whether a different part is moving in S76. In the case of image G4 in Fig. 4, the moving object avoidance mode is not entered, and detection of a part different from the normal operation mode S77 is selected.

[0068] An example of this is shown in image G4. Compared to the reference image G1, rice stumps 20C to 44C were matched, but no movement of objects was confirmed. Also, this corresponds to the case where rice stumps 50C to 54C on the left side are not detected.

[0069] As another example, image G5 is shown. Here, object 80 is detected as a different part. This time, it is the case that the number of objects has increased. In this case, the type of object 80 is analyzed through a neural network. Since information about foreign objects found in fields has been accumulated through a lot of experience, comparison images are generally available. Image data of stones, cardboard boxes, plastic boxes, umbrellas, nylon bags, wrapping paper, magazines, newspapers, tools, etc. are registered and are compared based on similar shapes and colors. Data on living creatures such as humans, dogs, cats, weasels, and mice is also registered.

[0070] However, foreign objects are classified according to the type of foreign object, and are divided into cases where the foreign object is interfering with work and driving is not possible unless the foreign object is removed, cases where conditional driving is possible, cases where normal driving is possible, and cases where the vehicle enters the moving object avoidance mode S78 described above.

[0071] The criteria for classifying such foreign objects must be improved through a learning function. To achieve this, the system has a function to learn correlations by using image synthesis processing, work vehicle data, and map data as input data when detecting changes between a reference image and a newly captured image, and by using a neural network to output data on changed parts of the image as a method for controlling future autonomous driving.

[0072] The fifth aspect of the present invention will now be described.

[0073] This is an example of an embodiment of the present invention. The input data 70 of the neural network includes the reference image G1, image data at the time of foreign object detection (images of G3 to G3C), driving data of the work vehicle at the time of detection (each setting of the tractor 10 when G3 to G3C are imaged and detection data), and map data of the foreign object detection position (position information of the moving object 60 when G3 to G3C are imaged), and the output data 73 is used to determine whether the moving object avoidance mode S78 responded correctly, and is evaluated based on whether there were any abnormalities after the content of the automatic driving response, or whether there were any elements that were manually changed by a human being.

[0074] In this way, the process of detecting anomalies or making manual changes is judged to be an error or poor accuracy in the criteria for the AI's judgment, and the changes must be learned. These anomaly detections and manual changes are added as change elements to the input data 70. By repeating this process, the correlation is further increased, making it possible to calculate the best output conditions for the field environment.

[0075] The learning function portion that can be given as an example is the classification standard for foreign objects described above. This is improved by the learning function, and classification is performed according to the method of subsequent driving response, such as when it is determined that the foreign object is interfering with work and driving is not possible unless the foreign object is removed, when it is determined that conditional driving is possible, when it is determined that normal driving is possible, and when the above-mentioned moving object avoidance mode S78 is entered, and the standard value is changed.

[0076] For example, the criteria for determining that a foreign object must be removed before driving can be improved through learning. If a captured image indicates that the object is a stone, but it is actually a black lump of soil, there is no problem with driving even if the object is not removed, so the vehicle will learn to distinguish between images of stones and soil lumps. In this case, the unevenness of the surface, the difference in color with the surrounding area, the distance from the ridge, etc. are used by the neural network to change the weighting of color discrimination and change the accuracy of distinguishing between stones and soil lumps, and are used in the function to learn the criteria for classifying foreign objects.

[0077] The third aspect of the present invention will now be described.

[0078] When detecting changes between a reference image and a newly captured image, the system is set to capture an image of the growing area of ​​the crop, and if a change in color is confirmed within this area, the system enters a pre-set pest response mode.

[0079] Image G6 in Fig. 6 is a reference image, and image G7 in Fig. 7 is a newly captured image. In comparison, changes are seen in plant 102B, plant 103C, plant 111A, plant 112B, and plant 113C in image G7. These changes are mainly due to the detection of differences in color rather than changes in shape.

[0080] When the plant 111 becomes the plant 111A, the area 120 and area 121 where the change was detected are enlarged and analyzed, and a pest is found, and the color is also different. When such a detection is made, the corresponding area is disinfected. Although disinfection is performed in the pest response mode, the location of the corresponding plant is registered because pest damage has occurred, and the location is registered on the map so that the plant is excluded when harvesting crops.

[0081] The fourth aspect of the present invention will now be described.

[0082] When detecting changes between the reference image and a newly captured image, the system is set to capture the water surface of a rice paddy, and if a change in color or shape is confirmed within this range, the system will enter a pre-set weed control mode.

[0083] Image 9 in Fig. 9 is the reference image, and image G10 in Fig. 10 is the new image. Both images were taken after rice planting and the time when the paddy fields were being managed. This is the time when weeds start to appear in paddy fields. Weeds include Monochoria vaginalis, Kuroguai, and Japanese barnyard grass. If left unattended, weeds 161-174 will spread all over the waterway between rice plants 141A-154A as shown in Image 10. This image shows a significant amount of weeds growing, but if weeds are detected by image comparison, we will immediately begin to capture the weeds and muddy the water in the paddy field to block sunlight from reaching the field surface in the areas between the rice plants, thereby preventing weeds from growing. This response is the weed response mode.

[0084] The focus of image judgment in the third and fourth inventions is color judgment. The reference image and the new image are superimposed, and the focus is on detecting whether there are any differences in color tone in specific areas. Color tone is judged by the degree of the three primary colors. Reference colors for each position are registered in the neural network, and if the area difference between different colors from these colors exceeds a predetermined value, an abnormality is detected. However, in the early stages of both image G7 and image G10, the changes are small and judgment is difficult. Therefore, it is advisable to use a neural network to comprehensively compare and judge the changes in shape and color.

[0085] The second, third and fourth aspects of the present invention relate to ways of responding when an abnormality is detected in an image, and more accurate responses are possible by launching a dedicated work vehicle in addition to the tractor 10, which is the representative type of the present invention.

[0086] The moving object avoidance mode of the second invention will be described with reference to FIG. 13. The moving object-compatible dedicated machine 200 is a dedicated machine used in the moving object avoidance mode. It enables automatic travel with a satellite positioning device 204. It is equipped with a water gun 207, which shoots water like a bullet to target the moving object and check the situation. If it is an animal, it is considered to retreat, and this is the main response of the moving object avoidance mode. The imaging device 266 is a LIDAR and a CCD camera, and detects the shape and distance from the captured image to estimate the moving object. Since it is equipped with crawlers 209, it can travel on some uneven terrain. By rotating the head 210, the light 205 is rotated and emitted to intimidate moving objects and also serves as a warning signal to nearby residents. The figures are a front view 201, a top view 203, and a side view 202.

[0087] In addition, this system can be used in a different manner as shown in FIG. 14(B), in which an imaging device 211 is provided in the corner of a farm field, and when an intrusion of a moving object 220 into the farm field 212 is confirmed, a dedicated machine 200 for handling moving objects is dispatched from a warehouse 213.

[0088] 14(A) uses a drone 230 instead of the moving object handling dedicated machine 200. Similarly, when an intrusion of a moving object 220 into a farm field 212 is confirmed, the drone 230 is dispatched from a warehouse / charging device 231. The drone 230 is a system that emits sound to scare away the moving object 220.

[0089] The pest control mode of the third invention will be described with reference to Fig. 15. A top view 251 of the pest control dedicated machine 250 is shown during spraying, while a front view 252 is shown during charging. A connection state with a charger 254 is shown in Fig. 253.

[0090] The pest control dedicated machine 250 is capable of automatic travel using a satellite positioning device 262. The travel section can travel using wheels 260 and crawlers 259, the main body is equipped with a disinfectant tank 257, and a charging device and battery section 258 are placed below. A spraying device 255 is provided above, which shoots out a mist of disinfectant and sprays it on the part of the plant 261 where the pests have infested. It is linked to images captured by the tractor 10, and the position of the plant where the pests have infested is confirmed by latitude and longitude, and the height above ground is also confirmed from the image, the target position is confirmed, and the amount of spraying is increased at this position.

[0091] It is also possible to provide an ultraviolet lamp 256 and irradiate it at the same time as spraying the disinfectant. The ultraviolet lamp is used to determine if eggs have been laid in the presence of pests, and to prevent hatching and to remove the eggs.

[0092] The pest control dedicated machine 270 in Fig. 16 is designed for crops 275 that are tall plants. In addition to disinfecting, it also sprays water, and is equipped with piping 271 and nozzles 272 for use at high altitudes. The tank 273 is installed at a slightly higher altitude, and is slightly pressurized to the pump pressure for spraying. It is equipped with a satellite positioning device 280 and is capable of automatic operation. The traveling section is the same as that of the pest control dedicated machine 250.

[0093] 16(A) is a diagram during spraying, and FIG. 16(B) is a diagram during charging. Charging section 276 is non-contact connected to power supply section 278 for charging. At this time, disinfectant 277 is supplied.

[0094] The weeding mode of the fourth invention will be described using the weeding dedicated machine 300 in Fig. 17. The diagrams are a front view 303, a top view 301, and a side view 302. The weeding dedicated machine 300 is capable of automatic travel using a satellite positioning device 307. It is possible to weed by mounting a dedicated weeding machine on the tractor 130, but when the tractor 130 detects weeds in a paddy field, it is more effective to detect their position from an image, transmit the positional relationship data to the work vehicle 300, and have the weeding dedicated machine carry out the weeding.

[0095] This weed-removal dedicated machine 300 is equipped with a small chain 304 between front wheels 305 and rear wheels 306, and as it moves, it drags the small chain 304 along the muddy layer at the top of the rice paddy, stirring up mud. In addition to stirring up mud, it can also remove already grown weeds by entangling the grass in the paddy field with the chain and tearing it off.

[0096] A number of small chains 304 are arranged in a row, and no chains are placed where the rice grows, so the rice is protected and only the weeds are entangled. The front wheels 305 have a larger diameter, so that the chains do not get tangled when turning.

[0097] Working in this way in conjunction with data from the weed control dedicated machine 300 is effective and can be done as part of the weed control mode.

[0098] 18A shows a state in which the tractor 130 has entered a weeding mode for the entire field, regardless of whether it has detected weeds. The tractor 130 takes a travel route for the entire field 308, and automatically travels to weed the field.

[0099] Fig. 18(B) shows a manual operation using the external controller 305. The tractor 130 detects the position of weeds, but when weeding is performed at that position, weeds such as floating weeds may drift into a position where there are no weeds. To deal with this, the imaging device 310 is equipped with an imaging device, and the captured image is sent to the external controller 305, allowing weeding to be performed while checking the image. [Explanation of symbols]

[0100] 10 Tractor 20 Rice stubble (reference image) 20A Rice stump (new image) 60 Objects (Human) 70 Data (input, neural network) 73 Data (output, neural network) 91 images (composite of reference and new) 111 Plants (reference image) 111A Plants (new image) 120 Pests 161 Weeds 200 Dedicated machine for moving objects 250 Pest control machine 300 Weed-controlling machine

Claims

1. In detecting the difference between a reference image and a newly captured image, The same work vehicle captures images of the same location in the field and in the same direction. Furthermore, an image is captured so that a part of the work vehicle is included in the image of the imaging device, The reference part of the work vehicle that entered the image is superimposed, A work vehicle that is capable of detecting differences between a reference image and a newly captured image by performing a correction process that matches the scale and inclination of the size of the work vehicle and the color of the work vehicle.

2. In detecting changes between a reference image and a newly captured image, 2. The work vehicle of claim 1, wherein if there is a change of a magnitude greater than a predetermined range, the part where the change occurred is set as a specific part, the work vehicle is stopped, and continuous image capture is performed, and if movement is confirmed in the specific part, it is determined to be a moving object, and a preset moving object avoidance mode is entered.

3. In detecting changes between a reference image and a newly captured image, Set it to capture the growing part of the crop, and if a change in color is confirmed in this range, The work vehicle of claim 1 , which enters a preset pest response mode.

4. In detecting changes between a reference image and a newly captured image, 2. The work vehicle of claim 1, which is set to capture images of the water surface of a paddy field, and when a change in color or shape is confirmed within this range, the work vehicle goes into a preset weed-killing mode.

5. In detecting changes between a reference image and a newly captured image, The input data is the difference between the reference image and the new image, the driving data of the work vehicle at the time of the image capture, and the map data of the foreign object detection position. When an abnormality is detected or the settings are changed manually by controlling the work vehicle based on the output results of the neural network, 2. The work vehicle according to claim 1, further comprising a function for learning correlations between inputs and outputs of each piece of data from the changed data.

Images