Agricultural assistance system and method for detecting crop rows

Abstract

This agriculture assistance system comprises a processing device that processes a top view image of a field including a plurality of crop rows to detect the plurality of crop rows. The processing device is configured to: generate a planted portion image by extracting, from an image, a planted portion that corresponds to the plurality of crop rows; aggregate the pixel values of a group of pixels forming the planted portion image along N-number (N is an integer of at least two) of aggregate lines of the planted portion image to obtain N-number of aggregate values related to the N-number of aggregate lines; apply a filter to a one-dimensional array storing the N-number of aggregate values to perform a convolution operation, thereby obtaining N-number of weighted aggregate values; and detect the plurality of crop rows on the basis of the N-number of weighted aggregate values.

Description

Agricultural support system and crop row detection method

[0001] The present disclosure relates to an agricultural support system and a method for detecting crop rows.

[0002] Research and development is underway on smart agriculture, which utilizes ICT (Information and Communication Technology) and IoT (Internet of Things), as the next generation of agriculture. Research and development is underway on agricultural support systems that centrally manage information about farm fields on the cloud and use the data on the cloud to support farming. By using agricultural support systems, it becomes possible to efficiently carry out all the tasks required for farming, such as pest control, tilling, sowing, fertilizing, planting crops, and harvesting.

[0003] Various methods have also been proposed for detecting crop rows by processing images that include the rows of crops in a field. Patent Document 1 discloses a control method for moving a farm implement along the rows of ridges in a field where crops are planted. By processing an image that includes the field, the angle between the direction in which the ridges extend and the direction in which the farm implement travels, or the amount of positional deviation of the farm implement from the center of the ridge, is calculated.

[0004] Patent No. 6418604

[0005] Due to environmental changes such as the type of crop or the growing season, crop rows may not be detected properly. Therefore, it is desirable to improve the accuracy and robustness of crop row detection.

[0006] An agricultural support system according to one aspect of the present disclosure includes a processing device that processes an image of a field including multiple crop rows viewed from above to detect the multiple crop rows. The processing device is configured to generate a vegetation canopy image by extracting the vegetation canopy, which is the multiple crop rows, from the image, integrate pixel values ​​of groups of pixels that make up the vegetation canopy image along N integration lines (N is an integer greater than or equal to 2) of the vegetation canopy image to obtain N integrated values ​​related to the N integration lines, apply a filter to a one-dimensional array storing the N integrated values ​​and perform a convolution operation to obtain N weighted integrated values, and detect the multiple crop rows based on the N weighted integrated values.

[0007] A method according to one aspect of the present disclosure is a computer-implemented method for detecting multiple crop rows in a field, which causes a computer to perform the following steps: acquire an image of the field viewed from above; generate a vegetation canopy image by extracting the vegetation canopy, which is the multiple crop rows, from the image; accumulate pixel values ​​of groups of pixels that make up the vegetation canopy image along N accumulation lines (N is an integer greater than or equal to 2) of the vegetation canopy image to obtain N accumulation values ​​related to the N accumulation lines; apply a filter to a one-dimensional array storing the N accumulation values ​​and perform a convolution operation to obtain N weighted accumulation values; and detect the multiple crop rows based on the N weighted accumulation values.

[0008] A general or specific aspect of the present disclosure may be realized by an apparatus, a system, a method, an integrated circuit, a computer program, or a computer-readable non-transitory storage medium, or any combination thereof. The computer-readable storage medium may include a volatile storage medium or a non-volatile storage medium. An apparatus may be composed of multiple devices. When an apparatus is composed of two or more devices, the two or more devices may be located in a single device or may be located separately in two or more separate devices.

[0009] Embodiments of the present disclosure enable improved accuracy and robustness of crop row detection.

[0010] 1 is a schematic diagram showing an example configuration of an agricultural support system according to an embodiment of the present disclosure; FIG. 2 is a block diagram showing an example hardware configuration of an agricultural support system according to an embodiment of the present disclosure; FIG. 3 is a flowchart showing a processing procedure according to an implementation example of an algorithm for detecting crop rows; FIG. 4 is a diagram showing an example input image including rice paddy rows; FIG. 5 is a diagram showing an example vegetation canopy image including vegetation canopies that are rice paddy rows; FIG. 6 is a diagram showing an example input image before preprocessing is applied; FIG. 7 is a diagram showing an example binary image after preprocessing is applied; FIG. 8 is a diagram showing another example input image before preprocessing is applied; FIG. 9 is a diagram showing another example binary image after preprocessing is applied; FIG. 10 is a flowchart showing an example of specific processing procedures included in step S30; FIG. 11 is a diagram showing an example of a rotated vegetation canopy image obtained by rotating a vegetation canopy image by a first rotation angle around the origin of a two-dimensional coordinate system; FIG. 12 is a diagram showing an example of a rotated vegetation canopy image obtained by rotating a vegetation canopy image by a second rotation angle around the origin of the two-dimensional coordinate system; FIG. 13 is a diagram showing an example of a rotated vegetation canopy image obtained by rotating a vegetation canopy image by a third rotation angle around the origin of the two-dimensional coordinate system; and FIG. 14 is a diagram showing an example of a rotated vegetation canopy image obtained by rotating a vegetation canopy image by a fourth rotation angle around the origin of the two-dimensional coordinate system. 8B. FIG. 8C is a schematic diagram showing an example of a frequency spectrum image generated by applying DFT to a vegetation canopy image. FIG. 8D is a schematic diagram showing a plurality of integrated lines of the vegetation canopy image or the rotated vegetation canopy image shown in FIG. 8A. FIG. 8E is a schematic diagram showing a plurality of integrated lines of the rotated vegetation canopy image shown in FIG. 8B. FIG. 8F is a flowchart showing an example of a specific processing procedure included in step S40. FIG. 8G is a schematic diagram for explaining a convolution operation. FIG. 8H is a schematic diagram showing an overview of a list obtained by sorting N weighted integrated values ​​of each rotated vegetation canopy image included in a set of weighted integrated values ​​in descending order. FIG. 8H is a flowchart showing an example of a specific processing procedure included in step S30 according to another implementation example. FIG. 8I is a flowchart showing an example of a specific processing procedure included in step S40 according to another implementation example. FIG. 8I is a flowchart showing a processing procedure according to an implementation example of an algorithm for transmitting a control instruction to an agricultural machine when weeds are detected. FIG. 8I is an enlarged view of a portion of a vegetation canopy image. FIG. 8I is a diagram showing an example of a detection result image in which the detection results of crop rows and weeds are shown.

[0011] Hereinafter, embodiments of the present disclosure will be described. However, more detailed descriptions than necessary may be omitted. For example, detailed descriptions of already well-known matters and redundant descriptions of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the inventors provide the accompanying drawings and the following description to enable those skilled in the art to fully understand the present disclosure, and do not intend for them to limit the subject matter described in the claims. In the following description, components having the same or similar functions are designated by the same reference numerals.

[0012] The following embodiments are examples, and the technology of the present disclosure is not limited to the following embodiments. For example, the numerical values, shapes, materials, steps, the order of the steps, the layout of the display screen, and the like shown in the following embodiments are merely examples, and various modifications are possible as long as no technical contradiction occurs. Furthermore, one aspect can be combined with another aspect as long as no technical contradiction occurs.

[0013] 1 and 2, an example of the hardware configuration of an agricultural support system 1000 according to this embodiment and various devices connected to the agricultural support system 1000 will be described. Hereinafter, the agricultural support system 1000 will be simply referred to as the "system 1000."

[0014] Fig. 1 is a schematic diagram illustrating an example configuration of a system 1000 according to an embodiment of the present disclosure. Fig. 2 is a block diagram illustrating an example hardware configuration of the system 1000 according to an embodiment of the present disclosure. Fig. 2 also illustrates an example hardware configuration of an unmanned aerial vehicle (UAV) 400 communicatively connected to the system 1000 via a network 60.

[0015] An unmanned aerial vehicle (UAV) is an aircraft that cannot carry a human being due to its structure, but can fly by remote control or automatic piloting. A rotary-wing UAV is an UAV that obtains lift using propellers, i.e., rotors, that rotate around an axis. Small UAVs equipped with multiple rotors (multi-rotor UAVs) are also called "drones," "multirotors," or "multicopters," and are widely used for applications such as aerial photography, surveying, and pesticide spraying. In this specification, UAVs are referred to as drones.

[0016] The system 1000 has various functions for supporting agricultural work. For example, the system 1000 may have a function for managing a field, a function for checking the growth status of crops in the field, a function for sending weed control instructions to agricultural machinery based on the results of detecting weeds growing in the field, a function for creating a work log, a function for analyzing taste and yield data for each field to design fertilization, a function for creating a harvesting plan, and a function for simulating the costs of materials such as pesticides and fertilizers for each field. By linking an agricultural machine 300 such as a tractor with the system 1000, it is possible to improve the yield and quality of crops.

[0017] The system 1000 is substantially realized as a computer system. The system 1000 in the embodiment of the present disclosure includes a server computer 100 (hereinafter referred to as the "server 100") and one or more terminals 200. For example, one or more agricultural machines 300 and / or one or more drones 400 may be connected to the system 1000 so as to be able to communicate with each other via a wired or wireless network 60. FIG. 1 shows a connection example in which two terminals 200A and 200B, one agricultural machine 300, and one drone 400 are connected so as to be able to communicate with each other via the network 60. However, the number of terminals 200, agricultural machines 300, and drones 400 connected to the network 60 may be any number. Each component will be described below.

[0018] (Server 100) The server 100 may be a computer, such as a cloud server or an edge server, that is installed in a location remote from the agricultural machine 300 and the drone 400. The server 100 includes a communication interface (I / F) 110, a processing device 120, and a storage device 130. In an embodiment of the present disclosure, the server 100 functions as a cloud server that centrally manages information related to the field and supports agriculture by utilizing the managed data.

[0019] The communication I / F 110 is an interface for communicating with the terminal 200, the agricultural machine 300, and the drone 400 via the network 60. For example, the communication I / F 110 can perform wired communication in accordance with various protocols. The communication I / F 110 may perform wireless communication in accordance with the Bluetooth (registered trademark) standard and / or the Wi-Fi (registered trademark) standard. Both standards include wireless communication standards that use frequencies in the 2.4 GHz band.

[0020] The processing device 120 includes, for example, a processor 121, a read-only memory (ROM) 122, and a random access memory (RAM) 123. Software (or firmware) for the processor 121 to execute at least one process may be implemented in the ROM 122. Such software may be recorded on a computer-readable recording medium, such as an optical disk, and sold as packaged software, or provided to users via the network 60.

[0021] The processor 121 is a semiconductor integrated circuit and includes a central processing unit (CPU). The processor 121 may be implemented by a microprocessor or a microcontroller. The processor 121 sequentially executes a computer program stored in the ROM 122, which describes a group of instructions for executing at least one process, to achieve the desired process.

[0022] In addition to or instead of the processor 121, the processing device 120 may include an FPGA (Field Programmable Gate Array) equipped with a CPU, a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an ASSP (Application Specific Standard Product), or a combination of two or more circuits selected from these circuits.

[0023] The ROM 122 is, for example, a writable memory (e.g., a PROM), a rewritable memory (e.g., a flash memory), or a read-only memory. The ROM 122 stores a program that controls the operation of the processor 121. The ROM 122 does not need to be a single recording medium, but may be a collection of multiple recording media. Some of the collection of multiple recording media may be removable memories.

[0024] The RAM 123 provides a working area for temporarily loading the control program stored in the ROM 122 at boot time. The RAM 123 does not have to be a single recording medium, but may be a collection of multiple recording media.

[0025] The storage device 130 mainly functions as database storage. An example of the storage device 130 is cloud storage. The storage device 130 may be, for example, a semiconductor storage device, a magnetic storage device, an optical storage device, or a combination thereof. However, the storage device 130 may also be an external storage device connected to the server 100 via the network 60.

[0026] The storage device 130 may store field image data. An example of field image data is aerial image data acquired from the sky by a drone, which includes one or more fields as subjects. The aerial image data is, for example, a remote sensing image such as an aerial photograph or a satellite image. The remote sensing image data may include geographic information or location information.

[0027] A "remote sensing image" is digital data acquired by remotely sensing the ground using a sensor mounted on a satellite, a manned aircraft, a drone, or the like. Remote sensing images include, for example, RGB images, grayscale images, multispectral images, or vegetation index images. Examples of sensors include an RGB image sensor and a multispectral sensor. A multispectral sensor can simultaneously acquire images in five wavelength bands, for example, red, green, blue, red edge, and near-infrared. Images in five wavelength bands include RGB images, red edge images, and near-infrared images. Remote sensing images may also be aerial survey images for creating maps or field maps.

[0028] (Terminal 200) Examples of the terminal 200 are a personal computer (PC), a laptop computer, a tablet computer, a smartphone, or a personal digital assistant (PDA). In Fig. 1, a laptop computer 200A and a tablet computer 200B are illustrated as examples of the terminal 200.

[0029] The terminal 200 includes an input device 210, a display device 220, a processor 230, a ROM 240, a RAM 250, a storage device 260, and a communication I / F 270. These components are connected to each other via a bus so that they can communicate with each other.

[0030] The input device 210 is a device for converting instructions from a user into data and inputting the data into a computer. Examples of the input device 210 include a keyboard, a mouse, and a touch panel.

[0031] Examples of the display device 220 are a liquid crystal display or an organic EL display.

[0032] The processor 230, the ROM 240, the RAM 250, and the storage device 260 are the same as those described in the hardware configuration example of the server 100, and therefore will not be described here.

[0033] The communication I / F 270 is an interface for communicating with the server 100, the agricultural machine 300, and the drone 400 via the network 60. For example, like the communication I / F 110, the communication I / F 270 can perform wired communication in accordance with a communication standard such as USB, IEEE 1394 (registered trademark), or Ethernet (registered trademark). Like the communication I / F 110, the communication I / F 270 can perform wireless communication in accordance with the Bluetooth (registered trademark) standard and / or the Wi-Fi standard.

[0034] (Agricultural Machine 300) In this disclosure, the term "agricultural machine" broadly includes machines that perform basic agricultural tasks in fields, such as tilling, planting, and harvesting. Agricultural machines are machines with functions and structures that perform agricultural tasks on the ground in a field, such as tilling, sowing, pest control, fertilizing, planting crops, or harvesting. These agricultural tasks are sometimes referred to as "ground work" or simply "work." Agricultural machines do not necessarily have a traveling device for their own movement; they may travel by being attached to or towed by another vehicle that has a traveling device. Furthermore, the term "agricultural machine" is not limited to cases where a work vehicle, such as a tractor, functions alone as an "agricultural machine." When a work vehicle is attached to or towed by a work vehicle (an implement), the entire work vehicle may function as a single "agricultural machine." Examples of agricultural machines include tractors, pest control machines, ride-on cultivators, and mobile field robots. Examples of pest control machines include speed sprayers, boom sprayers, ride-on cultivators, and drones.

[0035] 1 is a tractor used in a field. The tractor includes, for example, a vehicle body provided with a cabin, wheels (tires), a prime mover (engine), a transmission, a control device having one or more electronic control units (ECUs), a GNSS unit, and a communication device. The tractor may further include an edge computer.

[0036] The GNSS unit is a GNSS receiver equipped with an antenna for receiving signals from GNSS satellites and a processing circuit. The GNSS unit receives GNSS signals transmitted from GNSS satellites, such as the Global Positioning System (GPS), GLONASS, Galileo, BeiDou, or the Quasi-Zenith Satellite System (QZSS, e.g., Michibiki), and performs positioning based on the signals. The communication device equipped in the tractor can perform wired communication using a Controller Area Network (CAN) or the like, or wireless communication compliant with the Bluetooth (registered trademark) standard and / or Wi-Fi (registered trademark) standard.

[0037] (Drone 400) The drone 400 includes, for example, an imaging device 410, an RTK (Real Time Kinematic GPS) positioning unit 420, an inertial measurement unit (IMU) 430, a driving device 440, a control device 450, a data generating device 460, and a communication I / F 470. The flight of the drone 400 can be controlled using, for example, the user's terminal 200.

[0038] The imaging device 410 includes an image sensor, such as an RGB image sensor or a multispectral sensor, a lens optical system, and a data processing circuit. The image sensor may be, for example, a CMOS sensor having pixels of several Mpx to several tens of Mpx. The imaging device 410 is capable of acquiring, for example, an RGB image including a field area, a multispectral image or a vegetation index image, which may include a grayscale image, an RGB image, a red edge image, and a near-infrared image. The imaging device 410 may acquire, for example, one frame of image per second.

[0039] The RTK positioning unit 420 includes, for example, a GNSS receiver, a Wi-Fi communication module, and a microcontroller. The RTK positioning unit outputs a signal indicating the position of the drone 400 in a world coordinate system, such as a geographic coordinate system fixed relative to the Earth. The RTK positioning unit is capable of performing positioning with an accuracy of several centimeters. Position information including latitude, longitude, and altitude information is acquired through high-precision positioning.

[0040] The IMU 430 is equipped with an acceleration sensor and an angular acceleration sensor, and outputs a signal indicating the amount of movement and attitude of the drone 400 (or the imaging device 410).

[0041] The drive unit 440 includes various devices necessary for the flight of the drone 400, such as an electric motor for drive and multiple rotors.

[0042] The control device 450 includes, for example, a flight control device such as a flight controller and a host computer (companion computer). The control device 450 controls the operation of the drive device 440 based on output signals from the RTK positioning unit 420 and the IMU 430.

[0043] The data generation device 460 generates metadata including shooting information, for example, in accordance with the Exchangeable Image File Format (Exif) standard. The metadata may include information such as the shooting date and time, location information (geotag), posture information, image resolution, shutter speed, aperture (F-number), and ISO sensitivity. The data generation device 460 processes the data output from the image capture device 410, adds metadata to the processed data, and can generate image data in a file format such as RAW, DNG, TIFF, GeoTIFF, or JPEG.

[0044] The communication I / F 470 is an interface for communication between the drone 400 and an external device. For example, the communication I / F 470 may perform wireless communication in accordance with mobile communication standards such as LTE (Long Term Evolution), 4G, and 5G, and / or Wi-Fi (registered trademark) standards.

[0045] In one aspect, the system 1000 according to the embodiment of the present disclosure includes a processing device that processes an image of a field including multiple crop rows viewed from above to detect the multiple crop rows. The processing device 120 of the server 100, the edge computer of the agricultural machine 300, the processor 230 of the terminal 200, or at least two of these computing devices may work together to implement the functions of the processing device of the system 1000 described above. In the embodiment of the present disclosure, the processing device 120 of the server 100 functions as the processing device of the system 1000.

[0046] The processing device of system 1000 is configured to generate a canopy image by extracting a canopy that is a plurality of crop rows from the image, integrate pixel values ​​of pixel groups constituting the canopy image along N integration lines (N is an integer greater than or equal to 2) of the canopy image to obtain N integration values ​​related to the N integration lines, apply a filter to a one-dimensional array storing the N integration values ​​and perform a convolution operation to obtain N weighted integration values, and detect the plurality of crop rows based on the N weighted integration values. The processing device may further be configured to identify an area between adjacent crop rows based on the detection result of the crop rows, and to transmit a control instruction to the agricultural machine when it detects weeds growing in the identified area.

[0047] 2. Example of Implementation of Crop Row Detection Algorithm FIG. 3 is a flowchart showing the processing procedure of an example of implementation of the algorithm for detecting crop rows.

[0048] In the embodiment of the present disclosure, the crop in the crop row is paddy rice. However, the crop is not limited to paddy rice. The method for detecting a crop row in the embodiment of the present disclosure can be suitably applied to detecting a crop row other than a crop cultivated by broadcast seeding.

[0049] A method for detecting multiple crop rows in a field according to an embodiment of the present disclosure is implemented in a computer such as the processing device of the server described above. The method causes the computer to execute the following steps: acquire an image of the field viewed from above (step S10), generate a canopy image by extracting the canopy, which is the multiple crop rows, from the image (step S20), integrate pixel values ​​of pixel groups constituting the canopy image along N integration lines in the canopy image to obtain N integrated values ​​associated with the N integration lines (step S30), apply a filter to a one-dimensional array storing the N integrated values ​​and perform a convolution operation to obtain N weighted integrated values ​​(step S40), and detect the multiple crop rows based on the N weighted integrated values ​​(step S50).

[0050] (Step S10) The processing device acquires an image of the field including multiple crop rows viewed from above. For example, the processing device accesses the storage device 130 of the server 100 and acquires the image of the field viewed from above by reading out remote sensing image data stored in the storage device 130. Hereinafter, the image of the field viewed from above will be referred to as the "input image" to the system.

[0051] The input image is not limited to a remote sensing image. It may also be a top-view image (or a bird's-eye view image) of the ground, obtained by, for example, performing a homography transformation (planar projection transformation) on an image captured by a camera mounted on an agricultural machine (e.g., a tractor) photographing the cultivated field from an obliquely upward angle. Alternatively, the storage device 130 of the server 100 may store field map data containing positional information of seedlings planted in the field. In this case, the processing device can generate an image comparable to a remote sensing image or a top-view image by visualizing the positional information of the seedlings indicated by the field map data. An image generated in this manner is also an example of an input image. Figure 4A shows an example of an input image (multispectral image) including rows of rice paddies.

[0052] (Step S20) Next, the processing device generates a vegetation canopy image by extracting vegetation canopy areas, which are multiple crop rows, from the input image. In other words, the processing device applies preprocessing to the input image to generate a vegetation canopy image by extracting vegetation canopy areas, which are multiple crop rows. Figure 4B shows an example of a vegetation canopy image including vegetation canopy areas, which are rice paddy rows. The preprocessing includes, for example, Lab conversion, Otsu's binarization, adaptive binarization, application of deep learning, Gaussian filtering, and expansion / contraction processing.

[0053] If the input image is a multispectral image, the processing device may, for example, apply Lab conversion and Otsu's binarization to the input image in this order to generate a vegetation canopy image. Alternatively, the processing device may apply a Gaussian filter and adaptive binarization to the input image in this order to generate a vegetation canopy image. The processing device preferably applies dilation and shrinkage to the binary image after applying Otsu's binarization or adaptive binarization until the average pixel value of the image falls within a predetermined range. Applying dilation and shrinkage improves the accuracy of crop row detection, as described below, regardless of the crop's growth state or growth period.

[0054] Fig. 5A shows an example of an input image before preprocessing is applied, and Fig. 5B shows an example of a binary image after preprocessing is applied. Fig. 6A shows another example of an input image before preprocessing is applied, and Fig. 6B shows another example of a binary image after preprocessing is applied. Both the binary images shown in Fig. 5B and Fig. 6B are images after dilation / erosion processing has been applied.

[0055] An example of deep learning is segmentation (region division), such as semantic segmentation, instance segmentation, or panoptic segmentation, which is a combination of these two algorithms. By applying semantic segmentation to an input image instead of binarization, for example, the processing device can classify the input image into two classes: vegetation areas and non-vegetation areas.

[0056] (Step S30) Next, the processing device integrates pixel values ​​of pixel groups constituting the vegetation canopy image along N integration lines of the vegetation canopy image to obtain N integrated values ​​related to the N integration lines. The N integration lines will be described in detail later.

[0057] Fig. 7 is a flowchart showing an example of a specific processing procedure included in step S30. Figs. 8A to 8D each show an example of a rotated vegetation cover image obtained by rotating the vegetation cover image by a certain rotation angle around the origin of a two-dimensional coordinate system.

[0058] First, the processing device rotates the vegetation canopy image 70 through multiple rotation angles around the origin O of a two-dimensional coordinate system to generate multiple (M pieces, where M is an integer equal to or greater than 2) rotated vegetation canopy images 71 (step S31). The two-dimensional coordinate system is defined by a first axis and a second axis perpendicular to the first axis. In the examples shown in Figures 8A to 8D, the Y axis is the first axis and the X axis is the second axis.

[0059] FIG. 8A shows a rotated vegetation canopy image 71A obtained by rotating the vegetation canopy image 70 counterclockwise at a rotation angle of 0° around the origin O of the two-dimensional coordinate system. In other words, the original vegetation canopy image 70 before rotation is shown. FIG. 8B shows a rotated vegetation canopy image 71B obtained by rotating the vegetation canopy image 70 counterclockwise at a rotation angle of 75° around the origin O of the two-dimensional coordinate system. FIG. 8C shows a rotated vegetation canopy image 71C obtained by rotating the vegetation canopy image 70 counterclockwise at a rotation angle of 90° around the origin O of the two-dimensional coordinate system. FIG. 8D shows a rotated vegetation canopy image 71D obtained by rotating the vegetation canopy image 70 counterclockwise at a rotation angle of 105° around the origin O of the two-dimensional coordinate system.

[0060] The processing device may rotate the vegetation canopy image 70 counterclockwise, for example, in increments of 0.5°, to generate 360 ​​(180 / 0.5) rotated vegetation canopy images 71. Alternatively, the processing device may rotate the vegetation canopy image 70 counterclockwise, for example, in increments of 1°, to generate 180 (180 / 1) rotated vegetation canopy images 71. However, the rotation angle is not particularly limited.

[0061] In an embodiment of the present disclosure, the processing device applies a discrete Fourier transform (DFT) to the vegetation canopy image to generate a frequency spectral image, and from the spectral image, estimates the angle between the direction in which multiple crop rows extend in a two-dimensional coordinate system and the first axis.

[0062] Fig. 9 is a diagram showing an example of a frequency spectrum image generated by applying DFT to a vegetation canopy image. A vegetation canopy image 70 is shown on the left side of Fig. 9, and an example of a spectrum image 72 is shown on the right side. In the vegetation canopy image 70 shown in Fig. 9, multiple lines 80 are shown as dashed lines along multiple crop rows. For simplicity, the multiple lines 80 are all drawn parallel to each other, i.e., each line 80 intersects with the Y-axis at the same angle. However, in reality, the direction in which each crop row extends may vary.

[0063] 9, dotted lines 81 extending in a direction intersecting the direction in which the multiple lines 80 extend are shown. In the spectral image 72, the change in brightness is small in the direction in which the crop row lines 80 extend, and the change in brightness is large along the lines 81 extending in a direction intersecting the direction in which the lines 80 extend. Therefore, the amplitude of the high-frequency region is large in the direction in which the lines 81 extend. Theoretically, the change in brightness is greatest along the lines 81 extending in a direction perpendicular to the direction in which the lines 80 extend, and therefore the amplitude of the high-frequency region is largest in that direction.

[0064] The processing device applies a two-dimensional DFT to the vegetation canopy image 70 and integrates the spectrum of frequencies corresponding to the distance between adjacent crop rows from the spectral image 72 at regular angles (e.g., 3.0°). In the example shown in FIG. 9 , the processing device integrates the spectrum of frequencies at regular angles and determines that the integrated value of the spectrum of frequencies along the direction of the line 81 is the largest among the multiple integrated values. From the direction of the line 81, the processing device can estimate the angle between the direction in which the multiple crop rows extend and the X-axis or Y-axis in the two-dimensional coordinate system. In this way, the processing device can narrow down the direction in which the crop rows extend from the spectral image based on, for example, the spacing between the crop rows set during transplanting.

[0065] In an embodiment of the present disclosure, the processing device may further determine multiple rotation angles based on the estimated angle and generate multiple rotated canopy images by rotating the canopy image at the multiple rotation angles around the origin of the two-dimensional coordinate system. For example, the processing device may generate 21 rotated canopy images by rotating the canopy image in 0.5° increments within a range of ±5° based on the estimated angle. Narrowing down the number of rotation angle candidates based on the estimated angle reduces the number of rotated canopy images to be generated, thereby reducing the computational load of the integrated values ​​along the pixel lines described below.

[0066] Next, the processing device calculates N integrated values ​​for each of the M rotated vegetation cover images 71 by integrating the pixel values ​​of the pixel groups that make up the rotated vegetation cover image 71 along N integration lines (step S32).

[0067] Fig. 10A is a schematic diagram illustrating a plurality of integration lines 82 of the vegetation canopy image 70 or the rotated vegetation canopy image 71A shown in Fig. 8A. Fig. 10B is a schematic diagram illustrating a plurality of integration lines 82 of the rotated vegetation canopy image 71B shown in Fig. 8B.

[0068] As shown in FIG. 10A, multiple (N lines: N is an integer greater than or equal to 2) integration lines 82 extending parallel to the first axis, the Y-axis, are drawn at equal intervals on the image. For example, if the image size is 1920 x 1080, 1080 integration lines 82 corresponding to the 1080 pixels in the X-axis direction can be drawn at equal intervals on the 1080 pixels. Multiple integration lines 82 extending parallel to the first axis, the Y-axis, can also be drawn on the rotated vegetation canopy image. As illustrated in FIG. 10B, a number N of integration lines 82 determined based on the horizontal size of the circumscribing rectangle 75 of the rotated vegetation canopy image 71B in the X-axis direction are drawn at equal intervals on the rotated vegetation canopy image 71B.

[0069] For each of the M rotated vegetation canopy images 71, the processing device integrates the pixel values ​​of the pixel groups constituting the rotated vegetation canopy image 71 along N integration lines 82 to obtain N integrated values. In this manner, the processing device calculates integrated values ​​by summing the number of pixels in the vertical direction of the rotated vegetation canopy image along the integration lines. As described above, if the rotated vegetation canopy image or the vegetation canopy image is a binary image, the integrated value of the pixel values ​​along the integration line is equal to the total number of white pixels along the integration line. As illustrated in FIG. 10B , when integrating pixel values ​​along the integration line 82 of the rotated vegetation canopy image 71, the processing device places dummy pixels with a pixel value of 0 in the area of ​​the circumscribing rectangle 75 other than the rotated vegetation canopy image 71 at coordinates determined based on the pixels constituting the rotated vegetation canopy image 71. The processing device then integrates the pixel values ​​of the pixels, including the dummy pixels, along the integration line 82. The processing device stores the N integrated values ​​obtained in a one-dimensional array.

[0070] Referring again to Figures 8A to 8D, each of Figures 8A to 8D illustrates a portion of a waveform in which N integrated values ​​along N integration lines are plotted. The vertical axis represents the integrated value, and the horizontal axis represents the X coordinate of integration line 82, i.e., the intersection ρ of integration line 82 with the X axis. Among rotated vegetation canopy images 71A to 71D, the amplitude of the integrated value waveform obtained from rotated vegetation canopy image 71C is the largest, while the amplitude of the integrated value waveform obtained from rotated vegetation canopy image 71A or the original vegetation canopy image 70 is the smallest. In this way, the amplitude of the waveform changes depending on the rotation angle of the image.

[0071] 7, the processing device repeatedly executes the process of step S32 (step S33) until it acquires a set of N integrated values ​​for M rotated vegetation cover images 71. In other words, the processing device repeatedly executes the process of step S32 until it acquires all N integrated values ​​for M rotated vegetation cover images 71.

[0072] (Step S40) Next, the processing device applies a filter to a one-dimensional array storing N accumulated values ​​for each of the multiple rotated vegetation cover images 71 and performs a convolution operation (moving average or weighted average) to obtain N weighted accumulated values.

[0073] Fig. 11 is a flowchart showing an example of a specific processing procedure included in step S40. Fig. 12 is a schematic diagram for explaining the convolution operation.

[0074] The processing device applies a filter to a one-dimensional array storing N accumulated values ​​for each of the M rotated vegetation canopy images 71 and performs a convolution operation to obtain N weighted accumulated values ​​(step S41).

[0075] In an embodiment of the present disclosure, as illustrated in FIG. 12 , weighting is performed such that the weight increases as the distance to the center of the crop increases. This weighting allows the center of the crop row to be detected with high accuracy. If the center of the crop row can be detected with high accuracy, the accuracy of crop row detection can be improved. Examples of filters used in the convolution operation are linear filters or Gaussian filters. FIG. 12 shows an example of a linear filter.

[0076] The shape and size of the crop may vary depending on the type of crop. Furthermore, the shape and size of the crop may vary depending on the growing season of the crop. The weight of the filter in the embodiment of the present disclosure may vary depending on the type of crop or growing season. Furthermore, the size ρ of the filter ｗｉｄｔｈ The weight or size of the filter may vary depending on the type of crop or growing season. For example, the system 1000 may manage a work plan including information on the type of crop. The work plan data may be stored, for example, in the storage device 130 of the server 100. Table data describing the relationship between the type of crop or growing season and the weight and size may be stored in the storage device 130. The processing device may acquire the work plan and table data from the storage device 130, refer to the table, and set the filter weight or size according to the crop type information included in the work plan data. The processing device may also estimate the time from, for example, date or year / month information managed by the system 1000, and set the filter weight or size based on the estimation result. However, the filter weight and size may also be set by a user operating the terminal 200 and inputting information into the system 1000.

[0077] By changing the size or weight of the filter depending on the type of crop or growing season, the impact of environmental changes such as the type of crop or growing season on crop row detection can be reduced, thereby improving the accuracy and robustness of crop row detection.

[0078] The processing device repeatedly executes the process of step S41 (step S42) until it acquires a set of N weighted integrated values ​​for M rotated vegetation cover images 71. In other words, the processing device repeatedly executes the process of step S41 until it acquires all N weighted integrated values ​​for M rotated vegetation cover images 71.

[0079] (Step S50) Referring again to the flowchart in Fig. 3, the processing device detects a plurality of crop rows based on the set of N weighted integrated values ​​obtained from the plurality of rotated vegetation canopy images 71 acquired in step S42.

[0080] The weighted integrated value in the embodiment of the present disclosure is expressed by the function f(θ, ρ), where θ and ρ are the rotation angle and the X coordinate of the integration line 82, respectively, as described above.

[0081] 13 is a schematic diagram showing an outline of a list obtained by sorting the N weighted integrated values ​​of each rotated vegetation cover image included in the set of weighted integrated values ​​in descending order. When there are a total of M rotated vegetation cover images, the processing device The weighted sums are sorted in descending order.

[0082] Assume that the angle estimated by the processing device in step S30 is 80°. In FIG. 13, N weighted integrated values ​​obtained from each rotated vegetation canopy image 71 rotated in 0.5° increments within a range of ±5° (i.e., 75°≦θ≦85°) based on the angle 80° are represented by the function f(θ,ρ). In this example, the number M of rotated vegetation canopy images is 21. The processing device in this embodiment of the present disclosure sorts the N weighted integrated values ​​of each of the M rotated vegetation canopy images 71 included in the set of weighted integrated values ​​in descending order and generates a list from the sorted results. For example, the processing device adopts the weighted integrated values ​​in descending order from the top of the list.

[0083] 13, the function f(75, -50) = 8.0 indicating the weighted integrated value for a rotation angle θ = 75 and an X coordinate of the integration line = -50 is at the top, followed by the function f(85, -40) = 7.2 indicating the weighted integrated value for a rotation angle θ = 85 and an X coordinate of the integration line = -40, and then the function f(75.5, -60) = 6.5 indicating the weighted integrated value for a rotation angle θ = 75.5 and an X coordinate of the integration line = -60. The processing device adopts the weighted integrated values ​​of the functions f(75, -50), f(85, -40), and f(75.5, -60) in order from the top of the list. The processing device may adopt the weighted integrated values ​​based on the list, taking into account constraints.

[0084] The processing device according to the embodiment of the present disclosure estimates that a crop row formed along an integration line located at the coordinate ρ exists in the rotated vegetation canopy image 71 at a rotation angle θ indicated by the adopted function f(θ, ρ). For example, the processing device identifies a crop row corresponding to one function f(θ, ρ) from among multiple crop rows. Finally, the processing device can use the crop row estimation result obtained from the function f(θ, ρ) based on the rotated vegetation canopy image 71 to obtain position information for the crop row present in the vegetation canopy image or the input image, and detect the crop row based on the position information.

[0085] According to the embodiment of the present disclosure, since the angle at which the direction of extension of a crop row intersects with the X-axis or Y-axis can be determined for each crop row, it is possible to detect the direction in which multiple crop rows extend in a vegetation canopy image or an input image with high accuracy. In particular, even if an error occurs in generating the vegetation canopy image in the process of step S20, robust crop row detection can be achieved by subsequent processes.

[0086] Another implementation example of the method for detecting crop rows will be described with reference to Figures 14 and 15. The processing device may detect crop rows based on vegetation cover ratio instead of a set of weighted integrated values. Among steps S10 to S50 in the processing procedure shown in Figure 3, steps S30 and S40 differ from the process in the implementation example described above. The differences will be described in detail below.

[0087] 14 and 15 are flowcharts illustrating an example of a specific process procedure included in step S30 and step S40, respectively, according to another implementation example.

[0088] The processing device generates M rotated vegetation canopy images by executing the process of step S31 described above, and then calculates N first and second integrated values ​​for each rotated vegetation canopy image (step S34). Specifically, for each of the M rotated vegetation canopy images 71, the processing device integrates the pixel values ​​of the pixel groups constituting the rotated vegetation canopy image 71 along each integration line to obtain a first integrated value, and integrates the number of pixel groups in the rotated vegetation canopy image located on each integration line to obtain a second integrated value, thereby obtaining N first integrated values ​​and N second integrated values, respectively. The N first integrated values ​​correspond to the N integrated values ​​described above. If the vegetation canopy image is a binary image, the first integrated value corresponds to the number of pixels corresponding to the vegetation canopy, and the second integrated value corresponds to the number of all pixels located on the integration line, including pixels not corresponding to the vegetation canopy.

[0089] The processing device repeatedly executes the process of step S34 until a set of N first integrated values ​​and N second integrated values ​​of M rotated vegetation cover images 71 is obtained, as shown in FIG. 14 (step S35).

[0090] Next, as shown in Figure 15, the processing device applies a filter to a one-dimensional array storing N first accumulated values ​​and N second accumulated values ​​for each rotated vegetation cover image 71 and performs a convolution operation to obtain N weighted first accumulated values ​​and N weighted second accumulated values ​​(step S43).

[0091] Next, the processing device calculates the vegetation cover ratio of the weighted first integrated value relative to the weighted second integrated value associated with each of the N integration lines, thereby obtaining N vegetation cover ratios for the N integration lines (step S44). The vegetation cover ratio is represented by the ratio of the weighted first integrated value to the weighted second integrated value. The processing device calculates the vegetation cover ratio for each integration line, thereby obtaining N vegetation cover ratios for the N integration lines.

[0092] 15 , the processing device repeatedly executes the processes of steps S43 and S44 (step S45) until it acquires a set of N vegetation cover ratios for M rotated vegetation cover images 71. In other words, the processing device repeatedly executes the processes of steps S43 and S44 until it acquires all N vegetation cover ratios for M rotated vegetation cover images 71.

[0093] As described above, the processing device can detect multiple crop rows based on a set of N vegetation cover ratios obtained from multiple rotated vegetation cover images. Even in cases where image rotation processing is involved, as in the implementation example of the present disclosure, the use of vegetation cover ratios can improve the accuracy of crop row estimation based on rotated images.

[0094] [3. Example of Implementation of Algorithm for Detecting Weeds Growing Between Crop Rows] The above-described method may further include a step of, after detecting the crop rows, identifying an area between adjacent crop rows based on the detection results of the crop rows, and transmitting a control instruction to the agricultural machine when a weed growing in the identified area is detected.

[0095] FIG. 16 is a flowchart showing a processing procedure according to an implementation example of an algorithm for transmitting a control instruction to an agricultural machine when a weed is detected.

[0096] The processing procedure shown in Fig. 16 includes steps S10 to S80. Of steps S10 to S80, steps S10 to S50 are the same as those described with reference to Fig. 3. Specific processing of steps S60, S70, and S80 will be described below.

[0097] The processing device can be configured to detect crop rows according to the procedure from steps S10 to S50, identify areas between adjacent crop rows based on the results of the crop row detection, and transmit weed control instructions to the agricultural machine when it detects weeds growing in the identified areas.

[0098] (Step S60) The processing device identifies an area between adjacent crop rows based on the detection result of the crop rows. For example, the processing device identifies an area located between two detection lines along the adjacent crop rows from the vegetation canopy image.

[0099] (Step S70) The processing device detects weeds growing in the area identified in step S60. As described above, the crop in the embodiment of the present disclosure is paddy rice. Barnyard grass grows as a weed in the area between the rows of paddy rice. The shape of paddy rice closely resembles that of barnyard grass. In particular, barnyard grass before ears appear is difficult to distinguish from paddy rice.

[0100] Fig. 17 is an enlarged view of a portion of the vegetation cover image 70. Fig. 18 is a view showing an example of a detection result image showing the detection results of crop rows and weeds.

[0101] The processing device according to an embodiment of the present disclosure applies a labeling process to the vegetation canopy image 70 to generate a labeled image. The processing device may also apply a dilation process to the vegetation canopy image before applying the labeling process. The dilation process can improve the accuracy of weed detection. From the labeled image, the processing device determines pixel regions (hereinafter referred to as "connected regions") that exist in the region identified in step S60 and are composed of one or more connected components. The processing device detects weeds growing in the identified region, for example, based on shape characteristic parameters that characterize the shape of the connected region.

[0102] Examples of shape characteristic parameters include the coordinates of the center of gravity of the shape of the connected region and the area of ​​the connected region. For example, the processing device compares the distance from the coordinates of the center of gravity of the shape of the connected region to the crop row with a threshold and detects weeds based on the comparison result. This threshold can be determined, for example, by providing a training dataset to a decision tree and allowing it to learn. As another example, the processing device can compare the area of ​​the connected region with a threshold and detect weeds based on the comparison result.

[0103] 17 shows how objects that are candidates for weeds have been detected in the area between the detection lines 83 for the crop rows. For example, for object 91, the processing device determines that object 91 is a weed based on the results of comparing the shape characteristic parameters with a threshold. For object 92, the area of ​​the connected region is smaller than the threshold. In this case, the processing device determines, for example, that object 92 is noise. For object 93, the distance from the coordinates of the center of gravity of the shape of the connected region to the crop row is smaller than the threshold. In this case, the processing device determines that object 93 is close to rice plants and determines object 93 to be, for example, the lower leaves of rice plants, and excludes it from the list of weed candidates.

[0104] For weeds that can be distinguished from crops by visual inspection, for example, a machine learning model for object detection can be used. However, as mentioned above, the shape of paddy rice closely resembles the shape of barnyard grass, making it extremely difficult to distinguish between paddy rice and barnyard grass. Even if a machine learning model for object detection is used to detect barnyard grass, the results are the same as when distinguishing by visual inspection. On the other hand, according to an embodiment of the present disclosure, even when it is difficult to distinguish between crops and weeds by visual inspection, it is possible to accurately detect weeds. For example, it may be easy to distinguish between paddy rice and barnyard grass, or between paddy rice and weedy rice.

[0105] 18 , the processing device may generate a detection result image 78 by superimposing the detection line of the crop row and / or a rectangular area that contacts the connected area of ​​the weeds on the vegetation canopy image, and may display the detection result image 78 on the display device 220 of the terminal 200, for example. Alternatively, the processing device may store data of the detection result image 78 in the storage device 130 of the server 100, for example.

[0106] (Step S80) When the processing device detects weeds in step S70, it transmits a control instruction to an agricultural machine such as a pesticide-control machine. The agricultural machine performs a pest control task in response to the control instruction transmitted from the processing device. The agricultural machine can be operated with or without a driver, and can be operated manually or automatically. For example, an agricultural machine capable of automatic operation can automatically begin a pest control task in response to the pest control instruction transmitted from the processing device.

[0107] According to an embodiment of the present disclosure, an agricultural support system can improve the accuracy and robustness of crop row detection, for example, by detecting all of the crop rows included in a multispectral image acquired by a drone from above a farm field at once.

[0108] The agricultural support system, crop row or weed detection method according to the embodiments of the present disclosure can also be applied to crops other than rice, such as soybeans or vegetables such as cabbage.

[0109] For crops other than rice, it is possible to identify the crop row using the method disclosed herein, and then observe changes in the crop's vegetation cover over time to determine its growth status, or to identify vegetation other than the crop as weeds. In this case, the results of this determination can be used to identify areas of poor growth or weed infestation and take preventive measures. The growth status can also be used as a source of information for yield prediction. As another example, in a field using a crop rotation system, crops grown in the previous season may also grow in the current season. In this case, there is concern that the growth of the current season's crops may be poor due to the spread of pests and diseases and / or nutrient competition transmitted by the previous season's crops. However, by using the crop row detection method disclosed herein, it is possible to detect the occurrence of weeds in the previous season's crops early and take preventive measures.

[0110] A system providing various functions according to the embodiments of the present disclosure can also be retrofitted to an agricultural machine that does not have those functions. Such a system can be manufactured and sold independently of the agricultural machine. A computer program used in such a system can also be manufactured and sold independently of the agricultural machine. The computer program can be provided, for example, by being stored in a computer-readable non-transitory storage medium. The computer program can also be provided by downloading via a telecommunications line (for example, the Internet).

[0111] As described above, the present disclosure includes an agricultural support system and a crop row detection method described in the following items.

[0112] [Item 1] An agricultural support system comprising a processing device that processes an image of a field including a plurality of crop rows viewed from above to detect the plurality of crop rows, wherein the processing device is configured to: generate a vegetation canopy image by extracting the vegetation canopy that is the plurality of crop rows from the image; accumulate pixel values ​​of pixel groups that make up the vegetation canopy image along N accumulation lines (N is an integer of 2 or greater) of the vegetation canopy image to obtain N accumulated values ​​related to the N accumulation lines; apply a filter to a one-dimensional array that stores the N accumulated values ​​and perform a convolution operation to obtain N weighted accumulated values; and detect the plurality of crop rows based on the N weighted accumulated values.

[0113] [Item 2] The agricultural support system according to Item 1, wherein the weights of the filters vary depending on the type of crop or the growing season.

[0114] [Item 3] The agricultural support system according to Item 1, wherein the size of the filter varies depending on the type of crop or growing season.

[0115] [Item 4] The agricultural support system according to any one of items 1 to 3, wherein the filter is a linear filter or a Gaussian filter.

[0116] [Item 5] The agricultural support system described in any one of Items 1 to 4, wherein the processing device generates a plurality of rotated canopy images by rotating the canopy image at a plurality of rotation angles around the origin of a two-dimensional coordinate system defined by a first axis parallel to the direction in which the N integration lines extend and a second axis perpendicular to the first axis, and for each of the plurality of rotated canopy images, accumulates pixel values ​​of a group of pixels constituting the rotated canopy image along the N integration lines to obtain the N integrated values, applies the filter to a one-dimensional array storing the N integrated values ​​to perform a convolution operation to obtain the N weighted integrated values, and detects the plurality of crop rows based on the set of N weighted integrated values ​​obtained from the plurality of rotated canopy images.

[0117] [Item 6] The processing device generates a plurality of rotated vegetation canopy images by rotating the vegetation canopy image at a plurality of rotation angles around the origin of a two-dimensional coordinate system defined by a first axis parallel to the direction in which the N integration lines extend and a second axis orthogonal to the first axis, and for each of the plurality of rotated vegetation canopy images, the processing device accumulates pixel values ​​of pixel groups constituting the rotated vegetation canopy image along each integration line to obtain a first accumulated value and accumulates the number of pixel groups of the rotated vegetation canopy image located on each integration line to obtain a second accumulated value, thereby obtaining N first accumulated values ​​and N second accumulated values, respectively, and applies the filter to one-dimensional arrays storing the N first accumulated values ​​and the N second accumulated values ​​to perform a convolution operation to obtain N weighted first accumulated values ​​and N weighted second accumulated values, 5. The agricultural support system according to any one of items 1 to 4, further comprising: calculating a vegetation cover ratio of a weighted first integrated value relative to a weighted second integrated value associated with each of the N integrated lines to obtain N vegetation cover ratios for the N integrated lines; and detecting the plurality of crop rows based on the set of N vegetation cover ratios obtained from the plurality of rotated vegetation cover images.

[0118] [Item 7] The agricultural support system according to Item 5 or 6, wherein the processing device applies a discrete Fourier transform to the vegetation canopy image to generate a frequency spectrum image, and estimates, from the spectrum image, an angle between the first axis and a direction in which the plurality of crop rows extend in the two-dimensional coordinate system.

[0119] [Item 8] The agricultural support system described in Item 7, wherein the processing device determines the plurality of rotation angles by referring to the estimated angle, and rotates the vegetation canopy image by the plurality of rotation angles around the origin of the two-dimensional coordinate system to generate the plurality of rotated vegetation canopy images.

[0120] [Item 9] The agricultural support system according to any one of items 1 to 8, wherein the processing device applies Lab transformation or deep learning to the image to generate the vegetation cover image in which the vegetation cover is extracted.

[0121] [Item 10] The agricultural support system according to Item 1, wherein the processing device is further configured to identify an area between adjacent crop rows based on the detection result of the crop rows, and to transmit a control instruction when detecting weeds growing in the identified area.

[0122] [Item 11] The agricultural support system according to Item 10, wherein the processing device applies a labeling process to the vegetation cover image to generate a labeled image, obtains a pixel region composed of one or more connected components present in the identified region from the labeled image, and detects the weeds growing in the identified region based on shape characteristic parameters that characterize the shape of the pixel region.

[0123] [Item 12] The agricultural support system according to Item 10 or 11, further comprising an agricultural machine that performs work for pest control in response to the pest control instruction transmitted from the processing device.

[0124] [Item 13] The agricultural support system according to any one of Items 1 to 12, wherein the crop in the crop row is paddy rice, and the weeds are barnyard grass.

[0125] [Item 14] The agricultural support system according to any one of Items 1 to 13, further comprising an unmanned aerial vehicle that photographs the farm field from above the farm field to acquire a multispectral image.

[0126] [Item 15] A method for detecting multiple crop rows in a field, implemented on a computer, comprising: acquiring an image of the field viewed from above; generating a vegetation canopy image by extracting the vegetation canopy, which is the multiple crop rows, from the image; accumulating pixel values ​​of pixel groups constituting the vegetation canopy image along N integration lines (N is an integer of 2 or greater) of the vegetation canopy image to obtain N integration values ​​related to the N integration lines; applying a filter to a one-dimensional array storing the N integration values ​​and performing a convolution operation to obtain N weighted integration values; and detecting the multiple crop rows based on the N weighted integration values.

[0127] [Item 16] The method according to item 15, wherein the filter weights vary depending on the type of crop or growing season.

[0128] [Item 17] The method according to Item 15, wherein the size of the filter varies depending on the type of crop or growing season.

[0129] [Item 18] The method according to any one of items 15 to 17, wherein the filter is a linear filter or a Gaussian filter.

[0130] [Item 19] The method according to any one of Items 15 to 18, further comprising: generating a plurality of rotated canopy images by rotating the canopy image at a plurality of rotation angles around the origin of a two-dimensional coordinate system defined by a first axis parallel to the direction in which the N integration lines extend and a second axis perpendicular to the first axis; determining the N integrated values ​​includes, for each of the plurality of rotated canopy images, integrating pixel values ​​of a group of pixels constituting the rotated canopy image along the N integration lines to determine the N integrated values; determining the N weighted integrated values ​​includes, for each of the plurality of rotated canopy images, applying the filter to a one-dimensional array storing the N integrated values ​​and performing a convolution operation to determine the N weighted integrated values; and detecting the plurality of crop rows based on the set of N weighted integrated values ​​obtained from the plurality of rotated canopy images.

[0131] [Item 20] The method includes generating a plurality of rotated vegetation canopy images by rotating the vegetation canopy image at a plurality of rotation angles around the origin of a two-dimensional coordinate system defined by a first axis parallel to the direction in which the N integration lines extend and a second axis orthogonal to the first axis, and obtaining the N integrated values ​​includes, for each of the plurality of rotated vegetation canopy images, integrating pixel values ​​of a group of pixels constituting the rotated vegetation canopy image along each integration line to obtain a first integrated value, and integrating the number of pixel groups of the rotated vegetation canopy image located on each integration line to obtain a second integrated value, thereby obtaining N first integrated values ​​and N second integrated values, respectively, and applying the filter to a one-dimensional array storing the N first integrated values ​​and the N second integrated values ​​to perform a convolution operation, thereby obtaining N weighted first integrated values ​​and N weighted second integrated values, 19. The method of any one of items 15 to 18, further comprising: determining a vegetation cover ratio of the weighted first integrated value relative to the weighted second integrated value associated with each of the N integrated lines to obtain N vegetation cover ratios for the N integrated lines; and detecting the plurality of crop rows based on the set of N vegetation cover ratios for the plurality of rotated vegetation cover images.

[0132] [Item 21] The method according to Item 19 or 20, further comprising applying a discrete Fourier transform to the vegetation canopy image to generate a frequency spectrum image, and estimating from the spectrum image an angle between the direction in which the plurality of crop rows extend in the two-dimensional coordinate system and the first axis.

[0133] [Item 22] The method according to Item 21, comprising determining the plurality of rotation angles by referring to the estimated angles, and rotating the vegetation canopy image by the plurality of rotation angles around the origin of the two-dimensional coordinate system to generate the plurality of rotated vegetation canopy images.

[0134] [Item 23] The method according to any one of Items 15 to 22, wherein generating the vegetation canopy image includes applying Lab transformation or deep learning to the image to generate the vegetation canopy image in which the vegetation canopy is extracted.

[0135] [Item 24] The method according to Item 15, further comprising identifying an area between adjacent crop rows based on the detection result of the crop rows, and transmitting a control instruction to an agricultural machine when weeds growing in the identified area are detected.

[0136] An embodiment of the present disclosure can be used, for example, in an agricultural support system that manages the growing conditions of crops on the cloud.

[0137] 100...Server (server computer), 110...Communication I / F, 120...Processing device, 121...Processor, 122...ROM, 123...RAM, 130...Storage device, 200...Terminal, 210...Input device, 220...Display device, 230...Processor, 240...ROM, 250...RAM, 260...Storage device, 270...Communication I / F, 300...Agricultural machinery, 400...Drone, 410...Imaging device, 420...RTK positioning unit, 420...Positioning unit, 440...Drive device, 450...Control device, 460...Data generation device, 470...Communication I / F, 1000...Agricultural support system

Claims

1. An agricultural support system comprising a processing device that processes an image of a field including a plurality of crop rows viewed from above to detect the plurality of crop rows, the processing device being configured to: generate a vegetation cover image by extracting the vegetation cover which is the plurality of crop rows from the image; accumulate pixel values ​​of pixel groups constituting the vegetation cover image along N accumulation lines (N is an integer equal to or greater than 2) of the vegetation cover image to obtain N accumulated values ​​related to the N accumulation lines; apply a filter to a one-dimensional array storing the N accumulated values ​​and perform a convolution operation to obtain N weighted accumulated values; and detect the plurality of crop rows based on the N weighted accumulated values.

2. The agricultural support system according to claim 1, wherein the filter weights vary depending on the type of crop or the growing season.

3. The agricultural support system according to claim 1, wherein the size of the filter varies depending on the type of crop or the growing season.

4. The agricultural support system according to any one of claims 1 to 3, wherein the filter is a linear filter or a Gaussian filter.

5. The agricultural support system described in any one of claims 1 to 3, wherein the processing device generates a plurality of rotated vegetation canopy images by rotating the vegetation canopy image at a plurality of rotation angles around an origin of a two-dimensional coordinate system defined by a first axis parallel to the direction in which the N accumulation lines extend and a second axis perpendicular to the first axis, and for each of the plurality of rotated vegetation canopy images, accumulates pixel values ​​of a group of pixels constituting the rotated vegetation canopy image along the N accumulation lines to obtain the N accumulated values, obtains the N weighted accumulated values ​​by applying the filter to a one-dimensional array storing the N accumulated values ​​and performing a convolution operation, and detects the plurality of crop rows based on the set of N weighted accumulated values ​​obtained from the plurality of rotated vegetation canopy images.

6. The processing device generates a plurality of rotated vegetation cover images by rotating the vegetation cover image at a plurality of rotation angles around the origin of a two-dimensional coordinate system defined by a first axis parallel to the direction in which the N accumulation lines extend and a second axis perpendicular to the first axis; for each of the plurality of rotated vegetation cover images, the processing device accumulates pixel values ​​of a group of pixels constituting the rotated vegetation cover image along each accumulation line to obtain a first accumulated value, and accumulates the number of groups of pixels of the rotated vegetation cover image located on each accumulation line to obtain a second accumulated value, thereby obtaining N first accumulated values ​​and N second accumulated values, respectively; and applies the filter to a one-dimensional array storing each of the N first accumulated values ​​and the N second accumulated values, thereby performing a convolution operation, thereby obtaining N weighted first accumulated values ​​and N weighted second accumulated values; 4. The agricultural support system according to claim 1, further comprising: determining a vegetation cover ratio of a weighted first integrated value relative to a weighted second integrated value associated with each of the N integrated lines to obtain N vegetation cover ratios for the N integrated lines; and detecting the plurality of crop rows based on a set of the N vegetation cover ratios obtained from the plurality of rotated vegetation cover images.

7. The agricultural support system of claim 5, wherein the processing device applies a discrete Fourier transform to the vegetation cover image to generate a frequency spectrum image, and estimates from the spectrum image the angle between the direction in which the multiple crop rows extend in the two-dimensional coordinate system and the first axis.

8. The agricultural support system of claim 7, wherein the processing device determines the multiple rotation angles by referring to the estimated angle, and rotates the vegetation cover image at the multiple rotation angles around the origin of the two-dimensional coordinate system to generate the multiple rotated vegetation cover images.

9. An agricultural support system according to any one of claims 1 to 3, wherein the processing device applies Lab conversion or deep learning to the image to generate the vegetation cover image in which the vegetation cover is extracted.

10. The agricultural support system of claim 1, wherein the processing device is further configured to identify an area between adjacent crop rows based on the detection result of the crop rows, and to transmit a weed control instruction when it detects weeds growing in the identified area.

11. The agricultural support system described in claim 10, wherein the processing device applies a labeling process to the vegetation cover image to generate a labeled image, determines from the labeling image a pixel area consisting of one or more connected components that exists in the identified area, and detects the weeds growing in the identified area based on shape characteristic parameters that characterize the shape of the pixel area.

12. The agricultural support system according to claim 10 or 11, further comprising an agricultural machine that performs work for pest control in response to the pest control instruction transmitted from the processing device.

13. An agricultural support system according to any one of claims 1 to 3, wherein the crop in the crop row is paddy rice and the weeds are barnyard grass.

14. An agricultural support system according to any one of claims 1 to 3, comprising an unmanned aerial vehicle that photographs the field from above the field to obtain a multispectral image.

15. A method for detecting multiple crop rows in a field, implemented on a computer, comprising: acquiring an image of the field viewed from above; generating a vegetation cover image by extracting the vegetation cover that is the multiple crop rows from the image; accumulating pixel values ​​of pixel groups that constitute the vegetation cover image along N integration lines (N is an integer equal to or greater than 2) of the vegetation cover image to obtain N integration values ​​related to the N integration lines; applying a filter to a one-dimensional array storing the N integration values ​​and performing a convolution operation to obtain N weighted integration values; and detecting the multiple crop rows based on the N weighted integration values.

16. The method of claim 15, wherein the filter weights vary by crop type or growing season.

17. The method of claim 15, wherein the size of the filter varies with crop type or growing season.

18. The method according to any one of claims 15 to 17, wherein the filter is a linear filter or a Gaussian filter.

19. The method according to any one of claims 15 to 17, further comprising: generating a plurality of rotated vegetation canopy images by rotating the vegetation canopy image at a plurality of rotation angles around an origin of a two-dimensional coordinate system defined by a first axis parallel to a direction in which the N integration lines extend and a second axis perpendicular to the first axis; determining the N integrated values ​​includes, for each of the plurality of rotated vegetation canopy images, integrating pixel values ​​of a group of pixels constituting the rotated vegetation canopy image along the N integration lines to determine the N integrated values; determining the N weighted integrated values ​​includes, for each of the plurality of rotated vegetation canopy images, applying the filter to a one-dimensional array storing the N integrated values ​​and performing a convolution operation to determine the N weighted integrated values; and detecting the plurality of crop rows based on the set of N weighted integrated values ​​obtained from the plurality of rotated vegetation canopy images.

20. The method includes generating a plurality of rotated vegetation cover images by rotating the vegetation cover image at a plurality of rotation angles around the origin of a two-dimensional coordinate system defined by a first axis parallel to the direction in which the N accumulation lines extend and a second axis perpendicular to the first axis, and obtaining the N integrated values ​​includes, for each of the plurality of rotated vegetation cover images, integrating pixel values ​​of a pixel group constituting the rotated vegetation cover image along each accumulation line to obtain a first integrated value, and integrating the number of pixel groups of the rotated vegetation cover image located on each accumulation line to obtain a second integrated value, thereby obtaining N first integrated values ​​and N second integrated values, respectively, and applying the filter to a one-dimensional array storing each of the N first integrated values ​​and the N second integrated values ​​to perform a convolution operation to obtain N weighted first integrated values ​​and N weighted second integrated values, 18. The method of claim 15, further comprising: determining a vegetation cover ratio of a weighted first integrated value relative to a weighted second integrated value associated with each of the N integrated lines to obtain N vegetation cover ratios for the N integrated lines; and detecting the plurality of crop rows based on the set of N vegetation cover ratios for the plurality of rotated vegetation cover images.

21. The method of claim 19, comprising applying a discrete Fourier transform to the vegetation canopy image to generate a frequency spectrum image, and estimating from the spectrum image an angle between a direction in which the plurality of crop rows extend in the two-dimensional coordinate system and the first axis.

22. A method as claimed in claim 21, comprising determining the plurality of rotation angles by reference to the estimated angles, and rotating the vegetation cover image by the plurality of rotation angles around the origin of the two-dimensional coordinate system to generate the plurality of rotated vegetation cover images.

23. A method according to any one of claims 15 to 17, wherein generating the vegetation cover image includes applying a Lab transform or deep learning to the image to generate the vegetation cover image in which the vegetation cover is extracted.

24. The method according to claim 15, further comprising: identifying an area between adjacent crop rows based on the detection result of the crop rows; and transmitting a control instruction to an agricultural machine when a weed is detected in the identified area.

Images