Agricultural support system and crop row detection method

Abstract

To improve the accuracy and robustness of crop row detection.SOLUTION: An agricultural support system includes a processing device that processes an image of a field including multiple crop rows viewed from above to detect the multiple crop rows. The processing device is configured to: generate, by extracting from the image a planted portion that corresponds to the multiple crop rows, a planted portion image; aggregate pixel values of a group of pixels constituting the planted portion image along N-number (N is an integer equal to or greater than 2) of aggregate lines of the planted portion image to obtain N-number of aggregate values related to the N-number of aggregate lines; apply a filter to a one-dimensional array storing the N-number of aggregate values to perform a convolution operation, thereby, obtaining N-number of weighted aggregate values; and detect the multiple crop rows based on the N-number of weighted aggregate values.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to an agricultural support system and a method for detecting crop rows. [Background technology]

[0002] Research and development is underway on smart agriculture, which utilizes ICT (Information and Communication Technology) and IoT (Internet of Things), as the next generation of agriculture. Research and development is underway on an agricultural support system that centrally manages information about farm fields on the cloud and uses the data on the cloud to support agriculture. By using the agricultural support system, it becomes possible to efficiently carry out all the tasks required for farming, such as pest control, tilling, sowing, fertilizing, planting crops, and harvesting.

[0003] Also, various methods have been proposed for detecting crop rows by processing images including the crop rows in a farm field. Patent Document 1 discloses a control method for making a working machine travel along the ridges in a farm field where rows of ridges are formed on which crops are planted. By processing an image including the farm field, the angle between the direction in which the ridges extend and the direction in which the working machine travels, or the amount of positional deviation of the working machine from the center position of the ridges, is calculated. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6418604 Summary of the Invention [Problem to be solved by the invention]

[0005] Due to the change in the crop type or the environment such as the growing season, the crop row may not be detected properly. Therefore, it is desirable to improve the accuracy and robustness of the crop row detection. [Means for solving the problem]

[0006] An agricultural support system according to one embodiment of the present disclosure includes a processing device that processes an image of a field including multiple crop rows viewed from above to detect the multiple crop rows, and the processing device is configured to generate a vegetation cover image by extracting the vegetation cover that is the multiple crop rows from the image, accumulate pixel values ​​of pixel groups that constitute the vegetation cover image along N accumulation lines (N is an integer greater than or equal to 2) of the vegetation cover image to obtain N accumulated values ​​related to the N accumulation lines, apply a filter to a one-dimensional array storing the N accumulated values ​​and perform a convolution operation to obtain N weighted accumulated values, and detect the multiple crop rows based on the N weighted accumulated values.

[0007] A method according to one aspect of the present disclosure is a computer-implemented method for detecting multiple crop rows in a field, which causes a computer to perform the following steps: acquire an image of the field viewed from above; generate a vegetation cover image by extracting vegetation cover that is the multiple crop rows from the image; accumulate pixel values ​​of pixel groups that constitute the vegetation cover image along N accumulation lines (N is an integer greater than or equal to 2) of the vegetation cover image to obtain N accumulated values ​​related to the N accumulation lines; apply a filter to a one-dimensional array storing the N accumulated values ​​and perform a convolution operation to obtain N weighted accumulated values; and detect the multiple crop rows based on the N weighted accumulated values.

[0008] The general or specific aspects of the present disclosure may be realized by an apparatus, a system, a method, an integrated circuit, a computer program, or a computer-readable non-transitory storage medium, or any combination thereof. The computer-readable storage medium may include a volatile storage medium or a non-volatile storage medium. The apparatus may be composed of multiple devices. When the apparatus is composed of two or more devices, the two or more devices may be arranged in one device or may be arranged separately in two or more separate devices. Effect of the Invention

[0009] According to embodiments of the present disclosure, it is possible to improve the accuracy and robustness of crop row detection. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram illustrating a configuration example of an agricultural support system according to an embodiment of the present disclosure. [Diagram 2] FIG. 1 is a block diagram illustrating an example of a hardware configuration of an agricultural support system according to an embodiment of the present disclosure. [Diagram 3] 1 is a flow chart showing the process steps of an exemplary implementation of the algorithm for detecting crop rows. [Figure 4A] FIG. 1 is a diagram showing an example of an input image including rows of rice paddies. [Figure 4B] FIG. 13 is a diagram showing an example of a vegetation cover image including a vegetation cover that is a row of rice plants. [Figure 5A] FIG. 13 is a diagram illustrating an example of an input image before preprocessing is applied. [Figure 5B] FIG. 13 is a diagram showing an example of a binary image after preprocessing has been applied. [Figure 6A] FIG. 13 is a diagram showing another example of an input image before preprocessing is applied. [Figure 6B] FIG. 13 is a diagram showing another example of a binary image after preprocessing has been applied. [Figure 7] 13 is a flowchart showing an example of a specific process procedure included in step S30. [Figure 8A] 1 is a diagram showing an example of a rotated vegetation cover image obtained by rotating a vegetation cover image at a first rotation angle around the origin of a two-dimensional coordinate system. FIG. [Figure 8B] 13 is a diagram showing an example of a rotated vegetation cover image obtained by rotating the vegetation cover image at a second rotation angle around the origin of the two-dimensional coordinate system. FIG. [Figure 8C] 13 is a diagram showing an example of a rotated vegetation cover image obtained by rotating the vegetation cover image at a third rotation angle around the origin of a two-dimensional coordinate system. FIG. [Figure 8D]13 is a diagram showing an example of a rotated vegetation cover image obtained by rotating the vegetation cover image at a fourth rotation angle around the origin of a two-dimensional coordinate system. FIG. [Figure 9] FIG. 1 is a schematic diagram showing an example of a frequency spectrum image generated by applying DFT to a vegetation cover image. [Figure 10A] 8B is a schematic diagram illustrating a plurality of integrated lines of the vegetation cover image or the rotated vegetation cover image shown in FIG. 8A. FIG. [Figure 10B] FIG. 8C is a schematic diagram illustrating a plurality of integrated lines of the rotated vegetation cover image shown in FIG. 8B. [Figure 11] 13 is a flowchart showing an example of a specific process procedure included in step S40. [Figure 12] FIG. 1 is a schematic diagram for explaining a convolution operation. [Figure 13] FIG. 13 is a schematic diagram showing an outline of a list obtained by sorting, in descending order, N weighted integrated values ​​of each rotated vegetation cover image included in a set of weighted integrated values. [Figure 14] 13 is a flowchart showing an example of a specific process procedure included in step S30 according to another implementation example. [Figure 15] 13 is a flowchart showing an example of a specific process procedure included in step S40 according to another implementation example. [Figure 16] 10 is a flowchart showing a processing procedure according to an implementation example of an algorithm for transmitting a control instruction to an agricultural machine when a weed is detected. [Figure 17] FIG. 13 is an enlarged view of a portion of the vegetation cover image. [Figure 18] 13 is a diagram illustrating an example of a detection result image showing the detection results of crop rows and weeds. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present disclosure will be described. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters and overlapping explanation of substantially the same configuration may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Note that the inventor provides the accompanying drawings and the following explanation so that those skilled in the art can fully understand the present disclosure, and does not intend to limit the subject matter described in the claims by them. In the following description, components having the same or similar functions are given the same reference numerals.

[0012] The following embodiments are illustrative, and the technology of the present disclosure is not limited to the following embodiments. For example, the numerical values, shapes, materials, steps, the order of the steps, the layout of the display screen, and the like shown in the following embodiments are merely examples, and various modifications are possible as long as no technical contradiction occurs. In addition, one aspect can be combined with another aspect as long as no technical contradiction occurs.

[0013] [1. System configuration] 1 and 2, a description will be given of an example of the hardware configuration of an agricultural support system 1000 according to this embodiment and various devices connected to the agricultural support system 1000. Hereinafter, the agricultural support system 1000 will be simply referred to as the "system 1000."

[0014] Fig. 1 is a schematic diagram showing a configuration example of a system 1000 according to an embodiment of the present disclosure. Fig. 2 is a block diagram showing a hardware configuration example of the system 1000 according to an embodiment of the present disclosure. Fig. 2 also illustrates a hardware configuration of an unmanned aerial vehicle (UAV) 400 communicably connected to the system 1000 via a network 60.

[0015] An unmanned aerial vehicle is an aircraft that cannot carry a person due to its structure, and can fly by remote control or automatic pilot. A rotary-wing unmanned aerial vehicle is an unmanned aerial vehicle that obtains lift by using a propeller that rotates around an axis, i.e., a rotor. Small unmanned aerial vehicles (Multi-Rotor UAVs) equipped with multiple rotors are also called "drones," "multirotors," or "multicopters," and are widely used for applications such as aerial photography, surveying, and pesticide spraying. In this specification, an unmanned aerial vehicle is called a drone.

[0016] The system 1000 has various functions for supporting agricultural work. The system 1000 may have, for example, a function for managing a field, a function for checking the growth status of crops in the field, a function for sending control instructions to agricultural machinery in response to the detection result of weeds growing in the field, a function for creating a work log, a function for designing fertilization by analyzing taste and yield data for each field, a function for creating a harvesting plan, and a function for simulating the cost of materials such as pesticides and fertilizers for each field. By linking an agricultural machine 300 such as a tractor with the system 1000, it becomes possible to improve the yield and quality of crops.

[0017] The system 1000 is substantially realized as a computer system. The system 1000 in the embodiment of the present disclosure includes a server computer 100 (hereinafter, referred to as "server 100") and one or more terminals 200. For example, one or more agricultural machines 300 and / or one or more drones 400 may be connected to the system 1000 via a wired or wireless network 60 so as to be able to communicate with each other. FIG. 1 shows a connection example in which two terminals 200A and 200B, one agricultural machine 300, and one drone 400 are connected to the network 60 so as to be able to communicate with each other. However, the number of each of the terminals 200, agricultural machines 300, and drones 400 connected to the network 60 is arbitrary. Each component will be described below.

[0018] (Server 100) The server 100 may be a computer installed at a location remote from the agricultural machine 300 and the drone 400, such as a cloud server or an edge server. The server 100 includes a communication interface (I / F) 110, a processing device 120, and a storage device 130. In the embodiment of the present disclosure, the server 100 functions as a cloud server that centrally manages information related to the field and supports agriculture by utilizing the managed data.

[0019] The communication I / F 110 is an interface for communicating with the terminal 200, the agricultural machine 300, and the drone 400 via the network 60. For example, the communication I / F 110 can perform wired communication in accordance with various protocols. The communication I / F 110 may perform wireless communication in accordance with the Bluetooth (registered trademark) standard and / or the Wi-Fi (registered trademark) standard. Both standards include wireless communication standards that use frequencies in the 2.4 GHz band.

[0020] The processing device 120 includes, for example, a processor 121, a read only memory (ROM) 122, and a random access memory (RAM) 123. Software (or firmware) for the processor 121 to execute at least one process may be implemented in the ROM 122. Such software may be recorded on a computer-readable recording medium such as an optical disk, sold as packaged software, or provided to a user via the network 60.

[0021] The processor 121 is a semiconductor integrated circuit and includes a central processing unit (CPU). The processor 121 may be realized by a microprocessor or a microcontroller. The processor 121 sequentially executes a computer program stored in the ROM 122, which describes a group of instructions for executing at least one process, to realize a desired process.

[0022] In addition to or instead of the processor 121, the processing device 120 may include an FPGA (Field Programmable Gate Array) equipped with a CPU, a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an ASSP (Application Specific Standard Product), or a combination of two or more circuits selected from among these circuits.

[0023] ROM 122 is, for example, a writable memory (e.g., PROM), a rewritable memory (e.g., flash memory), or a read-only memory. ROM 122 stores a program that controls the operation of processor 121. ROM 122 does not need to be a single recording medium, but may be a collection of multiple recording media. Part of the collection of multiple recording media may be a removable memory.

[0024] The RAM 123 provides a working area for temporarily loading, at boot time, the control program stored in the ROM 122. The RAM 123 does not have to be a single recording medium, but may be a collection of multiple recording media.

[0025] The storage device 130 mainly functions as a database storage. An example of the storage device 130 is a cloud storage. The storage device 130 may be, for example, a semiconductor storage device, a magnetic storage device, an optical storage device, or a combination thereof. However, the storage device 130 may be an external storage device connected to the server 100 via the network 60.

[0026] The storage device 130 may store field image data. An example of the field image data is data of an aerial image captured from above by a drone, including one or more fields as objects. The aerial image is, for example, a remote sensing image such as an aerial photograph or a satellite image. The remote sensing image may have geographic information or location information.

[0027] A "remote sensing image" is digital data acquired by remotely sensing the ground using a sensor mounted on a satellite, a manned aircraft, or a drone. The remote sensing image includes, for example, an RGB image, a grayscale image, a multispectral image, or a vegetation index image. Examples of sensors include an RGB image sensor, a multispectral sensor, and the like. The multispectral sensor is capable of simultaneously acquiring images in five wavelength bands, for example, red, green, blue, red edge, and near infrared. The images in the five wavelength bands include an RGB image, a red edge image, and a near infrared image. The remote sensing image may be an aerial survey image for creating a map or a field map.

[0028] (Terminal 200) Examples of the terminal 200 include a personal computer (PC), a laptop computer, a tablet computer, a smartphone, or a personal digital assistant (PDA). In FIG. 1, a laptop computer 200A and a tablet computer 200B are illustrated as examples of the terminal 200.

[0029] The terminal 200 includes an input device 210, a display device 220, a processor 230, a ROM 240, a RAM 250, a storage device 260, and a communication I / F 270. These components are connected to each other via a bus so as to be able to communicate with each other.

[0030] The input device 210 is a device for converting instructions from a user into data and inputting the data to a computer. Examples of the input device 210 include a keyboard, a mouse, or a touch panel.

[0031] Examples of the display device 220 are a liquid crystal display or an organic electroluminescence display.

[0032] The processor 230, the ROM 240, the RAM 250, and the storage device 260 are the same as those described in the hardware configuration example of the server 100, and therefore will not be described here.

[0033] The communication I / F 270 is an interface for communicating with the server 100, the agricultural machine 300, and the drone 400 via the network 60. For example, like the communication I / F 110, the communication I / F 270 can perform wired communication conforming to a communication standard such as USB, IEEE1394 (registered trademark), or Ethernet (registered trademark). Like the communication I / F 110, the communication I / F 270 can perform wireless communication conforming to the Bluetooth (registered trademark) standard and / or the Wi-Fi standard.

[0034] (Agricultural machinery 300) In this disclosure, the term "agricultural machinery" broadly includes machinery that performs basic agricultural tasks such as "plowing," "planting," and "harvesting" in a field. Agricultural machinery is a machine that has a function and structure for performing agricultural tasks such as plowing, sowing, pest control, fertilization, planting crops, or harvesting on the ground in a field. These agricultural tasks are sometimes referred to as "ground tasks" or simply "tasks." Agricultural machinery does not need to have a traveling device for moving by itself, and may travel by being attached to or towed by another vehicle that has a traveling device. In addition, the function of a work vehicle such as a tractor is not limited to a single "agricultural machine," and when the work vehicle is attached to or towed by the work vehicle (implement), the entire work vehicle may function as one "agricultural machine." Examples of agricultural machinery include tractors, pest control machines, riding management machines, and field mobile robots. Pest control machines include, for example, speed sprayers, boom sprayers, riding management machines, and drones.

[0035] 1 is a tractor used in a farm field. The tractor includes, for example, a vehicle body provided with a cabin, wheels (tires), a prime mover (engine), a transmission, a control device having one or more electronic control units (ECUs), a GNSS unit, and a communication device. The tractor may further include an edge computer.

[0036] The GNSS unit is a GNSS receiver equipped with an antenna for receiving signals from GNSS satellites and a processing circuit. The GNSS unit receives GNSS signals transmitted from GNSS satellites such as GPS (Global Positioning System), GLONASS, Galileo, BeiDou, or Quasi-Zenith Satellite System (QZSS, e.g., Michibiki), and performs positioning based on the signals. The communication device equipped in the tractor can perform wired communication using CAN (Controller Area Network) or the like, or wireless communication compliant with the Bluetooth (registered trademark) standard and / or Wi-Fi (registered trademark) standard.

[0037] (Drone 400) The drone 400 includes, for example, an imaging device 410, an RTK (Real Time Kinematic GPS) positioning unit 420, an inertial measurement unit (IMU) 430, a driving device 440, a control device 450, a data generating device 460, and a communication I / F 470. The flight of the drone 400 can be controlled using, for example, the user's terminal 200.

[0038] The imaging device 410 has an image sensor such as an RGB image sensor or a multispectral sensor, a lens optical system, and a data processing circuit. The image sensor may be, for example, a CMOS sensor having pixels of several Mpx to tens of Mpx. The imaging device 410 is capable of acquiring a multispectral image or a vegetation index image, which may include, for example, an RGB image, a grayscale image, an RGB image, a red edge image, and a near-infrared image, including a field area. The imaging device 410 can acquire an image of one frame per second, for example.

[0039] The RTK positioning unit 420 includes, for example, a GNSS receiver, a Wi-Fi communication module, and a microcontroller. The RTK positioning unit outputs a signal indicating the position of the drone 400 in a world coordinate system, such as a geographic coordinate system fixed relative to the Earth. The RTK positioning unit can perform positioning with an accuracy of several centimeters. Position information including latitude, longitude, and altitude information is obtained by high-precision positioning.

[0040] The IMU 430 is equipped with an acceleration sensor and an angular acceleration sensor, and outputs a signal indicating the amount of movement and attitude of the drone 400 (or the imaging device 410).

[0041] The drive system 440 includes various devices required for flying the drone 400, such as an electric motor for drive and multiple rotors.

[0042] The control device 450 includes, for example, a flight control device such as a flight controller and a higher-level computer (companion computer). The control device 450 controls the operation of the drive device 440 based on output signals from the RTK positioning unit 420 and the IMU 430.

[0043] The data generating device 460 generates metadata including shooting information, for example, conforming to the Exif (Exchangeable image file format) standard. The metadata may include information such as shooting date and time, location information (geotag), posture information, image resolution, shutter speed, aperture (F value), and ISO sensitivity. The data generating device 460 processes data output from the imaging device 410, and can add metadata to the processed data to generate image data in a file format such as RAW, DNG, TIFF, GeoTIFF, or JPEG.

[0044] The communication I / F 470 is an interface for communicating between the drone 400 and an external device. For example, the communication I / F 470 may perform mobile communication such as LTE (Long Term Evolution) / 4G / 5G and / or wireless communication in accordance with the Wi-Fi (registered trademark) standard.

[0045] In one aspect, the system 1000 in the embodiment of the present disclosure includes a processing device that processes an image of a field including a plurality of crop rows viewed from above to detect the plurality of crop rows. The processing device 120 of the server 100, the edge computer of the agricultural machine 300, the processor 230 of the terminal 200, or at least two of these computing devices may cooperate to realize the functions of the processing device of the system 1000 described above. In the embodiment of the present disclosure, the processing device 120 of the server 100 functions as the processing device of the system 1000.

[0046] The processing device of the system 1000 is configured to generate a vegetation cover image by extracting vegetation cover that is a plurality of crop rows from an image, integrate pixel values ​​of a pixel group constituting the vegetation cover image along N integration lines (N is an integer equal to or greater than 2) of the vegetation cover image to obtain N integration values ​​related to the N integration lines, apply a filter to a one-dimensional array storing the N integration values ​​and perform a convolution operation to obtain N weighted integration values, and detect the plurality of crop rows based on the N weighted integration values. The processing device may be further configured to identify an area between adjacent crop rows based on the detection result of the crop rows, and transmit a control instruction to the agricultural machine when it detects weeds growing in the identified area.

[0047] [2. Example of implementation of crop row detection algorithm] FIG. 3 is a flow chart showing the steps of an exemplary implementation of the algorithm for detecting crop rows.

[0048] In the embodiment of the present disclosure, the crop of the crop row is paddy rice. However, the crop is not limited to paddy rice. The method for detecting a crop row in the embodiment of the present disclosure can be suitably applied to the detection of a crop row other than a crop cultivated by broadcasting.

[0049] In the embodiment of the present disclosure, a method for detecting multiple crop rows in a field is implemented in a computer such as the processing device of the server described above. The method causes the computer to execute the following steps: acquiring an image of the field viewed from above (step S10), generating a vegetation cover image by extracting vegetation covers, which are multiple crop rows, from the image (step S20), integrating pixel values ​​of pixel groups constituting the vegetation cover image along N integration lines of the vegetation cover image to obtain N integrated values ​​related to the N integration lines (step S30), applying a filter to a one-dimensional array storing the N integrated values ​​and performing a convolution operation to obtain N weighted integrated values ​​(step S40), and detecting multiple crop rows based on the N weighted integrated values ​​(step S50).

[0050] (Step S10) The processing device acquires an overhead image of a field including multiple crop rows. For example, the processing device acquires the overhead image of the field by accessing storage device 130 of server 100 and reading remote sensing image data stored in storage device 130. Hereinafter, the overhead image of the field is referred to as an "input image" to the system.

[0051] The input image is not limited to a remote sensing image, but may be a top-view image (or an overhead image) of the ground, obtained by, for example, performing homography transformation (planar projection transformation) on an image obtained by photographing a cultivated field from diagonally above with a camera mounted on an agricultural machine (e.g., a tractor). Alternatively, the storage device 130 of the server 100 described above may store data of a field map having position information of seedlings planted in the field. In this case, the processing device can generate an image comparable to a remote sensing image or a top-view image by visualizing the position information of the seedlings indicated by the data of the field map. The image generated in this way is also an example of an input image. FIG. 4A shows an example of an input image (multispectral image) including rice paddy rows.

[0052] (Step S20) The processing device then generates a vegetation cover image by extracting vegetation cover that is a plurality of crop rows from the input image. In other words, the processing device applies pre-processing to the input image to generate a vegetation cover image by extracting vegetation cover that is a plurality of crop rows. FIG. 4B shows an example of a vegetation cover image including vegetation cover that is a rice paddy row. The pre-processing includes, for example, Lab transform, Otsu's binarization, adaptive binarization, application of deep learning, Gaussian filter, and expansion / contraction processing.

[0053] When the input image is a multispectral image, the processing device generates a vegetation canopy image by, for example, applying a Lab transform and Otsu's binarization in this order to the input image. Alternatively, the processing device may generate a vegetation canopy image by applying a Gaussian filter and adaptive binarization in this order to the input image. It is preferable that the processing device applies a dilation / contraction process to the binary image after applying Otsu's binarization or adaptive binarization until the average value of the pixels of the image falls within a predetermined range. By applying the dilation / contraction process, it is possible to improve the detection accuracy of the crop row, which will be described later, regardless of the growth state or growth period of the crop.

[0054] Fig. 5A shows an example of an input image before preprocessing is applied, and Fig. 5B shows an example of a binary image after preprocessing is applied. Fig. 6A shows another example of an input image before preprocessing is applied, and Fig. 6B shows another example of a binary image after preprocessing is applied. Both the binary images shown in Fig. 5B and Fig. 6B are images after dilation / erosion processing has been applied.

[0055] An example of deep learning is segmentation, such as semantic segmentation, instance segmentation, or panoptic segmentation, which is a combination of these two algorithms. By applying, for example, semantic segmentation to an input image instead of binarization, the processing device can classify the input image into two classes: vegetation cover areas and non-vegetation cover areas.

[0056] (Step S30) The processing device then integrates pixel values ​​of pixels constituting the vegetation cover image along N integration lines of the vegetation cover image to obtain N integrated values ​​associated with the N integration lines. The N integration lines will be described in detail later.

[0057] Fig. 7 is a flowchart showing an example of a specific processing procedure included in step S30. Fig. 8A to Fig. 8D are diagrams showing examples of rotated vegetation cover images obtained by rotating the vegetation cover image by a certain rotation angle around the origin of a two-dimensional coordinate system.

[0058] First, the processing device rotates the vegetation cover image 70 at a plurality of rotation angles around the origin O of the two-dimensional coordinate system to generate a plurality of (M pieces: M is an integer of 2 or more) rotated vegetation cover images 71 (step S31). The two-dimensional coordinate system is defined by a first axis and a second axis perpendicular to the first axis. In the examples shown in Figures 8A to 8D, the Y axis is the first axis and the X axis is the second axis.

[0059] FIG. 8A shows a rotated vegetation cover image 71A obtained by rotating the vegetation cover image 70 counterclockwise at a rotation angle of 0° around the origin O of the two-dimensional coordinate system. In other words, the original vegetation cover image 70 before rotation is shown. FIG. 8B shows a rotated vegetation cover image 71B obtained by rotating the vegetation cover image 70 counterclockwise at a rotation angle of 75° around the origin O of the two-dimensional coordinate system. FIG. 8C shows a rotated vegetation cover image 71C obtained by rotating the vegetation cover image 70 counterclockwise at a rotation angle of 90° around the origin O of the two-dimensional coordinate system. FIG. 8D shows a rotated vegetation cover image 71D obtained by rotating the vegetation cover image 70 counterclockwise at a rotation angle of 105° around the origin O of the two-dimensional coordinate system.

[0060] The processing device may rotate the vegetation cover image 70 counterclockwise, for example, at 0.5° intervals, to generate 360 ​​(180 / 0.5) rotated vegetation cover images 71. Alternatively, the processing device may rotate the vegetation cover image 70 counterclockwise, for example, at 1° intervals, to generate 180 (180 / 1) rotated vegetation cover images 71. However, the rotation angle is not particularly limited.

[0061] In an embodiment of the present disclosure, a processing device applies a discrete Fourier transform (DFT) to a vegetation canopy image to generate a frequency spectral image, and from the spectral image, estimates the angle between the direction in which multiple crop rows extend in a two-dimensional coordinate system and a first axis.

[0062] Fig. 9 is a diagram showing an example of a frequency spectrum image generated by applying DFT to a vegetation canopy image. A vegetation canopy image 70 is shown on the left side of Fig. 9, and an example of a spectrum image 72 is shown on the right side. In the vegetation canopy image 70 shown in Fig. 9, a plurality of lines 80 along a plurality of crop rows are shown by dashed lines. For simplicity, the plurality of lines 80 are all drawn parallel to each other, in other words, the angle at which each line 80 intersects with the Y-axis is the same, but in reality, the direction in which each crop row extends may vary.

[0063] 9, a line 81 extending in a direction intersecting the direction in which the lines 80 extend is indicated by a dotted line. In the spectral image 72, the change in luminance is small in the direction in which the lines 80 of the crop rows extend, and the change in luminance is large along the line 81 extending in a direction intersecting the direction in which the lines 80 extend, so that the amplitude of the high frequency region is large in the direction in which the line 81 extends. Theoretically, the change in luminance is largest along the line 81 extending in a direction perpendicular to the direction in which the lines 80 extend, so that the amplitude of the high frequency region is largest in that direction.

[0064] The processing device applies a two-dimensional DFT to the vegetation cover image 70, and integrates the spectrum of frequencies corresponding to the distance between adjacent crop rows from the spectral image 72 at every fixed angle (e.g., 3.0°). In the example shown in FIG. 9, the processing device integrates the spectrum of frequencies at every fixed angle, and determines that the integrated value of the spectrum of frequencies along the direction in which the line 81 extends is the largest among the multiple integrated values. From the direction in which the line 81 extends, the processing device can estimate the angle between the direction in which the multiple crop rows extend in the two-dimensional coordinate system and the X-axis or Y-axis. In this way, the processing device can narrow down the direction in which the crop rows extend from the spectral image, for example, based on the spacing between the crop rows set at the time of transplanting.

[0065] In the embodiment of the present disclosure, the processing device may further determine a plurality of rotation angles with reference to the estimated angle, and generate a plurality of rotated vegetation cover images by rotating the vegetation cover image at the plurality of rotation angles around the origin of the two-dimensional coordinate system. For example, the processing device may generate 21 rotated vegetation cover images by rotating the vegetation cover image at 0.5° intervals within a range of ±5° based on the estimated angle. By narrowing down the candidates for the plurality of rotation angles based on the estimated angle, the number of rotated vegetation cover images to be generated can be reduced, thereby reducing the computation load of the integrated value along the pixel line described later.

[0066] Next, the processing device calculates N integrated values ​​for each of the M rotated vegetation cover images 71 by integrating the pixel values ​​of the pixel groups constituting the rotated vegetation cover image 71 along the N integration lines (step S32).

[0067] Fig. 10A is a schematic diagram illustrating a plurality of integrated lines 82 of the vegetation cover image 70 or the rotated vegetation cover image 71A shown in Fig. 8A. Fig. 10B is a schematic diagram illustrating a plurality of integrated lines 82 of the rotated vegetation cover image 71B shown in Fig. 8B.

[0068] As shown in FIG. 10A, a number of (N: N is an integer of 2 or more) integration lines 82 extending parallel to the Y-axis, which is the first axis, are drawn at equal intervals on the image. For example, when the image size is 1920×1080, 1080 integration lines 82 corresponding to the number of pixels in the X-axis direction may be drawn at equal intervals on the 1080 pixels. A number of integration lines 82 extending parallel to the Y-axis, which is the first axis, may also be drawn on the rotated vegetation cover image. As illustrated in FIG. 10B, a number N of integration lines 82 determined according to the horizontal size in the X-axis direction of the circumscribing rectangle 75 of the rotated vegetation cover image 71B are drawn at equal intervals on the rotated vegetation cover image 71B.

[0069] The processing device calculates N accumulated values ​​by accumulating the pixel values ​​of the pixel groups constituting the rotated vegetation cover image 71 along N accumulation lines 82 for each of the M rotated vegetation cover images 71. In this way, the processing device calculates an accumulated value by adding up the number of pixels in the vertical direction of the rotated vegetation cover image along the accumulation lines. As described above, when the rotated vegetation cover image or the vegetation cover image is a binary image, the accumulated value of the pixel values ​​along the accumulation line is equal to the total number of pixels corresponding to white that exist along the accumulation line. As illustrated in FIG. 10B, when accumulating pixel values ​​along the accumulation line 82 of the rotated vegetation cover image 71, the processing device places a dummy pixel with a pixel value of 0 in the area of ​​the circumscribing rectangle 75 other than the rotated vegetation cover image 71 at coordinates determined based on the pixels constituting the rotated vegetation cover image 71. Then, the processing device accumulates the pixel values ​​of the pixels including the dummy pixel along the accumulation line 82. The processing device stores the calculated N accumulated values ​​in a one-dimensional array.

[0070] Please refer to Fig. 8A to Fig. 8D again. Each of Fig. 8A to Fig. 8D illustrates a portion of a waveform in which N integrated values ​​along N integrated lines are plotted. The vertical axis indicates the integrated value, and the horizontal axis indicates the X coordinate of the integrated line 82, that is, the intersection ρ of the integrated line 82 and the X axis. Among the rotated vegetation cover images 71A to 71D, the amplitude of the waveform of the integrated value obtained from the rotated vegetation cover image 71C is the largest, and the amplitude of the waveform of the integrated value obtained from the rotated vegetation cover image 71A or the original vegetation cover image 70 is the smallest. In this way, the amplitude of the waveform changes depending on the rotation angle of the image.

[0071] 7, the processing device repeatedly executes the process of step S32 until it acquires a set of N integrated values ​​of the M rotated vegetation cover images 71 (step S33). In other words, the processing device repeatedly executes the process of step S32 until it acquires all N integrated values ​​of the M rotated vegetation cover images 71.

[0072] (Step S40) Next, the processing device applies a filter to the one-dimensional array storing the N integrated values ​​for each of the multiple rotated vegetation cover images 71, and performs a convolution operation (moving average or weighted average) to obtain N weighted integrated values.

[0073] Fig. 11 is a flowchart showing an example of a specific process procedure included in step S40. Fig. 12 is a schematic diagram for explaining the convolution operation.

[0074] The processing device applies a filter to a one-dimensional array storing the N accumulated values ​​for each of the M rotated vegetation cover images 71, and performs a convolution operation to obtain N weighted accumulated values ​​(step S41).

[0075] In the embodiment of the present disclosure, as illustrated in FIG. 12, weighting is performed such that the weight increases as the distance approaches the center of the crop. Such weighting allows the center of the crop row to be detected with high accuracy. If the center of the crop row can be detected with high accuracy, the detection accuracy of the crop row can be improved. An example of a filter used for the convolution operation is a linear filter or a Gaussian filter. FIG. 12 shows an example of a linear filter.

[0076] The shape and size of the crop may vary depending on the type of crop. Also, the shape and size of the crop may vary depending on the growing season of the crop. The weight of the filter in the embodiment of the present disclosure may vary depending on the type of crop or growing season. Also, the filter size ρ width may also vary depending on the type of crop or the growing season. For example, the system 1000 may manage a work plan including information on the type of crop. The data of the work plan may be stored, for example, in the storage device 130 of the server 100. Table data describing the relationship between the type of crop or the growing season and the weight and size may be stored in the storage device 130. The processing device may obtain the work plan and the table data from the storage device 130, and set the weight or size of the filter according to the information on the type of crop included in the work plan data by referring to the table. The processing device may also estimate the time from, for example, date or year / month information managed by the system 1000, and set the weight or size of the filter based on the estimation result. However, the weight and size of the filter may be set by the user operating the terminal 200 and inputting it into the system 1000.

[0077] By changing the size or weight of the filter depending on the type of crop or growing season, the effect of environmental changes such as the type of crop or growing season on crop row detection can be reduced, thereby improving the accuracy and robustness of crop row detection.

[0078] The processing device repeatedly executes the process of step S41 (step S42) until it acquires a set of N weighted integrated values ​​of the M rotated vegetation cover images 71. In other words, the processing device repeatedly executes the process of step S41 until it acquires all N weighted integrated values ​​of the M rotated vegetation cover images 71.

[0079] (Step S50) Referring again to the flow chart of Figure 3, the processing device detects a number of crop rows based on a set of N weighted integrated values ​​obtained from a number of rotated vegetation cover images 71 acquired in step 42.

[0080] In the embodiment of the present disclosure, the weighted integration value is expressed by a function f(θ, ρ), where θ and ρ are the rotation angle and the X coordinate of the integration line 82, respectively, as described above.

[0081] 13 is a schematic diagram showing an outline of a list obtained by sorting the N weighted integrated values ​​of each rotated vegetation cover image included in the set of weighted integrated values ​​in descending order. When there are a total of M rotated vegetation cover images, the processing device JPEG2025055365000002.jpgSort the 14,170 weighted accumulated values ​​in descending order.

[0082] Assume that the angle estimated by the processing device in step S30 is 80°. In FIG. 13, N weighted integrated values ​​obtained from each rotated vegetation cover image 71 rotated at 0.5° intervals within a range of ±5° (i.e., 75°≦θ≦85°) based on the angle 80° are expressed by a function f(θ,ρ). The number M of rotated vegetation cover images in this example is 21. The processing device in the embodiment of the present disclosure sorts the N weighted integrated values ​​of each of the M rotated vegetation cover images 71 included in the set of weighted integrated values ​​in descending order, and generates a list obtained from the sorting results. The processing device adopts the weighted integrated values, for example, from the top of the list in order.

[0083] In the list illustrated in FIG. 13, the function f(75,-50)=8.0 indicating the weighted integrated value of the rotation angle θ=75 and the X coordinate of the integration line=-50 is at the top, followed by the function f(85,-40)=7.2 indicating the weighted integrated value of the rotation angle θ=85 and the X coordinate of the integration line=-40, and then the function f(75.5,-60)=6.5 indicating the weighted integrated value of the rotation angle θ=75.5 and the X coordinate of the integration line=-60. The processing device adopts the weighted integrated values ​​of the functions f(75,-50), f(85,-40) and f(75.5,-60) in order from the top of the list. The processing device may adopt the weighted integrated value based on the list while taking into account the constraint condition.

[0084] The processing device in the embodiment of the present disclosure estimates that a crop row formed along an integration line located on a coordinate ρ exists in the rotated vegetation canopy image 71 at a rotation angle θ indicated by the adopted function f(θ,ρ). For example, the processing device identifies a crop row corresponding to one function f(θ,ρ) from among a plurality of crop rows. Finally, the processing device may obtain position information of the crop row existing in the vegetation canopy image or the input image using the estimation result of the crop row estimated from the function f(θ,ρ) based on the rotated vegetation canopy image 71, and detect the crop row based on the position information.

[0085] According to the embodiment of the present disclosure, since the angle at which the direction in which the crop rows extend and the X-axis or Y-axis intersect is obtained for each crop row, it is possible to detect the directions in which multiple crop rows extend in the vegetation canopy image or the input image with high accuracy. In particular, even if an error occurs in the generation of the vegetation canopy image in the process of step S20, robust crop row detection can be achieved by the subsequent process.

[0086] Another implementation example of the method for detecting a crop row will be described with reference to Fig. 14 and Fig. 15. The processing device may detect the crop row based on the vegetation coverage ratio instead of the set of weighted integrated values. The processes of steps S30 and S40 among steps S10 to S50 of the processing procedure shown in Fig. 3 are different from the processes according to the implementation example described above. The differences will be described in detail below.

[0087] Fig. 14 is a flowchart showing an example of a specific process procedure included in step S30 according to another implementation example. Fig. 15 is a flowchart showing an example of a specific process procedure included in step S40 according to another implementation example.

[0088] The processing device generates M rotated vegetation cover images by executing the process of step S31 described above, and then calculates N first and second integrated values ​​for each rotated vegetation cover image (step S34). Specifically, the processing device obtains a first integrated value by integrating the pixel values ​​of the pixel groups constituting the rotated vegetation cover image 71 along each integration line for each of the M rotated vegetation cover images 71, and obtains a second integrated value by integrating the number of pixel groups of the rotated vegetation cover image located on each integration line, thereby obtaining N first integrated values ​​and N second integrated values. The N first integrated values ​​correspond to the N integrated values ​​described above. When the vegetation cover image is a binary image, the first integrated value corresponds to the number of pixels corresponding to the vegetation cover, and the second integrated value corresponds to the number of all pixels located on the integration line, including the number of pixels not corresponding to the vegetation cover.

[0089] The processing device repeatedly executes the process of step S34 until a set of N first integrated values ​​and N second integrated values ​​of M rotated vegetation cover images 71 is obtained, as shown in FIG. 14 (step S35).

[0090] Next, as shown in FIG. 15, the processing device applies a filter to a one-dimensional array storing each of the N first accumulated values ​​and the N second accumulated values ​​for each rotated vegetation cover image 71 and performs a convolution operation to obtain N weighted first accumulated values ​​and N weighted second accumulated values ​​(step S43).

[0091] Next, the processing device obtains the vegetation coverage ratio of the weighted first integrated value relative to the weighted second integrated value associated with each of the N integrated lines, and obtains N vegetation coverage ratios for the N integrated lines (step S44). The vegetation coverage ratio is represented by the ratio of the weighted first integrated value / the weighted second integrated value. The processing device obtains the vegetation coverage ratio for each integrated line, and obtains N vegetation coverage ratios for the N integrated lines.

[0092] 15, the processing device repeatedly executes the processes of steps S43 and S44 until it acquires a set of N vegetation coverage ratios of M rotated vegetation cover images 71 (step S45). In other words, the processing device repeatedly executes the processes of steps S43 and S44 until it acquires all N vegetation coverage ratios of M rotated vegetation cover images 71.

[0093] As described above, the processing device can detect multiple crop rows based on a set of N vegetation cover ratios obtained from multiple rotated vegetation cover images. Even in the case where image rotation processing is involved as in the implementation example of the present disclosure, the vegetation cover ratio can be used to improve the accuracy of estimating crop rows based on rotated images.

[0094] [3. Example of implementation of algorithm for detecting weeds growing between crop rows] The above-mentioned method may further include a step of, after detecting the crop rows, identifying an area between adjacent crop rows based on the detection result of the crop rows, and transmitting a control instruction to the agricultural machine when weeds growing in the identified area are detected.

[0095] FIG. 16 is a flowchart showing a processing procedure according to an implementation example of an algorithm for transmitting a control instruction to an agricultural machine when a weed is detected.

[0096] The processing procedure shown in Fig. 16 includes the processing of steps S10 to S80. Among steps S10 to S80, the processing of steps S10 to S50 has been described with reference to Fig. 3. Specific processing of steps S60, S70, and S80 will be described below.

[0097] The processing device may be configured to detect crop rows according to the procedure from steps S10 to S50, identify areas between adjacent crop rows based on the results of the crop row detection, and transmit weed control instructions to the agricultural machine when it detects weeds growing in the identified areas.

[0098] (Step S60) The processing device identifies an area between adjacent crop rows based on the crop row detection results, for example, the processing device identifies an area from the vegetation canopy image that is located between two detection lines along the adjacent crop rows.

[0099] (Step S70) The processing device detects weeds growing in the area identified in step S60. As described above, the crop in the embodiment of the present disclosure is rice. Barnyard grass grows as a weed in the area between the rows of rice. The shape of rice closely resembles that of barnyard grass. In particular, barnyard grass before ears appear is difficult to distinguish from rice.

[0100] Fig. 17 is an enlarged view of a portion of the vegetation cover image 70. Fig. 18 is a view illustrating a detection result image showing the detection results of crop rows and weeds.

[0101] The processing device in the embodiment of the present disclosure applies a labeling process to the vegetation cover image 70 to generate a labeled image. The processing device may apply an expansion process before applying the labeling process to the vegetation cover image. The expansion process may improve the accuracy of weed detection. The processing device determines, from the labeling image, a pixel region consisting of one or more connected components (hereinafter referred to as a "connected region") that exists in the region identified in step S60. The processing device detects weeds growing in the identified region, for example, based on shape characteristic parameters that characterize the shape of the connected region.

[0102] Examples of the shape characteristic parameters include the centroid coordinates of the shape of the connected region and the area of ​​the connected region. For example, the processing device compares the distance from the centroid coordinates of the shape of the connected region to the crop row with a threshold value, and detects weeds based on the comparison result. The threshold value may be determined, for example, by providing a learning data set to a decision tree and training it. As another example, the processing device may compare the area of ​​the connected region with a threshold value, and detect weeds based on the comparison result.

[0103] FIG. 17 shows how objects that are candidates for weeds are detected in the area between the detection lines 83 of the crop rows. For example, for object 91, the processing device determines that object 91 is a weed based on the result of comparing the shape characteristic parameters with a threshold value. For object 92, the area of ​​the connected region is smaller than the threshold value. In this case, the processing device determines, for example, that object 92 is noise. For object 93, the distance from the center of gravity coordinate of the shape of the connected region to the crop row is smaller than the threshold value. In this case, the processing device determines that object 93 is close to rice, and determines object 93 to be, for example, the lower leaves of rice, and excludes it from the candidates for weeds.

[0104] For weeds that can be distinguished from crops by visual inspection, for example, a machine learning model for object detection can be used. However, as described above, the shape of rice closely resembles the shape of barnyard grass, making it extremely difficult to distinguish between rice and barnyard grass. Even if a machine learning model for object detection is used to detect barnyard grass, the result is the same as when they are distinguished by visual inspection. On the other hand, according to the embodiment of the present disclosure, even when it is difficult to distinguish between crops and weeds by visual inspection, it is possible to accurately detect weeds. For example, it may be easy to distinguish between rice and barnyard grass, or between rice and weedy rice.

[0105] 18, the processing device may generate a detection result image 78 by superimposing a rectangular area adjacent to the detection line of the crop row and / or the connected area of ​​the weeds on the vegetation cover image, and may display the detection result image 78 on the display device 220 of the terminal 200, for example. Alternatively, the processing device may store data of the detection result image 78 in the storage device 130 of the server 100, for example.

[0106] (Step S80) When the processing device detects weeds in step S70, it transmits a control instruction to an agricultural machine such as a pest control machine. The agricultural machine performs a pest control task in response to the control instruction transmitted from the processing device. The agricultural machine can be operated with or without a human driver, and can be operated either manually or automatically. For example, an agricultural machine capable of automatic operation can automatically start a pest control task in response to the pest control instruction transmitted from the processing device.

[0107] According to the agricultural support system of the embodiment of the present disclosure, it is possible to improve the accuracy and robustness of crop row detection. For example, it is possible to detect all crop rows included in a multispectral image acquired by taking pictures of a farm field by a drone at once.

[0108] The agricultural support system, crop row or weed detection method according to the embodiments of the present disclosure can also be applied to crops other than rice, for example soybeans or vegetables such as cabbage.

[0109] In the case of crops other than rice, it is possible to determine the growth status by observing the time-dependent changes in the vegetation cover of the crop after identifying the crop row using the method disclosed herein, or to determine that the vegetation cover other than the crop is a weed. In this case, it is possible to use the determination result to identify areas of poor growth or areas where weeds have appeared and take preventive measures. The growth status can also be used as an information source for yield prediction. As another example, in a field where a crop rotation system is used, a crop cultivated in the previous season may grow in the current season. In that case, there is a concern that the current season's crop will grow poorly due to the spread of pests and diseases and / or nutritional competition mediated by the previous season's crop. However, by using the crop row detection method disclosed herein, it is possible to detect the emergence of crops in the previous season early and take preventive measures.

[0110] A system providing various functions in the embodiments of the present disclosure can also be attached to an agricultural machine that does not have those functions. Such a system can be manufactured and sold independently of the agricultural machine. A computer program used in such a system can also be manufactured and sold independently of the agricultural machine. The computer program can be provided, for example, by being stored in a computer-readable non-transitory storage medium. The computer program can also be provided by downloading via a telecommunication line (for example, the Internet).

[0111] As described above, the present disclosure includes an agricultural support system and a crop row detection method described in the following items.

[0112] [Item 1] a processing device for processing an image of a top view of a field including a plurality of rows of crops to detect the plurality of rows of crops; The processing device includes: generating a vegetation cover image by extracting the vegetation cover, which is the plurality of crop rows, from the image; integrating pixel values ​​of a pixel group constituting the vegetation cover image along N integration lines (N is an integer equal to or greater than 2) of the vegetation cover image to obtain N integration values ​​related to the N integration lines; Applying a filter to the one-dimensional array storing the N integrated values ​​and performing a convolution operation to obtain N weighted integrated values; An agricultural support system configured to detect the plurality of crop rows based on the N weighted integrated values.

[0113] [Item 2] 2. The agricultural support system according to item 1, wherein the filter weights vary depending on the type of crop or the growing season.

[0114] [Item 3] 2. The agricultural support system according to item 1, wherein the size of the filter varies depending on the type of crop or growing season.

[0115] [Item 4] The agricultural support system according to any one of items 1 to 3, wherein the filter is a linear filter or a Gaussian filter.

[0116] [Item 5] The processing device includes: generating a plurality of rotated vegetation cover images by rotating the vegetation cover image at a plurality of rotation angles around an origin of a two-dimensional coordinate system defined by a first axis parallel to a direction in which the N integration lines extend and a second axis perpendicular to the first axis; For each of the plurality of rotated vegetation cover images, integrating pixel values ​​of a pixel group constituting the rotated vegetation cover image along the N integration lines to obtain the N integrated values; applying the filter to the one-dimensional array storing the N integrated values ​​to perform a convolution operation to obtain the N weighted integrated values; 5. The agricultural support system according to any one of claims 1 to 4, wherein the plurality of crop rows are detected based on a set of the N weighted integrated values ​​obtained from the plurality of rotated vegetation cover images.

[0117] [Item 6] The processing device includes: generating a plurality of rotated vegetation cover images by rotating the vegetation cover image at a plurality of rotation angles around an origin of a two-dimensional coordinate system defined by a first axis parallel to a direction in which the N integration lines extend and a second axis perpendicular to the first axis; For each of the plurality of rotated vegetation cover images, integrating pixel values ​​of a pixel group constituting the rotated vegetation cover image along each integration line to obtain a first integrated value, and integrating the number of pixel groups of the rotated vegetation cover image located on each integration line to obtain a second integrated value, thereby obtaining N first integrated values ​​and N second integrated values, respectively; applying the filter to a one-dimensional array storing each of the N first accumulated values ​​and the N second accumulated values ​​to perform a convolution operation, thereby obtaining N weighted first accumulated values ​​and N weighted second accumulated values; determining a vegetation coverage ratio of the weighted first integrated value relative to the weighted second integrated value associated with each of the N integrated lines to obtain N vegetation coverage ratios for the N integrated lines; 5. The agricultural support system according to any one of claims 1 to 4, wherein the plurality of crop rows are detected based on the set of N vegetation coverage ratios obtained from the plurality of rotated vegetation coverage images.

[0118] [Item 7] 7. The agricultural support system according to item 5 or 6, wherein the processing device applies a discrete Fourier transform to the vegetation cover image to generate a frequency spectrum image, and estimates from the spectrum image an angle between a direction in which the plurality of crop rows extend in the two-dimensional coordinate system and the first axis.

[0119] [Item 8] The agricultural support system described in item 7, wherein the processing device determines the multiple rotation angles by referring to the estimated angle, and rotates the vegetation cover image at the multiple rotation angles around the origin of the two-dimensional coordinate system to generate the multiple rotated vegetation cover images.

[0120] [Item 9] The agricultural support system according to any one of claims 1 to 8, wherein the processing device applies Lab transformation or deep learning to the image to generate the vegetation cover image in which the vegetation cover is extracted.

[0121] [Item 10] The agricultural support system according to item 1, wherein the processing device is further configured to identify an area between adjacent crop rows based on the detection result of the crop row, and to transmit a control instruction when detecting weeds growing in the identified area.

[0122] [Item 11] The processing device includes: applying a labeling process to the vegetation cover image to generate a labeling image; A pixel region consisting of one or more connected components present in the specified region is obtained from the labeling image; Item 11. The agricultural support system according to item 10, wherein the weeds growing in the identified region are detected based on shape characteristic parameters that characterize the shape of the pixel region.

[0123] [Item 12] Item 12. The agricultural support system according to item 10 or 11, comprising an agricultural machine that performs work for pest control in response to the pest control instruction transmitted from the processing device.

[0124] [Item 13] 13. The agricultural support system according to any one of items 1 to 12, wherein the crop in the crop row is paddy rice and the weed is barnyard grass.

[0125] [Item 14] 14. An agricultural support system according to any one of claims 1 to 13, comprising an unmanned aerial vehicle that photographs the field from above the field to acquire a multispectral image.

[0126] [Item 15] 1. A computer-implemented method for detecting a plurality of crop rows in a field, comprising: Obtaining an image of the field viewed from above; generating a vegetation cover image by extracting the vegetation cover that is the plurality of crop rows from the image; integrating pixel values ​​of a pixel group constituting the vegetation cover image along N integration lines (N is an integer equal to or greater than 2) of the vegetation cover image to obtain N integration values ​​related to the N integration lines; Applying a filter to the one-dimensional array storing the N integrated values ​​to perform a convolution operation to obtain N weighted integrated values; detecting the plurality of crop rows based on the N weighted integrated values; A method for getting a computer to run

[0127] [Item 16] 16. The method according to claim 15, wherein the filter weights vary depending on the type of crop or the growing season.

[0128] [Item 17] Item 16. The method according to item 15, wherein the size of the filter varies depending on the type of crop or growing season.

[0129] [Item 18] 18. The method according to any one of items 15 to 17, wherein the filter is a linear filter or a Gaussian filter.

[0130] [Item 19] generating a plurality of rotated vegetation cover images by rotating the vegetation cover image at a plurality of rotation angles around an origin of a two-dimensional coordinate system defined by a first axis parallel to a direction in which the N integration lines extend and a second axis perpendicular to the first axis; obtaining the N integrated values ​​includes integrating pixel values ​​of a pixel group constituting each of the plurality of rotated vegetation cover images along the N integration lines to obtain the N integrated values; obtaining the N weighted integrated values ​​includes obtaining the N weighted integrated values ​​by applying the filter to a one-dimensional array storing the N integrated values ​​for each of the plurality of rotated vegetation cover images and performing a convolution operation; 19. The method of any one of claims 15 to 18, further comprising detecting the plurality of crop rows based on the set of N weighted integrated values ​​obtained from the plurality of rotated vegetation cover images.

[0131] [Item 20] generating a plurality of rotated vegetation cover images by rotating the vegetation cover image at a plurality of rotation angles around an origin of a two-dimensional coordinate system defined by a first axis parallel to a direction in which the N integration lines extend and a second axis perpendicular to the first axis; The calculation of the N integrated values ​​is carried out for each of the plurality of rotated vegetation cover images, integrating pixel values ​​of a pixel group constituting the rotated vegetation cover image along each integration line to obtain a first integrated value, and integrating the number of pixel groups of the rotated vegetation cover image located on each integration line to obtain a second integrated value, thereby obtaining N first integrated values ​​and N second integrated values, respectively; applying the filter to a one-dimensional array storing each of the N first accumulated values ​​and the N second accumulated values ​​to perform a convolution operation, thereby obtaining N weighted first accumulated values ​​and N weighted second accumulated values; determining a vegetation coverage ratio of the weighted first integrated value relative to the weighted second integrated value associated with each of the N integrated lines to obtain N vegetation coverage ratios for the N integrated lines; 19. The method of any one of items 15 to 18, further comprising detecting the plurality of crop rows based on the set of N vegetation cover ratios for the plurality of rotated vegetation cover images.

[0132] [Item 21] 21. The method of claim 19 or 20, further comprising: applying a discrete Fourier transform to the vegetation canopy image to generate a frequency spectrum image; and estimating from the spectrum image an angle between a direction in which the plurality of crop rows extend in the two-dimensional coordinate system and the first axis.

[0133] [Item 22] 22. The method of claim 21, further comprising: determining the plurality of rotation angles by reference to the estimated angles; and rotating the vegetation cover image by the plurality of rotation angles around the origin of the two-dimensional coordinate system to generate the plurality of rotated vegetation cover images.

[0134] [Item 23] 23. The method according to any one of items 15 to 22, wherein generating the vegetation cover image includes applying a Lab transform or deep learning to the image to generate the vegetation cover image in which the vegetation cover is extracted.

[0135] [Item 24] Item 16. The method according to item 15, further comprising: identifying an area between adjacent crop rows based on the detection result of the crop row; and transmitting a control instruction to an agricultural machine when detecting weeds growing in the identified area. [Industrial Applicability]

[0136] An embodiment of the present disclosure can be used, for example, in an agricultural support system that manages crop growth conditions on the cloud. [Explanation of symbols]

[0137] 100...server (server computer), 110...communication I / F, 120...processing device, 121...processor, 122...ROM, 123...RAM, 130...storage device, 200...terminal, 210...input device, 220...display device, 230...processor, 240...ROM, 250...RAM, 260...storage device, 270...communication I / F, 300...agricultural machine, 400...drone, 410...imaging device, 420...RTK positioning unit, 420...positioning unit, 440...driving device, 450...control device, 460...data generating device, 470...communication I / F, 1000...agricultural support system

Claims

1. a processing device for processing an image of a top view of a field including a plurality of rows of crops to detect the plurality of rows of crops; The processing device includes: generating a vegetation cover image by extracting the vegetation cover, which is the plurality of crop rows, from the image; integrating pixel values ​​of a pixel group constituting the vegetation cover image along N integration lines (N is an integer equal to or greater than 2) of the vegetation cover image to obtain N integration values ​​related to the N integration lines; Applying a filter to the one-dimensional array storing the N integrated values ​​to perform a convolution operation to obtain N weighted integrated values; An agricultural support system configured to detect the plurality of crop rows based on the N weighted integrated values.

2. The agricultural support system according to claim 1 , wherein the filter weights vary depending on the type or growing season of the crop.

3. The agricultural support system according to claim 1 , wherein the size of the filter varies depending on the type of crop or the growing season.

4. The agricultural support system according to claim 1 , wherein the filter is a linear filter or a Gaussian filter.

5. The processing device includes: generating a plurality of rotated vegetation cover images by rotating the vegetation cover image at a plurality of rotation angles around an origin of a two-dimensional coordinate system defined by a first axis parallel to a direction in which the N integration lines extend and a second axis perpendicular to the first axis; For each of the plurality of rotated vegetation cover images, integrating pixel values ​​of a pixel group constituting the rotated vegetation cover image along the N integration lines to obtain the N integrated values; applying the filter to a one-dimensional array storing the N integrated values ​​to perform a convolution operation to obtain the N weighted integrated values; The agricultural support system according to claim 1 , wherein the plurality of crop rows are detected based on the set of N weighted integrated values ​​obtained from the plurality of rotated vegetation cover images.

6. The processing device includes: generating a plurality of rotated vegetation cover images by rotating the vegetation cover image at a plurality of rotation angles around an origin of a two-dimensional coordinate system defined by a first axis parallel to a direction in which the N integration lines extend and a second axis perpendicular to the first axis; For each of the plurality of rotated vegetation cover images, a first integrated value is obtained by integrating pixel values ​​of a pixel group constituting the rotated vegetation cover image along each integration line, and a second integrated value is obtained by integrating the number of pixel groups of the rotated vegetation cover image located on each integration line, thereby obtaining N first integrated values ​​and N second integrated values, respectively; applying the filter to a one-dimensional array storing the N first accumulated values ​​and the N second accumulated values, and performing a convolution operation to obtain N weighted first accumulated values ​​and N weighted second accumulated values; determining a vegetation coverage ratio of the weighted first integrated value relative to the weighted second integrated value associated with each of the N integrated lines to obtain N vegetation coverage ratios for the N integrated lines; The agricultural support system according to claim 1 , wherein the plurality of crop rows are detected based on the set of N vegetation cover ratios obtained from the plurality of rotated vegetation cover images.

7. 6. The agricultural support system according to claim 5, wherein the processing device applies a discrete Fourier transform to the vegetation cover image to generate a frequency spectrum image, and estimates from the spectrum image an angle between a direction in which the plurality of crop rows extend in the two-dimensional coordinate system and the first axis.

8. The agricultural support system of claim 7, wherein the processing device determines the multiple rotation angles by referring to the estimated angle, and rotates the vegetation cover image at the multiple rotation angles around the origin of the two-dimensional coordinate system to generate the multiple rotated vegetation cover images.

9. The agricultural support system according to claim 1 , wherein the processing device applies Lab conversion or deep learning to the image to generate the vegetation cover image in which the vegetation cover is extracted.

10. The agricultural support system according to claim 1 , wherein the processing device is further configured to identify an area between adjacent crop rows based on the detection result of the crop rows, and to transmit a control instruction when detecting weeds growing in the identified area.

11. The processing device includes: applying a labeling process to the vegetation cover image to generate a labeling image; A pixel region composed of one or more connected components existing in the specified region is obtained from the labeling image; The agricultural support system according to claim 10, wherein the weeds growing in the identified region are detected based on shape characteristic parameters that characterize the shape of the pixel region.

12. The agricultural support system according to claim 10 or 11, further comprising an agricultural machine that performs work for pest control in response to the pest control instruction transmitted from the processing device.

13. 4. The agricultural support system according to claim 1, wherein the crop in the crop row is paddy rice, and the weeds are barnyard grass.

14. The agricultural support system according to claim 1 , further comprising an unmanned aerial vehicle that photographs the farm field from above the farm field to acquire a multispectral image.

15. 1. A computer-implemented method for detecting a plurality of crop rows in a field, comprising: Obtaining an image of the field viewed from above; generating a vegetation cover image by extracting the vegetation cover that is the plurality of crop rows from the image; integrating pixel values ​​of a pixel group constituting the vegetation cover image along N integration lines (N is an integer equal to or greater than 2) of the vegetation cover image to obtain N integration values ​​related to the N integration lines; Applying a filter to the one-dimensional array storing the N integrated values ​​to perform a convolution operation to obtain N weighted integrated values; detecting the plurality of crop rows based on the N weighted integrated values; A method for getting a computer to run

16. The method of claim 15 , wherein the filter weights vary by crop type or growing season.

17. 16. The method of claim 15, wherein the size of the filter varies with crop type or growing season.

18. The method according to any one of claims 15 to 17, wherein the filter is a linear filter or a Gaussian filter.

19. generating a plurality of rotated vegetation cover images by rotating the vegetation cover image at a plurality of rotation angles around an origin of a two-dimensional coordinate system defined by a first axis parallel to a direction in which the N integration lines extend and a second axis perpendicular to the first axis; The step of calculating the N integrated values ​​includes integrating pixel values ​​of a pixel group constituting each of the plurality of rotated vegetation cover images along the N integration lines to calculate the N integrated values; obtaining the N weighted integrated values ​​includes obtaining the N weighted integrated values ​​by applying the filter to a one-dimensional array storing the N integrated values ​​for each of the plurality of rotated vegetation cover images and performing a convolution operation; 18. The method of claim 15, further comprising detecting the plurality of crop rows based on the set of N weighted integrated values ​​obtained from the plurality of rotated vegetation cover images.

20. generating a plurality of rotated vegetation cover images by rotating the vegetation cover image at a plurality of rotation angles around an origin of a two-dimensional coordinate system defined by a first axis parallel to a direction in which the N integration lines extend and a second axis perpendicular to the first axis; The calculation of the N integrated values ​​is carried out for each of the plurality of rotated vegetation cover images, The method includes integrating pixel values ​​of a pixel group constituting the rotated vegetation cover image along each integration line to obtain a first integrated value, and integrating the number of pixel groups of the rotated vegetation cover image located on each integration line to obtain a second integrated value, thereby obtaining N first integrated values ​​and N second integrated values, respectively; applying the filter to a one-dimensional array storing the N first accumulated values ​​and the N second accumulated values, and performing a convolution operation to obtain N weighted first accumulated values ​​and N weighted second accumulated values; determining a vegetation coverage ratio of the weighted first integrated value relative to the weighted second integrated value associated with each of the N integrated lines to obtain N vegetation coverage ratios for the N integrated lines; 18. The method of claim 15, further comprising detecting the plurality of crop rows based on the set of N vegetation cover ratios for the plurality of rotated vegetation cover images.

21. 20. The method of claim 19, comprising applying a discrete Fourier transform to the vegetation canopy image to generate a frequency spectrum image, and estimating from the spectrum image an angle between a direction in which the plurality of crop rows extend in the two-dimensional coordinate system and the first axis.

22. 22. The method of claim 21, comprising determining the plurality of rotation angles by reference to the estimated angles, and rotating the vegetation cover image by the plurality of rotation angles around an origin of the two-dimensional coordinate system to generate the plurality of rotated vegetation cover images.

23. 18. The method of claim 15, wherein generating the vegetation cover image comprises applying a Lab transform or deep learning to the image to generate the vegetation cover image in which the vegetation cover is extracted.

24. The method according to claim 15, further comprising: identifying an area between adjacent crop rows based on the detection result of the crop rows; and transmitting a control instruction to an agricultural machine when detecting weeds growing in the identified area.

Images