Time-series video-based lawn disease detection system

Abstract

We provide a time-series video-based lawn disease spot detection system that detects and extracts disease spots based on objective criteria and analyzes and understands the progression pattern of disease spots by continuously tracking changes in the state of the lawn. [Solution] In the time-series video-based turf disease detection system, the detection service providing server 300 includes an administrator terminal that uploads video data of the turf and a receiving unit that receives video data from the administrator terminal, a time-series alignment unit that accumulates the video data in chronological order, a global trend analysis unit that grasps the overall condition of the turf from the chronologically accumulated video data and extracts the global trend, which is the overall condition of the turf, a local trend analysis unit that analyzes the video data on a pixel-by-pixel basis and compares the local trend, which is the regional condition of the turf, with the global trend, and a disease detection unit that, if the local trend deviates from the global trend standard, estimates and extracts the area where the local trend occurred as the area where the disease is detected.

Description

[Technical Field]

[0001] The present invention relates to a time-series image-based turf disease detection system, which detects disease spots by extracting a global trend, which is a change across the entire turf area, based on time-series images and setting it as a standard, and checking whether a local trend, which is a regional change in the turf, deviates from the global trend. [Background technology]

[0002] Turf growth and disease surveys are often conducted through subjective visual assessments by researchers and field experts. Visual assessment results are quantitative rather than qualitative, and can vary greatly depending on the expert's experience and training. In turf research and field practice, quantitative assessments cover a wide range of traits, including both the growing and dormant seasons. For example, quantitative assessments include green-up time, dormancy onset time, coverage rate, quality, winter survival rate, and pest and disease damage. Accurate assessment of turf condition is extremely important not only for researchers but also for producers and consumers. This is because it can make a big difference in the amount of manpower, time, and money invested in turf production and management. In the case of turf diseases, this can be crucial in determining when to prevent the disease. Turf has limitations, such as the difficulty of individual assessments when investigating growth conditions and pest and disease damage. When a disease occurs in a turf field, it is not possible to evaluate each individual plant like in fruit trees, which is why disease is evaluated by the diseased plant area rate or index rather than the infected plant rate.

[0003] At this time, methods for detecting and managing turf disease spots have been researched and developed. In this regard, prior art patent documents 1-2 (Korean Patent Registration No. 10-1633710 (published June 27, 2016) and Korean Patent Registration No. 10-2076015 (published February 11, 2020) disclose a configuration for analyzing turf images from a camera, detecting diseased and burned areas of the turf, and collecting environmental information on the diseased and burned areas, which is then provided to an administrator's terminal. Another configuration involves receiving turf images using a drone, comparing and analyzing the normal state of the turf with the current state of the turf, and prescribing medicine to the affected areas if turf disease is determined to have occurred based on the Normalized Difference Vegetation Index (NDVI).

[0004] However, when receiving and analyzing lawn images from a single point in time rather than time-series data, as in the former case, it is difficult to distinguish whether the lawn has a diseased spot or is simply covered from the image at a single point in time. In the latter case, the current lawn condition is compared to the normal state, but since the hue of the grass community varies between morning and evening and can also change depending on the day, month, and season, if the normal state of the lawn is set as an absolute value, natural changes in hue may be recognized as diseased spots, or vice versa, resulting in errors. Therefore, research and development of a system that can detect lawn disease spots using time-series video is needed. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Korean Patent No. 10-1633710 [Patent Document 2] Korean Patent No. 10-2076015 Summary of the Invention [Problem to be solved by the invention]

[0006] One embodiment of the present invention provides a time-series image-based lawn disease spot detection system that accumulates image data of lawn in a time series, compares local trends that grasp the lawn condition regionally with a global trend that grasps the lawn condition globally, and detects a disease spot if the local trend does not match the standard, thereby preventing natural color changes that occur in the lawn community from being recognized as a disease spot, and compares changes in a partial area with changes in the lawn community, thereby detecting and extracting disease spots based on objective criteria rather than subjective criteria, and continuously tracks changes in the lawn condition to analyze and understand the progression pattern of disease spots. However, the technical problem to be solved by this embodiment is not limited to the above technical problem, and other technical problems may exist. [Means for solving the problem]

[0007] As a technical means for achieving the above-mentioned technical object, one embodiment of the present invention includes a detection service providing server including an administrator terminal that uploads video data of the grass and a receiving unit that receives video data from the administrator terminal, a time series alignment unit that accumulates the video data in chronological order, a global trend analysis unit that grasps the overall state of the grass from the chronologically accumulated video data and extracts a global trend, which is the global state of the grass, a local trend analysis unit that analyzes the video data on a pixel-by-pixel basis and compares the local trend, which is the local state of the grass, with the global trend, and a disease spot detection unit that, if the local trend deviates from the global trend standard, estimates and extracts the area where the local trend occurs as an area where a disease spot has been detected. [Effects of the Invention]

[0008] According to any one of the above-mentioned problem-solving means of the present invention, video data of the lawn is accumulated in a chronological order, and a global trend that grasps the overall state of the lawn is used as a standard to compare local trends that grasp the state of the lawn regionally. If the local trend deviates from the standard, it is detected as a disease spot, so that natural changes in color that occur in the grass community are not recognized as disease spots. By comparing changes in partial areas with changes in the grass community as a standard, disease spots can be detected and extracted based on objective criteria rather than subjective criteria, and the progression pattern of disease spots can be analyzed and understood by continuously tracking changes in the state of the lawn. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating a time-series image-based turf disease detection system according to an embodiment of the present invention; [Figure 2] 2 is a block diagram illustrating a detection service providing server included in the system of FIG. 1. FIG. [Figure 3] 1 is a diagram illustrating an embodiment of a time-series image-based lawn disease spot detection service according to an embodiment of the present invention; [Figure 4] 1 is a diagram illustrating an embodiment of a time-series image-based lawn disease spot detection service according to an embodiment of the present invention; [Figure 5] 1 is an operational flowchart illustrating a method for providing a lawn disease spot detection service based on time-series images according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. In order to clearly explain the present invention in the drawings, parts that are not relevant to the description are omitted, and similar parts are designated by similar reference numerals throughout the specification. Throughout the specification, when a part is said to be "connected" to another part, this includes not only "directly connected" but also "electrically connected" with another element therebetween. Furthermore, when a part is said to "comprise" a certain component, this does not mean excluding other components, but means that it may further include other components, unless otherwise specified, and does not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof. The terms "about," "substantially," and the like used throughout the specification are used to mean a numerical value or a close approximation of a numerical value when manufacturing and material tolerances inherent in the stated meaning are given, and are used to prevent unscrupulous infringers from unfairly exploiting the disclosure in which precise or absolute numerical values ​​are recited to aid in the understanding of the present invention. The terms "steps of (doing)" or "steps of" used throughout the specification of the present invention do not mean "steps for." As used herein, the term "module" includes hardware-implemented units, software-implemented units, and units implemented using both hardware and software. Also, one unit may be implemented using two or more pieces of hardware, or two or more units may be implemented by a single piece of hardware. Meanwhile, the term "module" is not limited to software or hardware; a "module" may reside on an addressable storage medium or execute one or more processors. Thus, by way of example, a "module" includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within a component and a "module" may be combined into fewer components and "modules" or further separated into additional components and "modules." Furthermore, a component and a "module" may be embodied to execute one or more CPUs within a device or security multimedia card. Some of the operations and functions described herein as being performed by a terminal, apparatus, or device may instead be performed by a server connected to the terminal, apparatus, or device. Similarly, some of the operations and functions described herein as being performed by a server may instead be performed by a terminal, apparatus, or device connected to the server. In this specification, some of the operations or functions described as mapping or matching with a terminal may be interpreted as mapping or matching the terminal's unique number or personal identification information of the terminal's identifying data. The present invention will now be described in detail with reference to the accompanying drawings.

[0011] Figure 1 is a diagram illustrating a time-series image-based turf disease detection system according to one embodiment of the present invention. Referring to Figure 1, the time-series image-based turf disease detection system 1 may include at least one administrator terminal 100, a detection service providing server 300, at least one camera 400, and at least one drone 500. However, the time-series image-based turf disease detection system 1 of Figure 1 is merely one embodiment of the present invention, and the present invention should not be construed as being limited by Figure 1.

[0012] At this time, each component of Fig. 1 is generally connected via a network 200. For example, as shown in Fig. 1, at least one administrator terminal 100 may be connected to a detection service providing server 300 via the network 200. The detection service providing server 300 may be connected to at least one administrator terminal 100, at least one camera 400, and at least one drone 500 via the network 200. The at least one camera 400 may be connected to the detection service providing server 300 via the network 200. The at least one drone 500 may be connected to at least one administrator terminal 100, the detection service providing server 300, and at least one camera 400 via the network 200.

[0013] Here, a network refers to a connected structure that allows information exchange between nodes such as multiple terminals and servers, and examples of such networks include local area networks (LANs), wide area networks (WANs), the Internet (WWW), wired / wireless data communication networks, telephone networks, and wired / wireless television communication networks. Examples of wireless data communication networks include, but are not limited to, 3G, 4G, 5G, 3GPP (registered trademark) (3rd Generation Partnership Project), 5GPP (5th Generation Partnership Project), 5G NR (New Radio), 6G (6th Generation of Cellular Networks), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, the Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth (registered trademark) network, NFC (Near-Field Communication) network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc.

[0014] In the following, the term "at least one" is defined as a term including both singular and plural, and it is clear that even if the term "at least one" is not present, each component can exist in singular or plural and can mean singular or plural. Furthermore, it can be said that the presence of each component in singular or plural can be changed depending on the embodiment.

[0015] At least one administrator terminal 100 may be an administrator terminal that uploads image data of grass captured using a web page, app page, program, or application related to the time-series image-based grass disease spot detection service to the detection service providing server 300, and receives and outputs the area where disease spots are detected from the detection service providing server 300.

[0016] Here, at least one administrator terminal 100 may be implemented as a computer that can connect to a remote server or terminal through a network. Here, the computer may include, for example, a notebook computer, desktop, laptop, etc. equipped with a navigation system or a web browser. At this time, at least one administrator terminal 100 may be implemented as a terminal that can connect to a remote server or terminal through a network. The at least one administrator terminal 100 may be, for example, a wireless communication device that ensures portability and mobility, and may include all kinds of handheld-based wireless communication devices such as navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminals, smartphones, smartpads, tablet PCs, etc.

[0017] The detection service providing server 300 may be a server that provides a web page, app page, program, or application for a lawn disease detection service based on time-series video. The detection service providing server 300 may receive video data from a camera 400 or drone 500 that captures the lawn via the administrator terminal 100, or may receive video data directly from the camera 400 or drone 500 and accumulate the data in a time-series manner. The detection service providing server 300 may also be a server that extracts global trends, which are the overall condition of the lawn. The detection service providing server 300 may also be a server that extracts local trends, which are the regional condition of the lawn. When the local trends deviate from the global trend set as the standard, the detection service providing server 300 may estimate the deviating area as an area where disease spots have occurred and provide the estimate to the administrator terminal 100.

[0018] Here, the detection service providing server 300 may be implemented as a computer that can connect to a remote server or terminal through a network. Here, the computer may include, for example, a notebook computer, desktop computer, laptop computer, etc. equipped with a navigation system or a web browser.

[0019] At least one camera 400 may be a device that transmits image data of the lawn captured using or without using a web page, app page, program, or application related to the time-series image-based lawn disease spot detection service to the detection service providing server 300 via the administrator terminal 100 or directly.

[0020] Here, the at least one camera 400 may be implemented as a computer that can connect to a remote server or terminal through a network. Here, the computer may include, for example, a notebook computer, desktop computer, laptop computer, etc. that is equipped with a navigation system or a web browser. In this case, the at least one camera 400 may be implemented as a terminal that can connect to a remote server or terminal through a network. The at least one camera 400 may be, for example, a wireless communication device that ensures portability and mobility, and may include all kinds of handheld-based wireless communication devices such as navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminals, smartphones, smartpads, tablet PCs, etc.

[0021] At least one drone 500 may be a UAV (Unmanned Aerial Vehicle) that transmits video data of the lawn captured using a webpage, app page, program, or application related to the time-series video-based lawn disease spot detection service to the detection service providing server 300 via the administrator terminal 100 or directly.

[0022] Here, at least one drone 500 may be embodied as a computer that can connect to a remote server or terminal through a network. Here, the computer may include, for example, a notebook computer, desktop, laptop, etc. equipped with a navigation system and a web browser. In this case, at least one drone 500 may be embodied as a terminal that can connect to a remote server or terminal through a network. At least one drone 500 may be, for example, a wireless communication device that ensures portability and mobility, and may include all kinds of handheld-based wireless communication devices such as navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminals, smartphones, smartpads, tablet PCs, etc.

[0023] Figure 2 is a block diagram illustrating the detection service providing server included in the system of Figure 1, and Figures 3 and 4 are diagrams illustrating an embodiment in which a time-series image-based lawn disease detection service according to one embodiment of the present invention is implemented.

[0024] Referring to FIG. 2, the detection service providing server 300 may include a receiving unit 310, a time series alignment unit 320, a global trend analysis unit 330, a local trend analysis unit 340, a lesion detection unit 350, a progress tracking unit 360, a pattern analysis unit 370, a pre-processing unit 380, a lesion display unit 390, and a spread prevention unit 391.

[0025] When the detection service providing server 300 according to an embodiment of the present invention or another server (not shown) operating in conjunction therewith transmits a time-series image-based lawn disease detection service application, program, application page, web page, etc. to at least one administrator terminal 100, at least one camera 400, and at least one drone 500, the at least one administrator terminal 100, at least one camera 400, and at least one drone 500 can install or open the time-series image-based lawn disease detection service application, program, application page, web page, etc. In addition, the service program may be driven by at least one administrator terminal 100, at least one camera 400, and at least one drone 500 using a script executed in a web browser. Here, a web browser is a program that enables use of web (World Wide Web) services and refers to a program that receives and displays hypertext written in HTML (HyperText Mark-up Language), and includes, for example, Chrome, Microsoft Edge, Safari, Firefox, Whale, UC Browser, etc. Furthermore, an application refers to an application program on a terminal, and includes, for example, an app executed on a mobile terminal (smartphone).

[0026] 2, the receiving unit 310 can receive video data from the manager terminal 100. The manager terminal 100 can upload video data of the lawn. In this case, the manager terminal 100 can upload video data of the lawn captured by the camera 400 or the drone 500.

[0027] The image data can include RGB data captured by an RGB camera and multispectral data collected from a multispectral camera. Using only RGB data requires preprocessing due to the effects of shadows. Even with preprocessing, it is difficult to distinguish whether the grass color is due to shadows or disease spots, resulting in a low disease spot detection rate. Therefore, as shown in Figure 4c, multispectral data obtained from a multispectral camera capable of detecting near-infrared (NIR), red, green, and blue light is used to merge the spectral characteristics of the image data. A multispectral camera is a collection of monochromatic images of the same scene in multiple wavelength bands, each captured with a different sensor. For example, if an image of space A is captured, each wavelength band, such as red, blue, and green, is captured separately and then grafted onto spatial data. This can be considered the result of grafting spatial information and spectroscopic technology. In this case, not only a multispectral camera but also a hyperspectral camera may be used, and the result captured by the multispectral camera may be a segmented result as shown on the left side of Figure 4f, while the result captured by the hyperspectral camera may be a continuous result as shown on the right side of Figure 4f.

[0028] The time series alignment unit 320 can accumulate image data in a time series. For example, in a typical single image captured in RGB, it can be difficult to distinguish whether the grass is covered or damaged by disease. Coverage refers to areas covered by the shadow of a golf club or debris. For example, Figure 4g shows grass covered with dollar spots. While such disease (dollar spot conditions) exhibit a continuous and connected pattern, coverage exhibits sporadic and spatially discontinuous characteristics. That is, disease spots exhibit reduced reflectance in specific spectra, while covered areas exhibit abnormally low reflectance across all spectra. Furthermore, because disease spots exhibit more distinct changes in specific bands, image data is collected and accumulated in a time series and as multispectral data.

[0029] The basis for the aforementioned characteristics of lesions and cover is found in the paper (Radocz, Laszlo, Csaba Juhasz, Andras Tamas, Arpad Illes, Peter Ragan, and Laszlo Radocz. 2024. “Multispectral UAV-Based Disease Identification Using Vegetation Indices for Maize Hybrids.” Agriculture 14, no. 11:2002. https: / / doi.org / 10.3390 / agriculture14112002) and the paper (Zhang, Jing, Simerjeet Virk, Wesley Porter, Kevin Kenworthy, Dana Sullivan, and Brian Schwartz. “Applications of unmanned aerial vehicle-based imagery in turfgrass field trials.” Frontiers in plant science). 10(2019):279.) and the paper (Lee Young-chan, Kang Jun-oh and Oh Seong-jung, "Time series analysis of clover eradication range in lawns based on drone footage," Journal of the Korean Society of Surveying 39, no.4(2021):211-221.doi:https: / / doi.org / 10.7848 / ksgpc.2021.39.4.211).

[0030] The global trend analysis unit 330 can extract a global trend, which is the global condition of the grass, by grasping the overall condition of the grass from the time-series accumulated image data. The grass condition can be evaluated using at least one vegetation index (VE) extracted from the RGB data and multispectral data. Here, the at least one vegetation index can be an index corresponding to the Normalized Difference Vegetation Index (NDVI), the Visible Atmospherically Resistant Index (VARI), and the Green Normalized Difference Vegetation Index (GNDVI). It should be clear that the vegetation indexes listed here are merely provided for illustrative purposes and are not intended to be limiting. Various other vegetation indices, such as those listed in Table 1, can also be used, and the present invention is not limited to any one index.

[0031] [Table 1] TIFF0007823955000003.tif248166TIFF0007823955000004.tif121166

[0032] <ndvi> Let us explain NDVI, a representative vegetation index. First, NDVI is calculated by calculating the visible and near-infrared light reflected by plants. NDVI reflects the photosynthetic activity of plants, making it easy to understand their health. Referring to Figures 4b and 4c, NDVI is one of the vegetation indices widely used in remote sensing since its introduction in the 1970s, and can be used to determine the vitality of vegetation. It estimates a dimensionless index between -1 and 1 using the near-infrared (NIR) and red (RED) bands of the electromagnetic spectrum, based on the method of reflecting energy and light. The formula for NDVI to calculate this is shown in Mathematical Equation 1. Equation 1

[0033] TIFF0007823955000005.tif1277

[0034] NDVI is a theory that utilizes the reflectance of green plants in the spectral range, and utilizes two key bands: NIR (near infrared) and RED (red). This is because the green chlorophyll in plants absorbs most red light and does not reflect it. This means that plants with high vitality have low red light reflectance, and conversely, near infrared light is not absorbed due to the characteristics of plant cells. By utilizing this, it is possible to measure the vitality of vegetation by measuring the reflectance in the near infrared (NIR) and red (RED) bands at a specific location.

[0035] [Table 2]

[0036] As shown in Figure 4b and Table 2, a value of -1 or 0 indicates that the plant is dead or inanimate, while a value closer to 1 indicates that the plant is healthy. However, a drawback is that changes in hue due to soil moisture affect the normalized difference vegetation index (NDVI). As a result, the amount of change in the vegetation index may be displayed as small, and to compensate for this, the soil-adjusted vegetation index (SAVI), enhanced vegetation index (EVI), etc. are sometimes used. For the sake of convenience, in one embodiment of the present invention, NDVI will be used as the basis. However, as mentioned above, it is not excluded to use various vegetation indices.

[0037] Here, the global condition may be, for example, the average value of the NDVI of the grass in a 10x10m area, assuming that area B in area A is 10x10m (meters) in size and covered with grass. For example, if the average NDVI value was 0.8 in May and changed to 0.6 in June, this means that the color of the grass in area B has faded slightly overall.

[0038] The local trend analysis unit 340 can compare the local trend, which is the local state of the grass obtained by analyzing the image data pixel by pixel, with the global trend. While the global trend identifies changes in the grass as a whole, the local trend identifies changes in each individual grass by measuring the NDVI pixel by pixel. Continuing with the example above, if the NDVI of the grass is checked pixel by pixel and pixel C in region A had an NDVI of 0.8 in May but dropped to 0.2 in June, the global trend (NDVI 0.8 → 0.6), which is the overall trend, would drop by 25%, while the local trend (NDVI 0.8 → 0.2) would drop by 75%, which can be seen as values ​​far exceeding the 25% error range (e.g., ±10%) of the reference value (global trend). In this case, it can be inferred that the grass is not fading overall, but that there is a diseased spot in this area, and this area can be identified as having a diseased spot.

[0039] <Crowding> Commonly used classification algorithms can be divided into supervised classification and unsupervised classification. Supervised classification is a method of classifying candidate sites using accurate labels for the object to be classified. Representative methods include logistic regression, decision trees, and random forests. Unsupervised classification is used to classify unlabeled data. It analyzes and groups similar data using only the internal structure and patterns of the data, and assigns meaningful labels to each cluster. Since lesions are a random phenomenon, unsupervised classification is an effective method for detecting them. Unsupervised classification methods include K-Means, the Iterative Self-Organizing Data Analysis Technique (ISODATA) algorithm, the Gaussian Mixture Model (GMM), and Density-Based Spatial Clustering of Application with Noise (DBSCAN).

[0040] Cluster analysis requires a pre-determined number of clusters (K), but this approach is possible when prior knowledge of the data is available. When analyzing data patterns based on reflectance characteristics across multiple spectral regions, including multiple light data, prior information is often insufficient. In such cases, ISODATA, a self-organizing clustering algorithm, can be effectively utilized. ISODATA has the ability to process noise and outliers, making it advantageous for removing or ignoring the noise that inevitably occurs in satellite data analysis, thereby obtaining more precise clustering results. Furthermore, even without pre-labeled data, i.e., pre-trained data, ISODATA can automatically classify the appropriate number of clusters and data patterns from lawn video data, enabling it to cluster pixels with similar characteristics, such as pixels with disease spots, without supervision.

[0041] The ISODATA clustering algorithm dynamically adjusts clusters according to their size. If the number of samples belonging to a parent class becomes small, the class is deleted. If the number of samples belonging to a parent class becomes large or the degree of variance is relatively large, the class is divided into two subclasses. In the case of video data of grass, an arbitrary mean can be assigned to each cluster through an iterative process of calculating the minimum Euclidean distance when assigning spectral domain cells to clusters. All cells are assigned to the closest of these means, and after repeating the initial process, a new mean for each cluster is calculated based on the attribute distance of the cells belonging to the cluster, and this process is repeated in a loop.

[0042] [Table 3]

[0043] Of course, the method of clustering NDVI to search for disease spots is not limited to the above, and it goes without saying that a variety of methods can be used.

[0044] If the local trend deviates from the global trend standard, the disease spot detection unit 350 can estimate and extract the area where the local trend occurred as an area where disease spots have been detected. Continuing with the example above, if the global trend is a 25% change in NDVI and the local trend is a 75% change, this is a value that deviates even when the global trend of 25% is added with an error range of ±5-10%. Therefore, if the value deviates from the standard or is outside the value obtained by adding the error range to the standard, the area can be estimated and extracted as an area where disease spots have been detected.

[0045] The progression tracking unit 360 can monitor the progression of disease spots within the chronologically accumulated image data. For example, it can check in which direction and how quickly a disease spot that started in area A spreads. In this case, data on which seasons and weather conditions are most likely to cause disease spots and under what conditions the disease spots are most likely to spread can be collected and used as basic information for analyzing the progression and patterns of disease spots.

[0046] The pattern analysis unit 370 extracts and accumulates the progression of disease spots, analyzes the progression pattern of the disease spots, and compares the changes between areas where disease spots progress and areas where disease spots do not progress. Analyzing the pattern in this way not only enables future diagnosis of disease spots, but also predicts the conditions for disease spot development, enabling advance control and pest prevention. The collected data may include factors that affect the occurrence of disease and pests, such as location information, weather information, soil information, and pest information. In more detail, soil information may include, for example, fertilizer magnesium usage, electrical conductivity, base exchange capacity, available silica concentration, fertilizer potassium usage, nitrate nitrogen capacity, fertilizer lime usage, available phosphate concentration, lime requirement, acidity, organic matter content, and ammonia nitrogen capacity. Weather information may include, but is not limited to, average ground temperature, maximum temperature, minimum temperature, minimum temperature on grass, daily precipitation, average local barometric pressure, average relative humidity, maximum wind speed, solar radiation, humidity, air temperature, soil moisture, underground temperature, and grass temperature.

[0047] To predict pests and diseases, a dataset of [location information - weather information - soil information - pest and disease information] can be constructed and a predictive model can be modeled so that when the [location information - weather information - soil information - pest and disease information] of the turfed area is input into the predictive model, [pest and disease information] is output. In this case, the pest and disease prediction model can be composed of, for example, a binary classification model that predicts the presence or absence of pests and diseases, and a multi-label classification model that predicts the type of pest that will occur if pests and diseases are predicted. Both the binary classification model and the multi-label classification model are selected as the model with the highest accuracy from a combination of each machine learning model (ensemble model). Of course, it is obvious that patterns can be learned and predicted using a variety of methods other than those mentioned above.

[0048] The preprocessing unit 380 performs image preprocessing on the image data to analyze it. First, distortion correction must be performed according to the characteristics of the camera lens. This is because radial distortion must be flattened. Second, radiometric calibration must be performed to uniformly process light. Each image may have uneven lighting conditions due to sunlight, shadows, time of day, or terrain. Therefore, uniform lighting conditions are essential to accurately identify vegetation characteristics. Third, a histogram smoothing process must be performed. After radiometric calibration, the brightness of the RGB data becomes darker, making it difficult to identify the hue of the grass. This process is performed to improve this, and it evens out the histogram, which has brightness values ​​concentrated on the dark side of the image, thereby improving brightness and darkness.

[0049] [Table 4]

[0050] When image data is collected by a drone (Unmanned Aerial Vehicle, 500), the disease spot display unit 390 can have the drone launch a pre-installed indicator toward the location to display the location of the area where the disease spot was detected. Since the reason for searching for disease spots is to prioritize eradication or preventive measures, ultimately, the manager must know where this area is. Accordingly, the drone 500 can be operated to launch a flag-like indicator toward the location of the disease spot.

[0051] In addition, a route guidance service can be provided so that the location can be identified through the administrator's terminal 100. For example, in a large area such as a golf course, accurately locating the location of each disease spot facilitates pest control, reduces the amount of pesticide used, and protects the environment. Therefore, a drone 500 can be used to plant a flag, and the administrator's terminal 100 can operate navigation to help locate the exact location. In addition, if the golf cart (not shown) is an autonomous cart, navigation can be linked to automatically locate the location of the disease spot. For example, the locations of the disease spots can be set as waypoints to generate a route, and the golf cart can automatically travel along this route. In this case, the administrator does not need to drive or walk; instead, he or she can simply sit in the golf cart, move around, and when he or she arrives at the location of the disease spot, get off to pest control, and then get back on and move again, repeating the process.

[0052] The spread prevention unit 391 transmits the location of the area where the disease spot was detected to the manager terminal 100, and when the manager terminal 100 outputs a restoration event indicating that the disease spot at the location has been restored, it can monitor whether the local tendency compared to the global tendency based on the location deviates from the standard. In other words, after treating the turf where the disease spot appeared, it goes through the process of checking whether the treatment was successful again. This monitoring process is important because if a chemical B is used to treat disease A, but the chemical B is not effective and disease A continues to spread, immediate measures must be taken, such as changing the chemical B to another chemical. In this way, it is possible to monitor whether treatment was successful at the location where the restoration event occurred, and transmit the results to the manager terminal 100.

[0053] Hereinafter, the operation process according to the configuration of the detection service providing server of Figure 2 will be described in detail with reference to Figures 3 and 4. However, it is obvious that the embodiment is merely one of various embodiments of the present invention and is not limited thereto.

[0054] Referring to FIG. 3a, (a) the detection service providing server 300 can receive video data captured by the camera 400 or drone 500, either directly or via the administrator terminal 100. After determining the global trend of the grass (b), the detection service providing server 300 uses this as a reference and compares it with (c) the local trend. Accordingly, (d) if there is a change in the grass color that deviates from the reference color in a portion of the grass while the overall color of the grass changes, the detection service providing server 300 estimates and extracts this as a disease spot. As shown in (a) of FIG. 3b, the detected disease spot is transmitted to the administrator terminal 100, and (b) the drone 500 can be controlled by the detection service providing server 300 to emit a display to display the location. By accumulating and storing patterns of disease spot changes over time, the detection service providing server 300 can build a model to predict the occurrence of disease and the speed and form of disease spot spread. The detection service providing server 300 can then monitor whether the pesticide used by the administrator is effective, as shown in (d).By accumulating and storing such data, the detection service providing server 300 can build a system that can provide guidance on what pesticide to use for what disease under what conditions.

[0055] Figure 4a is a diagram summarizing a disease spot detection service according to one embodiment of the present invention, Figure 4b is a diagram explaining the concept of NDVI, Figure 4c is a diagram explaining multispectral data, Figure 4d is a screen showing an NDVI application, and Figure 4e is a screen comparing the difference between GNVDI and NVDI. Figure 4b is an example diagram used by Auravant to explain the concept of NDVI, and Figures 4d and 4e are screen shots of a program provided by Auravant's platform (https: / / www.auravant.com / en / articles / precision-agriculture / vegetation-indices-and-their-interpretation-ndvi-gndvi-msavi2-ndre-and-ndwi / ). Figure 4f is a diagram explaining the difference between multispectral data and hyperspectral data, and was published in JAI News (https: / / news.jai.com / blog / ko / multi-spectral-imaging). Figure 4g is a photograph (Golf Industry News) of turf covered in dollar spot. It should be made clear that FIGS. 4b to 4g are merely screen shots and photographs attached for the purpose of explaining the concept, and are not screen shots and photographs according to an embodiment of the present invention.

[0056] Matters not explained in the method for providing a lawn disease spot detection service based on time-series video in Figures 2 to 4 are the same as or can be easily inferred from the content previously explained in the method for providing a lawn disease spot detection service based on time-series video through Figure 1, so they will not be explained further below.

[0057] Figure 5 is a diagram showing a process of transmitting and receiving data between components included in the time-series image-based turf disease detection system of Figure 1 according to one embodiment of the present invention. Hereinafter, an example of a process of transmitting and receiving data between components will be described with reference to Figure 5, but the present application is not limited to this embodiment, and it will be obvious to those skilled in the art that the process of transmitting and receiving data shown in Figure 5 can be changed according to the various embodiments described above. Referring to FIG. 5, the detection service providing server receives video data from the administrator terminal (S5100).

[0058] The detection service providing server then accumulates the image data in a time series (S5200), and extracts the global trend, which is the global condition of the grass, by grasping the average of the grass condition from the time series accumulated image data (S5300).

[0059] In addition, the detection service providing server compares the local tendency, which is the local state of the grass obtained by analyzing the video data pixel by pixel, with the global tendency (S5400), and if the local tendency deviates from the global tendency standard, it estimates and extracts the area where the local tendency occurs as the area where disease spots are detected (S5500).

[0060] The order of the above steps (S5100 to S5500) is merely an example and is not limited thereto, that is, the order of the above steps (S5100 to S5500) may be changed, and some steps may be performed simultaneously or deleted.

[0061] Matters not described in the method for providing a lawn disease spot detection service based on time-series video in Figure 5 are either the same as or can be easily inferred from the content described in the method for providing a lawn disease spot detection service based on time-series video through Figures 1 to 4, so they will not be described further below.

[0062] The method for providing a lawn disease detection service based on time-series video according to one embodiment described with reference to FIG. 5 may also be embodied in the form of a recording medium containing computer-executable instructions, such as an application or program module executed by a computer. A computer-readable medium may be any available medium that can be accessed by a computer, and includes both volatile and nonvolatile media, and both separable and non-separable media. Furthermore, a computer-readable medium may include all computer storage media. A computer storage medium includes all volatile and non-volatile, separable and non-separable media embodied in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data.

[0063] The method for providing a lawn disease spot detection service based on time-series video according to one embodiment of the present invention may be implemented by an application that is pre-installed on the terminal (which may include a program included in a platform or operating system that is pre-installed on the terminal), or by an application (i.e., a program) that the user directly installs on the master terminal through an application providing server such as an application store server, an application, or a web server related to the service. In this sense, the method for providing a lawn disease spot detection service based on time-series video according to one embodiment of the present invention may be implemented by an application (i.e., a program) that is pre-installed on the terminal or that the user installs directly, and may be recorded on a computer-readable recording medium such as a terminal.

[0064] The above description of the present invention is for illustrative purposes only, and those skilled in the art will readily appreciate that the present invention may be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. Therefore, the above-described embodiments are illustrative in all respects and are not limiting. For example, each component described as a single component may be implemented in a distributed form, and similarly, each component described as a distributed component may be implemented in a combined form. The scope of the present invention is indicated by the claims that follow rather than by the above detailed description, and all modifications and variations that come within the meaning and scope of the claims and their equivalents are included within the scope of the present invention.< / ndvi>

Claims

1. and a detection service providing server including: an administrator terminal that uploads video data of the lawn; a receiving unit that receives video data from the administrator terminal; a time series sorting unit that accumulates the video data in time series; a global trend analysis unit that grasps the overall state of the lawn from the accumulated video data in time series and extracts a global trend, which is the global state of the lawn; a local trend analysis unit that analyzes the video data in pixel units and compares the local trend, which is the local state of the lawn, with the global trend; and a disease spot detection unit that, if the local trend deviates from the global trend standard, estimates and extracts the area where the local trend occurs as an area where a disease spot has been detected; The detection service providing server a progression tracking unit that monitors the progression of the lesion within the time-series accumulated image data; and a pattern analysis unit that analyzes the progression pattern of the lesions after extracting and accumulating the progression process of the lesions and compares the changes between the areas where the lesions progress and the areas where the lesions do not progress; and a diffusion prevention unit that transmits the location of the area where the lesion is detected to the administrator terminal, and when the administrator terminal outputs a restoration event indicating that the lesion at the location has been restored, monitors whether a local tendency compared to the global tendency based on the location deviates from the reference; A time-series video-based lawn disease spot detection system characterized by:

2. The video data is The data includes RGB data captured by an RGB camera and multispectral data collected from a multispectral camera, The condition of the grass is evaluated using at least one vegetation index extracted from the RGB data and multispectral data; The at least one vegetation index is It is an index corresponding to NDVI (Normalized Difference Vegetation Index), VARI (Visible Atmospherically Resistant Index) and GNDVI (Green Normalized Difference Vegetation Index). The time-series image-based turf disease detection system according to claim 1.

3. The detection service providing server a pre-processing unit that performs image pre-processing on the image data to analyze the image data; The time-series image-based turf disease detection system according to claim 1.

4. The detection service providing server a lesion display unit that, when the image data is collected by a drone, causes the drone to project an indicator pre-installed on the drone toward the position where the lesion is detected so as to display the position. The time-series image-based turf disease detection system according to claim 1.

Images