Method for classifying invasive plants using drone orthophotos, method for mapping the distribution of invasive plants using the same, and method for weed control using the same

Abstract

A method for classifying disturbing plants using drone orthophotos is disclosed, and the method for classifying disturbing plants using drone orthophotos may include: (a) forming a drone orthophoto in which a target area is shown during a preset period; and (b) determining a candidate object of a disturbing plant from the drone orthophoto as the disturbing plant.

Description

Technology Field

[0001] The present invention relates to a method for classifying invasive plants using drone orthophotos, a method for mapping the distribution of invasive plants using the same, and a method for weed control using the same. Background Technology

[0002] Invasive plants are plant species that have been introduced to areas where they do not naturally exist due to human material activities. These plants are not present in their original ecosystems, but as they thrive in new environments, they can have a negative impact on the local ecosystem or biodiversity.

[0003] Furthermore, invasive plants are plants that are introduced from foreign countries, either artificially or naturally, and exist outside their original places of origin or habitats; they are defined as, firstly, plants judged to pose a significant threat to ecosystems based on national risk assessments, and secondly, among naturalized plants, those that disrupt or are likely to disrupt the ecological balance.

[0004] Specifically, invasive plants typically reproduce rapidly, possess strong competitiveness, and can be highly adaptable. Invasive plants can be a major factor in disrupting the natural balance of ecosystems.

[0005] In other words, invasive plants leave their original habitats and spread rapidly into new environments, threatening the survival of native plants and causing serious environmental problems that reduce biodiversity. These invasive plants not only destroy the balance of natural ecosystems but also inflict socioeconomic damage, such as decreased agricultural productivity, water pollution, and landscape degradation.

[0006] Currently in Korea, various alien invasive plant species, including prickly pear, Japanese knotweed, goldenrod, Japanese knotweed, ragweed, American aster, prickly lettuce, common tansy, hairy waterweed, waterweed, devil's weed, small sorrel, western goldenrod, sea sedge, English cordgrass, garlic mustard, ragweed, and water knotweed, are threatening the native ecosystem. These invasive species have strong reproductive capabilities and excellent adaptability; once established, they are very difficult to remove, requiring continuous and systematic monitoring and management.

[0007] To date, the monitoring and management of ecosystem-disturbing plants have primarily relied on direct surveys by field experts. This approach requires significant time and manpower to survey large areas, and accurate surveys are often impossible in hard-to-reach regions. Furthermore, this method has limitations as it is frequently limited to one-off surveys, making it difficult to track changes in distribution over time.

[0008] Although remote sensing techniques utilizing satellite imagery or aerial photographs have been introduced in some regions, it is difficult to accurately distinguish various plant species using only simple image analysis, and small communities were often missed due to limitations in resolution. Furthermore, consistent monitoring was difficult because the data processing relied heavily on the subjective judgment of experts.

[0009] In other words, conventional manual surveys had the disadvantage of being difficult to cover large areas and required excessive time and cost.

[0010] While accurately identifying the current distribution of invasive plants is a prerequisite for effective management, manually surveying the distribution of invasive species across wide areas is extremely time-consuming and labor-intensive. Furthermore, since the spread of invasive species is highly sensitive to seasonal and climate changes, near-real-time monitoring is required; however, current technological limitations prevent these requirements from being met.

[0011] Therefore, there is a need for methods to efficiently detect ecosystem-disturbing plants and manage them systematically.

[0012] In particular, among invasive plants, prickly pear, Japanese knotweed, and goldenrod pose a significant threat. Accordingly, there has been a need to quantitatively map the distribution and area range of prickly pear, Japanese knotweed, and goldenrod over large areas using GIS (Geographic Information System) photogrammetry technology in a short period of time to confirm, study, or weed their distribution. This is also used for verification after weeding operations.

[0013] The background technology related to the present invention is disclosed in Registered Patent Publication No. 10-2362424. The problem to be solved

[0014] The present invention aims to solve the problems of the aforementioned conventional technology by providing a method for classifying prickly pear, Japanese knotweed, and goldenrod using drone orthophotos that can determine the distribution and area range of prickly pear, Japanese knotweed, and goldenrod over a wide area in a short time through quantitative (coordinate, area) surveying, a method for mapping the distribution of disturbed plants using the same, and a weed control method using the same.

[0015] However, the technical problems that the embodiments of the present invention aim to solve are not limited to those described above, and other technical problems may exist. means of solving the problem

[0016] As a technical means for achieving the above-mentioned technical task, a method for classifying invasive plants using drone orthophotos according to one embodiment of the present invention may include: (a) forming image data including orthophotos in which a target area is depicted during a preset period; (b) inputting the image data into a pre-trained machine learning model; and (c) the machine learning model determining from the image data whether the candidate object of the invasive plant is the invasive plant.

[0017] A method for mapping the distribution of disturbed plants according to one embodiment of the present invention may include: a step of classifying disturbed plants using a method of classifying disturbed plants using orthophotos according to one embodiment of the present invention; and a step of mapping the area where the plants classified as disturbed plants are located using Geographic Information System (GIS) technology.

[0018] A method for weeding disturbed plants according to one embodiment of the present invention may include: a step of classifying disturbed plants using a method for classifying disturbed plants using orthophotos according to one embodiment of the present invention; and a step of remotely weeding plants classified as disturbed plants using a weeding robot. Effects of the invention

[0019] According to the solution to the problem described above, invasive plants can be identified by distinguishing the characteristics of the invasive plants during a predetermined period using orthophotos formed from images captured during that period. Accordingly, the distribution and area range of prickly pear, Japanese knotweed, and goldenrod can be determined quickly through quantitative (coordinates, area) surveying by observing the target area without having to walk around the area. Furthermore, even if the target area is large, the distribution and area range of prickly pear, Japanese knotweed, and goldenrod can be easily determined through quantitative (coordinates, area) surveying. Additionally, this can be mapped. Moreover, after weeding, the effectiveness of the work on the target site can be verified using the orthophotos.

[0020] In addition, by using a weeding robot, the efficiency of weeding operations can be improved and the safety of workers from the risks associated with weeding can be ensured. Brief explanation of the drawing

[0021] FIGS. 1 to 5 are photographs illustrating the characteristics of prickly pear and the identification of prickly pear using orthophotos of a method for classifying disturbed plants using orthophotos according to one embodiment of the present invention. FIGS. 8 to 10 are examples of images that may be included in an orthophoto input to a machine learning model of a method for classifying disturbed plants using an orthophoto according to one embodiment of the present invention. FIGS. 11 and 12 are conceptual diagrams illustrating a method for classifying disturbed plants using orthophotos according to one embodiment of the present invention. FIGS. 13 and 14 are photographs illustrating the method of classifying disturbed plants using orthophotos according to one embodiment of the present invention, wherein an area where a target determined to be a disturbed plant is located is painted with a preset color to output an image. FIG. 15 is a conceptual diagram illustrating a method for classifying disturbed plants using orthophotos according to one embodiment of the present invention. FIG. 16 is a conceptual diagram illustrating a method for classifying disturbed plants using orthophotos according to one embodiment of the present invention, which outputs removal results and reappearance results. FIG. 17 is a photograph illustrating a robotic weeder of a weeding method according to one embodiment of the present invention. Specific details for implementing the invention

[0022] Embodiments of the present invention are described below with reference to the attached drawings to enable those skilled in the art to easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0023] Throughout this specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" with other elements interposed between them.

[0024] Throughout the entire specification, when a component is described as being located "on," "on top," "on top," "under," "on bottom," or "on bottom" of another component, this includes not only cases where the component is in contact with the other component but also cases where another component exists between the two components.

[0025] Throughout this specification, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0026] The present invention relates to a method for classifying invasive plants using drone orthophotos, a method for mapping the distribution of invasive plants using the same, and a method for weed control using the same.

[0027] Hereinafter, a method for classifying invasive plants using drone orthophotos according to one embodiment of the present invention (hereinafter referred to as the "present method") will be described.

[0028] The present method includes a step (step 1) of forming image data including a drone orthophoto in which a target area is depicted during a preset period. Step 1 may form the image using one or more drone cameras.

[0029] For example, when an orthophoto is formed by a drone, the drone (e.g., an autonomous drone) can form the image by flying over a target area along a pre-set path and capturing high-resolution images.

[0030] According to this, drone orthophotos can refer to planar images that accurately reflect geographical information based on photographs taken by a drone. Additionally, the image may be one that reflects accurate location information of the surface by capturing multiple aerial photographs, combining them in a mosaic form, and removing terrain distortion.

[0031] In addition, as previously mentioned, images can be formed by drones, and drone images may be created using footage captured by a drone. In this case, the drone can photograph the ground within an altitude range of 100m to 150m, and the images (videos or photos) captured by the drone can be formed as orthophotos. For reference, drones can capture ultra-high-resolution images, which can play a significant role in overcoming time, cost, and environmental constraints.

[0032] In addition, drone orthophotos can be linked with a GIS system, enabling quantitative analysis of the area distribution of invasive plants. In other words, drone orthophotos are linked with GIS to enable the mapping of the area distribution of invasive plants through photogrammetry, allowing for quantitative analysis (display of area and distribution coordinates). Accordingly, the distribution of invasive plants (spiny gourd, Japanese knotweed, and goldenrod) can be identified in a short period of time at low cost.

[0033] In addition, aerial photography by drone can be performed by selecting the location and scope of the target area using satellite maps and acquiring the aerial photographs. For reference, flight approval and authorization for filming may be required prior to drone aerial photography.

[0034] In addition, the image acquisition software may be the free app version Pix4dCapture Ver. 4.4 from Pix4d, and the image processing software for generating drone orthophotos may be Metashape Ver. 2.1 from Agisoft.

[0035] In addition, video recording can be performed by DJI MAVIC 2 PRO at an altitude of 100m to 150m, the resolution can be 2.3cm / pixel, and WGS84 global geodetic coordinates >>>>> GRS80 coordinate conversion >>>>> digital topographic map overlay.

[0036] In addition, aerial photography can be performed by a drone at an altitude of 150m or less, with a resolution of 3cm / pixel, and the drone imagery can be superimposed with digital topographic maps published by the National Geographic Information Institute.

[0037] That is, drone footage is captured (DJI MAVIC 2 PRO, PIX4D CAPTURE), drone orthophotos are produced (Metashape), and the drone orthophotos are corrected (QGIS, GCP correction, digital topographic map 1 / 5,000) to produce the final drone orthophoto. Then, after distinguishing prickly pear, Japanese knotweed, and goldenrod through artificial intelligence analysis, mapping of the disturbed plants can be performed in GIS (QGIS, Mapping), and subsequently, the area and location of the disturbed plants can be verified (QGIS, Excel).

[0038] In other words, a drone can take aerial photographs, an orthophoto is produced, and it can be orthocorrected (QGIS, GCP correction, digital topographic map 1 / 5,000). After identifying the distribution range of the disturbing species through artificial intelligence, the distribution of the disturbing plants can be mapped in GIS, and subsequently, the area and location (coordinates) of the disturbing plants can be verified.

[0039] That is, the first step involves photographing a target area for a preset period and forming a drone orthophoto from the captured photos or videos. According to this, the drone orthophoto may depict the shape or appearance of the target area (which may be referred to as an area of ​​invasive species distribution) during the preset period.

[0040] In addition, video data may include not only video (image) but also one or more of metadata such as time, location (GPS), and altitude. Such video data may be formed by capturing a video using a capturing device that includes one or more of a camera module, a GPS module, an altitude sensor, and a timestamp recorder. Accordingly, in the present invention, video and video data may be the same concept.

[0041] Such image data can be defined as follows.

[0042] [Equation 1]

[0043]

[0044] In addition, the present method includes a step (step 2) of inputting image data into a pre-trained machine learning model. The machine learning model will be described later.

[0045] Examples of images included in the video input in the second stage are illustrated in FIGS. 1 to 7.

[0046] In addition, other examples of images included in the video input in the second stage are illustrated in FIGS. 8 to 10.

[0047] Additionally, the present method includes a step (step 3) of determining whether a candidate object of a disturbed plant is a disturbed plant from a drone orthophoto.

[0048] The third step can be performed by a machine learning model.

[0049] In addition, the invasive plants may include at least one of prickly pear, Japanese knotweed, and goldenrod.

[0050] Referring to FIGS. 11 and 12, the machine learning model may be a pre-trained deep learning model. For example, the deep learning model may be a Convolutional Neural Network (CNN) or a Generative AI-based model. Additionally, the CNN-based model may include at least one of YOLO, Faster R-CNN, RetinaNet, U-Net, etc.

[0051] A machine learning model can be expressed as the following function.

[0052] [Equation 2]

[0053]

[0054] Specifically, the third step is to obtain location information (b) of a disturbed plant candidate object from the drone orthophoto collected at time t. i ), classification result(c i ) and reliability(s iThe following set of objects including ) can be produced.

[0055] [Object Set]

[0056]

[0057] Drone orthophotos collected at time t are input into deep learning models for object detection (Faster R-CNN, YOLO, etc.), and location information (b) of each candidate invasive plant object i ), classification result(c i ), reliability(s i A set of objects Y = {(b i , c i , s i It can produce )}. Such a deep learning model can be constructed based on a pre-trained invasive plant image dataset, and

[0058] In addition, the detected object can generally be represented in the form of a bounding box or a segmentation mask.

[0059] In addition, the third step is, among the object set, reliability s i a preset threshold s t Only objects that meet or exceeding the criteria are selected to form a trusted object set, and within the trusted object set, identical classification results c i If objects having are clustered together, those objects can be determined to be the same plant community.

[0060] Reliability calculated per object (s i ) can generally have a value of 0.0 to 1.0, and for improved accuracy, a threshold s t Only objects with a value of (e.g., 0.5 or 0.6) or higher can be selected as trusted objects. This type of filtering is a post-processing method commonly used in deep learning-based object detection.

[0061] Also, the same classification result (c i For objects having ), location information (b iClustering can be determined based on the distance between center points or spatial density. In this case, clustering algorithms such as DBSCAN, MeanShift, or K-means can be utilized, and if GPS-based coordinates are provided, radius-based clustering can also be determined.

[0062] Accordingly, in the third step, if time t is from May 1 to October 31 according to Korean standards and the disturbed plant candidate object is a plant community consisting of clustered objects that include a stem structure (forming a plant community), the plant community can be determined to be prickly pear.

[0063] In addition, in the third stage, if time t is from December 1 to March 31 according to Korean standards and the plant community consists of individuals that are fallen and clustered, the plant community may be identified as Japanese knotweed.

[0064] In addition, in the third stage, if time t is from October 1 to December 31 according to Korean standards and the plant community consists of individuals that are in bloom and clustered, the plant community can be identified as *Scutellaria baicalensis*.

[0065] For reference, the external appearance of candidate invasive plant objects, such as stem shape, lodging, and flowering status, can be determined through segmentation results or morphological analysis of object masks. This morphological classification can be processed through class subdivision of a single model, or it can be implemented using a separate morphological recognition auxiliary model (e.g., a stem detection CNN) for auxiliary judgment.

[0066] In addition, the time t, which is the time of capture, is determined based on the metadata of the drone video or the time and date information of the capture, and

[0067] Conditional classification judgments can be performed in conjunction with the appearance time of each invasive plant. For example, from May to October, objects showing stem structures are likely to be *Xanthium strumarium*, from December to March, objects in a fallen form are likely to be *Ambrosia artemisiifolia*, and from October to December, flowering objects are likely to be *Aster tataricus*.

[0068] For reference, in Figures 11 and 12, the images are described as collected images.

[0069] Meanwhile, the judgment using the characteristics of each disturbing plant is explained in more detail below.

[0070] If the invasive plant is prickly burr, the preset period may be from May 1 to October 31 according to Korean time (Korean time if the target area is Korea). Preferably, if the invasive plant is prickly burr, the preset period may be from mid-June to the end of October according to Korean time (Korean time if the target area is Korea).

[0071] In addition, the third step can identify plants that form colonies with stems as prickly burdock. Specifically, in the third step, the machine learning model can identify plants that form colonies with tendrils (runners) and are covered in a green carpet-like pattern in drone orthophotos as prickly burdock.

[0072] Referring to FIGS. 1 to 5, when the target area is Korea, during the period from May 1 to October 31 (preferably from mid-June to the end of October) in Korean time, the prickly pear may have characteristics such as being greener than other plants, covering the tops of other plants like a carpet, gathering in colonies, and having developed tendrils (runners) as a vine. In drone orthophotos, a plant having one or more of the above characteristics can be identified as prickly pear.

[0073] The three above-mentioned features are not found at any other time outside of the period from May 1 to October 31 (preferably from mid-June (June 15) to the end of October (October 31)) based on Korean standards. Therefore, drone orthophotos can be produced using images taken within the period from May 1 to October 31 (preferably from mid-June (June 15) to the end of October (October 31)) based on Korean standards.

[0074] For reference, when performing the third step, referring to Figures 1 to 5, the drone orthophoto can be magnified to identify disturbed plants.

[0075] In addition, if the invasive plant is Japanese knotweed, the pre-set period may be from December to the end of March according to Korean time, for example, from December 1 to March 31 (in Korean time if the target area is Korea).

[0076] In addition, the third step allows identifying plants with higher brightness than other plants in the drone orthophoto as Japanese knotweed. In other words, in the third step, the machine learning model can identify grayish-white plants with higher brightness than other plants in the drone orthophoto as Japanese knotweed.

[0077] Referring to Fig. 6, Japanese knotweed can be observed in drone orthophotos as white sticks in a fallen shape resembling a mop handle (Feature 1). Additionally, Japanese knotweed can be gathered in colonies (Feature 2). Anything having one or more of the above features in a drone orthophoto can be identified as Japanese knotweed.

[0078] The two above-mentioned features are not found at any other time outside of the period from December 1 to March 31 in Korea. Therefore, drone orthophotos can be produced using images taken within the period from December 1 to March 31 in Korea.

[0079] In addition, if the invasive plant is *Aster tataricus*, the preset period may be from early October to late November (e.g., from October 1 to November 31) according to Korean standards. The most ideal preset period may be from mid-October to early November (the 15th, from October 15 to November 1), which is the flowering period when the most ideal identification is possible.

[0080] In this case, the third step can determine that the flowering plant in the drone orthophoto is a type of goldenrod. Specifically, in the second step, the machine learning model can determine that the colony of bright yellow flowers in the drone orthophoto is a type of goldenrod.

[0081] The flowering period of *Solidago can be among the latest in Korea. Accordingly, referring to Fig. 7, the flowering plant may be *Solidago canadensis* (Feature 1). As such, when *Solidago canadensis* is in bloom, it can be clearly distinguished from other plants. This is because other plants may not be in bloom during the period when *Solidago canadensis* is flowering. Additionally, the color of the *Solidago canadensis* flowers may be yellow (Feature 2). Furthermore, *Solidago canadensis* can reproduce by gathering in colonies (Feature 3). In drone orthophotos, if a plant possesses one or more of the above features, it can be determined to be *Solidago canadensis*.

[0082] That is, the third step can be performed by a machine learning model, and the machine learning model can determine if at least one of prickly pear, Japanese knotweed, and goldenrod is present in the input image according to the judgment method described above.

[0083] In addition, the set of trust objects has different confidence thresholds (s) depending on the classification result (c1). t1 , s t2 , s t3 It can be configured by applying etc. In other words, an object classified as prickly pear is s t1It is determined to be a valid object only if it possesses the above level of reliability, and for Japanese knotweed and goldenrod respectively, s t2 , s t3 Whether an object is valid can be determined based on the above reliability.

[0084] Specifically, since the reliability distribution of object classification results varies depending on each type of invasive plant, plant class (c i It is desirable to set different reliability thresholds for each.

[0085] For example, prickly pear (c i = In the case of "spiny gourd", s i ≥ s t1 = 0.6, Japanese knotweed(c i = In the case of "ragweed," s i ≥ s t2 = 0.55, Yangmiyeokchwi(c i = In the case of "Yangmiyeokchwi", s i ≥ s t3 It can be set to 0.5, and this class-specific multi-threshold-based filtering can reduce the risk of over-classification or misclassification.

[0086] In this way, since separate confidence criteria are applied to each class for the same deep learning model output, multi-stage filtering is possible without operating multiple models, and classification accuracy can be improved with a feasible structure.

[0087] In other words, by applying different threshold values ​​(reliability standards by type of invasive plant) for each class (types of invasive plants (spiny gourd, Japanese knotweed, goldenrod)) rather than a single criterion to object classification reliability, the present invention enables more precise identification of invasive plants than general filtering methods and achieves the technical effect of improved accuracy without increasing model complexity.

[0088] In particular, the reliability of plant classification results can vary depending on external similarity and background; therefore, adjusting class-specific thresholds in consideration of this can be considered a refined post-processing technique not proposed in existing literature or disclosed inventions.

[0089] In addition, the third step can identify the area of ​​the disturbed plants. That is, in the third step, if the machine learning model determines that at least one of prickly pear, Japanese knotweed, or goldenrod is present, it can determine this and identify the area where it is determined to exist. Furthermore, the machine learning model can make this determination using bounding box or segmentation techniques.

[0090] In addition, in the third stage, the machine learning model can produce information including the location and distribution area of ​​the detected disturbed plants.

[0091] For example, referring to FIGS. 13 and 14, an image (or video containing an image) can be output by painting an area where a target determined to be a disturbing plant is located with a preset color.

[0092] In addition, in the third stage, the machine learning model can establish a management plan for disturbed plants based on the generated information.

[0093] Additionally, referring to FIG. 15, the first step may generate GPS information of the location where the image is captured. In this case, the present method may include the step of mapping the geographical location of a plant determined to be an invasive plant based on the GPS information.

[0094] As described above, according to the present method, the distribution and area range of prickly pear, Japanese knotweed, and goldenrod can be determined in a short time through quantitative (coordinates, area) surveying by observing a wide area. Accordingly, an image (or video containing the image) in which the distribution location and distribution area of ​​the invasive plants in the target area are calculated can be output as a result.

[0095] In addition, as described above, the present method can be performed as follows.

[0096] In one embodiment, the present method involves an autonomous flying drone flying along a pre-set path over a target area and capturing high-resolution images, and the captured images being transmitted to a system in real-time or post-processing mode to be analyzed through an object detection model (machine learning model) based on the YOLO algorithm. The drone is equipped with a GPS module to record accurate location information of the captured images, which can be used for mapping the location of detected disturbing species.

[0097] In this embodiment, the flight altitude of the drone can be adjusted to perform a general scan of the entire area and detailed imaging of suspected areas in stages. This enables efficient investigation of a wide area while allowing for high-accuracy detection in necessary areas.

[0098] The present method according to such examples can implement a drone-based monitoring system.

[0099] In addition, as another example, the present method can be implemented by periodically collecting drone orthophotos and analyzing them with a ResNet-based deep learning model.

[0100] In addition, as another example, the present method can be performed by an integrated monitoring and management system.

[0101] The management system utilizes Geographic Information System (GIS) technology to visualize the distribution of detected invasive species on a map and track changes over time. It can also provide advanced analytical functions, such as comparative analysis with historical data, diffusion prediction modeling, and recommendations for optimal removal timing and methods.

[0102] Additionally, the present method may include, after the third step, a step of calculating an area where disturbing plants need to be removed based on one or more of the determined type of disturbing plants, the determined area of ​​disturbing plants, and the determined density of disturbing plants.

[0103] In addition, in the present method, the step of inputting an image into a pre-trained machine learning model may involve preprocessing the image data and inputting the preprocessed image data into the pre-trained machine learning model. The preprocessing may include at least one of image correction, noise removal, and resolution adjustment.

[0104] In addition, the present method may include a step of visualizing (e.g., visualizing the location) of what is identified as an invasive plant by linking it with a Geographic Information System (GIS) after the third step.

[0105] In addition, the present method may include a step of selecting and applying a model suitable for the shooting time from among a plurality of detection models optimized by season or growth stage.

[0106] In addition, in the present method, the third step may determine only results based on the judgment that are greater than or equal to a preset confidence level as valid values. In other words, the present method calculates a confidence score for the result based on the judgment of the third step and may process only results greater than or equal to a threshold set prior to the third step as valid detections.

[0107] In addition, the present method may include, after the third step, a step of generating a future diffusion prediction model by analyzing the trend of increase or decrease in the area of ​​detected disturbed plants.

[0108] In addition, the present method may include a step of collecting user feedback after the third step to continuously improve the performance of the detection model.

[0109] Additionally, with reference to FIG. 16, the present method can determine whether disturbed plants have recurred in the same area after the third step. To this end, the present method can store data including an image (or a video containing an image) formed according to the determination after the third step, and can determine whether disturbed plants have recurred by comparing it with data for the same area (a past model or a comparison model) that was stored and formed at a past point in time.

[0110] In addition, the present invention provides a system for classifying invasive plants (hereinafter referred to as "the system") according to one embodiment of the present invention that performs the aforementioned method.

[0111] The present system includes an image processing module that forms image data. The image may be captured by a drone, and the image processing module may be a module that removes geometric distortion of the image based on the image captured by the drone and location information (GPS, IMU, etc.) and combines multiple images to generate an orthophoto. Such an image processing module may be implemented in a server-based image processing system or in a computing device mounted on the drone. For example, as described above, the image processing module may include one or more of Pix4D, DJI Terra, Metashape, and GDAL.

[0112] In addition, the system includes a model input module that inputs image data into a machine learning model. The model input model converts orthophoto data into a multidimensional tensor form suitable for input into a machine learning model and may include preprocessing operations based on the TensorFlow or PyTorch framework.

[0113] In addition, the present system includes a classification module that determines whether a candidate object of an invasive plant is an invasive plant from image data. The classification module can classify whether a plant is an invasive plant through a pre-trained machine learning model. As previously mentioned, it may be a Convolutional Neural Network (CNN) or Generative AI-based model. Furthermore, the CNN-based model may include at least one of YOLO, Faster R-CNN, RetinaNet, U-Net, etc.

[0114] This system may be a computer, server, or cloud-based analysis system. That is, this method may be performed by a computer, server, or cloud-based analysis system.

[0115] In addition, the present invention provides a method for mapping the distribution of disturbed plants according to one embodiment of the present invention using the aforementioned method.

[0116] A method for mapping the distribution of disturbed plants according to one embodiment of the present invention includes the step of classifying disturbed plants from plants in a target area using the present method.

[0117] In addition, a method for mapping the distribution of disturbed plants according to one embodiment of the present invention includes the step of mapping the portion of a target area where plants classified as disturbed plants are located using Geographic Information System (GIS) technology.

[0118] The mapping step can visualize the location, area, etc., of the parts where plants classified as invasive species are located, for example, by plotting (representing) them on a map.

[0119] According to this, by observing a wide area, the distribution and area ranges of prickly pear, Japanese knotweed, and goldenrod can be mapped in a short time and determined quantitatively (coordinates, area) through photogrammetry methods.

[0120] In addition, the acquired image data may be preprocessed (one or more of noise removal, resolution adjustment, spectrum correction, etc.). Furthermore, object detection (identification of disturbed plants) may be performed by a machine learning model, specifically by one or more of YOLO, Faster R-CNN, RetinaNet, and U-Net. Additionally, location and time may be integrated, and GPS and capture time information may be linked.

[0121] For reference, in this invention, the algorithm (model) for determining disturbed plants (object detection) may be as follows.

[0122] YOLOv5 / YOLOv8: Strengths in real-time object detection

[0123] Faster R-CNN: Accuracy-Centric Object Detection

[0124] U-Net: Pixel-level identification (segment-level detection)

[0125] DeepLab v3+: Precise segmentation in complex backgrounds

[0126] In addition, the algorithm for tracking time series prediction changes in the present invention may be as follows.

[0127] LSTM / GRU: Time-series-based recurrence probability prediction

[0128] 1D CNN + Attention: Mixed Analysis of Temporal and Spatial Patterns

[0129] In addition, the algorithms for GIS and location analysis in this invention may be as follows.

[0130] GeoPandas + Folium: Location Information Visualization

[0131] NDVI Analysis: Vegetation Index-Based Health Assessment (When Multispectral Imagery is Applied)

[0132] In addition, the algorithm for diffusion prediction and modeling in this invention may be as follows.

[0133] Diffusion Models / SIR Model Variants: Prediction of Interspecies Diffusion Pathways and Rates

[0134] Random Forest + Environmental Variables: Prediction of Dominant Spread Areas Based on Terrain and Weather

[0135] In addition, the present invention proposes a method for controlling invasive plants according to one embodiment of the present invention using the aforementioned method.

[0136] A method for controlling invasive plants according to one embodiment of the present invention includes the steps of classifying invasive plants among plants in a target area using drone orthophotos, confirming the location of the classified invasive plants, and moving a remotely controlled robot to the location of the invasive plants to control the invasive plants.

[0137] According to this, the present institution maps the distribution area of ​​invasive plants among the plants in the target area using drone orthophotos and can weed the invasive plants using a remotely controlled weeding robot.

[0138] Invasive plants may include one or more of prickly pear, Japanese knotweed, and goldenrod.

[0139] Since prickly pear, Japanese knotweed, and goldenrod are invasive plants harmful to humans, controlling them remotely using a weeding robot can minimize the impact on workers during pest control operations.

[0140] In addition, since weeding robots can perform the work of multiple people, the speed of pest control can be fast, and safety can be ensured by allowing work to be done in places that are difficult for humans to access (e.g., minefields, river slopes, areas infested with venomous snakes and wasps).

[0141] According to the method for controlling invasive plants according to one embodiment of the present invention, labor shortages in rural areas caused by population decline and aging can be resolved, a machine can take over the task of removing plants harmful to people, and an orthophoto taken by a drone after control can be used to compare the before and after of the operation.

[0142] In addition, a method for controlling invasive plants according to one embodiment of the present invention may include a step of monitoring changes in the distribution of invasive plants. The monitoring step may be performed before, simultaneously with, or after the control step.

[0143] In addition, a method for controlling invasive plants according to one embodiment of the present invention may include, after the control step, a step of generating an image of the same area and a step of calculating the completion rate of invasive plant removal through the image of the same area.

[0144] The step of calculating the removal completion rate involves inputting an image of the same area into a machine learning model, determining that a disturbing plant is identified as such in the input image of the same area by the machine learning model, and calculating the disturbing plant removal rate (or removal completion rate) by comparing it with the image in the aforementioned method based on the determination result.

[0145] In addition, the present invention proposes an ecosystem-disturbing plant detection and management device (hereinafter referred to as the "this device") that performs the aforementioned method.

[0146] The device includes an image acquisition unit that forms an image in which a target area is shown during a preset period.

[0147] In addition, the device includes a data storage unit that stores acquired images.

[0148] The image acquisition unit and the data storage unit can perform the first step.

[0149] In addition, the device includes an invasive species determination unit that determines invasive plants within an image using the aforementioned pre-trained machine learning model. The invasive species determination unit may perform a third step. As described above, the invasive species determination unit may use a Convolutional Neural Network (CNN)-based object detection or segmentation algorithm. Additionally, the algorithm may include at least one of YOLO, Faster R-CNN, ResNet, and U-Net.

[0150] In addition, the device includes an analysis unit that analyzes the location and distribution area of ​​the determined (detected) disturbed plants. The analysis unit may perform one or more of the third step or steps following the third step of the method described above.

[0151] In addition, the device includes an output unit that displays analysis results and provides a user interface.

[0152] In addition, the device may include an image preprocessing unit that performs preprocessing of the acquired image.

[0153] In addition, the device may include a GPS module. The GPS module can record geographical location information of the acquired image.

[0154] In addition, the device may include a geographic information processing unit that implements a Geographic Information System (GIS) function to visualize the detected results on a map according to the third step.

[0155] In addition, the device may include a time series analysis unit that tracks and analyzes changes in the distribution of disturbed plants over time.

[0156] In addition, the device may include a management plan establishment unit that establishes a plan for removing disturbing plants and evaluates the completion rate after the removal operation.

[0157] In addition, the device may include a model improvement unit that collects user feedback and improves the detection model based on it.

[0158] In addition, the device may include a prediction modeling unit that analyzes the spread pattern of detected disturbing plants and predicts future spread.

[0159] As described above, the present invention provides a comprehensive system and method for effectively detecting and managing ecosystem-disturbing plants by utilizing the latest deep learning technology. By precisely analyzing data obtained from various image sources, the location and distribution of invasive species can be identified, and based on this, systematic management plans can be established. The main advantage of the present invention is that it comprehensively provides efficient monitoring of large areas, high-accuracy detection of invasive species, tracking of changes over time, and verification of the effectiveness of removal operations. Through this, effective management of ecosystem-disturbing plants becomes possible even with limited personnel and resources, thereby making a significant contribution to the conservation of natural ecosystems and the maintenance of biodiversity.

[0160] Furthermore, this system possesses excellent scalability and adaptability, allowing it to be utilized in various environments and conditions, and enabling continuous improvement through user feedback. In the future, system performance can be further enhanced by securing training data on more invasive species and improving algorithms.

[0161] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical concept or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0162] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and the concept of equivalents thereof should be interpreted as being included within the scope of the present invention.

Claims

Claim 1 A method for classifying invasive plants using drone orthophotos, performed by an information processing device having one or more processors, comprising: (a) forming image data including drone orthophotos in which a target area is depicted during a preset period; (b) inputting the image data into a pre-trained machine learning model; and (c) the machine learning model determining from the image data whether a candidate invasive plant object is an invasive plant, wherein the invasive plant includes one or more of prickly pear, Japanese knotweed, and goldenrod, and step (c) comprises the following object set Y including location information (bi), classification result (ci), and confidence (si) of the candidate invasive plant object from the drone orthophotos collected at time t. t Step to calculate: A step of constructing a set of trusted objects by selecting only candidate objects as trusted objects for candidate objects whose classification result (ci) among the set of objects Yt corresponds to any one of prickly burr, Japanese knotweed, and goldenrod, and whose confidence (si) is greater than or equal to the threshold corresponding to the plant class among the first confidence threshold st1 corresponding to prickly burr, the second confidence threshold st2 corresponding to Japanese knotweed, and the third confidence threshold st3 corresponding to goldenrod; a step of determining that objects having the same classification result (c_i) are clustered within the set of trusted objects as the same plant community if such objects are clustered together; a step of determining that the plant community is prickly burr if the time t is from May 1 to October 31 according to Korean standards and the plant community consists of clustered objects including stem structures; a step of determining that the plant community is prickly burr if the time t is from December 1 to March 31 according to Korean standards and the plant community consists of clustered objects that are fallen over A method for classifying disturbed plants using drone orthophotos, comprising: a step of determining that the plant is a Japanese knotweed; and a step of determining that the plant community is a Japanese knotweed if the time t is from October 1 to December 31 according to Korean standards and the plant community consists of objects that are in bloom and clustered, wherein the image data includes the drone orthophoto collected at time t and the GPS coordinates, altitude, and time of shooting of the area corresponding to the image included in the drone orthophoto. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 delete

Images