System and method for analyzing time series growth of crops based on receptacle analysis and tracking

Abstract

A method for analyzing the time-series growth of a crop based on flower cluster analysis and tracking is provided. The method comprises the steps of: acquiring an image of a crop (hereinafter, crop image) captured by a camera; recognizing a predetermined unit object included in the crop from the crop image; clustering the unit object to form a plurality of clusters; reconstructing the plurality of clusters into flower cluster units; and generating linkage information between the unit object and clusters or between the plurality of clusters in each image having different time information.

Description

Technology Field

[0001] The present invention relates to a method for analyzing the time-series growth of crops based on flower cluster analysis and tracking. Background Technology

[0002] Various deep learning-based image recognition technology services are becoming active, and the application of these technologies is expanding not only in the IT sector but also in various fields such as manufacturing and agriculture.

[0003] However, its application in the agricultural sector has been relatively slow. This can be attributed to the difficulty in reflecting the characteristics of crops, which do not possess a fixed form and undergo continuous changes.

[0004] Taking strawberries as an example, while it is possible to sufficiently analyze the characteristics of crops such as strawberries and flowers from a single image, it is difficult to apply techniques such as object tracking to analyze the growth of strawberries because the similarity of the objects is very high. Prior art literature

[0005] Published Patent Application No. 10-2015-0000435 (2015.01.02) The problem to be solved

[0006] An embodiment of the present invention provides a time-series growth analysis system and method for a crop based on flower cluster analysis and tracking, which enables efficient crop tracking by reflecting the characteristics of crop growth in a tracking technology for crop growth analysis, and particularly enables growth analysis by utilizing image analysis technology such as deep learning to confirm the composition of flower clusters.

[0007] However, the technical problem that this embodiment aims to solve is not limited to the technical problem described above, and other technical problems may exist. means of solving the problem

[0008] As a technical means for achieving the aforementioned technical problem, a time-series growth analysis method for crops based on flower cluster analysis and tracking according to the first aspect of the present invention comprises: a step of acquiring an image of a crop (hereinafter, crop image) captured by a camera; a step of recognizing a predetermined unit object included in the crop from the crop image; a step of clustering the unit object to form a plurality of clusters; a step of reconstructing the plurality of clusters into flower cluster units; and a step of generating linkage information between the unit object and clusters or between the plurality of clusters in each image having different time information.

[0009] In some embodiments of the present invention, the step of recognizing a predetermined unit object included in the crop from the crop image may include: a step of recognizing a predetermined object included in the crop from the crop image; and a step of recognizing the predetermined object as a unit object classified by growth stage.

[0010] In some embodiments of the present invention, the step of clustering the unit objects to form a plurality of clusters may include: a step of calculating distance information between a first unit object and a second unit object among the unit objects; and a step of forming the first and second unit objects into a cluster when the distance information is within a preset threshold distance.

[0011] In some embodiments of the present invention, the step of reorganizing the plurality of clusters into flower cluster units may be performed by reorganizing the clusters into flower cluster units when the number of the plurality of unit objects included in the clusters is greater than or equal to the minimum number of unit objects.

[0012] In some embodiments of the present invention, the step of reorganizing the plurality of clusters into flower cluster units may include: detecting a stem for any one unit object within the cluster; detecting the first intersection point among the points where the detected stem is continuous; and reorganizing the unit objects continuous from the first intersection point into a single flower cluster unit.

[0013] In some embodiments of the present invention, the step of reorganizing consecutive unit objects at the first intersection into a single flower box unit may be performed if the number of consecutive unit objects at the first intersection satisfies a minimum number of unit objects required to constitute a flower box unit.

[0014] In some embodiments of the present invention, the step of detecting a stem for any one unit object within the cluster is to detect a straight or curved stem in a direction upward relative to the boundary line of the unit object, and to detect the stem for the unit object by tracking a path in which energy is maximized relative to the boundary line of the unit object.

[0015] Some embodiments of the present invention further include the step of storing information of the recognized unit object and information of the reconstructed cluster, respectively, wherein the information of the unit object includes a unit object index, a class, a location on a crop image, and a time of acquiring the crop image, and the information of the cluster may include a cluster index, a list of unit objects within the cluster, location information of the cluster center point on the crop image, cluster size information on the crop image, and a time of acquiring the crop image.

[0016] In some embodiments of the present invention, the step of generating linkage information between the unit object and clusters or between a plurality of clusters in each crop image having different time information may include: a step of obtaining location information of at least one unit object identified from a crop image at a first time point; a step of obtaining location information of a plurality of clusters identified from a crop image at a second time point that follows the first time point; a step of extracting a cluster having the closest distance among the plurality of clusters having a preset threshold distance from the location information of the unit object; and a step of generating linkage information for the extracted cluster and the unit object.

[0017] In some embodiments of the present invention, the step of extracting a cluster having the closest distance from the first location information among the plurality of clusters may include: a step of comparing the growth stage of a unit object at the first time point with the maximum growth stage in the cluster having the closest distance; and, if the cluster having the closest distance has a maximum growth stage earlier than the growth stage of the unit object, a step of extracting a cluster located at the next closest distance.

[0018] In some embodiments of the present invention, the step of generating linkage information between the unit objects and clusters or between a plurality of clusters in each crop image having different time information may include: a step of obtaining location information of each unit object within a cluster identified from a crop image at a first time point; a step of obtaining location information of a cluster identified from a crop image at a second time point that follows the first time point; a step of extracting a cluster at a second time point that has the closest distance and belongs to a preset threshold distance for each unit object at the first time point; and a step of generating linkage information between the extracted cluster and each unit object.

[0019] In some embodiments of the present invention, the step of extracting a cluster at a second time point having the nearest distance to each unit object at a first time point may include: a step of comparing the maximum growth stage of each unit object at the first time point with the maximum growth stage in the cluster having the nearest distance; and, if the cluster having the nearest distance has a maximum growth stage earlier than the maximum growth stage of the unit object, a step of extracting a cluster located at the next nearest distance.

[0020] Additionally, a time-series growth analysis system for crops based on flower cluster analysis and tracking according to the second aspect of the present invention comprises: a communication module that receives an image of a crop captured by a camera; a memory in which a program for analyzing the crop image and generating linkage information for the crop is stored; and a processor that, by executing the program stored in the memory, recognizes a predetermined unit object included in the crop from the crop image, clusters the unit object to form a plurality of clusters, reconstructs the plurality of clusters into flower cluster units, and generates linkage information between the unit object and clusters or between the plurality of clusters in each image having different time information.

[0021] In addition to this, other methods for implementing the present invention, other systems, and computer-readable recording media for recording a computer program for executing said method may be further provided. Effects of the invention

[0022] According to one embodiment of the present invention described above, the stepwise growth characteristics of a crop can be recorded based on crop images, and in particular, appropriate crop management according to the season can be made possible through flower cluster tracking technology.

[0023] The effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below. Brief explanation of the drawing

[0024] FIG. 1 is a block diagram of a time-series growth analysis system for crops based on flower cluster analysis and tracking according to one embodiment of the present invention. FIG. 2 is a flowchart of a time-series growth analysis method for crops based on flower cluster analysis and tracking according to one embodiment of the present invention. FIG. 3 is a diagram illustrating an example of a clustered result in one embodiment of the present invention. FIG. 4 is a diagram illustrating the content of generating linkage information in one embodiment of the present invention. Specific details for implementing the invention

[0025] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the present invention, and the present invention is defined only by the scope of the claims.

[0026] The terms used in this specification are for describing embodiments and are not intended to limit the invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. The terms "comprises" and / or "comprising" used in this specification do not exclude the presence or addition of one or more other components in addition to the components mentioned. Throughout the specification, the same reference numerals refer to the same components, and "and / or" includes each of the mentioned components and all combinations of one or more. Although terms such as "first," "second," etc., are used to describe various components, these components are not limited by these terms. These terms are used merely to distinguish one component from another. Therefore, the first component mentioned below may be the second component within the technical scope of the invention.

[0027] Unless otherwise defined, all terms used herein (including technical and scientific terms) may be used in a meaning commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0028] In the following, to aid the understanding of those skilled in the art, the background of the invention will be described first, followed by a detailed explanation of the invention. Meanwhile, the crop in this invention will be described using strawberries as an example. However, the crop in this invention is not necessarily limited to strawberries; any crop capable of forming a flower cluster, in other words, where multiple small fruits gather to form a single cluster (such as raspberries, blackberries, etc.), may be the subject.

[0029] Driven by advancements in deep learning technology, various image analysis technologies are being developed and utilized. Similarly, in the agricultural sector, technologies are being developed to analyze crop characteristics and growth based on images. However, the agricultural sector has the disadvantage of being difficult to apply general image analysis technologies to because it deals with growing organisms.

[0030] While the recognition of individual objects is achieved at a very high level, the application of image analysis technology for growth analysis is very limited. The cause of this can be inferred from the following reasons.

[0031] First, there is a problem in identifying the unique characteristics of each object itself. Taking strawberries as an example, it is difficult to find features to distinguish each individual strawberry. For instance, one needs to be able to determine how much a strawberry in a photo taken yesterday has changed compared to a photo taken today at the same location, but in this case, it is very difficult to apply tracking based on object characteristics.

[0032] Secondly, there is a problem in that the location of the object continuously changes due to growth and agricultural activities.

[0033] In other words, crops are living organisms whose shapes continuously change. For example, if an object perceived at image coordinates (30, 100) moves to (50, 120) in the second image over time, and a new object is perceived at the same location (30, 100), the criteria for determining what kind of object the object perceived at (30, 100) in the second image is becomes ambiguous with existing image processing techniques.

[0034] In addition, leaves are constantly moving due to the influence of sunlight, wind, etc., acting as a hindrance to tracking objects.

[0035] In addition, farming activities of agricultural workers also pose difficulties in object recognition. For example, thinning leaves, pruning, and fruit harvesting act as major factors that disrupt the continuity between acquired images.

[0036] Due to these characteristics of crops, there are limitations to tracking changes in crops using general tracking techniques.

[0037] To solve these problems, one embodiment of the present invention aims to provide a growth tracking technology at the inflorescence unit level utilizing the growth characteristics of a crop. Strawberries do not bear fruit as independent individuals, but rather have the characteristic of bearing fruit in inflorescence units, where about five fruits emerge from a single branch end. Furthermore, the fruits emerging from a single inflorescence unit share very similar ripening stages. One embodiment of the present invention can provide a crop growth tracking technology by reflecting these characteristics of the inflorescence unit level.

[0038] In addition, one embodiment of the present invention can provide analysis information at the flower cluster level, such as the location of the flower cluster, the number of flower clusters, the number of unit objects included in the flower cluster, and the growth stage for each unit object, from a crop image. Furthermore, by generating linked information to track how the existing flower cluster has changed over time, it enables the analysis of the continuous growth characteristics of the object.

[0039] Hereinafter, a time-series growth analysis system for crops based on flower cluster analysis and tracking according to one embodiment of the present invention will be described with reference to FIG. 1.

[0040] FIG. 1 is a block diagram of a time-series growth analysis system for crops based on flower cluster analysis and tracking (100, hereinafter referred to as the growth analysis system) according to one embodiment of the present invention.

[0041] A growth analysis system (100) according to one embodiment of the present invention includes a communication module (110), a memory (120), and a processor (130).

[0042] The communication module (110) can receive crop images captured by the camera directly from the camera or receive crop images received from the camera from a storage medium. Additionally, the communication module (110) supports data transmission and reception between internal components or data transmission and reception with an external device. Such a communication module (110) may include both a wired communication module and a wireless communication module. The wired communication module may be implemented as a power line communication device, a telephone line communication device, a cable home (MoCA), Ethernet, IEEE1294, an integrated wired home network, and an RS-485 control device. In addition, the wireless communication module may be composed of modules for implementing functions such as WLAN (wireless LAN), Bluetooth, HDR WPAN, UWB, ZigBee, Impulse Radio, 60GHz WPAN, Binary-CDMA, wireless USB technology and wireless HDMI technology, as well as 5G (5th generation communication), LTE-A (long term evolution-advanced), LTE (long term evolution), and Wi-Fi (wireless fidelity).

[0043] The memory (120) stores programs for generating linked information about crops by analyzing crop images. Here, the memory (120) is a general term for non-volatile storage devices and volatile storage devices that retain stored information even when power is not supplied. For example, the memory (120) may include NAND flash memory such as compact flash (CF) cards, SD (secure digital) cards, memory sticks, solid-state drives (SSDs), and micro SD cards, magnetic computer memory devices such as hard disk drives (HDDs), and optical disc drives such as CD-ROMs and DVD-ROMs.

[0044] The processor (130) can execute software, such as a program, to control at least one other component (e.g., hardware or software component) of the growth analysis system (100) and can perform various data processing or operations.

[0045] The processor (130) recognizes a predetermined unit object included in the crop from a crop image, clusters the unit object to form a plurality of clusters, reconstructs the plurality of clusters into flower cluster units, and generates linkage information between the unit object and clusters or between the plurality of clusters in each image having different time information.

[0046] Meanwhile, in one embodiment of the present invention, the processor (130) may use at least one of machine learning, neural network, or deep learning algorithms as an artificial intelligence algorithm for crop image analysis, and examples of neural networks may include models such as CNN (Convolutional Neural Network), DNN (Deep Neural Network), and RNN (Recurrent Neural Network).

[0047] Hereinafter, a method performed by a growth analysis system (100) according to one embodiment of the present invention will be specifically described with reference to FIGS. 2 to 4.

[0048] FIG. 2 is a flowchart of a time-series growth analysis method for crops based on flower cluster analysis and tracking according to one embodiment of the present invention.

[0049] First, a crop image captured by a camera is acquired (S110). Step S110 can be implemented in various ways, such as acquiring the crop image by directly controlling the camera or acquiring it by receiving a stored crop image.

[0050] Next, a predetermined unit object included in the crop is recognized from the crop image (S120).

[0051] At this time, the present invention recognizes a specific object included in a crop from a crop image, and then recognizes the specific object as a unit object classified by growth stage. That is, a unit object refers to an object recognized from a crop image classified by a growth stage. For example, in the case of a strawberry crop, the unit objects may be classified into a flower bud, a flower, pollination, an unripe strawberry, a ripe strawberry, etc. In other words, one embodiment of the present invention means that the unit object is not simply recognized as a strawberry or a flower, but is specifically classified according to the growth stage, such as a flower bud, a flower, pollination, an unripe strawberry, a ripe strawberry, etc.

[0052] Next, the unit objects are clustered to create multiple clusters (S130).

[0053] In one embodiment, the present invention calculates distance information between a first unit object and a second unit object among unit objects, and if the distance information is within a preset threshold distance, the first and second unit objects can be configured into a cluster.

[0054] FIG. 3 is a diagram illustrating an example of a clustered result in one embodiment of the present invention.

[0055] FIG. 3 illustrates an example of clustering for recognized unit objects. Clustering is performed by calculating distance information on an image between each unit object and grouping unit objects that are close to each other. In one embodiment of the present invention, clustering may be performed through k-means clustering, etc.

[0056] Next, multiple clusters are reorganized into flower cluster units (S140). Since the clusters formed in step S120 were simply clustered based on distance between unit objects, a process is required to verify the composition of each flower cluster for the clustered objects. For example, in the case of strawberry crops, since the ripening and harvesting times of strawberries differ for each flower cluster, it is important to reorganize the clusters into flower cluster units.

[0057] A flower cluster consists of a unit object connected to multiple stems originating from a single branch. In order to reorganize into such a cluster of flower cluster units, one embodiment of the present invention may reorganize a cluster into a cluster of flower cluster units if the number of multiple unit objects included in the cluster is greater than or equal to the minimum number of unit objects. In the case of the strawberry example, a strawberry flower cluster unit may contain at least 5 unit objects. Therefore, if the number of unit objects within a cluster formed in the previous step is less than 5, which is the minimum number of unit objects, the cluster is treated as not being formed. That is, the cluster is treated as existing as individual unit objects rather than as a cluster.

[0058] In addition to or separately, one embodiment of the present invention detects a stem for any one unit object within a cluster to reconstruct a cluster of flower cluster units, and detects the first junction among the points where the detected stem is continuous. Then, the unit objects continuous from the first detected junction can be reconstructed into a single flower cluster unit.

[0059] In this case, one embodiment of the present invention may apply various image processing techniques to track stems from unit objects, and as an example, may apply an energy maximization technique. That is, one embodiment of the present invention may track a specific object (stem) within a crop image by applying an energy maximization technique. Specifically, one embodiment of the present invention may detect a stem in the form of a straight line or a curve in a direction pointing upward relative to the boundary line of the unit object, and may detect the stem for a unit object by tracking the path where energy is maximized relative to the boundary line of the unit object.

[0060] Even when applying the above energy maximization technique, if the number of consecutive unit objects at the first intersection satisfies a minimum number of unit objects required to constitute a firebox unit, it can be reorganized into a firebox unit.

[0061] Meanwhile, one embodiment of the present invention can record crop growth information regarding flower cluster unit classification after reconstructing clusters at the flower cluster unit level. That is, for the utilization of growth information, cluster information recognized through the flower cluster unit reconstruction process must be systematically recorded.

[0062] To this end, one embodiment of the present invention may store information of a recognized unit object and information of a reconstructed cluster, respectively.

[0063] Here, the unit object information refers to objects not included in the clustering classification, that is, objects prior to being included in the flower cluster unit. The unit object information may include at least one of the following: unit object index, class (object classification), location on the crop image, and crop image acquisition time.

[0064] Additionally, the cluster information may include at least one of the following: a cluster index, a list of unit objects within the cluster, location information of the cluster center point on the crop image, cluster size information on the crop image, and the crop image acquisition time. In this case, the list of unit objects within the cluster may include the class of the unit objects and location information of each unit object on the crop image. Additionally, the location information of the center point may be defined by reflecting the distribution of the unit objects.

[0065] Referring again to FIG. 2, next, linkage information is generated between unit objects and clusters in each crop image having different time information, or between multiple clusters (S150).

[0066] FIG. 4 is a diagram illustrating the content of generating linkage information in one embodiment of the present invention.

[0067] Step S140 is a step of storing information for each crop image while reflecting the temporal characteristics of clustering. For example, by confirming that a single unit object (flower) has transitioned into a single cluster (flower bud) in an image acquired several days later, it is possible to track linkage information between unit objects and clusters, or between multiple clusters.

[0068] Figure 4 shows three crop images taken continuously at the same location. In the top crop image, a cluster (G1) on the left and a unit object (O) on the right were recognized, in the middle crop image, three clusters (G1-1, G-1-2, G2) were recognized, and in the bottom crop image, two clusters (G1-1, G2) were recognized.

[0069] In this process, to provide efficient information to farmers, it is necessary to provide information regarding whether the clusters (G1, G1-1, G1-2) located on the left side of each crop image are the same cluster and how they have changed. Additionally, it is necessary to provide information regarding how the unit object (O) on the right side is clustered (G2) and how it changes.

[0070] To this end, one embodiment of the present invention may generate and provide linkage information between unit objects and clusters.

[0071] Specifically, location information of at least one unit object identified from a crop image at a first time point is obtained, and location information of a plurality of clusters identified from a crop image at a second time point following the first time point is obtained.

[0072] Next, among multiple clusters having the unit object's location information and a preset threshold distance, the cluster with the closest distance can be extracted, and linkage information can be generated for the extracted cluster and the unit object. That is, for the cluster targeted for linkage information generation, the cluster's central location must satisfy the condition that it is within the unit object's location information and the preset threshold distance. The linkage information may be a link between the unit object's ID at a first time point and the cluster's ID at a second time point.

[0073] For example, let A be the location information of a unit object identified in a crop image at a first time point (t0), and let B(0, t, N) be the location information of a cluster identified in a crop image at a second time point (t1) (central location). Then, cluster B, which has the closest distance to A, can be extracted. In this case, the distance can be Euclidean distance.

[0074] In this case, one embodiment of the present invention compares the growth stage of a unit object at a first time point with the maximum growth stage in a cluster having the closest distance, and if the comparison result confirms that the cluster having the closest distance has a maximum growth stage earlier than the growth stage of the unit object, the cluster located at the next closest distance can be extracted.

[0075] That is, in the above example, by comparing the growth stage of A with the maximum growth stage of the clusters, if a maximum growth stage earlier than the growth stage of A is identified, that cluster is discarded even if it has the closest distance, and if the maximum growth stage of the next closest cluster has progressed further than the growth stage of A, linkage information can be generated for the next closest cluster.

[0076] In addition, one embodiment of the present invention can generate and provide linkage information between a plurality of clusters.

[0077] Specifically, location information of each unit object within a cluster is obtained from a crop image at a first time point, and location information of the cluster is obtained from a crop image at a second time point that follows the first time point.

[0078] Next, for each unit object at the first time point, a cluster at the second time point that has the closest distance within a preset threshold distance can be extracted, and linkage information between the extracted cluster and each unit object can be generated. In this case, the cluster targeted for linkage information generation must satisfy that the central location of the cluster is within the preset threshold distance from the location information of each unit object included within the cluster. Additionally, the linkage information may link the IDs of the unit objects within the cluster at the first time point with the IDs of the cluster at the second time point.

[0079] For example, let A(0, ⪋, N) be the cluster location information (central location) identified in the crop image at the first time point (t0), and let B(0, ⪋, N) be the cluster location information (central location) identified in the crop image at the second time point (t1). Then, cluster B, which has the closest distance to unit object A0, can be extracted. In this case, the distance can be Euclidean distance.

[0080] In this case, one embodiment of the present invention compares the maximum growth stage of each unit object at a first time point with the maximum growth stage of a cluster having the closest distance, and if the comparison result confirms that the cluster having the closest distance has a maximum growth stage earlier than the maximum growth stage of the unit object, the cluster located at the next closest distance can be extracted.

[0081] That is, in the above example, if the maximum growth stage of A0 and the maximum growth stage of cluster B are compared and a maximum growth stage earlier than the growth stage of A is identified, cluster B is discarded even if it has the closest distance, and if the maximum growth stage of the next closest cluster has progressed further than the maximum growth stage of A, linkage information can be generated for the next closest cluster.

[0082] When linkage information is generated for such clusters, it is possible to perform a more accurate, error-free analysis of growth characteristics at the cluster level. For example, let's assume that in crop images from the same location at the first and second time points, a single cluster was perceived at the first time point, while two clusters were perceived at the second time point. In this case, the stage-by-stage growth characteristics of the crop can be verified by linking the cluster information from the first and second time points.

[0083] In the case of strawberries, at the first time point, a single flower cluster unit is detected and constitutes a single cluster, but there are cases where it is not possible to accurately confirm this through image analysis alone, even if there are actually two flower cluster units. If the collected and stored information is maintained without modification until the second time point, and the strawberries in the first flower cluster unit are ready for harvest while the strawberries in the second flower cluster unit are not yet ripe, they are recognized as a single flower cluster unit (cluster), making it impossible to provide accurate strawberry harvest information, such as missing the harvest time or harvesting too early.

[0084] In contrast, one embodiment of the present invention has the advantage of enabling continuous tracking of crop growth characteristics by linking cluster information at a first time point and a second time point, thereby recognizing that at the first time point a first cluster (unit object: flower) exists, and at the second time point a 1-1 cluster (unit object: ripe strawberry) and a 1-2 cluster (unit object: unripe strawberry) linked to the first cluster exist.

[0085] Furthermore, in one embodiment of the present invention, when the cluster information for the same location in the crop image at the first and second time points in the above example differs, this can be constructed as training data to train an analysis model. That is, the number of cluster information points at the same location at the first and second time points is calculated respectively, and the difference between them is calculated and stored as error information. Subsequently, the cluster information at the first and second time points corresponding to the error information is set as the input, and the analysis model can be trained so that the cluster information at the first and second time points becomes identical.

[0086] Through this, instead of recognizing that only one cluster exists at the first time point in the aforementioned example, it is possible to recognize that two clusters exist, thereby enabling more accurate and continuous cluster tracking. At this time, surrounding information corresponding to the cluster information included in the crop images captured at the first and second time points (distribution of objects, absolute position of objects and relative position between objects, background area, etc.), cluster area information, the number of unit objects within the cluster, and information about surrounding clusters and unit objects can be additionally added and learned as training data for the analysis model.

[0087] Meanwhile, in one embodiment of the present invention, linkage information may be stored in a form such as (ID@timestamp). Additionally, each unit object and cluster may be connected by a link with a unit object or cluster identified at a different time (e.g., object1@2020.11.28.T04:28:11 -> cluster3@2020.12.04.T05.38.11 ->...).

[0088] Meanwhile, in the above description, steps S110 to S150 may be further divided into additional steps or combined into fewer steps according to an embodiment of the present invention. Also, some steps may be omitted as necessary, and the order between steps may be changed. Furthermore, even if other omitted details are included, the details described in FIG. 1 may be mutually applicable with the details in FIG. 2 to 4.

[0089] The time-series growth analysis method for crops based on flower bud analysis and tracking according to one embodiment of the present invention described above may be implemented as a program (or application) and stored on a medium to be executed in combination with a server, which is hardware.

[0090] The aforementioned program may include code encoded in a computer language such as C, C++, JAVA, or machine language, which can be read by the computer's processor (CPU) through the computer's device interface, in order for the computer to read the program and execute the methods implemented in the program. Such code may include functional code related to functions that define the necessary functions for executing the methods, and may include control code related to execution procedures necessary for the computer's processor to execute the functions according to a predetermined procedure. Additionally, such code may further include memory reference code regarding where (address) additional information or media necessary for the computer's processor to execute the functions should be referenced in the computer's internal or external memory. In addition, if the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the above functions, the code may further include communication-related code regarding how to communicate with any other computer or server located remotely using the communication module of the computer, and what information or media to transmit or receive during communication.

[0091] The above-mentioned storage medium refers to a medium that stores data semi-permanently and is readable by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, examples of the above-mentioned storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the above-mentioned program may be stored on various recording media on various servers that the computer can access, or on various recording media on the user's computer. Additionally, the above-mentioned medium may be distributed across networked computer systems, and computer-readable code may be stored in a distributed manner.

[0092] The steps of the method or algorithm described in connection with embodiments of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. The software module may reside in RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), Flash Memory, a hard disk, a removable disk, a CD-ROM, or any form of computer-readable recording medium well known in the art to which the present invention belongs.

[0093] Although embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will understand that the present invention may be implemented in other specific forms without altering its technical concept or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. Explanation of the symbols

[0094] 100: Growth Analysis System 110: Communication module 120: Memory 130: Processor

Claims

Claim 1 A method for analyzing the time-series growth of a crop based on flower cluster analysis and tracking performed by a computer comprises: a step of acquiring an image of a crop (hereinafter, crop image) captured by a camera; a step of recognizing a predetermined unit object included in the crop from the crop image; a step of clustering the unit object to form a plurality of clusters; a step of reconstructing the plurality of clusters into flower cluster units; and a step of generating linkage information between the unit object and cluster or between the plurality of clusters in each image having different time information, wherein the step of generating linkage information between the unit object and cluster or between the plurality of clusters in each crop image having different time information comprises: a step of acquiring location information of each unit object within a cluster identified from a crop image at a first time point; a step of acquiring location information of a cluster identified from a crop image at a second time point that follows the first time point; and a step of extracting a cluster at a second time point that has the closest distance and belongs to a preset threshold distance for each unit object at the first time point. A method for time-series growth analysis of crops based on flower cluster analysis and tracking, comprising the step of generating linkage information between the extracted clusters and each of the unit objects, wherein the step of extracting a cluster at a second time point having the closest distance to each of the unit objects at the first time point includes: a step of comparing the maximum growth stage of each unit object at the first time point with the maximum growth stage in the cluster having the closest distance; and, if the cluster having the closest distance has a maximum growth stage earlier than the maximum growth stage of the unit object, a step of extracting a cluster located at the next closest distance. Claim 2 A method for time-series growth analysis of a crop based on flower cluster analysis and tracking, wherein, in claim 1, the step of recognizing a predetermined unit object included in the crop from the crop image comprises: a step of recognizing a predetermined object included in the crop from the crop image; and a step of recognizing the predetermined object as a unit object classified by growth stage. Claim 3 A method for time-series growth analysis of crops based on flower bud analysis and tracking, wherein the step of clustering the unit objects to form a plurality of clusters comprises: a step of calculating distance information between a first unit object and a second unit object among the unit objects; and a step of forming the first and second unit objects into a cluster when the distance information is within a preset threshold distance. Claim 4 A method for analyzing time-series growth of crops based on flower cluster analysis and tracking, wherein, in claim 1, the step of reorganizing the plurality of clusters into flower cluster units is to reorganize the clusters into flower cluster units when the number of the plurality of unit objects included in the clusters is greater than or equal to the minimum number of unit objects. Claim 5 A method for analyzing time-series growth of a crop based on flower cluster analysis and tracking according to claim 1, wherein the step of reorganizing the plurality of clusters into flower cluster units comprises: a step of detecting a stem for any one unit object within the cluster; a step of detecting the first intersection point among the points where the detected stem is continuous; and a step of reorganizing the unit objects continuous from the first intersection point into one flower cluster unit. Claim 6 In claim 5, the step of reorganizing consecutive unit objects at the first intersection into a single flower cluster unit is to reorganize into a single flower cluster unit when the number of consecutive unit objects at the first intersection satisfies a minimum number of unit objects required to constitute a flower cluster unit, a method for time-series growth analysis of crops based on flower cluster analysis and tracking. Claim 7 In claim 5, the step of detecting a stem for any one unit object within the cluster is to detect a straight or curved stem in a direction upward relative to the boundary line of the unit object, and to detect the stem for the unit object by tracking a path in which energy is maximized relative to the boundary line of the unit object, a time-series growth analysis method for crops based on flower bud analysis and tracking. Claim 8 A method for time-series growth analysis of a crop based on flower bud analysis and tracking according to claim 1, further comprising the step of storing information of the recognized unit object and information of the reconstructed cluster, respectively, wherein the information of the unit object includes a unit object index, a class, a location on a crop image, and a time of crop image acquisition, and the information of the cluster includes a cluster index, a list of unit objects within the cluster, location information of the cluster center point on the crop image, cluster size information on the crop image, and a time of crop image acquisition. Claim 9 A method for time-series growth analysis of a crop based on flower bud analysis and tracking according to claim 1, wherein the step of generating linkage information between the unit object and clusters or between multiple clusters in each crop image having different time information comprises: a step of obtaining location information of at least one unit object identified from a crop image at a first time point; a step of obtaining location information of multiple clusters identified from a crop image at a second time point that follows the first time point; a step of extracting a cluster having the closest distance among the multiple clusters having a preset threshold distance from the location information of the unit object; and a step of generating linkage information for the extracted cluster and the unit object. Claim 10 A time-series growth analysis method for crops based on flower cluster analysis and tracking according to claim 9, wherein the step of extracting a cluster having the first location information and the nearest distance among the plurality of clusters comprises: a step of comparing the growth stage of a unit object at the first time point with the maximum growth stage in the cluster having the nearest distance; and, if the cluster having the nearest distance has a maximum growth stage earlier than the growth stage of the unit object, a step of extracting a cluster located at the next nearest distance. Claim 11 delete Claim 12 delete Claim 13 A communication module that receives an image of a crop (hereinafter, crop image) captured by a camera; a memory in which a program for analyzing the crop image and generating linkage information for the crop is stored; and a processor that, by executing the program stored in the memory, recognizes a predetermined unit object included in the crop from the crop image, clusters the unit object to form a plurality of clusters, reconstructs the plurality of clusters into flower cluster units, and generates linkage information between the unit object and cluster or between the plurality of clusters in each image having different time information, wherein the processor, when generating linkage information between the unit object and cluster or between the plurality of clusters in each crop image having different time information, obtains location information of each unit object within a cluster identified from the crop image at a first time point, obtains location information of a cluster identified from the crop image at a second time point that follows the first time point, extracts a cluster at the second time point that has the closest distance and belongs to a preset threshold distance for each unit object at the first time point, generates linkage information between the extracted cluster and each unit object, and each at the first time point A time-series growth analysis system for crops based on flower cluster analysis and tracking, characterized by extracting a cluster at a second time point having the closest distance to each unit object, comparing the maximum growth stage of each unit object at the first time point with the maximum growth stage of the cluster having the closest distance, and if the cluster having the closest distance has a maximum growth stage earlier than the maximum growth stage of the unit object, extracting a cluster located at the next closest distance.

Images