Device and method for predicting growth status from crop images based on artificial intelligence model

Abstract

The present invention relates to a method for predicting a growth state from an image of a crop based on an artificial intelligence model, and may include the steps of receiving images of a plurality of crops, extracting growth parts from the images of the plurality of crops, determining a growth pattern of the crop based on the images of the plurality of crops from which the growth parts have been extracted, and predicting a growth state of the crop based on the determination of the growth pattern.

Description

Technology Field

[0001] The present invention relates to an apparatus and method for predicting the growth state from an image of a crop based on an artificial intelligence model, and more specifically, to an apparatus and method for predicting the growth state from an image of a crop using segmentation deep learning. Background Technology

[0002] With the Fourth Industrial Revolution, smart agriculture is emerging as data replaces labor, knowledge, and experience through the automation and intelligence of agricultural operations. This is attracting attention as an industrial innovation aimed at securing future agricultural competitiveness and enhancing added value, as it links cutting-edge technologies of the Fourth Industrial Revolution to agriculture, a primary industry, and expands the potential of agriculture in the early stages of digital innovation.

[0003] Smart agriculture is essential for meeting food demand with limited land, resources, and manpower, and for responding to risks such as food security crises and climate change; furthermore, due to changes in the international situation, such as trade disputes between nations, the importance of securing food sovereignty from a national security perspective is significantly increasing.

[0004] The domestic smart agriculture market has been growing steadily over the past few years. Along with the government's support policies for smart agriculture, interest among farmers is increasing. In particular, smart farms utilizing IoT technology, precision pest control using drones, and AI-based crop management systems are receiving attention.

[0005] In developed countries such as the United States, Japan, and Europe, precision agriculture technology is rapidly advancing, and agricultural data analysis utilizing artificial intelligence and big data, crop monitoring and pest control using drones, and autonomous agricultural machinery have been commercialized.

[0006] Under these circumstances, open-field farming plays a pivotal role in Korea's agriculture, but its productivity and efficiency have recently been declining due to various problems. Consequently, there is an urgent need for solutions utilizing advanced technology. Prior art literature

[0007] Korean Published Patent No. 10-2020-0029657 The problem to be solved

[0008] The present invention aims to solve the problems of the prior art as described above by predicting the growth status from images of crops based on an artificial intelligence model. means of solving the problem

[0009] A method for predicting a growth state from an image of a crop based on an artificial intelligence model according to the present invention for achieving the above objective may include: receiving images of a plurality of crops; extracting growth parts from the images of the plurality of crops; determining a growth pattern of the crop based on the images of the plurality of crops from which the growth parts have been extracted; and predicting a growth state of the crop based on the determination of the growth pattern.

[0010] Here, the step of determining the growth pattern of the crop may include an artificial intelligence model applying images of a plurality of crops from which the growth parts have been extracted as input variables and applying the growth pattern of the crop as an output variable to learn.

[0011] In addition, it may include images of a plurality of crops from which the growth parts have been extracted being arranged in a time series and used for learning to determine the growth pattern of the crops.

[0012] Additionally, the images of a plurality of crops from which the growth parts have been extracted may be arranged in a time series, and then the growth parts may be labeled.

[0013] In addition, the above labeling may include at least one of germination, early growth, and maturation.

[0014] In addition, the images of a plurality of crops from which the growth parts have been extracted are converted into feature vectors, and the feature vectors are connected in a time series to learn the growth pattern of the crops.

[0015] Additionally, the step of predicting the growth state of the crop may include predicting the growth range of the growth part of the crop after a final point in time in a time-series manner from images of a plurality of crops from which the growth part has been extracted, based on the growth pattern.

[0016] In addition, after predicting the growth range of the growth part of the crop, it may include displaying the predicted growth range of the growth part of the crop on the image of the crop that is last arranged among the images of the plurality of crops from which the growth part was extracted.

[0017] In addition, the above artificial intelligence model can be implemented as an LSTM (Long Shot Term Memory) model.

[0018] Meanwhile, an apparatus for extracting growth parts from an image of a crop based on an artificial intelligence model according to the present invention for achieving the above objective may include: a receiving unit for receiving images of a plurality of crops; a growth part extraction unit for extracting growth parts from the images of the plurality of crops; and a growth state prediction unit for determining a growth pattern of the crop based on the images of the plurality of crops from which the growth parts have been extracted, and predicting a growth state of the crop based on the determination of the growth pattern. Effects of the invention

[0019] The apparatus and method for predicting the growth state from an image of a crop according to the present invention have the following effects.

[0020] First, accuracy can be improved. The apparatus and method for predicting the growth state in an image of a crop according to the present invention can predict the growth state in an image of a crop by using an LSTM model to configure a layer that learns data changing over time and learning it. Accordingly, there is an effect of predicting the growth state in an image of a crop with high accuracy.

[0021] Second, agricultural productivity is improved. The device and method for predicting the growth status from crop images according to the present invention can improve the overall agricultural productivity of the nation by utilizing artificial intelligence technology to precisely diagnose and manage the growth status of crops. In other words, the device and method for predicting the growth status from crop images according to the present invention can improve the growth speed and quality of crops by providing optimal agricultural management methods through precise data analysis and real-time monitoring, increase yield per unit area, and promote the efficient use of agricultural resources. Furthermore, by detecting and responding to risks such as climate change and pests in advance through an AI-based prediction model, it can minimize crop damage and ensure stable production, thereby increasing the national food self-sufficiency rate and strengthening food security by reducing dependence on imports.

[0022] Third, sustainable agriculture can be realized. The apparatus and method for predicting growth status from crop images according to the present invention can realize sustainable agriculture by responding to climate change and increasing resource efficiency, such as water, fertilizers, and pesticides, through artificial intelligence-based data analysis. Brief explanation of the drawing

[0023] FIG. 1 is a block diagram of a device for predicting the growth state from an image of a crop according to an embodiment of the present invention. FIG. 2 is a flowchart of a method for extracting a growth part from an image of a crop according to an embodiment of the present invention. FIG. 3 is a diagram showing the recognition of a growth area performed by a device for predicting the growth state in an image of a crop according to an embodiment of the present invention. Figure 4 is a diagram showing the U-Net model structure used in a device for predicting the growth state from an image of a crop according to an embodiment of the present invention. FIG. 5 is a diagram schematically illustrating a device for predicting the growth state in an image of a crop according to an embodiment of the present invention, which divides the growth area in an image of a crop. FIG. 6 is a diagram schematically illustrating the extraction of a growth area from an image of a crop in a device for predicting the growth state from an image of a crop according to an embodiment of the present invention. FIG. 7 is a flowchart of a method for predicting the growth state from an image of a crop according to an embodiment of the present invention. FIG. 8 is a diagram showing the LSTM model structure used in a device for predicting the growth state from an image of a crop according to an embodiment of the present invention. FIG. 9 is a diagram illustrating the process of learning the growth state in an image of a crop using an LSTM model structure used in a device for predicting the growth state in an image of a crop according to an embodiment of the present invention. FIG. 10 is a diagram exemplarily illustrating the prediction of a crop growth state in an image of a crop according to an embodiment of the present invention. Specific details for implementing the invention

[0024] Specific structural or functional descriptions regarding embodiments according to the concept of the present invention disclosed herein are provided merely for the purpose of explaining embodiments according to the concept of the present invention, and embodiments according to the concept of the present invention may be implemented in various forms and are not limited to the embodiments described herein.

[0025] Since embodiments according to the concept of the present invention may be subject to various modifications and may take various forms, embodiments are illustrated in the drawings and described in detail in this specification.

[0026] However, this is not intended to limit embodiments according to the concept of the present invention to specific disclosed forms, and includes all modifications, equivalents, or substitutions that fall within the spirit and scope of the present invention.

[0027] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0028] The terms used in this specification are used merely to describe specific embodiments and are not intended to limit the invention.

[0029] The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, terms such as “comprising” or “having” are intended to specify the existence of the disclosed features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0030] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which this invention pertains.

[0031] Terms such as those defined in commonly used dictionaries should be interpreted as having meanings consistent with their meanings in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0032] Hereinafter, the specific technical configuration of the present invention will be examined with reference to the attached drawings.

[0033] <생육 상태 예측 장치>

[0034] FIG. 1 is a block diagram of a device for predicting the growth state from an image of a crop according to an embodiment of the present invention.

[0035] As illustrated in FIG. 1, a growth state prediction device (10) for extracting a growth part from an image of a crop according to an embodiment of the present invention may include a receiving unit (100), a growth part extraction unit (200), and a growth state prediction unit (300).

[0036] The receiver (100) is configured to receive an image containing a crop. This receiver (100) can receive an image containing a crop, such as an onion or garlic, from an external device. Alternatively, it may receive an image containing a crop captured from a shooting module.

[0037] Here, the image containing the crop may be an image that can determine the growth status of the crop. Specifically, the image containing the crop may be an image that includes plant height, number of leaves, leaf length, leaf width, etc.

[0038] The growth part extraction unit (200) is configured to extract growth parts from an image containing crops. This growth part extraction unit (200) can perform segmentation deep learning based on an artificial intelligence model such as a U-Net model.

[0039] That is, the growth part extraction unit (200) can perform learning by configuring a learning layer that reflects the input image size and characteristics of the crop data.

[0040] In addition, the growth part extraction unit (200) can extract the growth part from an image containing a crop by extracting features of the crop from an image containing the crop based on the learned results.

[0041] In addition, the growth part extraction unit (200) can evaluate the performance of the U-Net model by applying a cross-entropy loss function to the U-Net model, which is an artificial intelligence model.

[0042] Additionally, the growth part extraction unit (200) may apply an early learning stoppage technique to prevent overfitting caused by overlearning that occurs when the U-Net model exceeds the set learning speed.

[0043] The growth state prediction unit (300) is configured to predict the growth state based on an image of a crop from which a growth part has been extracted. This growth state prediction unit (300) can predict the growth state of the crop using an LSTM model. That is, it can learn the growth state of the crop while receiving multiple images of the crop from which a growth part has been extracted in a time series, and then predict the growth state of the crop.

[0044] <생육 부위 추출 방법>

[0045] Below, the process of extracting the growth parts will be explained in detail with reference to the drawings.

[0046] FIG. 2 is a flowchart of a method for extracting a growth part from an image of a crop according to an embodiment of the present invention.

[0047] As illustrated in FIG. 2, a method for extracting a growth part from an image of a crop according to an embodiment of the present invention can initially receive an image containing a crop. <s200>

[0048] Specifically, the growth state prediction device (10) can receive an image containing a crop, such as an onion or garlic, from an external device. This external device may be a database that stores images containing crops. Accordingly, it can receive an image of a crop stored in the database.

[0049] Alternatively, images of crops captured from the camera module can be received in real time.

[0050] Here, the image containing the crop may be an image that can determine the growth status of the crop. That is, the image containing the crop may be an image that captures a specific area including the height, number of leaves, leaf length, leaf width, etc.

[0051] Next, based on an artificial intelligence model, the crop growth area and the area outside the growth area can be recognized in the image. <s210>This will be explained with reference to Fig. 3.

[0052] FIG. 3 is a diagram showing the recognition of a growth area performed by a device for predicting the growth state in an image of a crop according to an embodiment of the present invention.

[0053] As illustrated in FIG. 3, the growth state prediction device (10) according to an embodiment of the present invention can learn to recognize the growth area of ​​a crop and an area outside the growth area based on an original image of an image containing a crop such as an onion or garlic.

[0054] Here, the artificial intelligence model included in the growth state prediction device (10) can learn images containing crops based on segmentation deep learning.

[0055] Segmentation deep learning is an algorithm that analyzes images or videos at the pixel level to classify which class each pixel belongs to. This is one of the most important tasks in the field of computer vision and can be used in various application fields, such as medical image analysis, autonomous driving, object detection, and robot vision.

[0056] Such segmentation deep learning can be classified into semantic segmentation and instance segmentation.

[0057] Semantic segmentation assigns every pixel in an image to a specific class (e.g., person, road, sky, etc.) and does not distinguish individual objects within that class. For example, even if there are multiple people in an image, they are all classified only as the "person" class.

[0058] Instance segmentation distinguishes objects belonging to the same class within an image individually. For example, if there are multiple people in an image, it identifies each person as an independent object.

[0059] Models used in such segmentation deep learning may include FCN (Fully Convolutional Network), Mask R-CNN, DeepLab (v3, v3+), and U-Net.

[0060] Here, the FCN (Fully Convolutional Network) is a fully convolutional network that extends the general CNN structure to generate pixel-level output.

[0061] In addition, Mask R-CNN is a model that combines object detection and instance segmentation, and can add pixel-level mask generation based on Faster R-CNN.

[0062] In addition, DeepLab(v3, v3+) uses atrous convolution to effectively process spatial information and has excellent segmentation performance in boundary regions.

[0063] Finally, U-Net is frequently used in medical image processing and is based on an encoder-decoder structure. It has the advantage of being able to perform segmentation with high accuracy.

[0064] Such a U-Net model is used in the present invention to learn to extract growth parts from an image containing a crop. That is, the growth state prediction device (10) may learn to extract growth parts from an image containing a crop. This will be explained with reference to FIG. 4.

[0065] Figure 4 is a diagram showing the U-Net model structure used in a device for predicting the growth state from an image of a crop according to an embodiment of the present invention.

[0066] As illustrated in FIG. 4, the growth state prediction device (10) according to an embodiment of the present invention can learn to recognize the growth area of ​​a crop and the area outside the growth area based on an original image of an image containing a crop using an artificial intelligence model with a U-Net model.

[0067] Here, we will explain the U-Net model in more detail.

[0068] The U-Net model generates segmentation results by classifying input images at the pixel level. Its name is derived from the U-shaped network structure.

[0069] The U-Net model is based on an encoder-decoder architecture and can be broadly divided into two parts.

[0070] The encoder extracts features by downsampling the image. Accordingly, it consists of multiple convolution and pooling layers and captures contextual features of the image. Furthermore, as the network progresses, spatial resolution decreases and the number of channels increases.

[0071] The decoder restores the image to its original size by upsampling the features extracted from the encoder. It consists of an upsampling layer and a convolutional layer. In the decoder stage, each step is connected to the corresponding layer of the encoder via a skip connection. This allows for the compensation of spatial information lost at low resolutions.

[0072] The main characteristic of such a U-Net model is its symmetric structure. It is a U-shaped symmetric network, so named because the encoder and decoder stages have similar depth and structure.

[0073] In addition, another feature of the U-Net model is the Skip Connection, which passes the encoder's output (feature map) to the decoder to compensate for information lost at low resolutions, which can help to better restore segmentation boundaries and details.

[0074] In addition, since the U-Net model consists entirely of convolutional layers, it can be used regardless of input size and can output pixel-level results through global downsampling and upsampling processes.

[0075] Accordingly, in the present invention, an encoder can be designed by configuring a learning layer that reflects the size of the image containing the crop and the characteristics of the crop in an image containing the crop, and by configuring a convolutional layer that extracts features within the image containing the crop. Accordingly, features of the crop can be extracted from the image containing the crop after downsampling.

[0076] Furthermore, through upsampling of images containing crops in the decoder, the crop-containing images that were downsampled to extract crop features in the convolutional layer can be restored to their original size. In addition, a layer can be designed to minimize feature loss regarding crop-containing images by adding a skip connection that connects directly to the decoder.

[0077] In this way, the growth state prediction device (10) of the present invention learns to recognize the growth area in an image containing a crop based on a U-Net model, thereby enabling the recognition of the growth area of ​​the crop and the area outside the growth area in an image containing a crop.

[0078] That is, the growth part device (10) can construct a learning dataset by applying an image containing a crop as an input variable and recognizing the growth part according to the characteristics of the crop as an output variable. Accordingly, the growth part device (10) can be trained to output the recognition of the growth part according to the characteristics of the crop when an image containing a crop is input by adjusting hyperparameters based on a U-Net model.

[0079] In addition, the growth state prediction device (10) can learn to remove surrounding background areas, such as soil or structures, in addition to crops.

[0080] Returning to Fig. 2, the image containing the crop can then be divided into a growth area and an area outside the growth area. <s220>This will be explained with reference to Fig. 5.

[0081] FIG. 5 is a diagram schematically illustrating a device for predicting the growth state in an image of a crop according to an embodiment of the present invention, which divides the growth area in an image of a crop.

[0082] As shown in FIG. 5, the artificial intelligence model can learn to divide the growth area and the area outside the growth area in the image containing the crop based on the fact that when an image containing the crop is received, the model learns to recognize the image containing the crop into the growth area and the area outside the growth area in the image containing the crop.

[0083] Specifically, the growth area device (10) can be trained to output a division of the growth area and an area other than the growth area according to the characteristics of the crop by applying an image of a crop with the surrounding background area removed as an input variable and an image set as an output variable in which the growth area of ​​the crop is valid and the area not corresponding to the crop is invalid, and by adjusting hyperparameters based on a U-Net model, when an image of a crop with the surrounding background area removed is input, the device can be trained to output a division of the growth area and the area other than the growth area according to the characteristics of the crop.

[0084] Here, the growth area may refer to, for example, the leaf portion. Or, the growth area may refer to the stem portion.

[0085] Accordingly, the growth state prediction device (10) can divide and distinguish the growth area of ​​the crop and the area other than the growth area, which is not valid, in an image containing the crop, based on learning to divide the valid growth area of ​​the crop and the area other than the growth area, such as weeds, in an image containing the crop.

[0086] For example, the growth state prediction device (10) can distinguish between them by dividing the valid growth area of ​​the crop in an image containing the crop into green and dividing the area other than the invalid growth area into brown.

[0087] This is just one example, but the growth state prediction device (10) can divide the valid growth area and the area other than the invalid growth area in an image containing a crop into various categories.

[0088] Returning to Fig. 2, based on the last segmented image, only the growth area can be extracted. <s230>This will be explained with reference to Fig. 6.

[0089] FIG. 6 is a diagram schematically illustrating the extraction of a growth area from an image of a crop in a device for predicting the growth state from an image of a crop according to an embodiment of the present invention.

[0090] As shown in FIG. 6, the growth state prediction device (10) can divide the valid growth area and the area other than the invalid growth area in an image containing a crop, and then separate the growth area and the area other than the growth area in the image containing the crop to extract only the growth area.

[0091] Specifically, the growth state prediction device (10) can separate the image containing the crop into an image containing the crop from which only the valid growth area is extracted and an image containing the crop from which only the area other than the invalid growth area is extracted, and then generate the image of the crop from which only the growth area is extracted as a growth prediction image.

[0092] Accordingly, the growth state prediction device (10) can extract and store only the growth area of ​​the crop, which is the growth part, from an image containing the crop, and this can be used as image information to determine the growth state of the crop. This extraction can be implemented using the U-Net model in the same way.

[0093] Meanwhile, the growth state prediction device (10) can evaluate the performance of the artificial intelligence model by applying a cross-entropy loss function to the artificial intelligence model.

[0094] Here, the Cross-Entropy loss function is a loss function used to evaluate the performance of artificial intelligence models in classification problems. It can be used to measure how well a model matches the actual correct answer in classification problems that primarily deal with probability-based predictions (softmax output).

[0095] Specifically, cross-entropy loss measures the difference between the predicted probability distribution and the actual label distribution (correct answer). It can be expressed mathematically as follows.

[0096]

[0097] Here, C is the number of classes.

[0098] Also, y i is the actual label (correct answer) value, in the form of a one-hot vector with 1 for the correct class and 0 for the remaining classes.

[0099] In addition, is the model's predicted probability value (softmax output).

[0100] In the above equation, the higher the probability the model assigns to the correct class, the smaller the loss value becomes.

[0101] The model's output is a probability distribution normalized by the softmax function. These values ​​represent the probability of belonging to each class. Accordingly, the cross-entropy loss is the probability of the correct class ( It is designed so that the higher ) is, the smaller the loss, and the lower it is, the larger the loss. This can induce the model to assign a high probability to the correct class.

[0102] Therefore, the cross-entropy loss function is used to reduce the loss value as the model learns. Since the model's prediction accuracy increases as the loss decreases, the value of the cross-entropy loss function is used as one of the important indicators of model performance.

[0103] Consequently, cross-entropy loss is a tool that can precisely evaluate how well a deep learning model correctly predicts the answer through probabilistic prediction. This allows for continuous improvement of the model and enhancement of learning performance.

[0104] Accordingly, in the present invention, the growth state prediction device (10) can evaluate the performance of the U-Net model by applying a cross-entropy loss function to the U-Net model, which is an artificial intelligence model. At this time, when evaluating the performance of the U-Net model, the performance can be performed by setting the Mean Absolute Error (MAE), which is a regression model performance evaluation metric.

[0105] Meanwhile, in the present invention, the growth state prediction device (10) applies a fast and stable Adam optimizer learning algorithm and can adjust the learning speed of the artificial intelligence model by appropriately adjusting the learning rate value.

[0106] Here, the Adam Optimizer is a widely used algorithm to optimize the training process of deep learning models. It can effectively control the model's training speed by adaptively adjusting the learning rate. In this context, Adam refers to Adaptive Moment Estimation.

[0107] In the Adam Optimizer, the learning rate value significantly influences the speed and quality of the AI ​​model's learning. Selecting an appropriate learning rate value enables fast convergence and stable learning, but an inappropriate learning rate value can cause the following problems.

[0108] In other words, if the learning rate value is too large, the cross-entropy loss function value may oscillate or diverge. Also, if the learning rate value is too small, learning may proceed very slowly or get stuck at a local optimum.

[0109] Therefore, the growth state prediction device (10) can optimize the process of the U-Net model learning the characteristics of the crop from an image containing the crop by appropriately adjusting the learning rate value based on the artificial intelligence model.

[0110] At this time, one of the following methods can be used to adjust the learning rate value.

[0111] First, Step Decay can be used. This Step Decay can decrease the learning rate value at regular intervals.

[0112] Second, exponential decay can be used. This exponential decay can decrease the learning rate value exponentially.

[0113] Third, cosine annealing can be used. This cosine annealing can gradually decrease the learning rate value along the cosine curve.

[0114] Here, the growth state prediction device (10) may apply an early learning stoppage technique to prevent overfitting caused by over-learning that occurs when the artificial intelligence model exceeds the set learning speed.

[0115] At this time, the number of training iterations, that is, the number of times the entire dataset passes through the neural network or the number of times the entire dataset is trained, can be set through the number of early cessation of training.

[0116] Meanwhile, in the present invention, a growth state prediction device (10) learns an image containing a crop to recognize a growth area of ​​the crop and an area other than the growth area of ​​the crop, and a batch normalization technique that appropriately distributes the image containing the crop can be applied in relation to dividing the image into a growth area of ​​the crop and an area other than the growth area of ​​the crop.

[0117] Specifically, Batch Normalization (BN) is a technique used to accelerate and stabilize the training process of deep learning models. It can mitigate changes in data distribution (Internal Covariate Shift) by normalizing input data in mini-batch units at each layer of the neural network.

[0118] Batch normalization calculates the batch mean and variance, performs normalization, and can also carry out scaling and shifting. Consequently, variations in the input distribution are mitigated, allowing for the use of larger learning rates and thus accelerating training. Additionally, it becomes less sensitive to weight initialization, enabling more stable training. Furthermore, normalization provides a normalization effect that contributes to reducing overfitting. It can also mitigate the vanishing gradient problem, even in very deep networks.

[0119] Applying batch normalization allows the model to train stably, which generally enables the use of larger learning rates. This accelerates the training process and enables better convergence.

[0120] <생육 상태 예측 방법>

[0121] Below, we will explain in detail the process of predicting growth conditions by referring to the drawings.

[0122] FIG. 7 is a flowchart of a method for predicting the growth state from an image of a crop according to an embodiment of the present invention.

[0123] As illustrated in FIG. 7, a method for predicting the growth state from an image of a crop according to an embodiment of the present invention can initially receive images of a plurality of crops. <s700>

[0124] Specifically, the growth state prediction device (10) can receive an image containing a specific crop, such as an onion or garlic, from an external device as described above. This external device may be a database that stores images containing crops. Accordingly, it can receive images of crops stored in the database.

[0125] Alternatively, images of crops captured from the camera module can be received in real time.

[0126] Next, the growth part of a specific crop can be extracted from images of multiple crops. <710>

[0127] Specifically, the growth state prediction device (10) can generate an image of a crop from which the growth part is extracted by extracting only the growth area of ​​the crop from an image containing the crop as described above, and the image of the crop from which the growth part is extracted can be used to determine the growth state of the crop.

[0128] Subsequently, based on the image of the crop from which the growth part of a specific crop has been extracted, the growth pattern of the crop can be determined. <s720>

[0129] Specifically, in the present invention, a growth state prediction device (10) can determine the growth pattern of a crop based on an artificial intelligence model. A Long Short-Term Memory (LSTM) model may be used as such an artificial intelligence model. This will be explained with reference to FIGS. 8 and 9.

[0130] FIG. 8 is a diagram showing the LSTM model structure used in a device for predicting the growth state from an image of a crop according to an embodiment of the present invention, and FIG. 9 is a diagram showing the entire process of learning the growth state from an image of a crop using the LSTM model structure used in a device for predicting the growth state from an image of a crop according to an embodiment of the present invention.

[0131] As illustrated in FIGS. 8 and 9, in the present invention, a growth state prediction device (10) can determine the growth pattern of a crop from an image of a crop from which a growth part has been extracted, based on an LSTM model.

[0132] Here, the LSTM (Long Shot Term Memory) model is a type of RNN (Recurrent Neural Networks) model. An RNN model is a type of artificial neural network in which hidden nodes are connected by directed edges to form a recurrent structure. Such RNN models are known to be suitable for processing sequentially appearing data, such as speech and text.

[0133] A chronic problem with RNN models is that gradient vanishing occurs during the learning process for long-term memory. LSTM models are a means to solve this existing problem of RNN models by preserving state values ​​while transmitting them, thereby increasing the durability of long-term memory.

[0134] Accordingly, in the present invention, the growth state prediction device (10) utilizes this to track the change value in the accumulated time unit frames and extracts the predicted value from the image of the crop from which the growth part is extracted, thereby determining the growth pattern according to the data accuracy.

[0135] Specifically, the growth state prediction device (10) can receive and arrange images of a plurality of crops from which growth parts have been extracted in a time series and then perform labeling. This labeling involves labeling the growth state on the image of each crop from which growth parts have been extracted, and the labeling may include germination, early growth, maturity, etc., and at least one of these may be included.

[0136] In addition, images of multiple crops from which growth parts have been extracted can be prepared as input to an LSTM model by adjusting and normalizing the size of each image through preprocessing after labeling.

[0137] In addition, the growth state prediction device (10) may have images of multiple crops from which growth parts have been extracted as input variables, and growth patterns at each point in time may be applied as output variables. Accordingly, the images of multiple crops from which growth parts have been extracted may be converted into feature vectors and configured as a time-series learning dataset.

[0138] In addition, the LSTM model processes feature vectors along the time axis, learns temporal relationships, and can improve learning performance by stacking multiple LSTM layers. Accordingly, images of multiple crops from which growth parts have been extracted are accumulated over time, and based on this, smooth advancement can be achieved through the optimization of the LSTM model and the adjustment of hyperparameters.

[0139] At this time, when the initial LSTM model is used, one or two layers are designed, and layers may be added later depending on the amount and complexity of the training dataset. Each layer is set to have 50 to 100 neurons, and the number of neurons can be expanded later in the same way as the layer design.

[0140] In addition, dropout may be set to prevent overfitting caused by overtraining that exceeds the set learning model speed. Dropout may be set at a rate of 0.2 to 0.5.

[0141] In this way, the growth state prediction device (10) can determine the growth pattern at each point in time based on images of multiple crops from which growth parts have been extracted.

[0142] Meanwhile, in the present invention, the growth state prediction device (10) can apply an MSE loss function to evaluate the performance of the LSTM model. This MSE (Mean Squared Error) is a loss function commonly used when dealing with regression problems in machine learning and deep learning. MSE is a value calculated by squaring the difference between the model's predicted value and the actual value and calculating the average; it has the characteristic that the value becomes smaller as the prediction is closer to the actual value, and the value becomes larger as it is further away.

[0143] In addition, MSE can measure the performance of the entire data because it averages the error over all data points, treats both negative and positive errors as positive by squaring the error, imposes a greater penalty on large errors, and is a smooth function, making it suitable for optimization algorithms such as Gradient Descent.

[0144] In addition, the growth state prediction device (10) adopts a fast and stable Adam optimizer learning algorithm, sets the initial learning rate to 0.001, and can adjust the learning rate according to the learning results.

[0145] In addition, the growth state prediction device (10) sets epochs based on the number of neurons designed in each layer (50 to 100) and proceeds with learning, and uses an Early Stopping technique to stop learning when the performance of the LSTM model is not improved or when overfitting occurs, and when Early Stopping occurs, the number of neurons and epochs can be adjusted to optimize the model learning performance.

[0146] Additionally, the growth state prediction device (10) can evaluate the performance of the LSTM model by setting MAE evaluation indicators and adjust hyperparameters. Accordingly, the growth state prediction device (10) can proceed with retraining to improve the performance and result of the LSTM model after setting optimized parameters.

[0147] Finally, based on the determination of specific growth patterns, the growth status of the crop can be predicted. <s730>This will be explained with reference to Fig. 10.

[0148] FIG. 10 is a diagram exemplarily illustrating the prediction of a crop growth state in an image of a crop according to an embodiment of the present invention.

[0149] As illustrated in FIG. 10, the growth state prediction device (10) can determine the growth pattern of a crop based on images of a plurality of crops from which growth parts have been extracted, and use this to predict the growth state.

[0150] Specifically, let us assume that images of multiple crops in which the stems and leaves have grown in a time-series manner are received. At this time, when images of multiple crops in which only the stems and leaves of the crops are extracted are input, the growth state prediction device (10) can determine the growth pattern of the crop and predict the growth state.

[0151] For example, in the first image of a crop from which the growth part has been extracted, the crop stem is the first length and the leaf area is the first area, and in the second image of the crop from which the growth part has been extracted, the crop stem is the second length, which is 2 cm longer than the first length, and the leaf area is 2 cm from the first area. 2 Let's assume that is the second area that has increased.

[0152] Accordingly, the growth state prediction device (10) for the stem and leaves of the crop has a third length that is 2 cm longer than the second length, and a leaf area that is 2 cm longer than the second area. 2 The growth status can be predicted based on the increased third area.

[0153] In other words, the growth state is a prediction of the degree or range of growth of a specific crop's growth part from the final point in time in a time series from images of multiple crops from which the growth part of a specific crop is extracted, based on the growth pattern learned by the growth state prediction device (10).

[0154] In addition, the growth state prediction device (10) can predict the degree or range of growth of a specific crop and then display the predicted degree or range of growth of the crop on the last arranged crop image among the multiple crop images from which the specific crop's growth part was extracted, using a dotted line or a solid line with different colors, and at this time, a numerical value regarding the degree or range of growth can also be displayed. Accordingly, information regarding the degree or range of growth of a specific crop can be provided to the user.

[0155] In this way, the growth state prediction device (10) learns to determine the growth pattern of a crop based on images of a plurality of crops from which the growth parts of a specific crop have been extracted, and then, when images of a plurality of crops from which the growth parts of the crop have been extracted are input in a time series, it can predict the growth state of the crop from the final input crop image.

[0156] The preferred embodiments of the present invention described above are disclosed for illustrative purposes only, and those skilled in the art with ordinary knowledge of the present invention will be able to make various modifications, changes, and additions within the spirit and scope of the present invention, and such modifications, changes, and additions should be considered to fall within the scope of the claims of the present invention. Explanation of the symbols

[0157] 10: Growth status prediction device 100 : Receiver 200 : Growth part extraction section 300 : Growth status prediction section

Claims

Claim 1 A method for predicting a growth state from an image of a crop based on an artificial intelligence model by a growth state prediction device, comprising: receiving images of a plurality of crops; extracting growth parts from the images of the plurality of crops; and determining a growth pattern of the crop based on the images of the plurality of crops from which the growth parts have been extracted. A method comprising a step of predicting the growth state of a crop based on the determination of the growth pattern, wherein the step of determining the growth pattern of the crop includes the artificial intelligence model applying images of a plurality of crops from which the growth parts are extracted as input variables and applying the growth pattern of the crop as an output variable for learning, wherein the artificial intelligence model is implemented as a Long Shot Term Memory (LSTM) model, and the images of a plurality of crops from which the growth parts are extracted are arranged in a time series to perform learning to determine the growth pattern of the crop, wherein the images of a plurality of crops from which the growth parts are extracted are converted into feature vectors, and the LSTM model learns the growth pattern of the crop by connecting the feature vectors in a time series along a time axis to learn the temporal relationship regarding the feature vectors, wherein the step of extracting the growth parts is performed using a U-Net model, and the learning rate value of the U-Net model is adjusted using one of Step Decay, Exponential Decay, and Cosine Annealing. Claim 2 delete Claim 3 delete Claim 4 A method according to claim 1, comprising arranging images of a plurality of crops from which the growth parts are extracted in a time series and then labeling the growth parts. Claim 5 In paragraph 4, the method comprises at least one of germination, early growth, and maturation. Claim 6 delete Claim 7 A method according to claim 1, wherein the step of predicting the growth state of the crop comprises predicting the growth range of the growth part of the crop after a final point in time in a time-series manner from images of a plurality of crops from which the growth part is extracted, based on the growth pattern. Claim 8 A method according to claim 7, comprising predicting the growth range of the growth part of the crop, and then displaying the predicted growth range of the growth part of the crop on the image of the crop that is last arranged among the images of the crops from which the growth part was extracted. Claim 9 delete Claim 10 A device for predicting the growth state from an image of a crop based on an artificial intelligence model, comprising: a receiving unit for receiving images of a plurality of crops; and a growth part extraction unit for extracting growth parts from the images of the plurality of crops. The apparatus includes a growth state prediction unit that determines the growth pattern of the crop based on images of a plurality of crops from which the growth parts have been extracted, and predicts the growth state of the crop based on the determination of the growth pattern, wherein determining the growth pattern of the crop includes the artificial intelligence model applying the images of the plurality of crops from which the growth parts have been extracted as input variables and applying the growth pattern of the crop as an output variable to learn, wherein the artificial intelligence model is implemented as a Long Shot Term Memory (LSTM) model, the images of the plurality of crops from which the growth parts have been extracted are converted into feature vectors, and the LSTM model learns the growth pattern of the crop by connecting the feature vectors chronologically along a time axis to learn the temporal relationship regarding the feature vectors, wherein the growth part extraction unit includes a U-Net model, and the learning rate value of the U-Net model is adjusted using one of Step Decay, Exponential Decay, and Cosine Annealing.

Images