Server providing management information about crop using artificial inteligence

Abstract

A server providing management information for crops using AI (Artificial intelligence) is disclosed. A server providing management information for crops according to one embodiment of the present invention may include a storage unit and a processor that acquires an image of a crop to be managed and at least one hand of a user, acquires growth status information of the crop, acquires activity information of the hand for managing the crop, and performs learning by matching the growth status information of at least a part of the crop with the activity information of the hand.

Description

Technology Field

[0001] The present invention relates to a server that provides management information for crops using AI, and more specifically, to a server that provides management information for crops related to the know-how of a worker. Background Technology

[0002] AI technology is being utilized in various ways to meet the demand for improved agricultural productivity driven by population growth and climate change. Data is collected through sensors, drones, and satellites, and big data analysis and machine learning can predict crop growth and the occurrence of pests and diseases. Autonomous agricultural machinery and harvesting robots automate tasks, while climate data analysis and market forecasting support efficient decision-making. Furthermore, AI is contributing to the development of disease-resistant varieties through genetic analysis and crop simulation. These technologies are essential for realizing resource conservation and sustainable agriculture.

[0003] As an example of the technology forming the background of the present invention, U.S. Patent Publication 2023-0017425 A1 (January 19, 2023) discloses a method utilizing a deep learning-based image analysis approach to estimate the effect of herbicides on crops and analyze the condition of plants in order to optimize crop yields and minimize crop damage. The problem to be solved

[0004] Meanwhile, farmers tend to manage crops based on experience and know-how. Nevertheless, this experience and know-how are difficult to document, making it hard to pass them on to novice farmers.

[0005] The present invention has been devised to solve the aforementioned problems, and aims to provide a method for providing information on the activities of a farmer's hands to other farmers that must be performed at a specific growth state of a crop by matching and learning information on the growth state of a crop with information on the activities of a farmer's hands. means of solving the problem

[0006] A server providing management information for a crop using artificial intelligence (AI) according to various embodiments of the present invention may include a storage unit and a processor that acquires an image of a crop to be managed and at least one hand of a user, acquires information on the growth status of the crop, acquires information on the activity of the hand for managing the crop, and performs learning by matching the growth status information for at least a part of the crop with the activity information of the hand. Effects of the invention

[0007] According to various embodiments of the present invention, by matching and learning information on the growth status of a crop with information on the activity of a farmer's hand, the present invention can efficiently provide information on the activity of a farmer's hand that must be performed at a specific growth status of a crop to other farmers. Brief explanation of the drawing

[0008] FIG. 1 is a drawing of a system for providing management information for crops according to one embodiment of the present invention. FIG. 2 is a block diagram of a crop management server according to one embodiment of the present invention. FIG. 3 is a detailed block diagram of a crop management server according to one embodiment of the present invention. FIG. 4 is a flowchart of a method of operation of a crop management server according to one embodiment of the present invention. Specific details for implementing the invention

[0009] The operating principles of preferred embodiments of the present invention will be described in detail below with reference to the attached drawings. Furthermore, in describing embodiments of the invention, detailed descriptions of related known functions or configurations will be omitted if it is determined that such detailed descriptions could obscure the essence of the present disclosure. Additionally, the terms used below are defined considering their functions in the present invention, and these may vary depending on the intentions or conventions of the user or operator. Therefore, the definitions of the terms used should be interpreted based on the content throughout this specification and the corresponding functions.

[0010] AI smart farm technology, which manages crop growth conditions, is used to increase agricultural productivity, utilize resources efficiently, and cultivate crops in an environmentally friendly manner. This technology combines advanced technologies such as sensors, data analysis, artificial intelligence (AI), and the Internet of Things (IoT) to optimize the crop growth environment and enable real-time management.

[0011] Key elements and technologies include a sensor network system. Environmental sensors measure growth environment data such as temperature, humidity, CO₂ concentration, and light intensity, while soil sensors detect soil moisture, temperature, pH, and nutrient status to monitor the crop root environment. Additionally, image sensors utilize cameras and drones to visually analyze the condition of crop leaves, stems, and fruits.

[0012] Furthermore, AI smart farm technology for managing crop growth conditions transmits data collected from Internet of Things (IoT) sensors to a cloud server via a wireless network, and the IoT-based control system can remotely control greenhouses, irrigation systems, lighting, and more. Here, AI technology analyzes crop growth data to provide an optimal cultivation environment and performs data analysis and prediction, such as forecasting the likelihood of pest and disease outbreaks or harvest times. Additionally, computer vision technology can be integrated to analyze crop images to identify issues such as pests, nutrient deficiencies, and excessive moisture. Moreover, machine learning models can learn weather changes and growth patterns based on historical data to recommend optimal farming operations.

[0013] In the following, a method for providing management information for crops using artificial intelligence is described in detail according to various embodiments of the present invention.

[0014] FIG. 1 is a drawing of a system for providing management information for crops according to one embodiment of the present invention.

[0015] A system (10) that provides management information for crops using artificial intelligence (AI) according to one embodiment of the present invention may include a user terminal (100) and a crop management server (200).

[0016] A user terminal (100) can be mounted on the body of a user (or farmer, worker) to obtain images of a crop being managed and at least one hand of the user. Here, the user terminal (100) may be a mobile device such as a smartphone, or a wearable device that can be attached to the user's body.

[0017] Here, the user terminal (100) can obtain information on the growth status of a crop and information on the activity of a hand for managing a crop based on the acquired image. For example, the information on the growth status of a crop may include at least one of information on the disease status of a crop and information on the growth stage of a crop.

[0018] Additionally, the user terminal (100) can perform learning by matching growth status information for at least a part of the crop with activity information of the hand. For example, when at least a part of the crop undergoes a physical change (e.g., a physical change in which at least a part of the crop detaches from a branch or stem) according to a first movement of the hand, the user terminal (100) can learn by matching first activity information, which includes information about at least a part of the crop and information about the physical change, with growth status information for at least a part of the crop. As a more specific example, the user terminal (100) can determine that the fruit of the crop has been harvested when the fruit is located on a movement path where the hand moves away from the worker's body according to the first movement of the hand, and the hand moves along the movement path and then moves away from the movement path and the fruit moves away from its position.

[0019] For example, the user terminal (100) can determine whether to harvest based on the maturity of the fruit to be harvested. For example, if the user terminal (100) is set to harvest grapes with a maturity of 5 or higher, it can notify that only grapes with a maturity of 5 or higher should be harvested.

[0020] Here, the user terminal (100) can determine the priority of harvesting for multiple fruits based on the distance between the user's hand and the center of the fruit and the camera exposure of the fruit. For example, the user terminal (100) can set a fruit with a short distance between the user's hand and the center of the fruit and a high camera exposure as a high harvest priority. The corresponding ranking can be displayed in each area of ​​the multiple fruits displayed on the user terminal (100). The camera exposure of the crop described above can be calculated as (area of ​​the area where the fruit is exposed to the camera) / (area of ​​the area where the fruit is exposed to the camera + area of ​​the area where the fruit is obscured by an object (= area where the entire fruit is exposed to the camera)) x 100 (%).

[0021] In one embodiment of the present invention described above, information regarding physical change may be one of an event in which at least one part of the crop is separated, an event in which a support member is attached to the crop, or an event in which a pesticide or fertilizer is sprayed on the crop.

[0022] For example, a user terminal (100) can learn by matching user activity information regarding a crop, crop disease stage information, and crop growth stage information. For example, it can learn by matching first activity information regarding an event in which at least a part of the crop, for example, a branch of the crop, is separated from the stem by the user's hand, first disease stage information, for example, leaf blight manifestation stage information, and a first growth stage among a plurality of growth stages of the crop.

[0023] When such learning is advanced and a crop management model is generated, the user terminal (100) acquires an image of a crop and, if it is determined that the crop included in the image is in the stage of the first disease manifestation and the first growth stage of the crop, it can provide a notification to perform first activity information corresponding to an event in which at least one part of the crop is separated by the user's hand.

[0024] The crop management server (200) can generally control the system (10) that provides management information for crops. The crop management server (200) can perform learning by matching growth status information and hand activity information for at least a part of the crop described above based on image information received from the user terminal (100), or can create a crop management model by receiving learning results received from the user terminal (100) and performing updates. When an image of a specific crop is received from the user terminal (100), the crop management server (200) can acquire hand activity information corresponding to the current crop growth status information based on the crop management model created through the learning and provide it to the user terminal (100).

[0025] According to various embodiments of the present invention described above, the present invention can efficiently provide information on the activities of a farmer's hands to other farmers that need to be performed at a specific growth state of a crop by matching and learning information on the growth state of a crop with information on the activities of a farmer's hands.

[0026] Generally, the devices, systems, and methods disclosed herein may include a number of different devices and computer systems, such as, for example, general-purpose computing systems, server-client computing systems, consumer-merchant computing systems, mainframe computing systems, cloud computing infrastructure, telephone computing systems, laptop computers, desktop computers, smartphones, cellular phones, personal digital assistants (PDAs), tablet computers, and other mobile devices, and may be implemented within the number of different devices and computer systems. The devices and computer systems may include one or more databases and other storage devices, servers, and additional components, for example, processors, modems, terminals and displays, computer-readable media, algorithms, modules and applications, and other computer-related components. The devices and computer systems and / or computing infrastructure are configured, programmed, and adapted to perform the functions and processes of the systems and methods disclosed herein.

[0027] According to one embodiment of the present invention, the location of a fruit within an image can be determined through deep learning using an image input from a camera for monitoring the cultivation environment of a crop or fruit. The process of determining the presence or absence of a fruit area through weakly supervised learning can be trained through data collection and learning, and can be performed in the order of data collection, labeling, model selection, CAM generation, fruit detection using the CAM, and model training and evaluation.

[0028] For example, the crop management server (200) may first perform data collection. Images related to the fruit may be collected for training. In this case, it is desirable for the crop management server (200) to collect images from various angles and lighting conditions.

[0029] Next, the crop management server (200) can perform labeling. The crop management server (200) labels the collected images to assign a label indicating whether the fruit is present. For example, if the fruit is present in a specific image, it can be labeled "1," and if it is not present, it can be labeled "0."

[0030] Additionally, the crop management server (200) can perform model selection. For example, the crop management server (200) can select a model for weakly supervised learning. A commonly used deep learning model is a Convolutional Neural Network (CNN), and a Class Activation Map (CAM) can be generated based on this.

[0031] Next, the crop management server (200) can perform CAM generation. The crop management server (200) generates the CAM using a selected deep learning model. The CAM is one of the techniques for visualizing network activation and can highlight important areas related to a specific object or class (fruit).

[0032] Additionally, the crop management server (200) can perform fruit detection using CAM. It detects areas containing fruit in an input image using CAM. The crop management server (200) can find areas associated with the corresponding class (fruit) in the input image based on the activation map of the CAM.

[0033] Finally, the crop management server (200) can perform model training and evaluation. The crop management server (200) trains and evaluates a selected model using collected data. Using the trained model, it can determine whether there is a fruit in the input image.

[0034] One embodiment of the present invention described above may be explained as utilizing deep learning or an artificial neural network, such as a Convolutional Neural Network (CNN). However, the present invention is not limited thereto, and in addition to the Convolutional Neural Network (CNN), network models such as Deep Neural Network (DNN), Recurrent Neural Network (RNN), Bidirectional Recurrent Deep Neural Network (BRDNN), and Multilayer Perceptron (MLP) may also be utilized.

[0035] Furthermore, the aforementioned artificial neural network models the operating principles of biological neurons and the connection relationships between them; it is an information processing system in which multiple neurons, referred to as nodes or processing elements, are connected in a layered structure. An artificial neural network can refer to a model in which artificial neurons (nodes), forming a network through synaptic connections, change the strength of these connections through learning to possess problem-solving capabilities.

[0036] In addition, artificial neural networks can be used interchangeably with neural networks, and artificial neural networks may include multiple layers, each of which may include multiple neurons. Furthermore, artificial neural networks may include synapses connecting neurons. Here, an artificial neural network can generally be defined by the following three factors: (1) connection patterns between neurons of different layers, (2) a learning process for updating the weights of the connections, and (3) an activation function that generates an output value from a weighted sum of inputs received from the previous layer.

[0037] FIG. 2 is a block diagram of a crop management server according to one embodiment of the present invention.

[0038] Referring to FIG. 2, the crop management server (200') may include a storage unit (210) and a processor (220).

[0039] The storage unit (210) may include a volatile or non-volatile recording medium.

[0040] The storage unit (210) is connected to one or more processors and can store code that causes the processor (220) to control detailed components when executed by the processor (220). Additionally, the storage unit (210) can store a payment information library, a hardware control information library, an internal display unit control information library, an external display touch control library, etc.

[0041] Here, the storage unit (210) may include a magnetic storage medium or a flash storage medium, but the scope of the present invention is not limited thereto. The storage unit (210) may include internal memory and / or external memory, and may include volatile memory such as DRAM, SRAM, or SDRAM, non-volatile memory such as OTPROM (one time programmable ROM), PROM, EPROM, EEPROM, mask ROM, flash ROM, NAND flash memory, or NOR flash memory, flash drives such as SSD, CF (compact flash) card, SD card, Micro-SD card, Mini-SD card, or memory stick, or storage devices such as HDD.

[0042] The processor (220) can control the overall operation of the crop management server (200').

[0043] For example, the processor (220) can receive from the user terminal (100) an image of a crop being managed and at least one hand of the user, which is mounted on the body of the user (or farmer, worker). Here, the user terminal (100) may be a mobile device such as a smartphone, or a wearable device that can be attached to the user's body.

[0044] Here, the processor (220) can obtain information on the growth status of the crop and information on the activity of the hand for managing the crop based on the received image. For example, the information on the growth status of the crop may include at least one of information on the disease status of the crop and information on the growth stage of the crop.

[0045] Additionally, the processor (220) can perform learning by matching growth status information for at least a part of the crop with activity information of the hand. For example, when at least a part of the crop undergoes a physical change (e.g., a physical change in which at least a part of the crop detaches from a branch or stem) according to the first movement of the hand, the processor (220) can learn by matching first activity information, which includes information about at least a part of the crop and information about the physical change, with growth status information for at least a part of the crop. As a more specific example, the processor (220) can determine that the fruit of the crop has been harvested when the fruit is located on a movement path where the hand moves away from the worker's body according to the first movement of the hand, and after the hand moves along the movement path, it deviates from the movement path and the fruit moves away from its position.

[0046] In one embodiment of the present invention described above, information regarding physical change may be one of an event in which at least one part of the crop is separated, an event in which a support member is attached to the crop, or an event in which a pesticide or fertilizer is sprayed on the crop.

[0047] For example, the processor (220) can learn by matching user activity information regarding a crop, crop disease stage information, and crop growth stage information. For example, it can learn by matching first activity information regarding an event in which at least a part of the crop, for example, a branch of the crop, is separated from the stem by the user's hand, first disease stage information, for example, leaf blight manifestation stage information, and a first growth stage among a plurality of growth stages of the crop.

[0048] When such learning is advanced and a crop management model is generated, the processor (220) acquires an image of a crop and, if it is determined that the crop included in the image is in the first disease manifestation period and the first growth stage of the crop, it can provide a notification to perform first activity information corresponding to an event in which at least a part of the crop is separated by the user's hand.

[0049] For example, the developmental stages of a fruit according to one embodiment of the present invention are classified into three stages, such as an immature stage (e.g., a first stage), a discoloration stage (e.g., a second stage), and a mature stage (e.g., a third stage). Among these, the immature stage can be further subdivided and specifically classified into two or more stages, for example, four stages, namely a first immature stage, a second immature stage, a third immature stage, and a fourth immature stage. Additionally, the mature stage can also be more specifically classified, for example, into a first mature stage, a second mature stage, and a third mature stage. These fruit developmental stages are classified according to physiological changes in the fruit and days before harvest (DBH).

[0050] For example, the aforementioned first immaturity stage can be classified based on approximately +35 DBH, where fruit expansion begins; the second immaturity stage can be classified based on 25–35 DBH, where fruit expansion is almost complete; the third immaturity stage can be classified based on 15–25 DBH, where the fruit reaches full size; and the fourth immaturity stage can be classified based on 15 DBH, which is the final stage of immaturity, the discoloration stage.

[0051] For example, the processor (220) may perform learning by matching growth status information and hand activity information for at least a part of the crop described above based on image information received from the user terminal (100), or may create a crop management model by receiving the learning results received from the user terminal (100) and performing an update. When an image of a specific crop is received from the user terminal (100), the processor (220) may obtain hand activity information corresponding to the current crop growth status information based on the crop management model created through the learning and provide it to the user terminal (100).

[0052] According to various embodiments of the present invention described above, the present invention can efficiently provide information on the activities of a farmer's hands to other farmers that need to be performed at a specific growth state of a crop by matching and learning information on the growth state of a crop with information on the activities of a farmer's hands.

[0053] The processes for performing the method of providing management information for crops according to various embodiments of the present invention described above should be interpreted as being performed at a user terminal (100) or at a crop management server (200, 200').

[0054] FIG. 3 is a detailed block diagram of a crop management server according to one embodiment of the present invention.

[0055] Referring to FIG. 3, the crop management server (300) includes a communication unit (310), a storage unit (320), and a processor (330).

[0056] The communication unit (310) performs communication. The communication unit (310) can communicate with external electronic devices through various communication methods such as BT (Bluetooth), WI-FI (Wireless Fidelity), ZigBee, IR (Infrared), NFC (Near Field Communication), etc.

[0057] The storage unit (320) can store an O / S (Operating System) software module for operating the crop management server (300), data for configuring various UI screens provided in the display area, etc.

[0058] In addition, the storage unit (320) is readable and writable.

[0059] The processor (330) controls the overall operation of the crop management server (300) using various programs stored in the storage unit (320).

[0060] Specifically, the processor (330) includes RAM (331), ROM (332), main CPU (333), graphics processing unit (334), first to n interfaces (335-1 to 335-n) and a bus (336).

[0061] Here, RAM (331), ROM (332), main CPU (333), graphics processing unit (334), first to n interfaces (335-1 to 335-n), etc. can be connected to each other via a bus (336).

[0062] The first to n interfaces (335-1 to 335-n) are connected to the various components described above. One of the interfaces may be a network interface connected to an external device through a network.

[0063] The ROM (332) stores a set of instructions for booting the system, etc. When a turn-on command is input and power is supplied, the main CPU (333) copies the O / S stored in the storage unit (320) to the RAM (331) according to the instructions stored in the ROM (332), and runs the O / S to boot the system.

[0064] When booting is complete, the main CPU (333) copies various stored application programs to RAM (331) and executes the application programs copied to RAM (331) to perform various operations.

[0065] The main CPU (333) accesses the storage unit (320) and performs booting using the O / S stored in the storage unit (320). Then, the main CPU (333) performs various operations using various programs, content, data, etc. stored in the storage unit (320).

[0066] The graphics processing unit (334) uses the operation unit and the rendering unit to generate a screen containing various objects such as icons, images, and text.

[0067] FIG. 4 is a flowchart of a method of operation of a crop management server according to one embodiment of the present invention.

[0068] Referring to FIG. 4, the operation method of a server (hereinafter referred to as a crop management server) that provides management information for crops using AI (Artificial intelligence) may include a process of acquiring an image of a crop to be managed and at least one hand of a user (S410), a process of acquiring information on the growth status of the crop (S420), a process of acquiring information on the activity of the hand for crop management (S430), and a process of performing learning by matching the information on the growth status of at least a part of the crop with the information on the activity of the hand (S440).

[0069] For example, crop growth status information may include at least one of crop disease status information and crop growth stage information.

[0070] For example, the method of operation of the crop management server described above may further include a process of learning by matching growth status information of at least a part of the crop with first activity information, which includes information about at least a part of the crop and information about physical changes, when at least a part of the crop is physically changed according to a first movement of a hand.

[0071] In addition, as an example, the method of operation of the crop management server described above may further include a process of determining that the fruit has been harvested when the fruit is located on a path where the hand moves away from the worker's body according to the first movement of the hand, and the hand moves along the path and then moves away from the path, and the fruit moves away from the position.

[0072] According to one embodiment of the present invention, the information regarding the physical change described above may be one of an event in which at least one part of the crop is separated, an event in which a support member is attached to the crop, or an event in which a pesticide or fertilizer is sprayed on the crop.

[0073] In addition, as an example, the method of operation of the crop management server described above may further include a process of learning by matching first activity information regarding an event in which at least one part of the crop is separated, a first disease manifestation period, and a first growth stage of the crop. Here, by learning activity information by matching the disease manifestation period with the growth stage, the conditions for disease manifestation according to the growth stage can be reflected more precisely during the learning process.

[0074] In addition, as an example, the method of operation of the crop management server described above may further include a process of acquiring an image of a crop and, if the crop included in the image is in the period of manifestation of a first disease and the crop is in the first growth stage, a process of performing a notification to perform first activity information regarding an event in which at least a part of the crop is separated.

[0075] Meanwhile, the method of operation of a crop management server according to various embodiments of the present invention described above may be provided to each server or device to be executed by a processor while being implemented as computer-executable program code and stored on various non-transitory computer-readable media.

[0076] For example, a non-transitory computer-readable medium may be provided that stores a program for acquiring images of a crop to be managed and at least one hand of a user, acquiring information on the growth status of the crop, acquiring information on the activity of the hand for managing the crop, and performing learning by matching the growth status information and the activity information of the hand for at least a part of the crop.

[0077] A non-transient readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, the various applications or programs described above may be stored and provided on non-transient readable media such as CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, and ROMs.

[0078] Although embodiments of the present invention have been illustrated and described above, those skilled in the art will understand that various modifications in form and details may be made without departing from the spirit and scope of the embodiments as defined by the appended claims and equivalents. Explanation of the symbols

[0079] System providing management information on crops: 10 User terminal: 100 Crop Management Server: 200, 200', 300 Storage: 210, 320 Processors: 220, 330 Communications Department: 310

Claims

Claim 1 A crop management server comprising: a storage unit; and a processor that acquires an image of a crop to be managed and at least one hand of a user, acquires growth status information of the crop, acquires first activity information including information on at least a part of the crop and information on the physical change when at least a part of the crop is physically changed according to a first movement of the hand for managing the crop, performs learning by matching the growth status information of at least a part of the crop, disease stage information of the crop, and the first activity information, acquires a camera exposure degree of the fruit, which is the ratio of the area of ​​the fruit exposed to the camera to the area when the entire fruit is exposed to the camera, based on the image of the crop, and determines the priority of harvesting for a plurality of fruits based on the distance between the user's hand and the center of the fruit and the camera exposure degree of the fruit. Claim 2 A crop management server according to claim 1, wherein the growth status information of the crop includes at least one of the disease status information of the crop and the growth stage information of the crop. Claim 3 delete Claim 4 A crop management server according to claim 1, wherein the processor determines that the fruit is harvested when the fruit is located on a path where the hand moves away from the worker's body in accordance with the first movement of the hand, and the hand moves away from the path and then moves away from the path, and the fruit moves away from the location. Claim 5 A crop management server, wherein the information regarding the physical change is one of an event in which at least a part of the crop is separated, an event in which a support member is attached to the crop, or an event in which a pesticide or fertilizer is sprayed on the crop. Claim 6 In paragraph 5, the processor is a crop management server that learns by matching first activity information regarding an event in which at least one part of the crop is separated, a first disease manifestation period, and a first growth stage of the crop. Claim 7 A crop management server according to claim 6, wherein the processor acquires an image of the crop and, when the crop included in the image is in the period of manifestation of a first disease and the crop is in a first growth stage, performs a notification to perform first activity information regarding an event in which at least a part of the crop is separated.

Images