Artificial intelligence-based system for diagnosing and predicting malfunction of smart farm ict equipment

Abstract

This malfunction diagnosis system comprises: a prediction module for predicting smart farm data at the current time point from smart farm data of a past time point; a statistical analysis module for setting a threshold value for diagnosis of a malfunction by statistically analyzing, in a normal situation and a malfunctioning situation, the differences between the smart farm data at the current time point, predicted by the prediction module, and actually measured values; and a malfunction diagnosis module which calculates the difference between the smart farm data at the current time point, predicted by the prediction module, and actual smart farm data at the current time point, and which compares the calculated difference to the threshold value set by the statistical analysis module, thereby diagnosing whether there is a malfunction.

Description

AI-based malfunction diagnosis and prediction system for smart farm ICT equipment

[0001] The technology described below is a system that can diagnose smart farm malfunctions.

[0002] This application is the result of research conducted with the support of the Information and Communications Technology Planning and Evaluation Institute with funding from the government (Ministry of Science and ICT) as follows.

[0003] 1. Project ID: 1711193344, Research Project Name: Regional Intelligence Innovation Talent Development (Grand ICT Research Center), Research Project Name: Grand ICT Research Center (Suncheon National University).

[0004] 2. Project ID: 1711198848, Research Project Name: Training of Information, Communication, and Broadcasting Innovation Talents, Research Project Title: Implementation of Smart Distribution for Low-Carbon Agricultural Technology and Wellness Agricultural and Food Value Enhancement (Suncheon National University).

[0005] Recently, major countries, including Korea, are developing intelligent smart farms through convergence with ICT. Specifically, they are focusing on developing differentiated smart farm technologies centered on "software and hardware platforms," ​​"data intelligence," and "integration with various technologies such as AI, cloud computing, and the Internet of Things." Smart farms, focusing on facility horticulture, provide services tailored to the needs of farmers, such as crop growth, environmental information management, control, disease prevention, and growth algorithms. However, there still remains a significant gap between these technologies and practical, field-based solutions.

[0006] [Prior Art Literature]

[0007] [Patent Document]

[0008] Korean Patent Publication No. 10-2020-0017054

[0009] Traditional smart farm technology has been developed with a focus on labor savings and convenience. Traditional smart farm technology has focused on remote monitoring and control of agricultural sites. Therefore, it has been difficult to verify proper system operation in the field. To address this issue, numerous devices (CCTVs, sensors, etc.) have been installed in smart farms. However, identifying malfunctions based on measured values ​​has been difficult and costly. When applying smart farm technology, device malfunctions can result in irreparable damage. Therefore, rapid identification of malfunctions is crucial.

[0010] The technology described below is intended to disclose a system capable of diagnosing malfunctions in smart farms.

[0011] The malfunction diagnosis system includes a prediction module that predicts smart farm data at a current point in time from smart farm data at a past point in time; a statistical analysis module that statistically analyzes the difference between the smart farm data at a current point in time predicted by the prediction module and an actually measured value in a normal situation and a situation where a malfunction occurs to set a threshold for malfunction diagnosis; and a malfunction diagnosis module that calculates the difference between the smart farm data at a current point in time predicted by the prediction module and the actual smart farm data at a current point in time, and compares the calculated difference with the threshold set by the statistical analysis module to diagnose whether a malfunction exists.

[0012] The technology described below can automatically diagnose malfunctions in smart farms without requiring direct human intervention. This eliminates labor and cost issues. It also prevents irreparable damage caused by equipment malfunctions in smart farms.

[0013] The technology described below allows for the detection of various malfunction scenarios by establishing various rules for malfunction detection. This allows for flexible response.

[0014] Figure 1 is one embodiment of a malfunction diagnosis system (100).

[0015] Figure 2 is a flowchart (200) of one embodiment in which a malfunction diagnosis system performs a smart farm malfunction diagnosis method.

[0016] Figure 3 is a flowchart (300) of one embodiment in which a malfunction diagnosis system performs a method for diagnosing a smart farm malfunction.

[0017] FIG. 4 is a configuration of another embodiment of a malfunction diagnosis system (400).

[0018] The technology described below is susceptible to various modifications and embodiments. Specific embodiments of the technology described below may be illustrated in the drawings of the specification. However, these are intended to illustrate the technology described below and are not intended to limit the technology described below to any specific embodiments. Therefore, it should be understood that all modifications, equivalents, or alternatives that fall within the spirit and scope of the technology described below are encompassed by the technology described below.

[0019] In the terms used hereinafter, singular expressions should be understood to include plural expressions unless the context clearly dictates otherwise, and terms such as "comprises" should be understood to mean the presence of a described feature, number, step, operation, component, part, or combination thereof, but not to exclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0020] Before going into a detailed description of the drawings, it should be made clear that the division of components in this specification is merely a division based on the main function of each component. That is, two or more components described below may be combined into one component, or one component may be further divided into two or more components with more detailed functions. In addition to its own main function, each component described below may additionally perform some or all of the functions of other components, and of course, some of the main functions of each component may be exclusively performed by other components.

[0021] Additionally, in performing a method or method of operation, each process constituting the method may occur in a different order than the stated order, unless the context clearly indicates a specific order. That is, each process may occur in the same order as the stated order, may be performed substantially simultaneously, or may be performed in the opposite order.

[0022] Figure 1 is one embodiment of a malfunction diagnosis system (100).

[0023] The malfunction diagnosis system (100) can be physically implemented in various forms. For example, the malfunction diagnosis system (100) can take the form of a PC, laptop, smart device, server, or data processing-dedicated chipset.

[0024] The malfunction diagnosis system (100) may include a smart farm database (110), a prediction module (120), a statistical analysis module (130), a malfunction diagnosis module (140), a rule module (150), a smart farm cloud server (160), and an external linkage module (170).

[0025] The smart farm database (110) may include smart farm data.

[0026] Smart farm data can include various types of data that can be measured on smart farms.

[0027] Smart farm data can be time-series data. That is, it can be measured at regular intervals and presented in sequential order. For example, smart farm data might be measured at 30-minute intervals from 8:00 AM to 12:00 PM.

[0028] Smart farm data may include device data. Device data may include data obtained from various devices installed in a smart farm. Devices installed in a smart farm may include environmental control devices, imaging devices, and sensor devices.

[0029] Environmental control devices may be devices that regulate the internal environment of a smart farm. These devices may include ventilators, air conditioners, irrigation systems, window opening / closing devices, and nutrient supply devices. The smart farm database (110) may include data on the operating hours, operating frequency, operating results, and power consumption of each environmental control device.

[0030] The video recording device may be a device that records the interior of a smart farm. In one embodiment, the video recording device may include a CCTV, an infrared camera, or an ultraviolet camera. The smart farm database (110) may include data regarding the video captured by the video recording device and the time at which the video was captured.

[0031] The sensor device may be a sensor that measures environmental information within the smart farm. In one embodiment, the sensor device may include a thermometer, hygrometer, illuminometer, oxygen measurement sensor, and carbon dioxide measurement sensor. The smart farm database (110) may include temperature data, humidity data, illuminance data, oxygen concentration data, and carbon dioxide concentration data.

[0032] Smart farm data included in the smart farm database (110) can be used to train the prediction module (120) and statistical analysis module (130).

[0033] The prediction module (120) may be a module that predicts smart farm data at the current point in time based on smart farm data at a past point in time.

[0034] The prediction module (120) may be a module trained to predict current smart farm data from past smart farm data based on normal data. Accordingly, the prediction module (120) predicts current smart farm data from past smart farm data by assuming only normal situations.

[0035] For example, the prediction module (120) predicts the humidity data measured on the 14th based on the humidity data measured on the 11th, the humidity data measured on the 12th, and the humidity data measured on the 13th. If there is a difference between the current smart farm data predicted by the prediction module (120) and the actual acquired smart farm data, it may be diagnosed that there is a malfunction in the humidity sensor (or humidity control device). This is because the actual measured data is different even though the prediction module (120) is trained to predict the current smart farm data by assuming only normal situations.

[0036] The prediction module (120) can predict smart farm data at the current point in time based on smart farm data at a past point in time using a time series data processing model. The time series data processing model may be a model that processes time series data. The time series data processing model may be a machine learning (ML)-based model. Furthermore, the time series data processing model may be an artificial neural network (ANN)-based model. In one embodiment, the time series data processing model may be at least one of a recurrent neural network (RNN), a long short-term memory (LSTM), a gated recurrent unit (GRU), and a transformer.

[0037] After learning is completed, the prediction module (120) can receive smart farm data from the smart farm cloud server (160). The prediction module (120) can predict smart farm data at the current point in time based on the smart farm data received from the smart farm cloud server (160).

[0038] The statistical analysis module (130) may be a module that statistically calculates the difference between the current smart farm data predicted by the prediction model in normal situations and in situations where a malfunction has occurred and the actual measured value, and sets a threshold for diagnosing a malfunction. In other words, the statistical analysis module (130) may be a module that calculates the difference between the current smart farm data predicted by the prediction module (120) in normal situations and in situations where a malfunction has occurred and the actual measured value, and then statistically analyzes the calculated difference to set a threshold for diagnosing a malfunction.

[0039] The malfunction diagnosis module (140) may be a module that diagnoses whether a malfunction has occurred in a smart farm device installed in a smart farm.

[0040] The malfunction diagnosis module (140) can receive smart farm data from the smart farm cloud server (160). The malfunction diagnosis module (140) can receive smart farm data at the current point in time predicted by the prediction module (120). The malfunction diagnosis module (140) can receive a threshold value for malfunction diagnosis calculated by the statistical analysis module. The malfunction diagnosis module (140) can receive rules for malfunction diagnosis from the rule module (150). Furthermore, the malfunction diagnosis module (140) can also receive data from the Korea Meteorological Administration from the smart farm cloud server (160).

[0041] The malfunction diagnosis module (140) can calculate the difference between the smart farm data of the current point in time transmitted by the prediction module (120) and the actual measurement value transmitted by the smart farm cloud server (160). The malfunction diagnosis module (140) can compare the calculated difference value with the threshold value for malfunction diagnosis transmitted by the statistical analysis module (130). The malfunction diagnosis module (140) can diagnose whether a malfunction has occurred based on the comparison result. This is because, as described above, the prediction module (120) can diagnose that a malfunction has occurred if the actual measurement value is different from the predicted result even though it predicts the smart farm data of the current point in time from the past point in time by assuming only normal situations.

[0042] The malfunction diagnosis module (140) can also analyze whether there is a malfunction using the rules received from the rule module (150).

[0043] The rule module (150) can set rules in advance to prevent malfunctions and transmit the set rules to the malfunction diagnosis module (140).

[0044] Rules for diagnosing malfunctions may be rules that define situations in which malfunctions occur. Rules for diagnosing malfunctions may include actuator malfunction diagnosis rules, sensor malfunction diagnosis rules, and user-defined diagnosis rules. Actuator malfunction diagnosis rules may include rules that diagnose an actuator malfunction when a certain result is present. Sensor malfunction diagnosis rules may include rules that diagnose a sensor malfunction when a certain result is present. User-defined diagnosis rules may include rules that the user has set to indicate a malfunction when a certain result is present.

[0045] In one example, a rule for diagnosing a malfunction could include a rule stating that if the values ​​obtained from other equipment of the same type installed in the same area of ​​a smart farm differ from each other, a malfunction should be considered. For example, if two thermometers in area A of a smart farm measure -10°C and -20°C, respectively, the difference is so large that a malfunction should be considered.

[0046] In one example, a rule for diagnosing a malfunction could include a rule stating that if the values ​​obtained from different types of equipment installed in the same area of ​​a smart farm are inconsistent, a malfunction should be considered. For example, a rule could include that if a thermometer in area A of a smart farm measures the current temperature below 5°C and a hygrometer measures the humidity below 10%, a malfunction should be considered. This is because lower temperatures tend to lead to higher humidity, and conversely, higher temperatures tend to lead to lower humidity.

[0047] In one example, rules for diagnosing malfunctions may include a rule stating that a malfunction is detected when different types of equipment are within a certain distance. For example, a rule stating that a malfunction is detected when a heating or cooling device and a thermometer are very close together inside a smart farm. This is because the thermometer may have difficulty measuring the temperature inside the smart farm if it is too close to the heating or cooling device. In this case, images captured by a video camera can be utilized.

[0048] In one embodiment, a rule for diagnosing a malfunction may include a rule stating that if the power consumption of a device is 0, it should be considered a malfunction. For example, a rule may include stating that if the power consumption of a heating or cooling device within a smart farm is 0, it should be considered a malfunction.

[0049] In one example, rules for diagnosing malfunctions may include a rule stating that if the values ​​measured by a sensor or similar remain unchanged for an extended period of time, a malfunction is considered present. For example, a rule stating that if a thermometer consistently measures the temperature inside a smart farm at 23°C, a malfunction is considered present.

[0050] For example, a rule for diagnosing malfunctions might include a rule that, if heavy rainfall occurs around a smart farm, resulting in high humidity inside the farm, a malfunction has occurred in the humidity control device installed in the smart farm. In such cases, data from the Korea Meteorological Administration can be utilized.

[0051] The smart farm cloud server (160) may be a server that stores smart farm data measured by the smart farm and data from the Korea Meteorological Administration. The smart farm cloud server (160) may transmit the stored smart farm data and data from the Korea Meteorological Administration to the prediction module (120). The smart farm cloud server (160) may transmit the stored smart farm data and data from the Korea Meteorological Administration to the malfunction diagnosis module (140).

[0052] The external linkage module (170) can transmit the results of the malfunction diagnosis module (140) to the Wabu system. The external system may be the malfunction diagnosis system (100). In one embodiment, the external system may be a personal terminal of an administrator managing a smart farm. For example, the malfunction diagnosis system (100) may notify the administrator's personal terminal that a malfunction has occurred in the current smart farm internal device.

[0053] The Korea Meteorological Administration data may include information about the environment surrounding the smart farm, provided by the Korea Meteorological Administration. For example, the Korea Meteorological Administration data may include at least one of temperature data, precipitation data, wind speed data, wind direction data, atmospheric pressure data, humidity data, and sunlight amount data. The Korea Meteorological Administration data may be transmitted to a malfunction diagnosis module (140). The malfunction diagnosis module (140) can diagnose whether a malfunction exists based on the transmitted Korea Meteorological Administration data.

[0054] Figure 2 is a flowchart (200) of one embodiment in which a malfunction diagnosis system (100) performs a smart farm malfunction diagnosis method.

[0055] The smart farm cloud server (160) can transmit smart farm data to the prediction module (120) and malfunction diagnosis module (140) (210). The smart farm data may be time series data measured from the smart farm.

[0056] The prediction module (120) can predict the current smart farm data from the past smart farm data based on the received smart farm data (220). The prediction module (120) may be a module that has been trained in advance using the smart farm data stored in the smart farm database (110). The prediction result predicted by the prediction module (120) may be a result of predicting the current smart farm data from the past smart farm data, assuming that the smart farm data is in a normal state.

[0057] The malfunction diagnosis module (140) can receive smart farm data at the current point in time predicted from the prediction module (120) (230).

[0058] The malfunction diagnosis module (140) can calculate the difference between the smart farm data at the current point in time predicted by the prediction module (120) and the actual smart farm data at the current point in time (240).

[0059] The malfunction diagnosis module (140) can receive a threshold value from the statistical analysis module (130) (250). The statistical analysis module may be a module that has been trained in advance using smart farm data stored in the smart farm database (110). The threshold value of the statistical analysis module is used to diagnose whether there is a malfunction due to a large difference between the result predicted by the prediction module (120) and the actual measured value.

[0060] The malfunction diagnosis module (140) can diagnose whether a malfunction has occurred by comparing the calculated difference with the received threshold value (260).

[0061] Furthermore, the malfunction diagnosis system (100) can transmit the diagnosis results to an external system through an external linkage module (170).

[0062] Figure 3 is a flowchart (300) of one embodiment in which a malfunction diagnosis system (100) performs a method for diagnosing a smart farm malfunction.

[0063] The smart farm cloud server (160) can transmit smart farm data and weather data to the malfunction diagnosis module (140) (310).

[0064] The rule module (150) can transmit rules for malfunction diagnosis to the malfunction diagnosis module (140) (320). As described above, the rules for malfunction diagnosis can include actuator malfunction diagnosis rules, sensor malfunction diagnosis rules, and user-defined diagnosis rules.

[0065] The malfunction diagnosis module (140) can diagnose whether a malfunction has occurred by comparing the received smart farm data, weather data, and rules for malfunction diagnosis (330).

[0066] Fig. 4 is a configuration of another embodiment of (400).

[0067] The malfunction diagnosis system (400) may correspond to the malfunction diagnosis system (100) described in Fig. 1. That is, the malfunction diagnosis system (400) may be a device that performs the smart farm malfunction diagnosis method described above.

[0068] The malfunction diagnosis system (400) may include an input device (410), a storage device (420), an operation device (430), an output device (440), an interface device (450), and a communication device (460).

[0069] The input device (410) may include an interface device (keyboard, mouse, touch screen, etc.) that receives a certain command or data. The input device (410) may also include a configuration that receives information through a separate storage device (USB, CD, hard disk, etc.). The input device (410) may receive the input data through a separate measuring device or a separate database. The input device (410) may also receive data through wired or wireless communication through a communication device (460). The input device (410) may receive data required to perform the aforementioned smart farm malfunction diagnosis method. The input device (410) may receive a model required to perform the aforementioned smart farm malfunction diagnosis method. The input device (410) may receive smart farm data and Korea Meteorological Administration data. The input device (410) may receive a time series data processing model. The input device (410) may receive rules for malfunction diagnosis.

[0070] The storage device (420) may be a device that stores certain information. The storage device (420) may store information input through the input device (410). The storage device (420) may store information generated during the operation of the computing device (430). That is, the storage device (420) may include a memory. The storage device (420) may store data required to perform the aforementioned smart farm malfunction diagnosis method. The storage device (420) may store a model required to perform the aforementioned smart farm malfunction diagnosis method. The storage device (420) may store smart farm data and Korea Meteorological Administration data. The storage device (420) may store learning data. That is, the storage device (420) may store the smart farm database (110). The storage device (420) may store rules for malfunction diagnosis.

[0071] The computing device (430) may be a device such as a processor, AP, or a chip embedded with a program that processes data and performs certain operations. The computing device (430) may generate a control signal that controls the malfunction diagnosis system (400). The computing device (430) may generate a control signal that controls the input device (410), storage device (420), output device (440), interface device (450), and communication device (460) included in the malfunction diagnosis system (400). The computing device (430) may perform the operations necessary to perform the aforementioned smart farm malfunction diagnosis method.

[0072] The computing device (430) can predict smart farm data at the present time from smart farm data at a past time. If necessary, the computing device (430) can utilize a time series data processing model. The computing device (430) can calculate the difference between the smart farm data at the present time predicted from the smart farm data at a past time and the currently measured smart farm data. The computing device (430) can diagnose whether a malfunction has occurred based on the calculated difference and a pre-calculated threshold value. The computing device (430) can diagnose whether a malfunction has occurred based on smart farm data, Korea Meteorological Administration data, and rules for diagnosing malfunctions.

[0073] The output device (440) may be a device that outputs certain information. The output device (440) may output interfaces required for data processing, input data, analysis results, etc. The output device (440) may be physically implemented in various forms, such as a display, a device that outputs documents, a speaker, etc. The output device (440) may output information stored in the storage device (430). The output device (440) may output information generated during the process of the calculation device (430). The output device (440) may output the result of the calculation performed by the calculation device (430).

[0074] The interface device (450) may be a device that receives certain commands and data from the outside. The interface device (450) may receive a control signal for controlling the malfunction diagnosis system (400). The interface device (450) may output the results analyzed by the malfunction diagnosis system (400). The interface device (450) may receive data necessary for performing the aforementioned smart farm malfunction diagnosis method from a physically connected input device or an external storage device.

[0075] The communication device (460) may refer to a configuration that receives and transmits certain information via a wired or wireless network. The communication device (460) may perform network communication such as Wi-Fi (Wireless Fidelity), Wi-Fi Direct, Bluetooth, UWB (Ultra Wide Band), NFC (Near Field Communication), USB (Universal Serial Bus), HDMI (High Definition Multimedia Interface), LAN (Local Area Network), etc. The communication device (460) may receive a control signal necessary to control the malfunction diagnosis system (400). The communication device (460) may transmit the results analyzed by the malfunction diagnosis system (400). The communication device (460) may receive data necessary to perform the aforementioned smart farm malfunction diagnosis method. The communication device (460) may receive a model necessary to perform the aforementioned smart farm malfunction diagnosis method. The communication device (460) may communicate with a smart farm cloud server and an external system.

[0076] The above-described smart farm malfunction diagnosis method can be implemented as a program (or application) including an executable algorithm that can be executed on a computer.

[0077] The above program may be provided stored on a non-transitory computer readable medium.

[0078] The above-mentioned temporarily readable medium refers to various RAMs such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous DRAM (Synclink DRAM, SLDRAM), and direct Rambus RAM (DRRAM).

[0079] The above non-transitory readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, the various applications or programs described above may be stored and provided in a non-transitory readable medium, such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM (read-only memory), PROM (programmable read only memory), EPROM (Erasable PROM, EPROM), EEPROM (Electrically EPROM), or flash memory.

[0080] The present embodiment and the drawings attached to the present specification only clearly illustrate a part of the technical idea included in the above-described technology, and it will be obvious that all modified examples and specific embodiments that can be easily inferred by a person skilled in the art within the scope of the technical idea included in the specification and drawings of the above-described technology are included in the scope of the rights of the above-described technology.

Claims

1. A prediction module that predicts current smart farm data from past smart farm data; A statistical analysis module that statistically analyzes the difference between the current smart farm data predicted by the above prediction module and the actual measured value in a normal situation or a situation where a malfunction occurs, and sets a threshold for diagnosing a malfunction; A malfunction diagnosis system, comprising: a malfunction diagnosis module that calculates the difference between the current smart farm data predicted by the above prediction module and the actual current smart farm data, and compares the calculated difference with a threshold value set by the above statistical analysis module to diagnose whether there is a malfunction.

2. In paragraph 1, The above prediction module is a malfunction diagnosis system that predicts smart farm data at the present time from smart farm data at the past time using a time series data processing model.

3. In paragraph 2, The above time series data processing model is a malfunction diagnosis system based on a Recurrent Neural Network (RNN).

4. In paragraph 1, The above smart farm data includes device data, A malfunction diagnosis system, wherein the above device data includes data that can be acquired from at least one of an environmental control device, an imaging device, and a sensor device installed in a smart farm.

5. In paragraph 1, The above malfunction diagnosis system further includes a rule module that sets rules for diagnosing malfunctions in advance; The above malfunction diagnosis module is a malfunction diagnosis system that diagnoses whether there is a malfunction by comparing the weather data, the smart farm data, and the rules for diagnosing malfunctions set in advance.

6. In paragraph 1, A malfunction diagnosis system, wherein the malfunction diagnosis system further includes an external linkage module that transmits the results diagnosed by the malfunction diagnosis module to an external system.

7. Step in which the smart farm cloud server transmits smart farm data to the prediction module and malfunction diagnosis module; A step in which the above prediction module predicts smart farm data at the present time from smart farm data at the past time based on the received smart farm data; A step in which the malfunction diagnosis module receives the smart farm data at the current point in time predicted from the prediction module; A step in which the malfunction diagnosis module calculates the difference between the current smart farm data predicted by the prediction module and the actual current smart farm data; The step of the above malfunction diagnosis module receiving a threshold value from the statistical analysis module; and A smart farm malfunction diagnosis method, comprising: a step of the malfunction diagnosis module comparing the calculated difference with the received threshold value to diagnose whether a malfunction has occurred.

8. In paragraph 7, A smart farm malfunction diagnosis method further comprising a step of an external linkage module transmitting the results diagnosed by the malfunction diagnosis module to an external system.

9. Step of the smart farm cloud server transmitting smart farm data and weather station data to the malfunction diagnosis module; A step of the rule module transmitting rules for malfunction diagnosis to the malfunction diagnosis module; and A method for diagnosing a malfunction of a smart farm, comprising: a step of diagnosing whether a malfunction has occurred by comparing the received smart farm data, weather data, and rules for diagnosing malfunctions by the malfunction diagnosis module.

Images