Electronic device for learning and predicting quality factor for each variety of kalanchoe and operating method thereof

Abstract

According to the present disclosure, provided are an electronic device and an operation method thereof, the electronic device: training an artificial intelligence model to determine the variety of kalanchoe on the basis of image data; training the artificial intelligence model to predict and determine a plurality of quality factors including the canopy of the kalanchoe, the number of flowers on the kalanchoe, the flowering rate of the kalanchoe, the percentage of peduncles of the kalanchoe, and the amount of damage to the kalanchoe due to pests; and inputting new image data, which is received from an image sensor, to the trained artificial intelligence model to predict and determine the plurality of quality factors.

Description

Electronic device and operating method for learning and predicting cultivar-specific quality factors of Kalanchoe

[0001] Embodiments of the present disclosure relate to artificial intelligence, and more particularly, to an electronic device and an operating method thereof for learning and predicting quality factors of each variety of Kalanchoe.

[0002] The plant market is constantly expanding. It is crucial to ensure high-quality plants by inspecting their quality before shipment and delivery. If consumers receive low-quality plants, their satisfaction with the purchase may decrease or they may become disappointed, which may in turn diminish their willingness to purchase. It is extremely difficult for humans to assess the quality of plants, especially kalanchoes. Handling kalanchoes during quality inspection can cause significant damage, potentially degrading their quality. Therefore, there is a pressing need for a technology that can assess plant quality without touching the plants themselves, including kalanchoes.

[0003] In the past, when people handled Kalanchoe, they caused a lot of damage to the Kalanchoe, which resulted in the deterioration of the quality of the Kalanchoe, and there was a problem that it was difficult for people to check each Kalanchoe with the naked eye due to the small shape of the Kalanchoe.

[0004] Embodiments of the present disclosure aim to address various issues, including the aforementioned ones, by providing an electronic device and its operating method for learning and predicting quality factors of each cultivar of Kalanchoe in a non-contact manner without contacting the plant, including Kalanchoe. However, these tasks are exemplary and are not intended to limit the scope of the present disclosure.

[0005] According to one aspect of the present disclosure, there is provided an electronic device comprising: an image sensor for photographing a flowerpot including a Kalanchoe to generate image data including an image of the Kalanchoe and the flowerpot; a memory for storing one or more instructions; and a processor for executing the one or more instructions stored in the memory, wherein the processor trains an artificial intelligence model to determine a variety of the Kalanchoe based on the image data received from the image sensor, trains the artificial intelligence model to predict and determine a plurality of quality factors including a canopy of the Kalanchoe, the number of flowers of the Kalanchoe, a flowering rate of the Kalanchoe, a ratio of peduncles of the Kalanchoe, and an amount of damage to the Kalanchoe caused by pests based on the image data, and inputs new image data received from the image sensor into the artificial intelligence model for which training has been completed to predict and determine the plurality of quality factors.

[0006] According to the present embodiment, the processor may receive first image data including a front image in which the front of the Kalanchoe is captured from the image sensor, detect the front image from the first image data, measure the width and length of the Kalanchoe excluding the flowerpot from the front image, and train the artificial intelligence model to determine the canopy according to the variety based on learning data including the variety and the canopy.

[0007] According to the present embodiment, the processor may receive a plurality of first image data each including a front view of the Kalanchoe photographed while the Kalanchoe is rotating, measure the width and the length in the front view image included in each of the plurality of first image data, and set the canopy including the maximum width and maximum length among the plurality of measured widths and lengths as the learning data.

[0008] According to the present embodiment, the processor receives a plurality of second image data including a top-view image of the Kalanchoe taken from a top-view perspective from the image sensor, detects a plurality of top-view images according to a flowering state of a flower from the plurality of second image data, and trains the artificial intelligence model to predict and determine the number of flowers of the Kalanchoe and the flowering rate of the Kalanchoe according to the variety based on learning data including each top-view image according to the flowering state.

[0009] According to the present embodiment, the processor can detect a first top-view image according to non-blooming, a second top-view image according to partial blooming, and a third top-view image according to full blooming from the plurality of second image data.

[0010] According to the present embodiment, the processor may receive third image data including a top-view image of the Kalanchoe taken from a top-view perspective from the image sensor, distinguish an image representing a flower of the Kalanchoe from an image representing a leaf of the Kalanchoe in the top-view image of the third image data, measure an area of ​​the flower in the image representing the flower of the Kalanchoe, and train the artificial intelligence model to predict and determine a ratio of a flower diameter of the Kalanchoe according to the variety based on learning data including the area.

[0011] According to the present embodiment, the processor may set an outermost circumscribed circle centered on the flowerpot in the third image data and an inscribed circle touching a flower located closest to the center, divide the circumscribed circle and the inscribed circle into four parts each to set eight regions, measure a partial area of ​​a flower included in each of the eight regions, and calculate the area of ​​the flower based on the partial area.

[0012] According to the present embodiment, the processor may receive fourth image data including a top-view image of the Kalanchoe taken from a top-view perspective from the image sensor, select an image representing a first area damaged by the pest and an image representing a second area not damaged by the pest from the top-view image of the fourth image data, and train the artificial intelligence model to determine the amount of damage to the Kalanchoe caused by the pest according to the variety based on learning data including the image representing the first area.

[0013] According to the present embodiment, the processor can further select an image representing the pest from the top view image and include the image representing the pest in the learning data.

[0014] According to another aspect of the present disclosure, a method of operating an electronic device is provided, comprising: photographing a flower pot including a Kalanchoe to generate image data including an image of the Kalanchoe and the flower pot; training an artificial intelligence model to determine a variety of the Kalanchoe based on the image data; training the artificial intelligence model to predict and determine a plurality of quality factors including a canopy of the Kalanchoe, a number of flowers of the Kalanchoe, a flowering rate of the Kalanchoe, a ratio of peduncles of the Kalanchoe, and an amount of damage to the Kalanchoe caused by pests based on the image data; and inputting new image data into the artificial intelligence model, which has completed training, to predict and determine the plurality of quality factors.

[0015] Other aspects, features and advantages other than those described above will become apparent from the following detailed description, claims and drawings for carrying out the invention.

[0016] Additionally, these general and specific aspects may be implemented using a system, method, computer program, or combination of any system, method, or computer program.

[0017] According to the exemplary embodiments of the present disclosure, as described above, an electronic device and its operating method can be implemented for learning and predicting quality factors specific to Kalanchoe varieties without damaging or harming the plants or degrading the quality of delivered plants. Of course, the scope of the present disclosure is not limited by these effects.

[0018] FIG. 1 is a block diagram schematically illustrating an electronic device according to an exemplary embodiment of the present disclosure.

[0019] FIG. 2 is a flowchart illustrating an operation method of an electronic device according to an exemplary embodiment of the present disclosure.

[0020] FIG. 3 is a flowchart conceptually illustrating the operation of an electronic device according to an exemplary embodiment of the present disclosure.

[0021] FIG. 4 is a flowchart illustrating steps for training an artificial intelligence model for determining a canopy according to an exemplary embodiment of the present disclosure.

[0022] FIG. 5 is a diagram illustrating a method for sensing the width and length of a Kalanchoe according to an exemplary embodiment of the present disclosure.

[0023] FIG. 6 is a flowchart illustrating steps for training an artificial intelligence model for predicting and determining the number of flowers and / or flowering rate of Kalanchoe according to an exemplary embodiment of the present disclosure.

[0024] FIG. 7 is a diagram illustrating a method for detecting images at each flowering stage according to an exemplary embodiment of the present disclosure.

[0025] FIG. 8 is a flowchart illustrating steps for training an artificial intelligence model for predicting and determining the flower diameter ratio of Kalanchoe according to an exemplary embodiment of the present disclosure.

[0026] FIGS. 9 and 10 are schematic drawings illustrating equally divided regions in concentric circles according to exemplary embodiments of the present disclosure.

[0027] FIG. 11 is a flowchart illustrating steps for training an artificial intelligence model for determining the amount of damage to Kalanchoe caused by pests according to an exemplary embodiment of the present disclosure.

[0028] FIG. 12 is a diagram visually illustrating a process for detecting pest damage according to an exemplary embodiment of the present disclosure.

[0029] The present disclosure is capable of various modifications and embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present disclosure, as well as methods for achieving them, will become clearer with reference to the embodiments described in detail below, along with the drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various forms.

[0030] In the examples below, the terms first, second, etc. are not used in a limiting sense, but are used for the purpose of distinguishing one component from another.

[0031] In the examples below, singular expressions include plural expressions unless the context clearly indicates otherwise.

[0032] In the following examples, terms such as “include” or “have” mean that a feature or component described in the specification is present, and do not preclude the possibility that one or more other features or components may be added.

[0033] In the following examples, when a part such as a layer, region, component, etc. is said to be on or above another part, it includes not only the case where it is directly above the other part, but also the case where another region, component, etc. is interposed in between.

[0034] For convenience of explanation, the sizes of components in the drawings may be exaggerated or reduced. For example, the sizes and thicknesses of each component shown in the drawings are arbitrarily indicated for convenience of explanation, and thus the present disclosure is not necessarily limited to the figures shown.

[0035] In some embodiments, where implementations are otherwise feasible, specific sequences of operations may be performed in a different order than described. For example, two steps described in succession may be performed substantially simultaneously, or in a reverse order from the described order.

[0036] In this specification, “A and / or B” refers to the case where it is A, or B, or both A and B. And, “at least one of A and B refers to the case where it is A, or B, or both A and B.

[0037] In the following examples, when it is said that layers, regions, components, etc. are connected, it includes cases where the layers, regions, components, etc. are directly connected, and / or cases where other layers, regions, components, etc. are interposed between the layers, regions, and components and are indirectly connected. For example, when it is said in this specification that layers, regions, components, etc. are electrically connected, it refers to cases where the layers, regions, components, etc. are directly electrically connected, and / or cases where other layers, regions, components, etc. are interposed between them and are indirectly electrically connected.

[0038] The x-axis, y-axis, and z-axis are not limited to the three axes in the Cartesian coordinate system, but can be interpreted in a broader sense that includes them. For example, the x-axis, y-axis, and z-axis may be orthogonal to each other, but they can also refer to different directions that are not orthogonal to each other.

[0039] The advantages and features of the present disclosure, and methods for achieving them, will become clearer with reference to the embodiments described below in detail with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure that the disclosure is complete and to fully inform those skilled in the art of the scope of the present disclosure, and the present disclosure is defined solely by the scope of the claims.

[0040] The terminology used in this disclosure is for the purpose of describing embodiments only and is not intended to limit the present disclosure. In this disclosure, the singular may also include the plural unless specifically stated otherwise. The terms "comprises" and / or "comprising" as used herein do not exclude the presence or addition of one or more other components in addition to the mentioned components. Like reference numerals refer to like components throughout the disclosure, and "and / or" may include each and any combination of one or more of the mentioned components. Although "first", "second", etc. are used to describe various components, these components are not limited by these terms. These terms are only used to distinguish one component from another. Therefore, it should be understood that a first component mentioned below may also be a second component within the technical spirit of the present disclosure.

[0041] The word "exemplary" is used herein to mean "serving as an example or illustration." Any embodiment described herein as "exemplary" should not necessarily be construed as preferred or advantageous over other embodiments.

[0042] Embodiments of the present disclosure may be described in terms of a function or a block that performs a function. A block, which may be referred to as a "unit" or a "module" in the present disclosure, may be physically implemented by analog or digital circuits such as logic gates, integrated circuits, microprocessors, microcontrollers, memories, passive electronic components, active electronic components, optical components, hardwired circuits, etc., and may optionally be driven by firmware and software. Furthermore, the term "unit" as used in the disclosure refers to software, hardware elements such as FPGAs or ASICs, and the "unit" may perform certain roles. However, the "unit" is not limited to software or hardware. The "unit" may be configured to reside on an addressable storage medium and may be configured to execute one or more processors. Thus, as an example, a "part" may include elements such as software elements, object-oriented software elements, class elements, and task elements, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functionality provided within the elements and "parts" may be combined into a smaller number of elements and "parts" or further separated into additional elements and "parts."

[0043] Embodiments of the present disclosure can be implemented using at least one software program running on at least one hardware device and capable of performing network management functions to control elements.

[0044] Spatially relative terms such as "below," "beneath," "lower," "above," and "upper" may be used to readily describe the relationship between one component and other components as depicted in the drawings. Spatially relative terms may be understood to encompass different orientations of components during use or operation in addition to the orientations depicted in the drawings. For example, if a component depicted in the drawings were flipped over, a component described as "below" or "beneath" another component may end up "above" the other component. Thus, the exemplary term "below" may encompass both the above and below orientations. Components may also be oriented in other directions, and thus spatially relative terms may be interpreted accordingly.

[0045] Unless otherwise defined, all terms (including technical and scientific terms) used in this disclosure may be used with the meaning commonly understood by those skilled in the art to which this disclosure pertains. Furthermore, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0046] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings. When describing with reference to the drawings, identical or corresponding components are given the same reference numerals and redundant descriptions thereof will be omitted.

[0047] The terms in this disclosure are defined as follows. Differentiation: a plant growing in a pot; Plant height: the length from the root and stem boundary to the tip of the leaf (vertical plant length); Plant width: the horizontal length of the plant; Canopy: the shape of the above-ground part of the plant (plant height and plant width); Flowering: the phenomenon of flowers blooming; Non-flowering: the phenomenon of flowers not blooming yet; Flowering rate: the rate of flowers blooming; Peduncle: a peduncle, or the part of the peduncle that connects to the flower stalk when one flower blooms on the inflorescence; Wiping: the phenomenon of a plant withering and drying up; Insect damage: damage caused by insects; Stock: plants that are not marketable or plants that are not sold; Selection: the work of organizing items according to size or grade.

[0048] FIG. 1 is a block diagram schematically illustrating an electronic device (100) according to an exemplary embodiment of the present disclosure.

[0049] Referring to FIG. 1, an electronic device (100) can communicate with a user terminal and execute a program of program data. In the present specification, the 'electronic device for learning and predicting quality factors for each variety of Kalanchoe (hereinafter, the device according to the present disclosure)' includes various devices that can perform computational processing and provide results to a user. For example, the device according to the present disclosure may be a computing device, including all of a computer, a server device, and a portable terminal, or may be in the form of any one of them. Here, the computer may include, for example, a notebook, desktop, laptop, tablet PC, slate PC, etc. equipped with a web browser. The server device is a server that communicates with an external device to process information, and may include an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, and a web server.A portable terminal is, for example, a wireless communication device that ensures portability and mobility, and may include all kinds of handheld-based wireless communication devices such as PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handy-phone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminals, smart phones, etc., and wearable devices such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted devices (HMD).

[0050] The image sensor (110) can photograph a plant. In exemplary embodiments, the image sensor (110) can photograph a Kalanchoe and a flowerpot containing the Kalanchoe. In exemplary embodiments, the image sensor (110) can photograph a flowerpot containing the Kalanchoe and generate image data including images of the Kalanchoe and the flowerpot. The image sensor (110) can photograph the Kalanchoe from various viewpoints, such as the front, side, top, and back of the Kalanchoe. In exemplary embodiments, the image sensor (110) can be implemented as a camera, but the present disclosure is not limited to the exemplary embodiments. In the present disclosure, an image photographing the top surface of the Kalanchoe is referred to as a 'top-view image'.

[0051] The memory (120) can store data for an algorithm for controlling the operation of components within the device or a program that reproduces the algorithm, and can be implemented with at least one processor (130) that performs the aforementioned operation using the data stored in the memory (120). Here, the memory (120) and the processor (130) can each be implemented as separate chips. In addition, the memory (120) and the processor (130) can also be implemented as a single chip.

[0052] The memory (120) can store data supporting various functions of the device, programs for the operation of the processor (130), input / output data, and a plurality of application programs (or applications) run on the device, data for the operation of the device, commands, and one or more instructions. At least some of these application programs can be downloaded from an external server via wireless communication.

[0053] The memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, an SSD (Solid State Disk type), an SDD (Silicon Disk Drive type), a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. In addition, the memory (120) may be a database that is separate from the device but is connected by wire or wirelessly.

[0054] The processor (130) may execute one or more instructions stored in the memory (120). The processor (130) may train an artificial intelligence model to determine the variety of the Kalanchoe based on image data received from the image sensor (110). In addition, the processor (130) may train the artificial intelligence model to predict and determine a plurality of quality factors based on the image data through an artificial neural network processing unit. In exemplary embodiments, the plurality of quality factors may include the canopy of the Kalanchoe, the number of flowers of the Kalanchoe, the flowering rate of the Kalanchoe, the ratio of the peduncle of the Kalanchoe, and the amount of damage to the Kalanchoe due to pests. The processor (130) may input new image data received from the image sensor (110) into the trained artificial intelligence model to predict and determine a plurality of quality factors.

[0055] An artificial intelligence model can be implemented through an artificial neural network processing unit. For example, the artificial neural network processing unit can implement an artificial intelligence model by calling functions, libraries, parameter values, hyperparameters, etc. stored in memory (120).

[0056] In exemplary embodiments, the processor (130) may suitably preprocess an image of image data as input to an artificial intelligence model.

[0057] The artificial intelligence-related functions according to the present disclosure are operated through a processor (130) and a memory (120). The processor (130) may be composed of one or more processors. In this case, the one or more processors may be a general-purpose processor such as a CPU, an AP, a DSP (Digital Signal Processor), a graphics-only processor such as a GPU, a VPU (Vision Processing Unit), or an artificial intelligence-only processor such as an NPU. The one or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in the memory (120). Alternatively, when the one or more processors are artificial intelligence-only processors, the artificial intelligence-only processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0058] The predefined operation rules or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that the basic artificial intelligence model is learned by a learning algorithm using a plurality of learning data, thereby creating a predefined operation rules or artificial intelligence model set to perform a desired characteristic (or purpose). This learning may be performed on the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and / or system. Examples of the learning algorithm include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0059] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values, and performs neural network operations through operations between the operation results of the previous layer and the multiple weights. The multiple weights of the multiple neural network layers may be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained from the artificial intelligence model is reduced or minimized during the learning process. The artificial neural network may include a deep neural network (DNN), and examples thereof include, but are not limited to, a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or deep Q-networks.

[0060] According to an exemplary embodiment of the present disclosure, the processor (130) can implement artificial intelligence. Artificial intelligence refers to a machine learning method based on an artificial neural network that imitates human neurons (biological neurons) to enable a machine to learn. Artificial intelligence methodologies can be divided into supervised learning, in which input data and output data are provided together as training data depending on the learning method, so that the solution (output data) to the problem (input data) is determined, unsupervised learning, in which only input data is provided without output data, so that the solution (output data) to the problem (input data) is not determined, and reinforcement learning, in which a reward is given from an external environment whenever an action is taken in the current state, and learning is performed in a direction to maximize this reward. In addition, artificial intelligence methodologies can be categorized according to the architecture of the learning model. The architectures of widely used deep learning technologies can be categorized into convolutional neural networks (CNNs), recurrent neural networks (RNNs), transformers, and generative adversarial networks (GANs).

[0061] The present device and system may include an artificial intelligence model. The artificial intelligence model may be a single artificial intelligence model or may be implemented as multiple artificial intelligence models. The artificial intelligence model may be composed of a neural network (or artificial neural network) and may include statistical learning algorithms that mimic biological neurons in machine learning and cognitive science. A neural network may refer to a model in general that has problem-solving capabilities by changing the binding strength of synapses through learning, formed by artificial neurons (nodes) that form a network by combining synapses. The neurons of the neural network may include a combination of weights or biases. The neural network may include one or more layers composed of one or more neurons or nodes. For example, the device may include an input layer, a hidden layer, and an output layer. The neural network constituting the device can infer a desired result (output) from an arbitrary input (input) by changing the weights of neurons through learning.

[0062] The processor (130) may generate a neural network, train a neural network, perform a calculation based on received input data, generate an information signal based on the calculation result, or retrain a neural network. The models of the neural network may include various types of models such as CNN (Convolution Neural Network) such as GoogleNet, AlexNet, VGG Network, R-CNN (Region with Convolution Neural Network), RPN (Region Proposal Network), RNN (Recurrent Neural Network), S-DNN (Stacking-based deep Neural Network), S-SDNN (State-Space Dynamic Neural Network), Deconvolution Network, DBN (Deep Belief Network), RBM (Restrcted Boltzman Machine), Fully Convolutional Network, LSTM (Long Short-Term Memory) Network, Classification Network, etc., but are not limited thereto. The processor (130) may include one or more processors for performing calculations according to the models of the neural network. There are. For example, a neural network can include a deep neural network.

[0063] Neural networks include CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), perceptron, multilayer perceptron, FF (Feed Forward), RBF (Radial Basis Network), DFF (Deep Feed Forward), LSTM (Long Short Term Memory), GRU (Gated Recurrent Unit), AE (Auto Encoder), VAE (Variational Auto) Encoder), DAE (Denoising Auto Encoder), SAE (Sparse Auto Encoder), MC (Markov Chain), HN (Hopfield Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Depp Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), Generative Adversarial Network (GAN), Liquid State Machine (LSM), Extreme Learning Machine (ELM), It will be understood by those skilled in the art that any neural network may be included, including but not limited to ESN (Echo State Network), DRN (Deep Residual Network), DNC (Differentiable Neural Computer), NTM (Neural Turning Machine), CN (Capsule Network), KN (Kohonen Network), and AN (Attention Network).

[0064] According to an exemplary embodiment of the present disclosure, the processor (130) may be configured to perform a CNN (Convolution Neural Network) such as GoogleNet, AlexNet, VGG Network, etc., R-CNN (Region with Convolution Neural Network), RPN (Region Proposal Network), RNN (Recurrent Neural Network), S-DNN (Stacking-based deep Neural Network), S-SDNN (State-Space Dynamic Neural Network), Deconvolution Network, DBN (Deep Belief Network), RBM (Restrcted Boltzman Machine), Fully Convolutional Network, LSTM (Long Short-Term Memory) Network, Classification Network, Generative Modeling, eXplainable AI, Continual AI, Representation Learning, AI for Material Design, BERT for natural language processing, SP-BERT, MRC / QA, Text Analysis, Dialog System, GPT-3, GPT-4, Visual Analytics for vision processing, Visual Understanding, Video Synthesis, ResNet, Anomaly Detection, Prediction, Time-Series for data intelligence Various artificial intelligence structures and algorithms can be used, including but not limited to Forecasting, Optimization, Recommendation, and Data Creation.

[0065] Although not shown, the electronic device (100) may further include a communication unit. The communication unit may perform communication with at least one user terminal. At this time, the communication unit may include a wireless communication module that supports various wireless communication methods such as GSM (global System for Mobile Communication), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), UMTS (universal mobile telecommunications system), TDMA (Time Division Multiple Access), LTE (Long Term Evolution), 4G, 5G, and 6G, in addition to a Wi-Fi module and a WiBro (Wireless broadband) module.

[0066] According to the exemplary embodiments described above, by identifying the quality factors of each kalanchoe variety through an artificial intelligence model in a non-contact manner without directly contacting or destroying the kalanchoe, damage and harm to plants including kalanchoe can be prevented, the quality of the plants can be prevented from deteriorating, and the marketability of the plants can be improved.

[0067] FIG. 2 is a flowchart for explaining an operation method of an electronic device (100) according to an exemplary embodiment of the present disclosure.

[0068] Referring to FIG. 2, a step (S100) of generating image data including images of a Kalanchoe and a flowerpot is performed. In an exemplary embodiment, an image sensor (110) may capture a flowerpot included in a Kalanchoe to generate image data including images of the Kalanchoe and the flowerpot. In addition, the image sensor (110) may provide the image data to a processor (130). In the present disclosure, the image data is exemplified as a two-dimensional RGB image, but is not limited thereto, and various imaging techniques such as a thermally captured image, a spectral spectrum-based image, and a depth image including a depth value may be used.

[0069] A step (S120) of training an artificial intelligence model to determine the variety of Kalanchoe based on image data is performed. In an exemplary embodiment, the processor (130) may train an artificial intelligence model to determine the variety of Kalanchoe based on the image data. It is important to establish a reference point for determining the quality of Kalanchoe for each variety and to establish guidelines for each variety. Accordingly, the electronic device (100) may collect a frontal image of Kalanchoe, detect an image of Kalanchoe, detect characteristic objects such as flower color and shape for each Kalanchoe variety, and identify and determine the variety through the artificial intelligence model. In an exemplary embodiment, the processor (130) may box an entire image of a flowerpot, extract an image of the flowerpot included in a box, detect characteristics such as flower color and size from the flowerpot image, and train an artificial intelligence model to determine the variety of Kalanchoe using a dataset including the detected characteristics and labeled Kalanchoe varieties.

[0070] A step (S130) of training an artificial intelligence model to predict and determine multiple quality factors of a Kalanchoe based on image data is performed. In an exemplary embodiment, the processor (130) may train an artificial intelligence model to determine multiple quality factors, including a Kalanchoe canopy, the number of Kalanchoe flowers, a Kalanchoe flowering rate, a Kalanchoe peduncle ratio (a state in which flowers are arranged on an inflorescence axis, or a shape in which flowers bloom), and an amount of damage to the Kalanchoe caused by pests, based on the image data.

[0071] In the present disclosure, the number of flowers is the sum of the number of bloomed flowers and the number of unbloomed flowers, and the flowering rate is the value (percentage) obtained by dividing the number of bloomed flowers by the number of flowers and multiplying by 100.

[0072] A step (S140) is performed to input new image data into a trained artificial intelligence model to predict and determine multiple quality factors. In an exemplary embodiment, the image sensor (110) may generate new image data and provide it to the processor (130), and the processor (130) may input the new image data received from the image sensor (110) into the trained artificial intelligence model to predict and determine multiple quality factors. In an exemplary embodiment, the prediction and determination results of the quality factors predicted and determined by the artificial intelligence model may be output as a score. The score will be described in more detail with reference to FIG. 9.

[0073] FIG. 3 is a flowchart conceptually illustrating the operation of an electronic device according to an exemplary embodiment of the present disclosure.

[0074] Referring to FIG. 3, a sensing device such as an image sensor (110) included in an electronic device (100) can sense Kalanchoe. Kalanchoe is a succulent plant of the Crassulaceae family in the rose order, and its scientific name is Kalanchoe Blossfeldiana.

[0075] According to an exemplary embodiment, the electronic device (100) can detect the flower color, flower shape, and leaf shape of the Kalanchoe, and identify and determine the variety of the Kalanchoe through an operation of the processor (130).

[0076] FIG. 4 is a flowchart illustrating steps for training an artificial intelligence model for determining a canopy according to an exemplary embodiment of the present disclosure.

[0077] Referring to Figure 4, consumers do not prefer Kalanchoes that are too long or too short. Unlike cut flowers, which can be adjusted in length by cutting, Kalanchoes are a crop whose length cannot be arbitrarily adjusted during distribution. Therefore, the canopy length and width of the Kalanchoe when shipped can be important quality factors for the Kalanchoe. The electronic device (100) can measure the width and length of the Kalanchoe by detecting only the portion corresponding to the Kalanchoe flower in the image.

[0078] In an exemplary embodiment, a step (S210) of receiving first image data including a frontal image of the front of Kalanchoe is performed. In an exemplary embodiment, the processor (130) may receive first image data including a frontal image of the front of Kalanchoe from an image sensor (110).

[0079] A step (S220) of detecting a front image from the first image data is performed. In an exemplary embodiment, the processor (130) may detect a front image from the first image data. In this case, the front image may be an image of one side of a flowerpot containing Kalanchoe, captured by an image sensor (110).

[0080] A detection step (S230) of measuring the width and length of a Kalanchoe in a frontal image is performed. In an exemplary embodiment, the processor (130) can measure the width and length of a Kalanchoe excluding the flowerpot in the frontal image.

[0081] A step (S240) of training an artificial intelligence model for determining a canopy according to a variety is performed based on learning data including varieties and canopies. In an exemplary embodiment, the processor (130) may train an artificial intelligence model for determining a canopy according to a variety based on learning data including varieties and canopies. Accordingly, when modeling an artificial intelligence model based on values ​​measured by an image sensor (110) spaced a certain distance from a flower pot, there is an advantage in that values ​​almost identical to actual values ​​can be obtained.

[0082] According to an exemplary embodiment of the present disclosure, the image sensor (110) can not only secure a pot containing Kalanchoe, but also rotate the entire pot to measure the shape of the pot from all angles. Therefore, since the shape is captured at all angles, the maximum values ​​of the maximum length, maximum width, etc. of the Kalanchoe can be accurately measured.

[0083] According to an exemplary embodiment, the image sensor (110) may sense a plurality of image data and then provide the data to the processor (130), and the processor (130) may call a function or algorithm for extracting a maximum value among the sensed image data from a memory, or may extract the length and width of a Kalanchoe having a maximum value among the plurality of image data through an artificial neural network model trained exclusively for image processing. In an exemplary embodiment, the processor (130) may plot a length graph and a width graph for the length and width extracted from each of the plurality of image data, and may apply a first derivative to the length graph and the width graph to calculate the maximum value, thereby calculating the length maximum value and the width maximum value.

[0084] In an exemplary embodiment, the processor (130) may receive a plurality of first image data each including a front view of the Kalanchoe photographed while the Kalanchoe is rotating. Then, the processor (130) may measure a width and a length from a front view image included in each of the plurality of first image data. Then, the processor (130) may set a canopy including a maximum width and a maximum length among the plurality of measured widths and lengths as learning data. According to this, since a numerical value can be read for each photographing viewpoint, there is an advantage in that the maximum length and maximum width among the plurality of numerical values ​​can be measured more accurately.

[0085] In an exemplary embodiment, the electronic device (100) can collect a frontal image, detect an image of a Kalanchoe, measure the width and length of the plant excluding the pot, and evaluate and score the quality of the Kalanchoe based on its variety and characteristics using an artificial intelligence model. The score will be described in more detail with reference to FIG. 9.

[0086] According to the exemplary embodiments described above, there is an effect of providing advantageous conditions, convenience, and marketability to farmers and farmers by photographing kalanchoes and judging the quality of kalanchoes in a non-contact manner without touching or destroying the plants containing kalanchoes.

[0087] FIG. 5 is a diagram illustrating a method for sensing the width and length of a Kalanchoe according to an exemplary embodiment of the present disclosure.

[0088] Referring to FIG. 5 together with FIG. 4, the electronic device (100) can sense a flower pot containing Kalanchoe and sense the width and plant height of the plant. For example, the horizontal length (plant width) of the Kalanchoe may be 191.84 mm (millimeters), and the vertical length (plant height) may be 147.62 mm. The sensed values ​​of multiple Kalanchoes can be plotted as a graph, and a trend line of the plant width can be fitted.

[0089] FIG. 6 is a flowchart illustrating steps for training an artificial intelligence model for predicting and determining the number of flowers and / or flowering rate of Kalanchoe according to an exemplary embodiment of the present disclosure.

[0090] Referring to Figure 6, a single pot of Kalanchoe can typically contain over 100 flowers. The number of flowers (or number of flowers, amount of flowers) and flowering rate can be important quality factors in assessing the quality of Kalanchoe. However, due to the relatively large number of flowers, non-contact imaging may be necessary to accurately assess these quality factors.

[0091] In an exemplary embodiment, a step (S310) of receiving a plurality of second image data including a top-view image of Kalanchoe is performed. In an exemplary embodiment, the processor (130) may receive a plurality of second image data including a top-view image of Kalanchoe captured from a top-view perspective from an image sensor (110). A step (S320) of detecting a plurality of top-view images according to a flowering state of a flower from the plurality of second image data is performed. In an exemplary embodiment, the processor (130) may detect a plurality of top-view images according to a flowering state of a flower from the plurality of second image data.

[0092] A step (S330) of training an artificial intelligence model for predicting and determining the number of flowers and the flowering rate of Kalanchoe according to a variety is performed based on learning data including each top-view image according to the flowering state. In an exemplary embodiment, the processor (130) may train an artificial intelligence model for predicting and determining the number of flowers and the flowering rate of Kalanchoe according to a variety based on learning data including each top-view image according to the flowering state.

[0093] Meanwhile, when flowers bloom (i.e., when flowers open), they bloom in various forms. In particular, since the flowers are in bloom when they are shipped, not all flowers in the Kalanchoe may be in bloom. In other words, the flowering states of all flowers in the Kalanchoe may be different. Therefore, it is necessary to individually train an artificial intelligence model with images of each stage of the Kalanchoe's flowering. In an exemplary embodiment, the processor (130) may detect a first top-view image according to non-blooming, a second top-view image according to partial blooming, and a third top-view image according to full blooming from a plurality of second image data. In addition, the processor (130) may train the artificial intelligence model with the first to third top-view images as individual learning datasets. In an exemplary embodiment for partial blooming, partial blooming may mean that half of the total number of flowers have bloomed, but is not limited thereto.

[0094] In an exemplary embodiment, the processor (130) may distinguish, detect, generate, and preprocess the first to third top-view images by boxing non-blooming, half-blooming, and / or blooming flowers in a rectangular manner from a plurality of second image data. In an exemplary embodiment, the processor (130) may also process blurred portions, which are flaws in the images.

[0095] Petals that are out of focus or outside the region of interest are typically treated as exceptions or interpreted as noise, but according to exemplary embodiments of the present disclosure, objects such as petals that appear blurry or noisy are also labeled and sensed, thereby further improving the accuracy of the object recognition model.

[0096] In an exemplary embodiment, the electronic device (100) collects an image from a top view, detects an image of a Kalanchoe, selects an object by distinguishing between non-blooming, half-blooming, and fully blooming, and determines and scores the total number of flowers and the flowering rate through an artificial intelligence model. According to an exemplary embodiment of the present disclosure, flowers bloom in various forms, and most of the flowers are not in full bloom when they are released. Therefore, the accuracy of image sensing can be improved by individually learning flowering images for each stage of flower blooming. According to an exemplary embodiment of the present disclosure, the shape change pattern of Kalanchoe for each flowering stage can be broadly divided into three stages, and each stage can be distinguished into a non-blooming state, a half-blooming (half-blooming) state, and a fully blooming state. As a result of performing image learning according to the present disclosure, it was confirmed that distinguishing between the three flowering states improved the performance of the learning model.

[0097] According to the exemplary embodiments described above, by photographing kalanchoe in a non-contact manner and predicting and judging the quality of kalanchoe, it is possible to provide advantageous conditions, convenience, and marketability to farmers and farmers.

[0098] FIG. 7 is a diagram illustrating a method for detecting images at each flowering stage according to an exemplary embodiment of the present disclosure.

[0099] Referring to FIG. 7 together with FIG. 6, the electronic device (100) can receive an image of a Kalanchoe and perform object recognition according to its flowering state. According to an exemplary embodiment, the electronic device (100) can detect petals from the Kalanchoe petal image by learning that the flowering stage is divided into non-blooming, partially blooming, and fully blooming states. For example, the electronic device (100) can detect Kalanchoe petals one by one by generating a bounding box.

[0100] According to an exemplary embodiment, the electronic device (100) can label and sense objects such as petals that are out of focus or outside the region of interest, or petals that appear blurry or noisy, thereby further improving the accuracy of the object recognition model.

[0101] FIG. 8 is a flowchart illustrating steps for training an artificial intelligence model for predicting and determining the flower diameter ratio of Kalanchoe according to an exemplary embodiment of the present disclosure. FIGS. 9 and 10 are diagrams schematically illustrating equally divided regions in concentric circles according to an exemplary embodiment of the present disclosure.

[0102] Referring to Figure 8, multiple flower stems, or peduncles, can be attached to a single kalanchoe plant in a single pot. Furthermore, since multiple flowers are attached to a single stem, a relatively thin stem may sag under the weight of the numerous flowers. When peduncles are positioned symmetrically and in equal proportions, the stem is less likely to sag or fall. Therefore, the initial shipment peduncle ratio can be an important quality factor.

[0103] In an exemplary embodiment, a step (S410) of receiving third image data including a top-view image of a Kalanchoe is performed. In an exemplary embodiment, the processor (130) may receive the third image data including a top-view image of a Kalanchoe captured from a top-view perspective from an image sensor (110). A step (S420) of distinguishing images representing flowers and leaves of a Kalanchoe from the top-view images of the third image data is performed. In an exemplary embodiment, the processor (130) may distinguish a first image representing a Kalanchoe flower and a second image representing leaves of the Kalanchoe from the top-view images of the third image data. A step (S430) of measuring an area of ​​a flower in the image representing a Kalanchoe flower is performed. In an exemplary embodiment, the processor (130) may measure an area of ​​a flower in the first image representing a Kalanchoe flower. A step (S440) of training an artificial intelligence model for predicting and determining the flower diameter ratio of Kalanchoe according to variety is performed based on learning data including area. In an exemplary embodiment, the processor (130) may train an artificial intelligence model for predicting and determining the flower diameter ratio of Kalanchoe according to variety based on learning data including area.

[0104] In an exemplary embodiment, to distinguish between flowers and leaves in a top-view image and accurately detect and determine the area of ​​a flower, it is necessary to determine not only the quantity of flowers but also their locations. Therefore, by dividing the inscribed circle (IC) and circumscribed circle (OC) set in the top-view image into four equal parts and dividing the inscribed circle (IC) and circumscribed circle (OC) into eight areas, a more accurate flower diameter ratio can be measured.

[0105] In an exemplary embodiment, the processor (130) may set an outermost circumscribed circle (OC) centered on the flowerpot in the third image data and an inscribed circle (IC) touching the flower located closest to the center, divide the circumscribed circle (OC) and the inscribed circle (IC) into four parts each to set eight regions, measure a partial area of ​​a flower included in each of the eight regions, and calculate the area of ​​the flower based on the partial area.

[0106] In an exemplary embodiment of the present disclosure, the reference point in the top-view image is the center of the flowerpot, not the center of the circumscribed circle (OC), and the center of the flowerpot may be fixed. Therefore, the judgment accuracy can be further improved compared to simply using the center of the circumscribed circle (OC) as the reference point.

[0107] In an exemplary embodiment, the electronic device (100) can collect images from a top view, detect images of Kalanchoe, detect a flower, measure the flower ratio, and determine and score the flower ratio through an artificial intelligence model.

[0108] According to the exemplary embodiments described above, by photographing kalanchoe in a non-contact manner and predicting and judging the quality of kalanchoe, it is possible to provide advantageous conditions, convenience, and marketability to farmers and farmers.

[0109] FIG. 11 is a flowchart illustrating steps for training an artificial intelligence model for determining the amount of damage to Kalanchoe caused by pests according to an exemplary embodiment of the present disclosure.

[0110] Referring to Figure 11, Kalanchoe plants are generally more susceptible to pest damage than diseases such as fungal and bacterial diseases. Because pests are relatively small, they can be difficult for humans to see. Since pest damage should not be delivered as supplies, it is difficult for humans to directly identify and screen for pest damage. Therefore, pests and / or the area of ​​pest damage can be important quality factors for Kalanchoe plants, and it is necessary to identify pests and / or the area of ​​pest damage in a top-view image.

[0111] In an exemplary embodiment, a step (S510) of receiving fourth image data including a top-view image of Kalanchoe is performed. In an exemplary embodiment, the processor (130) may receive fourth image data including a top-view image of Kalanchoe captured from a top-view perspective from an image sensor (110).

[0112] A step (S520) of selecting images representing the first and second areas, respectively, from the top-view image of the fourth image data is performed. In an exemplary embodiment, the processor (130) may select a third image representing the first area damaged by pests and a fourth image representing the second area not damaged by pests from the top-view image of the fourth image data.

[0113] A step (S540) of training an artificial intelligence model for determining the amount of damage to Kalanchoe caused by pests according to the variety is performed based on training data including an image representing the first area. In an exemplary embodiment, the processor (130) may train an artificial intelligence model for determining the amount of damage to Kalanchoe caused by pests according to the variety based on training data including a third image representing the first area.

[0114] In an exemplary embodiment, a step (S530) of selecting an image representing a pest from the top-view image of the fourth image data may be further included. In an exemplary embodiment, the processor (130) may further select a fifth image representing a pest from the top-view image, and include the fifth image representing the pest in the learning data. In an exemplary embodiment for S540, the processor (130) may train an artificial intelligence model for determining the amount of damage to Kalanchoe caused by pests according to the variety, based on learning data including the third image representing the first area and the fifth image representing the pest.

[0115] In an exemplary embodiment, the electronic device (100) can collect images from a top view, detect images of Kalanchoe, select objects damaged by pests and pests, and determine and score the items through an artificial intelligence model.

[0116] FIG. 12 is a diagram visually illustrating a process for detecting pest damage according to an exemplary embodiment of the present disclosure.

[0117] Referring to FIG. 12, the electronic device (100) can receive image data from Kalanchoe and extract objects including areas damaged by pests. In an exemplary embodiment, the electronic device (100) can perform bounding box processing on the area damaged by pests to calculate the size and number of pests damaged.

Claims

1. An image sensor that photographs a flowerpot containing a Kalanchoe and generates image data containing images of the Kalanchoe and the flowerpot; Memory that stores one or more instructions; and A processor comprising: a processor that executes one or more instructions stored in the memory; The above processor, An electronic device characterized in that an artificial intelligence model is trained to determine the variety of the Kalanchoe based on the image data received from the image sensor, and the artificial intelligence model is trained to predict and determine a plurality of quality factors including a canopy of the Kalanchoe, the number of flowers of the Kalanchoe, a flowering rate of the Kalanchoe, a ratio of a peduncle in which flowers are arranged on a flower axis of the Kalanchoe, and an amount of damage to the Kalanchoe caused by pests based on the image data, and new image data received from the image sensor is input to the artificial intelligence model that has completed training to predict and determine the plurality of quality factors, and the ratio of the peduncle is determined, and the quality score is learned in a direction in which the quality score increases as the ratio of flowers arranged on both sides of the flower axis becomes substantially the same.

2. In paragraph 1, The above processor, An electronic device characterized in that it receives first image data including a frontal image in which the front of the Kalanchoe is captured from the image sensor, detects the frontal image from the first image data, measures the width and length of the Kalanchoe excluding the flowerpot from the frontal image, and trains the artificial intelligence model to determine the canopy according to the variety based on learning data including the variety and the canopy.

3. In paragraph 2, The above processor, An electronic device characterized in that it receives a plurality of first image data each including a front view of the Kalanchoe photographed while the Kalanchoe is rotated, measures the width and the length from the front image included in each of the plurality of first image data, and sets the canopy including the maximum width and maximum length among the plurality of measured widths and lengths as the learning data.

4. In paragraph 1, The above processor, An electronic device characterized in that it receives a plurality of second image data including a top-view image taken from a top-view point of view of the Kalanchoe from the image sensor, detects a plurality of top-view images according to a flowering state of a flower from the plurality of second image data, and trains the artificial intelligence model to predict and determine the number of flowers of the Kalanchoe and the flowering rate of the Kalanchoe according to the variety based on learning data including each top-view image according to the flowering state.

5. In paragraph 4, The above processor, An electronic device characterized in that it detects a first top view image according to non-blooming, a second top view image according to partial blooming, and a third top view image according to full blooming from the plurality of second image data.

6. In paragraph 1, The above processor, An electronic device characterized in that it receives third image data including a top-view image of the Kalanchoe taken from a top-view point of view from the image sensor, distinguishes an image representing a flower of the Kalanchoe from an image representing a leaf of the Kalanchoe in the top-view image of the third image data, measures the area of ​​the flower in the image representing the flower of the Kalanchoe, and trains the artificial intelligence model to predict and determine the ratio of the flower diameter of the Kalanchoe according to the variety based on learning data including the area.

7. In paragraph 6, The above processor, In the third image data above, the outermost circumcircle centered on the flowerpot and the incircle touching the flower located closest to the center are set, Divide the above circumscribed circle and the above inscribed circle into four equal parts, and set up eight regions. Measure the partial area of ​​the flower contained in each of the above eight areas, An electronic device characterized in that the area of ​​the flower is calculated based on the above partial area.

8. In paragraph 1, The above processor, An electronic device characterized in that it receives fourth image data including a top-view image taken from a top-view point of view of the Kalanchoe from the image sensor, selects an image representing a first area damaged by the pest and an image representing a second area not damaged by the pest from the top-view image of the fourth image data, and trains the artificial intelligence model to determine the amount of damage to the Kalanchoe caused by the pest according to the variety based on learning data including the image representing the first area.

9. In paragraph 8, The above processor, An electronic device characterized in that it further selects an image representing the pest from the top view image and includes the image representing the pest in the learning data.

10. A step of photographing a flowerpot containing a Kalanchoe to generate image data including images of the Kalanchoe and the flowerpot; A step of training an artificial intelligence model to determine the variety of the Kalanchoe based on the image data; A step of training the artificial intelligence model to predict and determine a plurality of quality factors including the canopy of the Kalanchoe, the number of flowers of the Kalanchoe, the flowering rate of the Kalanchoe, the proportion of peduncles in which flowers are arranged on the inflorescence axis of the Kalanchoe, and the amount of damage to the Kalanchoe caused by pests, based on the image data; A step of learning in which the ratio of the above flower axis is determined, and the quality score increases as the ratio of flowers arranged on both sides of the flower axis becomes substantially the same; and An operating method of an electronic device, comprising a step of inputting new image data into the artificial intelligence model for which learning has been completed to predict and determine the plurality of quality factors.

Images