Learning-based pollen recognition device and method

Abstract

The present application relates to a pollen recognition device and method, and more particularly, to a device and method that accurately recognize whether pollen is attached to a bee and the amount of pollen using learning data acquired by learning features of a bee's form and the color of a bee and pollen, etc. through a deep learning model.

Description

Technical Field

[0001] This invention relates to a pollen identification device and method, and more particularly to a device and method that accurately identifies whether pollen has adhered to a bee and the amount of pollen by using learning data obtained through a deep learning model that learns the morphology of bees and the color of bees and pollen. Background Technology

[0002] Honeybees are bees used for storing and producing honey, while bumblebees (also known as garden bumblebees or red bumblebees) are bees used for pollinating plants, pollinating flowers through buzz pollination.

[0003] Bee pollination refers to the behavior of collecting pollen by vibrating the pectoral muscles to transfer pollen to the hairy tufts on the body. This vibration helps the pollen to fall onto the stigma of the pistil.

[0004] Therefore, bumblebees are mostly used for pollination of plants in the Solanaceae family, such as tomatoes, eggplants, and peppers, which are difficult to pollinate by bees due to the lack of nectar, or for the cultivation of crops such as peaches, plums, apples, apricots, strawberries, cantaloupes, pears, blueberries, raspberries, and mangoes.

[0005] For example, when cultivating tomatoes, bumblebees are released into the colony. Typically, 10% to 20% of the worker bees will collect pollen 5 to 12 times a day, and during each collection activity, they will fly to 50 to 220 flowers.

[0006] If the number of bees far exceeds the number of flowers, the excessive number can lead to deformed fruit or poor pollination. Therefore, using beehives with separate entrances and exits and a return port, and adjusting the switches according to the time period and bee activity, can control the bumblebee's foraging activities.

[0007] If a specified period of time passes after setting up a beehive, the number of individual bumblebees will decrease, and their activity will also decrease, thus requiring the beehive to be replaced. If the temperature exceeds 30 degrees Celsius, ventilation is needed to lower the temperature of the beehive, which will cause more bees to remain in the beehive, resulting in reduced pollination activity. If the temperature exceeds 33 degrees Celsius, the bees will enter survival mode, killing the larvae and ceasing pollination activities. To confirm this situation, there are already proposed bumblebee management methods that allow for remote monitoring of the bumblebee's status.

[0008] Korean Patent Publication No. 10-2016-0141224, "Bumblebee Management Device and Management System," discloses a technique that uses a sensor to detect the number of times bumblebees enter and exit through the entrance and exit of a bumblebee box to remind the bumblebee to change time. Korean Patent No. 10-1963648, "Greenhouse Bumblebee Management System and Method, Bumblebee Box," discloses a technique that uses multiple sensors to detect the direction and number of times bees enter and exit the bumblebee box to control the opening and closing of the entrance and exit.

[0009] However, the aforementioned existing technologies are limited to identifying bumblebees and have the following problems: they cannot analyze the amount of pollen on the bees, cannot provide information related to the appropriateness of the bees' pollination activities, and cannot provide the necessary information for hive and greenhouse management.

[0010] Existing technical documents

[0011] Patent documents

[0012] Patent Document 1: Korean Patent Publication No. 10-2016-0141224 (published on December 8, 2016)

[0013] Patent Document 2: Korean Patent No. 10-1963648 (Published on April 1, 2019) Summary of the Invention

[0014] To address the aforementioned problems, the present invention aims to provide an apparatus and method for tracking the trajectory of bees by comparing images of bees captured near a beehive with learning data.

[0015] Furthermore, the present invention aims to provide an apparatus and method for analyzing, using captured images, the amount of pollen collected by bees, the appropriateness of bee pollination activities, and the health status of bees.

[0016] Furthermore, the present invention aims to provide an apparatus and method for determining the appropriateness of bee pollination activities by analyzing bee trajectories and pollen status in captured images, and for determining the hive replacement time when necessary.

[0017] A learning-based pollen identification device according to an embodiment of the present invention for achieving the above-mentioned objective includes: a storage unit for storing learning data obtained by learning morphological features of bees and pollen through a deep learning model; and a control unit for identifying bees and pollen in the captured images using the learning data stored in the storage unit.

[0018] Furthermore, the aforementioned control unit detects boundary lines through image processing of the captured images and identifies the boundary lines predicted as pollen as pollen outlines.

[0019] Furthermore, the aforementioned learning data includes learning data related to various types of bees and learning data related to various types of pollen. The control unit uses the aforementioned learning data to set up beehives in areas presumed to be bees and pollen boxes in areas presumed to be pollen, thereby identifying bees and pollen respectively.

[0020] Furthermore, the control unit uses the IoU value, which is the intersection of the beehive and the pollen box, to identify whether pollen is present.

[0021] Furthermore, the control unit detects boundary lines through image processing of the captured images, identifies the boundary lines predicted to be pollen as pollen outlines, and only considers the outlines existing within the pollen box as pollen.

[0022] Furthermore, the control unit detects boundary lines through image processing of the captured images, identifies the boundary lines predicted to be pollen as pollen outlines, and only considers the outlines existing within the beehive as pollen.

[0023] Furthermore, the aforementioned control unit considers only pollen boxes that are present inside the beehive as valid pollen boxes, ignoring pollen boxes that are present outside the beehive.

[0024] Furthermore, the control unit detects boundary lines through image processing of the captured images, identifies the boundary lines predicted to be pollen as pollen outlines, and only considers the outlines existing within the effective pollen boxes as pollen.

[0025] Furthermore, the aforementioned control unit sets the outline as a baseline for controlling the color or concentration of pollen.

[0026] Furthermore, the control unit is configured with a quadrilateral inscribed within the outline, and the interior of the quadrilateral is defined as the target area for controlling the color or concentration of the pollen.

[0027] Furthermore, the control unit determines the presence of pollen based on at least one of the following: the area ratio R of the beehive and pollen box, the curvature E of the outline, the difference D between the color of the pollen and the background color, and the color ratio P inside the outline.

[0028] Furthermore, the control unit determines the presence of pollen based on at least one of the following: the area ratio R of the beehive and pollen box, the curvature E of the outline, the difference D between the color of the pollen and the background color, and the color ratio P inside the quadrilateral.

[0029] Furthermore, the control unit performs binary processing on the image of the pollen box area to standardize the color values, and analyzes the color values ​​using a numerical integration method to quantitatively determine the concentration and amount of pollen.

[0030] Furthermore, during the aforementioned binary processing, the control unit sets the color value of the outer region of the outline within the pollen box to 0.

[0031] Furthermore, in the image of the pollen box area, the control unit uses the color value as the Z-axis to display the distribution of pollen in three dimensions.

[0032] Furthermore, the present invention also includes: a camera unit for capturing the aforementioned images; and a display unit for displaying the aforementioned images, wherein the control unit displays the aforementioned beehive, pollen box, and outline on the display unit.

[0033] According to the present invention, an apparatus and method for tracking the trajectory of bees by comparing images of bees taken near a beehive with learning data will be provided.

[0034] Furthermore, according to the present invention, an apparatus and method will be provided for analyzing, using captured images, the amount of pollen collected by bees, the appropriateness of bee pollination activities, and the health status of bees.

[0035] Furthermore, according to the present invention, an apparatus and method will be provided to determine the appropriateness of bee pollination activities by analyzing bee trajectories and pollen status in captured images, and to determine the hive replacement time when necessary. Attached Figure Description

[0036] Figure 1 This is a structural diagram of a learning-based pollen recognition device according to an embodiment of the present invention.

[0037] Figures 2 to 4 This is an example image of a learning-based pollen recognition device according to an embodiment of the present invention.

[0038] Figures 5 to 7 This is an example image of pollen images in a learning-based pollen recognition device according to an embodiment of the present invention.

[0039] Figure 8 An example diagram of a beehive used in conjunction with a learning-based pollen identification device according to an embodiment of the present invention.

[0040] Figure 9 This is an example diagram illustrating the normal pollen recognition process in a learning-based pollen recognition device according to an embodiment of the present invention.

[0041] Figure 10 This is an example diagram illustrating an abnormal pollen identification scenario in a learning-based pollen identification device according to an embodiment of the present invention.

[0042] Figure 11 This is an example image of pollen images that have undergone binary processing in a learning-based pollen recognition device according to an embodiment of the present invention.

[0043] Figure 12 This is a quantitative pollen analysis diagram in a learning-based pollen identification device according to an embodiment of the present invention.

[0044] Figure 13 This is an analysis chart of the moving distance and occurrence frequency of bees in a learning-based pollen recognition device according to an embodiment of the present invention.

[0045] Figures 14 to 16 This is an example diagram illustrating trajectory tracking using a learning-based pollen identification device according to an embodiment of the present invention. Detailed Implementation

[0046] The advantages and features of the present invention, and methods for implementing them, will become clear from the accompanying drawings and the various embodiments described in detail therewith. However, the present invention is not limited to the embodiments disclosed below, but can be implemented in many different ways. The embodiments described herein are provided only to complete the disclosure of the invention and to give a full understanding of the scope of the invention to those skilled in the art. The invention is defined only by the scope of the claims. Throughout the specification, the same reference numerals denote the same structural elements.

[0047] The terminology used in this specification is for illustrative purposes only and is not intended to limit the invention. Unless otherwise expressly indicated in the context, singular expressions include plural expressions. It should be understood that, in this specification, terms such as "comprising" or "possessing" are used to indicate the presence of features, numbers, steps, actions, structural elements, components, or combinations thereof described in this specification, rather than precluding the presence or additional possibilities of one or more other features or numbers, steps, actions, structural elements, components, or combinations thereof.

[0048] In this specification, terms such as “part,” “module,” “device,” “terminal,” “server,” or “system” are used to refer to a combination of hardware and software driven by that hardware. For example, hardware may be a central processing unit (CPU) or a data processing device that includes other processors. Furthermore, software driven by hardware may be a running program, object, executable file, thread of execution, program, etc.

[0049] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0050] Figure 1 This is a structural diagram of a learning-based pollen recognition device according to an embodiment of the present invention.

[0051] The device 100 includes: a control unit 110, which tracks the trajectory of bees through a trajectory tracking module 120 and a pollen analysis module 130 and analyzes the amount of pollen collected; and a storage unit 150, which stores learning data obtained by learning various information related to bees and pollen through a deep learning model.

[0052] In addition, it may include: a camera unit 140 for capturing images of bees moving near the beehive; a display unit 160 for displaying the captured images; and a communication unit 170 for performing wired or wireless communication with external devices.

[0053] When an external camera (not shown) is used instead of the built-in camera unit 140, the device 100 can perform data transmission and reception with the external camera through the communication unit 170. An external display device (not shown) can be used instead of the display unit 160 built into the device 100, or the external display device can be used in conjunction with the display unit 160.

[0054] The device of the present invention can learn the characteristics of bumblebees, such as body structure or color distribution, through a deep learning model, and store them as learning data in the storage unit 150. It compares the video received from the camera unit 140 and stored in the storage unit, as well as the streaming video data received from the camera unit 140 and the learning data frame by frame, to find bumblebees in the images, track their movement paths, find pollen, and analyze the amount of pollen.

[0055] Figures 2 to 4 This is an example image of a learning-based pollen recognition device according to an embodiment of the present invention.

[0056] The display screen provided by the device of the present invention can have, for example, Figure 2 The structure shown.

[0057] Basically, bumblebees are identified by calculating the intersection between multiple quadrilaterals (hereinafter referred to as "boxes") virtually drawn near the entrance of the beehive and the quadrilaterals (hereinafter referred to as "beehives") automatically drawn when identifying bees.

[0058] exist Figure 2 In the diagram, the entrance and exit of the beehive are used as the center, and the area outside is divided into three regions: Area 1 (inside the red box), Area 2 (inside the blue box), and Area 3 (outside the blue box). The area around the bees is defined as the beehive (green box).

[0059] The device of the present invention distinguishes whether a bee is entering or leaving the beehive by recording or tracking the order in which the area inside the box intersects with the beehive.

[0060] For example, if the movement of the beehive (or the intersection of the beehive and the frame) is recorded in the order of area 1 → area 2 → area 3, it is considered as leaving the beehive; if it is recorded in the order of area 3 → area 2 → area 1, it is considered as entering the beehive; if it stays in area 1 for a certain period of time and then disappears, it is considered as entering the beehive.

[0061] exist Figure 2 In Chinese, "hive" refers to the number of bees present in a hive, which is a value obtained by counting the number of times multiple bees enter and exit.

[0062] "Pollination" refers to the amount of pollen collected by bees, "heavy" indicates a good state of pollen collection, and "light" indicates a relatively insufficient state of pollen collection.

[0063] "LoC" indicates the region where the bees (or beehives) are located.

[0064] On the other hand, in this invention, the IoU (intersection over union) metric, used as an indicator to evaluate the accuracy of object detection, is defined as follows.

[0065] IoU = Intersection area of ​​the frame and the beehive / Sum of the areas of the frame and the beehive

[0066] The calculated intersection value will be displayed in Figure 2 In the image, the "pIoU1", "pIoU2", "cIoU1", and "cIoU2" at the top right are the values. The first letter "p" means the previous value, and the first letter "c" means the current value.

[0067] “pIoU1” represents the value measured in the previous frame in region 1 (red box), “pIoU2” represents the value measured in the previous frame in region 2 (blue box), “cIoU1” represents the value measured in the current frame in region 1, and “cIoU2” represents the value measured in the current frame in region 2.

[0068] Not only "IoU", but also the values ​​of "Hive" and "Pollination" are updated in each frame of the image.

[0069] On the other hand, bee and pollen identification is performed by comparing stored learning data with captured images. In bee detection, the IoU between the bee's region (region 1, region 2, or region 3) and the beehive is calculated. In pollen detection, the presence of pollen is initially distinguished by calculating the IoU between the beehive and the pollen.

[0070] When data from individuals such as bees or pollen grains are applied to captured images, the probability that an individual in an image acquired at any given time period is a bee or pollen grain can be numerically displayed near the beehive or pollen box. For example, if "0.99" is displayed next to a beehive, it means that the probability that the individual is a "bee" is 99%.

[0071] Pollen boxes are generated inside beehives. The value displayed next to the pollen box represents the probability that an image that intersects with the beehive is predicted to be pollen. In other words, it represents the probability that an overlapping pollen image is predicted to be pollen.

[0072] To locate regions predicted to be pollen, image preprocessing is used to draw outlines in the corresponding areas. The color and concentration (i.e., the ratio of pixels to a specific color) of the regions within the outlines are calculated. Pollen colors vary depending on the flowers of the fruit pollinated by bees. By drawing outlines in the areas identified as pollen and calculating the average color and concentration within them, pollen can be identified.

[0073] It can be confirmed that the distribution pattern of pollen adhering to a bee's body has a prescribed correlation with the degree of pollination activity of the bee. The shape and radius of curvature of the outline can be used as a benchmark to distinguish the degree of pollination activity of the bee. For example, if the outline is elliptical or the pollen distribution area is wide, it can be judged that the bee has little or no pollination activity, while if the pollen distribution area is small and close to a circle, it can be judged that the bee has vigorous and smooth pollination activity.

[0074] Figure 3 This illustrates the case where the amount of pollen collected by bees is relatively abundant (heavy) due to exceeding a specified baseline, expressed numerically as 0.09998. Figure 4 This illustrates the case where the amount of pollen collected by bees is relatively insufficient (light) due to a lack of a specified baseline, and is expressed numerically as 0.448.

[0075] Figure 5 Show Figure 3 The pollen image shown, i.e., the pollen image when the pollen quantity is sufficient, is characterized by using a relatively bright color to represent the pollen, and its outline is also clearly defined. In contrast, Figure 6 Show Figure 4 The pollen image shown is characterized by pollen being represented by a relatively blurry or dark color, with its outline also blurred, and the pollen being dispersed in multiple areas, or having a wide area inside the outline.

[0076] Reference Figure 7 This will explain the methods for identifying pollen and measuring pollen quantity.

[0077] A green outline will be generated around the area predicted to be pollen. Boundary lines are detected through image processing and represented by outlines. However, relying solely on boundary line detection methods may result in cases where pollen, although similar to pollen, is not actually pollen.

[0078] Therefore, it is necessary to utilize learning data related to pollen morphology. By using pollen-related learning data to set pollen boxes (i.e., the yellow box on the left side of the image) in areas presumed to be pollen, and generating contour lines only within the pollen boxes, the accuracy of pollen identification can be improved.

[0079] For bees, beehives can also be set up in areas presumed to be bees (i.e., the blue box on the right side of the image) using bee-related learning data. Only outlines present inside the beehive are identified as pollen. In the image on the right, green outlines are generated inside and outside the beehive, but only those attached to the bee's body are identified as pollen, so the green outlines outside the beehive should be ignored.

[0080] In the left-hand image, a green outline is formed inside the pollen box. Cases where a green outline forms outside the pollen box should also be disregarded. In summary, even when generating beehives and pollen boxes separately, it is preferable to identify the area inside the outline as pollen only when the pollen box is located inside the beehive and there is an outline inside the pollen box. The outline formed in this way will establish a reference area for understanding the color, concentration, etc., of the pollen.

[0081] On the other hand, in the right-hand image, the green outline extending beyond the beehive can be interpreted as being related to false pollen. In this case, if the IoU value, which is the intersection of the beehive and the pollen region, is used, the green outline on the right side, which is almost or completely detached from the beehive, will result in a very low or non-existent IoU value, thus allowing correction to the identification results related to the presence or absence of pollen.

[0082] Classification tasks can be applied to many fields using data learned through deep learning, as long as the data quality is sufficient for learning. However, the core of this invention is not merely about learning / prediction, but about classifying individuals through algorithms.

[0083] This represents a novel methodology for improving and enhancing predictive models. When learning about multiple individuals such as bees, bumblebees, and pollen, the input image is processed by a deep learning model, which learns the image's annotations (category types: bees, honeybees, bumblebees, pollen, other individuals) and features (morphology, color distribution, concentration, etc.). The model's accuracy can then be calculated based on the features of each category of the learned image.

[0084] Accurate learning and prediction of the entire captured imagery yields the highest quality results, but this requires processing massive amounts of data, necessitating consideration of computer performance and efficiency. To address this issue, a method is employed that annotates only the individuals to be identified. In this invention, only features representing individual individuals or categories are learned, excluding overall learning of each individual, thus achieving high efficiency with low computational cost. Furthermore, reliability is improved by applying IoU values ​​to the classification of different categories (i.e., bees and pollen).

[0085] The learning data will indicate the accuracy of individual entities (bees or pollen) in the images. Therefore, for complex / delicate structures like insects, accuracy is heavily influenced by the quality of the input images during learning. In particular, for fast-moving objects like bees, image quality is significantly affected by camera resolution, ambient brightness, and object movement. Thus, it's possible to misidentify areas not of interest as pollen, or even to misidentify areas where pollen is not actually present.

[0086] Therefore, to address this problem, this invention also employs an algorithm for detecting pollen color. To this end, image preprocessing is used to distinguish only the regions identified as pollen in the training data. Colors are typically represented in RGB (red, green, blue), and a binary technique that re-corresponds them 1:1 within the black-to-white spectrum is used to extract the colors. This binary technique will be explained later.

[0087] On the other hand, beehives are often yellow, similar to pollen. In such cases, bees may mistake the area around the entrance for pollen when entering or leaving the hive. To prevent this, an algorithm can be used to confirm whether the pollen is present by comparing the color of the area around the hive.

[0088] Even images of the same individual beehive can be misidentified based on the brightness of the surrounding area. This problem can be addressed by using the color of the beehive's perimeter as a corrector. That is, instead of using a method of periodically photographing the beehive to remove colors that might be identified as pollen, the method involves correcting for the surrounding brightness by varying it over time (or based on the sun's position at latitude and longitude).

[0089] In this regard, we will refer to Figure 8 Please provide an explanation.

[0090] For example, Figure 8The primary color of the beehives in the case is yellow. Pollen is also primarily yellow, which could lead to confusion between the colors of the beehives and the pollen. In this situation, the color value of the beehives is used as a reference color for color similarity checks to determine whether the identified individuals belong to the pollen group.

[0091] The premise is that the color of pollen and the color of the beehive are similar but not identical. The color of the beehive is uniform within a specified area, but the color of pollen varies depending on its distribution, making this premise quite appropriate.

[0092] When individuals presumed to be pollen are identified in a certain area, if the similarity between the color inside the outline and the color of the beehive is very high (e.g., color difference less than 20%), it is considered a beehive area, not pollen. If the similarity is within a specified range (e.g., color difference 20%–40%), it is considered pollen. If the similarity is very low (e.g., color difference greater than 40%), it is considered not pollen or pollen in small quantities. The criteria, displayed as percentages, can be changed through data updates and learning.

[0093] The methodology of using the background color of the beehive, etc., as a corrector to accurately identify the color of pollen or bees can be used in a way that is independent of the type of color.

[0094] Figure 9 The diagram illustrates a normal pollen recognition process in a learning-based pollen recognition device according to an embodiment of the present invention, showing a beehive, a pollen box, and pollen color data.

[0095] After first locating bee regions in the image by utilizing learning data and comparing IoU values, regions identified as pollen are then located within the bee regions. Next, the outlines of the pollen-identified regions are located, and their internal information is retrieved; this process will continue in this order.

[0096] The pollen color data (Color) is supplemented with information such as the area ratio R of bees and pollen (or the area ratio of the beehive and pollen box), the curvature E of the outline (or the eccentricity of the ellipse), the difference D between the pollen color and the background color (i.e., the color of the beehive), and the color ratio P within the outline. The section displayed at the bottom as "Gate" indicates information related to the color of the beehive entrance, and the section displayed at the bottom as "Pollen" indicates information related to the color of the pollen. To determine the color ratio P, a box (red square) is defined within the inscribed outline, and the color distribution is measured within this box, which makes the calculation faster and simpler.

[0097] exist Figure 9In this case, the area ratio R of bees and pollen is as low as 3%, the difference D between pollen color and background color is within the specified range, and the pollen color ratio P inside the outline is uniform (~100%), so the individual is very likely to be pollen.

[0098] In comparison, Figure 10 In the process, first confirm the results of the parameters (R, E, D, P) for the location identified as a pollen region (i.e., the location shown by the green outline). If the individual is not a pollen, then... Figure 10 As shown, if it is determined that it is not pollen, it is more convenient to replace the area below the pollen color data with a specific color (e.g., black) for easy identification, rather than the pollen color.

[0099] Figure 11 This is an example image of pollen images that have undergone binary processing in a learning-based pollen recognition device according to an embodiment of the present invention.

[0100] By performing binary processing on the image to standardize the color values ​​expressed by R, G, and B according to values ​​between 0 and 255, and then analyzing these values ​​using numerical integration, the concentration and amount of pollen can be quantitatively determined. Figure 11 This is the result of extracting only the pollen region using binary-processed values.

[0101] and Figure 9 By comparison, it can be confirmed that only the color values ​​within the contour lines found in the pollen region are extracted. Figure 11 In the middle, all areas that are not pollen areas are filled with 0 to avoid affecting the calculation of pollen quantity. Figure 11 The colors of real-time input images are represented using binary processing, thus enabling quantitative analysis of regions identified as pollen.

[0102] Figure 12 This is a quantitative pollen analysis diagram in a learning-based pollen identification device according to an embodiment of the present invention.

[0103] Figure 12 3D display Figure 11 In the image, the Z-axis represents the color value, showing how the color varies according to pollen concentration, thus enabling easy observation of concentration changes. This makes it easier to identify pollen distribution within the pollen outline.

[0104] On the other hand, the device of the present invention can analyze the captured images to... Figure 13 The method shown generates a chart analyzing the bee's trajectory.

[0105] Figure 13This is an analysis chart of the moving distance and occurrence frequency of bees in a learning-based pollen recognition device according to an embodiment of the present invention.

[0106] Figure 13 To simultaneously display the bee's traveling distance and number of appearances in a trajectory chart, the movement and status of the bees can be understood by comprehensively analyzing the bee's travel distance (blue) representing the distance from the hive entrance to the hive and the number of appearances (red) representing the number of times the hive appears.

[0107] Figures 14 to 16 This is an example diagram illustrating the state analysis of bees using a learning-based pollen recognition device according to an embodiment of the present invention.

[0108] Compare the three attached images with the trajectory of a normal bee entering the hive. Figure 14 The trajectory of normal bees leaving the hive ( Figure 15 Compared to, such as Figure 16 As shown, it is clear that bees may fall to their deaths after their movement ends around the entrance / exit.

[0109] Although not illustrated, the device of the present invention can also provide various forms of bee state analysis diagrams. For example, it may include a bee trajectory chart representing the bee's flight path on an XY coordinate system, a bee observation chart representing the cumulative frequency of bees appearing in an image, and a bee travel distance chart and bee observation chart generated from a reference point.

[0110] In addition, by measuring the time it takes for bees to enter and exit the hive, we can understand bee aging and activity levels, and learn data on pollen morphology and color. Therefore, since pollen color can be distinguished in the captured images, it can be determined whether bees are seeking other plants different from the target plant, and whether bee movement is normal or abnormal. Thus, analyzing bee movement can provide information related to environmental changes or the appropriateness of bee pollination activities, and may also provide information needed for hive and greenhouse management.

[0111] The present invention has been described above through specific structural elements and other specific matters, limiting embodiments and accompanying drawings. However, this is only for a more comprehensive understanding of the present invention. The present invention is not limited to the above-described multiple embodiments. Those skilled in the art to which this invention pertains can make various modifications and variations based on this description.

[0112] Therefore, the concept of the present invention is not limited to the embodiments described above. All technical solutions that are modified in a manner equivalent to or equivalent to the scope of the invention claims are within the scope of the present invention.

Claims

1. A learning-based pollen identification device, characterized in that, include: The storage unit stores the learning data acquired through deep learning models of the morphological characteristics of bees and pollen; and The control unit uses the captured images and the learning data stored in the aforementioned storage unit to identify bees and pollen in the captured images. The aforementioned learning data includes learning data related to various types of bees and learning data related to various types of pollen. The control unit uses the learning data to set up beehives in areas presumed to be bees and pollen boxes in areas presumed to be pollen, thereby identifying bees and pollen respectively. The control unit detects boundary lines through image processing of the captured images, identifies the boundary lines predicted to be pollen as pollen outlines, and only considers the outlines existing within the beehive as pollen. The aforementioned control unit sets the outline as a baseline for determining the color or concentration of pollen. The control unit determines the presence of pollen based on at least one of the following: the area ratio R of the beehive and pollen box, the curvature E of the outline, the difference D between the color of the pollen and the background color, and the color ratio P inside the outline.

2. The learning-based pollen identification device according to claim 1, characterized in that, The control unit uses the IoU value, which is the intersection of the beehive and the pollen box, to identify whether pollen is present.

3. The learning-based pollen recognition device according to claim 1, characterized in that, The control unit considers only the outlines existing in the beehive and also in the pollen box as pollen.

4. The learning-based pollen identification device according to claim 1, characterized in that, The aforementioned control unit considers only pollen boxes that are present inside the beehive as valid pollen boxes, and ignores pollen boxes that are present outside the beehive.

5. The learning-based pollen identification device according to claim 4, characterized in that, The aforementioned control unit considers only the outlines existing within the aforementioned effective pollen boxes as pollen.

6. The learning-based pollen identification device according to claim 1, characterized in that, The control unit is set to a quadrilateral inscribed in the outline, and the interior of the quadrilateral is set as the target area for controlling the color or concentration of the pollen.

7. The learning-based pollen recognition device according to claim 6, characterized in that, The color ratio P mentioned above is the color ratio P inside the quadrilateral mentioned above.

8. The learning-based pollen identification device according to claim 1, characterized in that, The control unit performs binary processing on the image of the pollen box area to standardize the color values, and analyzes the color values ​​using a numerical integration method to quantitatively determine the concentration and amount of pollen.

9. The learning-based pollen identification device according to claim 8, characterized in that, During the binary processing described above, the control unit sets the color value of the outer region of the outline within the pollen box to 0.

10. The learning-based pollen identification device according to claim 1, characterized in that, In the image of the pollen box area, the control unit uses the color value as the Z-axis to display the distribution of pollen in three dimensions.

11. The learning-based pollen identification device according to claim 1, characterized in that, Also includes: A camera unit, used to capture the aforementioned images; and The display unit is used to display the aforementioned images. The aforementioned control unit displays the aforementioned beehive, pollen box, and outline on the aforementioned display unit.

Images