Bee behavior analysis system and bee behavior analysis method

Abstract

To provide a honeybee behavior analysis system capable of analyzing a feeding range of honeybees in real time in an actual apiary.SOLUTION: A honeybee behavior analysis system comprises an imaging device for capturing an image of behavior of a honeybee and a computer for analyzing the image captured by the imaging device. The computer includes an image processing part for executing prescribed image processing on the image, a frequency spectrum analysis part for analyzing a frequency spectrum representing a change in every pixel of the image after the image processing, a pixel specification part for specifying a pixel having a peak in a specific frequency component based on the frequency spectrum, a dance information calculation part for calculating dance information relating to dance of the honeybee based on the pixel specified by the pixel specification part, and a feeding position specification part for specifying a feeding position of the honeybee according to the dance information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a system and method for analyzing the behavior of honeybees. [Background technology]

[0002] In rearing species of the genus Apis, such as the European honeybee, it is important to understand the range that the European honeybees use for foraging, etc., in managing the rearing environment. One known method for understanding the foraging range of European honeybees is to decode the European honeybee's waggle dance (see, for example, Non-Patent Document 1). Because manual decoding takes time, efforts are underway to develop automatic computer-based decoding technology.

[0003] However, automatic computer-based decoding requires processing large amounts of data, which requires a lot of power. To automatically decode the dance in real time, data analysis must be performed on a computer located on-site, rather than on a remote server. However, it is difficult to secure sufficient power sources deep in the mountains where actual apiaries are located, making it difficult to analyze the dance in real time on-site and decode the foraging ranges of European honeybees. Therefore, there is a need for a system that allows beekeepers and others to decode the foraging ranges of actual apiaries in real time. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Wario et al. 2017. Automatic detection and decoding of honey bee waggle dances. PLOS ONE. 12(12): e0188626. [Non-patent document 2] Okubo et al. 2019. Forage area estimation in European honeybees (Apis mellifera) by automatic waggle decoding of videos using a generic camcorder in field apiaries. Apidologie. 50:243‐252. [Non-patent document 3] George et al. 2021. Distance estimation by Asian honey bees in two visually different landscapes. Journal of Experimental Biology (2021) 224, jeb242404. doi:10.1242 / jeb.242404. Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention provides a system for analyzing the behavior of Apis that can decipher the foraging range of Apis in real time in an actual apiary. [Means for solving the problem]

[0006] In order to solve the above problems, the present invention provides a honeybee behavior analysis system comprising an imaging device that captures images of honeybee behavior and a computer that analyzes images captured by the imaging device. The computer comprises an image processing unit that performs predetermined image processing on the images, a frequency spectrum analysis unit that analyzes a frequency spectrum that shows changes in each pixel of the image after the image processing, a pixel identification unit that identifies pixels having peaks in specific frequency components based on the frequency spectrum, a dance information calculation unit that calculates dance information related to the honeybee dance based on the pixels identified by the pixel identification unit, and a foraging position identification unit that identifies the foraging positions of the honeybees in accordance with the dance information.

[0007] A method for analyzing the behavior of honeybees according to the present invention includes the steps of capturing images of honeybee behavior with an imaging device and analyzing the images captured by the imaging device with a computer, wherein the step of analyzing with a computer includes the steps of: performing predetermined image processing on the images; analyzing a frequency spectrum showing changes in each pixel of the image after the image processing; identifying pixels having peaks in specific frequency components based on the frequency spectrum; calculating dance information related to the dance of honeybees based on the identified pixels; and identifying the foraging positions of honeybees according to the dance information. [Effects of the Invention]

[0008] According to the Apis behavior analysis system of the present invention, it is possible to provide a Apis behavior analysis system that can decipher the foraging range of Apis in real time in an actual apiary. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a schematic diagram illustrating the overall configuration of a behavioral analysis system for the genus Apis 1 according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing an example of the hardware configuration of the edge computer 20. [Figure 3] FIG. 1 is a schematic diagram illustrating a method for calculating dance information of honeybee B and for identifying a foraging location based on the calculation results. [Figure 4] 10 is a flowchart illustrating an example of a procedure for calculating dance information related to the dance of bees. [Figure 5] 10 is a schematic diagram illustrating the details of reduction processing in the image processing unit 112. FIG. [Figure 6] FIG. 2 is a schematic diagram illustrating a method of calculating a frequency spectrum executed by a frequency spectrum analysis unit 113. [Figure 7] FIG. 2 is a schematic diagram illustrating a method of calculating a frequency spectrum executed by a frequency spectrum analysis unit 113. [Figure 8]FIG. 10 is a schematic diagram illustrating the contents of step S24. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, the present embodiment will be described with reference to the accompanying drawings. In the accompanying drawings, functionally identical elements may be designated by the same numerals. Note that the accompanying drawings show embodiments and implementation examples according to the principles of the present disclosure, but these are for understanding the present disclosure and are not to be used to interpret the present disclosure in a limiting manner. The descriptions in this specification are merely typical examples and are not intended to limit the scope or application of the present disclosure in any way.

[0011] Although the present embodiment has been described in sufficient detail to enable those skilled in the art to implement the present disclosure, it should be understood that other implementations and forms are possible, and that changes in configuration and structure and substitutions of various elements are possible without departing from the scope and spirit of the technical ideas of the present disclosure. Therefore, the following description should not be interpreted as being limited thereto.

[0012] The overall configuration of a honeybee behavior analysis system 1 according to an embodiment of the present invention will be described with reference to Figure 1. The system 1 includes a camera (imaging device) 10, an edge computer 20, and a battery 30. The edge computer 20 can be connected to a server 40, a database 50, and a control computer 60 via a network NW.

[0013] The camera 10 is positioned facing a hive HB for raising honeybees B, and is capable of capturing images (video) inside the hive HB and capturing images of the behavior of the honeybees B. As one example, the camera 10 may be configured to capture images of the inside of the hive HB through a hole provided in the hive HB, or at least a portion of the side of the hive HB may be made of a transparent acrylic plate or the like, and the camera 10 may capture images of the inside of the hive HB through the transparent acrylic plate.

[0014] The edge computer 20 is a computer for analyzing image data captured by the camera 10. The battery 30 is a power supply device that supplies power to the camera 10 and the edge computer 20. For example, a Jetson Nano (registered trademark) provided by NVDIA can be used as the edge computer 20. As will be described later, the edge computer 20 is configured to analyze the dance of honeybees according to a built-in program and / or an AI engine, thereby enabling the immediate identification of the foraging locations of honeybees. By immediately identifying the foraging locations, it becomes possible to create sales promotion data indicating that the honeybee is a monofloral nectar with a high selling price, or to create data for measures against pesticide exposure.

[0015] The server 40 is connected to the edge computer 20 via the network NW, receives analysis result data from the edge computer 20, manages the data in the database 50, and provides various data (learned data, etc.) for edge computing to the edge computer 20. The control computer 60 is similarly connected to the edge computer 20 via the network NW, and has the function of displaying the analysis results from the edge computer 20 on a display and setting various parameters for the edge computer 20. In addition to the display, the control computer 60 may also be equipped with input devices (e.g., a keyboard, a mouse, etc.) not shown. The database 50 manages the analysis result data from the edge computer 20 as described above, as well as data such as the location of the hive HB, the topography, environment, and climate around the hive HB, and bee rearing information (population, harvest, etc.).

[0016] For the sake of simplicity, FIG. 1 illustrates one hive HB, one camera 10, one edge computer 20, and one server 40, but this is merely an example and the present invention is not limited to this. A configuration in which one server 40 is connected to multiple edge computers 20 is also within the scope of the present invention. Multiple cameras 10 may be provided in one hive HB. The camera 10 may be an infrared camera, and may be provided with a light source that illuminates the imaging area of ​​the camera 10.

[0017] The server 40 may also be configured with multiple server computers that are distributed. The edge computers 20 do not need to be arranged one-to-one with the beehives HB, and one edge computer 20 may be arranged for multiple beehives HB, or conversely, multiple edge computers 20 may be arranged for one beehive HB. In addition, the battery 30 may also be connected to multiple cameras 10 and edge computers 20. The control computer 60 may also be configured to control multiple edge computers 20.

[0018] As shown in FIG. 2, the edge computer 20 includes, for example, a CPU (Central Processing Unit) 101, a ROM 102, a RAM 103, a flash memory 104, an input / output control unit 105, and a communication control unit 106.

[0019] The CPU 101 is a central control unit that is responsible for various analyses of the behavior of honeybees in the beehives HB, where the honeybees are kept, among the operations of the Apidae behavior analysis system. In addition to the CPU 101, a GPU (Graphics Processing Unit) may be provided as a control unit for executing image processing and image recognition.

[0020] ROM 102 is a storage device that stores programs for image processing, frequency spectrum analysis, dance information calculation, and foraging location identification, as well as various data required to execute these programs. RAM 103 is a storage device that temporarily stores the calculation results of the programs. Flash memory 104 is a storage medium that stores programs read from ROM 102 and various update programs provided by server 40.

[0021] The input / output control unit 105 is a control unit that controls data input from the server 40 and the control computer 60, and data output to the server 40 and the control computer 60. The communication control unit 106 controls the sending and receiving of data exchanged between the server 40 and the control computer 60.

[0022] Using the aforementioned program and AI engine, the CPU 101 realizes an image processing unit 112, a frequency spectrum analysis unit 113, a pixel identification unit 114, a dance information calculation unit 115, a foraging position identification unit 116, a learned data storage unit 117, and a pixel restoration unit 118 within the edge computer 20.

[0023] The image processing unit 112 processes the image data captured by the camera 10 into a format suitable for analyzing the bee dance information. The frequency spectrum analysis unit 113 analyzes the frequency spectrum that shows the changes in each pixel that makes up the image data after image processing by the image processing unit 112. The pixel identification unit 114 identifies pixels whose frequency components in the frequency spectrum satisfy predetermined conditions, specifically, pixels that make up part of the image of the bee dance. The dance information calculation unit 115 calculates dance information related to the bee dance included in the image data, based on the data of the pixels identified by the pixel identification unit 114.

[0024] The foraging position identification unit 116 performs a calculation to identify the foraging position of the bee in accordance with the dance information calculated by the dance information calculation unit 115, and identifies the foraging position of the bee in accordance with the calculation result. The learned data storage unit 117 is a part that stores learned data provided by, for example, the server 40 in accordance with the results of analysis of past bee dances. The learned data is used for judgment in the foraging position identification unit 116. The pixel repair unit 118 has a function of repairing pixel data based on a comparison of pixel data between multiple frame groups.

[0025] With reference to Figure 3, we will explain how to calculate the dance information of honeybee B and how to identify the foraging position based on the result. The honeybee dance is a method of communicating information to other honeybees to inform the location of the foraging position HS for foraging or collecting water. The distance to the foraging position HS is indicated by the duration of the wagging, and the direction of the foraging position HS is indicated by the direction of the wagging (angle α with respect to the sun SN). By reading the dance information (duration, direction), the foraging position HS where the honeybee is foraging or collecting water can be determined.

[0026] By identifying the foraging location HS, overlapping of foraging locations between multiple apiaries can be prevented, and the safety of honeybee rearing can be ensured by evaluating the risk of pesticide exposure at the foraging location HS. It is known that the vibration frequency of honeybee dance is 9 to 16 Hz, with a center frequency of approximately 12.6 Hz.

[0027] Note that the programs of the edge computer 20 and the various functions of the AI ​​engine are not limited to the functions shown in Fig. 2. Functions other than those shown in Fig. 2 may be added, or some of the functions shown in Fig. 2 may be omitted or replaced with similar functions.

[0028] An example of the procedure for calculating dance information related to the dance of honeybees will be described with reference to the flowchart in Figure 4. First, the inside of the hive HB is photographed by the camera 10, and video data is acquired at, for example, 120 frames per second (step S11). The acquired video data is input to the image processing unit 112, where it is subjected to grayscale conversion and reduction processing (step S12).

[0029] As shown in Figure 5, for example, the reduction process in the image processing unit 112 can be performed so that the bee's tail is included in a 3x3 pixel area of ​​the image of the bee. The 3x3 pixels allow the frequency and angle of the bee's dance to be identified with sufficient accuracy. By performing this reduction process, sufficient data is obtained to analyze the movement of the bee's tail, reducing the amount of image processing data required to analyze the dance information, thereby reducing power consumption and shortening the time required for determination. The frame data after image processing is stored in the RAM 103 as a frame group, for example, a bundle of 20 frame images (step S13).

[0030] When 20 frames of frame data are stored in RAM 103, a Fourier transform is performed on each pixel of the 20 frames of frame data (step S21). Specifically, as shown in FIGS. 6 and 7, the time change in pixel brightness over 20 frames is calculated for each of pixels Pixel1, 2, ... n of the 20 frames of frame data. Then, a discrete Fourier transform is applied to this time change data in brightness to obtain frequency spectrum data for each pixel. Then, the maximum value (peak position) of the frequency spectrum data for each pixel is identified.

[0031] Next, it is determined whether the maximum value (peak value) of the frequency spectrum of each pixel matches the bee dance frequency band (9 to 16 Hz, representative value 12.6 Hz) (step S22). Pixels whose maximum value of the frequency spectrum is determined to match the dance frequency band are determined to be candidates for pixels constituting the image of the bee dancing, and the frequency spectrum data is stored in RAM 13 in association with pixel position data, etc. (step S23).

[0032] When the above-mentioned determination is completed for all pixels in a frame group consisting of 20 frame data, the result of determining the candidate pixels in the previous frame group FGi-1 is compared with the result of determining the candidate pixels in the current frame group FGi. That is, it is determined whether or not the positions of the pixels determined to be constituting the image of dancing bees in the current frame group FGi and the pixels determined to be constituting the image of dancing bees in the previous frame group FGi-1 match, according to the data stored in RAM 103 (step S24).

[0033] 8 shows a schematic diagram of the determination in step S24. For example, if the pixels constituting the image of dancing bees are identified in frame group FG2 and pixel groups DG12 to DG42 are obtained, it is determined whether pixel groups are also stored in RAM 103 in the previous frame group FG1 at the same positions as pixel groups DG12 to DG42.

[0034] If a pixel group exists at the same position among multiple frame groups, it is determined that the pixel is highly likely to be a pixel constituting an image of dancing bees, and an identification number indicating this is assigned (step S25). As described above, by calculating the frequency spectrum for each frame group consisting of 20 frames of data and comparing the maximum value with the dance frequency band, it is possible to extract images related to dancing bees quickly and while suppressing memory consumption. Note that while an example of calculating the frequency spectrum for a bundle of 20 frames of data has been described, this is merely an example, and it goes without saying that the number of frames of data to be bundled can be changed.

[0035] The image data acquisition to image processing procedures of steps S11 to S13 and the procedures of steps S21 to S25 can be executed simultaneously (in parallel) for different frame groups. In this way, while data for a plurality of frame groups FGi is being acquired, an operation to identify candidate pixels constituting the image of dancing bees can be executed for the previous frame group FGi-1, thereby shortening the processing time.

[0036] Next, data on candidate pixels determined to constitute an image of dancing bees, identified for each frame group, are stored in RAM 103 (step S26) in association with their position data, frequency spectrum data of the pixels, and identification numbers (step S25). Figure 8 is a schematic diagram illustrating the storage of candidate pixel data in step S26. Candidate pixels for dancing bees are identified in the order of frame groups FG1, FG2, ... FGm, and the positions of those pixels are determined. Figure 8 schematically shows pixel positions using dots.

[0037] 8 shows that pixel groups DG11 to DG51 are detected in frame group FG1 as candidates for pixels that make up the image of dancing bees. These pixel groups DG11 to DG51 are stored in RAM 103 together with their position data and frequency spectrum data.

[0038] Similarly, in frame group FG2, which follows frame group FG1, candidate pixels constituting the image of dancing bees are detected. As an example, as shown in FIG. 8, pixel groups DG12-DG42 are detected in frame group FG2. Then, in step S26, it is confirmed whether pixel groups at the same positions as pixel groups DG12-DG42 exist (are stored) in the previous frame group FG1. In the example of FIG. 8, pixel groups DG11-DG41 are confirmed at the same positions in frame group FG1 for pixel groups DG12-DG42 in frame group FG2. Therefore, pixel groups DG11-DG41 and DG12-DG42 are determined to have a high probability of being pixels constituting the image of dancing bees, and are stored in RAM 103 as candidate pixels for the dancing image.

[0039] Corresponding pixel groups are detected in consecutive frames, and even pixel groups determined to be highly likely to constitute images of dancing bees may contain missing pixels. Therefore, in step S27, pixel data determined to be missing is repaired in pixel repair unit 118. Well-known interpolation methods such as linear interpolation, cubic spline interpolation, and Lagrange interpolation may be used for the repair. Meanwhile, pixel groups determined to be noise are also subject to data deletion processing in pixel repair unit 118.

[0040] Once the restoration of the dance information has been completed in this manner, information about the bee dance (dance duration, direction, etc.) is calculated based on the data of the candidate pixels that make up the obtained dance image (step S28).Then, the calculation results of the bee dance information are output and displayed in the control computer 60, for example, in the form of a table or map (step S29).

[0041] As described above, according to the system of this embodiment, information on the dance of the honeybees is obtained by analyzing image data from a camera 10 placed near the hive HB in an edge computer 20 placed nearby. The edge computer 20 performs predetermined image processing on the obtained image data, analyzes the frequency spectrum of changes in each pixel, and calculates information on the dance of the honeybees based on the results, thereby identifying the foraging locations of the honeybees. By simplifying the data processing and calculation, it becomes possible to analyze the dance information of the honeybees in the edge computer 20, and to analyze the behavior of the honeybees without relying on the server 40.

[0042] The present invention is not limited to the above-described embodiments, and various modifications are possible. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to embodiments including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment. It is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to delete part of the configuration of each embodiment, or to add or replace other configurations. [Explanation of symbols]

[0043] 10...camera, 20...edge computer, 30...battery, 40...server, 50...database, 60...control computer, NW...network, HB...hive, B...honeybee, 112...image processing unit, 113...frequency spectrum analysis unit, 114...pixel identification unit, 115...dance information calculation unit, 116...foraging position identification unit, 117...learned data storage unit, 118...pixel restoration unit.

Claims

1. an imaging device for imaging the behavior of honeybees; a computer that analyzes the image captured by the imaging device; Equipped with The computer an image processing unit that performs predetermined image processing on the image; a frequency spectrum analysis unit that analyzes a frequency spectrum indicating a change in each pixel of the image after the image processing; a pixel specifying unit that specifies a pixel having a peak in a specific frequency component based on the frequency spectrum; a dance information calculation unit that calculates dance information related to the dance of the bees based on the pixels identified by the pixel identification unit; a foraging position identification unit that identifies a foraging position of the honeybee according to the dance information; Equipped with The image processing unit performs grayscale conversion and reduction processing on the image as the image processing, so that the bee's buttocks in the image after the reduction processing are included in a pixel range that is smaller than the bee's buttocks in the image before the reduction processing is performed; The frequency spectrum analysis unit performs the analysis on the image after the image processing, thereby performing the analysis using a smaller amount of data and less power consumption than when the analysis is performed on the image before the image processing. A honeybee behavior analysis system characterized by:

2. 2. The honeybee behavior analysis system according to claim 1, wherein the frequency spectrum analysis unit acquires data of a frame group including a plurality of frames that constitute the image, acquires temporal changes in brightness for each pixel of the plurality of frames, and analyzes the frequency spectrum by applying a discrete Fourier transform to the temporal changes.

3. The Ap behavior analysis system according to claim 2 , wherein the pixel specifying unit specifies pixels having a peak of a frequency component in a dance frequency band related to a dance of a honeybee.

4. The honeybee behavior analysis system according to any one of claims 1 to 3, further comprising a pixel restoration unit that restores the pixel data by comparing a plurality of consecutive frames.

5. A step of capturing images of the behavior of the honeybees with an imaging device; analyzing the image captured by the imaging device with a computer; Equipped with The step of analyzing by a computer includes: performing predetermined image processing on the image; analyzing a frequency spectrum indicative of pixel-by-pixel changes in the image after the image processing; Identifying a pixel having a peak in a specific frequency component based on the frequency spectrum; calculating dance information about the bee dance based on the identified pixels; determining a foraging location of the honeybee according to the dance information; Equipped with In the step of performing the image processing, the image processing includes grayscale conversion and reduction processing of the image, so that the rear end of the bee in the image after the reduction processing is included in a pixel range smaller than the rear end of the bee in the image before the reduction processing is performed; In the step of analyzing the frequency spectrum, the analysis is performed on the image after the image processing is performed, thereby performing the analysis using a smaller amount of data and less power consumption than when the analysis is performed on the image before the image processing is performed. A method for analyzing the behavior of honeybees, characterized by:

6. 6. The method for analyzing Apis behavior according to claim 5, wherein the step of analyzing the frequency spectrum comprises acquiring data of a frame group including a plurality of frames constituting the image, acquiring temporal changes in brightness for each pixel of the plurality of frames, and applying a discrete Fourier transform to the temporal changes to analyze the frequency spectrum.

7. The method for analyzing Ap behavior according to claim 6, wherein the step of identifying pixels identifies pixels having a peak of a frequency component in a dance frequency band related to a dance of honeybees.

8. The method for analyzing the behavior of the honeybee according to any one of claims 5 to 7, further comprising the step of comparing a plurality of consecutive frames to restore the pixel data.

Images